Efficient add-only file system for high performance key value repository
By designing a dedicated file system, using the log structure to only attach write operations and separate data and logs, the problems of tail delay, write amplification and storage costs in the prior art are solved, and higher system robustness and performance are achieved.
Patent Information
- Application Number
- CN202410998079.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-09-27
- Filing Date
- 2024-07-24
- Publication Date
- 2025-03-28
AI Technical Summary
When using the key-value repository, existing file systems have problems such as tail delay and write amplification, and the storage cost is high, and a large amount of files and data may be lost in the event of a failure, affecting the robustness of the system.
A dedicated file system is designed to be used with key-value repositories, adopting a log structure with an attached write operation, separating data and logs, pre-allocating data space, performing metadata management and task scheduling to optimize I/O mode and reduce metadata overhead.
By reducing tail delay and write amplification, reducing storage costs, limiting potential data loss, improving system robustness, and optimizing metadata management and I/O scheduling, significantly improving system performance and reliability.
Smart Images
Figure CN119719050A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to file systems, and in particular to file systems for use with key-value stores. Background Art
[0002] Key-value stores are often used as local storage engines for various infrastructure storage systems. Key-value stores typically save their persistent data on disk via a file system. These file systems are general-purpose and include several features and structures that are not suitable for the specific requirements of key-value stores. Summary of the invention
[0003] The present disclosure is directed to file systems, and in particular, to file systems for use with key-value stores.
[0004] By providing a dedicated file system configured for use with a key-value storage library, tail latency and write amplification can be reduced. In addition, storage costs can be reduced, and the potential number of files and / or the amount of data that may be lost in a failure can be limited, thereby improving system robustness. In addition, by being designed together with the key-value storage library and specifically for the workload of the key-value storage library, the file system can control metadata overhead and make the overhead more predictable. In addition, the file system can have operations, in particular, select input / output (I / O) modes suitable for the storage device used. In addition, by operating the file system in user space rather than kernel space, I / O scheduling can be established to make it flash-friendly, thereby allowing the use of NVMe solid-state drives (SSDs) and / or ZNS SSDs that provide partitioned storage.
[0005] In an embodiment, a file system is configured to perform log-structured append-only write operations. The file system is configured for use with a key-value store, the file system including a file manager configured to keep data and logs separate.
[0006] In an embodiment, the file manager is further configured to determine whether the file is a general file or an instant file based on the write characteristics of the file, and when the file is a general file, store the general file in one or more pre-allocated data spaces. In an embodiment, the size of each of the one or more pre-allocated data spaces is approximately one megabyte (MB).
[0007] In an embodiment, the file system further comprises a metadata manager configured to perform compression of a journal file containing persistent metadata of the file system. In an embodiment, the compression of the journal file is performed in a synchronous mode, wherein in the synchronous mode, a global flush lock is held until the compression of the journal and the update of the super block are completed. In an embodiment, the compression of the journal file is performed in an asynchronous mode, wherein in the asynchronous mode, the global flush lock is released once an in-memory metadata dump request is submitted.
[0008] In an embodiment, the file system further comprises a task scheduler configured to assign a priority to each write task, the priority being based on whether data of the write task is general data or immediate data.
[0009] In an embodiment, the file system includes a dedicated journal thread configured to generate a journal, wherein the journal includes a latest checkpoint from a file system operation log. In an embodiment, the journal includes one or more transactions, each of the one or more transactions including a universally unique identifier, a global sequence identifier, an encoded operation log entry, and a cyclic redundancy check.
[0010] In an embodiment, the disk layout of the file system includes a superblock, which includes journal inode content, universal unique identifier data, version information, and block size. In an embodiment, the disk layout provides a superblock at the second 4 kilobyte block provided in the storage device.
[0011] In an embodiment, the data storage of the file system includes one or more zoned state drives (ZNS SSDs).
[0012] In an embodiment, a system includes a key-value store and a file system as described herein.
[0013] In an embodiment, the file system is configured to update data based on the key-value store without updating metadata associated with the data.
[0014] In an embodiment, a system includes a key-value store and a file system collaborator, which is configured to provide an interface between the key-value store and the file system. In an embodiment, the key-value store and the file system collaborator are configured to expose an application programming interface that allows the key-value store to mark the priority of input / output requests to the file system. In an embodiment, the key-value store and the file system collaborator are configured to coordinate compression and / or garbage collection operations between the key-value store and the file system. In an embodiment, the key-value store and the file system collaborator are configured to provide data from the key-value store to the file system to achieve sector-level redundancy of file data of the key-value store in the file system. In an embodiment, the key-value store and the file system collaborator are configured to perform synchronization operations for the key-value store and the file system without synchronizing metadata.
[0015] In an embodiment, the file system is an append-only file system.In an embodiment, the file system is configured to independently write separate data and logs to persistent storage.
[0016] In an embodiment, a database system includes a key-value store, a file system configured to perform an append operation, a file system configured to be used with the key-value store, and a key-value store and file system coordinator configured to provide an interface between the key-value store and the file system. The file system includes a file manager configured to keep data and logs separate. In an embodiment, the key-value store and file system coordinator are configured to expose an application programming interface that allows the key-value store to mark the priority of input / output requests to the file system.
[0017] In an embodiment, a computer-readable medium contains instructions that, when executed by a processor, direct the establishment and operation of a file system, the file system configured to perform append operations, the file system configured for use with a key-value store, wherein the file system includes a file manager configured to keep data and logs separate. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 The architecture of a file system according to an embodiment is shown.
[0019] Figure 2 Modules of a file system according to an embodiment are shown.
[0020] Figure 3 A disk layout of a file system according to an embodiment is shown.
[0021] Figure 4 A superblock of a file system according to an embodiment is shown.
[0022] Figure 5 A journal of a file system according to an embodiment is shown.
[0023] Figure 6 Directory and file management in a file system according to an embodiment is shown.
[0024] Figure 7 A method of persisting metadata in a file system according to an embodiment is shown.
[0025] Figure 8 Journal file compression in a file system according to an embodiment is shown.
[0026] Fig. 9 Task scheduling in a file system according to an embodiment is shown.
[0027] Fig.10 Memory allocation in a file system according to an embodiment is shown.
[0028] Fig.11 A synchronous input / output (I / O) process in a file system according to an embodiment is shown.
[0029] Fig.12 Sector-level fault tolerance in a file system according to an embodiment is shown.
[0030] Fig.13 A key-value system integrated with a file system according to an embodiment is shown. DETAILED DESCRIPTION
[0031] The present disclosure relates to file systems, and in particular to file systems for use with key-value stores.
[0032] In the following detailed description, reference is made to the accompanying drawings that form a part of the description. In the accompanying drawings, similar symbols generally represent similar components unless the context dictates otherwise. In addition, unless otherwise stated, the description of the subsequent drawings may refer to the features in any previous drawings to provide a clearer context and substantive explanation of the current example embodiments. However, the example embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the various aspects of the present disclosure as generally described and narrated herein and shown in the accompanying drawings may be arranged, replaced, combined, separated, and designed in a variety of different configurations, all of which are expressly contemplated herein.
[0033] In addition, various parts of the present disclosure may be described in terms of functional block components and various processing steps. It should be understood that such functional blocks may be implemented by any number of firmware, software, and / or hardware components configured to perform the specified functions.
[0034] Figure 1 The architecture of a file system according to an embodiment is shown. The file system 100 includes a client library 102 , metadata 104 , metadata persistence 106 , a space allocator 108 , and a device access layer 110 configured to interface with a storage device 112 .
[0035] The file system 100 may be specifically designed to support key-value storage library workloads without supporting general file system workloads. The file system 100 may be a user space attached-only file system designed to operate as a file system for a corresponding key-value storage library. In an embodiment, the storage device 112 used in the file system 100 may be a solid-state drive (SSD), for example, an NVMe SSD. A device access layer 110 may be provided to implement the file system 100 on the storage device 112. The file system 100 may be configured to reduce synchronization write delays commonly encountered in key-value storage libraries. The file system 100 may include customized processing of metadata 104 and make metadata persistent 106. For example, the file system 100 may perform sequential writes to a pre-write log (WAL) and SST files from a corresponding key-value system. The file system 100 may also provide an ordered journaling mode, rather than a dedicated separate thread and thus requiring multiple metadata writes to journal the commit block. A dedicated journaling thread may result in significant write amplification, especially for small data sizes (4KB to 32KB), which may be avoided by the ordered journaling mode of the file system 100. Reducing or avoiding write amplification can improve tail latency, especially for synchronous writes to the write-ahead log.
[0036] In the file system 100, data and logs can be split. In addition, the write model used in the file system 100 can be an append-only write of a log structure. The file system 100 can use pre-allocated space blocks to store common data, so that synchronous writes can be performed by writing data without corresponding updates to metadata. In addition, the data persistence of files and logs can be performed separately. Performing these tasks separately can reduce the number of metadata persistence operations associated with data write operations.
[0037] The file system 100 can be configured to be an append-only file system in which random writes are not allowed. Supporting random writes is much more expensive than supporting append writes (for example, implementing atomic random writes typically requires data diaries in addition to metadata diaries, which actually doubles the amount of data written), and may not be friendly to NAND flash media. On traditional SSDs, random writes require a lot of garbage collection overhead, and such operations cannot be performed on partitioned (ZNS) SSDs. The file system 100 can be configured to perform append operations to append data to files. Since the key-value storage library does not require random writes, in an embodiment, the file system 100 can be designed to be append-only, for example, by making the file system 100 not allow rename, delete, or modify operations. The file system 100 can utilize ZNS SSDs, thereby significantly reducing storage costs. In some embodiments, the file system 100 can also optionally be used with a hard disk drive (HDD) such as a shingled magnetic recording (SMR) HDD used as a storage device.
[0038] The file system 100 may store the file system metadata 104 in memory during normal operation. Storing the metadata 104 in memory as much as possible may reduce latency during reads and allow writes to require only a single I / O operation.
[0039] The file system 100 may provide consistent high throughput and low (tail) latency for sequential writes and random / sequential reads. The file system 100 may be suitable for supporting online services such as key-value stores and databases.
[0040] The file system 100 can be configured to cater to general files and instant files. General files are characterized by low write intensity and low latency in sequential and random reads. As a non-limiting example, the file system 100 will treat the key-value store SST file (which can be written in batches without frequent disk flushing) as a general file. Instant files are characterized by the need for fast, incremental synchronous writes, while also requiring high sequential and random read performance. For example, the pre-write log (WAL) files in the key-value store need to be frequently flushed to the disk to achieve instant durability, which causes them to be designated as instant files in the file system 100.
[0041] The file system 100 can be accessed through a dedicated client library 102. The client library 102 can provide an API interface for creating, deleting, renaming, reading, and / or writing files.
[0042] Figure 2 Modules of a file system according to an embodiment are shown. The file system 200 includes a file manager 202 , a metadata manager 204 , a task scheduler 206 , a space allocator 208 , and a key-value file system coordinator 210 .
[0043] File manager 202 is configured to perform data placement so as to improve file system performance by reducing synchronous write overhead. Synchronous write overhead can be reduced by separating data and log, pre-allocating data space, and / or maintaining independence between data and log persistence. By separating data and log and adopting the only additional write method of log structure, the write model of file system 200 can be improved, thereby improving performance. File manager 202 can improve the efficiency of synchronous write by merging pre-allocated data space, because only data persistence is needed, eliminating the need for persistent metadata, thereby reducing overhead. By providing separate data persistence and global log persistence for different files, these operations can be performed independently, thereby alleviating a large amount of metadata persistence operations that are usually caused by single data write persistence in traditional file systems.
[0044] The file manager 202 can be configured to handle general files and instant files, as described above. General files can be characterized by relatively low write intensity, while instant files can be files that require fast, incremental, synchronous writes. Pre-allocation of data space can include discrete-sized chunks of storage allocations for general files, thereby allowing metadata to be used for such general files more efficiently. The file manager 202 can determine the nature of whether a file is a general file or an instant file. The nature of whether a file is a general file or an instant file can be determined based on an API, through which the file can be provided or accessed. The nature of whether a file is a general file or an instant file can be determined based on a tag applied to the file.
[0045] The metadata manager 204 is configured to manage file system metadata in memory and on disk. The metadata manager 204 may be configured to perform log compression to free up disk space occupied by stale records and to prepare the log for future metadata updates. Figure 8 A non-limiting example of a journal compression process performed by metadata manager 204 is provided in and described below.
[0046] The task scheduler 206 is configured to manage the data reading and writing process. The task scheduler 206 can classify the writing by task type, such as writing immediate data or general data, and schedule the data writing according to the priority based on the task type. In an embodiment, metadata writing can be sent directly to be executed without being scheduled by the task scheduler 206.
[0047] The space allocator 208 is configured to manage the space within the file system 200. In an embodiment, the space allocator 208 is configured to use three layers (L0, L1, L2) to describe the space. The space allocator 208 can thereby speed up the determination of the state of the space as clear or set without adding excessive overhead. Fig.10 A non-limiting example of the structure implemented by the spatial allocator 208 is provided in and described below.
[0048] The key-value file system coordinator 210 is configured to interface with an associated file system to improve the operation of the key-value system and the file system 200 by coordinating operations in the two systems. As non-limiting examples, the collaboration between the key-value system and the file system facilitated by the key-value file system coordinator 210 may include exposing an API that allows the key-value storage library to mark I / O request priorities, providing a dedicated file type to improve synchronous I / O, reducing or avoiding blocking caused by compression operations, coordinating garbage collection operations to reduce write amplification, providing sector-level redundancy for stored data to improve fault tolerance, and performing synchronization operations without synchronizing metadata.
[0049] Compared to traditional file synchronization and file data synchronization implementations, key-value and file system collaboration can achieve a more efficient approach. When this traditional file synchronization and file data synchronization approach is used with append-only files, two separate I / O operations are triggered: one for updating the data and the other for updating the file size. This results in increased input / output overhead and write amplification factor (WAF). Key-value and file system collaboration can allow a dedicated file data synchronization system call that only updates user data without synchronizing the file size unless a new general file storage unit is allocated for the file to accommodate the new user data. This can avoid unnecessary file metadata synchronization in most cases. Therefore, the file data synchronization implementation through key-value and file system collaboration can avoid performing additional I / O operations caused by traditional file synchronization and file data synchronization, thereby effectively reducing the overall WAF by 1. This can provide improved performance and enhanced data consistency without affecting necessary synchronization tasks.
[0050] Figure 3The disk layout of the file system according to an embodiment is shown. The disk layout 300 includes three different types of data: super block 302, journal 304, and data 306. Super block 302 includes necessary global file system metadata, including journal index node (inode) content, universal unique identifier (UUID) data, version information, block size, and other related parameters. In an embodiment, super block 302 is located at the second 4KB of the storage device, and the positioning of super block 302 is selected to improve accessibility and coordination with other data components. Journal 304 is the data that captures the latest checkpoint from the operation log of the file system. Journal 304 can be effectively replayed during the installation process of the file system, thereby improving the recovery process of file metadata in the memory. Finally, data 306 is dedicated to storing the actual content of each file in the file system. In an embodiment, each data block is organized as a continuous range of logical block addresses (LBA).
[0051] Figure 4A superblock of a file system according to an embodiment is shown. Superblock 400 may include file system metadata and define file system type, size, state, and information about other metadata structures (metadata of metadata). Superblock 400 may provide a high-level metadata structure for a file system. Superblock 400 includes UUID 402, version information 404, block size 406, and log file node data 408. UUID 402 may be a repository of unique and globally identified identifiers that uniquely identify a specific instance of a file system. UUID 402 may ensure that each file system instance remains distinguishable from other file system instances, thereby preventing conflicts and enabling effective management and tracking of each file system. Version information 404 indicates the version number of the file system. The version number may be increased separately after a journal compression operation is performed, thereby tracking the development stage and enhancement of the file system. Therefore, version information 404 may be used to help compatibility management and version control. Block size 406 indicates the size of the file system. Block size 406 may be used as the minimum size for read and write input / output (IO) requests within the file system. The block size 406 can thus support effective and efficient data processing, optimizing data transfer and management operations within the file system. Log file node data 408 can be dedicated to journaling. Log file node data 408 can contain information including the file inode number, file size, last modification time, and a list of discrete chunks of general file data. Each chunk in the list identifies a sequential range of logical block addresses (LBAs) on the device. LBAs can be represented in log file node data 408 as<offset,len,dev_identifier> . dev_identifier specifies the type of device, thereby allowing support for multiple devices. Support for different devices can facilitate the storage of metadata and data across different device types, thereby enhancing scalability and flexibility. In addition, supporting multiple storage devices per server can reduce the management overhead of applications. Therefore, super block 400 can provide improved data management, seamless version control, and adaptability to different hardware configurations.
[0052] Figure 5 FIG. 2 shows a journal file of a file system according to an embodiment. Figure 5 The journal file shown in is a journal that is configured to durably store the metadata of the file system to disk to make the metadata persistent. After the file system is mounted, the journal can be used to restore the metadata in memory. The journal consists of transactions that operate on the journal. The format of the transaction entry is as follows: Figure 6. UUID represents the UUID of the file system that owns the transaction. "seq" is a unique and global sequence identifier used by the transaction. "op_bl" is an encoded operation log entry. "crc" is a cyclic redundancy check for the transaction content, which can be used to verify correctness when reading the transaction content during file system recovery. As non-limiting examples, operation log entries may include: recording a new file node, linking a file to directory data, updating a file node, linking a file to an existing directory, unlinking a file from an existing directory, creating a directory, deleting a directory, updating a file node, such as by creating or appending a file, deleting a file, recording journal information during compression, and / or recording a transaction sequence identifier during compression.
[0053] In this file system, a metadata manager (such as described above and in Figure 2 The metadata manager 202 shown in FIG. 202 can manage file system metadata in memory and on disk. The metadata manager can handle journal compression to free up disk space occupied by outdated log records. The metadata manager can update metadata when changes occur. Figure 6 2 shows the directory and file management in the file system according to the embodiment. Figure 6 As shown in , "dir_map" provides a hash map that associates directory names with their corresponding "file_map". Figure 6 As shown in , "file_map" can manage all "fnodes" of files located in a specific directory. Each fnode maintains a mapping of file blocks to their corresponding physical disk locations, which are organized into a list of discrete chunks for common data.
[0054] according to Figure 6 The organized metadata can be exploited as follows. When provided with a file name, the file system can locate the associated fnode in a timely manner, thereby providing seamless file access. In addition, provided with the requested file offset and length, the file system can quickly determine the logical block address (LBA) of the relevant chunk for common data. Notably, all metadata lookup operations can be performed in memory, ensuring that critical metadata information within the file system can be quickly retrieved. This memory-centric approach enhances the overall performance and reliability of the file system.
[0055] Figure 7A method for making metadata persistent in a file system according to an embodiment is shown. Method 700 includes updating data in a memory 702, synchronizing data to a device 704, and synchronizing metadata to a device 706. Updating data in a memory at 702 includes using an operation to append data to an internal write buffer of a file. When appending data, the system checks whether the length of the write buffer has reached a specified threshold, a non-limiting example of such a threshold is 512KB. If the threshold is reached, the data is flushed to a storage device. By synchronizing data to a device at 704, the data can be flushed to the storage device. Synchronizing data to a device at 704 can include performing a flush operation, wherein the data in the write buffer is written to the storage device. In addition, synchronizing data to a device at 704 can include marking a file and adding the file to a global marked file list to indicate that the file has pending changes. In addition, metadata can be synchronized to a device at 706. Synchronizing metadata to a device at 706 can be performed by calling a function for ensuring data persistence. The function can force the internal write buffer of the file to be written to the storage device to ensure that the data is securely stored. The function may be an "fsync()" function. In an embodiment, all marked records marked during synchronization of data to the device at 704 may be synchronized to a journal file on the device after the last synchronization of log data. This step helps maintain durability and consistency of file metadata changes in various computing environments. In addition, when synchronizing metadata to the device at 706, transaction records for marked files may be updated. The transaction record may be formatted as described in the previous section and then flushed to the journal file. This process may ensure durability and consistency of file metadata changes, thereby enhancing the reliability and performance of key-value storage libraries and file systems in various computing environments.
[0056] Figure 8 FIG. 4 shows compression of journal files in a file system according to an embodiment. A file system may accumulate stale records that may potentially be updated by subsequent metadata changes, resulting in a growing journal size. To address this problem, it is necessary to, for example, Figure 8Free up disk space occupied by stale records and perform journal file compression. This process can be performed in either synchronous or asynchronous mode, each with different characteristics. In synchronous mode, the file system can maintain a global flush lock until the new compression log and update to the superblock are complete. This ensures that the compression operation is performed in a coordinated manner, thereby protecting data integrity. Conversely, in asynchronous mode, the file system can release the flush lock once the in-memory metadata dump request is submitted to the device. This allows data flushing to disk to continue unimpeded while the file system commits metadata update transaction records to disk. By using asynchronous mode, unnecessary delays caused by waiting for disk I / O to complete can be minimized, thereby improving overall system efficiency. These compression modes allow the file system to effectively manage the journal size, thereby improving performance.
[0057] Fig. 9 Task scheduling in a file system according to an embodiment is shown. A task scheduler is configured to manage data read and write operations of a file system. In an embodiment, all data read and write operations of a file system are managed by a task scheduler. For write operations, each write task can be scheduled with a specific priority based on the data type involved in the operation (such as instant data or general data). Metadata writes can be sent directly to a region manager without task scheduling. Therefore, in such an embodiment, there can be three categories of write operations, "instant writes", "general writes", and "metadata writes". Another role of a "polling worker" can also be provided in such an embodiment. For instant writes, each instant file can be bound to a separate region together with an instant queue. For general writes, each general write can be assigned to any available general queue. The polling worker task can send general data to any available region, or preferably attach the general data being written to the same file or general data of a tag prompt. For metadata writes, metadata writes are directed to metadata regions. The file system can give priority to the instant and secure storage of metadata information to maintain the integrity of the file structure and system operation. By sending metadata writes directly to the metadata area, the system can ensure that critical file system information is updated and available for retrieval in a timely manner. The polling worker can use a single thread to direct data flushing. This single-threaded approach processes all queues, effectively flushing their corresponding data. This consolidated approach simplifies the management and coordination of data flushing operations, thereby reducing unnecessary overhead and promoting seamless file management. Using a single polling thread ensures optimal use of system resources, thereby improving performance and responsiveness in the file management system.
[0058] Fig.10 Memory allocation in a file system according to an embodiment is shown. Fig.10In the allocation shown in , three hierarchical layers L0, L1 and L2 are used. The lowest layer is L0, which stores the status of individual allocation units (AUs). AU matches the block size of the file system, such as 4KB or 2MB, which is determined when the file system is made on one or more storage devices. L0 uses 1 bit to indicate the status of each AU, where the 1 bit is 0 means that the AU is allocated, and 1 means that the AU is free. L1 divides the AUs in L0 into groups of 512, which is aligned with the CPU cache line size (64 bytes) for easy search. In L1, each two bits represent the status of the 512 AU states in L0. Among them, two bits of 00 mean that all sub-L0 AUs are cleared, two bits of 01 mean that some sub-L0 AUs are cleared, and two bits of 11 mean that all sub-L0 AUs are set. In L2, each bit represents the status of slot 512 in L1, and the bit of 0 means that all sub-items in L1 are cleared, and the bit of 1 means that some sub-items in L1 are set. Since most of the memory consumption is still caused by L0, this three-level approach reduces the additional overhead while improving the flat bitmap system.
[0059] In an embodiment, the file system may include an API for allowing the key-value store to set I / O priorities, such as Figure 2 A portion of the key-value file system collaborator 210 shown in and described above. Collaboration between the key-value storage library and the file system regarding I / O priorities can reduce tail latency by implementing a user space I / O scheduler that assigns high priorities to foreground I / O tasks and low priorities to background I / O tasks. In addition, the key-value storage library can use a scheduler to perform background tasks to ensure consistent I / O amplification and latency for upper layer applications, thereby achieving stable and low tail latency. In addition, due to the use of storage segments with discrete sizes, metadata I / O is not required to read common files of the file system, so most reads only require a single I / O operation.
[0060] Fig.11 FIG. 2 shows a synchronous input / output (I / O) process in a file system according to an embodiment. Fig.11 The synchronous I / O process shown in the file system ( Fig.11 The "file system" in ) can take advantage of a specialized file type specifically tailored to support this efficient synchronous I / O. A key-value store that works with the file system ( Fig.11 The "key-value store" in the can utilize specially designed files to store and manage data from the logging engine included in the key-value system. Fig.11The synchronization of I / O operations in the file system can improve system performance and responsiveness. Custom file types and the joint design of the file system and key-value store can streamline the synchronization process of input and output operations, thereby improving data processing capabilities and ensuring smoother and faster data storage and retrieval. Fig.11 The synchronous I / O process can be done as follows Figure 2 and coordinated by at least a portion of the key-value file system coordinator 210 as shown in and discussed above.
[0061] Dedicated files allow the key-value store to perform compression tasks without any blocking or disruption to overall system performance. Traditional compression processes can be time-consuming and often cause system slowdowns, affecting overall performance. However, by employing a customized file type, the key-value store can more efficiently manage data organization and compression in a non-blocking manner, maintaining system responsiveness and optimal functionality.
[0062] To mitigate write amplification and optimize the garbage collection process, the file system can collaborate with the key-value store to perform garbage collection efficiently. This collaboration can include sharing the file system's region usage information with the key-value store, enabling smart decisions about file compaction or garbage collection execution. Garbage collection scheduling can be improved by considering garbage rates at the entry level rather than the space level. This ensures that large object (blob) files with the highest garbage space are selected, resulting in more efficient garbage collection operations. In addition, this awareness can allow the garbage collection process to be more flexible, diverse, concurrent, and aware of physical capacity. This will optimize space reclamation, improve concurrency at high watermarks, and enhance overall GC performance.
[0063] Periodic compaction can be performed in the key-value store to merge multiple layers of the same page from the log-structured merge tree into a final layer. This can reduce the amount of data processed during garbage collection and enable the upper layer to perform the corresponding functions. The reduced garbage data can reduce read operations, thereby reducing read amplification in read operations. In addition, "hot" and "cold" data can be separated through the cooperation of the file system and the key-value store, thereby reducing the write amplification problem caused by rewriting cold data when there is mixed hot and cold data.
[0064] Fig.12 Sector-level fault tolerance in a file system according to an embodiment is shown. Fig.12 The interaction between a data storage system, a file system ("file system"), and a key-value store ("key-value store") is shown. Fig.12As shown in , the file system is configured to provide sector-level redundancy for file system metadata. The structure of the file system allows data to be recovered from the data storage system within one or two sector failures. In an embodiment, the key-value system is configured to provide data to the file system to achieve sector-level redundancy for file data of the key-value storage library. Sector-level redundancy can reduce the blast radius of the failure, thereby reducing its impact. The present invention relates to a method for mitigating the potential adverse effects of failures in a data storage system, specifically aiming to reduce the extent of damage caused by such failures. In accordance with this goal, the file system provides a guarantee that data can be successfully recovered in the event of one or two sector failures. To achieve this, a sector-level redundancy approach is adopted, in which the file system is responsible for ensuring redundancy of the file system metadata, and the key-value storage library is entrusted to achieve sector-level redundancy of file data.
[0065] Sector-level fault tolerance techniques improve the robustness and reliability of data storage systems. By including sector-level redundancy in file system metadata and file data, the risk of data loss due to sector failures can be greatly mitigated. Therefore, the proposed method enhances the overall data recovery capability of the system, thereby helping to improve the stability and long-term viability of the system. Applying sector-level redundancy in the management of file system metadata and file data can protect critical information, ultimately helping to enhance user experience and improve system resilience.
[0066] Fig.13 A key-value system integrated with a file system according to an embodiment is shown. The key-value store and file system 1300 includes a key-value system 1302. The key-value system 1302 includes a log engine 1304, a hash engine 1306, a sort engine 1308, and a garbage collection manager 1310. The key-value store and file system 1300 also includes a collaboration layer 1312 and a file system 1314. The key-value store and file system 1300 can interact with a kernel space 1316, which includes one or more disks 1318. The key-value store and file system 1300 can interact with an application 1320.
[0067] The key-value and file system 1300 can be used for storage in cloud applications, for example, to provide data persistence required by cloud services. The key-value system 1302 is configured to provide a key-value storage repository, for example, as part of the storage backend of a cloud service. Non-limiting examples of cloud services using the key-value system 1302 include shopping, social media, metadata management, etc. The file system 1314 can be a dedicated user-level append-only file system that is configured to provide storage specifically to facilitate the operation of the key-value system 1302.
[0068] The log engine 1304 is configured to allow multiple log files to be written simultaneously, thereby reducing the number of compression and garbage collection operations. The logs written by the log engine 1304 can be configured so that strong ordering is not required to process the logs. The log engine 1304 is configured to improve throughput performance issues in log writing and increase recovery speed by reducing the synchronous write overhead of the log from multiple input / output (I / O) operations to a single I / O operation, using lock-free queues to aggregate writes to control latency and improve throughput, and / or providing asynchronous interfaces to enhance the thread model. In the case where the key-value system 1302 and the file system 1314 are integrated and collaborated, the log engine 1304 can be used to store a pre-written log (WAL) with a predefined structure, which has a defined actual file size. The defined file size for the WAL can in turn result in fewer I / O operations being required, thereby improving performance while mitigating potential trade-offs related to data consistency. The operation of the log engine 1304 is Figure 4-Figure 7 are further detailed in and described below.
[0069] The hash engine 1306 is configured to process point queries within the key-value system 1302. In particular, the hash engine 1306 is configured to reduce tail latency in point queries. The hash engine 1306 includes separation of data and index components, and maintaining the index in a cache memory, for example, by compressing the index and / or caching partial data. The partial data may be selected using, for example, a least recently used strategy. The operation of the hash engine 1306 is Figure 8 and Fig. 9 are further detailed in and described below.
[0070] The sorting engine 1308 is configured to perform range scan operations while reducing write amplification factors and / or read / write latency associated with such operations. The sorting engine 1308 is configured to use partitioned log structure merge (LSM) trees. Classification of I / O streams and scheduling of tasks may also be performed by the sorting engine 1308. The operations of the sorting engine 1308 are described in detail in the following sections. Figure 8 and 13 are further detailed in and described below.
[0071] The garbage collection manager 1310 is configured to perform garbage collection and / or compression operations in the key-value and file system 1300. The garbage collection manager 1310 can be configured to reduce unnecessary data movement during garbage collection and / or compression operations in the key-value and file system 1300. The garbage collection manager 1310 can perform garbage collection and / or compression operations based on knowledge of application-side data deletions (such as page expiration). The garbage collection and compression performed by the garbage collection manager 1310 can be configured to arrange data to support other modules such as the sorting engine 1308. The garbage collection manager 1310 can coordinate the preservation of data during garbage collection and / or compression operations. The operation of the garbage collection manager 1310 is Fig.13 are further detailed in and described below.
[0072] The collaboration layer 1312 is configured to facilitate collaboration between the key-value system 1302 and the file system 1314. Based on the collaboration between the key-value system 1302 and the file system 1314, the collaboration layer 1312 can also facilitate efficient compression and / or garbage collection operations in the key-value system 1302. The collaboration can reduce write amplification problems caused by compression and / or garbage collection operations. In an embodiment, the collaboration layer 1312 can expose region usage information from the key-value system 1302 to the file system 1314.
[0073] The file system 1314 can be configured to split data from the log and use a log-structured append-only write as a write model, where append-only operations can be performed. In an embodiment, the file system can also provide pre-allocated data space, where synchronous writes occur only for data persistence, and in an embodiment, metadata persistence is not required. In an embodiment, data persistence and global log persistence for different files can be performed separately. These aspects of the file system can allow the file system to avoid some metadata persistence operations, such as those caused by a single data write persistence operation.
[0074] File system 1314 can be configured to support general files and instant files. General files and instant files can both be written sequentially, and can both be read sequentially or randomly. General files can be optimized for continuous low latency in sequential or random reads. General files can be used to write data in batches, and there is no need to flush data (such as SST files) to disk after each write. Storage space is allocated in large units, and a non-limiting example of unit size is 1MB. Large allocation units can reduce the metadata size of general files so that the metadata of all general files can be stored in memory during normal file system operations. By storing metadata in memory, no matter how the read offset is, the read operation of general files does not require further I / O to access metadata. This can reduce the read tail latency of general files. Instant files can be optimized for fast, incremental synchronous writes, while having good sequential and random read performance near the tail. Instant files can be used to write data that needs to be frequently flushed to disk to achieve instant durability, such as pre-written log files of key-value systems. For instant files, data and metadata written separately each time can be bundled together. Bundled data and metadata can be written to a journal file that is shared by all instant files. Bundling data and writing it to a journal file can improve the speed of incremental writes and sync operations. This approach is structured to support sequential reads, but there may be tradeoffs in terms of random reads. Since instant files are expected to be mostly read sequentially, and random reads are mainly concentrated near the tail, the most recently written data for each instant file that is actively being written can be cached to improve read performance.
[0075] The file system 1314 may include a user space I / O scheduler to assign I / O priorities to different I / O types. The I / O scheduler marks the foreground I / O as a high priority, while the background I / O will be marked as a low priority. In addition, the key value system 1302 may include a scheduler for scheduling its background tasks to ensure that each I / O issued by the upper application has a consistent I / O amplification. By jointly designing I / O scheduling in the key value system 1302 and the file system 1314, since both I / O amplification and I / O latency are consistent, the tail latency can remain stable and low. In addition, reading common files from the file system does not require the I / O of metadata, and using large spaces for common files can ensure that most read operations only require a single I / O.
[0076] Kernel space 1316 may contain disks 1318. Disks 1318 may include one or more storage media, such as solid-state drives (SSDs). In an embodiment, at least some of disks 1318 are zoned storage (ZNS) SSDs.
[0077] Application 1320 is any suitable application that utilizes key-value and file system 1300, for example, online shopping, social media, metadata management applications, etc. Application 1320 may interface with key-value and file system 1300 through any suitable application programming interface (API). In an embodiment, the API may be specific to a particular type of file, for example, whether the nature of the file is a general file or an instant file is determined by the API that receives the file.
[0078] All aspects:
[0079] Aspect 1. A file system, wherein the file system is configured to perform a log-structured append-only write operation, the file system is configured to be used with a key-value store, the file system comprising a file manager configured to keep data and logs separate.
[0080] Aspect 2. A file system according to Aspect 1, wherein the file manager is further configured to determine whether the file is a general file or an instant file based on the writing characteristics of the file, and store the general file in one or more pre-allocated data spaces when the file is a general file.
[0081] Aspect 3. A file system according to Aspect 2, wherein the one or more pre-allocated data spaces are each approximately one megabyte (MB).
[0082] Aspect 4. The file system according to any one of aspects 1-3 further comprises a metadata manager configured to perform compression of a journal file, the journal file containing persistent metadata of the file system.
[0083] Aspect 5. The file system according to aspect 4, wherein compression of the journal file is performed in synchronous mode, wherein in synchronous mode, a global flush lock is held until compression of the journal and update of the super block are completed.
[0084] Aspect 6. The file system according to aspect 4, wherein the compression of the journal file is performed in an asynchronous mode, wherein in the asynchronous mode, once the in-memory metadata dump request is submitted, the global flush lock is released.
[0085] Aspect 7. The file system according to any one of aspects 1-6 further comprises a task scheduler configured to assign a priority to each write task, the priority being based on whether data of the write task is general data or immediate data.
[0086] Aspect 8. The file system according to any one of aspects 1-7, comprising a dedicated journal thread configured to generate a journal, wherein the journal comprises the latest checkpoint from the operation log of the file system.
[0087] Aspect 9. The file system according to aspect 8, wherein the journal comprises one or more transactions, each of the one or more transactions comprising a universally unique identifier, a global sequence identifier, an encoded operation log entry, and a cyclic redundancy check.
[0088] Aspect 10. A file system according to any one of Aspects 1-9, wherein the disk layout of the file system includes a super block, which includes journal index node content, universal unique identifier data, version information, and block size.
[0089] Aspect 11. A file system according to Aspect 10, wherein the disk layout provides the super block at the second 4 kilobyte block provided in the storage device.
[0090] Aspect 13. The file system according to any one of aspects 1-12, wherein the data storage of the file system comprises one or more zoned solid state drives (ZNS SSDs).
[0091] Aspect 14. A system comprising a key-value storage library and a file system according to any one of Aspects 1-13.
[0092] Aspect 15. The system according to Aspect 14, wherein the file system is configured to update data based on the key-value store without updating metadata associated with the data.
[0093] Aspect 16. The system according to any one of Aspects 14-15, comprising a key-value store and file system coordinator configured to provide an interface between the key-value store and the file system.
[0094] Aspect 17. The system according to Aspect 16, wherein the key-value store and the file system coordinator are configured to expose an application programming interface, which allows the key-value store to mark the priority of the input / output request to the file system.
[0095] Aspect 18. The system according to any one of Aspects 16-17, wherein the key-value store and file system coordinator is configured to coordinate compaction and / or garbage collection operations between the key-value store and the file system.
[0096] Aspect 19. A system according to any one of Aspects 16-18, wherein the key-value storage library and the file system collaborator are configured to provide data from the key-value storage library to the file system to achieve sector-level redundancy of file data of the key-value storage library in the file system.
[0097] Aspect 20. The system according to any one of Aspects 16-19, wherein the key-value store and file system coordinator is configured to perform synchronization operations between the key-value store and the file system without synchronizing metadata.
[0098] Aspect 21. The system according to any one of aspects 1-20, wherein the file system is an append-only file system.
[0099] Aspect 22. A system according to any one of Aspects 1-20, wherein the file system is configured to independently write the separated data and log to a persistent memory.
[0100] Aspect 23. A database system comprising: a key-value store, a file system configured to perform log-structured append-only write operations, a file system configured to be used with the key-value store, and a key-value store and file system collaborator configured to provide an interface between the key-value store and the file system, wherein the file system includes a file manager configured to keep data and logs separate.
[0101] Aspect 24. The database system according to Aspect 23, wherein the key-value store and the file system coordinator are configured to expose an application programming interface, which allows the key-value store to mark the priority of the input / output request to the file system.
[0102] Aspect 25. A computer-readable medium comprising instructions that, when executed by a processor, direct the establishment and operation of a file system, the file system being configured to perform log-structured append-only write operations, the file system being configured for use with a key-value store, the file system including a file manager configured to keep data and logs separate.
[0103] The examples disclosed in this application should be considered in all respects as illustrative and not restrictive. The scope of the invention is indicated by the appended claims rather than the foregoing description; and all changes within the meaning and equivalent range of the claims are intended to be embraced therein.
Claims
1. A file system, wherein the file system is configured to perform log-structured append-only write operations, the file system being configured for use with a key-value store, the file system comprising a file manager configured to keep data and logs separate.
2. The file system of claim 1, wherein the file manager is further configured to: determine whether the file is a universal file or an instant file based on the write characteristics of the file, and store the universal file in one or more pre-allocated data spaces when the file is a universal file.
3. The file system of claim 1, further comprising a metadata manager configured to perform compression of a journal file containing persistent metadata for the file system.
4. The file system of claim 3, wherein the compression of the journal file is performed in a synchronous mode, wherein in the synchronous mode a global flush lock is held until the compressed journal and updates to the super block are completed.
5. The file system of claim 3, wherein the compression of the journal file is performed in an asynchronous mode, wherein in the asynchronous mode, a global flush lock is released once an in-memory metadata dump request is submitted.
6. The file system of claim 1, comprising a dedicated journal thread configured to generate a journal, wherein the journal comprises a latest checkpoint from an operational log of the file system.
7. The file system of claim 6, wherein the journal comprises one or more transactions, each of the one or more transactions comprising a universally unique identifier, a global sequence identifier, an encoded operation log entry, and a cyclic redundancy check.
8. The file system of claim 1, wherein the disk layout of the file system includes a superblock, the superblock including journal inode content, universally unique identifier data, version information, and block size.
9. The file system of claim 8, wherein the disk layout provides the superblock at a second 4 kilobyte block provided in a storage device.
10. The file system of claim 8, wherein the data storage of the file system comprises one or more zoned solid state drives (ZNS SSDs).
11. A system comprising the key-value storage library and the file system according to claim 1.
12. The system of claim 11, wherein the file system is configured to update data based on the key-value store without updating metadata associated with the data.
13. The system of claim 11, comprising a key-value store and file system coordinator configured to provide an interface between the key-value store and the file system.
14. The system of claim 13, wherein the key-value store and file system coordinator are configured to expose an application programming interface, the application programming interface allowing the key-value store to mark the priority of input / output requests to the file system.
15. The system of claim 13, wherein the key-value store and file system coordinator is configured to coordinate compaction and / or garbage collection operations between the key-value store and the file system.
16. The system of claim 13, wherein the key-value store and file system coordinator is configured to provide data from the key-value store to the file system to achieve sector-level redundancy for the file data of the key-value store in the file system.
17. The system of claim 16, wherein the key-value store and file system coordinator is configured to perform synchronization operations for the key-value store and the file system without synchronizing metadata.
18. A database system comprising: a key-value store, a file system configured to perform an append operation, a file system configured for use with the key-value store, and a key-value store and file system coordinator configured to provide an interface between the key-value store and the file system, The file system includes a file manager configured to keep data and logs separate.
19. The database system according to claim 18, wherein the key-value store and file system coordinator are configured to expose an application programming interface, the application programming interface allowing the key-value store to mark the priority of input / output requests to the file system.
20. A computer-readable medium comprising instructions that, when executed by a processor, direct the establishment and operation of a file system, the file system being configured to perform additional operations, the file system being configured for use with a key-value store, wherein the file system includes a file manager configured to keep data and logs separate.