Methods and systems for facilitating rapid crash recovery in storage devices

Abstract

A system for facilitating crash recovery is provided. The system receives input / output (I / O) requests for data associated with logical block addresses. The system retrieves the physical location corresponding to the logical block address from a first mapping table associated with a first storage drive, wherein the first mapping table is stored in random access memory including the block device, and the block device driver is stored separately from the first mapping table stored in the block device in system memory. The system accesses the physical location to perform the I / O request. In response to determining a crash associated with the driver, the system restarts the driver to restore access to the first mapping table without rebuilding the first mapping table, which involves reading data from the first storage drive and extracting the mapping between logical and physical addresses.

Description

Technical Field

[0001] This disclosure relates to the field of data storage. More specifically, this disclosure relates to methods and systems for facilitating rapid crash recovery in storage devices.

[0002] Related technologies

[0003] Today, various storage systems are used to store and access ever-increasing volumes of digital content. Storage systems can include a variety of storage devices that provide persistent memory, such as solid-state drives (SSDs) and hard disk drives (HDDs). Open Channel SSDs are a type of SSD that offers transparency and flexibility in managing NAND flash memory. In Open Channel SSDs, the Flash Translation Layer (FTL), along with the associated mapping table of logical address information to physical address information, resides on the host side (e.g., in kernel mode or user space). This host-based FTL allows internal SSD information to be shared with software and can further optimize FTL operations while the host application is executing. This sharing and optimization can improve the performance, cost, reliability, and operational efficiency of the SSD and the entire storage system.

[0004] In host-based FTL (Open Channel Transmission) SSDs, the FTL is typically a program running in system memory. This FTL program is responsible for maintaining the mapping table. The FTL program may crash for various reasons, such as memory problems or host crashes. In the event of an FTL program crash, the system may need to read large amounts of data from the SSD to recover the mapping table's contents in order to rebuild or repair it. This recovery process can be time-consuming and may make it difficult to ensure service restoration within a timeframe sufficient to meet Service Level Agreement (SLA) requirements.

[0005] Therefore, while the host-based FTL can provide transparency and flexibility in managing the physical media of storage drives, it presents some challenges in dealing with the time-consuming problem of rebuilding the mapping table due to FTL program crashes. Summary of the Invention

[0006] One embodiment provides a system for facilitating crash recovery. The system receives an input / output (I / O) request for data associated with a logical block address. The system retrieves a physical location corresponding to the logical block address from a first mapping table associated with a first storage driver, wherein the first mapping table is stored in random access memory containing the block device, and wherein the driver for the block device is stored separately from the first mapping table stored in the block device in system memory. The system accesses the physical location to execute the I / O request.

[0007] In some embodiments, the system determines the size of a mapping table associated with each of a plurality of storage drives, wherein the storage drives include the first storage drive, and the size of the mapping table is based on the capacity of each associated storage drive. The system adds a plurality of mapping tables associated with the plurality of storage drives in sequence to obtain a mapping file, wherein the mapping file includes the first mapping table. The system stores the mapping file in the block device.

[0008] In some embodiments, the I / O request for data is associated with one or more logical block addresses containing the logical block address. The system obtains the physical location corresponding to the one or more logical block addresses by: identifying a first mapping table associated with the first storage drive; identifying a first starting point of the first mapping table based on the sum of the sizes of multiple mapping tables associated with multiple storage drives, which precede the first storage drive and are arranged sequentially; determining a first offset based on the value of the first logical block address among the one or more logical block addresses; and determining a first length associated with the one or more logical block addresses. Each logical block address in the corresponding mapping table corresponds to metadata of the same size. The first mapping table contains logical block addresses ordered according to the values ​​of the contained logical block addresses. Accessing the physical location to execute the I / O request involves accessing the first storage drive at the first starting point, plus the first offset, with the number of cells equal to the first length.

[0009] In some embodiments, the first length is equal to the number of one or more logical block addresses, the number of logical block addresses being increased or decreased according to a predetermined size of the metadata stored in the first mapping table.

[0010] In some embodiments, the content management module communicates with the driver and the block device to manage the additional mapping table of the mapped file. The content management module includes a granular modulator, an access pattern analyzer, a random engine, and a sequential engine.

[0011] In some embodiments, the system determines the access pattern of the requested I / O data using the access pattern analyzer. The system adjusts the size of the cell to be accessed using the granularity modulator. The system determines whether the I / O request is associated with a random read / write operation or a sequential read / write operation.

[0012] In some embodiments, in response to determining a random read or write operation, the system accesses the block device's mapping file at a first-size granularity via the random engine, wherein the random engine includes a read cache. In response to determining a sequential read or write operation, the system accesses the block device's mapping file at a second-size granularity via the sequential engine, which is larger than the first size. The second size is determined based on a prediction of how much data to prefetch from the block device's mapping file.

[0013] In some embodiments, the driver communicates with the block device and the storage drive. The application communicates with the driver based on one or more of the following: communication between the application and the driver; and communication between the application and the driver via a hypervisor. The hypervisor communicates with the driver based on one or more of the following: communication between the hypervisor and the driver; and communication between the hypervisor and the driver via a distributed file system.

[0014] In some embodiments, in response to determining a crash associated with the driver of the block device, the system restarts the driver to restore access to the first mapping table without rebuilding the first mapping table, which involves reading data from the first memory and extracting the mapping between logical and physical addresses. A flash translation layer program running in the driver manages the first mapping table. Determining a crash associated with the driver includes determining a crash associated with the flash translation layer program. Restarting the driver includes restarting the flash translation layer program to restore access to the first mapping table.

[0015] In another embodiment, the system determines a first mapping table associated with a first storage drive, wherein the first mapping table is stored in random access memory (RAM) including a block device, and wherein the block device driver is stored separately from the first mapping table stored in the block device in system memory. In response to determining a crash associated with the driver of the block device, the system restarts the driver to restore access to the first mapping table without rebuilding the first mapping table, which involves reading data from the first storage drive and extracting the mapping between logical and physical addresses. Attached Figure Description

[0016] Figure 1 illustrates the architecture of an exemplary environment for data storage according to existing technology.

[0017] Figure 2A illustrates an exemplary environment according to the prior art, in which the flash conversion layer operates in system memory.

[0018] Figure 2B An exemplary environment according to an embodiment of this application is shown, wherein a flash translation layer operates in a RAM block device.

[0019] Figure 3 An exemplary access hierarchy according to an embodiment of this application is shown.

[0020] Figure 4A A diagram illustrating the recovery process following a host-FTL crash, including mapping table reconstruction, is shown according to existing technology.

[0021] Figure 4B A diagram illustrating the recovery process following a host-FTL crash according to an embodiment of this application is shown.

[0022] Figure 5 An environment with a content management module that provides dynamic access granularity is illustrated according to an embodiment of this application.

[0023] Figure 6 A flowchart of a method for facilitating recovery after a crash, according to embodiments of this application, is presented.

[0024] Figure 7A A flowchart of a method for facilitating the management and access of mapping files according to an embodiment of this application is shown.

[0025] Figure 7B A flowchart of a method for facilitating the management and access of mapping files according to an embodiment of this application is shown.

[0026] Figure 8 An exemplary computer system for facilitating recovery is shown according to an embodiment of this application.

[0027] Figure 9 An exemplary apparatus for facilitating recovery according to embodiments of this application is shown.

[0028] In the accompanying drawings, the same reference numerals refer to the same graphic elements. Detailed Implementation

[0029] The following description is intended to enable those skilled in the art to make and use these embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this disclosure. Therefore, the embodiments described herein are not limited to those shown, but should be considered within the broadest scope consistent with the principles and features disclosed herein.

[0030] Overview

[0031] The embodiments described herein provide a system that places an FTL mapping file (as multiple additional mapping tables corresponding to multiple storage drives) in a random access memory (RAM) disk including block devices, wherein the block device drivers are stored separately from the FTL mapping file or mapping tables in system memory, thereby avoiding the need to reconstruct the mapping tables by reading data from the storage drives and extracting the mapping relationship between logical addresses and physical addresses.

[0032] As described above, in host-based FTL (e.g., open-channel SSDs), the mapping table between logical address information and physical address information resides on the host side (e.g., in kernel mode or user space). This host-based FTL allows internal SSD information to be shared with software and can further optimize FTL operations while host applications are executing. This sharing and optimization can improve the performance, cost, reliability, and operational efficiency of the SSD and the entire storage system.

[0033] In host-based FTL (Open Channel Transmission) SSDs, the FTL is typically a program running in system memory. The FTL program is responsible for maintaining the mapping table. The FTL program can crash for various reasons, such as memory problems or host crashes. In the event of an FTL program crash, the system can rebuild or restore the mapping table by sequentially reading large amounts of data from the SSD. This recovery process can be time-consuming and may make it difficult to guarantee service restoration within a timeframe sufficient to meet Service Level Agreement (SLA) requirements.

[0034] Therefore, while host-based FTL can provide transparency and flexibility in managing the physical media of storage drives, it presents some challenges in dealing with the time-consuming problem of rebuilding the mapping table due to FTL program crashes.

[0035] One solution is to provide a device-based FTL, where the FTL operates as part of the DRAM of a single or embedded microprocessor and storage device. This device-based FTL can be separated from the host CPU and DRAM, allowing the FTL to remain independent of the application. If the host application crashes or fails, the FTL remains unaffected due to its isolation from the host CPU and DRAM. Therefore, this isolation provides decoupling between the application and the FTL.

[0036] However, as shown in Figure 1, this solution is subject to several constraints. First, because the FTL runs on the device side, the host lacks visibility into the FTL, which can make the management and operation of physical storage media (such as NAND) more difficult. Second, the increasing complexity of microprocessor architectures and the growing capacity of in-processor DRAM lead to increased power and cost consumption. Third, developing and debugging firmware (software running on the microprocessor) can be more complex than host-side software. Debugging can be particularly difficult due to the limited information dumped at the time of an incident. Fourth, if the device-based FTL program itself crashes, the system may need to perform a reboot, which can involve a time-consuming process of rebuilding the mapping table (e.g., by reading data from the storage medium and extracting the logical-to-physical mapping).

[0037] The embodiments described herein address the challenges associated with both traditional device-based FTL solutions and crash recovery from host-based FTLs by placing FTL mapping information on the block device's RAM disk. The block device can store the FTL mapping information and can be accessed by a block device driver (e.g., an open-channel SSD driver, as follows). Figure 3 The system manages or operates the FTL file (as shown). Block device drivers may include FTL drivers. Instead of storing the FTL mapping information (e.g., mapping tables and files) together with the FTL driver in system memory, the system can store FTL mapping information separately from the FTL driver in the block device. When the system encounters a crash related to the block device driver, the system only needs to restart the driver to restore access to the FTL file, as follows: Figure 4B As shown, this eliminates or avoids the need to reconstruct the mapping table by reading data from the storage drive and extracting the mapping relationship between logical and physical addresses. In other words, the recovery process can involve restarting the driver to restore access to the FTL mapping file / table without rebuilding the FTL mapping file / table. The block device can store the FTL mapping information as, for example, a mapping file that includes multiple mapping tables corresponding to multiple storage drives.

[0038] Mapping tables in FTL mapping files stored in block devices can be added based on an ordered sequence, as follows: Figure 5As shown. An I / O request can be a request for data associated with one or more logical block addresses. Data can be accessed by retrieving the physical location associated with one or more logical block addresses from a given mapping table corresponding to a physical storage drive (on which the requested data is stored or written). A first LBA can correspond to the first LBA appearing in one or more LBAs in ascending order. The physical location can be determined by: identifying the starting point of the given mapping table (based on the sizes of the preceding mapping tables arranged sequentially in the FTL file); using the value of the first LBA as an index in the given mapping table to determine the offset from the starting point; and determining the length associated with the one or more logical block addresses (based on a fixed size for each LBA entry in the given mapping table). The following relates to... Figure 3 and Figure 5 An exemplary communication is described for accessing information in a mapping table of an FTL file stored in a block device.

[0039] In the embodiments described herein, the system may further include a content management layer running between the Open Channel SSD driver and the block device. The content management layer can manage the mapping table file and adjust access granularity (e.g., I / O size) based on access patterns (e.g., access frequency). The content management layer may use random and sequential engines to perform these functions. The following relates to... Figure 5 An exemplary content management layer is described.

[0040] Therefore, the embodiments described herein improve the time-consuming process of rebuilding mapping tables in the event of a crash by placing FTL files (as additional multiple FTL mapping tables corresponding to storage drives in the system) in the RAM disk as a block device, wherein the FTL driver and the FTL files are stored separately in system memory. The system further uses a content management layer to manage and operate the block device based on access granularity. The FTL mapping files are shared among the storage drives and can be accessed based on the order of the additional mapping tables and the offset and length associated with one or more logical block addresses for an I / O request.

[0041] A "storage system" refers to a complete set of hardware and software components used to facilitate system storage. A storage system may include multiple storage servers and other server clusters. A "storage server" is a computing device that may include multiple storage devices or storage drives. A "storage device" or "storage drive" is a device or drive that provides persistent storage of data with non-volatile memory, such as a solid-state drive (SSD), a hard disk drive (HDD), or a flash-based storage device.

[0042] "Computing device" means any server, device, node, entity, drive, or any other entity that can provide any computing power.

[0043] A "mapping table" refers to a data structure that maps logical addresses to physical addresses or physical locations, as follows: Figure 3 and Figure 5 As shown. An "FTL mapping file" refers to a file or other data structure that includes multiple additional mapping tables.

[0044] "Block device" or "RAM disk" refers to random access memory, including block devices that are part of system memory. In this application, the FTL mapping file is stored in the block device. Following a crash related to a driver controlling, managing, interfaced with, or communicating with the block device, the FTL mapping file can be efficiently and quickly accessed via system calls after the crash.

[0045] The term "crash recovery" refers to the process by which a driver returns to a stable and usable state. In this application, the crash recovery process may include system calls to an FTL mapping file or FTL program running on a block device.

[0046] "Open Channel SSD" refers to a storage device that is part of a storage system in which the FTL program does not reside on the storage device (as in a device-based FTL), but rather resides on the host (as in a host-based FTL). In this disclosure, the host-based FTL is managed by the content management layer and the block device driver that stores the FTL mapping file.

[0047] Exemplary system architectures in the prior art

[0048] Figure 1 illustrates the architecture of an exemplary environment 100 for data storage according to the prior art. Environment 100 may include a host computer comprising central processing units (CPUs) 110 and 130. Each CPU may include multiple cores and may be coupled to multiple dual in-line memory modules (DIMMs). For example, CPU 110 may include cores 112, 114, and 116 and may be coupled to DIMMs 120, 122, and 124. Similarly, CPU 130 may include cores 132, 134, and 136 and may be coupled to DIMMs 140, 142, and 144. The host computer may communicate with a storage device (e.g., an SSD 150) via a host interface 170 and communications 172 and 174. The SSD 150 may also include a microprocessor 152; DRAMs 162 and 164; and NAND flash memory modules 154, 156, 158, and 160.

[0049] The SSD 150 may include an FTL program that runs on the (embedded) microprocessor 152 and is stored in DRAMs 162 and 164. In this device-based FTL of environment 100, because the device-based FTL is isolated from the host, any problems with the host application generally do not affect the operation of the device-based FTL in the SSD 150. While this solution protects the device-based FTL from problems related to the host application, some limitations remain.

[0050] First, because the FTL runs on the device side, the host lacks visibility into it, which can make managing and operating physical storage media (e.g., NAND) more difficult. Second, microprocessor architectures are becoming increasingly complex, driving larger and larger DRAM capacities. These developments lead to increased power and cost consumption. Third, developing and debugging firmware (software running on the microprocessor) can be more complex than host-side software, and debugging can be particularly difficult due to the limited information dumped in the event of an incident. Fourth, if the device-based FTL program itself crashes, the system may need to be restarted, and a time-consuming process of rebuilding the mapping table may be required (e.g., by reading data from the storage medium and extracting the logical-to-physical mapping).

[0051] Therefore, while current device-based FTL solutions can isolate the FTL from the host, some challenges still exist.

[0052] In the prior art, the FTL in system memory and the FTL in RAM block devices

[0053] Figure 2A illustrates an exemplary environment 200 in which a flash translation layer operates in system memory, according to prior art. Environment 200 may include a CPU 210 with a cache 212. CPU 210 may communicate with or be coupled to main memory 220 (e.g., a DRAM DIMM), which may include an operating system 222, an application 224, and an FTL / mapping table 226. In environment 200, FTL 226 may be a module or program that allocates memory in an allocation region and updates the mapping table. However, when the FTL / mapping table 226 program crashes (e.g., a system crash or an application-related crash), the mapping table disappears or becomes unavailable. To recover from such a crash, the system must rebuild the mapping table in a time-consuming, conventional manner, as described above with respect to Figure 1.

[0054] Figure 2BAn exemplary environment 230 is illustrated according to an embodiment of this application, wherein a flash translation layer operates in a RAM block device 256. Environment 230 may include a CPU 240 having a cache 242. The CPU 240 may communicate with or be coupled to main memory 250 (e.g., a DRAM DIMM), which may include an operating system 252, an application 254, and a RAM disk 256 that may store an FTL / mapping table 258. In environment 230, the RAM disk 256 is a block device formed from DRAM (e.g., main memory 250) of system memory. If the system encounters a crash associated with an application or a driver controlling the RAM disk 256, the system can quickly regain access to the FTL / mapping table 258 by restarting the FTL program, for example, by making a system call to the FTL program 258. Therefore, the system can avoid the time-consuming process of reconstructing the mapping table of the FTL mapping file 258 stored in the RAM disk 256.

[0055] Exemplary access hierarchy

[0056] Figure 3 An exemplary access hierarchy 300 according to an embodiment of this application is illustrated. The hierarchy 300 may include: an application 302, a hypervisor 304, a distributed file system 306, an open channel SSD driver 308, a RAM disk 310, and open channel SSDs 312, 314, and 316. During operation, the application 302 may communicate with or directly operate the open channel SSD driver 308 (via communication 346). The application 302 may also communicate with the open channel SSD driver 308 via the hypervisor 304 (via communication 324) for virtualization. The hypervisor 304 may communicate with or directly operate the open channel SSD driver 308 (via communication 344), or may communicate with the open channel SSD driver 308 via the distributed file system 306 (via communications 324 and 326).

[0057] The open channel SSD driver 308 can communicate with RAM disk 310 (which may be a block device formed from system memory) via communication 330. The open channel SSD driver 308 can also communicate with multiple open channel SSDs, such as open channel SSDs 312, 314 and 316 (via communication 332, 334 and 336 respectively).

[0058] RAM disk 310 may store FTL mapping file 340, which may include additional mapping tables 342, 344, and 346 associated with each of the Open Channel SSDs 312-316, the additional mapping tables being sorted in the same order as the associated storage drives. The order may be, for example, SSD 312, SSD 314, and SSD 316 drawn from left to right. Each individual mapping table may be stored as a single unit or as a block within FTL mapping file 340. Note that FTL mapping file 340 may be stored separately from the FTL driver of Open Channel SSD driver 308 (which may be stored in system memory) on RAM disk 310.

[0059] The size of each mapping table can be based on the size or capacity of the corresponding storage drive or SSD. Each SSD's size or capacity may differ, and the size of the associated mapping table may also differ. For example, given a 1000:1 ratio, if SSD 312 has a capacity of 4 terabytes (TB), then mapping table 342 (associated with SSD 312) could be 352 of size 4 gigabytes (GB). Similarly, if SSD 314 has a capacity of 2TB, then mapping table 344 (associated with SSD 314) could be 354 of size 2GB. Furthermore, if SSD 316 has a capacity of 8TB, then mapping table 346 (associated with SSD 316) could be 356 of size 8GB.

[0060] Each mapping table can be organized based on ascending order of LBA values, where each LBA can correspond to metadata of a predetermined size for the mapping table. For example, the metadata can indicate the physical location (including, for example, physical block address (PBA)) where the data corresponding to a given LBA value is stored, and the metadata itself stored in the mapping table can be of a fixed size, such as 10 bytes. Using fixed-size metadata in a given mapping table, the LBA values ​​are arranged in ascending order (e.g., {LBA_1, LBA_2, LBA_3, ... LBA_n}: LBA_1 might correspond to bytes 1 to 10; LBA_2 might correspond to bytes 11 to 20; LBA_3 might correspond to bytes 21 to 30; and LBA_n might correspond to bytes ((n*10)–9) to (n*10). This allows the system to establish addressing using known data formats and the size of data stored on the block device (i.e., in each mapping table). That is, the system can build the mapping table and the address information based on the data structure of the mapping table without requiring a complex content management layer, such as a file system. The following is combined with... Figure 5 An exemplary content management layer is described, which communicates with the driver and the block device.

[0061] Therefore, based on the ascending order of the LBA values ​​in a given mapping table and the known or predetermined size of the metadata stored in the given mapping table, the system can determine the offset and length. The system can determine the offset based on the value of the LBA associated with an incoming I / O request (or the first LBA among one or more LBAs associated with an incoming I / O request). The system can determine the length based on the number of one or more LBAs associated with an incoming I / O request and the predetermined or fixed size of the metadata stored in the given mapping table (e.g., length = number of LBAs * fixed size of metadata in the given mapping table).

[0062] Furthermore, the system can determine the starting point of each mapping table based on the cumulative size of prior mapping tables, wherein the prior mapping tables are determined based on an ordered sequence. That is, the system can determine the starting point of a given mapping table associated with a given storage drive, the starting point being determined based on the sum of the sizes of multiple mapping tables associated with multiple storage drives that precede the given mapping table in the sequence.

[0063] Since mapping table 342 is the first of several corresponding mapping tables appended to FTL mapping file 340, the starting point 362 of mapping table 342 can be 0. The starting point 364 of mapping table 344 can have a value equal to the sum of the sizes of the preceding mapping tables, i.e., the length or size 352 of the preceding mapping table 342. The starting point 366 of mapping table 346 can have a value equal to the sum of the sizes of the preceding mapping tables, i.e., a value at least equal to the size 352 of the preceding mapping table 342 plus the size 354 of the preceding mapping table 344 (plus the sizes of any other preceding mapping tables after mapping table 344 and before mapping table 346, not shown).

[0064] Furthermore, when processing an incoming I / O request, the system can determine the offset and length associated with the I / O request and access the physical location or physical block address in a given storage drive by adding the offset to the determined starting point, the number of units (e.g., bytes) being equal to the length. For example, suppose LBA_e corresponds to or maps to metadata 376, LBA_f corresponds to or maps to metadata 378, and LBA_g corresponds to or maps to metadata 380. When processing an I / O request for data associated with logical block addresses LBA_e, LBA_f, and LBA_g (“three incoming LBAs”), the open channel SSD driver 308 can receive the I / O request and determine that SSD 314 is the storage device to be accessed. Driver 308 can access RAM disk 310 to retrieve the physical location associated with the incoming LBA from mapping table 344 (associated with SSD 314). The system can determine the starting point 364 of the mapping table 344 (as described above) and the offset 392 (based on the value of LBA_e, the first of the three input LBAs, corresponding to metadata 376). The system can start reading data from the starting point 364 plus the offset 392, with a length of 394 (determined based on the number of input LBAs (three) multiplied by the fixed size of the metadata of the mapping table 344).

[0065] Therefore, hierarchy 300 describes both the communication hierarchy in the described embodiments and the way to access a given mapping table stored in an FTL mapping file, which is part of a plurality of ordered and additive mapping tables associated with an ordered sequence of storage drives.

[0066] Mapping table reconstruction in existing technologies and rapid recovery from crashes using block devices

[0067] Figure 4A Figure 400 illustrates a recovery process following a host-FTL crash according to prior art, including mapping table reconstruction. In Figure 400, when the system experiences a crash associated with the open-channel SSD drive (legacy) 402 (e.g., a host-side FTL crash), the system must restart the FTL program (as shown in "Crash Reboot"). The open-channel SSD drive (new) 412 must reallocate memory and load data from the open-channel SSD 414 (via communication 420) to rebuild a mapping table (memory allocation) 404 (as shown in "Crash Reconstruction"). As mentioned above, rebuilding or reconstructing the mapping table in this manner can be a time-consuming process, which may not adequately meet the terms of the SLA.

[0068] Figure 4BFigure 440 illustrates a recovery process following a host-FTL crash according to an embodiment of this application. In Figure 440, when the system experiences a crash associated with the open channel SSD driver (legacy) 442 (e.g., a host-side FTL crash), the system only needs to reboot or restart the FTL program (e.g., a "crash reboot"). Instead of reallocating memory from the open channel SSD 454 and loading data to rebuild or reconstruct the mapping table (e.g., ... Figure 4A According to the prior art (Figure 400), the open channel SSD driver (new) 452 only needs to use system call 462 to access the mapping table 444 (such as stored in RAM block device 444). As a result, the system essentially only loses the communication 460 between the open channel SSD drivers (old) 442, and does not need to communicate with the SSD 454 at all to rebuild or reconstruct the mapping table (such as missing...). Figure 4B (See the "Crash Rebuild" tab). Note that the FTL mapping table stored in RAM block device 444 can be stored separately from the FTL driver of the SSD driver or block device driver (442 or 452), rather than stored together, the latter of which can be stored in system memory. Therefore, the system can eliminate the need to rebuild or reconstruct the mapping table, as the mapping table remains unaffected based on its location in RAM block device 444, is storable, and can be quickly accessed via system call 462, which facilitates rapid crash recovery.

[0069] In this way, Figure 4B The system described herein can avoid, eliminate, or eliminate the need for mapping table reconstruction (as indicated by improvement 430 to the prior art environment of Figure 400), thereby eliminating the need for time-consuming processes that may not adequately meet SLA terms.

[0070] The described embodiments provide solutions and improvements for scenarios where the host-side FTL crashes, such as in open-channel SSDs. In rare cases of a complete server crash (e.g., during a reboot), the entire server will require time to restart correctly. Because the FTL mapping file is stored on a block device (RAM disk) and runs on volatile system memory, the system can rebuild the mapping table as needed during the time required for the entire server reboot. In other words, the embodiments described herein address scenarios where the FTL program, the FTL mapping file of a block device, or the associated block device driver crashes, and provide improvements thereto.

[0071] Content management module with dynamic granular access

[0072] Figure 5An embodiment of this application illustrates an environment 500 having a content management module 512 that provides dynamic access granularity. In environment 500, an OpenChannel driver 510 communicates with a block device 514 via the content management module 512. The content management module 512 may include: a granularity modulator 520 that adjusts the size of data units to access an FTL mapping file stored in the block device 514; an access pattern analyzer 522 that determines the access pattern of a given or requested data or corresponding metadata; a random engine 524; and a sequential engine 526. The block device 514 may store an FTL mapping file 540, which may include metadata of mapping tables associated with one or more storage drives. For example, the FTL mapping file 540 may include metadata corresponding to LBAs, which are ordered in ascending order based on LBA values, such as metadata 542, 544, 546, 548, 550, 552, and 554.

[0073] The system can determine whether an I / O request is associated with a random read / write operation or a sequential read / write operation. The system can use a random engine 524 in response to determining a random read or write operation. The random engine 524 can access the FTL mapping file 540 of the block device 514 at a granularity of a first size (e.g., small size). The random engine 524 may also include a read cache (not shown) to increase data hit rate and reduce the number of queries to the block device 514. The system can use a sequential engine 526 in response to determining a sequential read or write operation, for example, by predicting a relatively large I / O size and prefetching mapping information (e.g., metadata or physical location information) corresponding to one or more incoming LBAs. The sequential engine 526 can access the FTL mapping file 540 of the block device 514 at a granularity of a second size (e.g., large size or a size greater than the first size).

[0074] As described above, when processing I / O requests and accessing the FTL mapping file 540 stored in block device 514, the system can determine the starting point of a given mapping table in the FTL mapping file 540 (as described above). Figure 5 (Not shown). The system can also determine an {offset, length} 560 associated with the incoming I / O data. For multiple units equal to length 560 (represented as size or length 564 in the FTL mapping file 540), the system can retrieve data starting from position 562 (which may include a starting point plus offset 560). In some embodiments, offset 560 may include the starting point, wherein the system determines a single offset value (or the first specific LBA value among multiple LBA values) by first identifying the starting point of a given mapping table and moving to the correct position in the given mapping table based on an offset determined by a specific LBA.

[0075] Methods to facilitate data recovery

[0076] Figure 6 A flowchart 600 of a method for facilitating recovery after a crash, according to an embodiment of this application, is presented. During operation, the system receives an input / output (I / O) request for data associated with a logical block address (operation 602). The system retrieves the physical location corresponding to the logical block address from a first mapping table associated with a first storage driver, wherein the first mapping table is stored in random access memory (RAM) including the block device, and the driver for the block device is stored separately in system memory from the first mapping table stored in the block device (operation 604). The system accesses the physical location to perform the I / O request (operation 606). If the system does not determine a crash associated with the driver controlling the block device (or the driver of the block device) (decision 608), the operation returns.

[0077] If the system determines a crash associated with the driver controlling the block device (or the driver of the block device) (decision 608), the system restarts the driver to restore access to the first mapping table without rebuilding the first mapping table, which involves reading data from the first storage drive and extracting the mapping between logical addresses and physical addresses (operation 610). The operation returns.

[0078] A method for facilitating the management of mapping files and dynamic access granularity

[0079] Figure 7A A flowchart 700 illustrates a method for facilitating the management and access of mapping files according to embodiments of this application. During operation, the system determines the size of a mapping table associated with each of a plurality of storage drives, wherein the storage drives include at least a first storage drive, and the size of the mapping table is based on the capacity of each associated storage drive (operation 702). The system adds a plurality of mapping tables associated with the plurality of storage drives according to the sorting order of the storage drives to obtain a mapping file, wherein the mapping file includes at least the first mapping table (operation 704). The system stores the mapping file in random access memory (RAM) including block devices, wherein the drivers for the block devices are stored separately from the mapping file stored in the block devices in system memory (operation 706). The system receives I / O requests for data associated with one or more logical block addresses (operation 708). The system retrieves the physical location corresponding to one or more logical block addresses from the mapping file (operation 710). Figure 7B Continue operating at label A.

[0080] Figure 7BA flowchart 720 illustrates a method for facilitating the management and access of mapped files according to embodiments of this application. The system identifies a first mapping table associated with a first storage drive (operation 722). The system identifies a first starting point of the first mapping table based on the sum of the sizes of the mapping tables associated with the storage drives, which are sequentially arranged before the first storage drive (operation 724). The system determines a first offset based on the value of a first logical block address in one or more logical block addresses (operation 726). This "first" LBA is the first LBA value appearing in an ordered sequence of one or more LBAs. The system determines a first length associated with one or more logical block addresses, wherein each logical block address corresponds to metadata of the same size in its respective mapping table. The first mapping table includes logical block addresses that are ordered according to the values ​​of the included logical block addresses (operation 728).

[0081] The system determines a physical location based on one or more logical block addresses, a first starting point, a first offset, and a first length (operation 730). The system accesses the physical location to execute the I / O request based on accessing the first storage drive from the first starting point, plus a first offset equal to the number of cells equal to the first length (operation 732). The first length is equal to the number of one or more logical block addresses, which is incremented or decremented according to a predetermined size of metadata stored in the first mapping table. Figure 6 Continue the operation at point 608.

[0082] Determining and accessing physical locations can involve one or more of the following: access pattern analyzer, granular modulator, random engine, and sequential engine, as described above. Figure 5 As described.

[0083] Exemplary computer systems and devices

[0084] Figure 8An exemplary computer system facilitating recovery according to an embodiment of this application is illustrated. The computer system 800 includes a processor 802, volatile memory 806, and a storage device 808. In some embodiments, the computer system 800 may include a controller 804 (indicated by dashed lines). The volatile memory 806 may include, for example, random access memory (RAM) used as managed memory. The volatile memory 806 can be used to store one or more memory pools and form a block device. The storage device 808 may include persistent storage that can be managed or accessed by the processor 802 (or controller 804). Furthermore, the computer system 800 may be coupled to a peripheral input / output (I / O) user device 810, such as a display device 811, a keyboard 812, and a pointing device 814. The storage device 808 may store an operating system 816, a content processing system 818, and data 836.

[0085] Content processing system 818 may include instructions that, when executed by computer system 800, cause computer system 800 or processor 802 to perform the methods and / or processes described in this disclosure. Specifically, content processing system 818 may include instructions for receiving and transmitting data packets, including data to be read or written, input / output (I / O) requests (e.g., read requests or write requests), metadata, logical block addresses (LBAs), and physical block addresses (PBAs) or physical locations (communication module 820).

[0086] The content processing system 818 may further include instructions for receiving input / output (I / O) requests for data associated with a logical block address (communication module 820). The content processing system 818 may include instructions for retrieving a physical location corresponding to a logical block address from a first mapping table associated with a first storage drive, wherein the first mapping table is stored in random access memory (RAM) including block devices, and wherein the block device driver is stored separately from the first mapping table (mapping file management module 822) stored in the block device in system memory. The content processing system 818 may further include instructions for accessing the physical location to perform the I / O request (physical location access module 824). The content processing system 818 may include instructions for restarting the driver to restore access to the first mapping table in response to determining a crash associated with the driver controlling the block device (driver crash determination module 826) without rebuilding the first mapping table, which involves reading data from the first storage drive and extracting the mapping between logical and physical addresses (driver restart module).

[0087] The content processing system 818 may also include instructions for determining the size of a mapping table, adding a mapping table associated with a storage drive to obtain a mapping file, and storing the mapping file in a block device (mapping file management module 822). The content processing system 818 may include instructions for retrieving a physical location by identifying a starting point and determining an offset and length (physical location access module 824).

[0088] Content processing system 818 may further include instructions for determining the access pattern of requested I / O data (access pattern analysis module 832). Content processing system 818 may include instructions for adjusting the size of the cell to be accessed (granularity adjustment module 830). Content processing system 818 may include instructions for determining whether an I / O request is associated with a random read / write operation or a sequential read / write operation (data processing module 834). Content processing system 818 may further include instructions for accessing a block device's mapping file at a first-size granularity by a random engine in response to determining a random read or write operation (data processing module 834), wherein the random engine includes a read cache (mapping file management module 822). Content processing system 818 may include instructions for accessing a block device's mapping file with a second-size granularity by a sequential engine in response to determining a sequential read or write operation (data processing module 834), the second size being larger than the first size (mapping file management module 822).

[0089] Data 836 may include any data required as input or output by the methods and / or processes described in this disclosure. Specifically, data 836 may store at least: data; I / O requests; metadata; data associated with logical block addresses (LBAs); logical block addresses (LBAs); physical block addresses (PBAs); physical locations; mapping tables; mapping files or FTL mapping files; logical-to-physical mappings; identifiers or indicators for storage drives or associated mapping tables; indications; sequences, ascending or ordered sequences; starting points; offsets; lengths; dimensions; a number of cells; a number of LBAs; identifiers or indicators for content management modules, access pattern analyzers, granular modulators, random engines or sequential engines; identifiers or indicators for applications, drivers, hypervisors, distributed file systems or block devices; flash translation layer programs and related information; and system calls to restart drivers.

[0090] Figure 9 An exemplary device 900 for facilitating recovery according to embodiments of this application is illustrated. Device 900 may include multiple units or devices that can communicate with each other via wired, wireless, quantum optical, or electrical communication channels. Device 900 may be implemented using one or more integrated circuits and may include components larger than... Figure 9The illustrated unit or device may be fewer or more units or devices. Furthermore, device 900 may be integrated into a computer system, or implemented as a standalone device or a device capable of communicating with other computer systems and / or devices.

[0091] Device 900 may include modules or units 902-916 configured to perform similar functions. Figure 8 The functions or operations of modules 820-834 of the computer system 800 include: a communication unit 902; a mapping file management unit 904; a physical location access unit 906; a driver crash determination unit 908; a driver restart unit 910; a granularity adjustment unit 912; an access mode analysis unit 914; and a data processing unit 916.

[0092] The data structures and code described in this detailed description section are typically stored on a computer-readable storage medium, which can be any device or medium capable of storing code and / or data for use by a computer system. Computer-readable storage media include, but are not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tapes, CDs (optical discs), DVDs (digital versatile optical discs or digital video discs), or other computer-readable media now known or hereafter developed.

[0093] The methods and processes described in the detailed description section may be presented as code and / or data, which may be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and / or data stored on the computer-readable storage medium, the computer system executes the methods and processes presented as data structures and code and stored in the computer-readable storage medium.

[0094] Furthermore, the aforementioned methods and processes can be incorporated into a hardware module. For example, the hardware module may include, but is not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), and other programmable logic devices now known or developed in the future. When the hardware module is activated, it executes the methods and processes contained within it.

[0095] The foregoing embodiments described herein are for illustrative and descriptive purposes only. They are not intended to be exhaustive or to limit the embodiments described herein to the forms disclosed. Therefore, many modifications and variations will be apparent to those skilled in the art. Furthermore, the foregoing disclosure is not intended to limit the embodiments described herein. The scope of the embodiments described herein is defined by the appended claims.

Claims

1. A computer-implemented method comprising: determining, for each of a plurality of storage drives, a size of a mapping table associated with the respective storage drive, wherein the storage drives include a first storage drive and the size of the mapping table is based on a capacity of the associated storage drive; adding a plurality of mapping tables associated with the plurality of storage drives according to an order of the storage drives to obtain a mapping file, wherein the mapping file includes a first mapping table associated with the first storage drive, and storing the mapping file in a random access memory (RAM) of a block device, wherein a driver of the block device is stored in a system memory separately from the first mapping table stored in the block device; receiving an input / output (I / O) request for data associated with a logical block address; retrieving a physical location corresponding to the logical block address from the first mapping table associated with the first storage drive; and accessing the physical location to perform the input / output (I / O) request.

2. The method of claim 1, wherein, the input / output (I / O) request is associated with one or more logical block addresses containing the logical block address, and the method further comprises retrieving the physical location corresponding to the one or more logical block addresses by: identifying the first mapping table associated with the first storage drive; identifying a first starting point of the first mapping table based on a sum of sizes of a plurality of mapping tables associated with a plurality of storage drives that precede and are in order of the first storage drive; determining a first offset based on a value of a first logical block address of the one or more logical block addresses; and determining a first length associated with the one or more logical block addresses, wherein each logical block address in a respective mapping table corresponds to a same size of metadata, and wherein the first mapping table includes logical block addresses that are sequentially ordered based on values of the contained logical block addresses, and wherein accessing the physical location for performing the input / output (I / O) request is accessing the first storage drive at the first starting point plus the first offset, the number of units being equal to the first length.

3. The method of claim 2, the first length is equal to a number of the one or more logical block addresses, the number of logical block addresses being incremented or decremented by a predetermined size of metadata stored in the first mapping table. wherein 4. The method of claim 1, a content management module communicates with the driver and the block device to manage the additional mapping tables of the mapping file, and wherein wherein the content management module includes a granularity modulator, an access pattern analyzer, a random engine, and a sequential engine.

5. The method of claim 4, further comprising: determining, by the access pattern analyzer, an access pattern of the requested I / O data; adjusting, by the granularity modulator, a size of units to access; and determining whether the input / output (I / O) request is associated with a random read / write operation or a sequential read / write operation. ​ ​ 6. The method of claim 5, further comprising: in response to determining a random read or write operation, the random engine accessing the mapping file of the block device at a first size of granularity, wherein the random engine comprises a read cache; and in response to determining a sequential read or write operation, the sequential engine accessing the mapping file of the block device at a second size of granularity greater than the first size, wherein the second size is determined based on a prediction of how much data is to be prefetched from the mapping file of the block device.

7. The method of claim 1, wherein the driver in communication with the block device and the storage drives, wherein an application is in communication with the driver based on one or more of: communication between the application and the driver; and communication between the application and the driver through a hypervisor, wherein the hypervisor is in communication with the driver based on one or more of: communication between the hypervisor and the driver; and communication between the hypervisor and the driver through a distributed file system.

8. The method of claim 1, further comprising: in response to determining a crash associated with the driver of the block device, restarting the driver to resume access to the first mapping table without rebuilding the first mapping table, which involves reading data from the first storage drive and extracting mapping relationships between logical addresses and physical addresses; wherein a flash translation layer program running in the driver manages the first mapping table, wherein determining a crash associated with the driver includes determining a crash associated with the flash translation layer program, and wherein restarting the driver includes restarting the flash translation layer program to resume access to the first mapping table.

9. A computer system comprising: a processor; and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the processor to perform a method, the method comprising: determining, for each of a plurality of storage drives, a size of a mapping table associated with the respective storage drive, wherein the storage drives comprise a first storage drive and the size of the mapping table is based on a capacity of the respective storage drive; adding a plurality of mapping tables associated with the plurality of storage drives in an order of precedence of the storage drives to obtain a mapping file, storing the mapping file in a random access memory (RAM) of a block device, wherein the mapping file comprises a first mapping table associated with the first storage drive, a driver of the block device is stored in a system memory separately from the first mapping table stored in the block device; receiving an input / output (I / O) request for data associated with a logical block address; retrieving a physical location corresponding to the logical block address from the first mapping table associated with the first storage drive; and accessing the physical location to perform the input / output (I / O) request.

10. The computer system of claim 9, wherein, The input / output (I / O) request is associated with one or more logical block addresses including the logical block address, and the method further comprises retrieving a physical location corresponding to the one or more logical block addresses by: identifying a first mapping table associated with the first storage drive; identifying a first starting point of the first mapping table based on a sum of sizes of a plurality of mapping tables associated with a plurality of storage drives, the plurality of storage drives being arranged in a sequence before the first storage drive; determining a first offset based on a value of a first logical block address of the one or more logical block addresses; and determining a first length associated with the one or more logical block addresses, wherein each logical block address in a respective mapping table corresponds to a same size of metadata, and wherein the first mapping table includes logical block addresses that are sequentially ordered based on values of the included logical block addresses, and wherein accessing the physical location for performing the input / output (I / O) request is accessing the first storage drive at the first starting point plus the first offset, the number of units being equal to the first length.

11. The computer system of claim 10, the first length being equal to a number of the one or more logical block addresses, the number of logical block addresses being incremented or decremented by a predetermined size of metadata stored in the first mapping table. wherein 12. The computer system of claim 9, a content management module in communication with the driver and the block device to manage additional mapping tables of the mapping file, and wherein, wherein the content management module includes a granularity modulator, an access pattern analyzer, a random engine, and a sequential engine.

13. The computer system of claim 12, wherein the method further comprises: determining, by the access pattern analyzer, an access pattern of requested I / O data; adjusting, by the granularity modulator, a size of units to access; and determining whether the input / output (I / O) request is associated with a random read / write operation or a sequential read / write operation.

14. The computer system of claim 13, wherein the method further comprises: in response to determining a random read or write operation, the random engine accessing the mapping file of the block device at a first size of granularity, wherein the random engine includes a read cache; and in response to determining a sequential read or write operation, the sequential engine accessing the mapping file of the block device at a second size of granularity that is greater than the first size, wherein the second size is determined based on a prediction of how much data is to be prefetched from the mapping file of the block device.

15. The computer system of claim 9, the driver being in communication with the block device and the storage drive, wherein an application is in communication with the driver based on one or more of: wherein communication between the application and the driver; and communication between the application and the driver through a hypervisor, wherein the hypervisor is in communication with the driver based on one or more of: communication between the hypervisor and the driver; and communication between the hypervisor and the driver through the application. communication between the hypervisor and the driver; and communication between the hypervisor and the driver via a distributed file system.

16. The computer system of claim 9, wherein the method further comprises: in response to determining a crash associated with the driver of the block device, restarting the driver to regain access to the first mapping table without rebuilding the first mapping table, the rebuilding involving reading data from the first storage drive and extracting mapping relationships between logical addresses and physical addresses; wherein a flash translation layer program running in the driver manages the first mapping table, wherein determining a crash associated with the driver includes determining a crash associated with the flash translation layer program, and wherein restarting the driver includes restarting the flash translation layer program to regain access to the first mapping table.

17. A computer-implemented apparatus comprising: a mapping file management unit configured to determine a size of a mapping table associated with each of a plurality of storage drives, wherein the storage drives include a first storage drive and the size of the mapping table is based on a capacity of the associated storage drive; add a plurality of mapping tables associated with the plurality of storage drives in an order of precedence of the storage drives to obtain a mapping file, store the mapping file in a random access memory (RAM) of a block device, wherein the mapping file includes a first mapping table associated with the first storage drive, a driver of the block device is stored in a system memory separately from the first mapping table stored in the block device; a communication unit configured to receive an input / output (I / O) request for data associated with a logical block address; the mapping file management unit is further configured to retrieve a physical location corresponding to the logical block address from the first mapping table associated with the first storage drive; and a physical location access unit configured to access the physical location to perform the input / output (I / O) request.

18. The apparatus of claim 17, further comprising: a driver crash determination unit configured to determine a crash associated with a driver of a block device; a driver restart unit configured to, in response to determining a crash associated with the driver of the block device, restart the driver to regain access to the first mapping table without rebuilding the first mapping table, the rebuilding involving reading data from the first storage drive and extracting mapping relationships between logical addresses and physical addresses; wherein a flash translation layer program running in the driver manages the first mapping table, wherein determining a crash associated with the driver includes determining a crash associated with the flash translation layer program, and wherein restarting the driver includes restarting the flash translation layer program to regain access to the first mapping table.

19. A computer-implemented method comprising: determining, for each of a plurality of storage drives, a size of a mapping table associated with the respective storage drive, wherein the storage drives include a first storage drive and the size of the mapping table is based on a capacity of the associated storage drive; adding a plurality of mapping tables associated with the plurality of storage drives according to an order of the storage drives to obtain a mapping file, storing the mapping file in a random access memory (RAM) of a block device, wherein the mapping file includes a first mapping table associated with the first storage drive, and a driver of the block device is stored in a system memory separately from the first mapping table stored in the block device; determining the first mapping table associated with the first storage drive; and in response to determining a crash associated with the driver of the block device, restarting the driver to restore access to the first mapping table without rebuilding the first mapping table, which involves reading data from the first storage drive and extracting a mapping relationship between logical addresses and physical addresses.

Images