Storage device providing heterogeneous namespaces and its application in databases

Abstract

Provided are storage devices providing heterogeneous namespaces and their application in databases. The disclosed storage device includes a first logical storage device and a second logical storage device, wherein the effect of writing data to the first logical storage device on the life of the storage device is less than the effect of writing data to the second logical storage device on the storage device.

Description

Technical Field

[0001] This application relates to storage technologies, and more particularly to storage devices that provide heterogeneous namespaces, and to optimizing database performance using the provided storage devices. Background Technology

[0002] Figure 1 A block diagram of a storage device is shown. Storage device 100 is coupled to a host computer to provide storage capabilities. The host computer and storage device 100 can be coupled in various ways, including but not limited to connections via SATA (Serial Advanced Technology Attachment), SCSI (Small Computer System Interface), SAS (Serial Attached SCSI), IDE (Integrated Drive Electronics), USB (Universal Serial Bus), PCIe (Peripheral Component Interconnect Express), NVMe (NVM Express), Ethernet, Fibre Channel, and wireless communication networks. The host computer can be an information processing device capable of communicating with the storage device via the above methods, such as a personal computer, tablet computer, server, laptop computer, network switch, router, cellular phone, or personal digital assistant. Storage device 100 includes interface 110, control unit 120, one or more NVM chips 130, and DRAM (Dynamic Random Access Memory) 140.

[0003] NAND flash memory, phase change memory, FeRAM (Ferroelectric RAM), MRAM (Magnetic Random Access Memory), and RRAM (Resistive Random Access Memory) are common types of NVM.

[0004] Interface 110 is compatible with exchanging data with the host via methods such as SATA, IDE, USB, PCIe, NVMe, SAS, Ethernet, and Fibre Channel.

[0005] Control unit 120 is used to control data transfer between interface 110, NVM chip 130, and DRAM 140. It is also used for memory management, mapping host logical addresses to NVM chip physical addresses, erase leveling, bad block management, etc. Control unit 120 can be implemented in various ways, including software, hardware, firmware, or a combination thereof. For example, control unit 120 can be in the form of an FPGA (Field-programmable gate array), ASIC (Application Specific Integrated Circuit), or a combination thereof. Control unit 120 may also include a processor or controller, in which software executes to manipulate the hardware of control unit 120 to process I / O (Input / Output) commands. Control unit 120 can also be coupled to DRAM 140 and can access data in DRAM 140. FTL tables and / or cached I / O command data are stored in DRAM.

[0006] The control unit 120 includes a flash interface controller (or media interface, media interface controller, flash channel controller), which is coupled to the NVM chip 130 and issues commands to the NVM chip 130 in accordance with the interface protocol of the NVM chip 130 to operate the NVM chip 130, and receives the command execution results output from the NVM chip 130. Known NVM chip interface protocols include "Toggle" and "ONFI".

[0007] A memory target is one or more logic units (LUNs) within a NAND flash memory package that share a chip enable (CE) signal. A NAND flash memory package includes one or more dies. Typically, a logic unit corresponds to a single die. A logic unit may include multiple planes. Multiple planes within a logic unit can be accessed in parallel, while multiple logic units within a NAND flash memory chip can execute commands and report status independently of each other. The meanings of target, logic unit (LUN), and plane are provided in the "Open NAND Flash Interface Specification (Revision 3.0)" available at http: / / www.micron.com / ~ / media / Documents / Products / Other%20Documents / ONFI3_0Gold.ashx, which is part of the prior art.

[0008] NVM storage media typically store and retrieve data in pages, while erasing data in blocks. A block (also called a physical block) on an NVM storage medium contains multiple pages. A page (called a physical page) on the storage medium has a fixed size, such as 17664 bytes. Physical pages can also have other sizes.

[0009] In storage devices, the FTL (Flash Translation Layer) is used to maintain the mapping information from logical addresses to physical addresses. Logical addresses constitute the storage space of the storage device as perceived by upper-layer software such as the operating system. Physical addresses are the addresses used to access the physical storage units of the solid-state storage device. In existing technologies, address mapping can also be implemented using intermediate address formats. For example, a logical address can be mapped to an intermediate address, and then the intermediate address can be further mapped to a physical address. Optionally, the host accessing the storage device provides the FTL.

[0010] A table structure that stores mapping information from logical addresses to physical addresses is called an FTL table. Typically, the data entries in an FTL table record the address mapping relationships in the storage device, on a per-data-page basis.

[0011] Some storage devices expose their physical address space to upper-layer software such as operating systems. For example, Open-Channel Solid State Drives (OCDs) are described in detail in the "Open-Channel Solid State Drives Specification" available at http: / / lightnvm.io / docs / OCSSD-2_0-20180129.pdf, the full text of which is incorporated herein by reference. The Open-Channel Solid State Drives specification provides extensions to the NVMe protocol. According to the specification, hosts access storage devices using physical addresses, and the storage address space provided by the storage device to the host is an address space composed of physical addresses.

[0012] Databases use storage devices to store the data they manage. Typically, the performance of storage devices is inferior to the host memory running the database software. To achieve better performance, databases often maintain a cache in memory; data is written to memory for high performance, while cached data is written to storage devices for non-volatile capabilities.

[0013] However, in some cases, the page size of the cache in memory differs from the storage unit size of the I / O commands provided by the storage device. For example, the cache page size in memory might be 4KB, 8KB, or 16KB, while the smallest storage unit operated on by the I / O commands accessing the storage device might be 512B, 4KB, or another size. In this situation, to write a cache page to the storage device, the host driver or database might generate multiple I / O commands for writing a single cache page to the storage device. Maintaining the atomicity of multiple I / O commands is difficult for storage devices. While storage devices can ensure the atomicity of a single I / O command (the I / O command either all or none of them work on the entire address space accessed), they typically do not guarantee the atomicity of multiple I / O commands as a whole. Therefore, when a database writes a single cache page to the storage device, a partial page write problem may occur, where part of the cache page's content is non-volatilely recorded in the storage device, while another part is not. This will lead to database errors.

[0014] Some databases (e.g., MySQL) additionally maintain a double write buffer, for example, in host memory. Figure 2 This diagram illustrates the use of a dual-write cache in a database.

[0015] The database maintains page caches and double-write caches in the memory of the host running the database. The unit of access for the page cache is the cache page; data to be stored by the database is written to cache pages to improve performance. For example, double-write caches have larger page sizes, such as 1MB. The host running the database is coupled to the storage device.

[0016] To write the data of the cached page to the storage device, the database first copies the data of the cached page to the dual-write cache. Figure 2 In the middle, as instructed by (1), the data in the dual write cache is then non-volatilely written to the dual write cache file on the storage device (e.g., the ibdata1 file managed by the MySQL database). Figure 2 In the middle, as instructed by (2), the data in the double write cache is then non-volatilely written to the data file on the storage device (e.g., the *.idb file managed by the MySQL database). Figure 2 In the double-write cache file, data is organized according to the structure of the double-write cache, so that the data in the double-write cache is written to the double-write cache file sequentially. Data in the data file is organized according to the data table structure of the database, so the data written to the data file is difficult to present in order. After the data in the double-write cache is written to the data file, the database submits the database update operation related to the data written. At this time, even if the data in the data file is corrupted due to atomicity issues, the database can recover the correct data from the double-write cache file.

[0017] Understandably, dual write cache files and data files can be provided by a single file or more. Different regions or structures within the files store different data from the database.

[0018] In some cases, multiple instances of a database share a dual-write cache file, resulting in different regions of the dual-write cache file being allocated to different database instances. In this scenario, the data written to the dual-write cache file also exhibits randomness. Summary of the Invention

[0019] However, the cost of using dual write caching is an increase in the amount of data written to the storage device. Typically, the non-volatile storage media of a storage device has a limited capacity for data writes, which is directly related to the lifespan of the storage device. The data in the dual write cache files is only used during database error recovery, therefore a large portion of the data within them remains unused.

[0020] Furthermore, writing data to the dual-write cache file also increases the latency of database transaction processing and affects database performance.

[0021] The storage device according to embodiments of this application provides a high-performance storage means to accelerate the process of writing data to a dual-write cache file, thereby improving the performance of the database on the storage device. Embodiments of this application also reduce the amount of data written to the non-volatile storage medium of the storage device, thereby extending the lifespan of the storage device.

[0022] According to a first aspect of this application, a first storage device according to the first aspect of this application is provided, including a first logical storage device and a second logical storage device that can be accessed by a host coupled to the storage device, wherein the first logical storage device has higher performance in processing write commands than the second logical storage device.

[0023] According to the first storage device of the first aspect of this application, a second storage device according to the first aspect of this application is provided, wherein, under normal operating conditions of the storage device, data written to the first logical storage device is stored in a volatile storage medium and is not written to a non-volatile storage medium; data written to the second logical storage device is stored in a non-volatile storage medium.

[0024] According to the second storage device of the first aspect of this application, a third storage device according to the first aspect of this application is provided, wherein in response to a power failure of the storage device or an instruction, data stored in a volatile storage medium that is to be written to the first logical storage device is written to a non-volatile storage medium.

[0025] According to one of the first to third storage devices of the first aspect of this application, a fourth storage device according to the first aspect of this application is provided, wherein the storage space size of the first logical storage device is smaller than that of the second logical storage device.

[0026] According to one of the first to fourth storage devices of the first aspect of this application, a fifth storage device according to the first aspect of this application is provided, wherein the storage space size of the first logical storage device is used to accommodate the double write cache file of the database, and the storage space size of the second logical storage device is used to accommodate the data file of the database.

[0027] According to one of the first to fifth storage devices according to the first aspect of this application, a sixth storage device according to the first aspect of this application is provided, wherein the logical storage device is a namespace.

[0028] According to one of the first to sixth storage devices of the first aspect of this application, a seventh storage device according to the first aspect of this application is provided, wherein the first logical storage device uses an address mapping mechanism different from the address mapping mechanism of the second logical storage device to map the address space of the first logical address device to the storage medium.

[0029] According to a second aspect of this application, a method for writing updated data from a database to a storage device is provided, comprising: writing the updated data to a dual-write cache; and writing the updated data in the dual-write cache to a dual-write cache file stored in a first logical storage device.

[0030] And updated data in the dual write cache is written to the data file stored on the second logical storage device.

[0031] According to the first method of writing updated database data to a storage device according to the second aspect of this application, a second method of writing updated database data to a storage device according to the second aspect of this application is provided, wherein, under normal operating conditions of the storage device, data written to the first logical storage device is stored in a volatile storage medium and is not written to a non-volatile storage medium; data written to the second logical storage device is stored in a non-volatile storage medium.

[0032] According to the second method of writing updated database data to a storage device according to the second aspect of this application, a third method of writing updated database data to a storage device according to the third aspect of this application is provided, wherein in response to a power failure or an instruction being given to the storage device, the storage device writes data stored in a volatile storage medium that was written to the first logical storage device to a non-volatile storage medium.

[0033] According to one of the first to third methods of writing updated data of a database to a storage device according to the second aspect of this application, a fourth method of writing updated data of a database to a storage device according to the second aspect of this application is provided, wherein the storage space size of the first logical storage device is smaller than that of the second logical storage device.

[0034] According to one of the first to fourth methods of writing updated data of a database to a storage device according to the second aspect of this application, a fifth method of writing updated data of a database to a storage device according to the second aspect of this application is provided, wherein the storage space size of the first logical storage device is used to accommodate the database's double write cache file, and the storage space size of the second logical storage device is used to accommodate the database's data file.

[0035] One of the methods for writing updated data of a database to a storage device according to the first to fifth methods of the second aspect of this application is provided, and a sixth method for writing updated data of a database to a storage device according to the second aspect of this application is provided, wherein the logical storage device is a namespace.

[0036] One of the methods for writing updated database data to a storage device according to the first to fifth aspects of the second aspect of this application provides a seventh method for writing updated database data to a storage device according to the second aspect of this application, wherein the first logical storage device is NVRAM and the second logical storage device is a block device.

[0037] One of the methods for writing updated database data to a storage device according to the first to seventh aspects of the second aspect of this application provides an eighth method for writing updated database data to a storage device according to the second aspect of this application, wherein the first logical storage device uses an address mapping mechanism different from the address mapping mechanism of the second logical storage device to map the address space of the first logical address device to the storage medium.

[0038] One of the methods for writing updated data of a database to a storage device according to the first to eighth aspects of the second aspect of this application provides a ninth method for writing updated data of a database to a storage device according to the second aspect of this application, which further includes: setting a dual-write cache file of the database to be stored in the storage space provided by the first logical storage device.

[0039] According to the ninth method of writing updated database data to a storage device according to the second aspect of this application, a tenth method of writing updated database data to a storage device according to the second aspect of this application is provided, which further includes: setting the database data file to be stored in the storage space provided by a second logical storage device.

[0040] One of the methods for writing updated database data to a storage device according to the first to tenth aspects of the second aspect of this application is provided, and an eleventh method for writing updated database data to a storage device according to the second aspect of this application is provided, wherein the performance of the first logical storage device in processing write commands is higher than that of the second logical storage device.

[0041] One of the methods for writing updated database data to a storage device according to the first to eleventh aspects of the second aspect of this application provides a method for writing updated database data to a storage device according to the twelfth aspect of the second aspect of this application, further comprising: in response to data written to the data file being corrupted, retrieving data from the dual write cache file to recover the corrupted data.

[0042] According to a third aspect of this application, a system for writing updated data of a database to a storage device is provided, comprising: means for writing updated data to a dual write cache; means for writing updated data in the dual write cache to a dual write cache file stored in a first logical storage device; and means for writing updated data in the dual write cache to a data file stored in a second logical storage device.

[0043] According to a third aspect of this application, a first storage device according to the third aspect of this application is provided, comprising a first logical storage device and a second logical storage device, wherein writing data to the first logical storage device has a smaller impact on the lifespan of the storage device than writing data to the second logical storage device.

[0044] According to the first storage device of the third aspect of this application, a second storage device according to the third aspect of this application is provided, wherein, under normal operating conditions of the storage device, data written to the first logical storage device is stored in a volatile storage medium and is not written to a non-volatile storage medium; data written to the second logical storage device is stored in a non-volatile storage medium.

[0045] According to the second storage device of the third aspect of this application, a third storage device according to the third aspect of this application is provided, wherein in response to a power failure of the storage device or an instruction, data stored in a volatile storage medium that is to be written to a first logical storage device is written to a non-volatile storage medium.

[0046] According to one of the first to third storage devices of the third aspect of this application, a fourth storage device according to the third aspect of this application is provided, wherein the first logical storage device is used to accommodate a double write cache file of a database, and the second logical storage device is used to accommodate a data file of a database.

[0047] According to one of the first to fourth storage devices of the third aspect of this application, a fifth storage device according to the third aspect of this application is provided, wherein the address for accessing the storage space of the first logical storage device is mapped to the address of the volatile storage medium, and the address for accessing the storage space of the second logical storage device is mapped to the address of the non-volatile storage medium.

[0048] According to one of the first to fifth storage devices of the third aspect of this application, a sixth storage device according to the third aspect of this application is provided, wherein in response to writing first data to an address of the storage space of the first logical storage device, a space address is allocated from a volatile storage medium to carry the first data, and in response to writing second data to an address of the storage space of the second logical storage device, a space address is allocated from a non-volatile storage medium to carry the second data.

[0049] According to one of the first to sixth storage devices of the third aspect of this application, a seventh storage device according to the third aspect of this application is provided, wherein the first logical storage device and the second logical storage device can be accessed respectively by a host coupled to the storage device in a manner that accesses block devices.

[0050] According to one of the first to seventh storage devices of the third aspect of this application, an eighth storage device according to the third aspect of this application is provided, wherein the first logical storage device and the second logical storage device are namespaces conforming to the NVMe protocol.

[0051] According to one of the first to eighth storage devices of the third aspect of this application, a ninth storage device according to the third aspect of this application is provided, wherein the read / write performance of the first logical storage device is higher than that of the second logical storage device.

[0052] According to a fourth aspect of this application, a method for writing updated data of a database to a storage device is provided, comprising: writing the updated data to a dual-write cache file stored on a first storage device; and writing the updated data to a data file stored on a second storage device.

[0053] According to the first method of writing updated database data to a storage device according to the fourth aspect of this application, a second method of writing updated database data to a storage device according to the fourth aspect of this application is provided, wherein the first storage device and the second storage device are provided by the same physical storage device and follow the NVMe protocol namespace.

[0054] The method for writing updated database data to a storage device according to the first or second method of the fourth aspect of this application provides a method for writing updated database data to a storage device according to the third method of the fourth aspect of this application, wherein, under normal operating conditions of the first storage device, the data written to the first storage device is stored in a volatile storage medium and is not written to a non-volatile storage medium; the data written to the second storage device is stored in a non-volatile storage medium.

[0055] According to the third method of writing updated data of a database to a storage device according to the fourth aspect of this application, a fourth method of writing updated data of a database to a storage device according to the fourth aspect of this application is provided, wherein in response to the first storage device losing power or being instructed, data stored in a volatile storage medium that is to be written to the first logical storage device is written to a non-volatile storage medium.

[0056] One of the methods for writing updated data of a database to a storage device according to the first to fourth aspects of the fourth aspect of this application provides a fifth method for writing updated data of a database to a storage device according to the fourth aspect of this application, further comprising: in response to data written to the data file being corrupted, retrieving data from the dual write cache file to recover the corrupted data.

[0057] One of the methods for writing updated database data to a storage device according to the first to fifth aspects of the fourth aspect of this application provides a sixth method for writing updated database data to a storage device according to the fourth aspect of this application, wherein writing data to a first storage device has a smaller impact on the lifespan of the physical storage device than writing data to a second storage device.

[0058] According to a fifth aspect of this application, a system for writing updated data of a database to a storage device according to the fifth aspect of this application is provided, comprising: means for writing updated data to a dual write cache file stored in a first storage device; and means for writing updated data to a data file stored in a second storage device.

[0059] According to a sixth aspect of this application, a first computer according to the sixth aspect of this application is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements one of the methods for writing updated data of a database to a storage device according to the second or fourth aspect of this application. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0061] Figure 1 A block diagram of a storage device;

[0062] Figure 2 A diagram illustrating the use of dual write caching in a database is shown.

[0063] Figure 3 This is a schematic diagram illustrating the optimization of database dual-write caching operations according to an embodiment of this application;

[0064] Figure 4A and Figure 4B A schematic diagram of a host and storage device according to an embodiment of this application is shown;

[0065] Figure 5 A flowchart illustrating database data update according to an embodiment of this application is shown;

[0066] Figure 6 A schematic diagram of a host and storage device according to yet another embodiment of this application is shown;

[0067] Figure 7 A schematic diagram of a host and storage device according to another embodiment of this application is shown; and

[0068] Figure 8 A schematic diagram of a host and storage device according to yet another embodiment of this application is shown. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0070] Figure 3 This is a schematic diagram illustrating the optimization of database dual-write caching operations according to an embodiment of this application.

[0071] According to embodiments of this application, the storage device coupled to the host running the database provides at least two logical storage devices, such as a dual-write cache storage device and a block device storage device. For example, the performance of the dual-write cache storage device, particularly its performance in processing write commands, is superior to that of the block device storage device. The block device storage device is, for example, a physical storage device or virtual storage device (e.g., a namespace (NS) following the NVMe protocol) that conforms to protocols such as NVMe, OCSSD, and SATA. The dual-write cache storage device is, for example, a memory space provided by a CMB (Controller Memory Buffer) following the NVMe protocol, a namespace (NS) following the NVMe protocol, or a PCIe device. It can also be storage space provided by, for example, high-performance non-volatile storage media (e.g., phase-change memory, MRAM, FeRAM, RRAM, DRAM, SRAM, etc.).

[0072] Although Figure 3 The document demonstrates the use of a single storage device to provide both a dual-write cache storage device and a block device storage device. Alternatively, two storage devices can be used to provide both the dual-write cache storage device and the block device storage device.

[0073] The host manages and operates block device storage devices on a block device basis. The host operates dual-write cache storage devices on a block device or memory basis.

[0074] Databases maintain two or more types of caches in host memory, such as page cache and write cache. The storage unit of page cache is called a cache page, and the cache unit of write cache is called a write cache page.

[0075] According to embodiments of this application, a dual-write cache file for carrying data of dual-write cache pages is stored in a dual-write cache storage device, while the database data file is stored in a block device storage device. For example, the dual-write cache storage device is mounted to a first partition of the operating system, and the block device storage device is mounted to a second partition of the operating system. The dual-write cache file is created in the first partition, and the database data file is created in the second partition. Thus, without modifying the database and / or host drivers, high database operation performance can be achieved by configuring only the database files (dual-write cache file and data file) and using the storage device according to embodiments of this application.

[0076] The database needs to write non-volatile data to one or more cached pages in the page cache. To write the cached page data to storage, the database first copies the cached page data to the dual-write cache. Figure 3In the middle, as instructed by (1), the data of the dual write cache is then written to the dual write cache file on the dual write cache storage device. Figure 3 In the middle, as instructed by (2), the data in the dual write cache is then non-volatilely written to the data file on the block device storage device. Figure 3 In the middle, as indicated by (3).

[0077] Because dual-write cache storage devices have higher performance or write performance, data in the dual-write cache can be written non-volatilely to the dual-write cache file on the dual-write cache storage device. Figure 3 In the process indicated by (2), the required latency is low, thereby reducing the impact of dual write caching on database operations.

[0078] Optionally, the data in the dual-write cache is written to the dual-write cache file on the dual-write cache storage device. Figure 3 In the middle, as instructed by (2), the data of the double write cache is written to the data file on the block device storage device. Figure 3 In the process, the operations indicated by (3) can be performed concurrently. The database identifies the data to be written to the dual-write cache file on the dual-write cache storage device. Figure 3 In the middle, only after the operation indicated by (2) is completed will the response write the data of the double write cache to the data file on the block device storage device ( Figure 3 In the process, the operation instructed by (3) is completed and subsequent processing is carried out (e.g., committing the transaction).

[0079] Optionally or further, the dual-write cache storage device has a small capacity, for example, the same or slightly larger than the dual-write cache in host memory. The dual-write cache file is used as a non-volatile copy of the dual-write cache in host memory, and the database overwrites the storage space of the dual-write cache file, so that the size of the dual-write cache storage device only needs to accommodate the dual-write cache file.

[0080] Because dual-write cache storage devices have a small capacity, for example, 1MB-10MB, the cost of the storage device is not significantly increased even if expensive non-volatile storage media (e.g., DRAM, phase-change memory, FeRAM, MRAM, RRAM) are used to provide dual-write cache storage devices.

[0081] In one example, a dual-write cache storage device is provided by DRAM to achieve higher performance compared to, for example, flash memory. Although the database considers the dual-write cache file to be non-volatile, according to one embodiment of this application, when the storage device is operating normally, data written to the dual-write cache file stored on the DRAM-provided dual-write cache storage device is stored only in the DRAM, without flushing the data in the DRAM providing the dual-write cache device to a non-volatile storage medium such as flash memory. From the database's perspective, data is written to the dual-write cache file and can be read from it, thus the dual-write cache file functions correctly. Because the data in the DRAM providing the dual-write cache storage device is not flushed to a non-volatile storage medium, the dual-write cache storage device achieves higher write performance and lower complexity, thereby reducing the cost of providing the dual-write cache storage device.

[0082] Furthermore, since data in the DRAM providing the dual-write cache storage device is not flushed to the non-volatile storage medium in most cases, the amount of data written to the non-volatile storage device is reduced, thereby improving the lifespan of the storage device. The amount of data written to the dual-write cache file accounts for approximately 40% of the total data written to the storage device by the database. Therefore, according to the implementation of this application, the amount of data written to the storage device by the database is reduced by 40%, correspondingly, the latency of database update operations is reduced by nearly 40%, and the lifespan of the storage device is increased by nearly 40%.

[0083] When power is lost, the data stored in the DRAM is lost. According to embodiments of this application, in response to a power outage, the storage device writes the data stored in the dual-write cache storage device in the DRAM to a non-volatile storage medium such as NAND flash memory. After the storage device is subsequently powered on, the data stored in the dual-write cache storage device is recovered from the non-volatile storage medium and stored in the DRAM to respond to database access requests. Optionally, the storage device also includes a backup power supply for providing power to the storage device after a power outage, so as to write the data stored in the dual-write cache storage device in the DRAM to the non-volatile storage medium after the power outage. In one embodiment, the data stored in the dual-write cache storage device in the DRAM is written to the non-volatile storage medium only in response to a power outage; when the storage device is powered normally, the data stored in the dual-write cache storage device in the DRAM is not written to a volatile storage medium.

[0084] In another embodiment, the storage device includes NVRAM (Non-Volatile RAM) and uses NVRAM to provide a dual-write cache storage device. The dual-write cache file stored in the dual-write cache storage device is stored in NVRAM.

[0085] Therefore, even if an unexpected event such as a power outage occurs during the process of writing data to a data file on a block device storage device, resulting in the data in the data file being corrupted (for example, the data in the same page cache is inconsistent), the database can recover the correct data from the double write cache file.

[0086] Figure 4A and Figure 4B A schematic diagram of a host and storage device according to an embodiment of this application is shown.

[0087] See Figure 4A For example, the host's file system presents one or more ibdata1 files and *.idb files to a database running on the host. The host driver presents block devices and memory devices (optionally, the memory device is presented as a RAMDisk (a block device using memory)) to the database or other applications. The *.idb files are stored in the storage space provided by the block device, while the ibdata1 files are stored in the storage space provided by the memory device. Therefore, the ibdata1 files have better I / O performance compared to the *.idb files.

[0088] The storage device is coupled to the host via an interface. The storage device presents itself to the host as, for example, an NVMe storage device and an NVRAM device. As an example, the NVRAM storage device is provided according to Chinese Patent Application No. 2018100940602, entitled "Non-volatile Random Access Memory and Method for Providing the Same," the entire contents of which are incorporated herein by reference. Again, as an example, the NVMe storage device responds to NVMe commands obtained from the interface, while the NVRAM device responds to commands obtained from the interface to access a specified memory space of the device.

[0089] Figure 4B This is a schematic diagram of a host and storage device according to another embodiment of this application.

[0090] The host's file system presents one or more ibdata1 files and *.idb files to the database running on the host. Storage devices comprise two or more namespaces. These namespaces are, for example, namespaces defined by the NVMe specification (logical storage devices). One or more namespaces 0 through 31 are presented as block device 0 by the host driver and provide storage space for the *.idb files (e.g., presented as a file system partition).

[0091] according to Figure 4B The storage device also provides a special namespace (denoted as NSV), which has higher performance than namespaces 0-31. The special namespace is presented by the host driver as block device V and provides storage space for storage ibdata1 (e.g., presented as a file system partition).

[0092] according to Figure 4B In one embodiment, both namespaces (NS(0-31)) and namespace (NSV) are block devices, so the file system uses block device 0 and block device V in the same or basic way.

[0093] The host is coupled to the storage device through an interface. The host accesses the storage device's namespace (NS(0-31)) and namespace (NSV) using NVMe commands that follow the NVMe protocol, for example.

[0094] In one example, the storage device uses NVRAM devices to handle I / O commands accessing namespaces (NSVs). As another example, the storage device uses block storage devices to handle I / O commands accessing namespaces (NSVs). NVRAM or block storage devices are implemented using storage space provided by, for example, high-performance non-volatile storage media (e.g., phase-change memory, MRAM, FeRAM, RRAM, DRAM, SRAM, etc.). Optionally, the storage device provides a dedicated address mapping mechanism for namespaces (NSVs). For example, the storage space of the namespace (NSV) is directly mapped to the high-performance non-volatile storage media, or the storage space of the namespace (NSV) is directly mapped to the high-performance non-volatile storage media through a dedicated FTL table.

[0095] In another example, the storage device identifies the data to be written by I / O commands accessing namespace NSV and namespaces (NS 0-31) as different streams. The data to be written by I / O commands accessing namespace NSV is identified as a sequential stream, while the data to be written by I / O commands accessing namespaces (NS 0-31) is identified as a random stream. The sequential stream is written sequentially to the storage device, resulting in higher write performance.

[0096] Figure 5 A flowchart illustrating database data update according to an embodiment of this application is shown.

[0097] According to embodiments of this application, the host running the database is coupled to at least two (logical) storage devices—a high-performance storage device and a general-purpose storage device, wherein the high-performance storage device has at least higher performance in processing write commands than the general-purpose storage device. For example, a single physical storage device can provide both logical high-performance and general-purpose storage.

[0098] Configure the database's write cache files (e.g., the ibdata1 file managed by the MySQL database) to be stored in storage space provided by a high-performance storage device (510). Meanwhile, the database's data files (e.g., *.idb files managed by the MySQL database) are configured to be stored in storage space provided by a regular storage device.

[0099] When the database is updated (e.g., during data table insertion or update operations), data to be stored is generated (520). The generated data to be stored is written to a cache page of the database's page cache (530). The data in the page cache is then written to a dual-write cache page of the dual-write cache (540). The data in the dual-write cache page is written to a dual-write cache file stored in a high-performance storage device (550), and the data in the dual-write cache page is also written to a data file stored in a regular storage device (560). Steps 550 and 560 can be processed simultaneously, or step 550 can be executed first and then step 560.

[0100] Optionally, in step 550, the storage device stores the data of the double-write cache file in, for example, DRAM, instead of writing it to non-volatile memory. This results in lower latency and / or greater bandwidth for the data write operation in step 550, and also reduces the amount of data written to non-volatile memory, which helps extend the lifespan of the storage device.

[0101] Figure 6 A schematic diagram of a host and storage device according to yet another embodiment of this application is shown.

[0102] The database or application running on the host can access at least two types of storage space with different performance characteristics. The first type of storage space consists of memory, RAM disks, or non-volatile block devices ( Figure 6 The second type of storage space is accessed in the form of block devices (NV), which have relatively high write performance and / or can handle a relatively large amount of write data. Figure 6 It is accessed in the form of block device 0, which has relatively low write performance.

[0103] The storage device coupled to the host provides the host with a high-performance storage device 610 and a general-purpose storage device 620. The high-performance storage device 610 provides the first type of storage space, and the general-purpose storage device 620 provides the second type of storage space.

[0104] The control unit 600 of the storage device responds to I / O commands for accessing the ordinary storage device 620 using NVM (non-volatile memory) 650.

[0105] The cache 630 of the storage device is, for example, DRAM (Dynamic Random Access Memory). The control unit 600 uses the cache 630 to respond to I / O commands accessing the high-performance storage device 610. For example, data written to the high-performance storage device 610 is stored in the cache 630 by the control unit 610. Because the cache 630 is DRAM, it has better write performance than the NVM 650 and can hold more write data.

[0106] In one example, when the storage device is operating normally, the control unit 650 does not move data from the cache 630 to the NVM 650. The control unit 600 maintains a mapping between the storage space of the high-performance storage device 610 and the storage space of the cache 630. The storage space of the high-performance storage device 610 and the storage space of the cache 630 are, for example, mapped one-to-one, so that an overwrite to the storage space of the high-performance storage device 610 will cause an overwrite to the storage space of the cache 630.

[0107] The storage device also includes a backup power supply 640. In response to a power outage, the backup power supply 640 provides power to the cache 630, control unit 600, and NVM 650. The control unit 600 moves the data recorded in the cache 630 for the storage space of the high-performance storage device 610 to the NVM 650. When the storage device is powered on again, the data recorded in the cache 630 for the storage space of the high-performance storage device 610 (moved to the cache 630) is restored from the NVM 650. Thus, when the database in the host accesses the first type of storage space provided by the high-performance storage device 610, it can obtain the data written to the first type of storage space before the power outage.

[0108] Since the dual-write cache file of the database is usually small, the capacity of cache 630 does not need to be large (e.g., the same as or slightly larger than the dual-write cache file), and the backup power supply 640 does not need to provide much power to move the data of cache 630 to NVM 650, thus not significantly increasing the cost of backup power supply 640.

[0109] Even if an overwrite is performed on the storage space of cache 630, the data in cache 630 will not be written to NVM 650, thus the overwritten data will be lost and cannot be accessed again. This implementation does not affect the database's use of the dual-write cache file. From the database's perspective, the dual-write cache file is non-volatile; after the storage device loses power and is powered on again, the data written to the dual-write cache file can be read again.

[0110] In another example, while the storage device is operating, the control unit 600 continuously moves data from the cache 630 to the NVM 650, or moves data from the cache 630 to the NVM 650 in response to an update to the cache 630. Thus, in the event of a power failure, only data from the cache 630 that has not yet been written to the NVM 650 needs to be moved to the NVM 650, thereby reducing the power consumption of the backup power supply 640 and lowering its cost.

[0111] As another example, configure a MySQL database running on the host machine, setting directories such as datadir, innodb_log_group_home_dir, innodb_undo_directory, and log_bin to be stored in the second type of storage space provided by a regular storage device 620, while setting innodb_data_home_dir / innodb_data_file_path to be stored in the first type of storage space provided by a high-performance storage device.

[0112] As another example, configure the MySQL database by setting the ibdata1 file to be stored in the second type of storage space, and removing the undolog from the ibdata1 file and storing it in the first type of storage space to reduce the size of the ibdata1 file. Set the other files of the MySQL database to be stored in the first type of storage space.

[0113] Optionally, for other databases, the configuration database will store files or logs used to provide non-volatility for caching or to provide data recovery in the second type of storage space, while storing data files in the first type of storage space.

[0114] Figure 7 A schematic diagram of a host and storage device according to another embodiment of this application is shown.

[0115] The database or application running on the host can access at least two types of storage space with different performance characteristics. The first type of storage space consists of memory, RAM disks, or non-volatile block devices ( Figure 7 The first type of storage space is accessed in the form of a block device (NV), and has relatively high write performance. The second type of storage space is accessed in the form of a block device (NV). Figure 7 It is accessed in the form of block device 0, which has relatively low write performance.

[0116] The host-coupled storage device provides the host with the NVRAM 710 and the storage device namespace (NS 0-31) 720. The NVRAM 710 provides the first type of storage space, and the storage device namespace (NS 0-31) 720 provides the second type of storage device.

[0117] The storage device control unit 700 responds to I / O commands for accessing the storage device's namespace (NS 0-31) 720 using NVM (non-volatile memory) 750. The storage device control unit 700 responds to I / O commands for accessing NVRAM 710 using cache 730, NVM 755, and backup power supply 740.

[0118] The NVRAM 710 provides a contiguous address space. The NVRAM address space comprises multiple regions of equal size, each called a block. For example, the size of the NVRAM address space corresponding to each block could be, for example, 512 bytes, 1KB, or 4KB. The block corresponding to an address in the NVRAM address space can be calculated. For example, dividing the address of the NVRAM address space by the size of the NVRAM address space corresponding to the block yields the block index or block number (ID).

[0119] Control unit 700 maintains a small block translation table. The small block translation table includes multiple entries, each corresponding to one of the small blocks and indexed by the small block ID. The value of each entry records the address of the underlying large block frame or memory slice that provides the data for the small block. The underlying large block frame and memory slice will be described in detail later.

[0120] The small block translation table is stored in, for example, DRAM or SRAM. The control unit 700 calculates the corresponding small block ID based on the address of the accessed NVRAM address space, and uses the small block ID to query the small block translation table to obtain the address of the underlying large block frame or memory slice that provides data for the small block.

[0121] The NVM 755's storage space is organized into blocks. A block is, for example, a single physical block of the NVM chip, multiple physical blocks with the same physical block number in different planes of the NVM chip's LUN, or physical blocks from multiple LUNs.

[0122] Large blocks of memory are organized into frames. A large block consists of multiple frames. The size of a frame is the same as the size of a small block, so that the data stored in the NVRAM address space corresponding to a small block can be recorded in a single frame of a large block.

[0123] Large blocks include at least two types: base large blocks and log large blocks. The frames of base large blocks are called base large block frames. The values ​​of the entries in the small block translation table record the addresses of the base large block frames, indicating the base large block frames. Base large block frames can be accessed based on their addresses.

[0124] The storage space of cache 730 is organized into memory slices. A memory slice is, for example, a segment of storage space in DRAM. The size of a memory slice is the same as the size of a block, so that the data stored in the NVRAM address space corresponding to a block can be recorded in a memory slice. The value of the entry in the block translation table records the address of the memory slice, indicating the memory slice. The memory slice can be accessed based on its address.

[0125] Some entries in the small block translation table indicate the address of the underlying large block frame, while others indicate the address of the memory slice.

[0126] Log blocks are used to record modifications to the NVRAM address space. For example, writing data D to address NADDR in the NVRAM address space generates a log entry, recording the changes.<NADDR,D> The NVRAM management module will record...<NADDR,D> Log entries are written to log block frames. Control unit 700 organizes log block frames into, for example, linked lists to facilitate access to multiple log block frames.

[0127] Continuing with the example, the NVRAM address space corresponding to the entry in the same small block translation table (small block 1) is updated. Data D is written to address NADDR in this NVRAM address space, and the generated record is...<NADDR,D> Log entries are written to log block frames within the log block. Since the log block is the storage space of the NVM chip, logs written to it are non-volatile; even if the storage device loses power, the records in the log block can still be read when the storage device is powered on again.<NADDR,D> Log entries.

[0128] In response to the application writing data D to address NADDR in the NVRAM address space, the NVRAM management module also records data D in the memory chip corresponding to the same address NADDR. Since the memory chip is storage space provided by DRAM, it can be addressed byte-by-byte or accessed randomly, thus efficiently completing the operation of recording data D in the memory chip.

[0129] According to embodiments of this application, memory chips provide efficient random access capabilities to the NVRAM address space, while log block frames record log entries.<NADDR,D> This provides non-volatile capabilities for updating the NVRAM address space. The NVRAM address space, possessing both random access and non-volatile capabilities, provides NVRAM services to the host.

[0130] Figure 8 A schematic diagram of a host and storage device according to yet another embodiment of this application is shown.

[0131] The host running a database or application can access at least two types of storage space with different performance characteristics. The first type of storage space is block devices ( Figure 8 The second type of storage space is accessed in the form of block devices (V), which have relatively high read / write performance and / or their lifespan is insensitive to the amount of data written. Figure 8 When accessed in the form of block device 0, it has relatively low read / write performance compared to block device V.

[0132] Storage devices coupled to the host provide at least two namespaces to the host. Figure 8The namespaces are labeled NSV and NV0, respectively. The first type of storage space is provided by namespace NSV, and the second type of storage device is provided by namespace NS0. Namespaces NSV and NS0 are, for example, namespaces conforming to the NVMe protocol.

[0133] The storage device uses the FTL (Flash Translation Layer) to map requests to namespace NSV to addresses in cache 830, and requests to namespace NS0 to addresses in NVM 850. Cache 830 is, for example, DRAM (Dynamic Random Access Memory). Cache 830 is used to respond to I / O commands accessing namespace NSV. Because cache 830 is DRAM, it offers better read / write performance than NVM 850, and its lifetime is insensitive to the amount of data written. NVM 850 is used to respond to I / O commands accessing namespace NS0.

[0134] The FTL table records the mapping from the logical addresses of each namespace to the physical addresses of cache 830 / NVM 850. Therefore, based on the logical address of a read command accessing namespace NSV, the FTL table is consulted to obtain the physical address of cache 830; similarly, based on the logical address of a read command accessing namespace NS0, the FTL table is consulted to obtain the physical address of NVM 850. For write commands accessing namespace NSV, a cache unit is allocated from cache 830 to write the data to be written, and the logical address of the write command and the allocated cache unit's physical address are recorded in the FTL table. For write commands accessing namespace NS0, a cache unit is allocated from NVM 850 to write the data to be written, and the logical address of the write command and the allocated NVM 850's physical address are recorded in the FTL table.

[0135] Optionally, a separate FTL table can be provided for each namespace.

[0136] The logical address space of namespace NSV is mapped one-to-one with the storage space of cache 830, so a write to the logical address space of namespace NSV will cause a write to the storage space of cache 830 to be overwritten.

[0137] In one example, when the storage device is operating normally, data written to cache 830 is not migrated to NVM 850. NVM 850 is sensitive to the amount of writes; writing data will consume the lifetime of NVM 850, while cache 830 is not sensitive to the amount of write data, and writing data will not significantly consume the lifetime of cache 830. However, according to the embodiments of this application, data written to the namespace NSV (i.e., cache 830) is not written to NVM 850, thereby significantly reducing the consumption of the storage device's lifetime when serving database applications. Taking MySQL database as an example, the amount of data written to the dual write cache file can reach 40% of the total amount of data written to the file system by the MySQL database. When the storage device according to the embodiments of this application is used to serve the MySQL database, 40% of the total amount of write data will not cause the storage device's lifetime to be consumed, which is equivalent to significantly extending the lifespan of the storage device.

[0138] To reiterate, the logical address space of namespace NSV is much smaller than that of namespace NS0, for example, in the tens of megabytes, or even one ten-thousandth or less of the logical address space of NS0. Therefore, using cache 830 to provide namespace NSV will not significantly increase the cost of the storage device.

[0139] The storage device also includes a backup power supply 8640. In response to a power outage, the backup power supply 8640 provides power to the cache 830, the control unit, and the NVM 850. The control unit moves the data for the namespace NSV recorded in the cache 830 to the NVM 850. When the storage device is powered on again, the data for the namespace NSV recorded in the cache 830 (moved to the cache 830) is restored from the NVM 850. Thus, when the database in the host accesses the block device V, it can obtain the data written to the block device V before the power outage.

[0140] Since the capacity of cache 830 does not need to be large (e.g., tens of MB, or one ten-thousandth or less of the logical address of NS 0), the power required by backup power supply 840 to move data from cache 830 to NVM850 does not need to be much, thus not significantly increasing the cost of backup power supply 840.

[0141] The bandwidth of a storage device for processing write commands is typically limited by the bandwidth of the NVM. According to embodiments of this application, a namespace NSV is provided using a cache 830. Data written to the namespace NSV is not written to the NVM, thus the write bandwidth of the namespace NSV is not limited by the NVM bandwidth. Therefore, under high pressure (e.g., 32 threads, 64 threads, or more threads accessing the namespace NSV simultaneously), the storage device according to embodiments of this application demonstrates significantly better processing capabilities for database applications than traditional storage devices.

[0142] According to another embodiment of this application, the host is coupled to at least two storage devices. A database or application running on the host accesses both storage devices to gain the advantage of at least parallel processing of I / O commands. The database stores dual-write cache files on the first storage device, while storing the data files on the second storage device. Furthermore, the first storage device has higher read / write performance than the second storage device and / or its lifetime is insensitive to the amount of data written.

[0143] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application. Clearly, those skilled in the art can make various alterations and variations to this application without departing from its spirit and scope. Thus, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A storage device comprising a first logical storage device and a second logical storage device, wherein writing data to the first logical storage device has a smaller impact on the lifespan of the storage device than writing data to the second logical storage device; in, When the storage device is in normal working condition, the data written to the first logical storage device is stored in the volatile storage medium and is not written to the non-volatile storage medium. Data written to the second logical storage device is stored in a non-volatile storage medium; The address for accessing the storage space of the first logical storage device is mapped to the address of the volatile storage medium, and the address for accessing the storage space of the second logical storage device is mapped to the address of the non-volatile storage medium. The database maintains two or more types of caches within the host. The storage unit for page cache is a cache page, and the storage unit for dual-write cache is a dual-write cache page. To write the data of the cached page to the storage device, the database first copies the data of the cached page to the dual-write cache, then writes the data of the dual-write cache to the dual-write cache file on the dual-write cache storage device, and then writes the data of the dual-write cache to the data file on the block device storage device in a non-volatile manner. The dual-write cache storage device is the first logical storage device, and the block device storage device is the second logical storage device. The dual-write cache file is used as a non-volatile copy of the dual-write cache in the host memory. The database overwrites the storage space of the dual-write cache file. The dual-write cache storage device has a small capacity. Among them, volatile storage media are not sensitive to the amount of data written, while non-volatile storage media are sensitive to the amount of data written.

2. The storage device according to claim 1, wherein In response to a power failure of the storage device or an instruction, the data stored in the volatile storage medium that was written to the first logical storage device is written to the non-volatile storage medium.

3. The storage device according to any one of claims 1-2, wherein In response to writing first data to an address in the storage space of a first logical storage device, a space address is allocated from a volatile storage medium to carry the first data; and in response to writing second data to an address in the storage space of a second logical storage device, a space address is allocated from a non-volatile storage medium to carry the second data.

4. The storage device according to any one of claims 1-2, wherein The first logical storage device and the second logical storage device can be accessed by a host coupled to the storage device in a manner similar to accessing a block device.

5. The storage device according to any one of claims 1-2, wherein The first logical storage device and the second logical storage device are namespaces that conform to the NVMe protocol.

6. The storage device according to any one of claims 1-2, wherein The read / write performance of the first logical storage device is higher than that of the second logical storage device.

7. A method for writing updated data from a database to a storage device, comprising: Write the updated data to the dual-write cache file stored on the first storage device; Write the updated data to the data file stored on the second storage device; Specifically, under normal operating conditions of the first storage device, data written to the first storage device is stored in a volatile storage medium and is not written to a non-volatile storage medium; data written to the second storage device is stored in a non-volatile storage medium. Among them, the dual-write cache file is used as a non-volatile copy of the dual-write cache in the host memory, and the database writes overwrites the storage space of the dual-write cache file. Among them, volatile storage media are not sensitive to the amount of data written, while non-volatile storage media are sensitive to the amount of data written. The database maintains two or more types of caches within the host. The storage unit for page cache is a cache page, and the storage unit for dual-write cache is a dual-write cache page. To write the data of the cached page to the storage device, the database first copies the data of the cached page to the dual-write cache, then writes the data of the dual-write cache to the dual-write cache file on the dual-write cache storage device, and then writes the data of the dual-write cache non-volatilely to the data file on the block device storage device. The dual-write cache storage device is the first storage device, and the block device storage device is the second storage device. The dual-write cache file is used as a non-volatile copy of the dual-write cache pages in the host memory. The database overwrites the storage space of the dual-write cache file, and the dual-write cache storage device has a small capacity.

8. The method according to claim 7, wherein In response to a power failure of the first storage device or an instruction, the data stored in the volatile storage medium that was written to the first logical storage device is written to the non-volatile storage medium.

9. The method according to any one of claims 7-8, further comprising: In response to the corruption of data written to the data file, data is retrieved from the double-write cache file to recover the corrupted data.

10. A system for writing updated data from a database to a storage device, comprising: A means for writing updated data to a double-write cache file stored in a first storage device; as well as A means for writing updated data to a data file stored on a second storage device; Specifically, under normal operating conditions of the first storage device, data written to the first storage device is stored in a volatile storage medium and is not written to a non-volatile storage medium; data written to the second storage device is stored in a non-volatile storage medium. Among them, the dual-write cache file is used as a non-volatile copy of the dual-write cache in the host memory, and the database writes overwrites the storage space of the dual-write cache file. Among them, volatile storage media are not sensitive to the amount of data written, while non-volatile storage media are sensitive to the amount of data written. The database maintains two or more types of caches within the host. The storage unit for page cache is a cache page, and the storage unit for dual-write cache is a dual-write cache page. To write the data of the cached page to the storage device, the database first copies the data of the cached page to the dual-write cache, then writes the data of the dual-write cache to the dual-write cache file on the dual-write cache storage device, and then non-volatilely writes the data of the dual-write cache to the data file on the block device storage device; wherein the dual-write cache storage device is the first storage device and the block device storage device is the second storage device; The dual-write cache file is used as a non-volatile copy of the dual-write cache pages in the host memory. The database overwrites the storage space of the dual-write cache file, and the dual-write cache storage device has a small capacity.

Images