STORAGE SYSTEM AND STORAGE CONTROL METHOD

Abstract

To provide a storage system provided with good performance and high reliability.SOLUTION: A storage system comprises a nonvolatility storage device, a storage controller for controlling reading and writing from / in the storage device, and a memory, wherein the storage controller produces a log and stores it in a log memory when performing the reading and writing, writes the log stored in the memory in the storage device, and recovers a capacity relating to a storage area of the memory storing the log written on the storage device. When recovering a free area of the memory, a base-image saving method for writing on the storage device in a storage area unit having a plurality of logs to recover the free area and a garbage collection method for writing on the storage device in a log unit to recover the free area are carried out.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a storage system and a storage control method. [Background technology]

[0002] Conventionally, a redundant configuration has been adopted in storage systems to improve availability and reliability. For example, Patent Document 1 proposes the following storage system. In a storage system having multiple storage nodes, each storage node has one or more storage devices that provide storage areas, and one or more storage control units that read and write requested data from and to the corresponding storage devices in response to requests from a host device. Each storage control unit holds predetermined configuration information required to read and write requested data from and to the corresponding storage device in response to requests from the host device, multiple control software programs are managed as redundancy groups, the configuration information held by each control software program belonging to the same redundancy group is updated synchronously, and the multiple control software programs that make up the redundancy group are allocated to different storage nodes so as to distribute the load on each storage node. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-101703 Summary of the Invention [Problem to be solved by the invention]

[0004] According to Patent Document 1, a storage system that can continue reading and writing even in the event of a node failure can be constructed using a technology for constructing a storage system using software (Software-Defined Storage: SDS). To improve the performance and reliability of such a storage system, it is necessary to efficiently make various data nonvolatile. This invention proposes a method for efficiently storing control information, cache data, etc. in a storage system in a storage device. [Means for solving the problem]

[0005] In order to achieve the above object, one representative storage system of the present invention is a storage system comprising a non-volatile storage device, a storage controller that processes data to be read and written to the storage device using a storage function, and volatile memory, wherein the storage controller, upon receiving a data write request, stores the data related to the received write request in the memory, stores a log related to the data stored in the memory in the storage device, and once the log has been stored in the storage device, sends a completion response to the source of the data write request, processes the data stored in the memory using the storage function and destages it to the storage device, and if the data in the memory related to the write request is lost due to a power outage, recovers the data in the memory using the log stored in the storage medium. Furthermore, one representative storage control method of the present invention is a storage control method in a storage system comprising a non-volatile storage device, a storage controller that processes data to be read and written to the storage device using a storage function, and a volatile memory, wherein when the storage controller receives a data write request, it stores the data related to the received write request in the memory, stores a log related to the data stored in the memory in the storage device, and once the log has been stored in the storage device, it sends a completion response to the source of the data write request, processes the data stored in the memory using the storage function and destages it to the storage device, and if the data in the memory related to the write request is lost due to a power outage, it recovers the data in the memory using the log stored in the storage medium. [Effects of the Invention]

[0006] According to the present invention, it is possible to realize a storage system that combines high performance and high reliability. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating the configuration of a storage system according to a first embodiment of the present invention. [Figure 2] FIG. 1 illustrates an example of the physical configuration of a storage node. [Figure 3] FIG. 1 is a diagram showing an example of a logical configuration of a storage node. [Figure 4] FIG. 1 is a diagram illustrating an overview of the disclosed storage system and storage control method. [Figure 5] FIG. 1 is a diagram showing an example of a memory configuration diagram. [Figure 6] FIG. 1 is a diagram illustrating an example of a configuration diagram of a storage device. [Figure 7] FIG. 1 is a diagram illustrating an example of a software module structure of a storage control unit. [Figure 8] A diagram showing an example of a cache directory [Figure 9] Diagram showing the structure of a log header [Figure 10] Read processing flowchart [Figure 11] Write processing flowchart [Figure 12] Asynchronous destage processing flowchart [Figure 13] Flowchart of control information update process [Figure 14] Cache data update process flowchart [Figure 15] Log creation process flowchart [Figure 16] Flowchart of control information determination process [Figure 17] Log backup process flowchart [Figure 18] Flowchart of base image backup process [Figure 19] Cache Data Log Garbage Collection Process Flowchart [Figure 20] Log recovery process flowchart [Figure 21] Explanatory diagram of Example 2 [Figure 22] Flowchart of log saving process according to the second embodiment [Figure 23] Flowchart of base image backup process in the second embodiment [Figure 24] Flowchart of log recovery processing according to the second embodiment [Figure 25] Explanatory diagram of Example 3 DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings. The preferred embodiments relate to a storage system including a plurality of storage nodes, each of which has one or more SDSs implemented therein. In the disclosed embodiment, a storage node stores control information and cache data in memory. The storage node is equipped with a nonvolatile device. When updating control information or data in response to a write request from a host, the updated data is stored in this nonvolatile device in log format. This makes it possible to make the updated data nonvolatile. A response is then made to the host. Asynchronously, the data in memory is destaged to a storage device. Destaging is a process of reflecting data written to the storage system and writing it to a storage device. Destaging provides various storage functions such as thin provisioning, snapshots, and data redundancy, and includes processes such as creating logical-to-physical translation addresses to enable data search and random access. In contrast, nonvolatile device storage in log format is intended to restore data in memory if it is lost, so the storage process is lightweight and fast. Therefore, when using volatile memory, response performance can be improved by quickly storing data in a nonvolatile storage device in log format and responding to the host device with a completion response. When storing in log format, control information and data are stored in an append format. Since storage is appended, free space must be reclaimed. Two methods are used to reclaim free space: the base image backup method and the garbage collection method. The base image backup method writes the entire target area of ​​a certain control information or cache data to a non-volatile device, and discards all update logs during that time (reclaiming them as free space). The garbage collection method identifies unnecessary logs that are not the latest among the update logs and reclaims log space by transferring all logs other than the unnecessary logs to a separate area. In the event of a power outage, this base image backup and log can be used to restore control information and cache data to memory, preventing their loss. By using both methods to reclaim free space, the management information required for free space management can be reduced and the overhead for free space recovery can be reduced, improving storage performance. [Example]

[0009] (1) Example 1 (1-1) Configuration of the storage system of the first embodiment FIG. 1 shows the storage system of the first embodiment as a whole. The storage system 100, for example, comprises a plurality of host devices 101 (Host), a plurality of storage nodes 103 (Storage Node), and a management node 104. The host devices 101, storage nodes 103, and management node 104 are connected to one another via a network 102 configured from a Fibre Channel, Ethernet (registered trademark), a LAN (Local Area Network), or the like.

[0010] The host device 101 is a general-purpose computer device that transmits read requests or write requests (hereinafter, these are collectively referred to as I / O (Input / Output) requests) to the storage node 103 in response to requests from user operations, implemented application programs, etc. The host device 101 may also be a virtual computer device such as a virtual machine.

[0011] The storage node 103 is a computer device that provides a storage area for reading and writing data to the host device 101. The storage node 103 is, for example, a general-purpose server device. The management node 104 is a computer device used by a system administrator to manage the entire storage system 100. The management node 104 manages multiple storage nodes 103 as a group called a cluster. Although Fig. 1 shows an example in which only one cluster is provided, multiple clusters may be provided within the storage system 100.

[0012] FIG. 2 is a diagram illustrating an example of the physical configuration of the storage node 103. As shown in FIG. The storage node 103 includes a CPU (Central Processing Unit) 1031, a memory 1032, a plurality of storage devices 1033 (Drives), and a communication device 1034 (NIC: Network Interface Card).

[0013] The CPU 1031 is a processor that controls the operation of the entire storage node. The memory 1032 is composed of semiconductor memory such as SRAM (Static RAM (Random Access Memory)) and DRAM (Dynamic RAM). The memory 1032 is used to temporarily store various programs and necessary data. The CPU 1031 executes the programs stored in the volatile memory 1032, thereby executing various processes of the entire storage node 103, as described below.

[0014] The storage device 1033 is configured from one or more types of large-capacity nonvolatile storage devices, such as an SSD (Solid State Drive), a SAS (Serial Attached SCSI (Small Computer System Interface)) hard disk drive, a SATA (Serial ATA (Advanced Technology Attachment)) hard disk drive, etc. The storage device 1033 provides a physical storage area for reading or writing data in response to an I / O request from the host device 101.

[0015] The communication device 1034 is an interface that enables the storage node 103 to communicate with the host device 101, other storage nodes 103, or the management node 104 via the network 102. The communication device 1034 is configured, for example, with a NIC, an FC card, etc. The communication device 1034 controls protocols during communication with the host device 101, other storage nodes 103, or the management node 104.

[0016] FIG. 3 is a diagram showing an example of the logical configuration of the storage node 103. As shown in FIG. The storage node 103 includes a front-end driver 1081, a back-end driver 1087, one or more storage controllers 1083, and a data protection controller 1086.

[0017] The front-end driver 1081 is software that controls the communication device 1034 and has the function of providing the CPU 1031 with an abstracted interface for the storage control unit 1083 when communicating with the host device 101, other storage nodes 103, or management node 104.

[0018] The back-end driver 1087 is software that controls each storage device 1033 in its own storage node 103 and has the function of providing the CPU 1031 with an abstracted interface for communication with each storage device 1033 .

[0019] The storage control unit 1083 is software that functions as a controller for the SDS. The storage control unit 1083 accepts I / O requests from the host device 101 and issues I / O commands corresponding to the I / O requests to the data protection control unit 1086. The storage control unit 1083 also has a logical volume configuration function. The logical volume configuration function associates the logical chunks configured by the data protection control unit with the logical volumes provided to the host. For example, a straight mapping method (a one-to-one correspondence between logical chunks and logical volumes, with the logical chunk addresses and logical volume addresses being the same) may be used, or a virtual volume function (Thin Provisioning) method (a method in which logical volumes and logical chunks are divided into small-sized areas (pages) and the addresses of logical volumes and logical chunks are associated with each other on a page-by-page basis) may be used.

[0020] In the first embodiment, each storage control unit 1083 implemented in a storage node 103 is managed as a pair constituting a redundant configuration together with another storage control unit 1083 arranged in another storage node 103. Hereinafter, this pair will be referred to as a storage control unit group 1085. 3 shows a case where one storage control unit group 1085 is configured by two storage control units 1083. In the following description, the storage control unit group 1085 is also configured by two storage control units 1083, but one redundant configuration may also be configured by three or more storage control units 1083.

[0021] In the storage control unit group 1085, one storage control unit 1083 is set to a state (active system state, hereinafter referred to as active mode) in which it can accept I / O requests from the host device 101. In the storage control unit group 1085, the other storage control unit 1083 is set to a state (standby system state, hereinafter referred to as standby mode) in which it cannot accept I / O requests from the host device 101. In the storage control unit group 1085, if a failure occurs in a storage control unit 1083 set to active mode (hereinafter referred to as an active storage control unit) or in a storage node 103 in which the active storage control unit is located, the state of the storage control unit 1083 that had been set to standby mode (hereinafter referred to as a standby storage control unit) is switched to active mode. As a result, if the active storage control unit becomes unable to operate, the standby storage control unit can take over the I / O processing that the active storage control unit had been executing.

[0022] The data protection control unit 1086 is software that allocates physical storage areas provided by storage devices 1033 in its own storage node 103 or in other storage nodes 103 to each storage control unit group 1085, and has the function of reading or writing specified data to the corresponding storage device 1033 in accordance with the above-mentioned I / O command given from the storage control unit 1083. In this case, when the data protection control unit 1086 allocates a physical storage area provided by a storage device 1033 in another storage node 103 to the storage control unit group 1085, it cooperates with the data protection control unit 1086 implemented in that other storage node 103 to exchange data with that data protection control unit 1086 via the network 102, thereby reading or writing the data to that storage area in accordance with the I / O command given by the active storage control unit of that storage control unit group 1085.

[0023] FIG. 4 is a diagram illustrating an overview of the disclosed storage system and storage control method. The storage control unit updates control information and cache data for I / O processing from the host and various other processes. When doing so, the control information / cache data in memory is updated, and the log is stored in the storage device to make it non-volatile. To do this, an update log is created in the control information log buffer or cache log buffer. The log consists of the update data itself and a log header, and is information that indicates how the control information or cache data in memory was updated. The log header contains information indicating the update location, update size, and the order of updates, as shown in Figure 9.

[0024] The update log in the log buffer is written to the log area on the storage device in an append format. This writing can be done immediately or asynchronously. Because of the append writes, the free space in the log area on each device gradually decreases until it becomes impossible to write to it. To avoid this, it is necessary to reclaim the free space. Different methods are used for the log area for control information and the log area for cache data.

[0025] The base image backup method is used for control information. In the base image backup method, the entire control information is copied to the base image area on the storage device. Once the copy is complete, all update logs prior to the start of the copy are invalidated (reclaimed as free space). On the other hand, a garbage collection method is used to reclaim free space in the log area for cache data. When cache data is overwritten or deleted from the cache (by asynchronous destage, which will be described later), the log for that cache data becomes invalid. The garbage collection method removes invalid logs and copies valid old logs to the end of the log area as new logs, thereby reclaiming the log area as free space.

[0026] 5 is an example of a memory configuration diagram. The memory stores storage control information 10321, a cache data area 10323, a cache data log header management table 10324, a control information log buffer 10325, and a cache data log buffer 10326.

[0027] The storage control information 10321 is an area in which control information for realizing various storage functions is stored, and an example thereof is a cache directory 10322. The cache directory will be explained with reference to FIG. The cache data log header management table 10324 is a table that stores the log headers of all cache data logs on the disk. The control information log buffer 10325 temporarily stores a log of control information, and the cache data log buffer 10326 temporarily stores a log of cache data.

[0028] 6 is an example of a configuration diagram of a storage device. A control information base image area 10332, a control information log area 10333, a cache data log area 10334, and a permanent area 10335 exist on the storage device. The control information base image area is an area to which the entire control information is copied in the base image backup process described below. The control information log area 10333 and the cache data log area 10334 are areas to which logs are saved in the log backup process described below. The permanent area 10335 is an area that stores user data and is managed by the data protection control unit 1086. Each of these will be explained in the respective processing flows described later.

[0029] FIG. 7 is a diagram showing an example of the software module structure of the storage control unit 1083. The storage control unit 1083 executes log creation processing, log saving processing, control information update processing, cache data update processing, base image saving processing, user data cache area reclamation processing, read processing, write processing, and asynchronous destage processing. Each processing will be described in detail later.

[0030] The control information will be explained below. 8 is a diagram showing an example of a cache directory. The cache directory is management information for areas (cache segments) obtained by dividing the cache area, and has entries corresponding to each cache segment.

[0031] Each entry has a cache address, logical volume number, logical volume address, and attribute entry. The cache address indicates the address in memory of the cache segment to which each entry corresponds, and the logical volume number and logical volume address indicate which logical volume and which address the data stored in that cache segment belongs to. If no data is stored in the cache segment, a "-" is stored, indicating "no value." If data is stored in the cache segment, the attribute field has a value of Dirty or Clean. Dirty indicates that the data has not yet been destaged and written to the storage device, and Clean indicates that the data has been destaged and matches the value written to the storage device.

[0032] Figure 9 shows the structure of the log header. The log header is a table included in each log stored in the log buffer area in memory or the log area on the storage device. Each log header has fields for a log sequence number, an update address, an update size, an area type, and a valid flag.

[0033] The log sequence number field stores a log sequence number that is uniquely assigned to each log. The update address field stores the address of the control information or cache data to be updated for each log. The update size field stores the size of the update. The area type field stores a value that identifies either control information or cache data. Here, the character string "control information" or "cache data" is stored. The valid flag field is set to a value of "valid" or "invalid."

[0034] FIG. 10 is an example of a flowchart for read processing. The read processing is called when a read I / O command is received from the host and executed by the storage controller. First, the read command sent from the host is received via the front-end driver, and the read command is interpreted to obtain the logical volume number and logical volume address of the read target (step 4001). Next, it determines whether or not there is a cache hit (step 4002). Specifically, the cache directory is referenced to search for an entry corresponding to the logical volume number and logical volume address. If there is an entry (hit), the cache address is referenced from the entry, a Yes branch is taken, data is read from the cache area (step 4006), and the data is returned to the host (step 4007). If there is no entry corresponding to the logical volume number and logical volume address in the cache directory (miss), a No branch is taken, and staging processing is called (step 4003). The staging processing is executed by the data protection controller, and reads data corresponding to the logical volume number / logical volume address from the persistent area on the storage device. The read data is stored in the cache data area in memory (step 4004). At this time, the cache data update process, which will be described later, is called with the non-volatilization requirement set to "not required." The cache directory is updated (step 4005). At this time, since the cache directory is a piece of control information, the control information update process, which will be described later, is called. At this time, the non-volatilization requirement is set to "not required" as with the cache data update and called. Then, as in the case of a cache hit, the data is read from the cache (step 4006), and the data is returned to the host (step 4007).

[0035] FIG. 11 is an example of a flowchart illustrating write processing. The write processing is executed by the storage control unit. First, a write command sent from the host via the front-end driver is received, and the write command is interpreted to obtain the logical volume number and logical volume address of the write target (step 4101). Next, it is determined whether or not there is a cache hit (step 4102). Specifically, the cache directory is referenced, and an entry corresponding to the logical volume number and logical volume address is searched for in the cache directory. If there is an entry (hit), the cache address is referenced from the entry, a branch is taken to Yes, and the data is stored in the cache (step 4104). At this time, the cache data update process (described later) is called with the non-volatilization requirement set to "needed." Next, the corresponding cache directory is updated (step 4105). At this time, since the cache directory is a piece of control information, the control information update process (described later) is called. At this time, the non-volatilization requirement is set to "needed" as in the cache data update process. Next, the control information confirmation process (described later) is called (step 4106). Finally, a write success response is sent to the host (step 4107).

[0036] In the case of a cache miss, the process branches to No and a cache segment is reserved (4103). The cache segment is reserved by calling the control information update process. At this time, the non-volatile status is set to "yes." After that, the process proceeds to step 4104, and the subsequent steps are the same as in the case of a hit.

[0037] 12 is a flowchart of asynchronous destage processing. Asynchronous destage processing searches the cache directory for an entry with the attribute "dirty" (step 4201), and if no such entry is found, the process ends (branch No). If a dirty entry is found (branch Yes), destage processing is executed for that cache segment (step 4203). The destage process is a process executed by the storage control unit and the data protection control unit, and writes data corresponding to the logical volume number / logical volume address to a permanent area on the storage device. This destaging provides various storage functions, such as thin provisioning, snapshot, and data redundancy. For example, the thin provisioning function dynamically allocates capacity at a fixed size according to the write operation, and generates a logical-physical translation address that maps logical space to physical space. The snapshot function virtually saves data at a certain point in time (called a snapshot image), and when an update occurs, stores only the updated area in a separate area, generating a logical-physical translation address from the logical space of the snapshot image to the physical space of that separate area for reference. The data redundancy function uses technologies such as mirroring and EC (Erasure Coding) to make data redundant, protecting the data by creating a copy of the data and storing it in another node, or by creating a redundant code (parity) from the data and storing it in another node. The thin provisioning and snapshot functions are provided by the storage control unit. The data redundancy function is provided by the data protection unit. After the destage process, the entry is deleted from the cache directory (step 4204). At this time, since this is a cache directory update, the control information update process is called with non-volatilization necessity set to "necessary" and updated. Next, the user data cache log is invalidated (step 4205). This process searches the cache data log header management table and invalidates the user data cache log in the same address range as the data destaged in this process.

[0038] Figure 13 is a flowchart of the control information update process. The control information update process is called when updating control information in memory. When called, the memory address and size for identifying the control information to be updated, the update value, and information indicating whether non-volatilization is required are passed.

[0039] First, the control information in the memory is updated (step 4301). Next, the passed non-volatilization necessity is referenced to determine whether or not non-volatilization is necessary (step 4302). Only if necessary, the log creation process is called (4303).

[0040] FIG. 14 is a flowchart of the cache data update process. Steps 4401 to 4403 are the same as steps 4301 to 4303, except that the update target is cache data rather than control information. Unlike the control information update process, the cache data update process adds steps 4404 to 4406 when it is determined in step 4402 that non-volatilization is required. First, it is determined whether or not to overwrite (step 4404). That is, the cache data log header management table is referenced to search for a log with the same address, and if so, it is determined to be an overwrite. Only if an overwrite is required is the log with the same address in the log header management table invalidated (the valid flag is set to "invalid"). Finally, the log header of the log created in step 4403 is added to the log header management table (step 4406).

[0041] 15 is a flowchart of the log creation process. In this process, the "log buffer" indicates the control information log buffer when the update target is control information, and indicates the user data cache log buffer when the update target is user data cache.

[0042] First, a log sequence number is determined (step 4501). Log sequence numbers are assigned in the order in which logs are created, and one log always corresponds to one log sequence number. Next, an area in the log buffer into which the next log will be written is allocated (step 4502).

[0043] The log creation process may be performed by multiple processes operating in parallel, but in that case, exclusive processing must be performed to prevent the same log sequence number from being acquired by another process and to prevent the same log buffer area from being reserved by another process.

[0044] Next, a log header is created (step 4503). The above-mentioned log sequence number is stored in the sequence number field of the log header, and the values ​​of the update target address in memory and update size passed to this log creation process are stored in the update address field and update size field. The information type field stores "control information" when updating control information, and "cache data" when updating cache data.

[0045] Next, the log is stored in the log buffer (step 4504). The log consists of a log header and the data to be updated. The log header is stored at the beginning of the reserved area previously reserved in the log buffer, and the updated data itself is stored at a memory address obtained by adding the log header size to the reserved area. Finally, the log valid flag in the log header is set to "valid" (step 4505), and this processing ends.

[0046] The control information determination process is shown in Fig. 16. This process simply calls the log save process (step 4601). The flow of log backup processing is shown in Figure 17. First, the log buffer is referenced and any unsaved logs are read (step 4701). Next, the unsaved logs are stored in the log area on the storage device (step 4702). The write position is immediately after the last log written. When the writing is complete, the log is deleted from the log buffer in memory (step 4703).

[0047] Figure 18 shows the processing sequence for base image backup processing. This processing is executed asynchronously. It may also be called when it is detected that the free space in the control information log area is running low. First, the sequence number is referenced (step 4801). This processing is the same as allocating a sequence number in the log creation processing. Next, the entire control information is written to the base image area on the storage device (step 4802). Once this is complete, old logs are invalidated (step 4803). Old logs are logs with sequence numbers lower than the sequence number referenced in step 4801. Specifically, the "valid flag" of those logs on the storage device may be set to "invalid", or the number referenced in step 4801 may be stored on the storage device as the "base image backup sequence number".

[0048] FIG. 19 shows the cache data log garbage collection process. First, it is determined whether the free space in the cache data log is below a threshold (step 4901). If it is not below the threshold, the process terminates without doing anything. If it is below the threshold, the cache log header management table in memory is referenced, and a search is performed starting from the oldest cache log header (the one with the lowest log sequence number) to find the oldest valid log whose validity flag value is "valid" (step 4902). Once the oldest valid log in the cache data log area is identified, a new log is created for the data indicated by that log (step 4903). At this time, a new sequence number may be allocated and reassigned. Next, a save process is invoked to write this log to the storage device (step 4904), and the area in the log area containing this log and with a lower address than this log is reclaimed as free log space (step 4905).

[0049] Figure 20 is a flowchart of the log recovery process. This function is called before the storage control unit starts up when restarting after a power outage. First, the base image is read from the base image area on the storage device (step 41001). Next, the control information log and cache data log are sorted according to sequence number, arranging them from the oldest log (the log with the lowest sequence number) to the newest log (the log with the highest sequence number) (step 41002). After that, the oldest to newest logs are reflected in that order in the control information and cache data areas in memory based on the log addresses (step 41003). This completes the recovery of the control information / cache data after a power outage.

[0050] In this embodiment, the base image saving method is applied to the control information, and the cache data garbage collection method is applied to the cache data area, so that both methods are used selectively. The base image backup method does not require management of the validity / invalidity of each log, but instead backs up the entire target area regardless of whether it has been updated, so the amount of copying required for each backup is large. Therefore, it is suitable for information where it takes a relatively long time for the free space in the log area to be filled, in other words, for information where the size of each update log is small.

[0051] On the other hand, the garbage collection method copies only old valid logs, so the amount of data copied is small. However, because it must manage valid / invalid logs on a log-by-log basis, it is suitable for a small number of logs, i.e., for information that is updated relatively infrequently.

[0052] From this perspective, the base image backup method is suitable for control information, as it is characterized by small update granularity and frequent updates. On the other hand, the cache data area is updated less frequently (in most cases, cache data is updated once per I / O), but the size of each update is large. Therefore, the garbage collection method is suitable for cache data. By using both methods appropriately, the size of the log management information can be reduced, while the amount of copying required for free space recovery can be reduced, resulting in low overhead.

[0053] Furthermore, in this embodiment, when updating cache data or the cache directory, the non-volatile status is set to "not required" for read processing, and set to "required" for write processing, and the log creation process is then called. Since the data stored in the cache data during read processing is the same data on the storage device, even if it is lost due to a power outage, it can be re-staged, so there is no problem. Furthermore, since there is no overhead for creating and saving logs during read processing, performance is improved. [Example]

[0054] Next, a second embodiment will be described. FIG. 21 is a configuration diagram of a storage device, a base image area, and a log area according to the second embodiment. This embodiment is an example in which there are multiple control information log areas and cache data log areas on the storage devices. For example, the control information log area may be in multiple of the storage devices and the cache data log area in the remaining storage device, or both may be in the previous storage device.

[0055] Fig. 22 shows the log backup process in this embodiment. The difference from Fig. 17 is that step 4711 is added after step 4701. Step 4711 is the determination of the destination storage device. A storage device to which logs are to be saved is selected from multiple storage devices. Selection may be made in a round robin manner, or the load on each storage device may be monitored and the storage device with the least load may be selected. By writing logs to multiple storage devices in parallel, the write performance of the storage devices can be effectively utilized to speed up writing. The control information log and cache data log may be written to separate storage devices, or moreover, there may be multiple storage devices to write the control information log and multiple storage devices to write the cache data log.

[0056] Figure 23 shows the base image backup process in this embodiment. What differs from Figure 18 is that steps 4811 and 4812 are added after step 4801. In other words, the base image is divided and stored in base image areas on multiple storage devices. The division can be done simply by size (dividing into fixed length or equal sizes), or the size of each division can be changed depending on the load and available capacity.

[0057] Fig. 24 shows the log recovery process in this embodiment, which differs from Fig. 20 in that step 41001 has been changed to step 41011, and step 41002 has been changed to step 41012, with multiple read destination storage devices present in each case. In step 41011, the base images read from multiple devices are combined and processed as a single base image. In step 41012, the logs read from multiple storage devices are merged / sorted in the same way as the control information log / cache data log. [Example]

[0058] FIG. 25 shows a third embodiment of the present invention. The third embodiment is an example in which a compression function is supported. The compression function is one example of the various functions supported by the storage control unit. The compression function reduces capacity consumption, saves disk space, and reduces disk costs by compressing and storing data received from the host using a data compression algorithm such as LZ4. Although the compression process takes longer, the data is processed in a reduced size after compression, so overall processing time can be expected to be shorter.

[0059] As shown in Figure 25, data received from the host is stored in the cache data area. The processing at this time is the same as the write processing shown in Figure 11, but differs in that the data to be written to memory is compressed and metadata indicating the compressed data and its location is written. At this time, the compressed data and metadata are written to the cache data area in an append format because, with the compression function, the size of the compressed data received from the host is indeterminate (depending on the compression effect), and even if you try to overwrite the old data with the compressed data, it may be larger than the old data and unable to be replaced. Therefore, an append-write method is suitable, which always allocates new space for writing and releases the space for old data as free space. Also, because it is an append-write format, information (metadata) indicating the storage location of the compressed data is required. This metadata is also stored in the cache data area. The compressed data and metadata stored in the cache data area are stored in a cache data log area on the disk in synchronization with the host I / O (that is, before returning a response to the host).

[0060] When storing cache data and control information in the log area, multiple logs stored in the log buffer can be written together in a single disk write operation ("bulk write") to improve processing efficiency, reduce the number of disk writes, and improve performance. Also, because control information is smaller in size than data, it does not need to be compressed.

[0061] If more logs are to be consolidated during this consolidation, it may be possible to wait until more logs have accumulated in the log buffer before evacuation, but this would delay the response of the control information confirmation process, which is executed before the I / O response, and therefore the response to host I / O. To prevent this, the log evacuation process consolidates all logs accumulated in the log buffer at the time it is called, and does not delay the log evacuation process. This method naturally improves the efficiency of consolidation when the I / O load on the storage is high and many logs are simultaneously accumulating in the log buffer, but under low loads, the response to host I / O is not delayed more than necessary, achieving both the high throughput performance required under high loads and the reduced response time required under low loads.

[0062] The compressed data and metadata in the cache data area are written to the persistent area on disk asynchronously with host I / O. This process is the asynchronous destaging process shown in Figure 12. During this process, just like when writing to the cache data area, the destaging process also uses an append write format. Asynchronous destaging improves processing efficiency by bundling nearby data together and writing it in a single disk write process, reducing the number of disk writes and improving performance. This bundling is particularly easy to do with compressed data that uses an append write format. This is the same as when storing in the cache data log area mentioned above. Destaging involves more processing than log evacuation processing. As explained in the description of FIG. 12, in the destage process executed by the data protection unit during this process, data can be made redundant by mirroring or EC (Erasure Coding).

[0063] Figure 25 shows an example of EC, where parity data calculated with other data is stored in other nodes for redundancy. In this third embodiment, it was shown that by collectively writing the control information and cache data logs to the log area, processing efficiency can be improved while at the same time reducing the number of disk writes, improving performance. It was also shown that with destaging, the bulk writing effect is enhanced, particularly with the compression function, which improves processing efficiency, reduces the number of disk writes, and improves performance. In addition, the compression function reduces the amount of data flowing to the disk compared to when it is not compressed, so the reduced data flow to the disk allows for more efficient disk use and improved performance.

[0064] As described above, the disclosed storage system 100 comprises a storage control unit 1083 as a storage controller that controls reading and writing for a drive 1033 as storage having a non-volatile storage device, and when the storage controller performs the reading and writing, it generates a log and stores it in memory 1032, which is a log memory, and writes the log stored in the memory to the storage device, and the log includes a control information log and a cache data log, and when the log is generated, the generated control information log is stored in the log memory and written collectively to the storage device in storage area units having multiple control information logs, and the generated cache data log is stored in the log memory, and the cache data in the log memory is compressed and written to the storage device. In addition, when recovering free space in the memory, the storage controller uses a base image backup method in which free space is recovered by writing to the storage device in units of storage areas having multiple logs, and a garbage collection method in which free space is recovered by writing to the storage device in units of logs. Such a configuration and operation makes it possible to realize a storage system that combines high performance and reliability.

[0065] As an example, when the storage controller updates a log stored in the memory, it stores the updated log in a storage area separate from the storage area in which the pre-update log is stored and invalidates the pre-update log; in the base image backup method, it stores multiple logs that have not been invalidated in that storage area in the storage device and recovers that storage area as free space; and in the garbage collection method, it asynchronously performs the process of recovering the storage area of ​​the invalidated logs as free space and the process of storing the non-invalidated logs in the storage device and invalidating the logs in the storage area in which those logs were stored. The log also includes a log of control information related to storage control and a log of user data related to the reading and writing, and the control information log has a smaller update granularity than the user data log, and the base image backup method is applied to the control information log, and the garbage collection method is applied to the user data log. This allows control information, cache data, and the like in the storage system 100 to be efficiently stored in the storage devices, achieving high performance and reliability.

[0066] In addition, the storage controller manages the control information log and the user data log by attaching sequence control information indicating the processing sequence to the control information log and the user data log, thereby ensuring consistency in the sequence of the control information log and the user data log. Therefore, even if different management methods are used for the control information log and the user data log, consistency can be easily ensured.

[0067] In addition, in the log recovery process in which data is recovered from the control information log and the user data log, the storage controller recovers the control information using the base image backup method, and recovers the user data after reserving space in advance. This allows data recovery to be carried out efficiently.

[0068] Furthermore, the storage controller uses the base image saving method for data whose update size is less than a predetermined value, and uses the garbage collection method for data whose update size is equal to or greater than the predetermined value. By adopting such an operation, free space can be managed efficiently in accordance with the size of updated data.

[0069] The disclosed storage system 100 includes a plurality of the storage controllers, and has a configuration in which the storage controllers are made redundant. That is, in a storage system with redundant storage controllers, It efficiently stores information and cache data in the storage device, achieving high performance and reliability. do.

[0070] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, not only can the configurations be deleted, but also replacements and additions of configurations are possible. For example, in the above embodiment, a configuration in which a sequence number is used as the sequence control information is exemplified, but a timestamp may be assigned to a log and the timestamp may be used as the sequence control information. Also, a pointer to the next log may be assigned and used as the sequence control information. Furthermore, the sequence control information is not limited to being assigned to each log, but may be assigned to a certain range of logs. [Explanation of symbols]

[0071] 100: storage system, 101: host device, 102: network, 103: storage node, 104: management node, 1031: CPU, 1032: memory, 1033: drive, 1083: storage controller

Claims

1. a non-volatile storage device; a storage controller that processes data to be read and written from the storage device; a volatile memory; A storage system comprising: When the storage controller receives a data write request, storing data related to the received write request in the memory; storing a plurality of logs related to the data stored in the memory in parallel in a plurality of the storage devices; After storing the log in the storage device, a completion response is sent to the source of the data write request; destaging the data stored in the memory to the storage device; When the data in the memory related to the write request is lost due to a power outage, the data in the memory is recovered using the log stored in the storage device. A storage system comprising:

2. a non-volatile storage device; a storage controller that processes data to be read and written from the storage device; a volatile memory; A storage system comprising: When the storage controller receives a data write request, storing data related to the received write request in the memory; storing a log related to the data stored in the memory in the storage device; After storing the log in the storage device, a completion response is sent to the source of the data write request; destaging the data stored in the memory to the storage device; When the data in the memory related to the write request is lost due to a power outage, the data in the memory is recovered using a log stored in the storage device, When the data is destaged, invalidating the log in the storage device related to the destaged data; when data that updates data related to the log is written, the storage controller invalidates the log in the storage device related to the data before the update; The storage controller By managing the control information log and the user data log with sequence control information indicating the processing sequence, consistency between the control information log and the user data log is ensured. A storage system comprising:

3. a non-volatile storage device; a storage controller that processes data to be read and written from the storage device; a volatile memory; A storage system comprising: When the storage controller receives a data write request, compressing data related to the received write request and storing it in the memory; storing a compressed log relating to the compressed data in the storage device; After storing the log in the storage device, a completion response is sent to the source of the data write request; destaging the compressed data to the storage device; When the data in the memory related to the write request is lost due to a power outage, the data in the memory is recovered using a log stored in the storage device, When the data is destaged, invalidating the log in the storage device related to the destaged data; When data that updates data related to the log is written, the storage controller invalidates the log in the storage device related to the data before the update. A storage system comprising:

4. a non-volatile storage device; a storage controller that processes data to be read and written from the storage device; a volatile memory; A storage system comprising: When the storage controller receives a data write request, storing data related to the received write request in the memory; storing a log related to the data stored in the memory in the storage device; After storing the log in the storage device, a completion response is sent to the source of the data write request; destaging the data stored in the memory to the storage device; When the data in the memory related to the write request is lost due to a power outage, the data in the memory is recovered using a log stored in the storage device, The logs include a control information log and a cache data log, and the cache data log is larger in size than the control information log; The storage controller collects the control information log and the cache data log separately, and stores them in the storage device for each of the plurality of control information logs and for each of the plurality of cache data logs, compresses the cache data log and stores it in the storage device, and stores the control information log in the storage device without compressing it. A storage system comprising:

5. a non-volatile storage device; a storage controller that processes data to be read and written from the storage device; a volatile memory; A storage system comprising: When the storage controller receives a data write request, storing data related to the received write request in the memory; storing a log related to the data stored in the memory in the storage device; After storing the log in the storage device, a completion response is sent to the source of the data write request; destaging the data stored in the memory to the storage device; When the data in the memory related to the write request is lost due to a power outage, the data in the memory is recovered using a log stored in the storage device, The logs include a control information log and a cache data log, the storage controller collects the control information logs and the cache data logs separately, and stores the control information logs and the cache data logs together in the storage device, When recovering the capacity of the memory storage area in which the logs written to the storage device were stored, the storage controller performs a base image backup method for the control information log, in which the entirety of a certain target area is written to the storage device to recover the free space, and performs a garbage collection method for the cache data log, in which the free space is recovered by writing to the storage device in log units. A storage system comprising:

6. The storage system according to claim 5, The storage controller When updating the log, the post-update log is stored in a storage area different from the storage area in which the pre-update log is stored, and the pre-update log is invalidated; In the base image backup method, a plurality of logs that have not been invalidated in the storage area are stored in the storage device, and the storage area is reclaimed as free space; The garbage collection method asynchronously performs a process of recovering the storage area of ​​the invalidated logs as free space, and a process of storing logs that have not been invalidated in the storage device and invalidating the logs in the storage area where those logs were stored. A storage system comprising:

7. The storage system according to claim 1, wherein: Storing the log related to the data in the storage device requires less processing than destaging the data. A storage system comprising:

8. The storage system of claim 7, the log stored in the storage device is a log for restoring data in a memory, The destaging of data to the storage device is for writing / updating data stored in the storage system. A storage system comprising:

9. A storage control method in a storage system including a nonvolatile storage device, a storage controller that processes data read from and written to the storage device, and a volatile memory, comprising: the storage controller: When a data write request is received, storing data related to the received write request in the memory; storing a plurality of logs related to the data stored in the memory in parallel in a plurality of the storage devices; After storing the log in the storage device, a completion response is sent to the source of the data write request; destaging the data stored in the memory to the storage device; When the data in the memory related to the write request is lost due to a power outage, the data in the memory is recovered using the log stored in the storage device. A storage control method comprising:

Images