Storage device and its control method

Abstract

This paper proposes a storage device and a control method thereof that can improve performance and reliability. [Solution] When the processor performs an operation, it creates a log corresponding to the updates made by the operation and stores it in memory. The log stored in memory is then stored in a drive to make it non-volatile. The log is divided into a first log and a second log according to its type. The first log is divided into multiple groups. When the processor processes the logs, it processes the first log and the second log in a predetermined order, and for the first log, it processes within each group in a predetermined order and across groups in parallel.

Description

Technical Field

[0001] The present invention relates to a storage device and a control method thereof.

Background Art

[0002] Conventionally, in a storage system, a redundant configuration is adopted to improve availability and reliability. For example, Patent Document 1 proposes the following storage system.

[0003] This storage system is composed of a plurality of storage nodes. Each storage node is provided with one or more storage devices that provide storage areas respectively, and one or more control software (hereinafter referred to as storage control units) that read and write the requested data to the corresponding storage device in response to requests from the upper device.

[0004] Each storage control unit is managed as a redundant group in combination with other storage control units arranged in different storage nodes. Each storage control unit holds predetermined configuration information necessary for reading and writing the requested data to the corresponding storage device in response to requests from the upper device respectively, and the configuration information held by each storage control unit belonging to the same redundant group is updated synchronously.

[0005] According to such a storage system, there is an advantage that even if one of the storage control units constituting the redundant group fails due to a failure or the like, the other storage control units can take over the processing.

[0006] By the way, in order to further improve the performance and reliability of a storage system having such a configuration, it is required to efficiently make various data non-volatile so that data loss due to a failure or the like does not occur.

[0007] In this regard, Patent Document 1 discloses the following method for efficiently storing control information, cache data, etc., in a non-volatile storage device in a storage node that includes a non-volatile storage device, a storage controller that controls the reading and writing of data to the storage device, and memory.

[0008] Specifically, in such a storage node, the storage controller generates logs when reading or writing data to the storage device and stores them in log memory, and also writes the logs stored in log memory to the storage device. The logs include control information logs related to updates to control information and cache data logs related to updates to cache data. When logs are generated, the generated control information logs are stored in log memory and written to the storage device in units of multiple control information logs, the generated cache data logs are stored in log memory, and the cache data in log memory is compressed and written to the storage device. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2023-152247 [Overview of the project] [Problems that the invention aims to solve]

[0010] Incidentally, when logs are destaged from volatile memory to non-volatile storage devices to become non-volatile, it is usually necessary to guarantee their order. For this reason, conventional methods have employed a system in which logs are stored in a single queue when stored in memory, a unique sequence number is assigned to each log, and the logs are devolved in the order of their sequence numbers.

[0011] However, this method presented a problem: it was difficult to perform log non-volatility processing in parallel across multiple processors due to reasons such as the need for exclusive access to sequence numbers.

[0012] This invention was made in consideration of the above points, and aims to propose a storage device and a control method thereof that can perform various processes on logs in parallel, thereby improving performance and reliability. [Means for solving the problem]

[0013] To solve the above problems, the present invention provides a storage device that performs data input and output, comprising a processor, volatile memory, and a plurality of non-volatile drives, wherein when the processor performs an operation, it creates a log corresponding to the update content updated by the operation and stores it in the memory, stores the log stored in the memory in the drives and makes it non-volatile, the log is divided into a first log and a second log according to its type, the first log is divided into a plurality of groups, and when the processor processes the log, it processes the first log and the second log in a predetermined order, processes the first log in a predetermined order within each group, and processes the groups in parallel.

[0014] Furthermore, in the present invention, a control method for a storage device that performs data input and output, comprising a processor, volatile memory, and a plurality of non-volatile drives, wherein when the processor performs an operation, it creates a log corresponding to the update content updated by the operation and stores it in the memory, stores the log stored in the memory in the drives to make it non-volatile, the log is divided into a first log and a second log according to its type, the first log is divided into a plurality of groups, and when the processor processes the log, it processes the first log and the second log in a predetermined order, processes the first log in a predetermined order within each group, and provides processing steps to be performed in parallel between groups.

[0015] According to the storage device and control method of the present invention, the processing speed of processing the first and second logs can be improved, and the order of the first and second logs can be reliably guaranteed. [Effects of the Invention]

[0016] According to the present invention, a storage device and a control method thereof that can improve performance and reliability can be realized. [Brief explanation of the drawing]

[0017] [Figure 1] This is a block diagram showing the overall configuration of the storage system according to the first and second embodiments. [Figure 2] This block diagram shows the approximate physical configuration of a storage node. [Figure 3] This is a block diagram showing the logical configuration of the storage nodes. [Figure 4] This is a block diagram illustrating the characteristic configuration of the storage system according to this embodiment. [Figure 5] This is a block diagram showing the logical configuration of memory. [Figure 6] This is a diagram illustrating the cache directory. [Figure 7] It is a block diagram showing the logical configuration of a memory device. [Figure 8] It is a block diagram showing the module configuration of a storage control unit. [Figure 9] It is a chart for explaining a log header. [Figure 10] It is a chart showing the configuration of a control information type management table. [Figure 11] It is a chart showing the configuration of a group management table according to the first embodiment. [Figure 12] It is a flowchart showing the processing procedure of a read process. [Figure 13] It is a flowchart showing the processing procedure of a write process. [Figure 14] It is a flowchart showing the processing procedure of an asynchronous destination process. [Figure 15] It is a flowchart showing the processing procedure of a control information update process. [Figure 16] It is a flowchart showing the processing procedure of a cache data update process. [Figure 17A] It is a flowchart showing the processing procedure of a log creation process. [[ID=3=3]] [Figure 17B] It is a flowchart showing the processing procedure of a log creation process. [Figure 18] It is a flowchart showing the processing procedure of a control information update confirmation process. [Figure 19] [[ID=I]]It is a flowchart showing the processing procedure of a log evacuation process. [Figure 20] It is a flowchart showing the processing procedure of a log reflection process. [Figure 21] It is a flowchart showing the processing procedure of a log recovery process. [Figure 22] It is a chart showing the configuration of a group management table according to the second embodiment.

Embodiments for Carrying Out the Invention

[0018] The following describes one embodiment of the present invention in detail with reference to the drawings.​

[0019] The following embodiments relate to a storage system comprising multiple storage nodes, each implementing one or more SDS (Software Defined Storage). Here, SDS refers to a storage device constructed by implementing software with storage functionality on a general-purpose server device.

[0020] In the disclosed embodiment, the storage node includes memory and a non-volatile device, and temporarily stores control information and cache data in the memory. When the storage node updates the control information or data in response to a write request from a higher-level device, it stores the updated data in log format in the non-volatile device. This makes it possible to make the updated data non-volatile. After that, it responds to the higher-level device.

[0021] Furthermore, the storage node asynchronously destages the data in memory to the storage device. Here, "destaging" refers to the process of writing data to the storage device that reflects the data written to the storage system. The same applies below.

[0022] In this process, the storage system performs operations such as creating logical-to-physical address translation information to convert the logical address on the logical volume where the data recognized by the higher-level device is stored to the physical address on the storage device, in order to search for and randomly access the data destaged on the storage device.

[0023] On the other hand, storing data in log format on a non-volatile device is intended to allow recovery in case of data loss in memory, so there is no need to search for or randomly access the data. Therefore, the aforementioned data conversion information is not required, and the processing overhead for storage is low. As a result, when using volatile memory, response performance can be improved by quickly storing data in log format on a non-volatile storage device and sending a completion response to the higher-level device.

[0024] (1) First embodiment (1-1) Configuration of the storage system according to this embodiment In Figure 1, 1 represents the storage system according to this embodiment as a whole. This storage system 1 comprises one or more host nodes 3, multiple storage nodes 4, and a management node 5, all interconnected via a network 2 consisting of Fibre Channel (FC), Ethernet (registered trademark), or LAN (Local Area Network).

[0025] Host node 3 is a general-purpose computer device that functions as a host (higher-level device) for storage node 4. Host node 3 may also be a virtual computer device such as a virtual machine. Host node 3 sends read commands or write commands (hereinafter, these will be collectively referred to as I / O (Input / Output) commands) to storage node 4 via network 2 in response to user operations or requests from implemented application programs.

[0026] Storage node 4 is a general-purpose physical server device that provides storage space for reading and writing data to host node 3. The specific configuration of storage node 4 will be described later.

[0027] Management node 5 is a computer device used by the system administrator to manage the entire storage system 1. Management node 5 manages multiple storage nodes 4 by grouping them together into a cluster 6. Although Figure 1 illustrates a case where only one cluster 6 is provided, multiple clusters 6 may be provided within the storage system 1.

[0028] Figure 2 shows an example of the physical configuration of storage node 4. As shown in Figure 2, storage node 4 is configured to include a CPU (Central Processing Unit) 10, memory 11, multiple storage devices 12, and a communication device 13.

[0029] The CPU 10 is the processor that controls the operation of the entire storage node 4. The memory 11 consists of volatile semiconductor memory such as SRAM (Static RAM (Random Access Memory)) or DRAM (Dynamic RAM), and is used to temporarily hold programs and data. The CPU 10 executes the programs stored in the memory 11, thereby executing various processes for the entire storage node 4 as described later.

[0030] The storage device 12 consists of one or more types of non-volatile, high-capacity storage devices, such as an SSD (Solid State Drive), a SAS (Serial Attached SCSI (Small Computer System Interface)) hard disk drive, or a SATA (Serial ATA (Advanced Technology Attachment)) hard disk drive. The storage device 12 provides a physical storage area for reading and writing data in response to I / O commands from the host node 3.

[0031] The communication device 13 is an interface for the storage node 4 to communicate with external devices such as the host node 3, management node 5, or other storage nodes 4 via the network 2, and is composed of a NIC (Network Interface Card) or FC card. The communication device 13 performs protocol control when communicating with external devices.

[0032] Figure 3 shows an example of the logical configuration of storage node 4. As shown in Figure 3, storage node 4 is configured to include a front-end driver 20, a back-end driver 21, one or more storage control units 22, and a data protection control unit 23.

[0033] The front-end driver 20 is software that controls the communication device 13 and provides the storage control unit 22 with an abstracted interface for communication with the host node 3, management node 5, or other storage nodes 4 to the CPU 10.

[0034] Furthermore, the backend driver 21 is software that controls each of the storage devices 12 within its own storage node 4 and provides the CPU 10 with an abstracted interface for communication with each storage device 12.

[0035] The storage control unit 22 is software that functions as a controller for SDS (Software Defined Storage). The storage control unit 22 receives I / O commands from the host node 3 and issues I / O instructions to the data protection control unit 23 according to the received I / O commands.

[0036] The storage control unit 22 also has a logical volume creation function that creates logical volumes, and provides the created logical volumes to the host node 3 as storage areas for reading and writing data. Furthermore, the storage control unit 22 associates the logical chunks created by the data protection control unit 23 with the created logical volumes. A logical chunk is a logical storage area to which the physical storage area provided by the storage device 12 is associated.

[0037] Methods for mapping logical chunks and logical volumes include straight mapping and virtual volume functionality (Thin Provisioning). Straight mapping maps logical chunks and logical volumes on a one-to-one basis, making the address of the logical chunk the same as the address of the logical volume. Virtual volume functionality, on the other hand, divides both logical chunks and logical volumes into small areas called pages, and maps logical chunks to logical volumes on a page-by-page basis.

[0038] In this embodiment, each storage control unit 22 implemented in a storage node 4 is managed as a redundant group together with other storage control units 22 located in other storage nodes 4. Hereinafter, this group will be referred to as the storage control unit group 24. The logical volumes created by the storage control units 22 that constitute the storage control unit group 24 are associated with the storage control unit group 24.

[0039] Figure 3 shows a case where one storage control group 24 is composed of two storage control units 22. In the following explanation, we will also proceed assuming that one storage control group 24 is composed of two storage control units 22. However, one storage control group 24 may be composed of three or more storage control units 22.

[0040] In the storage control unit group 24, one storage control unit 22 is set to a state where it can accept I / O commands from the host node 3 (this is the active system state, hereafter referred to as active mode), while the other storage control unit 22 is set to a state where it does not accept I / O commands from the host node 3 (this is the standby system state, hereafter referred to as standby mode).

[0041] The storage control unit 22, which is set to active mode (hereinafter referred to as the active storage control unit), then receives an I / O command for a logical volume associated with the storage control unit group 24 to which it belongs, and executes I / O processing for that logical volume according to that I / O command.

[0042] Furthermore, in the storage control group 24, if a failure occurs in the storage node 4 where the active storage control unit 22 is located, the state of the storage control unit 22, which was previously set to standby mode, is switched to active mode. This allows the standby storage control unit (hereinafter referred to as the standby storage control unit) 22 to take over the I / O processing that the active storage control unit 22 was performing if the active storage control unit 22 becomes inoperable (failover function).

[0043] To realize this failover function, each storage control unit 22 belonging to the same storage control unit group 24 always maintains identical configuration information. This configuration information is necessary for the storage control unit 22 to perform processing related to various functions, such as capacity virtualization, hierarchical storage control, which moves frequently accessed data to storage areas with faster response speeds, data deduplication, which removes duplicate data from stored data, compression, which compresses and stores data, a snapshot function that preserves the state of data at a specific point in time, and a remote copy function that copies data to a remote location synchronously or asynchronously for disaster recovery.

[0044] When the configuration information of an active storage control unit 22 constituting the storage control unit group 24 is updated, the difference in that configuration information before and after the update is transferred as differential data to the standby storage control unit 22 constituting the storage control unit group 24, and the standby storage control unit 22 updates the configuration information it holds based on this differential data. In this way, the configuration information held by each storage control unit 22 constituting the storage control unit group 24 is always maintained in a synchronized state.

[0045] By ensuring that all storage control units 22 constituting the storage control unit group 24 always maintain the same configuration information, even if an active storage control unit 22 or the storage node 4 on which the active storage control unit 22 is located fails or the storage node 4 is removed, the standby storage control unit 22, which is part of the same storage control unit group 24 as the active storage control unit 22, can immediately take over the processing that the active storage control unit 22 was performing up to that point.

[0046] The data protection control unit 23 is software that has the function of creating the aforementioned logical chunks and allocating physical storage space provided by a storage device 12 within its own storage node 4 or another storage node 4 to the logical chunks. The data protection control unit 23 also has the function of reading and writing specified data to a storage device 12 associated with a specified logical chunk (providing the physical storage space allocated to that logical chunk) in accordance with the aforementioned I / O instructions given by the storage control unit 22.

[0047] In this case, when the data protection control unit 23 allocates physical storage space provided by a storage device 12 in another storage node 4 to a logical chunk, it cooperates with the data protection control unit 23 implemented in that other storage node 4 and exchanges data with that data protection control unit 23 via the network 2 (Figure 1) to read and write that data to the storage space in accordance with the I / O requests given by the active storage control unit 22 of the storage control unit group 24.

[0048] (1-2) Log creation and non-volatile treatment according to this embodiment Next, an overview of the log creation and non-volatility processing performed in the storage node 4 of this embodiment will be described with reference to Figure 4. In this regard, first, the types of logs created by the storage control unit 22 will be explained.

[0049] The active storage control unit 22 of storage node 4 updates control information and cache data for I / O processing in response to I / O commands from host node 3, as well as for various other processes. When updating the control information and cache data, the active storage control unit 22 creates a log related to the update. The logs created by the active storage control unit 22 at this time include a local order guarantee information log and a global order guarantee information log.

[0050] The local order guarantee information log includes update logs for cache directory information (hereinafter referred to as the CD (Cache Directory) log), which is a type of control information, and update logs for cache data (hereinafter referred to as the CM (Cache Memory) log).

[0051] Other logs related to logical information include L-SM (Local Shared Memory) logs and G-SM (Global Shared Memory) logs.

[0052] The L-SM log is a type of control information log containing I / O statistics and trace information, and falls under the category of local order assurance information logs. I / O statistics include log statistics such as the number of read and write commands received and the number of I / O operations per logical volume. Trace information is log information used for various analyses, such as the number of error logs. This control information does not require strict consistency with other information and is cleared during failover or startup after a power outage.

[0053] Furthermore, G-SM logs contain configuration information such as the creation of logical volumes and the setting of access paths, and fall under the category of global order assurance information logs. This control information requires strict consistency with other information.

[0054] In the following, control information and cache data for which CD logs, CM logs, or L-SM logs are created when updated will be referred to as local order guarantee information, and control information for which G-SM logs are created when updated will be referred to as global order guarantee information.

[0055] In the storage system 1 of this implementation, as shown in Figure 4, logical volumes (VOL0 to VOL11 in Figure 4) associated with one storage control group 24 are divided into two or more predetermined groups (hereinafter referred to as local groups) 25, and a first local order guarantee information log queue 26 is created in advance, associated with each local group 25. Then, CD logs and CM logs created in relation to each logical volume belonging to one local group 25 are stored (enqueued) as a log group in the first local order guarantee information log queue 26 created, associated with the local group 25 to which the logical volume belongs.

[0056] Furthermore, in this storage system 1, one second local order guarantee information log queue 27 is provided for each storage control group 24, and all L-SM logs created in relation to that storage control group 24 are stored in this second local order guarantee information log queue 27 as a single log group. In addition, in this storage system 1, one global order guarantee information log queue 28 is provided for each storage control group 24, and all G-SM logs created in relation to that storage control group 24 are stored in this global order guarantee information log queue 28 as a single log group.

[0057] The CD logs and CM logs stored in each of the first local order guarantee information log queues 26 are read sequentially from that first local order guarantee information log queue 26 in parallel with the other first local order guarantee information log queues 26, and processes such as non-volatilization and transfer are performed.

[0058] Furthermore, L-SM logs stored in the second local order guarantee information log queue 27 and G-SM logs stored in the global order guarantee information log queue 28 are read from the second local order guarantee information log queue 27 and the global order guarantee information log queue 28 in parallel with other log queues (the first local order guarantee information log queue 26, the second local order guarantee information log queue 27, and / or global order guarantee information log queue 28) and processed for non-volatility and transfer.

[0059] However, the groups of CD logs and CM logs created for each logical volume belonging to one local group 25 may be composed of a first group consisting of CD logs and a second group consisting of CM logs, and a first local order guarantee information log queue for the first group (hereinafter referred to as the first local order guarantee information log queue for CD logs) and a first local order guarantee information log queue for the second group (hereinafter referred to as the first local order guarantee information log queue for CM logs) may be provided.

[0060] In this case, the CD logs created for each logical volume belonging to the corresponding local group 25 are stored in the first local order guarantee information log queue for CD logs, and the CM logs created in relation to each logical volume belonging to the local group 25 are stored in the first local order guarantee information log queue for CM logs. By doing so, it becomes possible to perform processes such as non-volatilization and transfer on the CD logs and CM logs created for each logical volume belonging to the same local group 25 in parallel.

[0061] Figure 5 shows an example of the logical configuration of memory 11 in each storage node 4. As shown in Figure 5, the memory 11 of the storage node 4 is defined as follows: storage control information area 30, cache data area 31, local order guarantee information log queue area 32, and global order guarantee information log queue area 33.

[0062] The storage control information area 30 is an area where control information for realizing various storage functions is stored. One example of such control information is the cache directory 34. The cache directory 34 is the management information for individual sub-regions (hereinafter referred to as cache segments) that are subdivided from the cache data area 31, and as shown in Figure 6, it has entries (rows) corresponding to each cache segment.

[0063] Each entry includes a cache address field 34A, a logical volume number field 34B, a logical volume address field 34C, and an attribute field 34D. The cache address field 34A stores the starting address (cache address) of the cache segment in memory 11 corresponding to that entry.

[0064] Furthermore, the logical volume number field 34B stores the identification number of the logical volume to which the data stored in the cache segment corresponding to that entry is written, and the logical volume address field 34C stores the starting address (logical volume address) of the storage area in that logical volume to which the data is written. If no data is stored in the corresponding cache segment, "-" is stored to indicate that there is no value.

[0065] Furthermore, attribute field 34D stores the attributes of data (cache data) if it is stored in the cache segment. In this case, there are two attributes for the data: "Dirty," which means that the data has not yet been destaged and has not been written to the storage device 12 (Figures 1 and 2); and "Clean," which means that the data has been destaged and matches the value written to the storage device 12.

[0066] Returning to the explanation of Figure 5, the cache data area 31 is a cache area for temporarily storing data. The "cache data" mentioned above refers to the data stored in this cache data area 31, and in the following, the data stored in this cache data area 31 will also be referred to as cache data as appropriate.

[0067] Furthermore, in the local order guarantee information log queue area 32, the required number of first local order guarantee information log queues 26 and second local order guarantee information log queues 27 are created for each storage control group 24 as described in Figure 4. Then, the corresponding CD logs and CM logs created by the corresponding storage control group 24 are temporarily stored in each first local order guarantee information log queue 26, and the L-SM logs created by the corresponding storage control group 24 are temporarily stored in each second local order guarantee information log queue 27.

[0068] Furthermore, in the global order guarantee information log queue area 33, a global order guarantee information log queue 28 is created for each storage control group 24 as described in Figure 4, and the G-SM logs created by the corresponding storage control group 24 are temporarily stored in these global order guarantee information log queues 28.

[0069] On the other hand, Figure 7 shows an example of the logical configuration of the storage device 12 in the storage node 4. As shown in Figure 7, the storage device 12 of the storage node 4 has a control information base image area 35, a control information log area 36, ​​a cache data log area 37, and a persistence area 38 defined within it.

[0070] The control information base image area 35 is an area for storing the entirety of the control information at a specific point in the past. In practice, this control information base image area 35 stores, for example, the entirety of the control information stored in the storage control information area 30 (Figure 5) of memory 11 as a base image, at startup of storage node 4 or at periodic intervals.

[0071] Furthermore, the control information log area 36 and the cache data log area 37 are areas for saving various control information logs and cache data logs, respectively, in the log saving process described later. In addition, the persistence area 38 is an area for storing user data (data written by the user) managed by the data protection control unit 23 (Figure 3).

[0072] Figure 8 shows an example of the software module configuration of the storage control unit 22 (Figure 3) installed on the storage node 4. The storage control unit 22 includes a read processing module 40, a write processing module 41, an asynchronous destaging processing module 42, a log creation processing module 43, a log backup processing module 44, a control information update processing module 45, a cache data update processing module 46, a control information update confirmation processing module 47, and a log recovery processing module 48 as software modules. The functions of each of these software modules will be described later.

[0073] Figure 9 shows an example of the structure of the log header 50 in this storage node 4. The log header 50 is control information that is added to the beginning of each log stored in the local order guarantee information log queue area 32 and the global order guarantee information log queue area 33 on the memory 11 as described in Figure 5, and in the control information log area 36 and the cache data log area 37 on the storage device 12 as described in Figure 7.

[0074] As shown in Figure 9, the log header 50 has a table structure comprising a group number field 50A, a global log sequence number field 50B, a local log sequence number field 50C, an update address field 50D, an update size field 50E, an information type field 50F, and a valid flag field 50G.

[0075] The group number field 50A stores, if the log header 50 is added to the beginning of a local order guarantee information log, the unique identifier (group number) assigned to the local group 25 (Figure 4) to which the logical volume associated with the local order guarantee information log belongs. If the log header 50 is added to the beginning of a global order guarantee information log, a special number (e.g., "0") indicating that the log is a global order guarantee information log is stored in the group number field 50A.

[0076] Furthermore, the global log sequence number field 50B and the local log sequence number field 50C each store a unique combination of log sequence numbers assigned to each log.

[0077] For example, if the log is a global order guarantee information log, the sequence number assigned to that global order guarantee information log to guarantee its order (global log sequence number) is stored in the global log sequence number field 50B, and an invalid value ("-") is stored in the local log sequence number field 50C. Note that a sequential number starting from "1" can be used as the global log sequence number. However, any other number that can guarantee the order of the global order guarantee information log may be used.

[0078] Furthermore, if the log is a local order guarantee information log, the latest global log sequence number is stored in the global log sequence number field 50B, and the local log sequence number field 50C stores the sequence number (local log sequence number) assigned to that local order guarantee information log to guarantee the order of that local order guarantee log within that group. For example, a sequential number starting with "1" can be applied as the local log sequence number. However, any other number that can guarantee the order of the local order guarantee information log may be applied.

[0079] The update address field 50D stores the address on the storage control information area 30 (in the case of control information) or on the cache data area 31 (in the case of cache data) of the memory 11 where the control information or cache data to be updated is stored. The update size field 50E stores the data size of the control information or cache data to be updated.

[0080] The information type field 50F stores information indicating whether the target of the log (what was updated when the log was created) is control information or cache data. In the example in Figure 9, the string "Control Information" is stored if the target is control information, and the string "Cached Data" is stored if the target is cached data.

[0081] Furthermore, the valid flag field 50G stores a valid flag indicating whether the log is valid or not. In the example in Figure 9, the string "Valid" is stored if the log is valid, and "Invalid" is stored if the log is invalid.

[0082] Figure 10 shows an example of the configuration of the control information type management table 51. The control information type management table 51 is a table that stores various types of control information (global order guarantee information or local order guarantee information), and is created in advance and stored in memory 11.

[0083] As shown in Figure 10, the control information type management table 51 is configured to include a control information type column 51A and a global / local column 51B. The control information type column 51A stores the names of configuration information such as "cache directory information," "dirty / clean management bitmap table," "configuration information," "I / O statistics information," and "trace information."

[0084] Furthermore, the Global / Local field 51B stores the string "global" if the corresponding control information is global order guarantee information, and "local" if it is local order guarantee information.

[0085] Furthermore, since cache directory information and bitmap tables indicating whether the data stored in individual sub-regions within a logical volume is dirty data or green data (dirty / clean management bitmap tables) are managed separately for each local group 25 (Figure 4), the control information type column 51A in the control information type management table 51, which corresponds to "cache directory information" and "dirty / clean management bitmap tables," stores the volume number of the corresponding logical volume after the string "local".

[0086] Furthermore, since I / O statistics and trace information are not managed separately by the volume number of the corresponding logical volume, and because order guarantees are not required and the information pertains to shared memory, the control information type column 51A in the control information type management table 51, which corresponds to "I / O statistics" and "trace information," stores the string "local" followed by the string "(order guarantee not required SM)" to indicate this.

[0087] On the other hand, Figure 11 shows an example of the configuration of the group management table 52. This group management table 52 is created by the active storage control unit 22 (Figure 3) for each storage control unit group 24 (Figure 3). This group management table 52 is used to manage which local group 25 (Figure 4) each logical volume associated with the corresponding storage control unit group 24 belongs to.

[0088] In the group management table 52, each entry (row) corresponds to one logical volume. Whenever a new logical volume is created and assigned to one of the local groups 25, that logical volume is registered in the group management table 52. The group management table 52 is stored and managed in a designated location in memory 11.

[0089] As shown in Figure 11, the group management table 52 is configured to include a volume number column 52A and a group number column 52B. The volume number column 52A stores the volume numbers of all logical volumes associated with the corresponding storage control group 24. The group number column 52B stores the identification number (local group number) of the local group 25 to which the corresponding logical volumes (logical volumes with the same volume number in the same row) are assigned.

[0090] This allows the storage control unit 22 to search the group management table 52 using the logical volume number as a key to determine which local group 25 the logical volume belongs to. As a result, it can determine which first local order guarantee information log queue 26 (Figure 4) the local order guarantee information log should be stored in. Incidentally, if there is control information that uses information other than the logical volume number as a key, it is necessary to create a separate group management table corresponding to that information.

[0091] Furthermore, various distribution methods can be widely applied to assign logical volumes to each local group 25, such as assigning them to the local group 25 according to the volume number assigned to the created logical volume, or assigning them to each local group 25 using random numbers.

[0092] For example, in a method of assigning logical volumes to local group 25 based on their volume number, the volume number is divided by the number of local groups 25, and depending on the remainder, the logical volume is assigned to the first local group 25 if the remainder is "0", to the second local group 25 if the remainder is "1", and so on.

[0093] (1-3) Various processes performed on the storage node Next, we will explain the specific details of the various processes performed on storage node 4. In the following explanation, the processing entity for each process will be described as the software module of the storage control unit 22 ("...module") as described above in Figure 7. However, it goes without saying that in practice, the storage control unit 22 (and by extension, the CPU 10 of storage node 4 (Figure 2)) executes the process based on that software module.

[0094] (1-3-1) Read Processing Figure 12 shows the flow of the read process executed on storage node 4 when a read command specifying the target logical volume and logical volume address is given from host node 3 to storage node 4.

[0095] This read operation is performed by the read operation module 40, control information update operation module 45, and cache data update operation module 46, etc., in the active storage control unit 22 of the storage control unit group 24 to which the logical volume to be read is associated, as shown in Figure 8.

[0096] In practice, when the active storage control unit 22 of the storage control unit group 24 receives a read command from the host node 3, the read processing module 40 first interprets the contents of the received read command and obtains the volume number of the logical volume to be read and the logical volume address of the read destination on that logical volume (S1).

[0097] The read processing module 40 also determines whether the data to be read is stored in the cache data area 31 (Figure 5) of the memory 11 (Figure 5) (S2). Specifically, the read processing module 40 searches the cache directory 34 (Figure 6) of the memory 11 for entries of data stored at the logical volume address obtained in step S1 in the logical volume of the volume number obtained in step S1, and determines whether such entries were found.

[0098] If the read processing module 40 obtains a positive result in this determination (cache hit), it retrieves the cache address on the cache data area 31 of the memory 11 where the data to be read is stored from its entry, and reads the cache data stored at that cache address in the cache data area 31 (S6).

[0099] The read processing module 40 then sends the cached data read in step S6 to the host node 3 (S7), and after this, the read processing is terminated.

[0100] In response, if the read processing module 40 obtains a negative result in the judgment in step S2 (cache miss), it requests the data protection control unit 23 (Figure 3) to stage the data to be read, using the logical volume number and logical volume address obtained in step S1 as arguments (S3).

[0101] Upon receiving this request, the data protection control unit 23 works in cooperation with the cache data update processing module 46 to stage the requested data (S4). Specifically, the data protection control unit 23 reads the data corresponding to the logical volume number and logical volume address from the persistence area 38 (Figure 7) on the storage device 12 (Figure 7), sets non-volatility to "not required," and calls the cache data update processing module 46. The cache data update processing module 46, called by the data protection control unit 23, then executes the cache data update process described later with respect to Figure 16, thereby storing the data read by the data protection control unit 23 from the storage device 12 in the cache data area 31 on the memory 11.

[0102] After this, the cache data update processing module 46 sets non-volatility to "not required" and calls the control information update processing module 45. The control information update processing module 45, called by the cache data update processing module 46, then updates the cache directory 34 (Figure 6) stored in the storage control information area 30 of the memory 11 by executing the control information update process described later with respect to Figure 15 (S5).

[0103] Next, the read processing module 40 reads the data to be read, which was staged in step S4, from the cache data area 31 of the memory 11 (S6), and sends the read data to the host node 3 (S7). This completes the read process.

[0104] (1-3-2) Light Processing On the other hand, Figure 13 shows the flow of the write process executed on storage node 4 when a write command specifying the logical volume to be written to and the logical volume address, along with the data to be written, are provided from host node 3 to storage node 4.

[0105] This write operation is performed by the write operation module 41, control information update operation module 45, cache data update operation module 46, and control information update confirmation operation module 47, etc., in the active storage control unit 22 of the storage control unit group 24 to which the logical volume to be written is associated, as shown in Figure 8.

[0106] In practice, when the active storage control unit 22 of the storage control unit group 24 receives a write command from the host node 3, the write processing module 41 first interprets the contents of the received write command and obtains the volume number of the logical volume to be written to and the logical volume address of the logical volume to be written to on that logical volume (S10).

[0107] The write processing module 41 also determines whether data (hereinafter referred to as prior data) is already stored at the logical volume address obtained in step S1 in the logical volume of the logical volume number obtained in step S10 (S11).

[0108] Specifically, the write processing module 41 searches the cache directory 34 (Figure 6) of the memory 11 for data entries stored at the logical volume address obtained in step S10 in the logical volume of the volume number obtained in step S10, and determines whether or not such entries were found.

[0109] If the write processing module 41 obtains a positive result in this determination (cache hit), it retrieves the cache address in the cache data area 31 of memory 11 where the preceding data is stored from its entry, sets the requirement for non-volatility to "required", and calls the cache data update processing module 46. The cache data update processing module 46, called by the write processing module 41, then executes the cache data update process described later with respect to Figure 16, thereby overwriting the data to be written to the cache address obtained by the write processing module 41 as described above (S14).

[0110] Next, the write processing module 41 sets the requirement for non-volatilization to "required" and calls the control information update processing module 45 (Figure 8). The control information update processing module 45, called by the write processing module 41, then updates the cache directory 34 (Figure 6) according to the processing content of step S14 by executing the control information update process described later with respect to Figure 15 (S15).

[0111] Next, the write processing module 41 calls the control information update confirmation processing module 47 (Figure 8). The control information update confirmation processing module 47, called by the write processing module 41, then executes the control information update confirmation process described later with respect to Figure 18, thereby saving the corresponding local order guarantee information logs stored in the first and second local order guarantee information log queues 26 and 27 in the local order guarantee information log queue area 32 of memory 11, and the corresponding global order guarantee information log stored in the global order guarantee information log queue 28 in the global order guarantee information log queue area 33 of memory 11, to the control information log area 36 (Figure 7) of the storage device 12 (S16).

[0112] Furthermore, the write processing module 41 sends a response to the host node 3 indicating that the write operation of the data to be written was successful (S17), and then terminates the write operation.

[0113] In response, if the write processing module 41 obtains a negative result (cache miss) in the judgment of step S11, it allocates a cache segment in the cache data area 31 of the memory 11 (Figure 5) with a size corresponding to the data size of the data to be written (S12).

[0114] Furthermore, the write processing module 41 sets the requirement for non-volatility to "required" and calls the control information update processing module 45. Thus, the control information update processing module 45, called by the write processing module 41, executes the control information update process described later with respect to Figure 15, and registers necessary information such as the logical volume number and logical volume address related to the data to be written in the cache directory 34 (Figure 6) for the cache segment secured by the write processing module 41 in step S12 (S13).

[0115] Furthermore, the light processing module processes steps S14 to S17 in the same manner as described above, and then terminates the light processing.

[0116] (1-3-3) Asynchronous destaging Figure 14 shows the flow of asynchronous destaging, which is periodically executed by the asynchronous destaging processing module 42 (Figure 8) and the control information update processing module 45, etc. Here, "destaging" refers to making the data stored in memory 11 non-volatile by storing it in the storage device 12.

[0117] When the asynchronous destaging module 42 starts this asynchronous destaging process, it first searches the cache directory 34 in memory 11 for entries with the attribute "Dirty" (entries in which "Dirty" is stored in the attribute field 34D mentioned above in Figure 6) (S20).

[0118] Next, the asynchronous destaging module 42 determines whether or not it was able to find such an entry in the search in step S20 (S21). If the asynchronous destaging module 42 obtains a negative result in this determination, it terminates the asynchronous destaging process.

[0119] In response, if the asynchronous destaging module 42 obtains a positive result in the judgment in step S21, it selects one unprocessed entry with the attribute "Dirty" and requests the data protection control unit 23 (Figure 3) to destage the data stored in the cache segment corresponding to the selected entry (S22).

[0120] Thus, the data protection control unit 23, having received this request, performs destaging of the data (S23). Specifically, the data protection control unit 23 obtains information about the entry (cache address, logical volume number, and logical volume address) from the cache directory 34, and reads the corresponding data from the cache data area 31 (Figure 5) based on the obtained information. The data protection control unit 23 also writes the read data to the persistence area 38 (Figure 7) on the storage device 12. At this time, the data protection control unit 23 may make the data redundant using techniques such as mirroring or EC (Erasure Coding).

[0121] Subsequently, the asynchronous destaging processing module 42 sets the non-volatility requirement to "Required" and calls the control information update processing module 45, causing it to execute the control information change processing described later for Figure 15, thereby deleting the entry corresponding to the data destaged in step S23 from the cache directory 34 (S24).

[0122] Next, the asynchronous destaging module 42 invalidates the corresponding cache data log (S25). Specifically, the asynchronous destaging module 42 changes the value stored in the valid flag field 50G in the log header 50 (Figure 9) of the cache data log with the same address as the data that was destaged at this time to "invalid". The invalidated cache data log is then deleted from the first local order guarantee information log queue 26 (Figure 4) by a predetermined periodic process.

[0123] Next, the asynchronous destaging module 42 determines whether it has finished executing the processes in steps S23 to S26 for all entries in the cache directory 34 in which "Dirty" is stored in the attribute field 34D (Figure 6) (S26).

[0124] If the asynchronous destaging module 42 obtains a negative result in this determination, it returns to step S22, and then repeats the processing from step S22 to step S26, sequentially switching the entry selected in step S22 to other applicable entries that have not yet been processed in step S23 and beyond.

[0125] The asynchronous destaging module 42 then terminates the asynchronous destaging process when it obtains a positive result in step S26 by completing the processing in steps S23 to S25 for all relevant entries in the cache directory 34.

[0126] (1-3-4) Control Information Update Process Figure 15 shows the flow of control information update processing performed by the control information update processing module 45 and the log creation processing module 43, etc., in step S6 of the read processing described above for Figure 12, steps S13 and S16 of the write processing described above for Figure 13, and step S24 of the asynchronous destaging processing described above for Figure 14.

[0127] In practice, when the control information update processing module 45 is called by the read processing module 40, the write processing module 41, or the asynchronous destaging processing module 42 in step S6 of the read processing, step S13 or step S16 of the write processing, or step S24 of the asynchronous destaging processing, it starts the control information update processing shown in Figure 15.

[0128] At this time, the control information update processing module 45 is notified by the read processing module 40, the write processing module 41, or the asynchronous destaging processing module 42 of the address in the storage control information area 30 of the memory 11 where the control information to be updated is stored, the size of the control information, the update value, and information indicating whether or not non-volatilization is required.

[0129] The control information update processing module 45 then first updates the corresponding control information stored in the storage control information area 30 of the memory 11 with the updated value notified by the read processing module 40, write processing module 41, or asynchronous destaging processing module 42 as described above (S30).

[0130] Next, the control information update processing module 45 determines whether the requirement for non-volatilization, as notified by the read processing module 40, write processing module 41, or asynchronous destaging processing module 42, is "required" (S31). If the control information update processing module 45 obtains a negative result in this determination, it terminates the control information update process.

[0131] In response, if the control information update processing module 45 obtains a positive result in the judgment in step S31, it calls the log creation processing module 43 (Figure 8) to execute the log creation process described later for Figures 17A and 17B, thereby creating the corresponding log (global order guarantee information log or local order guarantee information log) (S32). After this, the control information update processing module 45 terminates the control information update process.

[0132] (1-3-5) Cache data update process Figure 16 shows the flow of the cache data update process executed by the cache data update processing module 46, which is called in step S4 of the read processing described above for Figure 12, or in step S14 of the write processing described above for Figure 13, and the log creation processing module 43, which is called by the cache data update processing module 46.

[0133] In practice, when the cache data update processing module 46 is called by the read processing module 40 or the write processing module 41 in step S4 of the read processing or step S14 of the write processing, it starts the cache data update processing shown in Figure 16.

[0134] At this time, the cache data update processing module 46 is notified by the read processing module 40 or the write processing module 41 of the address in the cache data area 31 of the memory 11 where the cache data to be updated is stored, the size of the cache data, the update value, and information indicating whether or not non-volatility is required.

[0135] The cache data update processing module 46 then first updates the corresponding data stored in the cache data area 31 of the memory 11 (hereinafter referred to as cache data) with the updated value notified by the read processing module 40 or the write processing module 41 as described above (S40).

[0136] Next, the cache data update processing module 46 determines whether the requirement for non-volatilization, as notified by the read processing module 40 or the write processing module 41, is "required" (S41). If the cache data update processing module 46 obtains a negative result in this determination, it terminates the cache data update process.

[0137] In response, if the cache data update processing module 46 obtains a positive result from this judgment, it calls the log creation processing module 43. Thus, the log creation processing module 43, called by the cache data update processing module 46, creates a log (local order guarantee information log) for the cache data update performed in step S40 by executing the log creation process described later for Figures 17A and 17B (S42).

[0138] The cache data update processing module 46 then terminates this cache data update process.

[0139] (1-3-6) Log creation process Figures 17A and 17B show the flow of the log creation process executed by the log creation processing module 43, which is called in step S32 of the control information update process described above for Figure 15, or in step S42 of the cache data update process described above for Figure 16. The log creation processing module 43 creates the corresponding log according to the processing procedure shown in Figures 17A and 17B.

[0140] In practice, when the log creation processing module 43 is called in step S32 of the control information update processing or step S42 of the cache data update processing, it starts the log creation processing shown in Figures 17A and 17B.

[0141] The log creation processing module 43 then first determines whether the updated information is global order guarantee information (S50). This determination can be made, for example, by determining whether the type of control information stored in the global / local column 51B of the row corresponding to that information in the control information type management table 51 (Figure 10) is "global".

[0142] If the log creation processing module 43 obtains a positive result in this determination, it sends an instruction to the corresponding CPU core to stop updating all of the local order guarantee information (hereinafter referred to as a local update stop instruction) if the updates of the local order guarantee information for each local group 25 described above in Figure 4 are being performed in parallel by multiple CPU cores of the CPU 10 (Figure 2) (S51). The log creation processing module 43 then waits for all of these CPU cores to stop updating the local order guarantee information log (S52).

[0143] Various methods can be applied to determine whether the log creation processing module 43 has stopped updating all local order guarantee information logs. For example, a CPU core that has stopped updating local order guarantee information logs may set a flag to that effect at a predetermined location in memory 11, and the log creation processing module 43 that sent the local update stop instruction may make a decision based on the state of these flags. Alternatively, the decision may be made using inter-processor communication functions such as semaphores or socket communication.

[0144] Then, when the log creation processing module 43 confirms that all such CPU cores have stopped updating the local order guarantee information log, it creates the corresponding global order guarantee information log by processing steps S53 to S57.

[0145] In practice, the log creation processing module 43 first secures a new global log sequence number (S53). As described above with respect to Figure 9, the global log sequence number is a unique identification number for the global order guarantee information log, assigned to the global order guarantee information log in the order of creation. The log creation processing module 43 also secures an area in the global order guarantee information log queue 28 (Figure 4) for writing the next global order guarantee information log (S54).

[0146] Furthermore, when log creation is performed in parallel by multiple CPU cores, it is necessary to implement mutual exclusion to prevent the same global log sequence number from being obtained by another process, and to prevent the same log queue area from being allocated by another process.

[0147] Next, the log creation processing module 43 creates the log header 50 (Figure 9) of the global order guarantee information log to be created at that time (S55). Specifically, the log creation processing module 43 stores the global log sequence number secured in step S53 in the global log sequence number field 50B of the log header 50. The log creation processing module 43 also stores the update destination address on memory 11 and the update size value, which were notified when it was called by the control information update processing module 45 (Figure 8) or the cache data update processing module 46 (Figure 8), in the update address field 50D and the update size field 50E. Furthermore, the log creation processing module 43 stores "control information" in the information type field 50F when control information is updated, and "cache data" when cache data is updated.

[0148] Next, the log creation processing module 43 stores the global order guarantee information log in the global order guarantee information log queue 28 (Figure 4) (S56). The global order guarantee information log consists of a log header 50 and the data to be updated itself. For this reason, the log creation processing module 43 stores the log header 50 created in step S55 at the beginning of the area allocated in step S54 within the global order guarantee information log queue 28 (hereinafter referred to as the allocated area), and stores the data to be updated at an address position obtained by adding the data size of the log header 50 to the beginning address of the allocated area.

[0149] Furthermore, the log creation processing module 43 sets the value stored in the valid flag field 50G of the log header 50 of the global order guarantee information log stored in the global order guarantee information log queue 28 in step S56 to "valid" (S57).

[0150] Next, the log creation processing module 43 sends an instruction to all CPU cores of the CPU 10 (Figure 2), which are performing the local order guarantee information update in parallel, to resume the local order guarantee information update process (hereinafter referred to as the local update resume instruction) (S58), and then terminates this log creation process.

[0151] On the other hand, if the log creation processing module 43 obtains a negative result in the judgment in step S50, it determines whether or not it has received a local update stop instruction from another CPU core of the CPU 10 (Figure 2) which is performing the local order guarantee information update in parallel (S59). If the log creation processing module 43 obtains a negative result in this judgment, it proceeds to step S62.

[0152] In response, if the log creation processing module 43 obtains a positive result in the judgment in step S59, it notifies the CPU core that sent the local update stop instruction that it has stopped updating the local order guarantee information (S60), and then waits for a local update restart instruction to be sent from that CPU core (S61).

[0153] Then, when the log creation processing module 43 receives a local update restart instruction from the CPU core, it creates the corresponding local order guarantee information log by processing steps S62 to S68.

[0154] In practice, the log creation processing module 43 first identifies the local group 25 (Figure 4) to which the logical volume corresponding to the local order guarantee information to be updated belongs, and obtains its local group number (S62).

[0155] For example, if the type of information to be updated is "local", the log creation processing module 43 retrieves the volume number of the logical volume corresponding to the local order guarantee information to be updated, because the global / local column 51B of the row corresponding to that information in the control information type management table 51 (Figure 10) stores the string "local" followed by the volume number of the logical volume. The log creation processing module 43 also retrieves the local group number of the local group 25 to which the logical volume to which that volume number belongs by searching the group management table 52 (Figure 11) using the volume number obtained in this way as a key.

[0156] Next, the log creation processing module 43 secures a new local log sequence number (S63) and obtains the current global log sequence number (S64). The log creation processing module 43 also secures an area in the corresponding first or second local order guarantee information log queue 26,27 (Figure 4) to write the next local order guarantee information log, in the same manner as in step S54 (S65).

[0157] Next, the log creation processing module 43 generates the log header 50 in the same manner as in step S55 (S66). At this time, the log creation processing module 43 stores the global log sequence number obtained in step S64 in the global log sequence number field 50B of the log header 50.

[0158] Furthermore, the log creation processing module 43 stores the local order guarantee information log, consisting of the log header 50 created in step S66 and the data to be updated itself, in the area reserved in step S65 of the corresponding first or second local order guarantee information log queues 26, 27, in the same manner as in step S56 (S67).

[0159] After this, the log creation processing module 43 sets the value stored in the valid flag field 50G of the log header 50 of the local order guarantee information log stored in the first or second local order guarantee information log queues 26,27 in step S67 to "valid" (S68). Then the log creation processing module 43 finishes creating the corresponding local order guarantee information log and then terminates this log creation process.

[0160] (1-3-7) Control information update confirmation process Figure 18 shows the flow of the control information update confirmation process executed by the control information update confirmation processing module 47, which is called in step S17 of the write process described above in Figure 13. The control information update confirmation processing module 47 confirms the update of the corresponding control information according to the processing procedure shown in Figure 18.

[0161] Furthermore, when the control information update confirmation processing module 47 is called by the write processing module 41 in step S17 of the write processing, the logical volume number of the logical volume to which the data is written is notified by the write processing module 41.

[0162] Then, when the control information update confirmation processing module 47 is called by the write processing module 41 in step S17 of the write processing, it starts the control information update confirmation processing shown in Figure 18 and determines whether or not there has been an update to the global order guarantee information since the last time this control information update confirmation processing was performed (S70).

[0163] As a method for making such a determination, when the global order guarantee information is updated, the software module that made the update sets a flag in a specific memory area within memory 11 or storage device 12, and this control information update confirmation process clears that flag. If the control information update confirmation processing module 47 obtains a negative result in this determination, it proceeds to step S74.

[0164] In response, if the control information update confirmation processing module 47 obtains a positive result in step S70, it selects one global sequence guarantee information from among the global sequence guarantee information updated since the last time this control information update confirmation processing was performed, in which steps S73 and S74 are unprocessed and the global sequence number is the smallest (S71). The control information update confirmation processing module 47 also performs a log saving process to save the global sequence guarantee information log created in conjunction with the update of the global sequence guarantee information selected in step S71 (hereinafter referred to as the selected global sequence guarantee information) to the control information log area 36 (Figure 7) of the storage device 12 (S72).

[0165] Furthermore, the control information update confirmation processing module 47 performs a log reflection process (S73) that reflects the changes in the control information stored in the control information log area 36 of the storage device 12, which occurred as a result of the log saving process in step S72, to the control information held by the corresponding standby storage control unit 22.

[0166] Next, the control information update confirmation processing module 47 obtains the local group number of the local group 25 (Figure 4) to which the logical volume of the volume number notified by the write processing module 41 belongs when starting this control information update confirmation processing from the group management table 52 (Figure 11) (S74).

[0167] Next, the control information update confirmation processing module 47 targets the local group 25 of the local group number obtained in step S74 and performs a log saving process to save the local order guarantee information log related to each logical volume belonging to that local group 25 to the storage device 12 (S75).

[0168] Furthermore, the control information update confirmation processing module 47 executes a log reflection process (S76) that reflects the changes in the control information stored in the control information log area 36 of the storage device 12, which occurred as a result of the log saving process in step S75, to the control information held by the corresponding standby storage control unit 22.

[0169] The control information update confirmation processing module 47 then determines whether it has finished executing steps S72 and S73 for all global order guarantee information that has been updated since the last time this control information update confirmation processing was performed (S77).

[0170] Then, if the control information update confirmation processing module 47 obtains a negative result in this judgment, it returns to step S71, and thereafter, steps S72 and S73 sequentially switch the global order guarantee information selected in step S71 to the unprocessed global order guarantee log, and the processing of steps S71 to S77 is repeated.

[0171] The control information update confirmation processing module 47 then completes the processing in steps S72 and S73 for all global order guarantee information that has been updated since the last time this control information update confirmation processing was performed, and when it obtains a positive result in step S77, it terminates this control information update confirmation processing.

[0172] (1-3-8) Log backup process Figure 19 shows the processing procedure for log saving that is performed by the control information update confirmation processing module 47 in step S72 or step S74 of the control information update confirmation processing. The control information update confirmation processing module 47 saves the necessary global order guarantee information logs and local order guarantee information logs to the storage device 12 according to the processing procedure shown in Figure 19.

[0173] In practice, when the control information update confirmation processing module 47 proceeds to step S72 or step S74 of the control information update confirmation processing, it starts the log saving process shown in Figure 19. First, in step S72 of the control information update confirmation processing, it reads the global order guarantee information log of the selected global order guarantee information from the global order guarantee information log queue 28 (Figure 4), and in step S74 of the control information update confirmation processing, it reads one unsaved local order guarantee information log from the corresponding first or second local order guarantee information log queues 26, 27 (Figure 4) (S80).

[0174] Next, the control information update confirmation processing module 47 writes the global order guarantee information log or local order guarantee information log read in step S80 to the storage device 12 (S81). At this time, the control information update confirmation processing module 47 writes the CD log, L-SM log, and G-SM log to the control information log area 36 (Figure 7) of the storage device 12, and the CM log to the cache data log area 37 (Figure 7).

[0175] Furthermore, when the control information update confirmation processing module 47 writes a global order guarantee information log or a local order guarantee information log to the control information log area 36 or cache data log area 37 of the storage device 12, it writes the global order guarantee information log or local order guarantee information log to the control information log area 36 or cache data log area 37 immediately after the last global order guarantee information log or local order guarantee information log written to that area.

[0176] Next, the control information update confirmation processing module 47 deletes the global order guarantee information log or local order guarantee information log that was written to the storage device 12 in step S81 from the original global order guarantee information log queue 28 or the first or second local order guarantee information log queues 26, 27 (S82).

[0177] After this, the control information update confirmation processing module 47 terminates this log saving process if it is step S72 of the control information update confirmation process. If it is step S75 of the control information update confirmation process, the control information update confirmation processing module 47 determines whether it has finished writing all of the target global order guarantee information logs or local order guarantee information logs to the storage device 12 (S83).

[0178] Then, if the control information update confirmation processing module 47 obtains a negative result in this determination, it returns to step S80, and thereafter repeats the processing from step S80 to step S83 while sequentially switching the global order guarantee information log or local order guarantee information log selected in step S80 to another applicable global order guarantee information log or local order guarantee information log that has not been saved to the storage device 12.

[0179] The control information update confirmation processing module 47 then terminates this log saving process when it obtains a positive result in step S83 by saving all the necessary global order guarantee information logs or local order guarantee information logs to the storage device 12.

[0180] (1-3-9) Log reflection process Figure 20 shows the processing procedure for log backup that is executed by the control information update confirmation processing module 47 in step S73 or step S76 of the control information update confirmation processing. Following the processing procedure shown in Figure 20, the control information update confirmation processing module 47 reflects the changes in the control information managed by the active storage control unit, the self-storage control unit 22, in the control information held by the standby storage control unit 22, which is part of the same storage control unit group 24.

[0181] In practice, the control information update confirmation processing module 47 starts this log reflection process when it proceeds to step S73 or step S76 of the control information update confirmation process. The control information update confirmation processing module 47 first reads all the global order guarantee information logs and local order guarantee information logs stored in the control information log area 36 (Figure 7) and cache data log area 37 (Figure 7) of the storage device 12, merges these global order guarantee information logs and local order guarantee information logs, and then sorts them in order of the global log sequence numbers stored in the global log sequence number field 50B (Figure 9) of the log header 50 (Figure 9) (S90). Specifically, the control information update confirmation processing module 47 arranges the merged global order guarantee information logs and local order guarantee information logs in order from the one with the smallest global log sequence number.

[0182] Next, the control information update confirmation processing module 47 selects the global log sequence number with the smallest value among the global log sequence numbers that have not been processed since step S92 (S91), and sends the control information corresponding to the selected global log sequence number (hereinafter referred to as the selected global log sequence number) to the standby storage control unit 22 which is part of the same storage control unit group 24 (S92). Thus, the standby storage control unit 22, having received this control information, reflects (overwrites) the received control information in the control information it holds.

[0183] Next, the control information update confirmation processing module 47 selects one local order guarantee information log from among the local order guarantee information logs whose selected global log sequence number is stored in the global log sequence number field 50B (Figure 9) of the log header 50 (S93) that has not been processed since step S104 and has the smallest local log sequence number stored in the local log sequence number field 50C (Figure 9) of the log header 50 (S93), and transmits the control information or cache data corresponding to the selected local order guarantee information log (hereinafter referred to as the selected local order guarantee information log) to the standby storage control unit 22 which is part of the same storage control unit group 24 (S94). The standby storage control unit 22, upon receiving this control information, reflects (overwrites) the received control information in the control information it holds.

[0184] After this, the control information update confirmation processing module 47 determines whether it has finished executing the process in step S94 for all local order guarantee information logs in which the selected global log sequence number is stored in the global log sequence number field 50B of the log header 50 (S95).

[0185] Then, if the control information update confirmation processing module 47 obtains a negative result in this judgment, it returns to step S93, and thereafter repeats the processing from step S93 to step S95 while sequentially switching the local order guarantee information log selected in step S93 to other local order guarantee information logs that have not yet been processed in step S94.

[0186] Furthermore, when the control information update confirmation processing module 47 obtains a positive result in step S95, as the selected global log sequence number eventually reflects all local order guarantee information logs stored in the global log sequence number field 50B of the log header 50, it determines whether or not it has finished executing the processes in steps S92 to S95 for all the global log sequence numbers sorted in step S90 (S96).

[0187] If the control information update confirmation processing module 47 obtains a negative result in this determination, it increments the global log sequence number (by "1") (S97) and then returns to step S91. The control information update confirmation processing module 47 then repeats the process from steps S91 to S97, sequentially switching the global log sequence number selected in step S91 to the global storage number sorted in step S90, which has the smallest value among the global log sequence numbers that have not been processed since step S92.

[0188] The control information update confirmation processing module 47 then completes the processing from step S92 onward for all global storage numbers sorted in step S90, and when it obtains a positive result in step S96, it terminates this log reflection process. As a result, the update of the control information held by the active storage control unit 22 is reflected in the control information held by the standby storage control unit 22, which is part of the same storage control unit group 24.

[0189] (1-3-10) Log recovery process Figure 21 shows the processing procedure for the log recovery process executed by the log recovery processing module 48 (Figure 8) when the system restarts after a power outage. The log recovery processing module 48 executes this log recovery process when the system restarts after a power outage, thereby restoring the state of the log in memory 11 to the state of the base image stored in the control information base image area 35.

[0190] In practice, when a restart is initiated after a power outage, the log recovery processing module 48 starts this log recovery process and first reads the base image stored in the control information base image area 35 (Figure 7) of the memory device 12 (S100).

[0191] Next, the log recovery processing module 48 reads all the global order guarantee information logs and local order guarantee information logs stored in the control information log area 36 (Figure 7) and cache data log area 37 (Figure 7) of the storage device 12, merges these global order guarantee information logs and local order guarantee information logs, and sorts them in order of the global log sequence numbers stored in the global log sequence number field 50B (Figure 9) of the log header 50 (Figure 9) (S101). Specifically, the log recovery processing module 48 arranges the merged global order guarantee information logs and local order guarantee information logs in order from the one with the smallest global log sequence number.

[0192] Next, the log recovery processing module 48 selects one global log sequence number with the smallest value from among the global log sequence numbers that have not been processed since step S93 (S102), and recovers the control information corresponding to the selected global log sequence number (hereinafter referred to as the selected global log sequence number) (S103).

[0193] Next, the log recovery processing module 48 selects one local order guarantee information log from among the local order guarantee information logs whose selected global log sequence number is stored in the global log sequence number field 50B (Figure 9) of the log header 50, and which has not been processed since step S105 and has the smallest local log sequence number stored in the local log sequence number field 50C (Figure 9) of the log header 50 (S104). Then, it recovers control information or cache data corresponding to the selected local order guarantee information log (hereinafter referred to as the selected local order guarantee information log) (S105).

[0194] In steps S103 and S105, "recovery" refers to reflecting (writing) the corresponding control information or cache data to the storage control information area 30 (Figure 5) or cache data area 31 (Figure 5) on the memory 11, based on the address stored in the update address field 50D (Figure 9) of the log header 50 of the corresponding global order guarantee information log or local order guarantee information log.

[0195] After this, the log recovery processing module 48 determines whether it has finished executing the process in step S105 for all local order assurance information logs in which the selected global log sequence number is stored in the global log sequence number field 50B of the log header 50 (S106).

[0196] If the log recovery processing module 48 obtains a negative result in this determination, it returns to step S104, and thereafter repeats the processing from step S104 to step S106, sequentially switching the local order guarantee information log selected in step S104 to other local order guarantee information logs that have not yet been processed in step S105.

[0197] Furthermore, when the log recovery processing module 48 obtains a positive result in step S106 by eventually recovering all local order guarantee information logs stored in the global log sequence number field 50B of the log header 50 for the selected global log sequence number, it determines whether or not it has finished executing the processes in steps S103 to S106 for all global log sequence numbers sorted in step S101 (S107).

[0198] If the log recovery processing module 48 obtains a negative result in this determination, it increments the global log sequence number (by "1") (S108) and then returns to step S102. The log recovery processing module 48 then repeats the process from steps S102 to S108, sequentially switching the global log sequence number selected in step S102 to the global storage number sorted in step S101, which has the smallest value among the global log sequence numbers that have not been processed since step S103.

[0199] The log recovery processing module 48 then completes the processing from step S103 onward for all global storage numbers sorted in step S101, and when it obtains a positive result in step S107, it terminates the log recovery process. This completes the recovery of control information and cache data after a power outage.

[0200] Within local group 25 (Figure 4), it is necessary to recover the local order guarantee information logs in ascending order of local log sequence numbers. However, between different local groups 25, order guarantee is not required when recovering local order guarantee information logs. Therefore, this log recovery process can be performed in parallel between local groups 25.

[0201] (1-4) Effects of this embodiment As described above, in the storage system 1 according to this embodiment, for each storage control group 24, the logs created in relation to each logical volume associated with that storage control group 24 are divided into local order guarantee information logs, which only need to guarantee the order within the group, and global order guarantee information logs, which need to guarantee the order even with other groups, and the local order guarantee information logs are divided into multiple groups.

[0202] For global order guarantee information logs, sequence numbers (global log sequence numbers and / or local log sequence numbers) are assigned to guarantee order between them and other global order guarantee information logs, and for local order guarantee information logs, sequence numbers (global log sequence numbers and / or local log sequence numbers) are assigned to guarantee order between them and other local order guarantee information logs within the same group.

[0203] Furthermore, various processes such as asynchronous destaging, transmission, and reception are performed in parallel on the local and global order guarantee information logs for each group.

[0204] Therefore, this storage system 1 can improve the processing speed of various processes such as asynchronous destaging, transmission, and reception of logs, thereby improving performance. Furthermore, each storage system 1 can reliably guarantee the order of local order-guaranteed information logs and global order-guaranteed information logs. Thus, this storage system 1 can improve performance and reliability.

[0205] (2) Second embodiment In Figure 1, 60 represents the storage system according to the second embodiment. This storage system differs significantly from the storage system 1 of the first embodiment in that the configuration of the local group 25 described above in Figure 4 is different, but in all other respects, it is configured similarly to the storage system 1 of the first embodiment.

[0206] In practice, in the storage system 1 of the first embodiment, local groups 25 were formed by grouping logical volumes together, but in the storage system 60 of this embodiment, local groups 25 are formed by grouping the address ranges of multiple logical volumes, with the address range of the storage area of ​​each logical volume as the unit.

[0207] The first embodiment is similar in that a first local order guarantee information log queue 26 is provided in association with each local group 25, a second local order guarantee information log queue 27 is provided in association with the L-SM log, and a global order guarantee information log queue 28 is provided in association with the G-SM log.

[0208] Figure 22 shows the configuration of the group management table 62 held by the storage node 61 in this embodiment, as an alternative to the group management table 52 in the first embodiment described above with respect to Figure 11.

[0209] This group management table 62 is created by the active storage control unit 22 (Figure 3) for each storage control unit group 24 (Figure 3), and is used to manage which local group 25 (Figure 4) each address range of the divided storage area of ​​the logical volume associated with that storage control unit group 24 belongs to.

[0210] In the group management table 62, each entry (row) corresponds to an address range of a divided portion or all of the storage area of ​​a logical volume. Whenever a new logical volume is created and a portion or all of the address range of its storage area is allocated to one of the local groups 25, that portion or all of the address range is registered in the group management table 62. The group management table 62 is also stored and managed in the control information base image area 35 of the storage device 12.

[0211] This group management table 62 is configured to include a volume number field 62A, an address range field 62B, and a group number field 62C. The volume number field 62A stores the volume numbers of all logical volumes associated with the corresponding storage control group 24.

[0212] The address range field 62B stores the corresponding address range in the storage area of ​​the corresponding logical volume, and the group number field 62C stores the identification number (local group number) of the local group 25 to which the corresponding address range in the storage area of ​​the corresponding logical volume is assigned.

[0213] Furthermore, various methods can be applied to partition the storage area of ​​a logical volume, such as partitioning it into storage areas of a predetermined size or using a method set by the user. In addition, various distribution methods can be widely applied to distribute the partitioned storage areas of these logical volumes to each local group 25, such as distributing them sequentially to a fixed number of local groups 25 using a round-robin method or distributing them to each local group 25 using random numbers.

[0214] With the storage system 60 of this embodiment having the above configuration, even with respect to I / O performance to one or fewer logical volumes, processes such as log creation, backup, and reflection can be performed in parallel, thereby further improving the overall performance of the storage system 60.

[0215] (3) Other embodiments In the first and second embodiments described above, the present invention was applied to a storage node 4 constituting a storage system 1 as shown in Figure 1. However, the present invention is not limited to this and can be broadly applied to storage devices constituting various other storage systems.

[0216] Furthermore, while the first and second embodiments described above describe cases where sequential numbers starting from "1" are applied as local log sequence numbers and global log sequence numbers assigned to local order guarantee information logs and global order guarantee information logs, the present invention is not limited to these cases, and various other numbers can be broadly applied.

[0217] Furthermore, in the first and second embodiments described above, we have described cases where a method is applied to distribute logical volumes to local groups 25 that is independent of the function of the storage system 1, such as a method using volume numbers or random numbers. However, the present invention is not limited to these, and for example, the load of the CPU cores, such as the number of I / O operations, may be estimated and the load may be averaged, or the logical volumes may be distributed to each local group 25 in a way that averages the capacity. [Industrial applicability]

[0218] This invention can be widely applied to storage devices with various configurations that create logs corresponding to the updated information when the information is updated. [Explanation of symbols]

[0219] 1, 60...Storage system, 3...Host node, 4, 61...Storage node, 10...CPU, 11...Memory, 12...Storage device, 22...Storage control unit, 23...Data protection control unit, 24...Storage control unit group, 25...Local group, 26...First local order guarantee information log queue, 27...Second local order guarantee information log queue, 28...Global order guarantee information log queue, 30...Storage control information area, 31...Cache data area, 32...Local order guarantee information log queue area, 33...Global order guarantee information log queue area, 34... Cache directory, 35... Control information base image, 36... Control information log area, 37... Cache data log area, 38... Persistence area, 40... Read processing module, 41... Write processing module, 42... Asynchronous destaging processing module, 43... Log creation processing module, 44... Log backup processing module, 45... Control information update processing module, 46... Cache data update processing module, 47... Control information update confirmation processing module, 48... Log recovery processing module, 50... Log header, 51... Control information type management table, 52, 62... Group management table.

Claims

1. A storage device comprising a processor, volatile memory, and multiple non-volatile drives, which performs data input and output, When the aforementioned processor performs an operation, A log corresponding to the update content updated by the above operation is created and stored in the memory. The log stored in the memory is stored in the drive and made non-volatile. The aforementioned logs are divided into a first log and a second log, depending on their type. The first log described above is divided into multiple groups, When the processor processes the log, The first log and the second log shall be performed in a predetermined order. The first log is performed in a predetermined order within each group, and in parallel between groups. A storage device characterized by the following features.

2. The log has a sequence number indicating the processing order, and the processing is performed in the order based on the sequence number. The first log includes group identification information, a first sequence number indicating the processing order within the group, and a second sequence number indicating the processing order across logs, including those outside the group. The second log has the second sequence number, The order of the first log and the second log is according to the second sequence number, The order of the first logs is determined according to the first sequence number within each group, and the order of the groups can be processed in parallel. The storage device according to feature 1.

3. The system comprises multiple nodes, each containing the aforementioned processor, the aforementioned volatile memory, and the aforementioned multiple non-volatile drives. Configure a redundant group with multiple nodes. The processing of the log involves transferring and storing the log on another node in the redundancy group, thereby duplicating the log by storing it on at least two nodes. The aforementioned processor, When creating the log, the sequence number is assigned to the log. When processing the aforementioned logs, the logs are processed in the order according to the sequence number. The storage device according to feature 2.

4. The first log includes an update log of cache data and an update log of the cache directory related to the cache stored in the memory when the data is input or output. The second log includes a configuration change log that modifies the logical volume that the processor provides to the higher-level device or the path connecting the logical volume to the higher-level device. The storage device according to feature 1.

5. The aforementioned processor, When a write command is received as the aforementioned operation, The update content updated by the aforementioned write command is stored in the memory, A log corresponding to the aforementioned update is created, the created log is processed and stored in the goodwill drive, After storing the log to the drive, send a response to the write command. The updated contents of the aforementioned memory are destaged to the aforementioned drive. The storage device according to feature 1.

6. The group of the first log includes a first group consisting of cache data update logs and a second group consisting of cache directory update logs. The storage device according to feature 2.

7. The aforementioned processor provides multiple logical volumes to the higher-level device as storage areas. The logical volume is divided according to the plurality of groups relating to the first log, The aforementioned processor, The first log created according to the update content of the operation performed on each of the logical volumes is added to the group corresponding to the first log. The storage device according to feature 1.

8. The aforementioned processor provides a logical volume as storage space to the higher-level device. The storage area of ​​the provided logical volume is divided into the multiple groups relating to the first log, with address ranges as the unit. The aforementioned processor, The first log, created according to the update details of the operations performed on each address range of the logical volume, is added to the group corresponding to the first log. The storage device according to feature 1.

9. A control method for a storage device that performs data input and output, comprising a processor, volatile memory, and multiple non-volatile drives, When the aforementioned processor performs an operation, A log corresponding to the update content updated by the above operation is created and stored in the memory. The log stored in the memory is stored in the drive and made non-volatile. The aforementioned logs are divided into a first log and a second log, depending on their type. The first log described above is divided into multiple groups, When the processor processes the logs, it processes the first log and the second log in a predetermined order, and processes the first log in a predetermined order within each group, and in parallel between groups. A method for controlling a storage device, characterized by comprising the following features.

10. The log has a sequence number indicating the processing order, and the processing is performed in the order based on the sequence number. The first log includes group identification information, a first sequence number indicating the processing order within the group, and a second sequence number indicating the processing order across logs, including those outside the group. The second log has the second sequence number, In the processing step, the processor The order of the first log and the second log is according to the second sequence number, The order of the first logs is determined according to the first sequence number within each group, and the order of the groups can be processed in parallel. The method for controlling a storage device according to feature 9.

11. The aforementioned storage device is The system has multiple nodes, each comprising the aforementioned processor, the aforementioned volatile memory, and the aforementioned multiple non-volatile drives. Configure a redundant group with multiple nodes. The processing of the log involves transferring and storing the log on another node in the redundancy group, thereby duplicating the log by storing it on at least two nodes. The processor, when creating the log, assigns the second sequence number to the log. In the processing step, the processor The logs are processed in the order according to the sequence numbers mentioned above. A method for controlling a storage device according to the feature described in 10.

12. The first log includes an update log of cache data and an update log of the cache directory related to the cache stored in the memory when the data is input or output. The second log includes a configuration change log that modifies the logical volume that the processor provides to the higher-level device or the path connecting the logical volume to the higher-level device. The method for controlling a storage device according to feature 9.

13. The aforementioned processor, When a write command is received as the aforementioned operation, The update content updated by the aforementioned write command is stored in the memory, A log corresponding to the aforementioned update is created, the created log is processed and stored in the goodwill drive, After storing the log to the drive, send a response to the write command. The updated contents of the aforementioned memory are destaged to the aforementioned drive. The method for controlling a storage device according to feature 9.

14. The group of the first log includes a first group consisting of cache data update logs and a second group consisting of cache directory update logs. A method for controlling a storage device according to the feature described in 10.

15. The aforementioned processor provides multiple logical volumes to the higher-level device as storage areas. The logical volume is divided according to the plurality of groups relating to the first log, The aforementioned processor, The first log created according to the update content of the operation performed on each of the logical volumes is added to the group corresponding to the first log. The method for controlling a storage device according to feature 9.

16. The aforementioned processor provides a logical volume as storage space to the higher-level device. The storage area of ​​the provided logical volume is divided into the multiple groups relating to the first log, with address ranges as the unit. The aforementioned processor, The first log, created according to the update details of the operations performed on each address range of the logical volume, is added to the group corresponding to the first log. The method for controlling a storage device according to feature 9.

Images