Storage system and storage control method

Abstract

To appropriately reduce the frequency of writing from a cache to a nonvolatile medium in a storage system composed of a plurality of nodes.SOLUTION: In each of a plurality of nodes constituting a storage system, when there is data to be stored in a segment allocated from a cache, the node determines, for the segment, whether redundancy (redundant data of data in the segment is transferred to another node) is required and whether non-volatilization (data in the segment is stored in a nonvolatile area) is required according to a type of the data, and controls, on the basis of a result of determination, whether to perform the redundancy of data in the segment and whether to perform the non-volatilization of data in the segment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates generally to storage control. [Background technology]

[0002] One example of storage control is remote copy. With regard to remote copy, for example, the technology disclosed in Patent Document 1 is known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-18736 Summary of the Invention [Problem to be solved by the invention]

[0004] Of the primary storage system (storage system at the primary site) and the secondary storage system (storage system at the secondary site), at least the secondary storage system may employ Software Defined Storage (SDS). SDS is based on one or more (typically multiple) storage nodes. These storage nodes may be located, for example, in an on-premise environment or a cloud environment. A storage node (hereinafter referred to as a node) is, for example, a general-purpose computer, and has a cache and a VOL (logical volume). The cache is typically located in volatile memory, and the VOL is typically based on persistent storage.

[0005] The nodes on which SDS is based generally do not have batteries. Therefore, if a power outage occurs in a node, data in the node's cache (typically volatile memory) may be lost. To prevent such data loss, the node performs a data protection process by writing data from the cache to a VOL.

[0006] Specifically, for example, in asynchronous remote copying, the data stored in the cache includes a JNL (journal) and data to be copied from a PVOL (primary VOL) ​​to an SVOL (secondary VOL). The JNL includes the data to be copied (replica) and metadata for that data.

[0007] Assume that a secondary storage system includes a first and a second node. Assume that the first node has a first cache, a first JVOL (JNL VOL), a first memory save area, and a first SVOL. Assume that the first JVOL, the first memory save area, and the first SVOL are areas based on persistent storage devices inside or outside the first node. Assume that the second node has a second cache, a second JVOL, a second memory save area, and a second SVOL. Assume that the second JVOL, the second memory save area, and the second SVOL are areas based on persistent storage devices inside or outside the second node. Assume that the second SVOL is a mirror VOL of the first SVOL.

[0008] When the first node receives a JNL from the primary storage system, for example, the following processing is carried out. The first node writes the received JNL to the first cache, copies the data in the JNL to the first cache, and writes the JNL and data to the first memory backup area. The first node also transfers the JNL and data stored in the first cache to the second storage node for data redundancy. The second node writes the JNL and data to the second cache and writes them to the second memory backup area. This increases the likelihood that the JNL and data can be restored even if the JNL and data are lost from the first cache due to a power outage in the first node. The first node writes the log to the first JVOL, and then writes the JNL to the first JVOL. The first node writes the log to the first SVOL, and then writes the data in the JNL to the first SVOL. Similarly, the second node writes the log to the first JVOL, and then writes the JNL to the first JVOL. The second node writes the log to the second SVOL, and then writes the data in the JNL to the second SVOL. This makes data redundant on the first SVOL and the second SVOL, and even if a failure occurs in one of the first or second nodes, the data can be restored from the other node.

[0009] However, in this process, writing from the cache to the VOL occurs frequently during asynchronous remote copying, which raises concerns about a decrease in the performance of asynchronous remote copying.

[0010] The high frequency of writes from cache to non-volatile media (for example, non-volatile media that forms the basis of a VOL) ​​can pose an issue for storage control other than asynchronous remote copy in storage systems consisting of multiple nodes (storage systems in which data redundancy is implemented between nodes). [Means for solving the problem]

[0011] When each of the multiple nodes that make up the storage system has data to be stored in a segment secured from the cache, the node determines, depending on the type of data, whether or not the segment needs to be made redundant (transferring redundant data of the data in the segment to another node) and whether or not the data in the segment needs to be made non-volatile (storing the data in the segment in a non-volatile area), and based on the results of the determination, controls whether or not to make the data in the segment redundant and whether or not to make the data in the segment non-volatile. [Effects of the Invention]

[0012] According to the present invention, it is possible to appropriately reduce the frequency of writing data from a cache to a non-volatile medium in a storage system configured with a plurality of nodes. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram illustrating an overview of one embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of the physical configuration of the entire system. [Figure 3] FIG. 1 illustrates an example of the configuration of a software platform for a site. [Figure 4] FIG. 1 is a schematic diagram showing an overview of a remote copy configuration. [Figure 5] FIG. 1 is a schematic diagram illustrating an overview of I / O request processing. [Figure 6] FIG. 10 is a schematic diagram illustrating an overview of a recovery process from a node failure. [Figure 7] FIG. 2 is a diagram illustrating an example of data and programs stored in a memory. [Figure 8] FIG. 10 illustrates an example of a system configuration management table. [Figure 9] FIG. 10 illustrates an example of a pair configuration management table. [Figure 10] FIG. 10 illustrates an example of a cache management table. [Figure 11] FIG. 10 is a diagram showing the flow of a pair forming process. [Figure 12] FIG. 10 is a diagram showing the flow of the initial JNL creation process at the main site. [Figure 13] FIG. 10 is a diagram showing the flow of restore processing at a secondary site. [Figure 14A] FIG. 10 is a diagram showing the flow of JNL read processing at the primary site. [Figure 14B] FIG. 10 is a diagram showing the flow of the JNL purge process at the main site. [Figure 15A] FIG. 10 is a diagram showing the flow of cache storage processing at the secondary site. [Figure 15B] FIG. 10 is a diagram showing the flow of destage processing at the secondary site. [Figure 16]FIG. 10 is a diagram showing the flow of an update JNL creation process at the main site. [Figure 17] FIG. 10 is a diagram showing the flow of pair recovery processing. DETAILED DESCRIPTION OF THE INVENTION

[0014] In the following description, an "interface device" may refer to one or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., an NIC and an HBA (Host Bus Adapter)).

[0015] In the following description, "memory" refers to one or more memory devices, which are an example of one or more storage devices, and may typically be a primary storage device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

[0016] In the following description, a "persistent storage device" may refer to one or more persistent storage devices, which are an example of one or more storage devices. A persistent storage device may typically be a non-volatile storage device (e.g., an auxiliary storage device), and specifically may be, for example, a hard disk drive (HDD), a solid state drive (SSD), or a non-volatile memory express (NVMe) drive.

[0017] Furthermore, in the following description, a "processor" may refer to one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a processor device in a broader sense, such as a hardware circuit that performs part or all of the processing (e.g., an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit)).

[0018] In the following description, information that provides an output for an input may be described using expressions such as "xxx table." However, this information may be data of any structure (for example, structured data or unstructured data), or may be a neural network that generates an output for an input, or a learning model such as a genetic algorithm or random forest. Therefore, the "xxx table" may be referred to as "xxx information." In the following description, the structure of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0019] In the following description, processing may be described using a "program" as the subject. However, because a program is executed by a processor to perform a predetermined process using a storage device and / or an interface device, etc., as appropriate, the subject of the process may also be the processor (or a device such as a controller having the processor). A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0020] In the following description, when describing elements of the same type without distinguishing between them, common parts of the reference symbols will be used, and when describing elements of the same type with distinction between them, reference symbols or identifiers of the elements will be used. For example, for PVOL, a reference symbol such as "PVOL102P1" may be used, or an identifier such as "PVOL1" may be used.

[0021] FIG. 1 is a schematic diagram showing an overview of one embodiment of the present invention.

[0022] The primary site 201P has a host 51 and a primary storage system 100P. The host 51 may be a physical computer or a logical computer (for example, a virtual machine). The primary storage system 100P may be a so-called disk array system, but in this embodiment, it is a system made up of multiple primary nodes 210P.

[0023] The secondary site 201S has a secondary storage system 100S. The secondary storage system 100S is a system made up of multiple secondary nodes 210S.

[0024] The node 210 is typically a general-purpose computer, but may be a device other than a general-purpose computer. The node 210 typically does not have a battery. The node 210 has a cache 55, a VOL (logical volume) 102, and a non-volatile area 103. The cache 55 is typically volatile memory. The VOL 102 and the non-volatile area 103 are based on persistent storage devices inside or outside the node 210.

[0025] The multiple primary nodes 210P include, for example, a first primary node 210P1 and a second primary node 210P2. VOL102 includes PJVOL102JP and PVOL102P. "PJVOL" is a JVOL located in the primary site 201P. "JVOL" is a VOL in which a JNL (journal) is stored. "JNL" includes data to be copied and its metadata. The metadata in the JNL includes a sequence number (SEQ#), which is a value for specifying the order in which the data to be copied was written, and the write destination address of the data. "PVOL" is a primary VOL.

[0026] The multiple secondary nodes 210S include, for example, a first secondary node 210S1 and a second secondary node 210S2. VOL102 includes SJVOL102JS and SVOL102S. "SJVOL" is a JVOL located in the secondary site 201S. "SVOL" is a secondary VOL that forms a pair with PVOL.

[0027] In this embodiment, a redundancy flag 2 and a non-volatilization flag 3 are provided for each segment (an example of a cache area) in the cache 55 in each of the primary storage system 100P and the secondary storage system 100S.

[0028] Redundancy flag 2 indicates whether redundancy is required. When the value of redundancy flag 2 is "on", redundancy (copying to other nodes) of data within a segment in cache 55 is implemented. When the value of redundancy flag 2 is "off", redundancy of data within a segment in cache 55 is not implemented.

[0029] The non-volatile flag 3 indicates whether non-volatilization is required. If the value of the non-volatile flag 3 is "on", the data in the segment in the cache 55 is made non-volatile (written to the non-volatile area 103). If the value of the redundancy flag 2 is "off", the data in the segment in the cache 55 is not made non-volatile.

[0030] Depending on the type of data in a segment, the values ​​of the redundancy flag 2 and non-volatilization flag 3 corresponding to that segment are controlled.

[0031] For example, in this embodiment, the following processing is performed. Note that in FIG. 1, PJVOL102JP1S in the second primary node 210P2 is a standby VOL (mirror VOL) ​​for PJVOL102JP1A in the first primary node 210P1. PVOL102P1S in the second primary node 210P2 is a standby VOL for PVOL102P1A in the first primary node 210P1. SJVOL102JS1S in the second secondary node 210S2 is a standby VOL for SJVOL102JS1A in the first secondary node 210S1. SVOL102S2S in the second secondary node 210S2 is a standby VOL for SVOL102S1A in the first secondary node 210S1.

[0032] The first primary node 210P1 receives a write request specifying PVOL 102P1A and writes the write-target data associated with the write request to the first segment of cache 55P1. The first primary node 210P1 sets the values ​​of the redundancy flag 2P1 and non-volatilization flag 3P1 of the first segment to "on." As a result, the first primary node 210P1 transfers the data in the first segment to the second primary node 210P2 (redundancy) and writes it to non-volatile area 103P1 (non-volatilization). The second primary node 210P2 receives the data, writes it to a segment of cache 55P2, and writes it to non-volatile area 103P2. Although not shown, the redundancy flag may be set to "off" and the non-volatilization flag may be set to "on" for the segment of cache 55P2 (the segment to which data from the first primary node 210P1 is written). As a result, data redundancy is skipped and data is made nonvolatile (data is written to the nonvolatile area 103P2).

[0033] The first primary node 210P1 generates a JNL including data in the first segment of cache 55P1 and writes the JNL to the second segment of cache 55P1. The first primary node 210P1 sets the values ​​of redundancy flag 2P2 and non-volatilization flag 3P2 of the second segment to "on." Therefore, the first primary node 210P1 transfers the JNL in the second segment to the second primary node 210P2 (redundancy) and writes it to non-volatile area 103P1 (non-volatilization). The second primary node 210P2 receives the data, writes it to the segment of cache 55P2, and writes it to non-volatile area 103P2. Although not shown, the redundancy flag may also be set to "off" and the non-volatilization flag may also be set to "on" for the segment of cache 55P2 (the segment to which the JNL from the first primary node 210P1 is written). As a result, redundancy of the JNL is skipped and non-volatilization of the JNL is performed.

[0034] Although not shown, the first primary node 210P1 writes data in cache 55P1 to PVOL102P1A, and writes JNL in cache 55P1 to PJVOL102JP1A. Similarly, the second primary node 210P2 writes data in cache 55P2 to PVOL102P1S, and writes JNL in cache 55P2 to PJVOL102JP1S.

[0035] The first secondary node 210S1 receives a JNL from the first primary node 210P1. In this embodiment, the JNL is received in response to a JNL read request from the first secondary node 210S1 to the first primary node 210P1 (for example, a read request specifying the SEQ# of the JNL to be read), but the JNL may also be received as a JNL write request from the first primary node 210P1 to the first secondary node 210S1 (for example, a write request associated with the JNL to be written).

[0036] The first secondary node 210S1 writes the received JNL to the first segment of the cache 55S1. The first secondary node 210S1 sets the redundancy flag 2S1 and non-volatilization flag 3S1 of the first segment to "off." As a result, both redundancy and non-volatilization of the JNL are skipped.

[0037] The first secondary node 210S1 writes the data in the JNL in the first segment of cache 55S1 (data to be copied) to the second segment of cache 55S1. The first secondary node 210S1 sets the value of redundancy flag 2S2 of the second segment to "on" and the value of non-volatilization flag 3S2 to "off." As a result, data redundancy is implemented, but non-volatilization of the data is skipped. In other words, the first secondary node 210S1 writes the data in the second segment of cache 55S1 to SVOL102S1A and transfers the data to the second primary node 210P2. The second primary node 210P2 receives the data, writes it to the segment of cache 55S2, and then writes it to SVOL102S1S. Although not shown, for the segment of cache 55S2 (the segment to which data from the first secondary node 210S1 (data in the JNL) is written), both the redundancy flag and the non-volatilization flag may be set to "off." This causes both redundancy and non-volatilization of the data in the JNL to be skipped.

[0038] According to the above processing, redundancy and nonvolatilization of JNL in the first segment of cache 55S1 is skipped, and nonvolatilization of data in the second segment of cache 55S1 is skipped. Even if redundancy and nonvolatilization are skipped, if JNL or data is lost from cache 55S1 due to a power outage or the like in the first secondary node 210S1, the JNL or data can be restored from primary site 201P.

[0039] Specifically, JNL containing data is stored in PJVOL102JP1A (and 102JP1S) until the data is written to the first and second SVOL102S1A (and 102S1S). That is, when data is written to SVOL102S1A (and 102S1S), the first secondary node 210S1 notifies the first primary node 210P1 of the SEQ# of the JNL containing the data that has been copied to SVOL102S1A (and 102S1S). When the first primary node 210P1 receives notification of the SEQ# of the JNL to which data has been copied (reflected to SVOL102S), it purges the JNL with that SEQ# from JVOL102JP1A (the second primary node 210P2 also purges the JNL with that SEQ# from JVOL102JP1S). In this way, the JNL containing the data is stored in the primary site 201P until the data is written to SVOL102JS1A (and 102S1S). In order to restore the JNL and data that were lost from cache 55S1 before being written to JVOL102JS1A or SVOL102S1A, the first secondary node 210S1 sends a JNL read request specifying the SEQ# of the JNL to be restored to the first primary node 210P1. This enables the first secondary node 210S1 to obtain the JNL with that SEQ# from the first primary node 210P1, and also to obtain the data within that JNL.

[0040] This embodiment will be described in detail below.

[0041] FIG. 2 is a diagram showing an example of the physical configuration of the storage system 101. As shown in FIG.

[0042] There are multiple sites 201. The sites 201 are communicably connected via a network 202. The network 202 is, for example, a wide area network (WAN), but is not limited to a WAN. The site 201 is a data center or the like, and includes multiple (or one) nodes 210.

[0043] The node 210 may be a general-purpose computer. The node 210 includes, for example, one or more processor packages 213 including a processor 211 and memory 212, one or more drives 214, and one or more ports 215. These components are connected via an internal bus 216. The drive 214 is an example of a persistent storage device.

[0044] The processor 211 is, for example, a CPU (Central Processing Unit) and performs various types of processing.

[0045] The memory 212 is typically a volatile memory, and stores control information and data necessary to realize the functions of the node 210. The memory 212 also stores, for example, programs executed by the processor 211. The drive 214 stores various types of data, programs, etc.

[0046] The port 215 is connected to a network 220 within the site 201, and connects the node to other nodes 210 within the site 201 so as to be able to communicate with each other via the network 220. The network 220 is, for example, a LAN (Local Area Network), but is not limited to a LAN.

[0047] The physical configuration of the system is not limited to the above-described configuration. For example, the networks 202 and / or 220 may be redundant. For example, the network 220 may be separated into a management network and a storage network, the connection standard may be Ethernet (registered trademark), Infiniband, or wireless, and the connection topology is not limited to the configuration shown in Fig. 2. For example, the drive 214 may be configured independent of the node 210.

[0048] FIG. 3 is a diagram showing an example of the configuration of the software platform of the site 201.

[0049] For example, a secondary site 201S may employ a software platform having the configuration illustrated in Fig. 3. The site 201 has a network storage service 30 that provides multiple persistent stores 32 to multiple nodes 210 via a network 220. The persistent stores 32 are storage areas based on one or more drives 214.

[0050] The node 210 includes an instance store 65 , a hypervisor 64 , and a virtual machine 61 .

[0051] The instance store 65 provides temporary block-level storage for instances, which may reside on drives 214 physically attached to the node 210.

[0052] The hypervisor 64 dynamically creates and deletes the virtual machines 61 .

[0053] The virtual machine 61 manages one or more virtual drives 63 and runs storage control software (SCS) 720 .

[0054] The SCS 720 controls I / O (Input / Output) for the virtual drive 63. The storage control program 720 is made redundant between the nodes 210. That is, if a failure occurs in a node 210, the SCS (Standby) 720 of another node 210 changes from Standby to Active, replacing the SCS (Active) 720 of the node 210.

[0055] The virtual drive 63 is a storage area to which the instance store 65 or the persistent store 32 is allocated. The virtual drive 63 may be treated as a VOL 102.

[0056] In this way, the site 201 uses an instance store 65 using DAS (Direct Attached Storage) and storage (network storage service 30) via a network 220 such as iSCSI. The hypervisor 64 may not be required, and for example, the DAS and network storage service 30 may be configured using bare metal.

[0057] FIG. 4 is a schematic diagram showing an overview of a remote copy configuration.

[0058] A remote copy pair is established between a plurality of VOLs 102 between the primary site 201P and the secondary site 201S. Specifically, for example, two consistency groups 401a and 401b are established between the primary site 201P and the secondary site 201S. A consistency group 401 is made up of a plurality of (or one) remote copy pair VOLs 102, and in the consistency group 401, a plurality of PVOLs are copied to an SVOL while maintaining consistency. More specifically, for example, in the consistency group 401, update differential data up to the same time for a plurality of PVOLs 102 is copied to a plurality of SVOLs. Furthermore, control of the consistency group 401 (consistency control) is managed by the PJVOL. The PJVOL stores update differential data for a plurality of (or one) PVOLs 102P together with metadata such as the write time. When the primary site 201P transfers data from the PVOL to the secondary site 201S, it transfers the update differential data written to the PJVOL up to the same time to the secondary site 201S. This makes it possible to copy data to the SVOL while maintaining consistency in the update times between multiple PVOLs.

[0059] For example, according to consistency group 401a, data is copied to SVOL1 and SVOL2 in secondary node 210S1 via PJVOL1 and SJVOL1 while maintaining the consistency of PVOL1 and PVOL2 in primary node 210P1. According to consistency group 401b, data is copied to SVOL3 in secondary node 210S2 and SVOL4 in secondary node 210S3 while maintaining the consistency of PVOL3 in primary node 210P2 and PVOL4 in primary node 210P3, via PJVOL2 in primary node 210P2, PJVOL3 in primary node 210P3, SJVOL2 in secondary node 210S2, and SJVOL3 in secondary node 210S3. PJVOLs and SJVOLs do not necessarily have a one-to-one correspondence (for example, one-to-many, many-to-one, or many-to-many may also be used), and PJVOLs may be an area on memory 212.

[0060] As can be seen from the specific configuration described above, the consistency group 401 may be made up of VOLs 102 in a specific node 210 in the site 201, or may be made up of VOLs 102 in multiple nodes 210 in the site 201.

[0061] FIG. 5 is a schematic diagram showing an overview of I / O request processing.

[0062] First, the application 502 running on the host 51 issues a write request specifying PVOL1 to the primary node 210P1. Upon receiving the write request, the primary node 210P1 writes data A and B associated with the write request to PVOL1, and further writes a JNL including data A and B as update differential data to PJVOL1.

[0063] Next, the primary node 210P1 transfers the JNL (update difference data) written to PJVOL1 to SJVOL1 and SJVOL1 (standby) of the secondary site 201S. At this time, if multiple communication paths have been established between the primary site 201P and the secondary site 201S, the JNL may be transferred using any of the communication paths. Normally, the primary node 210P1 transfers the JNL to the secondary node 210S1 that has ownership of SVOL1 paired with PVOL1. However, if a failure occurs in the communication path that has ownership, the primary node 210P1 may transfer the JNL to a secondary node 210S2 or the like that does not have ownership. For example, when primary node 210P1 transfers a JNL to secondary node 210S2 which does not have ownership, secondary node 210S2 transfers the received JNL to secondary node 210S1 which does have ownership, and secondary node 210S1 writes the JNL to SJVOL1.

[0064] Next, secondary node 210S1 writes the JNL written to SJVOL1 to SVOL1. Data A and B written to SVOL1 are then written to drive 214a via storage pool 504a. If drive 214a is configured as a Direct Attached Storage (DAS) in which the node 210 and drive 214 are connected one-to-one, the JNL is written to drive 214a mounted on secondary node 210S1. By writing all of the data to be copied to SVOL1 to drive 214a of secondary node 210S1, which has ownership of SVOL1, in this way, when data is later read from SVOL1, there is no need to read the data from another node. This makes it possible to eliminate inter-node transfer processing and achieve high-speed read processing.

[0065] The storage pool 504 may be an area based on one or more drives 214. Storage functions such as thin-provisioning, compression, or deduplication are provided, and processing of the storage functions required for data written to the storage pool 504 is performed.

[0066] When writing data to drive 214a, secondary node 210S1 also writes redundant data of the written data to drive 214b of secondary node 210S2 to protect the data from node failure. When writing redundant data, if the data protection policy is replication, a replica of the data is written to drive 214b as redundant data. On the other hand, if the data protection policy is erasure coding, parity is calculated from the data, and the calculated parity is written to drive 214b as redundant data.

[0067] FIG. 6 is a schematic diagram showing an outline of a recovery process from a node failure.

[0068] SCS720 is running on secondary nodes 210S1, 210S2, and 210S3. Each secondary node 210S has an active SCS (Active) and a standby SCS (Standby) corresponding to the SCS (Active) in another secondary node 210S. For example, secondary node 210S1 has SCS1 (Active) and SCS3 (Standby), secondary node 210S2 has SCS2 (Active) and SCS1 (Standby), and secondary node 210S3 has SCS3 (Active) and SCS2 (Standby). SCSx (Active) and SCSx (Standby) belong to the redundancy group of SCSx, and there can be multiple SCSx (Standby) rather than just one (x is a natural number).

[0069] The process of recovering from a node failure will be explained using the specific example shown in FIG.

[0070] In order to take over the remote copy pair information of the secondary node 210S1, the secondary node 210S2 has a copy of the configuration information of SVOL1 and SJVOL1 that the secondary node 210S1 has. The secondary node 210S2 also stores redundant data of the data written to the drive 214a of the secondary node 210S1 in the drive 214d. Furthermore, the secondary node 210S2 has established a communication path with the primary node 210P1.

[0071] For example, if secondary node 210S1 stops due to a failure, secondary node 210S2, which detects the failure of secondary node 210S1, takes over the processing of SCS1 (Active) of secondary node 210S1, and SCS1 (Standby) becomes SCS1 (Active). Secondary node 210S2 communicates with primary node 210P1 and continues remote copy processing between PVOL1 and SVOL1. In other words, failover occurs from SCS1 of secondary node 210S1 to SCS1 of secondary node 210S2. As a result, even if a node failure occurs at one of the secondary sites 201S, another secondary site 201S can continue remote copy from the primary site 201P.

[0072] FIG. 7 is a diagram showing an example of data and programs stored in the memory 212. As shown in FIG.

[0073] Information is read from the drive 214 to the memory 212. For example, the various tables included in the control information table 710 and the various programs included in the SCS 720 are deployed in the memory 212 while the processes in which they are used are being executed, but at other times they are stored in a non-volatile storage area such as the drive 214 in case of a power outage or the like.

[0074] The control information table 710 includes a system configuration management table 711 , a pair configuration management table 712 , and a cache management table 713 .

[0075] The SCS 720 includes a pair creation processing program 721, an initial JNL creation processing program 722, a restore processing program 723, a JNL read processing program 724, a JNL purge processing program 725, a cache store processing program 726, a destage processing program 727, an updated JNL creation processing program 728, and a pair recovery processing program 729.

[0076] FIG. 8 is a diagram showing an example of the system configuration management table 711. As shown in FIG.

[0077] The system configuration management table 711 includes a node configuration management table 810, a drive configuration management table 820, and a port configuration management table 830. Each site 201 has a node configuration management table 810 for the multiple nodes 210 present in that site 201, and each node 210 has a drive configuration management table 820 and a port configuration management table 830 for the drives 214 within its own node 210.

[0078] The node configuration management table 810 is provided for each site 201, and stores information indicating the configuration of the nodes 210 provided in the site 201 (such as the relationship between the nodes 210 and the drives 214). More specifically, the node configuration management table 810 stores, for each node 210, information such as a node ID 811, a status 812, a drive ID list 813, and a port ID list 814.

[0079] The node ID 811 is the ID of the node 210. The status 812 indicates the status of the node 210 (for example, "Normal", "Warning", or "Failure"). The drive ID list 813 is a list of IDs of the drives 214 provided in the node 210. The port ID list 814 is a list of IDs of the ports 215 provided in the node 210.

[0080] The drive configuration management table 820 is provided for each node 210, and stores information indicating the configuration of the drives 214 provided in the node 210. More specifically, the drive configuration management table 820 stores information such as a drive ID 821, a status 822, and a size 823 for each drive 214.

[0081] The drive ID 821 is the ID of the drive 214. The status 822 indicates the status of the drive 214. The size 823 indicates the capacity of the drive 214.

[0082] The port configuration management table 830 is provided for each node 210, and stores information indicating the configuration of the ports 215 provided in the node 210. More specifically, the port configuration management table 830 stores information such as a port ID 831, a state 832, and an address 833 for each port.

[0083] The port ID 831 is the ID of the port 215. The status 832 indicates the status of the port 215. The address 833 indicates an address on the network assigned to the port 215. The address may be in the form of an IP (Internet Protocol), a WWN (World Wide Name), a MAC (Media Access Control) address, or the like.

[0084] FIG. 9 is a diagram showing an example of the pair configuration management table 712.

[0085] The pair configuration management table 712 includes a VOL management table 910 , a pair management table 920 , and a JNL management table 930 .

[0086] The VOL management table 910 stores information indicating the configuration of the VOLs 102. More specifically, the VOL management table 910 stores, for each VOL 102, information such as a VOL ID 911, an owner node ID 912, a retreat node ID 913, a size 914, and attributes 915.

[0087] The VOL ID 911 is the ID of the VOL 102. The owner node ID 912 is the ID of the node 210 that has ownership of the VOL 102. The fallback node ID 913 is the ID of the node 210 that takes over processing in the event of a failure in the node 210 that has ownership of the SVOL. The size 914 indicates the capacity of the VOL 102.

[0088] The attribute 915 indicates the attribute of the VOL 102. "NML_VOL" means a normal VOL that does not belong to a consistency group. "PAIR_VOL" means a PVOL or SVOL that belongs to a consistency group. "JNL_VOL" means a JVOL.

[0089] The pair management table 920 stores information indicating the configuration of a remote copy pair. More specifically, the pair management table 920 stores, for each consistency group, information such as a pair group ID 921, a PJVOL ID 922, a PVOL ID 923, an SJVOL ID 924, an SVOL ID 925, and a status 926.

[0090] The pair group ID 921 is the ID of the consistency group. The PJVOL ID 922 is a list of the IDs of the PJVOL102JP that belong to the consistency group. The PVOL ID 923 is a list of the IDs of the PVOL102P that belong to the consistency group. The SJVOL ID 924 is a list of the IDs of the SJVOL102JS that belong to the consistency group. The SVOL ID 925 is a list of the IDs of the SVOL102S that belong to the consistency group. The status 926 indicates the status of each remote copy pair in the consistency group (for example, "PAIR", "COPY", "SUSPEND", etc.). "PAIR" is a status in which writing to PVOL102P is periodically reflected in SVOL102S. "COPY" is a status in which initial copying is in progress. "SUSPEND" is a status in which the pair is suspended (a status in which synchronization between PVOL102P and SVOL102S is not performed).

[0091] The JNL management table 930 stores information related to JNLs. More specifically, the JNL management table 930 stores, for each JNL, information such as a pair group ID 931, a JNL ID 932, a P / SVOL ID 933, a P / SVOL address 934, a size 935, and a cache segment ID 936.

[0092] The pair group ID 931 is the ID of the consistency group to which the JNL belongs. The JNL ID 932 is the ID of the JNL. The JNL ID corresponds to SEQ# and is, for example, a consecutive number in the consistency group. In other words, the JNL ID indicates the order of writing, and the data in the JNL is stored in SVOL102S in the consistency group in the order of the JNL IDs.

[0093] The P / SVOL ID 933 includes the ID of the PVOL 102P to which the data in the JNL is written, and the ID of the SVOL 102S to which the data in the JNL is written. The P / SVOL address 934 includes the storage address of the data in the PVOL 102P to which the data in the JNL is written, and the storage address of the data in the SVOL 102S to which the data in the JNL is written.

[0094] The size 935 indicates the size of the JNL. For example, one JNL contains one or more pieces of data. The cache segment ID 936 is the ID of the cache segment to which the data in the JNL is written.

[0095] FIG. 10 is a diagram showing an example of the cache management table 713.

[0096] The cache management table 713 is a table related to the cache 55. The cache management table 713 includes a dirty queue 1001, a clean queue 1002, a free queue 1003, and a cache segment management table 1004. In this embodiment, there are a plurality of different segment sizes, and segments of a plurality of different segment sizes are prepared in advance. However, the segment size may be variable depending on the cache allocation request.

[0097] The dirty queue 1001 is a queue of IDs (addresses) of dirty segments in which dirty data is stored, for each drive 214, and for which the drive 214 is the write destination. A "dirty segment" is a segment in which dirty data is stored. "Dirty data" is data that has not been written to the drive 214.

[0098] The clean queue 1002 is a queue of IDs (addresses) of clean segments having each segment size. A "clean segment" is a segment in which clean data is stored. "Clean data" is data that has been written to the drive 214.

[0099] The free queue 1003 is a queue of IDs (addresses) of free segments having each segment size. A "free segment" is a segment to which new data may be written. A segment of the desired size is reserved from the free queue 1003, and data is written to the reserved segment.

[0100] The cache segment management table 1004 stores information about cache segments. More specifically, the cache segment management table 1004 stores, for each cache segment, information such as a segment ID 1041, a memory address 1042, a size 1043, a VOL ID 1044, a VOL address 1045, and a redundancy-non-volatilization flag 1046.

[0101] The segment ID 1041 is the ID of the cache segment. The memory address 1042 is the address of the cache segment (an address in the cache 55). The size 1043 is the size of the cache segment.

[0102] The VOL ID 1044 is the ID of the VOL 102 to which the data in the cache segment is written. The VOL address 1045 is the address in the VOL 102 to which the data is written (the address to which the data is written).

[0103] The redundancy-non-volatilization flag 1046 includes a redundancy flag 2 and a non-volatilization flag 3 corresponding to the cache segment. "1" means "on" and "0" means "off."

[0104] An example of processing performed in this embodiment will be described below. In the description with reference to Figures 11 to 17, to avoid confusion, the reference signs of programs in the primary site 201P will have a suffix "P" and the reference signs of programs in the secondary site 201S will have a suffix "S". In the description below, "initial JNL" refers to the JNL generated in the initial copy, and "updated JNL" refers to the JNL generated in response to an update of PVOL 102P after the initial copy.

[0105] FIG. 11 is a diagram showing the flow of the pair forming process.

[0106] According to the pair creation processing, a remote copy pair is created through communication between the primary site 201P and secondary site 201S. In the explanation of Figure 11, the pair creation processing program 721P is a program in the primary node 210P that has a PVOL candidate. The pair creation processing program 721S is a program in the secondary node 210S that has an SVOL candidate.

[0107] The pair forming processing program 721P sends a pre-check request to the pair forming processing program 721S (S1101). The pair forming processing program 721S receives the request (S1102) and performs predetermined pre-checks, such as checking whether a VOL102 to be paired exists and whether the information on the partner device is correct. The pair forming processing program 721S returns a response to the pre-check request (S1103). The response indicates the results of the pre-check. The pair forming processing program 721P receives the response (S1104).

[0108] If the response is a predetermined response, the pair creation processing program 721P sends a pair creation request to the pair creation processing program 721S, specifying PVOL102P as the PVOL candidate and SVOL102S as the SVOL candidate (S1105). The pair creation processing program 721S receives the request (S1106), creates a VOL pair (registers the information in the pair management table 920), and sets the pair status 926 to "COPY" (S1107). The pair creation processing program 721S starts the restore process (FIG. 13) (S1108) and returns a response to the pair creation request (S1109). The pair creation processing program 721S waits for completion of the initial copy (S1110). When the initial copy is complete (S1111: Yes), specifically, when the restore processing program 723S has completed synchronization of the data in the PVOL and the data in the SVOL, the pair creation processing program 721S sets the status 926 of the pair to "PAIR" (S1112).

[0109] The pair creation processing program 721P receives the response sent in S1109 (S1113). If the response is a specified response, the pair creation processing program 721P creates a VOL pair (registers the information in the pair management table 920) and sets the status 926 of the pair to "COPY" (S1114). If the pair has a resynchronization option (S1115: Yes), the pair creation processing program 721P sets the resynchronization option (S1116).

[0110] The pair creation processing program 721P starts the initial JNL creation processing (S1117). The pair creation processing program 721P waits for the completion of the initial copy (S1118). When all the initial JNLs have been purged (S1119: Yes), specifically when the restore processing program 723S has completed synchronization of the PVOL data and the SVOL data, the pair creation processing program 721P sets the status 926 of the pair to "PAIR" (S1120).

[0111] FIG. 12 is a diagram showing the flow of the initial JNL creation process at the primary site 201P.

[0112] When the initial JNL creation process is started, the process shown in Figure 12 is performed. In this process, data from PVOL 102P is generated as the initial JNL. The JNL is basically stored in the cache 55P, but if the cache is full, it is destaged to PJVOL 102JP, and the destaged data is purged from the cache 55P. The initial copy can be a full copy or a differential copy. If the pair status 926 is "SUSPEND," a differential copy is used to create a JNL only for the update differences while the pair is suspended. A redundancy flag and a non-volatile flag are set for the cache segment, and writing is controlled according to these flags.

[0113] If the resynchronization option is not set (S1201: No), the initial JNL creation processing program 722P targets the data in all areas of PVOL102P for creating the initial JNL (S1202).If the resynchronization option is set (S1201: Yes), the initial JNL creation processing program 722P references a difference management table (not shown) that indicates the differences between PVOL102P and SVOL102S, and targets the data in the difference area for creating the initial JNL (S1203).

[0114] The initial JNL creation processing program 722P reserves a cache segment for the initial JNL from the free queue 1003 (S1204), and obtains the VOL address 1045 (the PVOL address of the PVOL area where the data to be included in the initial JNL is located) corresponding to that segment (S1205). The initial JNL creation processing program 722P reads data from that address (S1206), creates metadata for the initial JNL (S1207), creates an initial JNL that includes the data read in S1206 and the metadata created in S1207, and sets the initial JNL as the storage target (S1208). The metadata includes, for example, the JNL ID, LBA (VOL address), transfer length, pair ID, PVOL ID, and SVOL ID.

[0115] The initial JNL creation processing program 722P sets both the redundancy flag and the non-volatile flag in the redundancy-non-volatile flag 1046 to "1" for the segment secured in S1204 (S1209). The initial JNL creation processing program 722P starts the cache storage processing (FIG. 15A) (S1210).

[0116] If the cache usage rate (for example, the ratio of the total capacity of dirty segments and clean segments to the total capacity of cache 55P) exceeds a predetermined value (S1211: Yes), the initial JNL creation processing program 722P targets the initial JNL in the cache segment for destaging (S1212) and starts the destaging process (Fig. 15B) (S1213). The initial JNL creation processing program 722P releases the cache segment containing the initial JNL destaged to PJVOL102JP, that is, makes it a free segment (S1214).

[0117] If there is an uncreated initial JNL (S1215: No), the process returns to S1204. If all initial JNLs have been created (S1215: Yes), the initial JNL creation process ends.

[0118] FIG. 13 is a diagram showing the flow of restore processing at the secondary site 201S.

[0119] When restore processing is initiated, the processing shown in Fig. 13 is performed. In this processing, the JNL from the primary site 201P is reflected in SVOL 102S. The redundancy flag and non-volatilization flag of the cache segment to which the JNL from the primary site 201P is written are each set to "0", thereby reducing the processing load. If the JNL is lost when a secondary node fails, the JNL is retransferred from the primary site 201P, and the synchronization state between the PVOL and SVOL is restored. After the JNL is reflected in SVOL 102S, a purge notification is issued to the primary site 102P to discard the JNL in question at the primary site 102P.

[0120] The restore processing program 723S waits for a fixed period of time (S1301), and then sends a JNL read request (S1302). The request may specify the ID (e.g., SEQ#) of the JNL to be read. The request is sent to the primary node 210P that has the PVOL represented by the metadata of the JNL. In response to this request, S1401 in FIG. 14A is performed.

[0121] The restore processing program 723S reserves the cache segment of the JNL to be read from the free queue 1003 (S1303).

[0122] 14A is performed, the restore processing program 723S receives a response to the JNL read request sent in S1302 (S1304). The restore processing program 723S stores the JNL included in the response in a buffer (S1305).

[0123] The restore processing program 723S sets both the redundancy flag and the non-volatile flag in the redundancy-non-volatile flag 1046 to "0" for the segment secured in S1303 (S1306). The restore processing program 723S sets the JNL stored in S1305 as a cache storage target (S1307), and starts the cache storage process (FIG. 15A) (S1308).

[0124] If the cache usage rate exceeds a predetermined value (S1309: Yes), the restore processing program 723S targets the JNL in the cache segment for destaging (S1310) and starts the destaging process (FIG. 15B) (S1311). The restore processing program 723S releases the cache segment containing the JNL destaged to SJVOL102JS (S1312).

[0125] If the determination result in S1309 is false (S1309: No), or after S1312, the restore processing program 723S reserves from the free queue 1003 a cache segment to which the data in the JNL stored in the segment in S1308 is to be copied (S1313). The restore processing program 723S sets both the redundancy flag and the non-volatile flag for the segment reserved in S1313 to "0" (S1314). The restore processing program 723S sets the data in the JNL stored in the segment in S1308 to be stored in the cache (S1315), and starts the cache store processing (FIG. 15A) (S1316).

[0126] The restore processing program 723S sets the redundancy flag to "1" and the non-volatile flag to "0" for the segment secured in S1313 (S1317). The restore processing program 723S targets the data in the segment secured in S1313 for destaging (S1318) and starts the destage processing (Fig. 15B) (S1319). The restore processing program 723S releases the cache segment containing the data destaged to SVOL102S (S1320). The restore processing program 723S updates the ID (SEQ#) of the restored (reflected) JNL (S1321) and sends a purge notification including the updated JNL ID to the primary site 201P (S1322). In response to this notification, S1451 in Fig. 14B is performed. 14B is performed, the restore processing program 723S receives a response to the notification sent in S1322 (S1323). The processing returns to S1301.

[0127] FIG. 14A is a diagram showing the flow of JNL read processing in the primary site 201P.

[0128] The JNL read processing program 724P receives a JNL read request (S1401), and determines whether there is any JNL that has not been transferred to the secondary site 201S (S1402). For example, it may be determined whether the JNL ID specified in the request is the ID of an untransferred JNL.

[0129] If the result of the determination in S1402 is true (S1402: Yes), the JNL read processing program 724P determines whether or not the untransferred JNL is cached (S1403).

[0130] If the determination result in S1403 is false (S1403: No), the JNL read processing program 724P reserves a cache segment for the JNL (S1404), and reads the JNL from PJVOL102JP to the buffer (S1405). The JNL read processing program 724P sets both the redundancy flag and the non-volatile flag for the segment reserved in S1404 to "0" (S1406). The JNL read processing program 724P sets the JNL read in S1405 to be stored in the cache (S1407), and starts the cache storage process (Fig. 15A) (S1408). The JNL read processing program 724P includes the JNL in question in the response to the JNL read request received in S1401 (S1410), and returns the response (S1412).

[0131] If the determination result of S1403 is true (S1403: Yes), the JNL read processing program 724P acquires the JNL to be transferred from the cache 55P (S1409), includes that JNL in the response to the JNL read request (S1410), and returns that response (S1412). The "JNL to be transferred" may be the JNL with the smallest JNL ID among the JNLs that have not yet been transferred.

[0132] If the determination result at S1402 is false (S1402: No), the JNL read processing program 724P includes a value indicating that there is no JNL in the response to the JNL read request (S1411), and returns this response (S1412).

[0133] FIG. 14B is a diagram showing the flow of the JNL purge process at the primary site 201P.

[0134] The JNL purge processing program 725P receives a purge notification including a JNL ID (S1451), purges all JNLs having JNL IDs up to the JNL ID indicated in the notification from PJVOL102JP, and returns a response indicating completion (S1453). Note that the purge notification may also be included as a parameter in the JNL read request.

[0135] FIG. 15A is a diagram showing the flow of the cache storage process at the secondary site 201S.

[0136] When the cache storage process is started, the process shown in Fig. 15A is performed. In this process, the JNL or data to be stored in the cache is stored in the secured cache segment, and whether or not the JNL or data is made redundant or non-volatile is controlled according to the flag.

[0137] The cache store processing program 726S stores the JNL or data in the secured cache segment (S1501).

[0138] If the redundancy flag corresponding to the segment is "1" (S1502: Yes), the cache store processing program 726S transfers the JNL or data in the segment to another secondary node 210S (S1503). In other words, if the redundancy flag corresponding to the segment is "0" (S1502: No), the cache store processing program 726S skips transferring the JNL or data in the segment to another secondary node 210S.

[0139] If the non-volatile flag corresponding to the segment is "1" (S1504: Yes), the cache storage processing program 726S writes the JNL or data in the segment to the non-volatile area 103S (S1505). In other words, if the non-volatile flag corresponding to the segment is "0" (S1504: No), the cache storage processing program 726S skips writing the JNL or data in the segment to the non-volatile area 103S.

[0140] FIG. 15B is a diagram showing the flow of destage processing at the secondary site 201S.

[0141] When the destage process is initiated, the process shown in Figure 15B is performed. In this process, JNL is written from a segment in cache 55S to SJVOL102JS, or data is written from the segment to SVOL102S. At that time, if the redundancy flag corresponding to the segment is "1", the JNL or data is made redundant in another secondary node. If the redundancy flag is "0", the JNL or data may be written to the instance store of the secondary node.

[0142] The destage processing program 727S selects the JNL or data to be destaged (S1551). Specifically, a dirty segment is selected from the dirty queue 1001.

[0143] If the redundancy flag corresponding to the segment containing the selected JNL or data is "1" (S1552: Yes), the destage processing program 727S transfers the JNL or data to another secondary node 210S (S1553) and writes the JNL or data to SJVOL102JS or SVOL102S of its own secondary node 210S (S1555).

[0144] If the non-volatile flag corresponding to the segment containing the selected JNL or data is "0" (S1552: No, S1554: Yes), the destage processing program 727S writes the JNL or data to the instance store of its own secondary node 210S (S1556). Note that S1555 may be performed instead of S1556.

[0145] FIG. 16 is a diagram showing the flow of the updated JNL creation process at the primary site 201P.

[0146] In this process, an updated JNL is created in response to a write request from the host 51. The updated JNL is transferred to the secondary site 201S by the restore process of the secondary site 201S in the same way as the initial JNL, and is reflected in SVOL102S. If the updated JNL in the primary site 201P is lost due to a node failure, the updated JNL cannot be recovered. For this reason, the data and JNL in the cache are made redundant and non-volatile.

[0147] The update JNL creation processing program 728P receives a write request (S1601) and stores the write data (data to be written) accompanying the write request in a buffer (S1602). The update JNL creation processing program 728P secures a cache segment for the write data from the free queue 1003 (S1603), sets the redundancy flag of the segment to "1", and sets the write data to be stored (S1605).

[0148] The updated JNL creation processing program 728P determines whether the attribute 915 of the VOL 102 specified in the write request is "PAIR_VOL," that is, whether the VOL 102 is a PVOL 102P (S1606).

[0149] If the determination result at S1606 is true (S1606: Yes), the update JNL creation processing program 728P determines whether the status 926 of the pair including the VOL in question is "SUSPEND" (S1607). If the determination result at S1607 is true (S1607: Yes), the update JNL creation processing program 728P updates the difference management table indicating the difference between the PVOL 102P and SVOL 102S in question, in accordance with the address of the write destination according to the write request (S1608).

[0150] If the determination result of S1607 is false (S1607: No), the updated JNL creation processing program 728P reserves a cache segment for the updated JNL (S1609). The updated JNL creation processing program 728P creates metadata for the updated JNL (S1610), sets the updated JNL to be stored in the cache (S1611), and sets the redundancy flag of the segment to "1" (S1612).

[0151] If the determination result of S1606 is false (S1606: No), after S1608 or after S1612, the updated JNL creation processing program 728P starts the cache storage processing (Fig. 15A) (S1613) and returns a response to the write request to the host 51 (S1614).

[0152] The updated JNL creation processing program 728P sets the write data as the destage target (S1615) and determines whether or not the cache usage rate exceeds a predetermined value (S1616). If the determination result of S1616 is true (S1616: Yes), the updated JNL creation processing program 728P sets the updated JNL as the destage target (S1617).

[0153] If the determination result of S1616 is false (S1616: No), or after S1617, the updated JNL creation processing program 728P starts the destage processing (Figure 15B) (S1618), releases the cache segment where the destaged updated JNL is located (S1619: Yes, S1620), and releases the cache segment where the destaged write data is located (S1621).

[0154] In the processing shown in Figure 16, for both the cache segment of the write data and the cache segment of the updated JNL, the updated JNL creation processing program 728P may set the non-volatile flag to "1" and write the write data and updated JNL in those segments to the non-volatile area 103P.

[0155] FIG. 17 is a diagram showing the flow of the pair recovery process.

[0156] In this process, the secondary site 201S detects the failure, transitions the pair state to "SUSPEND," and notifies the primary site 201P of the failure. In response to this notification, the JNL is retransferred from the primary site 201P to the secondary site 201S. The pair is restored using this retransferred JNL. For example, if the network 202 is temporarily out of service and the pair state becomes "SUSPEND," but the network 202 subsequently recovers, pair restoration processing may be performed. In this embodiment, pair restoration is performed automatically when the pair state transitions to "SUSPEND." The "failure" referred to in this paragraph may include, in addition to a network failure, a node failure, a power supply failure, or a drive failure. In response to a failure at the primary site 201P, the pair state may be set to "SUSPEND" at the primary site 201P, and this state may be notified to the secondary site 201S, and pair restoration may be performed in response to this notification.

[0157] The pair recovery processing program 729S detects a failure at the secondary site 201S (S1701) and sets the status 926 of the VOL pair affected by the failure to "SUSPEND" (S1702). The pair recovery processing program 729S determines whether the number of times the series of processes from S1704 to S1706 has been performed has exceeded the retry threshold (S1703).

[0158] If the determination result at S1703 is false (S1703: No), the pair recovery processing program 729S carries out a series of processes from S1704 to S1706. That is, the pair recovery processing program 729S notifies the primary node 210P that has the PVOL 102P that belongs to the pair of the failure of the pair whose status 926 was set to "SUSPEND" at S1702 (S1704), monitors the recovery of the pair (S1705), and determines whether the pair status has recovered to normal (S1706). If the determination result at S1705 is true (S1705: Yes), the processing of the pair recovery processing program 729S ends. If the determination result at S1705 is false (S1705: No), the processing returns to S1703.

[0159] The pair recovery processing program 729P receives notification of a pair failure from the pair recovery processing program 729S (S1751), sets a resynchronization option (S1752), and starts the pair creation processing (FIG. 11) (S1753).

[0160] Although one embodiment of the present invention has been described above, this is merely an example for explaining the present invention, and the scope of the present invention is not limited to this embodiment. The present invention can be implemented in various other forms.

[0161] The above description can be summarized as follows. The following summary may include supplementary explanations and explanations of variations of the above description. In the following description, the subject of processing is the processor 211, but more specifically, for example, the subject of processing may be the SCS 720. In the following description, for reference, the reference numerals of elements are described mainly with reference to FIG. 1.

[0162] The storage system 100 includes a plurality of nodes 210, each including a memory 212 provided with a cache 55 and a processor 211 connected to the memory 212. When data to be stored in a segment secured from the cache 55 exists in each of the plurality of nodes 210, the processor 211 determines, depending on the type of data, whether redundancy is required for the segment (transferring redundant data of the data in the segment to another node) and whether nonvolatilization is required (storing the data in the segment in the nonvolatile area 103). Based on this determination, the processor 211 controls whether redundancy is required for the data in the segment and whether nonvolatilization is required for the data in the segment. For each of the plurality of nodes 210, the nonvolatile area 103 is an area based on one or more nonvolatile media inside or outside the node 210 (nonvolatile media may be referred to as persistent storage devices, and an example of nonvolatile media is the drive 214). This allows the frequency of writing data from the cache 55 to nonvolatile media to be appropriately reduced in a storage system 100 configured with a plurality of nodes 210.

[0163] In the following description, determining the value of redundancy flag 2 is used as an example to determine whether redundancy is required, but in the present invention, determining whether redundancy is required is not limited to determining the value of redundancy flag 2. Similarly, in the following description, determining the value of non-volatilization flag 3 is used as an example to determine whether non-volatilization is required, but in the present invention, determining whether redundancy is required is not limited to determining the value of non-volatilization flag 3 (for example, information in a format other than a flag may be used). Also, in the following description, "redundant data" may be duplicate data (for example, a JNL or a duplicate of data included in the JNL) or parity data (a JNL or parity data of data included in the JNL).

[0164] The multiple nodes 210 may be multiple secondary nodes 210S constituting the secondary storage system 100S. Of the multiple secondary nodes 210, a first secondary node 210S1 may have SJVOL102JS1A, which is VOL102 in which the JNL is stored, SVOL102S1A, which forms a pair with PVOL102P1A in the primary storage system 100P, a first secondary cache 55S1, which is the cache 55 in the secondary node 210S1, and a first secondary processor (hereinafter, for convenience, designated "211S1"), which is the processor 211 in the secondary node 210S1. The JNL may include data to be written to PVOL102P1A in the primary storage system 100P and metadata including a SEQ# (sequence number) indicating the order in which the data is written. In the above-described embodiment, an example of the SEQ# is the JNL ID. The first secondary processor 211S1 may reserve a segment in which JNL is stored from the first secondary cache 55S1. If the data to be stored in the reserved segment is JNL, the first secondary processor 211S1 may set both redundancy flag 2 and non-volatilization flag 3 for that segment to "off." If both redundancy flag 2 and non-volatilization flag 3 for that segment are "off," the first secondary processor 211S1 does not need to perform redundancy, which involves writing the JNL in that segment to SJVOL102JS1A and transferring redundant data of the JNL in that segment to the second secondary node 210S2 having SVOL102S1S, which is a mirror VOL of SVOL102S1A, nor does it need to perform non-volatilization, which involves storing the JNL in that segment in the non-volatile area 103S1. This makes it possible to appropriately reduce the frequency of writing from the cache 55S to non-volatile media in asynchronous remote copying using JNL. Furthermore, even if such write frequency is reduced, if JNL or data is lost in secondary node 210S1 or 210S2, it is possible to restore the JNL or data using JNL from primary storage system 100P. Note that for each of multiple secondary nodes 210S, VOL102 (SJVOL102JS or SVOL102S) may be an area based on one or more non-volatile media inside or outside the secondary node 210S.

[0165] The first secondary processor 211S1 reserves a segment in which data within the JNL is stored from the first secondary cache 55S1, and if the data stored in the reserved segment is data within the JNL, it may set redundancy flag 2 for that segment to "on" and non-volatilization flag 3 to "off." If redundancy flag 2 is "on" and non-volatilization flag 3 is "off" for that segment, the first secondary processor 211S1 performs redundancy by writing the data in that segment to SVOL102S1A and transferring the redundant data of the data in that segment to the second secondary node 210S2, but does not perform non-volatilization by storing the data in that segment in the non-volatile area 103S1. This makes it possible to appropriately reduce the frequency of writing from the cache 55S to non-volatile media during asynchronous remote copying. The second secondary processor (hereinafter referred to as "211S2" for convenience), which is the processor 211 in the second secondary node 210S2, may reserve a segment from the second secondary cache 55S2, which is the cache 55 in that secondary node 210S2, store the redundant data from the first secondary node 210S1 in that segment, and write the data in that segment to SVOL102S1S. The second secondary processor 211S2 may set both redundancy flag 2 and non-volatilization flag 3 for that segment to "off" because the data in that segment is redundant data from another secondary node 210S.

[0166] The primary storage system 100P may be made up of a plurality of primary nodes 210P1 that are a plurality of nodes 210. Of the plurality of primary nodes 210P1, a first primary node 210P1 may have PJVOL102PJ1A that is a VOL102 in which JNL is stored, the PVOL102P1A, a first primary cache 55P1 that is a cache 55 in the primary node 210P1, and a first primary processor (hereinafter, for convenience, designated "211P1") that is a processor 211 in the primary node 210P1. The first primary processor 211P1 reserves a segment in which JNL is stored from the first primary cache 55P1, and when the data stored in the reserved segment is JNL, both the redundancy flag 2 and the non-volatilization flag 3 for the segment may be set to "on". If both the redundancy flag 2 and the non-volatile flag 3 for the segment are "on," the first primary processor 211P1 may write the JNL in the segment to PJVOL102PJ1A and transfer the redundant data of the JNL in the segment to the second primary node 210P2 having PVOL102JP1S, which is a mirror VOL of PVOL102P1A, thereby performing redundancy, and may also perform non-volatile storage by storing the JNL in the segment in non-volatile area 103P1. This increases the certainty that the JNL required for restoration in the secondary storage system 100S is present in the primary storage system 100P. For each of the multiple primary nodes 210P1, VOL102 (PJVOL102JP or PVOL102P) may be an area based on one or more non-volatile media inside or outside the primary node 210P1. In addition, the second primary processor (hereinafter, for convenience, referred to as "211P2"), which is processor 211 in the second primary node 210P2, may reserve a segment from the second primary cache 55P2, which is cache 55 in the primary node 210P2, store redundant data (JNL redundant data) from the first primary node 210P1 in the segment, and write the data in the segment to PJVOL102JP1S.The second primary processor 211P2 may set the redundancy flag 2 to "off" but the non-volatile flag 3 to "on" for the segment in question, since the data in the segment in question is redundant data from another primary node 210P. Therefore, the second primary processor 211P2 may store the data in the segment in the non-volatile area 103P2.

[0167] When the first primary processor 211P1 reserves from the first primary cache 55P1 a segment in which data from the host 51 is stored and which is to be included in the JNL, and the data stored in the reserved segment is data from the host 51 (data to be written to PVOL102JP1A), both the redundancy flag 2 and the non-volatilization flag 3 for that segment may be set to "on." When both the redundancy flag 2 and the non-volatilization flag 3 for that segment are set to "on," the first primary processor 211P1 may write the data in that segment to PVOL102P1A and perform redundancy by transferring redundant data of the data in that segment to the second primary node 210P2, and may also perform non-volatilization by storing the data in that segment in the non-volatile area 103P1. This increases the likelihood that the JNL required for restoration in the secondary storage system 100S can be re-created in the primary storage system 100P. The second primary processor 211P2 may reserve a segment from the second primary cache 55P2, store the redundant data from the first primary node 210P1 in that segment, and write the data in that segment to PVOL102P1S. The second primary processor 211P2 may set redundancy flag 2 to "off" but non-volatile flag 3 to "on" for that segment, because the data in that segment is redundant data from another primary node 210P. Therefore, the second primary processor 211P2 may store the data in that segment in non-volatile area 103P2.

[0168] If the non-volatile flag 3 for a segment is "off", the processor 211 (for example, either the secondary processor 211S or the primary processor 211P) may write the data in the segment to the instance store 65 in the node 210 that has the processor 211. The instance store 65 may be an area based on a volatile storage medium.

[0169] After data is written to SVOL102S1A (and SVOL102S1S), the first secondary processor 211S1 may update the SEQ# of the reflected JNL and notify the primary storage system 100P of the updated SEQ#. The primary storage system 100P may purge from the primary storage system 100P any JNL having a SEQ# up to the SEQ# notified by the first secondary processor 211S1. In this way, the JNL can be saved in the primary storage system 100P until it is reflected in SVOL102S1A (and SVOL102S1S).

[0170] When the first secondary processor 211S1 detects that a failure has occurred in the pair, it may notify the primary storage system 100P of the failure in the pair. The first secondary node 210S1 receives a JNL containing data representing the difference between SVOL102S1A and PVOL102P1A from the primary storage system 100P that has received the notification, and the first secondary processor 211S1 may reserve a segment from the first secondary cache 55S1 and store the JNL in that segment. As a result, the above-mentioned processing is performed, and therefore recovery from the failure is performed automatically. For example, the pair of SVOL102S1A and PVOL102P1A may be formed in response to a pair formation request from the primary storage system 100P. In response to the above-mentioned failure notification, the first secondary node 210S1 may receive a pair formation request from the primary storage system 100P, and in the processing performed in response to the pair formation request, the first secondary node 210S1 may receive a JNL including differential data from the primary storage system 100P. In other words, when a pair failure is detected, recovery from the failure can be performed automatically by running the pair formation processing again. [Explanation of symbols]

[0171] 201 sites 210 nodes

Claims

1. a plurality of nodes each including a memory in which a cache is provided and a processor connected to the memory; In each of the plurality of nodes, when there is data to be stored in the segment allocated from the cache, a processor Depending on the type of data, a determination is made as to whether redundancy is required for the segment, which is to transfer redundant data of the data in the segment to another node, and whether non-volatilization is required, which is to store the data in the segment in a non-volatile area; Based on the determination, control is performed to determine whether or not redundancy is to be implemented for the data in the segment and whether or not non-volatilization of the data in the segment is to be implemented; For each of the plurality of nodes, the non-volatile area is an area based on one or more non-volatile media within or outside the node; Storage system.

2. the plurality of nodes are a plurality of secondary nodes constituting a secondary storage system, A first sub-node among the plurality of sub-nodes is a secondary journal volume that stores a journal containing data to be written to a primary volume in a primary storage system and metadata including a sequence number indicating the order in which the data is written; a secondary volume paired with a primary volume in a primary storage system; a first secondary cache, which is a cache in the secondary node; a first secondary processor that is a processor in the secondary node; and The first co-processor: Reserving a segment in which a journal is stored from the first secondary cache; If the data stored in the reserved segment is a journal, it is determined that redundancy is not required for the segment, and that non-volatilization is also not required; If neither redundancy nor non-volatilization is required for the segment, neither redundancy, which is writing the journal in the segment to the secondary journal volume and transferring the redundant data of the journal in the segment to a second secondary node having a secondary mirror volume that is a mirror volume of the secondary volume, nor non-volatilization, which is storing the journal in the segment in a non-volatile area, is performed. For each of the plurality of secondary nodes, a volume is an area based on one or more non-volatile media within or outside the secondary node. The storage system according to claim 1 .

3. The first co-processor: a segment in which data in the journal is stored is reserved from the first secondary cache; If the data stored in the reserved segment is data in a journal, it is determined that redundancy is required for the segment and that non-volatilization is not required; If redundancy is required for the segment but non-volatilization is not required, redundancy is performed by writing the data in the segment to the secondary volume and transferring the redundant data of the data in the segment to the second secondary node, and non-volatilization is not performed by storing the data in the segment in a non-volatile area. The storage system according to claim 2 .

4. The primary storage system is configured with a plurality of primary nodes, a first primary node of the plurality of primary nodes has a primary journal volume that is a volume in which a journal is stored, the primary volume, a first primary cache that is a cache in the primary node, and a first primary processor that is a processor in the primary node; The first main processor a segment in which a journal is stored is secured from the first primary cache; If the data stored in the reserved segment is a journal, it is determined that the segment needs redundancy and also needs to be made nonvolatile; If both redundancy and non-volatilization are required for the segment, the journal in the segment is written to the primary journal volume and the redundant data of the journal in the segment is transferred to a second primary node having a primary mirror volume that is a mirror volume of the primary volume, thereby performing both redundancy and non-volatilization, which is storing the journal in the segment in a non-volatile area. For each of the plurality of primary nodes, a volume is an area based on one or more non-volatile media inside or outside the secondary node. The storage system according to claim 2 .

5. The first main processor Reserving from the first primary cache a segment in which data to be included in a journal and data from a host is stored; If the data to be stored in the reserved segment is data from the host, determine that the segment needs redundancy and also needs non-volatilization; If both redundancy and non-volatilization are required for the segment, the data in the segment is written to the primary volume and redundant data of the data in the segment is transferred to the second primary node, thereby performing both redundancy and non-volatilization, which is storing the data in the segment in a non-volatile area. The storage system according to claim 4 .

6. If the processor does not require non-volatilization for the segment, it writes the data in the segment to an instance store in the node that has the processor. The storage system according to claim 1 .

7. The redundant data is duplicated data or parity data. The storage system according to claim 1 .

8. the first secondary processor updates the sequence number of the journal that has been reflected after the data has been written to the secondary volume, and notifies the primary storage system of the updated sequence number; the primary storage system purges from the primary storage system journals having sequence numbers up to the sequence number notified by the first secondary processor; The storage system according to claim 2 .

9. When the first secondary processor detects that a failure has occurred in the pair, it notifies the primary storage system of the failure in the pair; The first secondary node receives a journal including data representing the difference between the secondary volume and the primary volume from the primary storage system that has received the notification, and the first secondary processor secures a segment from the first secondary cache and stores the journal in the segment. The storage system according to claim 2 .

10. The pair formation between the secondary volume and the primary volume is performed in response to a pair formation request from the primary storage system, In response to the notification, the first secondary node receives a pair formation request from the primary storage system, and in a process performed in response to the pair formation request, the first secondary node receives a journal including the data as the difference from the primary storage system. The storage system according to claim 9.

11. In each of the multiple nodes constituting the storage system, when there is data to be stored in a segment secured from the cache, the node: Depending on the type of data, a redundancy-non-volatilization decision is made for the segment, which is a decision as to whether redundancy is required, i.e., transferring redundant data of the data in the segment to another node, and whether non-volatilization is required, i.e., storing the data in the segment in a non-volatile area; Based on the determination, whether or not to implement redundancy for the data in the segment and whether or not to make the data in the segment nonvolatile are controlled. Storage control method.

Images