Management apparatus, management method, and storage system

Abstract

To provide a management apparatus, a management method, and a storage system capable of stably operating remote copies even in an environment where the line speed and the amount of data written from a host are uncertain.SOLUTION: In a management apparatus that comprises a processor and manages a plurality of storage apparatuses communicating with each other via a network and each having a volume, remote copies in which data is transferred from replication source volumes to replication destination volumes among a plurality of volumes via a network for replication are set for a plurality of combinations of the replication source volumes and the replication destination volumes. The management apparatus monitors a replication delay time, which is the time required for replication of the remote copy, and controls so as to limit the transfer amount of other remote copies based on the replication delay time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a management apparatus, a management method, and a storage system, and more particularly to a management apparatus and the like that can appropriately manage replication delay time when performing remote copying. [Background technology]

[0002] In storage systems that require high reliability, data copies are often prepared in remote locations to prepare for storage device failures or natural disasters. Some storage devices have a function for replicating data to a remote location (also called a remote copy function or remote replication function). In this case, there are two replication methods, generally known as the journal method and the snapshot method.

[0003] Each of these replication methods has its advantages and disadvantages, and they are used according to the requirements. However, a common problem is that the communication line speed must be sufficient to transfer data to a remote location in response to the amount of data written from the host. If the communication line speed is insufficient, the amount of data that is not replicated in the remote storage device increases, increasing the amount of data that will be lost in the event of a failure, making it impossible to maintain the RPO (Recovery Point Objective). Also, with the journal method, the journal area can become full and remote copy operation cannot be maintained. In such cases, remote copy operation becomes unfeasible.

[0004] Patent Document 1 discloses that a storage system maintains a journal of multiple journal entries and at least one snapshot of one or more data volumes. In this storage system, a unique sequence number is assigned to each journal and snapshot in the order of their occurrence, making it easy to find the journal to be applied to the snapshot.

[0005] Patent Document 2 discloses a storage system connected to another storage system via a network. This storage system includes a memory, a processor that executes a program stored in the memory, and a storage device. By executing the program, the processor provides a virtual volume, generates multiple snapshots of the virtual volume at different times, stores a base snapshot, which is a reference snapshot of the multiple snapshots, in the storage device, and transmits and stores difference data indicating the difference between a snapshot generated after the base snapshot and a snapshot generated immediately before the base snapshot, to the other storage system. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-18738 [Patent Document 2] Japanese Patent Application Publication No. 2023-1471 Summary of the Invention [Problem to be solved by the invention]

[0007] Traditionally, remote copy was performed between on-premises storage devices, but in recent years, hybrid cloud configurations have become more common, with the goal of reducing operational costs, in which the destination storage device is located in a public cloud. In remote copy configurations between on-premises devices, it was common to use highly stable dedicated lines as the communication line between storage devices. However, hybrid cloud configurations, which aim to reduce operational costs, often use internet lines, which have less stable line speeds than before. Furthermore, with the emphasis on agility in software development these days, it is difficult to accurately estimate in advance the amount of data that an application (host) will write to a storage device. These environmental changes have made it more difficult to operate remote copy stably in recent years. An object of the present invention is to provide a management device, a management method, and a storage system that can stably operate remote copy even in an environment where the line speed and the amount of data written from the host are uncertain. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention provides a management device that has a processor, communicates with each other via a network, and manages multiple storage devices, each having its own volume, in which remote copies, in which data is transferred via the network from a source volume to a destination volume among the multiple volumes, are set for multiple combinations of source volumes and destination volumes, and the management device monitors the replication delay time, which is the time required for remote copy replication, and controls to limit the transfer volume of other remote copies based on the replication delay time.In this case, a management device can be provided that can operate remote copies stably even in an environment where line speeds and the amount of data written from hosts are uncertain.

[0009] Here, for example, target information is provided that includes a target value for replication delay time, which is a performance target for remote copying, and the transfer volume of other remote copies that affect the remote copying is limited so that the remote copying satisfies the replication delay time of the target information. In this case, remote copying can be operated stably while satisfying the required performance. Furthermore, for example, the source volume accepts a write request and stores data, the data stored in the source volume is replicated to the destination volume by remote copy, and the management device restricts the amount of data transferred by other remote copies by restricting data writing to the source volume related to other remote copies. In this case, the replication delay time can be recovered. Furthermore, for example, the target information includes the write amount that the source volume can accept, and the management device limits the transfer amount of other remote copies based on the write amount of the target information and the replication latency time. In this case, by limiting the write amount, it is possible to indirectly limit the transfer speed from the source volume to the destination volume, and to recover the replication latency time. Furthermore, for example, the source volume accepts a write request and stores data, the data stored in the source volume is replicated to the destination volume by remote copy, the target information includes the amount of write data accepted by the source volume, and the management device improves the replication latency of the remote copy by limiting the writing of data to the source volume related to the remote copy based on the write amount and replication latency of the target information. In this case, remote copy can be operated stably. Furthermore, for example, when multiple other remote copies are candidates for transfer volume restrictions, the other remote copies for which transfer restrictions are to be applied are determined based on the difference between the performance information of the other remote copies and their target information. In this case, it is possible to more easily determine which remote copy is most deficient in required performance. For example, when a new remote copy is set for a source volume in which data is stored, an initial copy is performed to copy the stored data, and if multiple other remote copies become candidates for transfer rate restrictions, the management device stops the other remote copies whose operating status is the initial copy. In this case, resources for the initial copy, which requires a long time, can be allocated to remote copies whose replication delay time does not satisfy a predetermined range. For example, for multiple remote copies, at least one of the storage device where the source volume is located and the storage device where the destination volume is located is grouped, and another remote copy for which the transfer amount is limited is selected from the same group as the remote copy that requires the replication delay time. In this case, another remote copy can be selected from among remote copies that are likely to affect each other. Furthermore, for example, the transfer volume of a remote copy is limited so that the replication delay times of multiple remote copies in the same group are roughly the same. In this case, the transfer volume can be adjusted between remote copies that tend to affect each other. Furthermore, for example, another remote copy for which transfer is to be restricted is selected taking into consideration the journal usage rate of the remote copy using the journal method. In this case, even if the remote copy uses the journal method, the remote copy can be operated stably.

[0010] The present invention also provides a management method in which a management device having a processor manages multiple storage devices that communicate with each other via a network and each have their own volumes, in which remote copies are set for multiple combinations of source volumes and destination volumes, in which data is transferred and replicated from a source volume to a destination volume via the network, and the management device monitors the replication delay time, which is the time required for the remote copy replication, and controls to limit the transfer volume of other remote copies based on the replication delay time.In this case, a management method can be provided that allows remote copies to be operated stably even in an environment where line speeds and the amount of data written from hosts are uncertain.

[0011] Furthermore, the present invention provides a storage system comprising a plurality of storage devices that communicate with each other via a network, each having a volume, and a management device that has a processor and manages the plurality of storage devices, in which remote copies are set for a plurality of combinations of source volumes and destination volumes, in which data is transferred via the network from a source volume of the plurality of volumes to a destination volume, and the management device monitors the replication delay time, which is the time required for the remote copy replication, and controls to limit the transfer volume of other remote copies based on the replication delay time.In this case, a storage system can be provided that can operate remote copies stably even in an environment where line speeds and the amount of data written from hosts are uncertain. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide a management device, a management method, and a storage system that can stably operate remote copy even in an environment where the line speed and the amount of data written from the host are uncertain. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing the overall configuration and schematic operation of a storage system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of a storage system according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a management system according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of storage device information according to the present embodiment. [Figure 5] FIG. 2 is a diagram showing an example of the configuration of remote copy configuration information in this embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of the configuration of throttling setting information in the present embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of the configuration of service level target information according to the present embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of the configuration of remote copy performance information according to this embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of the configuration of service level target excess / shortage rate information in the present embodiment. [Figure 10] FIG. 2 is a diagram showing the flow of processing in which a remote copy establishment unit of a management system establishes a remote copy in response to a request from a user of a storage device, as shown in steps S1 to S3 in FIG. [Figure 11] FIG. 2 is a diagram showing a flow of processing shown in steps S4 to S5 in FIG. 1, in which a device monitoring unit of the management system monitors each storage device. [Figure 12]This figure shows the process flow for adjusting the operation of the replication process in each storage device in order for the service level guarantee unit of the management system to guarantee the service level target set for each remote copy, shown in steps S6 to S7 in Figure 1. [Figure 13] FIG. 10 is a diagram showing a processing flow for identifying a replication group that is a target for adjusting throttling settings in a service level guarantee unit. [Figure 14] FIG. 10 is a diagram showing a processing flow for determining adjustment contents of throttling settings in a service level guarantee unit. [Figure 15] FIG. 10 is a diagram showing a processing flow for calculating the excess / shortage rate with respect to the service level target of remote copy in the service level guarantee unit. [Figure 16] FIG. 10 is a diagram illustrating an example of a throttling adjustment process in a service level guarantee unit. [Figure 17] FIG. 10 is a diagram showing an example of changes in replication delay time when remote copy is managed by the management system of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the following description and drawings are merely examples for explaining the present invention, and have been omitted or simplified as appropriate for clarity of explanation, and do not limit the technical scope of the present invention. In addition, in the following explanation, when explaining the contents of various information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these can be used interchangeably. Furthermore, in the following explanation, the "program" may be used as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit)) to perform the specified processing while appropriately using storage resources (e.g., memory) and interface devices (e.g., communication devices), the processor may also be used as the subject in the explanation. Similarly, the subject of the processing performed by executing a program may be, for example, a controller, device, system, computer, node, storage device, server, client, or host that has a processor. Furthermore, part or all of the program may be processed using hardware circuits. The various programs may be installed on each computer via a program distribution server or storage media. In the following description, two or more programs may be realized as one program, and conversely, one program may be realized as two or more programs.

[0015] <Overall Description of Storage System 0100> First, an overview of the embodiment will be described with reference to Fig. 1. In order to simplify the overall description of the embodiment, some details may be omitted. Details of each component and each process will be described later. FIG. 1 is a diagram showing the overall configuration and general operation of a storage system 0100 according to this embodiment. 1 is composed of a management system 0110, a plurality of storage devices 0120, and a plurality of hosts 0130. In addition, although not constituting the storage system 0100, a user 0140 is also shown in the figure.

[0016] The management system 0110 is an example of a management device, and is a computer device that communicates with each other via a network and manages multiple storage devices 0120, each having a volume. The management system 0110 performs tasks such as establishing remote copies and managing service level objectives (performance required when performing remote copies) on behalf of users 0140 of the storage devices 0120. The management system 0110 is, for example, a server computer that operates on a cloud, and provides users 0140 with a service of performing remote copies.

[0017] The storage device 0120 is a computer device that provides a logical storage area called a volume to the host 0130. The illustrated storage device 0120 is made up of storage devices 0120A to 0120C. The storage device 0120A has a function for replicating volume data to other storage devices 0120B to 0120C installed in remote locations. In other words, the source volume accepts a write request and stores the data, and the data stored in the source volume is replicated to the destination volume by remote copy.

[0018] The host 0130 is a computer device that sends data read or write requests to volumes provided by the storage device 0120 in response to requests from application programs running on the host 0130. The illustrated host 0130 is made up of hosts 0130A to 0130D. In the example of Fig. 1, the storage device 0120A provides volumes PVOL0121A to 0121D to the hosts 0130A to 0130D, respectively.

[0019] The storage system 0100 shown in the figure can also be said to be configured for remote copying, in which data is transferred and replicated from a source volume of multiple volumes to a destination volume via a network, with multiple combinations of source volumes and destination volumes.

[0020] As described above, there are two types of remote copy methods: journal method and snapshot method. In the case of Fig. 1, remote copy using the journal method is configured for volumes PVOL0121A-0121B, and remote copy using the snapshot method is configured for volumes PVOL0121C-0121D. These remote copy methods are existing technologies, so detailed explanations will be omitted in this embodiment, but an overview of the operations and problems arising from the operations will be briefly explained.

[0021] The outline of the operation of remote copy using the journal method is as follows. First, the hosts 0130A to 0130B write data to the copy source volumes PVOL0121A to 0121B of the copy source storage device 0120A. As a result, the write request is stored as a log in the master journal areas MJNL0122A to 0122B. Thereafter, the log is transferred between the duplication source storage device 0120A and the duplication destination storage device 0120B, and the log is stored in the restore journal areas RJNL0124A-0124B. Finally, in the replication destination storage device 0120B, the logs are extracted from the restore journal areas RJNL0124A-0124B and reflected in the replication destination volumes SVOL0125A-0125B, thereby replicating data between PVOL0121A-0121B and SVOL0125A-0125B. When configuring remote copy later for a volume in which data has already been written, all data in the source volumes PVOL0121A-0121B is first transferred (initial copy) to the destination volumes SVOL0125A-0125B, and then the operation switches to the above. In this case, the initial copy is a copy of the data stored in a source volume in which data is stored when a new remote copy is set.

[0022] Because the above processes such as log generation, transfer, and reflection are all performed asynchronously, there is a certain amount of delay before the data written by hosts 0130A-0130B to source volumes PVOL0121A-0121B is reflected in destination volumes SVOL0125A-0125B. If the speed at which logs are transferred between the source storage device 0120A and the destination storage device 0120B is insufficient compared to the amount (speed) at which hosts 0130A-0130B write data to the source volumes PVOL0121A-0121B, the amount of logs remaining in the master journal areas MJNL0122A-0122B will increase, resulting in an increased delay until the logs are reflected in the destination volumes SVOL0125A-0125B. During this delay, data is not replicated, and if a failure occurs during that time, the data will be lost. Strictly speaking, data that has already been transferred to the restore journal area RJNL0124A-0124B will not be lost, but this will not be taken into consideration here. Furthermore, since the journal areas indicated by the master journal areas MJNL0122A-0122B and the restore journal areas RJNL0124A-0124B have limited capacity, if the log transfer speed is insufficient for a long period of time, the journal area becomes full and new logs can no longer be stored, and remote copy operation cannot be maintained.

[0023] For this reason, in journal-based remote copying, the system must be designed and operated appropriately so that replication delay time is less than RPO (Recovery Point Objective). Specifically, the system is designed and operated in such a way that, depending on the available transfer speed, the amount of IO (Input / Output) received from hosts 0130A to 0130B is limited to suppress the amount of logs generated, and in an environment where multiple remote copies are configured, the transfer speed for each remote copy is adjusted within a range that does not cause the jar area to become full. However, as mentioned above, the actual transfer speed is unstable in a hybrid cloud configuration, making such operation unrealistic. Furthermore, if the amount of I / O received from the host 0130 is limited, the operation of the host 0130 is affected, and the extent of this effect depends on the unstable transfer speed, which is a problem.

[0024] Next, the operation of remote copying using the snapshot method is outlined below. First, the storage device 0120A periodically acquires snapshots Snapshots 0123C to 0123D of the copy source volumes PVOLs 0121C to 0121D. Snapshots 0123C to 0123D are virtual copies of PVOLs 0121C to 0121D at the time the snapshot was taken, and are data objects that store only the differences from the previously taken snapshot as data entities. After the snapshot is acquired, the storage device 0120A transfers (the data objects of) Snapshots 0123C to 0123D to the storage device 0120C, which is the copy destination. In snapshot-based remote copying, the data replication process is now complete.

[0025] If a failure occurs in the source storage device 0120A and it becomes necessary to restore the data in the source volume, the source volumes PVOL0121C-0121D can be restored by applying Snapshots 0123C-0123D stored in the destination storage device 0120C from the beginning. As with the journal method, when configuring a remote copy later for a volume to which data has already been written, first Snapshots 0123C-0123D containing all the data of the source volumes PVOL0121C-0121D as differentials are transferred to the destination storage device 0120C (initial copy), and then the operation is switched to the above.

[0026] As described above, in remote copy using the snapshot method, data is replicated by periodically acquiring and transferring snapshots. During the period between when a snapshot is taken in a certain cycle and when the transfer of the snapshot in the next cycle is completed (replication delay time), data is not replicated, and if a failure occurs during this time, the data will be lost. If the amount (speed) of data written by the hosts 0130C to 0130D to the replication source volumes PVOL0121C to 0121D increases, the amount of differential data contained in the snapshots Snapshot0123C to 0123D increases accordingly, and the replication delay time increases.

[0027] For this reason, even in remote copy using the snapshot method, it is necessary to properly design and operate the system so that replication delay time < RPO (Recovery Point Objective). Specifically, the system is designed and operated in such a way that, depending on the available transfer speed, the amount of IO received from hosts 0130C to 0130D is limited to suppress the amount of differential generation, or in an environment where multiple remote copies are configured, the transfer speed for each remote copy is adjusted. However, as mentioned above, the actual transfer speed is unstable in a hybrid cloud configuration, making such operation unrealistic. Furthermore, if the amount of I / O received from the host 0130 is limited, the operation of the host 0130 is affected, and the extent of this effect depends on the unstable transfer speed, which is a problem.

[0028] So far, we have described the operation of the remote copy function and the resulting difficulties in system design and operation. In this embodiment, the management system 0110 performs the following processing to solve the above-mentioned problems.

[0029] The management system 0110 is a system that performs remote copy operations on behalf of users 0140 of the storage device 0120, taking into consideration the replication delay time and the usage rate of the journal area. Specifically, the management system 0110 collects performance information (remote copy performance information 0315) related to remote copying, such as replication delay time, from the storage device 0120. The management system 0110 then dynamically adjusts throttling for remote copying so as to maintain the RPO set by the user 0140 as a service level objective. At this time, the user 0140 sets service level targets not only for the RPO but also for performance items that affect the operation of the host 0130, such as the minimum IO volume, and the management system 0110 makes adjustments to ensure that each service level target is met, thereby achieving both PRO maintenance and stable operation of the host 0130.

[0030] First, the operation of the management system 0110 will be outlined with reference to FIG. The management system 0110 includes a control program 0320 that includes a remote copy construction module 0321 , a service level guarantee module 0323 , and a device monitoring module 0322 . The management information 0310 also includes service level target information 0314 , remote copy configuration information 0312 , and remote copy performance information 0315 . The operation of the management system 0110 in this embodiment will be outlined below with reference to steps S1 to S7.

[0031] Steps S1 to S3 are processing in which the management system 0110 establishes a remote copy in response to a request from the user 0140 of the storage device 0120. Conventionally, when constructing a remote copy, the user 0140 provides the storage device 0120 with information about the source and destination volumes (information about the source and destination volumes, information about the journal area to be used, etc.). On the other hand, in this embodiment, the user 0140 instructs the management system 0110 to construct a remote copy system after providing the management system 0110 with information on the service level target for the remote copy to be constructed in addition to the above information (step S1). The service level target is the performance required when performing a remote copy. Specifically, the service level target is the RPO and the minimum IO volume (lower limit IO volume) that needs to be issued to the source volumes (PVOLs 0121A to 0121D, etc.) for the host 0130 to operate stably. The management system 0110, which has received an instruction to establish a remote copy from the user 0140, stores the information provided by the user 0140 in the service level target information 0314 and the remote copy configuration information 0312 (step S2). Furthermore, the management system 0110 instructs each storage apparatus 0120 to establish a remote copy in accordance with the content of the instruction (step S3).

[0032] Steps S4 and S5 are processes in which the management system 0110 monitors each storage device 0120. The management system 0110 periodically communicates with each storage device 0120 and collects performance information related to remote copying (step S4). The management system 0110 then stores the collected information in the remote copy performance information 0315 (step S5). Specifically, the performance information includes the amount of IO issued by the host 0130 to the source volume, the replication delay time, and the like.

[0033] Steps S6 and S7 are processes in which the management system 0110 adjusts the operation of the replication process in each storage apparatus 0120 in order to guarantee the service level objectives set for each remote copy. The management system 0110 refers to the remote copy performance information 0315 and the service level target information 0314 (step S6). Then, when the replication delay time exceeds the RPO, the management system 0110 sets throttling for various processes related to remote copying so as to reduce the amount of data transferred from the source storage device 0120 to the destination storage device 0120 (step S7). The throttling settings include limits on the amount of IO that the host 0130 can issue to the source volume, and limits on the transfer speed of logs, snapshots, etc., and the service level guarantee unit 0323 makes optimal settings to prevent violations of service level targets (RPO and minimum IO amount).

[0034] An example of the configuration of the storage system 0100 in this embodiment has been explained above, but the above explanation has been limited to the parts relating to the essence of this embodiment, and other components may be included as necessary.

[0035] FIG. 2 is a diagram showing the hardware configuration of the storage system 0100 in this embodiment. The storage system 0100 comprises one management system 0110 , two or more storage devices 0120 (one or more source {storage devices} and one or more destination storage devices 0120 ), and one or more hosts 0130 . The various components that make up the storage system 0100 do not need to be installed in the same location; for example, the storage device 0120 that is the source of a remote copy and the storage device 0120 that is the destination of the remote copy may be installed in different data centers, and the management system 0110 may be installed in the cloud. Similarly, the multiple components that make up the storage system 0100 may be physically configured as a single device. For example, a configuration may be adopted in which the functions of the management system 0110, storage device 0120, and host 0130 are all present on a single physical server, as in a Hyper Converged Infrastructure.

[0036] The management system 0110, storage device 0120, and host 0130 are each computer devices comprised of a CPU 0201, memory 0202, storage device 0203, and communication port 0204, but the exact configuration is not important as long as they can perform their respective roles described below. The management system 0110, storage device 0120, and host 0130 may be equipped with multiple units of each of these components, or may be equipped with components other than those listed above. For example, the storage device 0203 may be equipped with multiple hard disk drives for redundancy, or an accelerator such as a GPU may be installed to perform some of the processing performed by the CPU 0201. Furthermore, each component may not be physical hardware, but may be virtual hardware based on virtualization technologies such as virtual machines and containers. Similarly, they may be configured using services that execute programs without regard to the actual computing environment, such as Function-as-a-Service and Serverless Computing. Furthermore, the storage device 0120, which is the replication destination in the snapshot method, may be configured using a cloud object storage service (such as AWS S3).

[0037] Each of the components that make up the storage system 0100 is connected to a network 0200, and can communicate with each other. The network 0200 is realized by any communication line such as Ethernet, InfiniBand, or optical fiber, or a combination thereof. In addition, the data center may include not only a closed local area network (LAN) within itself, but also a wide area network (WAN) such as the Internet, and a virtual network within a computer. Although not shown, network devices such as a network switch, router, and gateway may also be included as needed. Furthermore, while the diagram shows a configuration in which all components are connected to a single network 0200, a dedicated network used between specific components may also be included. For example, a storage area network (SAN) using a fiber channel may also be included to provide a high-speed connection between the storage device 0120 and the host 0130.

[0038] <Detailed explanation of the configuration of the management system 0110> Next, the management system 0110 will be described in detail. FIG. 3 is a diagram showing an example of the functional configuration of the management system 0110 in this embodiment. The management system 0110 includes, as management information 0310, storage device information 0311, remote copy configuration information 0312, throttling setting information 0313, service level target information 0314, remote copy performance information 0315, and service level target excess / deficiency rate information 0316. The management system 0110 also includes a remote copy construction unit 0321 , a device monitoring unit 0322 , and a service level guarantee unit 0323 as control programs 0320 .

[0039] The management information 0310 and the control program 0320 are stored in some storage device 0203 or memory 0202 depending on the computing environment in which the management system 0110 operates. The control program 0320 is executed using the CPU 0201 and the memory 0202 , and reads and writes the management information 0310 according to the program, and communicates with the storage device 0120 using the communication port 0204 .

[0040] The storage device information 0311 is information for managing information on the storage device 0120 that is the object of management by the management system 0110 . The remote copy configuration information 0312 is information for managing the configuration of the remote copy managed by the management system 0110 . The throttling setting information 0313 is information for managing the content of throttling set for each remote copy in order for the management system 0110 to maintain the service level target. The service level target information 0314 is information for managing the contents of the service level target for each remote copy that the user 0140 requests from the management system 0110 . The remote copy performance information 0315 is information for managing performance information relating to replication processing, which is collected by the management system 0110 from each storage apparatus 0120 . The service level target excess / deficiency rate information 0316 is information used by the management system 0110 to manage the degree of achievement of the service level target set for each remote copy.

[0041] The remote copy construction unit 0321 is a program that allows the management system 0110 to construct a remote copy in accordance with a request from the user 0140. As will be described in detail later, the remote copy construction unit 0321 functions as a performance information acquisition unit that acquires performance information as information on the performance required when performing a remote copy. The device monitoring unit 0322 is a program that monitors each storage device 0120 under the management of the management system 0110 and collects performance information related to replication processing. As will be described in detail later, the device monitoring unit 0322 monitors the replication delay time, which is the time required for remote copy replication. The service level guarantee unit 0323 is a program that allows the management system 0110 to maintain a service level target for each remote copy. As will be described in detail later, the service level guarantee unit 0323 controls the transfer volume of other remote copies to be limited based on the replication delay time.

[0042] FIG. 4 is a diagram showing an example of the configuration of the storage device information 0311 in this embodiment. The storage device information 0311 is information for managing information on the storage device 0120 that is the object of management by the management system 0110 . Each entry of the storage device information 0311 includes a device ID 0400 , a device name 0401 , a storage type 0402 , a model 0403 , and connection information 0404 .

[0043] The device ID 0400 is information that allows the management system 0110 to uniquely identify the storage device 0120. Note that in Fig. 4, the device ID 0400 is a sequential number, but it does not have to be a sequential number and may be an ID that uses characters other than numbers. The device name 0401 is information that allows the user 0140 to easily identify the storage device 0120 . The storage type 0402 and the model 0403 are information indicating the type of the storage device 0120. These are information for determining the functional differences between the storage devices 0120, such as when using object storage on the cloud as the storage device 0120 of the replication destination. The connection information 0404 is information used when the management system 0110 communicates with the storage device 0120. In Fig. 4, the connection information 0404 indicates an IP (IP address) and an AccessToken (authentication information), but may include other information as necessary.

[0044] In the example of Figure 4, the storage device 0120 whose device ID 0400 is "0" has a device name 0401 of "Primary-Str", a storage type 0402 of "Block", a model 0403 of "Block-Model-X", an IP address of the connection information 0404 of "1.1.1.1", and an AccessToken of "abc...".

[0045] A new entry is added to the storage device information 0311 when a new storage device 0120 is added to the management targets of the management system 0110 .

[0046] FIG. 5 is a diagram showing an example of the configuration of the remote copy configuration information 0312 in this embodiment. The remote copy configuration information 0312 is information for managing the configuration of the remote copy managed by the management system 0110 . Each entry of the remote copy configuration information 0312 includes a replication ID 0500 , a replication name 0501 , a replication mode 0502 , replication source information 0503 , replication destination information 0504 , and a replication group ID 0505 .

[0047] The replication ID 0500 is information that allows the management system 0110 to uniquely identify the remote copy configuration. Note that, although the replication ID 0500 is a sequential number in Fig. 5, it does not have to be a sequential number and may be an ID that uses characters other than numbers. The replication name 0501 is information that allows the user 0140 to easily identify the remote copy configuration. The replication mode 0502 is information indicating the replication processing method, such as the journal method or snapshot method described above. The replication source information 0503 and the replication destination information 0504 are information that respectively indicate the replication source and replication destination in a remote copy configuration. In Fig. 5, the replication source information 0503 and the replication destination information 0504 include a device ID 0400, a VOL ID (ID of the volume to be replicated), and, in the case of the journal method only, a JNL ID (ID of the journal area to be used), but other information may also be included as necessary. For example, in a snapshot-based remote copy, if object storage is used as the replication destination storage device 0120, identification information for the bucket where the object is stored may also be included. The replication group ID 0505 is information for identifying groups when remote copies that share a bottleneck that causes a deterioration in RPO are grouped together. Grouping of remote copies will be described later.

[0048] In the example of Figure 5, the remote copy with replication ID 0500 of "0" has replication name 0501 of "App-A", replication mode 0502 of "Journal" (journal method), replication source device ID 0400 of "0", replication source volume ID of "100", journal area ID used at the replication source of "10", replication destination device ID 0400 of "1", replication destination volume ID of "200", and journal area ID used at the replication destination of "20". It also shows that the remote copy with replication ID 0500 of "0" and the remote copy with replication ID 0500 of "1" each share a bottleneck that is causing a deterioration in RPO.

[0049] A new entry is added to the remote copy configuration information 0312 when the management system 0110 establishes a new remote copy in response to a request from the user 0140 .

[0050] FIG. 6 is a diagram showing an example of the configuration of the throttling setting information 0313 in this embodiment. The throttling setting information 0313 is information for managing the content of throttling set for each remote copy in order for the management system 0110 to maintain the service level target. Each entry in the throttling setting information 0313 includes a replication ID 0500 , a host IO limit 0600 , an inter-device transfer speed limit 0601 , and an initial copy operation limit 0602 .

[0051] The replication ID 0500 is information that allows the management system 0110 to uniquely identify the remote copy configuration, and is information that is used to associate the information in the remote copy configuration information 0312 with the information in the throttling setting information 0313 . The host IO limit 0600 is information indicating the limit content regarding the amount of IO that the host 0130 can issue, which is set for the source volume of the remote copy. The inter-device transfer speed limit 0601 indicates the limit on the transfer speed between the source and destination storage devices 0120 set for the remote copy. The initial copy operation restriction 0602 indicates the restriction content regarding the initial copy that is set for the remote copy.

[0052] In the example of Figure 6, the remote copies with replication ID 0500 of "0" and "2" have no restrictions, while the remote copy with replication ID 0500 of "1" has a setting that limits the amount of IO that the host 0130 can issue to "7.5k IOPS." The remote copy with replication ID 0500 of "3" has a setting that limits the transfer speed between the storage device 0120 to "20 MB / s," and the remote copy with replication ID 0500 of "4" has a restriction that stops the initial copy.

[0053] A new entry is added to the throttling setting information 0313 when the management system 0110 creates a new remote copy in response to a request from the user 0140 .

[0054] The management system 0110 shown in this embodiment uses three limits as items to be adjusted in order to maintain the service level target: a limit on the IO amount issued by the host 0130, a limit on the transfer speed between the storage devices 0120, and a limit on the initial copy operation, and therefore manages information corresponding to each of these as a host IO limit 0600, an inter-device transfer speed limit 0601, and an initial copy operation limit 0602. However, the management system 0110 may impose limits other than these three, and may add information corresponding to the limit contents to the throttling setting information 0313.

[0055] FIG. 7 is a diagram showing an example of the configuration of the service level target information 0314 in this embodiment. The service level target information 0314 is information for managing the contents of the service level target for each remote copy that the user 0140 requests from the management system 0110 . Each entry in the service level target information 0314 includes a replication ID 0500, a lower limit IO amount 0700, and an RPO 0701.

[0056] The replication ID 0500 is information that allows the management system 0110 to uniquely identify the remote copy configuration, and is information that is used to associate the information in the remote copy configuration information 0312 with the information in the service level target information 0314 . The lower limit IO volume 0700 is information indicating a service level objective related to the lower limit of the amount of IOs that can be accepted by the source volume of the remote copy, issued by the host 0130. RPO0701 is information indicating a service level objective regarding the replication delay time that occurs in the remote copy. In this case, it can also be said that the management system 0110 has service level target information 0314 as target information that is a remote copy performance target and includes a target value for replication latency. It can also be said that the service level target information 0314, which is target information, includes the lower limit IO amount 0700 as the amount of write that the source volume will accept.

[0057] In the example of FIG. 7, the remote copy with replication ID 0500 of "0" is requested by user 0140 to have a service level target of "10k IOPS" as the lower limit IO volume 0700 and "30 sec" as the RPO 0701.

[0058] A new entry is added to the service level target information 0314 when the management system 0110 creates a new remote copy in accordance with a request from the user 0140 .

[0059] The management system 0110 shown in this embodiment is configured to manage remote copying so as to satisfy two types of service level objectives: the lower limit IO volume 0700 and the RPO 0701 shown in FIG. 7. In this case, performance information important for stable remote copying can be set. However, this is not limited to this, and other service level objective items may also be included. For example, the configuration may be such that the time until the initial copy is completed can also be set as a service level objective. In this case, the configuration may be such that the throttling adjustment process to guarantee the service level objective, which will be described later, is adjusted so as not to violate the service level objective related to the initial copy.

[0060] Furthermore, in this embodiment, the service level target is treated as a target that must always be guaranteed, but this is not limited to this. For example, the system may be configured to allow a certain degree of tolerance for violating the service level target, such as "guaranteed for 99% of the last month (1% of the month (approximately 7 hours) may be violated)." In this case, the management system 0110 manages the remaining time (commonly referred to as the error budget) during which violation of the service level target is permitted in the service level target information 0314 and the service level target excess / shortage rate information 0316. The management system 0110 may be configured to allow throttling that intentionally violates the service level target within a margin of the error budget during the process of adjusting throttling to guarantee the service level target, which will be described later.

[0061] FIG. 8 is a diagram showing an example of the configuration of the remote copy performance information 0315 in this embodiment. The remote copy performance information 0315 is information for managing performance information relating to replication processing, which is collected by the management system 0110 from each storage apparatus 0120 . Each entry in the remote copy performance information 0315 includes a replication ID 0500 , an operation status 0800 , a host IO amount 0801 , a replication delay 0802 , a journal usage rate 0803 , and an inter-device transfer rate 0804 .

[0062] The replication ID 0500 is information that allows the management system 0110 to uniquely identify the remote copy configuration, and is information that is used to associate the information in the remote copy configuration information 0312 with the information in the remote copy performance information 0315 . The operation status 0800 is information indicating the current operation of the replication process, such as whether or not the remote copy is currently in the initial copy state. The host IO volume 0801 is information indicating the amount of IO issued by the host 0130 to the source volume of the remote copy. The replication delay 0802 is information indicating the time from when the host 0130 writes data to the source volume until the data is replicated to the destination storage device 0120 in the remote copy. The journal usage rate 0803 is information indicating the usage rate of the master journal area MJNL 0122 used at the replication source of the remote copy using the journal method. The inter-device transfer rate 0804 is information indicating the communication bandwidth between the source storage device 0120 and the destination storage device 0120 used to transfer data related to the remote copy in the remote copy.

[0063] In the example of Figure 8, the remote copy with replication ID 0500 of "0" has an operating status 0800 of "normal," a host IO volume 0801 of "12k IOPS," a replication latency 0802 of "31 seconds," a journal usage rate 0803 of "20%," and an inter-device transfer rate 0804 of "63 MB / s." The remote copy with replication ID 0500 of "2" has a replication mode 0502 of "snapshot" (see Figure 5), so the journal usage rate 0803 is "N / A." The remote copy with replication ID 0500 of "4" has an operating status 0800 of "initial copy," and replication processing is not running, so the replication latency 0802 and journal usage rate 0803 are "N / A."

[0064] In the remote copy performance information 0315, new entries are added and existing entries are updated during the process in which the device monitor 0322 of the management system 0110 acquires information related to remote copying from the storage device 0120.

[0065] FIG. 9 is a diagram showing an example of the configuration of the service level target excess / deficiency rate information 0316 in this embodiment. The service level target excess / deficiency rate information 0316 is information used by the management system 0110 to manage the degree of achievement of the service level target set for each remote copy. Each entry of the service level target excess / shortage rate information 0316 includes a replication ID 0500, a lower limit IO amount excess / shortage rate 0900, an RPO excess / shortage rate 0901, a maximum excess / shortage rate 0902, and a minimum excess / shortage rate 0903. The surplus / deficiency rate is information in which 100% means that the service level target has been exactly achieved, less than 100% means that the target has been violated, and more than 100% means that there is a margin for achieving the target.

[0066] The replication ID 0500 is information that allows the management system 0110 to uniquely identify the remote copy configuration, and is information that is used to associate the information in the remote copy configuration information 0312 with the information in the service level target excess / deficiency rate information 0316 . The lower limit IO volume excess / deficiency rate 0900 is information indicating the degree of achievement of the service level target for the lower limit IO volume 0700 set for the remote copy in question. The RPO surplus / deficiency rate 0901 is information indicating the degree to which the service level target of the RPO 0701 set for the remote copy in question has been achieved. The maximum surplus / deficiency rate 0902 is information indicating the surplus / deficiency rate of the item with the largest surplus / deficiency rate among the service level targets set for the remote copy. The smallest surplus / shortage rate 0903 is information indicating the surplus / shortage rate of the item with the smallest surplus / shortage rate among the service level targets set for the remote copy.

[0067] In the example of Figure 9, the remote copy with replication ID 0500 of "0" has a lower limit IO volume surplus / shortage rate 0900 of "120%" (a state with a margin relative to the service level target), an RPO surplus / shortage rate 0901 of "97%" (a state in which the service level target is violated), a maximum surplus / shortage rate 0902 of "120%", and a minimum surplus / shortage rate 0903 of "97%".

[0068] The service level target excess / deficiency rate information 0316 includes new entries and existing entries that are updated during the process in which the service level guarantee unit 0323 of the management system 0110 determines the details of throttling adjustment.

[0069] In this embodiment, the management system 0110 manages two types of service level targets, a lower limit IO volume 0700 and an RPO 0701, in the service level target information 0314. Therefore, the management system 0110 also manages a lower limit IO volume surplus / deficiency rate 0900 and an RPO surplus / deficiency rate 0901 corresponding to these in the service level target surplus / deficiency rate information 0316. As mentioned in the description of the remote copy performance information 0315 in Fig. 7, there may be other service level target items. In that case, information on the excess / deficiency rate corresponding to the added service level target is managed in the service level target excess / deficiency rate information 0316.

[0070] The above has described various types of information handled by the management system 0110 in this embodiment. The above explanation shows information used for control in this embodiment and information that is often included in a general management system 0110. However, new information may be added to the management information 0310 as needed, and unnecessary information may be deleted.

[0071] In the following description, various processes of the management system 0110 outlined using steps S1 to S7 in FIG. 1 will be described in detail. The processing described below is limited to the minimum processing required to implement the essence of the present invention, and explanations of parts that are not important for implementation are omitted, assuming that they are properly implemented. Furthermore, the following explanation omits, for example, the processing of the storage device 0120, which accepts data read / write requests to volumes from the host 0130, the replication processing itself using journaling or snapshot methods, the processing of accepting various instructions from the management system 0110, and the processing when an abnormality occurs in various processes or components. Furthermore, when implementing various processes, arbitrary processing may be added as necessary.

[0072] <Explanation of the operation of the management system 0110> Next, the operation of the management system 0110 will be described. FIG. 10 is a diagram showing the flow of processing in which the remote copy establishment unit 0321 of the management system 0110 establishes a remote copy in response to a request from the user 0140 of the storage device 0120, as shown in steps S1 to S3 in FIG. Steps S0100 to S0103 are performed by the remote copy construction unit 0321 present in the control program 0320 of the management system 0110. This process is started when the user 0140 of the storage device 0120 instructs the management system 0110 to construct a remote copy.

[0073] In this process, first, the user 0140 of the storage device 0120 requests the management system 0110 to establish a new remote copy, and the request is received by the remote copy establishment unit 0321 of the management system 0110 (S0100). This request includes the replication method to be used in the remote copy to be constructed, information on the replication source storage device 0120, the volume PVOL0121, and the master journal area MJNL0122, information on the replication destination storage device 0120, the volume SVOL0125, and the restore journal area RJNL0124, information on the service level target, etc. In addition to the above, this request may also include any other information, such as information that allows the management system 0110 to identify the user 0140.

[0074] Furthermore, the remote copy establishment unit 0321 may receive requirements regarding the replication destination, such as physical distance, instead of receiving information about the replication destination storage device 0120 from the user 0140. In this case, the remote copy establishment unit 0321 automatically selects a replication destination storage device 0120 that meets the requirements, based on the storage device information 0311, etc. Furthermore, the remote copy construction unit 0321 may select a destination storage device 0120 that is less likely to violate the service level target by referencing management information 0310 such as remote copy configuration information 0312 and remote copy performance information 0315. Specifically, one possible method is to avoid a storage device 0120 that already has a remote copy with a large lower limit IO volume 0700 in the service level target information 0314.

[0075] Next, the remote copy construction unit 0321 uses the received information and the remote copy configuration information 0312 to group remote copies that share a bottleneck that causes a deterioration in RPO (S0101). Any method for this grouping may be used, but the simplest method is to treat remote copies having the same pair of source and destination storage devices 0120 as one group. In the example of FIG. 1, remote copies performed from storage device 0120A to storage device 0120B are placed in the same group. Also, remote copies performed from storage device 0120A to storage device 0120C are placed in the same group. On the other hand, remote copies performed from storage device 0120A to storage device 0120B and remote copies performed from storage device 0120A to storage device 0120C are not placed in the same group, but in different groups. Note that a method may also be used in which remote copies having the same source or destination storage device 0120 are treated as one group. In this case, it can also be said that multiple remote copies are grouped by remote copies having a common storage device where at least one of the storage device in which the source volume is located and the storage device in which the destination volume is located is common.

[0076] More advanced methods include, for example, a method in which the same pair of source and destination storage devices 0120 are treated as separate groups if they use different communication ports 0204; a method in which the management system 0110 even manages the communication lines actually used and treats them as separate groups if the lines are different; and a method in which the management system 0110 manages information about some shared resource (e.g., CPU 0201 or memory 0202) used by the storage device 0120 for replication processing and groups the resources according to the shared range of the resource. In this case, the remote copy construction unit 0321 groups multiple remote copies, and the grouping can be said to be based on, for example, whether at least one of the source volume and the destination volume uses the same resource when performing the remote copy. This makes it easier to determine remote copies that are likely to affect each other. Furthermore, in this embodiment, this grouping process is performed only when a new remote copy is constructed, but the grouping may be reviewed periodically to accommodate changes in bottleneck locations.

[0077] Next, the remote copy construction unit 0321 adds a new entry to the management information 0310 based on the received information and the grouping results obtained in the previous step (S0102). Specifically, the remote copy establishment unit 0321 adds new entries to the remote copy configuration information 0312, the service level target information 0314, and the throttling setting information 0313. Furthermore, the remote copy establishment unit 0321 registers information in the throttling setting information 0313 indicating that no throttling is set.

[0078] Next, the remote copy establishment unit 0321 instructs the source and destination storage apparatuses 0120 to establish a remote copy (S0103). This construction instruction is realized by some kind of communication between the remote copy construction unit 0321 and each storage device 0120 via the communication port 0204 . Each storage device 0120 establishes a remote copy in accordance with the instructions, but details are omitted here.

[0079] FIG. 11 is a diagram showing the flow of processing shown in steps S4 and S5 in FIG. 1, in which the device monitor 0322 of the management system 0110 monitors each storage device 0120. This process is periodically started and executed by the management system 0110.

[0080] In this process, the device monitoring unit 0322 first refers to the storage device information 0311 and acquires a list of storage devices 0120 from which information is to be acquired (S0201). Next, the device monitoring unit 0322 references the remote copy configuration information 0312 and acquires a list of remote copies from which information is to be acquired (S0202). Next, the device monitoring unit 0322 acquires information about each remote copy acquired in step S0202 from each storage device 0120 acquired in step S0201 (S0203). This information acquisition is realized by some kind of communication between the device monitor 0322 and each storage device 0120 via the communication port 0204 .

[0081] Finally, the device monitor 0322 updates the remote copy performance information 0315 using the acquired information (S0204). At this time, the device monitoring unit 0322 may perform any processing on the collected information. For example, it may convert the data format used inside the storage device 0120 into the data format used in the management system 0110, convert units, etc. The device monitoring unit 0322 may also generate one piece of information by aggregating information acquired from both the source and destination storage devices 0120. For example, the replication delay 0802 in the journal method may be calculated using the difference between the time when data is written to the source volume PVOL0121 of the source storage device 0120 and a log is generated, and the time when the log is transferred to the destination storage device 0120 and reflected in the destination volume SVOL0125.

[0082] FIG. 12 is a diagram showing the process flow for adjusting the operation of the replication process in each storage device 0120 in order for the service level guarantee unit 0323 of the management system 0110 to guarantee the service level target set for each remote copy, as shown in steps S6 to S7 in FIG. 1. This process is periodically started and executed by the management system 0110.

[0083] In this process, the service level guarantee unit 0323 first performs a process to extract replication groups for which throttling settings need to be adjusted (S0301). This process will be described later with reference to FIG. Then, the adjustment process of steps S0303 to S0304 is executed sequentially for each of the extracted replication groups (S0302). In this case, the service level guarantee unit 0323 can also be said to select other remote copies for which the transfer amount is to be limited from the same group as the remote copy that requires the replication delay time, based on the grouping results. This makes it possible to adjust the transfer amount between remote copies that are likely to affect each other.

[0084] In step S0303, the service level guarantee unit 0323 determines the adjustment details of the throttling settings using the latest remote copy performance information 0315 collected by the device monitoring unit 0322. This processing will be described later with reference to FIG.

[0085] Thereafter, in step S0304, the service level guarantee unit 0323 sets the throttling settings adjusted in the previous step for each storage device 0120. This setting process is realized by some kind of communication between the device monitor 0322 and each storage device 0120 via the communication port 0204.

[0086] Although not shown, each storage device 0120 has a function for realizing the throttling setting instructed by the service level guarantee unit 0323 of the management system 0110, and performs throttling by using this function. For example, many storage devices 0120 generally have a Quality-of-Service function that limits the amount of IO received from the host 0130, and throttling of the host IO limit 0600 can be easily realized by using this function. In addition, the inter-device transfer speed limit 0601 and the initial copy operation limit 0602 are used to limit the remote copy process itself, and can be achieved by reducing the execution cycle and parallelism of the log transfer process within the storage device 0120 .

[0087] FIG. 13 is a diagram showing the processing flow for identifying a replication group for which the throttling setting in the service level guarantee unit 0323 is to be adjusted. This process is a subroutine executed in step S0301 of FIG.

[0088] In this process, a replication group including a remote copy for which some throttling has been set and a replication group including a remote copy for which an RPO violation has occurred are responded to the caller as targets for which throttling settings need to be adjusted. The former is because adjustments including the cancellation of the throttling that has already been set are required, while the latter is because adjustments to strengthen throttling are required to improve the RPO violation.

[0089] In this process, the service level guarantee unit 0323 sequentially executes the processes of steps S0402 to S0404 for each replication group defined in the remote copy configuration information 0312 (S0401).

[0090] First, in step S0402, the service level guarantee unit 0323 references the throttling setting information 0313 to determine whether or not any throttling has been set for any of the remote copies belonging to the replication group. As a result, if it has not been set (No in step S0402), the process proceeds to step S0403, and if it has been set (Yes in step S0402), the process proceeds to step S0404.

[0091] Next, in step S0403, the service level guarantee unit 0323 refers to the remote copy performance information 0315 and the service level target information 0314 to determine whether an RPO violation (replication delay 0802 > RPO 0701) has occurred in any of the remote copies belonging to the replication group. If the result is that there is a violation (Yes in step S0403), the process proceeds to S0404, and if there is no violation (No in step S0403), the process for the replication group ends and the process moves to the next replication group.

[0092] For simplicity's sake, we have included those instances where an RPO violation has actually occurred as an adjustment target, but this method means that the service level target requested by the user 0140 will not be achieved during the period until an appropriate adjustment is made. To prevent this, for example, the replication delay 0802 information in the remote copy performance information 0315 may be managed in chronological order, and future replication delay 0802 may be predicted by performing regression analysis or the like, and throttling adjustment may be performed in advance for those instances where it is determined that an RPO violation will occur in the future. In this case, the service level assurance unit 0323 will adjust throttling not when the replication delay time of remote copy has exceeded the service level target, but when it is likely to do so.

[0093] In step S0404, the service level guarantee unit 0323 determines that the replication group is a target for throttling setting adjustment, and adds the replication group to a list of targets for adjustment to be returned to the caller.

[0094] FIG. 14 is a diagram showing a processing flow for determining the adjustment contents of the throttling setting in the service level guarantee unit 0323. This process is a subroutine executed in step S0303 of FIG.

[0095] In this process, the service level guarantee unit 0323 first executes the process of step S0502 sequentially for each remote copy that belongs to the replication group to be adjusted (S0501).

[0096] In step S0502, the service level guarantee unit 0323 updates the service level target excess / deficiency rate information 0316 for each remote copy belonging to the replication group to be adjusted. This calculation process will be described later with reference to FIG. The surplus / deficiency rate calculated here means that 100% exactly meets the service level target, less than 100% means that the target is violated, and more than 100% means that there is a margin for achieving the target.

[0097] Next, the service level guarantee unit 0323 determines whether or not there is a minimum surplus / deficiency rate 0903 of less than 100% among the calculated surplus / deficiency rates of each remote copy (S0503). If there is anything less than 100% (Yes in step S0503), the process proceeds to step S0504; if there is nothing (if the excess / shortage rate is 100% or more in all remote copies) (No in step S0503), the process proceeds to step S0507. In step S0504, the service level guarantee unit 0323 determines whether or not any of the calculated maximum surplus / deficiency rates 0902 of the remote copies exceeds 100%. As a result, if there is anything that is 100% or more (Yes in step S0504), the process proceeds to step S0505, and if there is nothing that is not (No in step S0504), the process proceeds to step S0506.

[0098] In step S0505, the service level guarantee unit 0323 adjusts the throttling setting for the remote copy with the largest maximum surplus / deficiency rate 0902 and the greatest margin for target achievement so as to lower the maximum surplus / deficiency rate 0902. Details of this processing will be described later with reference to FIG. This adjustment means that the resources (such as the communication bandwidth between the storage devices 0120) consumed by the remote copy with the most leeway relative to the service level target are allocated to the remote copy that is violating the service level target, thereby equalizing the excess / shortage ratios of each remote copy. In this case, the service level assurance unit 0323 limits the remote copy transfer volume so that the degree of achievement of the target replication latency for multiple remote copies in the same group is approximately the same. In this case, the degree of achievement of the target replication latency can be expressed by the IO volume excess / shortage ratio 0900 and the RPO excess / shortage ratio 0901. Therefore, the service level assurance unit 0323 limits the remote copy transfer volume so that the IO volume excess / shortage ratio 0900 and the RPO excess / shortage ratio 0901 are approximately the same. This enables more stable remote copy operation. This is expected to eliminate or improve violations of service level targets.

[0099] In S0506, the service level guarantee unit 0323 notifies the user 0140 of the storage device 0120 that the service level target cannot be achieved, and prompts the user 0140 to intervene. This is because there is no maximum surplus / deficiency rate 0902 that exceeds 100%, and if throttling is made any stricter than this, it will result in a violation of another service level objective.

[0100] As mentioned in the description of the remote copy performance information 0315 in Fig. 7, the service level target information 0314 and the service level target excess / deficiency rate information 0316 may be configured to manage the remaining time (error budget) during which violation of the service level target is permitted. In this case, in step S0506, the service level guarantee unit 0323 may choose to throttle remote copies that have a surplus in the error budget so that they intentionally violate the service level target. The service level guarantee unit 0323 may then switch remote copies that are intentionally subject to violation of the service level target so that the error budgets of all remote copies are used up evenly, thereby postponing the depletion of the error budget and reducing the number of situations requiring intervention by the user 0140.

[0101] In step S0507, the service level guarantee unit 0323 adjusts the throttling settings to increase the minimum surplus / deficiency rate 0903 for the remote copy with the smallest minimum surplus / deficiency rate 0903 and the smallest margin for target achievement, among the remote copies for which throttling is set. This process is intended to sequentially release throttling, starting with the strictest throttling settings that left little room for target achievement, for throttling settings that were set due to past violations of service level targets, since the violation state has now been resolved.

[0102] The throttling can be lifted by simply lifting the restrictions to make it unlimited, or by gradually loosening the throttling at a fixed rate for each execution cycle of the service level guarantee processing flow (see Figure 12) to gradually approach an unlimited state. Alternatively, a more advanced method may be used. For example, the service level guarantee unit 0323 may repeatedly relax the throttling until the excess / shortage rate becomes equal to the second lowest rate (not including the tie rate), gradually approaching an unlimited state. Another possible method is for the service level guarantee unit 0323 to estimate the remote copy processing volume that can currently be used without violating the service level target, based on the remote copy performance information 0315, and then relax the throttling by the surplus amount.

[0103] FIG. 15 is a diagram showing the processing flow for calculating the excess / deficiency rate with respect to the service level target of remote copy in the service level guarantee unit 0323. This process is a process (subroutine) executed in step S0502 of Fig. 14. By using the excess / shortage rate, it is possible to more easily determine the remote copy that most shorts the service level target, which is the required performance.

[0104] In this process, the service level guarantee unit 0323 first refers to the remote copy performance information 0315 for the remote copy to be processed, and determines whether the operational state 0800 is "initial copy" (S0601). As a result, if the operation status 0800 is "initial copy" (Yes in step S0601), the process proceeds to step S0606, and if not (No in step S0601), the process proceeds to step S0602.

[0105] In step S0602, the service level guarantee unit 0323 determines whether or not any of the following conditions is met for the remote copy: the replication delay 0802 is greater than the RPO 0701, the replication mode 0502 is the snapshot method, or there is a predetermined margin in the journal usage rate 0803. If any of these conditions is met (Yes in step S0602), the process proceeds to step S0603; if not (No in step S0602), the process proceeds to step S0604. In step S0606, the service level guarantee unit 0323 determines whether the initial copy operation restriction 0602 for the remote copy is set to "stopped." As a result, if it is "stopped" (Yes in step S0606), the process proceeds to step S0607, and if not (No in step S0606), the process proceeds to step S0608.

[0106] In step S0603, the service level guarantee unit 0323 updates the service level target excess / deficiency rate information 0316 for the remote copy in question as follows: RPO excess / deficiency rate 0901=RPO 0701 / replication delay 0802, and then the process proceeds to step S0605. In step S0604, the service level guarantee unit 0323 updates the service level target excess / deficiency rate information 0316 for the remote copy in question, setting the RPO excess / deficiency rate 0901 to a fixed value of 100%, and then proceeds to step S0605.

[0107] In addition, the condition for proceeding to S0604 in the conditional branch of step S0602 means, in other words, that the replication delay 0802 is smaller than the RPO 0701, the replication mode 0502 is the journal method, and there is no specified margin in the journal usage rate 0803. Because the replication delay 0802 is smaller than the RPO 0701, the calculation in step S0603 would normally result in the RPO surplus / deficiency rate 0901 being greater than 100%. However, there is no room for throttling, such as limiting the log transfer speed between storage devices 0120 in the journal method, in order to lower the RPO surplus / deficiency rate 0901. For this reason, the RPO surplus / deficiency rate 0901 is set to a fixed value of 100%, so that it will not be selected as a target for throttling in future processing. In this case, remote copies include those that use a journal system, and if the journal usage rate, which is the usage rate of the journal area used when performing a remote copy, does not satisfy a predetermined range, the service level guarantee unit 0323 will not use this remote copy as another remote copy that limits the transfer amount. This allows for stable operation of remote copies even when the remote copy uses a journal system.

[0108] In step S0605, the service level guarantee unit 0323 updates the service level target surplus / deficiency rate information 0316 for the remote copy as follows: lower limit IO volume surplus / deficiency rate 0900=host IO volume 0801 / lower limit IO volume 0700, and then proceeds to step S0609.

[0109] In step S0607, the service level guarantee unit 0323 updates both the RPO excess / deficiency rate 0901 and the lower limit IO volume excess / deficiency rate 0900 for the remote copy to a fixed value of 100%, and then proceeds to step S0609. In step S0608, the service level guarantee unit 0323 updates both the RPO excess / deficiency rate 0901 and the lower limit IO volume excess / deficiency rate 0900 for the remote copy to a fixed value of ∞ (infinity), and then proceeds to step S0609. In this embodiment, since there is no service level target for the initial copy, when the initial copy is in operation (step S0608), the excess / deficiency rate is set to ∞ so that the remote copy is selected as the target for throttling with the highest priority. Note that instead of setting the excess / deficiency rate to ∞, the service level guarantee unit 0323 may set it to an unrealistically large value. In this case, the service level guarantee unit 0323 does not include remote copies whose operating state is the initial copy when determining the journal usage rate. In other words, even if the remote copy is a journal type and the journal area is insufficient, the initial copy is selected as the target for throttling with the highest priority. This allows resources for the initial copy, which takes a long time, to be preferentially allocated to other remote copies. In this case, the service level guarantee unit 0323 can be said to select another remote copy for which transfer is to be restricted, taking into consideration the journal usage rate of the journal-based remote copy.

[0110] In step S0609, the smaller of the RPO excess / shortage rate 0901 and the lower limit IO amount excess / shortage rate 0900 calculated in the steps up to this point is set as the minimum excess / shortage rate 0903. Then, the larger one is set as the maximum excess / shortage rate 0902, and the service level target excess / shortage rate information 0316 is updated.

[0111] FIG. 16 is a diagram showing an example of the throttling adjustment process in the service level guarantee unit 0323. This process is a subroutine executed in step S0505 of FIG.

[0112] In this process, the service level guarantee unit 0323 first determines whether the operation status 0800 of the remote copy that is the subject of throttling is "initial copy" (step S0701). As a result, if the operation status 0800 is "initial copy" (Yes in step S0701), the process proceeds to step S0702, and if not (No in step S0701), the process proceeds to step S0703.

[0113] In step S0702, the service level guarantee unit 0323 updates the throttling setting information 0313 to stop the initial copy running in that remote copy. This allows resources for the initial copy, which takes a long time, to be allocated to a remote copy whose replication delay 0802 does not satisfy the RPO 0701. As a result, the replication delay time can be recovered.

[0114] In step S0703, the service level guarantee unit 0323 determines whether the excess / shortage rate that is the maximum excess / shortage rate 0902 is the RPO excess / shortage rate 0901 or not. As a result, if the maximum surplus / shortage rate 0902=RPO surplus / shortage rate 0901 (Yes in step S0703), the process proceeds to step S0704; otherwise (No in step S0703), the process proceeds to step S0705.

[0115] In step S0704, the service level guarantee unit 0323 adjusts the throttling for the remote copy so as to reduce the transfer speed between the storage devices 0120, and updates the throttling setting information 0313. This makes it possible to recover the replication delay time. In step S0705, the service level guarantee unit 0323 adjusts throttling for the remote copy in question to reduce the host IO volume 0801, and updates the throttling setting information 0313. By limiting the amount of IOs issued by the host, the amount of transfer from the source volume to the destination volume is reduced. As a result, the transfer speed from the source volume to the destination volume can be indirectly limited, and the replication latency can be restored.

[0116] In the above method, it can be said that the service level guarantee unit 0323 limits the transfer amount of other remote copies that affect the remote copy so that the remote copy satisfies the replication delay time of the target information. Specifically, when multiple other remote copies are candidates for transfer amount restriction, the service level guarantee unit 0323 stops the other remote copies whose operating status is the initial copy.

[0117] Furthermore, when there are no other remote copies whose operating status is the initial copy and multiple other remote copies are candidates for transfer volume restriction, the service level guarantee unit 0323 determines which other remote copies are to be subject to transfer restriction based on the difference between the performance information of the other remote copies and their target information. Then, based on this difference, if the required performance that is most deficient is replication latency, the service level guarantee unit 0323 limits the transfer rate from the source volume to the destination volume. In this case, the service level guarantee unit 0323 can be said to limit the transfer volume of the other remote copy by restricting data writing to the source volume related to the other remote copy. This reduces the transfer volume of the other remote copy, improving the replication latency of the remote copy to be improved. This also reduces the transfer volume of the remote copy to be improved, improving the replication latency of the remote copy to be improved.

[0118] Furthermore, based on this difference, if the required performance that is most deficient is the transfer speed from the source volume to the destination volume, the service level guarantee unit 0323 will limit the amount of I / O that can be accepted by the host. In this case, it can be said that the service level guarantee unit 0323 limits the transfer amount of other remote copies based on the write amount of target information and the replication latency time. In this way, the service level guarantee unit 0323 improves the replication delay time of remote copying by restricting the data writing to the source volume involved in remote copying based on the write amount and replication delay time of the target information. Furthermore, it can be said that the service level guarantee unit 0323 determines other remote copies for which the transfer amount is to be limited based on the excess / deficiency rate, which is the value obtained by dividing the current performance by the required performance.

[0119] In both steps S0704 and S0705, throttling is tightened to adjust the excess / shortage rate, but the amount of adjustment in one execution cycle of the service level guarantee processing flow (see FIG. 12) may be simply tightened by a predetermined fixed amount or fixed rate, or a more advanced method may be used. An example of a more advanced method is to repeatedly tighten the throttling until the excess / shortage rate becomes equal to the second highest rate (not including the tie rate), and gradually tighten the throttling. Another possible method is to estimate the current amount of remote copy processing that can be used without violating the service level target based on the remote copy performance information 0315, and then tighten throttling to compensate for the shortfall.

[0120] FIG. 17 is a diagram showing an example of changes in replication delay time when remote copy is managed by the management system 0110 of this embodiment. In FIG. 17, the horizontal axis represents time, and shows the change over time in the speed of the communication line, the replication delay time, and the amount of IO issued by the host 0130. 17 shows that up to time t1, the situation was normal and the communication line speed was secured. At this time, the replication delay time was smaller than the RPO and the IO volume was larger than the lower limit IO volume. Here, it is shown that at time t1, the speed of the communication line slows down due to congestion, etc. As the speed of the communication line slows down, the replication delay time increases between time t1 and time t2. Then, at time t2, the replication delay time indicates that the RPO has been reached. The management system 0110 detects this and, at time t3, the management system 0110 reduces the IO amount to a range that satisfies the lower limit IO amount. As a result, the replication delay time maintains the RPO. After that, at time t4, the speed of the communication line is shown to have recovered. Accordingly, the replication delay time decreases from time t4 onwards. Furthermore, as the replication delay time recovers, the management system 0110 gradually returns the IO volume to the original transfer speed. If the IO volume is not limited between time t3 and time t4, the replication delay time will not meet the required performance, resulting in an RPO violation. Therefore, the management system 0110 avoids this by reducing the IO volume. Then, at time t5, the replication delay time returns to the original level up to time t1, and at time t5, the management system 0110 returns the IO volume to the original transfer rate up to time t1.

[0121] <Explanation of effect> The management system 0110 of this embodiment manages the source and destination storage apparatuses 0120. A user who uses remote copy registers service level objectives related to the amount of write data for each remote copy and the RPO as required performance with the management system 0110. The management system 0110 then allocates resources (such as communication bandwidth between storage apparatuses 0120) for remote copies that have a margin for the required performance among multiple remote copies to the remote copy that is violating the service level objective. This allows remote copy to be operated while satisfying the service level objective, which is the required performance. Therefore, remote copy can be operated stably even in an environment where the line speed and the amount of data written from the host 0130 are uncertain.

[0122] <Explanation of management method> The processing performed by the management system 0110 is realized by the cooperation of software and hardware resources, as shown in FIG. Therefore, the processing performed by the above-mentioned management system 0110 can be considered to be a management method in which a management device having a processor communicates with each other via a network and manages multiple storage devices, each having a volume, in which remote copies are set for combinations of multiple source volumes and destination volumes, in which data is transferred and replicated from a source volume of the multiple volumes to a destination volume via the network, and the management device monitors the replication delay time, which is the time required to replicate the remote copy, and controls to limit the transfer volume of other remote copies based on the replication delay time.

[0123] The program for realizing this embodiment can be provided not only by communication means but also by being stored on a recording medium such as a CD-ROM.

[0124] Although the main embodiment of the present invention has been described above, this is merely an example for explaining the present invention, and is not intended to limit the scope of the present invention to only this embodiment. Furthermore, it is not necessary to have all of the components described, and it is possible to replace or add part of the components of one embodiment with the components of another embodiment. Similarly, it is also possible to change or delete part of the components of each embodiment as necessary. [Explanation of symbols]

[0125] 0100... storage system, 0110... management system, 0120, 0120A to 120C... storage devices, 130, 130A to 130D... host, 0201... CPU, 0202... memory, 0310... management information, 0320... control program, 0321... remote copy construction unit, 0322... device monitoring unit, 0323... service level assurance unit

Claims

1. a processor; A management device that manages a plurality of storage devices that communicate with each other via a network and each have a volume, a remote copy in which data is transferred from a source volume of the plurality of volumes to a destination volume via the network and replicated is set for a plurality of combinations of the source volume and the destination volume; A management device monitors a replication delay time, which is the time required for the replication of the remote copy, and controls to limit the transfer amount of other remote copies based on the replication delay time.

2. a target value for the replication delay time, the target value being a performance target for the remote copy; 2. The management device according to claim 1, wherein the transfer amount of the other remote copy that affects the remote copy is limited so that the remote copy satisfies the replication delay time of the target information.

3. the source volume accepts a write request and stores data, and the data stored in the source volume is replicated to the destination volume by the remote copy; The management device limits the transfer amount of the other remote copy by limiting the writing of data to the source volume related to the other remote copy.

3. The management device according to claim 2.

4. the target information includes the amount of writes that the source volume can accept, The management device limits the transfer amount of the other remote copy based on the write amount and replication delay time of the target information.

4. The management device according to claim 3.

5. the source volume accepts a write request and stores data, and the data stored in the source volume is replicated to the destination volume by the remote copy; the target information includes the amount of writes that the source volume can accept, The management device improves the replication delay time of the remote copy by limiting the writing of data to the replication source volume for the remote copy based on the write amount of the target information and the replication delay time.

3. The management device according to claim 2.

6. When a plurality of other remote copies are candidates for the transfer amount limit, the other remote copies for which the transfer amount limit is to be applied are determined based on the difference between the performance information of the other remote copies and the target information.

3. The management device according to claim 2.

7. When a new remote copy is set for the source volume in which data is stored, an initial copy is performed to copy the stored data; 3. The management device according to claim 2, wherein when a plurality of other remote copies become candidates for the transfer amount limit target, the management device suspends the other remote copies whose operating status is an initial copy.

8. 2. The management device according to claim 1, wherein for multiple remote copies, at least one of the storage device in which the source volume is located and the storage device in which the destination volume is located groups common remote copies, and selects other remote copies for which the transfer volume is limited from the same group as the remote copy that requires the replication delay time.

9. 9. The management device according to claim 8, wherein the transfer amount of the remote copy is limited so that the degree of achievement of the target value of the replication delay time is approximately the same for a plurality of remote copies in the same group.

10. 2. The management device according to claim 1, wherein the other remote copy to be limited for transfer is selected in consideration of a journal usage rate of the remote copy using a journal method.

11. A management method in which a management device having a processor manages a plurality of storage devices each having a volume, the storage devices communicating with each other via a network, comprising: a remote copy in which data is transferred from a source volume of the plurality of volumes to a destination volume via the network and replicated is set for a plurality of combinations of the source volume and the destination volume; The management method in which the management device monitors a replication delay time, which is the time required for the replication of the remote copy, and controls so as to limit the transfer amount of other remote copies based on the replication delay time.

12. a plurality of storage devices that communicate with each other via a network and each have a volume; a management device having a processor and managing the plurality of storage devices; In a storage system comprising: a remote copy in which data is transferred from a source volume of the plurality of volumes to a destination volume via the network and replicated is set for a plurality of combinations of the source volume and the destination volume; The management device monitors a replication delay time, which is the time required for the replication of the remote copy, and controls the storage system so as to limit the transfer amount of other remote copies based on the replication delay time.

Images