Data generation control system and data generation control method

Abstract

To allow for a reduction in a time required to generate data.SOLUTION: In a calculator system 1, a management terminal 300 is configured to accept a specification of generation target data and a storage apparatus 500 to be generated, identify similar data similar to the generation target data, transmit from an object storage service 400 to the storage apparatus 500 the similar data and a first expected time expected to be required for first transmission processing of transmitting the generation target data from a storage apparatus 200 where the generation target is stored to the storage apparatus 500, calculate a second expected time expected to be required for second transmission processing of transmitting from the storage apparatus 200 to the storage apparatus 500 difference data between the generation target data and the similar data, and cause the second transmission processing to transmit the similar data and the difference data to the storage apparatus 500 to generate the generation target data when the second expected time is shorter than the first expected time.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for generating data by utilizing data in other devices connected via a network. [Background technology]

[0002] 2. Description of the Related Art Hybrid clouds that combine on-premise storage and public cloud storage have come to be used as storage for storing various types of data.

[0003] As the use of hybrid clouds progresses, it is expected that there will be an increasing number of situations in which data is copied not only between on-premise devices, but also between on-premise and public cloud devices, and between public cloud devices.

[0004] As a related technology, for example, Patent Document 1 discloses a technology in which a second storage system obtains a journal regarding data updates in a first storage system via a communication line and uses the journal to update the data corresponding to the first storage system. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Patent No. 7,152,079 Summary of the Invention [Problem to be solved by the invention]

[0006] For example, when multiple storage systems are located far apart, the storage systems are connected via a WAN (Wide Area Network), but generally, due to cost and other considerations, the bandwidth of the WAN is narrower than the bandwidth of the network within the storage system.

[0007] As a result, when copying a large amount of data between storage systems, the copying process takes a long time due to the bandwidth limitations of the WAN.

[0008] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide a technique capable of shortening the time required to generate data. [Means for solving the problem]

[0009] In order to achieve the above object, a data generation control system according to one aspect is a data generation control system having a plurality of storage devices and a management device, wherein the management device receives designation of target data and a destination storage device which is a destination storage device that generates the target data, identifies similar data similar to the target data, and calculates a first estimated time expected to be required for a first transmission process for transmitting basic data corresponding to the target data from a source storage device in which the basic data is stored to the destination storage device, and a time required for transmitting the similar data from the similar data storage storage device in which the similar data is stored to the destination storage device. a second predicted time expected to be required for a second transmission process to transmit differential data between the basic data and the similar data from the source storage device to the destination storage device, and if the second predicted time is shorter than the first predicted time, the second transmission process is used to transmit the similar data and the differential data to the destination storage device, thereby causing the destination storage device to generate the data to be generated, and if the second predicted time is not shorter than the first predicted time, the first transmission process is used to transmit the basic data to the destination storage device, thereby causing the destination storage device to generate the data to be generated. Effect of the Invention

[0010] According to the present invention, the time required to generate data can be reduced. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a computer system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an overview of a data copy process in the computer system according to the first embodiment. [Diagram 3] FIG. 3 is a configuration diagram of an on-premise data center according to the first embodiment. [Figure 4] FIG. 4 is a configuration diagram of a public cloud according to the first embodiment. [Diagram 5] FIG. 5 is a configuration diagram of a group of programs stored in the memory of the storage device according to the first embodiment. [Figure 6] FIG. 6 is a configuration diagram of a memory of a management terminal according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating a data backup process performed by the storage device according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating a restore process from backup data by the storage device according to the first embodiment. [Figure 9] FIG. 9 is a configuration diagram of catalog information according to the first embodiment. [Figure 10] FIG. 10 is a configuration diagram of the managed storage registration TBL according to the first embodiment. [Figure 11] FIG. 11 is a diagram showing the configuration of the transfer path information TBL according to the first embodiment. [Figure 12] FIG. 12 is a diagram showing the configuration of the VOL information TBL according to the first embodiment. [Figure 13] FIG. 13 is a diagram showing the configuration of the backup information TBL according to the first embodiment. [Figure 14] FIG. 14 is a configuration diagram of the optimal copy path determination TBL according to the first embodiment. [Figure 15] FIG. 15 is a flowchart of a volume copy process according to the first embodiment. [Figure 16]FIG. 16 is a flowchart of the normal COPY required time prediction process according to the first embodiment. [Figure 17] FIG. 17 is a flowchart of the similar-data-utilizing COPY required time prediction process according to the first embodiment. [Figure 18] FIG. 18 is a flowchart of the similar data utilizing COPY process according to the first embodiment. [Figure 19] FIG. 19 is a diagram illustrating an overview of a data copy process in the computer system according to the second embodiment. [Figure 20] FIG. 20 is a flowchart of a similar-data-utilizing COPY required time prediction process according to the second embodiment. [Figure 21] FIG. 21 is a flowchart of the similar data utilizing COPY process according to the second embodiment. [Figure 22] FIG. 22 is a diagram illustrating an overview of a data copy process in the computer system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Some embodiments will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0013] In the following description, information may be described using the expression "AAA table", but the information may be expressed in any data structure. In other words, to show that the information does not depend on the data structure, the "AAA table" can be called "AAA information".

[0014] In addition, in the following description, processing may be described with a "program" as the subject of operation, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using a storage resource (e.g., a memory) and / or a communication interface device (e.g., a NIC (Network Interface Card)) as appropriate, the subject of processing may also be the processor. Processing described with a program as the subject of operation may also be processing performed by a computer having a processor.

[0015] FIG. 1 is a diagram showing the overall configuration of a computer system according to the first embodiment.

[0016] The computer system 1 is an example of a data generation control system, and includes an on-premise data center 10 and a public cloud 20. The on-premise data center 10 and the public cloud 20 are located at separate locations. The on-premise data center 10 and the public cloud 20 are connected via the Internet 30, which is an example of a network.

[0017] The on-premise data center 10 includes one or more hosts 100, one or more storage devices 200, and a management terminal 300 as an example of a management device. The hosts 100, the storage devices 200, the management terminal 300, and the Internet 30 are connected via a switch (SW) 11.

[0018] The host 100 performs various processes according to the user's request, and performs input / output (I / O) of data related to the processes to the storage device 200. The management terminal 300 performs processing to manage devices to be managed in the computer system 1. In this embodiment, the management terminal 300 controls processing to generate (copy, etc.) certain data to a specified device.

[0019] The storage device 200 is, for example, a block storage that manages data in units of blocks, has a storage controller (Storage CTL) 210 including a storage OS, and stores various data in the storage area of ​​a physical drive 230 (see FIG. 3). In this embodiment, the storage device 200 stores volumes that store data and backup data of the volumes. The storage device 200 also stores the backup data of the volumes in an object storage service 400, which will be described later.

[0020] The public cloud 20 includes a storage device 500 and an object storage service 400. The storage device 500, the object storage service 400, and the Internet 30 are connected via a switch 21.

[0021] The object storage service 400 is an example of a storage or a storage device, and stores data in units of objects. In this embodiment, the object storage service 400 stores backup data of the volumes of the storage device 200 as objects.

[0022] The storage device 500 is, for example, a Software Defined Storage (SDS), has a storage controller (Storage CTL) 510 including a storage OS, and stores various data in the storage area of ​​a physical drive 531 (see FIG. 4).

[0023] In this embodiment, in a path (referred to as path 1) between the storage device 200 in the on-premise data center 10 and the object storage service 400 in the public cloud 20, a path (referred to as path 2) between the object storage service 400 and the storage device 500, and a path (referred to as path 3) between the storage device 200 in the on-premise data center 10 and the storage device 500 in the public cloud 20, path 2 includes only the network within the public cloud 20, and therefore has a higher bandwidth than paths 1 and 3, which include the Internet.

[0024] Fig. 2 is a diagram for explaining an outline of a data copy process in a computer system according to the first embodiment. The data copy process in Fig. 2 is a data copy process when data (data to be generated) in volume (Vol) #a of the storage device 200 in the on-premise data center 10 is copied to volume (Vol) #b in the storage device 500 in the public cloud 20. In this embodiment, it is assumed that backup data of Vol#a at a certain point in the past is stored in the storage device 200 in the on-premise data center 10 and the object storage service 400 in the public cloud 20.

[0025] When the management terminal 300 receives an instruction from the administrator to copy data (data to be copied (data to be generated), basic data) in Vol#a of the storage device 200 (source storage, source storage device, source storage device, first storage device, backup execution storage device) to Vol#b of the storage device 500 (destination storage, destination storage device, destination storage device, second storage device) (FIG. 2(1)), it searches for data similar to the data to be copied (similar data) and determines a data transfer method (FIG. 2(2)). Here, the subsequent processing will be described assuming that similar data exists in the object storage service 400 (similar data storing storage device, backup storage device, third storage device) and it has been determined that the data should be copied (generated) using a method of transmitting similar data from the object storage service 400 to the storage device 500.

[0026] The management terminal 300 instructs the destination storage (storage device 500 in the example of FIG. 2) to copy the found similar data (FIG. 2(3)). In accordance with this instruction, the storage device 500 reads the similar data (backup data of Vol#a) instructed by the object storage service 400 and stores it in Vol#b (FIG. 2(4)).

[0027] Next, the management terminal 300 instructs the source storage (storage device 200 in the example of FIG. 2) to extract the difference between the data to be copied and the similar data, and transfer the difference to the object storage service (FIG. 2(5)). In accordance with this instruction, the storage device 200 compares Vol#a with the backup data of Vol#a, extracts the difference data (FIG. 2(6)), and transfers the difference data to the object storage service 400 (FIG. 2(7)).

[0028] Next, the management terminal 300 instructs the destination storage to copy the differential data (FIG. 2(8)). In accordance with this instruction, the storage device 500 reads the differential data from the object storage service 400 and generates data in Vol#b that will be a copy of Vol#a by reflecting the differential data in the backup data of Vol#a stored in Vol#b. This completes the copying of the data of Vol#a to Vol#b of the storage device 500.

[0029] Next, the configuration of the on-premise data center 10 will be described in detail.

[0030] FIG. 3 is a configuration diagram of an on-premise data center according to the first embodiment.

[0031] The on-premise data center 10 includes a host 100, a storage device 200, a management terminal 300, and one or more switches 11.

[0032] The switch 11 switches the data to be transmitted. The host 100 includes a processor 101, a front-end (FE) interface (I / F) 102, and a management I / F 103. The processor 101 executes various processes and performs I / O of data to the storage device 200. The FE I / F 102 mediates communication with the storage device 200. The management I / F 103 mediates communication with the management terminal 300 related to management.

[0033] The storage device 200 has two controllers 210 and a plurality of physical drives 230. The physical drives 230 are, for example, hard disk drives (HDD) or solid state drives (SSD) and store various data such as user data used by the host 100.

[0034] The controller 210 includes a processor 211 , a memory 212 , a management I / F 213 , a copy I / F 214 , an IO I / F 215 , and a drive I / F 216 .

[0035] The management I / F 213 mediates communication related to management with the management terminal 300. The copy I / F 214 mediates communication related to data copying with other storage devices. The IO I / F 215 mediates communication related to data IO with the host 100. The drive I / F 216 mediates communication with the physical drive 230.

[0036] The processor 211 executes various processes in accordance with programs stored in the memory 212 .

[0037] The memory 212 is, for example, a RAM (RANDOM ACCESS MEMORY), and stores a group of programs 222 executed by the processor 211 and necessary information.

[0038] The management terminal 300 includes a processor 301, a memory 302, and a management I / F 303. The management I / F 303 mediates communications relating to management between the host 100 and the storage device 200 and the like.

[0039] The processor 301 executes various processes in accordance with programs stored in the memory 302 .

[0040] The memory 302 is, for example, a RAM, and stores a group of programs 311 executed by the processor 301, a group of tables 312 necessary for processing, and the like.

[0041] Next, the configuration of the public cloud 20 will be described in detail.

[0042] FIG. 4 is a configuration diagram of a public cloud according to the first embodiment.

[0043] The public cloud 20 includes one or more storage devices 500, an object storage service 400, and a switch 21.

[0044] The switch 21 switches the data to be transmitted. The object storage service 400 has physical drives for storing data, and manages data on an object-by-object basis.

[0045] The storage device 500 includes a compute service 520 that performs various processes, and a block storage service 530 that stores data in blocks. The compute service 520 includes a processor 521 and a memory 522. The processor 521 executes various processes in accordance with a program stored in the memory 522.

[0046] The memory 522 is, for example, a RAM, and stores a group of programs 523 executed by the processor 521, a group of tables 524 necessary for processing, and the like.

[0047] The block storage service 530 includes one or more physical drives 531 .

[0048] Next, the configuration of the program groups 222, 523 stored in the memories 212, 522 of the storage devices 200, 500 will be described.

[0049] FIG. 5 is a configuration diagram of a group of programs stored in the memory of the storage device according to the first embodiment.

[0050] The program groups 222, 523 of the storage apparatuses 200, 500 include a storage OS program 223. The storage OS program 223 includes a volume creation program 2231, a volume backup program 2232, a volume restore program 2233, a catalog information creation program 2234, a difference data extraction program 2235, a difference extraction feasibility determination program 2236, and a volume copy program 2237.

[0051] The volume creation program 2231 performs processing to create a volume in a storage device. The volume backup program 2232 performs processing to back up a volume in a storage device. The volume restore program 2233 performs processing to restore a volume from backup data. The catalog information creation program 2234 performs processing to create catalog information 411 (264) (see FIG. 9) related to the backup of a volume. The difference data extraction program 2235 performs processing to extract the difference between the data of a volume and similar data. The difference extraction availability determination program 2236 performs processing to determine whether or not a difference can be extracted. Whether or not a difference can be extracted may be determined, for example, by whether or not the same data as the similar data is stored in the storage device. The volume copy program 2237 performs processing to copy a volume.

[0052] Next, the configuration of the memory 302 of the management terminal 300 will be described.

[0053] FIG. 6 is a configuration diagram of a memory of a management terminal according to the first embodiment.

[0054] The memory 302 of the management terminal 300 stores a program group 311 and a table group 312. The program group 311 includes a volume copy control program 3111, a similar data search program 3112, and an optimal copy path determination program 3113.

[0055] The volume copy control program 3111 controls the process of copying volume data to a storage device. For example, the volume copy control program 3111 performs a process of issuing an instruction to copy a volume. The similar data search program 3112 performs a process of searching for similar data similar to the data to be copied. The optimal copy path determination program 3113 performs a process of determining the optimal path when copying (generating) the data to be copied.

[0056] The table group 312 includes a managed storage registration table (TBL) 3121, a transfer path information TBL 3122, a VOL information TBL 3123, a backup information TBL 3124, and an optimal copy path determination TBL 3125. Each table will be described in detail later.

[0057] Next, the data backup process performed by the storage device 200 will be described.

[0058] FIG. 7 is a diagram illustrating a data backup process performed by the storage device according to the first embodiment.

[0059] In the storage apparatus 200, a pool 250 is configured based on the storage area of ​​the physical drive 230. The storage area of ​​the pool 250 is allocated to a volume 260 and a snapshot (SS) volume 261.

[0060] The storage device 200 includes a volume backup function 262 and an object conversion function 270 .

[0061] The volume backup function 262 is a function that is realized by the processor 211 executing the volume backup program 2232. The volume backup function 262 is a function that acquires a snapshot of the volume 260, creates backup data 266 including differential data between the previous snapshot and the snapshot based on the snapshot, and metadata related to the differential data, and stores the backup data 266 in external storage (in this embodiment, the object storage service 400).

[0062] The volume backup function 262 uses a catalog information creation function 263 and a differential data extraction function 265. The catalog information creation function 263 is a function that is realized by the processor 211 executing a catalog information creation program 2234. The differential data extraction function 265 is a function that is realized by the processor 211 executing a differential data extraction program 2235.

[0063] The volume backup function 262 performs backup in an incremental format, for example, periodically. Specifically, the volume backup function 262 acquires a snapshot of the volume to be backed up, and uses a differential data extraction function 265 to compare data of the acquired snapshot with data of a snapshot one generation ago, extracts updated parts as differential data, and creates encapsulated backup data 266 by adding position information indicating the storage position of the differential data in the volume and size information indicating the size of the differential data to the differential data (data body). For example, since the backup is in an incremental format, the first backup data includes all data of the volume to be backed up, and the second and subsequent backup data include differential data between the newly acquired snapshot and the snapshot one generation ago, position information, and size information.

[0064] Furthermore, the volume backup function 262 creates catalog information 264 based on a snapshot of the data to be backed up and information on the backup source volume using a catalog information creation function 263. Note that in this embodiment, the catalog information creation function 263 obtains the catalog information 411 stored in the object storage service 400, and creates the catalog information 264 by adding changes to the catalog information 411.

[0065] Furthermore, the volume backup function 262 stores the backup data 266 and the catalog information 264 in the object storage service 400. After the backup data 266 and the catalog information 264 are stored in the object storage service 400, data of snapshots other than the latest snapshot in the pool 250 may be deleted.

[0066] The object conversion function 270 is a function that is realized by the processor 211 executing the volume backup program 2232, and is a function that converts data in the block format into the object format that the object storage service 400 handles.

[0067] The object storage service 400 has a bucket 401 that stores actual data. The bucket 401 is divided inside into a catalog information storage area 410 and a data storage area 420. The catalog information storage area 410 stores catalog information 411. The data storage area 420 stores data corresponding to the backup data 266.

[0068] Next, the data restoration process performed by the storage device 500 will be described.

[0069] FIG. 8 is a diagram illustrating a restore process from backup data by the storage device according to the first embodiment.

[0070] In the storage device 500, a pool 554 is configured based on the storage area of ​​a physical drive 531 of a block storage service 530. The storage area of ​​the pool 554 is allocated to a volume 553.

[0071] The storage device 500 has an object conversion function 551 and a backup restoration function 552 .

[0072] The object conversion function 551 is a function that is realized by the processor 521 executing the volume backup program 2232, and is a function that converts object format data handled by the object storage service 400 into block format data.

[0073] The backup restoration function 552 is a function that is realized by the processor 521 executing the volume restore program 2233. The backup restoration function 552 acquires the SS#0 backup data 421-0, which is the initial backup data, from the object storage service 400, and restores data based on the SS#0 backup data 421-0 to the copy destination volume. Next, it acquires backup data in sequence up to the backup data of the target snapshot to be restored (for example, a snapshot corresponding to the volume to be copied), and restores the difference from each of the acquired backup data to the volume. This makes it possible to restore (generate) the desired data to the volume.

[0074] Next, the catalog information 411 (264) will be described.

[0075] FIG. 9 is a configuration diagram of catalog information according to the first embodiment.

[0076] The catalog information 411 (264) manages information related to backups (snapshots). The catalog information 411 (264) stores an entry for each backup.

[0077] An entry of the catalog information 411 (264) includes fields for a backup ID (SS#) 411a, a backup source device ID 411b, a backup source VOL# 411c, a backup source VOL size 411d, a one generation previous backup ID 411e, an acquisition date and time 411f, an object key 411g, and an object size 411h.

[0078] The Backup ID (SS#) 411a stores the ID (Backup ID) of the backup (Backup) corresponding to the entry. In this embodiment, the number (SS#) of the snapshot is used as the Backup ID. The Backup source device ID 411b stores the ID of the device (Backup source device) that stores the volume to be backed up corresponding to the entry. The Backup source VOL# 411c stores the ID (Vol#) of the volume to be backed up corresponding to the entry (Backup source VOL). The Backup source VOL Size 411d stores the size of the Backup source volume corresponding to the entry. The One Generation Previous Backup ID 411e stores the Backup ID indicating the backup one generation before the backup corresponding to the entry. Note that if the backup corresponding to the entry is the first backup for the volume, there is no backup one generation before, so N / A is displayed. The acquisition date and time 411f stores the acquisition date and time of the backup corresponding to the entry. The object key 411g stores the identification information (object key) of the object of the object storage service 400 that stores the backup data of the volume corresponding to the entry. If multiple objects are used in the backup corresponding to the entry, multiple object keys are associated with the entry. The object size 411h stores the size of the object corresponding to the object key in the object key 411g of the entry.

[0079] Next, the managed storage registration TBL 3121 will be described.

[0080] FIG. 10 is a configuration diagram of the managed storage registration TBL according to the first embodiment.

[0081] The managed storage registration table 3121 manages information on storage that is the subject of management by the management terminal 300. The managed storage registration table 3121 stores an entry for each managed storage. The entry of the managed storage registration table 3121 includes fields for a device name 3121a, a device type 3121b, and a CTL IP address / backet name 3121c.

[0082] The device name 3121a stores a device name defined by the user for the storage corresponding to the entry. The device type 3121b stores the device type of the storage corresponding to the entry. Device types include, for example, block storage, SDS, object store, etc. The CTL IP address / bucket name 3121c stores the IP address or bucket name of the controller (CTL) of the storage corresponding to the entry.

[0083] Next, the transfer path information TBL 3122 will be described.

[0084] FIG. 11 is a diagram showing the configuration of the transfer path information TBL according to the first embodiment.

[0085] The transfer path information TBL 3122 manages information on paths between a plurality of devices. The transfer path information TBL 3122 stores an entry for each path. The entry of the transfer path information TBL 3122 includes fields for a path # 3122a, a source device name 3122b, a destination device name 3122c, and an average transfer speed 3122d.

[0086] The route #3122a stores the route number corresponding to the entry. The source device name 3122b stores the device name of the source of the route corresponding to the entry. The destination device name 3122c stores the device name of the destination of the route corresponding to the entry. The average transfer speed 3122d stores the average transfer speed of the route corresponding to the entry.

[0087] Next, the VOL information TBL 3123 will be described.

[0088] FIG. 12 is a diagram showing the configuration of the VOL information TBL according to the first embodiment.

[0089] The VOL information TBL 3123 manages information on volumes stored in each storage device. The VOL information TBL 3123 stores an entry for each volume. An entry in the VOL information TBL 3123 includes fields for device name 3123a, VOL# 3123b, and size 3123c.

[0090] The device name 3123a stores the device name of the device that stores the volume corresponding to the entry. The VOL# 3123b stores identification information (VOL#) of the volume corresponding to the entry. In this embodiment, the VOL# is, for example, a unique value within the device. The Size 3123c stores the size of the volume corresponding to the entry.

[0091] Next, the backup information TBL 3124 will be described.

[0092] FIG. 13 is a diagram showing the configuration of the backup information TBL according to the first embodiment.

[0093] The backup information TBL 3124 manages information related to backups. The backup information TBL 3124 is created based on the information in the catalog information 411 of the object storage service 400. The backup information TBL 3124 stores an entry for each backup. An entry in the backup information TBL 3124 includes fields for a backup ID 3124a, a source device 3124b, a source VOL 3124c, and an acquisition date and time 3124d.

[0094] The Backup ID 3124a stores the ID of the backup (Backup ID) corresponding to the entry. The Source Device ID 3124b stores the ID of the device (Backup source device) that stores the volume to be backed up corresponding to the entry. The Source VOL # 3124c stores the ID of the volume to be backed up corresponding to the entry (Backup source VOL). The Acquisition Date and Time 3124d stores the acquisition date and time of the backup corresponding to the entry.

[0095] Next, the optimal copy path determination TBL 3125 will be described.

[0096] FIG. 14 is a configuration diagram of the optimal copy path determination TBL according to the first embodiment.

[0097] The optimal copy path determination TBL 3125 manages information for determining the optimal copy path. The optimal copy path determination TBL 3125 includes fields for normal COPY 3125a and similar data utilization COPY 3125e.

[0098] Normal COPY 3125a stores a field for information about normal COPY, which copies data to a copy destination using the data to be COPY. In the example of Fig. 1, normal COPY is a process of copying data of Vol#a from storage device 200 to storage device 500 using path 3.

[0099] The normal COPY 3125a includes fields of a normal COPY target size 3125b, a normal COPY path speed 3125c, and a normal COPY expected time 3125d.

[0100] The size of the data to be copied is stored in the normal COPY target size 3125b. The communication speed of the path used for normal COPY (path 3 in the example of FIG. 1) is stored in the normal COPY path speed 3125c. The time expected to be required for normal COPY (expected normal COPY time) is stored in the normal COPY estimated time 3125d.

[0101] Similar data utilization COPY 3125e stores a field of information related to similar data utilization COPY, which creates data in the COPY destination by using similar data to the data to be COPY. In the example of Fig. 1, similar data utilization COPY is a process of copying similar data to the data to be COPY to the storage device 500 via path 2, transferring the difference between the data to be COPY and the similar data via paths 1 and 2, and generating the data to be COPY in the storage device 500.

[0102] The similar data utilization COPY 3125e includes fields of a similar data size 3125f, a similar data transfer path speed 3125g, a differential data size 3125h, a differential data transfer path 1 speed 3125i, a differential data transfer path 2 speed 3125j, and a similar data utilization COPY predicted time 3125k.

[0103] The similar data size 3125f stores the size of the similar data. The similar data transfer path speed 3125g stores the communication speed of the path used to transfer the similar data (path 2 in the example of FIG. 1). The differential data size 3125h stores the size of the differential data that is the difference between the data to be copied and the similar data. The differential data transfer path 1 speed 3125i stores the transfer speed of the first path (path 1 in the example of FIG. 1) among the paths used to transfer the differential data. The differential data transfer path 2 speed 3125j stores the transfer speed of the second path (path 2 in the example of FIG. 1) among the paths used to transfer the differential data. The similar data utilization COPY expected time 3125k stores the time expected to be required for the similar data utilization COPY.

[0104] Next, the processing operation in the computer system 1 will be described.

[0105] First, the volume copy process will be explained.

[0106] FIG. 15 is a flowchart of a volume copy process according to the first embodiment.

[0107] The volume copy control program 3111 of the management terminal 300 receives a volume copy instruction (an example of a volume COPY instruction or generation instruction) from the administrator (step S100). The volume COPY instruction includes information for identifying the volume to be copied (the volume to be copied) (for example, the VOL# of the volume to be copied, the device name of the storage of the copy source (copy source storage)) and information for identifying the copy destination (the device name of the storage of the copy destination (copy destination storage)). Next, the similar data search program 3112 determines whether data (similar data) similar to the data of the volume to be copied (the data to be copied) included in the volume COPY instruction exists in a storage other than the source storage in the computer system 1 (step S200). In this embodiment, the similar data search program 3112 determines whether backup data exists for the volume to be copied by referring to the Backup information TBL 3124.

[0108] As a result, if it is determined that similar data does not exist in any device other than the source storage (step S200: No), the volume copy control program 3111 executes a process (normal COPY: first transmission process) to transfer (copy) the data of the volume to be copied from the source storage to the destination storage (step S800), and terminates the process.

[0109] On the other hand, when it is determined that similar data exists in a device other than the copy source storage (step S200: Yes), the volume copy control program 3111 determines whether or not the difference between the similar data and the copy target data can be extracted in the copy source storage (step S300). Specifically, the volume copy control program 3111 inquires of the copy source storage whether or not the difference from the copy target data can be extracted, and receives a response from the copy source storage. In the copy source storage that has received the inquiry about whether or not the difference between the copy target data and the similar data can be extracted, the difference extraction availability determination program 2236 determines whether or not the difference between the copy target data and the similar data can be extracted, and returns the determination result to the volume copy control program 3111. Here, whether or not the difference between the copy target data and the similar data can be extracted may be determined based on whether or not the copy source storage stores the same data as the similar data.

[0110] As a result, if it is determined that it is not possible to extract the difference between the similar data and the data to be copied (step S300: No), the volume copy control program 3111 advances the process to step S800 and executes a normal COPY.

[0111] On the other hand, if it is determined that the difference between the similar data and the copy target data can be extracted (step S300: Yes), the volume copy control program 3111 executes a normal COPY required time prediction process to predict the required time (first predicted time) for normal COPY (step S400). Next, the volume copy control program 3111 executes a similar data utilizing COPY required time prediction process (step S500) to predict the required time (second predicted time) for similar data utilizing COPY (second transmission process) that creates the copy target data in the copy target storage by transferring the similar data to the copy destination device and transferring the difference between the similar data and the copy target data to the copy destination device.

[0112] Next, the volume copy control program 3111 judges whether the normal copy or the similar data utilization copy will be completed in a shorter time (step S600).

[0113] As a result, if it is determined that the similar data utilizing COPY will be completed in a shorter time, for example, if the predicted time required for the similar data utilizing COPY is shorter than the predicted time required for a normal COPY (step S600: similar data utilizing COPY), the volume copy control program 3111 executes a similar data utilizing COPY process (step S700) to execute the similar data utilizing COPY, and terminates the volume copy process.

[0114] On the other hand, if it is determined that the normal COPY will be completed in a shorter time, for example, if the predicted time required for the similar data utilization COPY is not shorter than the predicted time required for the normal COPY (step S600: normal COPY), the volume copy control program 3111 advances the process to step S800 and executes the normal COPY.

[0115] Next, the normal COPY required time prediction process (S400) will be described.

[0116] FIG. 16 is a flowchart of the normal COPY required time prediction process according to the first embodiment.

[0117] The optimal copy path determination program 3113 references the VOL information table 3123 and acquires the size of the volume to be copied (step S410). Next, the optimal copy path determination program 3113 references the transfer path information TBL 3122 and acquires the transfer speed of the path to be used (path 3 in this example) from the source storage to the destination storage (step S420).

[0118] Next, the optimum copy path determination program 3113 calculates the expected time by dividing the size of the volume to be copied by the transfer speed (step S430), stores the calculation result in the normal COPY expected time 3125d of the optimum copy path determination TBL 3125 (step S440), and terminates the process.

[0119] Next, the similar data utilizing COPY required time prediction process (S500) will be described.

[0120] FIG. 17 is a flowchart of the similar-data-utilizing COPY required time prediction process according to the first embodiment.

[0121] The optimal copy path determination program 3113 references the catalog information 264 acquired from the object storage service 400, and acquires the size of the similar data (backup size) (step S510).

[0122] Next, the optimal copy path determination program 3113 refers to the transfer path information TBL 3122, acquires the transfer speed of the path (path 2 in this example) from the storage that stores similar data (object storage service 400 in this example) to the destination storage (storage device 500) (step S520), and calculates the backup restoration time [A] by dividing the backup size by the transfer speed (step S530).

[0123] Next, the optimal copy path determination program 3113 requests the source storage to calculate the amount of difference between the copy target volume and similar data (step S540). Here, the calculation request includes, for example, the Vol# and SS# of the copy target volume.

[0124] The differential data extraction program 2235 of the copy source storage (storage device 200) calculates the amount of difference between the copy target volume and the similar data, and returns the calculated amount of difference to the optimal copy path determination program 31113 of the management terminal 300 (step S550).

[0125] The optimal copy path determination program 3113 obtains the differential amount returned from the source storage, and refers to the transfer path information TBL 3122 to obtain the transfer speed of the path (path 1) from the source storage to the object storage service 400, and the transfer speed of the path (path 2) from the object storage service 400 to the destination storage (step S560).

[0126] Next, the optimal copy path determination program 3113 calculates the time required to transfer the differential data (differential transfer time [B]) (step S570). Specifically, the optimal copy path determination program 3113 calculates the differential transfer time by adding together the value obtained by dividing the amount of difference by the transfer time of path 1 and the value obtained by dividing the amount of difference by the transfer speed of path 2.

[0127] Next, the optimal copy path determination program 3113 determines the total value of the backup restoration time [A] and the differential transfer time [B] as the similar data utilizing COPY predicted time (step S580), and ends the process.

[0128] Next, the similar data utilization COPY process (S700) will be described.

[0129] FIG. 18 is a flowchart of the similar data utilizing COPY process according to the first embodiment.

[0130] The volume copy control program 3111 of the management terminal 300 displays on the screen that VOL copy has started (step S701), and also displays on the screen that similar data is being transferred (step S702).

[0131] Next, the volume copy control program 3111 sends a backup restoration instruction to the copy destination storage (step S703). The volume copy control program 3111 includes in the backup restoration instruction the VOL# of the copy destination and the backup ID (SS#) of the data to be backed up (similar data).

[0132] When the destination storage receives the backup restoration instruction, the volume copy program 2237 restores the data of the backup ID included in the backup restoration instruction to the volume of the destination VOL# (step S704) and returns a completion response.

[0133] When the volume copy control program 3111 of the management terminal 300 receives a completion response corresponding to the backup restoration, it displays on the screen that the difference is being transferred (step S705), and sends a difference transfer instruction to the copy source storage (step S706). The volume copy control program 3111 includes in the difference transfer instruction the VOL# of the copy source and the backup ID (SS#) of the data from which the difference is to be extracted.

[0134] When the source storage receives the instruction for differential transfer, the differential data extraction program 2235 extracts the difference between the source VOL# volume included in the instruction for differential transfer and the data of the backup ID (step S707), transfers the extracted differential data to the object storage service 400 as a backup, and returns the backup ID (SS#) of the data storing the difference to the volume copy control program 3111 of the management terminal 300 (step S708).

[0135] The volume copy control program 3111 of the management terminal 300 sends an instruction to restore the differential data to the copy destination storage (step S709). The volume copy control program 3111 includes the copy destination VOL# and the backup ID (SS#) of the differential data in the instruction to restore the differential.

[0136] When the destination storage receives the instruction to restore the difference, the volume restore program 2233 restores the difference data of the backup ID included in the instruction to restore the difference to the destination volume VOL#, thereby generating data of the volume to be copied in the destination volume (step S704), and returns a completion response. This completes the copying of the volume to the destination storage.

[0137] Next, when the volume copy control program 3111 of the management terminal 300 receives a response indicating that the differential restoration has been completed, it displays on the screen that the VOL copy has been completed (step S711), and ends the similar data utilization COPY process.

[0138] According to the computer system 1 of the first embodiment described above, when it is determined that it is faster to generate (copy) data using similar data than to simply copy the data to be copied, the data can be generated using the similar data, thereby enabling data generation to be performed quickly.

[0139] Next, a computer system 1A according to the second embodiment will be described. In the computer system 1A, the same components as those in the computer system 1 according to the first embodiment are denoted by the same reference numerals.

[0140] FIG. 19 is a diagram illustrating an overview of a data copy process in the computer system according to the second embodiment.

[0141] The data copy process of Figure 19 is a data copy process when copying data of volume (Vol) #a of the storage device 200 in the on-premises data center 10A to volume (Vol) #b of the storage device 500 in the public cloud 20. In this embodiment, it is assumed that backup data of Vol #a at a certain point in the past is stored in the storage device 200 in the on-premises data center 10A and the object storage service 400 in the public cloud 20.

[0142] The computer system 1A includes a management terminal 300A. The management terminal 300A differs from the management terminal 300 according to the first embodiment in part of the processing by the volume copy control program 3111 and the optimal copy path determination program 3113. Specifically, in the volume copy processing, instead of the similar data utilizing COPY required time prediction processing (S500), a similar data utilizing COPY required time prediction processing (S800) is performed, and instead of the similar data utilizing COPY processing (S700), a similar data utilizing COPY processing (S900) is executed.

[0143] When the management terminal 300A receives an instruction from the administrator to copy data in Vol#a of the storage device 200 (source storage) to Vol#b of the storage device 500 (destination storage) (FIG. 19(1)), it searches for data similar to the data to be copied (similar data) and determines the method of transferring the data (FIG. 19(2)). Here, the subsequent processing will be explained assuming that similar data exists in the object storage service 400 and it has been determined that the data should be copied using a method of transferring the similar data from the object storage service 400 to the storage device 500.

[0144] The management terminal 300A instructs the destination storage (storage device 500 in the example of FIG. 19) to copy the found similar data (FIG. 19(3)). In accordance with this instruction, the storage device 500 reads the instructed similar data (backup data of Vol#a) from the object storage service 400 and stores it in Vol#b (FIG. 19(4)).

[0145] Next, the management terminal 300A instructs the source storage (storage device 200 in the example of FIG. 19) to extract the difference between the target data and the similar data and transfer the difference to the destination storage (FIG. 19(5)). In accordance with this instruction, the source storage (storage device 200) compares Vol#a with the backup data of Vol#a to extract the difference data (FIG. 19(6)), and transfers the difference data to the destination storage (FIG. 19(7)).

[0146] In response to this, the destination storage creates a copy of Vol#a by reflecting the differential data in the backup data of Vol#a, thereby completing the copying of data of Vol#a to Vol#b in the storage device 500.

[0147] Next, the similar data utilizing COPY required time prediction process (S800) according to the second embodiment will be described.

[0148] Fig. 20 is a flowchart of the similar data utilizing COPY required time prediction process according to the second embodiment. Note that the same reference numerals are used for the same parts as those in the similar data utilizing COPY required time prediction process according to the first embodiment shown in Fig. 17, and duplicated explanations may be omitted.

[0149] When the optimum copy path determination program 3113 acquires the differential amount returned from the copy source storage, it refers to the transfer path information TBL 3122 and acquires the transfer speed of the path (path 3) from the copy source storage to the copy destination storage (step S810).

[0150] Next, the optimal copy path determination program 3113 calculates the time required to transfer the differential data (differential transfer time [B]) (step S820). Specifically, the optimal copy path determination program 3113 calculates the differential transfer time by dividing the amount of difference by the transfer speed of path 3.

[0151] Next, the optimal copy path determination program 3113 determines the total value of the backup restoration time [A] and the differential transfer time [B] as the similar data utilizing COPY predicted time (step S580), and ends the process.

[0152] This similar data utilization COPY required time prediction process makes it possible to appropriately predict the expected time for the similar data utilization COPY process, which transfers similar data from the object storage service 400 to the destination storage and transfers differential data from the source storage to the destination storage.

[0153] Next, the similar data utilizing COPY process (S900) according to the second embodiment will be described.

[0154] Fig. 21 is a flowchart of the similar data utilization COPY process according to the second embodiment. Note that the same reference numerals are used to designate the same parts as those in the similar data utilization COPY process according to the second embodiment shown in Fig. 18, and duplicated explanations may be omitted.

[0155] When the source storage receives the instruction for differential transfer, the differential data extraction program 2235 extracts the difference between the volume of the source VOL# included in the instruction for differential transfer and the data of the backup ID (step S707). Next, the volume copy program 2237 transfers the extracted differential data and information indicating the write position in the volume for the differential data (write position information) to the destination storage (step S910).

[0156] Upon receiving the differential data, the volume restore program 2233 of the destination storage creates a volume to be copied by storing the differential data in the destination volume based on the write position information (step S920) and sends a completion response, thereby completing the copying of the volume to the destination storage.

[0157] When the volume copy program 2237 of the source storage receives the storage completion response from the destination storage, it sends a VOL copy completion response to the management terminal 300A (step S930).

[0158] Next, a computer system 1B according to the third embodiment will be described. In the computer system 1B, the same components as those in the computer system 1 according to the first embodiment are denoted by the same reference numerals.

[0159] FIG. 22 is a diagram illustrating an overview of a data copy process in the computer system according to the third embodiment.

[0160] The data copy process of FIG. 22 is a data copy process when copying (restoring) a volume (Vol#a) of the storage device 200A of the on-premise data center 10B at a predetermined time to the storage device 200B of the on-premise data center 10B. In this embodiment, the storage device 200A of the on-premise data center 10B stores backup data (backup (base)) that is a backup of the entire Vol#a at a certain time in the past, and the object storage service 400 of the public cloud 20 stores the same backup data as the storage device 200A, as well as differential data (backup (differential)) of Vol#a prior to a predetermined time. In this embodiment, the base backup data is defined as backup #0 (base), and the differential data at the nth backup time thereafter is defined as backup #n (differential).

[0161] The computer system 1B comprises a management terminal 300B. The management terminal 300B differs in part from the management terminal 300 according to the first embodiment in its processing.

[0162] The management terminal 300B receives an instruction from the administrator to copy (create) data of Vol#a of storage device 200A at a specific backup point in time (point in time n in this case) to Vol#c of storage device 200B (destination storage) (FIG. 22(1)). Here, in this embodiment, the copy instruction includes, for example, the Vol# to be restored, information (backup#) indicating the backup point in time to be restored, and the Vol# of the copy destination.

[0163] The management terminal 300B searches for data similar to the data at the backup point in time of the restoration target (similar data) and determines the method of transferring the data (FIG. 22(2)). Here, the storage device 200A has base backup data prior to the backup point in time n of the restoration target, i.e., Vol#a backup#0, and therefore this data is searched for as similar data, and the subsequent processing will be described assuming that it has been determined that the data will be copied using a method of transferring similar data from storage device 200A to storage device 200B.

[0164] The management terminal 300B instructs the storage device 200A to transfer the found similar data (FIG. 22(3)). In accordance with this instruction, the storage device 200A reads the instructed similar data (backup #0 of Vol#a) and stores it in Vol#c of the copy destination storage (storage device 200B) (FIG. 22(4)).

[0165] Next, the management terminal 300B instructs the destination storage (storage device 200B) to transfer the difference between the data to be restored and the similar data (FIG. 22(5)). In accordance with this instruction, the destination storage (storage device 200B) transfers the difference data from the similar data up to the backup point in time to be restored from the object storage service 400 (FIG. 22(6)). For example, if the backup point in time is #n, the destination storage (storage device 200B) transfers the backup data of backup #1 to backup #n from the object storage service 400. Here, backup #0 to backup #n correspond to the basic data corresponding to the data to be copied.

[0166] Next, the destination storage generates data of Vol#a at a predetermined backup point n by sequentially reflecting backup data from backup #1 to backup #n to the data of backup #0 of Vol#a stored in Vol#c. This completes the generation (copying) of data at the predetermined backup point n to the storage device 200B.

[0167] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

[0168] For example, in the above embodiment, the estimated time for normal copying and similar data utilizing copying are compared, and data is copied using the copy with the shorter estimated time, but the present invention is not limited to this. For example, as shown in Figure 1, if the communication speed of the path between the source storage and destination storage is slow and the communication speed of the path between the storage where the similar data is stored (in Figure 1, object storage service 400) and the destination storage is fast, and it is expected that the processing time will be shorter when similar data utilizing copying is used than when normal copying, then similar data utilizing copying may be executed without predicting and comparing the estimated time for each copy.

[0169] In the above-described embodiments, a part or all of the processing performed by the processor may be performed by a hardware circuit. The programs in the above-described embodiments may be installed from a program source. The program source may be a program distribution server or a storage medium (e.g., a portable storage medium). [Explanation of symbols]

[0170] 1, 1A, 1B... computer system, 10, 10A, 10B... on-premise data center, 20... public cloud, 100... host, 200, 200A, 200B... storage device, 300... management terminal, 400... object storage service, 500... storage device

Claims

1. A data generation control system having a plurality of storage devices and a management device, The management device includes: Accepting designation of target data and a destination storage device that is a destination storage device that generates the target data; Identifying similar data that is similar to the target data to be generated; calculating a first estimated time required for a first transmission process for transmitting basic data corresponding to the data to be generated from a source storage device in which the basic data is stored to the destination storage device, and a second estimated time required for a second transmission process for transmitting the similar data from a similar data storage device in which the similar data is stored to the destination storage device, and transmitting differential data between the basic data and the similar data from the source storage device to the destination storage device; When the second predicted time is shorter than the first predicted time, the similar data and the difference data are transmitted to the destination storage device by the second transmission process, thereby causing the destination storage device to generate the data to be generated; and when the second predicted time is not shorter than the first predicted time, the base data is transmitted to the destination storage device by the first transmission process, thereby causing the destination storage device to generate the data to be generated. Data generation control system.

2. The plurality of storage devices a backup execution storage device that stores backup data at a predetermined point in time for data managed by itself in a backup storage device that is a storage device for backup; The management device includes: Backup data in a backup storage device for the data to be generated is identified as similar data. The data generation control system according to claim 1 .

3. the backup execution storage device also stores the backup data to be stored in the backup storage device in itself; When the source storage device itself is the backup execution storage device, the source storage device identifies differential data based on the basic data and the backup data. The data generation control system according to claim 2 .

4. The management device includes: In the second transmission process, causing the source storage device to transmit the differential data to the backup storage device; The destination storage device acquires the differential data from the backup storage device. The data generation control system according to claim 3 .

5. The management device includes: In the second transmission process, The source storage device transmits the differential data to the destination storage device. The data generation control system according to claim 3 .

6. The source storage device is determining whether the differential data can be extracted and transmitting a result of the determination to the management device; The management device includes: performing the first transmission process when a determination result indicating that the differential data cannot be extracted from the source storage device is received; When a determination result indicating that the differential data can be extracted from the source storage device is received, the method determines whether to execute the first transmission process or the second transmission process. The data generation control system according to claim 1 .

7. The source storage device is Calculating the amount of the differential data and transmitting it to the management device; The management device includes: The second expected time is calculated based on a data amount of the similar data and a data amount of the difference data. The data generation control system according to claim 1 .

8. A data generation control system having a plurality of storage devices and a management device, the communication speed of the path between the third storage device and the second storage device is higher than the communication speed of the path between the first storage device and the second storage device; The management device includes: receiving an instruction to generate, in the second storage device, predetermined data to be generated that is stored in the first storage device, and determining whether or not similar data similar to the data to be generated exists in the third storage device; If the similar data exists in the third storage device, the similar data is transmitted to the second storage device; transmitting differential data between the target data and the similar data to the second storage device; The second storage device generates the target data based on the similar data and the differential data. Data generation control system.

9. A data generation control method for a data generation control system having a plurality of storage devices and a management device, comprising: The management device includes: Accepting designation of target data and a destination storage device that is a destination storage device that generates the target data; Identifying similar data that is similar to the target data to be generated; calculating a first estimated time required for a first transmission process for transmitting basic data corresponding to the data to be generated from a source storage device in which the basic data is stored to the destination storage device, and a second estimated time required for a second transmission process for transmitting the similar data from a similar data storage device in which the similar data is stored to the destination storage device, and transmitting differential data between the basic data and the similar data from the source storage device to the destination storage device; When the second predicted time is shorter than the first predicted time, the similar data and the difference data are transmitted to the destination storage device by the second transmission process, thereby causing the destination storage device to generate the data to be generated; and when the second predicted time is not shorter than the first predicted time, the base data is transmitted to the destination storage device by the first transmission process, thereby causing the destination storage device to generate the data to be generated. Data generation control method.

Images