Storage system and data control method

Abstract

The goal is to minimize the increase in storage consumption at the destination, whether the primary volume is updated for backup purposes or the snapshot volume is updated for virtual machine deployment purposes. [Solution] The source storage node transfers the parent-child differential data to the destination storage node if the difference in parent-child differential data is smaller than the difference in generational differential data, and transfers the generational differential data to the destination storage node if the difference in generational differential data is smaller than the difference in parent-child differential data. The destination storage node then uses the received volume data and the received generational differential data or parent-child differential data to associate the volume and multiple snapshot volumes with the destination storage node.

Description

Technical Field

[0001] The present invention relates to a storage system and a data control method, and is suitably applied to, for example, a storage system related to a technology using a snapshot function. Background Art

[0002] A snapshot function exists as an important function in storage apparatuses. The snapshot function is a function of creating a copy of data stored in the storage apparatus at a certain point in time. In copying by the snapshot function, instead of copying the actual data itself, mapping information between a logical storage area called a volume and a physical storage area such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) is copied. Therefore, copying by the snapshot function has the feature that it can be completed in a short time. Between the created snapshot volume and the primary volume that is the copy source, data sharing and differential management are performed using the mapping information. At this time, inside the storage apparatus, only the storage capacity corresponding to the data capacity of one volume plus the capacity of differential data indicating the difference between volumes is consumed. In addition, the restore processing for restoring the data of the primary volume using the snapshot volume has the feature that it can be processed in a short time by restoring only the differential data indicating the difference between the primary volume and the snapshot volume.

[0003] In general, IT (Information Technology) systems use snapshots with these characteristics for a variety of purposes. The most typical use is to take snapshot volumes as backups of primary volumes used for production operations. In recent years, frequent backups have become necessary as a countermeasure against ransomware, and the snapshot function is suitable for such backup purposes. Furthermore, the snapshot function can also be used for secondary purposes, such as analytical work to analyze the acquired snapshot volumes, i.e., for purposes different from production operations. On the other hand, a completely different use is in VDI (Virtual Desktop Infrastructure), for example, to deploy virtual machines by generating OS (Operating System) drives to be assigned to numerous virtual machines (VMs) from a master volume using the snapshot function.

[0004] Incidentally, in recent years, there has been an increase in storage devices that achieve high scalability by integrating multiple storage nodes, each having a primary volume and a snapshot volume, into a cluster. In such storage devices, I / O processing for volumes and the storage capacity for holding data stored in volumes are often independent for each node. Regarding the snapshot function, in order to maintain capacity consumption only for the differential data showing the difference between volumes, the primary volume and the snapshot volume are placed on the same node.

[0005] Typically, such storage devices implement controls to distribute volumes across each node or to migrate volumes between multiple storage nodes (rebalancing) to eliminate imbalances in storage capacity consumption and computational load associated with I / O processing. If the snapshot function is used, both the primary volume and the snapshot volume are migrated together.

[0006] Patent Document 1 discloses a method for migrating primary volumes and snapshot volumes from a source storage device to a destination storage device. Specifically, the method disclosed in Patent Document 1 fully copies the oldest generation of snapshot volumes from the source (hereinafter referred to as the "oldest generation") to the destination, extracts intergenerational differential data from the oldest generation snapshot volume toward the latest generation snapshot volume, and transfers the intergenerational differential data sequentially to the destination storage device. When the snapshot function is used as a backup for volumes used in production operations, the primary volume is updated from the host, while snapshot volumes are created from the primary volume at regular intervals. Therefore, each snapshot volume is arranged sequentially from the oldest generation to the latest generation, representing the data state of the primary volume at a given moment. Consequently, the method disclosed in Patent Document 1 allows migration to be performed while maintaining the same capacity consumption as the source storage device by sequentially transferring the intergenerational differential data from the oldest generation to the latest generation to the destination storage device. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2009-181206 [Overview of the project] [Problems that the invention aims to solve]

[0008] As mentioned above, when the snapshot function is used for backup purposes, the method disclosed in Patent Document 1 can be used without any problems. On the other hand, when the snapshot function is used for deploying virtual machines, different updates are performed on each snapshot volume created from the master volume, and the intergenerational differential data does not have any order. For this reason, when the snapshot function is used for deploying virtual machines, the method disclosed in Patent Document 1 resulted in increased storage capacity consumption at the migration destination.

[0009] This invention was made with the above points in mind, and aims to propose a storage system that can suppress the increase in storage space consumption at the migration destination, whether the primary volume is updated for backup purposes or the snapshot volume is updated for deploying virtual machines. [Means for solving the problem]

[0010] To solve the above problems, the present invention provides a storage system comprising a plurality of storage nodes having processors, wherein each storage node comprises a volume and a plurality of snapshot volumes which are snapshots of the volume, and when migrating the volume and the plurality of snapshot volumes from a source storage node to a destination storage node, the source storage node transfers the data of the volume to the destination storage node, compares the sum of the difference amounts of parent-child difference data indicating the generational difference between the volume to be migrated and the plurality of snapshot volumes with the sum of the difference amounts of intergenerational difference data indicating the differences between the plurality of snapshot volumes, and if the difference amount of parent-child difference data is smaller than the difference amount of intergenerational difference data, the parent-child difference data is transferred to the destination storage node, and if the difference amount of intergenerational difference data is smaller than the difference amount of parent-child difference data, the intergenerational difference data is transferred to the destination storage node, and the volume and the plurality of snapshot volumes are associated with the destination storage node using the intergenerational difference data or the parent-child difference data.

[0011] In the present invention, in a storage system comprising a plurality of storage nodes having processors, each storage node comprises a volume and a plurality of snapshot volumes which are snapshots of the volume, the snapshot volumes are created by duplicating the volume, are writable to the snapshot volumes after creation, and have a snapshot image at the time of creation of the snapshot volume and a snapshot image before writing to the snapshot volume, and when migrating the volume and the plurality of snapshot volumes from a source storage node to a destination storage node, the source storage node transfers the data of the volume, intergenerational difference data showing the difference between the snapshot volumes at the time of creation of the plurality of snapshots, and parent-child difference data which are the data written to the snapshot volumes after creation to the destination storage node, and the destination storage node uses the received data of the volume, the received intergenerational difference data, and the received parent-child difference data to associate the volume and the plurality of snapshot volumes at the destination storage node.

[0012] In the present invention, in a storage system comprising a plurality of storage nodes each having a processor, each storage node comprises a volume and a plurality of snapshot volumes which are snapshots of the volume, and when migrating the volume and the plurality of snapshot volumes from a source storage node to a destination storage node, the source storage node transfers the data of the volume and parent-child difference data indicating the difference between the volume and the plurality of snapshot volumes to the destination storage node, and the destination storage node uses the received data of the volume and the received parent-child difference data to create the volume and the plurality of snapshot volumes on the destination storage node. [Effects of the Invention]

[0013] According to the present invention, even when the primary volume is updated for backup purposes, or when the snapshot volume is updated for the purpose of deploying virtual machines, it is possible to suppress the increase in storage space consumption at the migration destination. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows an example of a system configuration including a storage system according to the first embodiment. [Figure 2] This is a block diagram showing an example of the logical configuration of a storage system according to the first embodiment. [Figure 3] This figure shows an overview of the process as a comparative example for migrating primary volumes and snapshot volumes from a source storage node to a destination storage node. [Figure 4] This is a functional block diagram showing a logical configuration example of a storage node according to the first embodiment. [Figure 5] Figure 4 shows an example of the configuration of the storage node management table. [Figure 6] Figure 4 shows an example of the configuration of the volume management table. [Figure 7] This figure shows an example of the configuration of the volume control table shown in Figure 4. [Figure 8] Figure 4 shows an example of the configuration of the snapshot generation management table. [Figure 9] This is a flowchart illustrating an example of the snapshot migration process. [Figure 10] This is a diagram illustrating the effects of the storage system according to the first embodiment. [Figure 11] This is a functional block diagram showing a logical configuration example of a storage node according to the second embodiment. [Figure 12] This figure shows an example configuration of a volume control table according to the second embodiment. [Figure 13] This is a diagram showing a configuration example of a mapping table according to the second embodiment. [Figure 14] This is a flowchart showing an example of the procedure of snapshot migration processing according to the second embodiment. [Figure 15] This is a diagram for explaining the effect of the second embodiment. [Figure 16] This is a block diagram showing a configuration example of the entire system including a storage system according to the third embodiment. [Figure 17] This is a block diagram showing an example of the logical configuration of the storage system according to the third embodiment. [Figure 18] This is a block diagram showing an example of the logical configuration of the storage system and the management server according to the third embodiment. [Figure 19] This is a block diagram showing a configuration example of a storage system management table according to the third embodiment. [Figure 20] This is a diagram showing a configuration example of a volume management table according to the third embodiment. [Mode for Carrying Out the Invention]

[0015] Hereinafter, the present embodiment of the present invention will be described in detail based on the drawings. However, the following description and drawings are examples for describing the present invention, are appropriately omitted and simplified for clarity of description, and do not limit the technical scope of the present invention.

[0016] Further, in the following description, expressions such as "xxx table" may be used to describe information that allows an output to be obtained for an input, and this information may be data of any structure, or may be a learning model such as a neural network that generates an output for an input. Accordingly, the "xxx table" can also be referred to as "xxx information".

[0017] Furthermore, in the following explanation, the configuration of each table is an example; one table may be divided into two or more tables, or all or part of two or more tables may constitute one table. Also, in the following explanation, the processing may be described with "program" as the subject, but since a program is executed by the processor unit and performs defined processing using the storage unit and / or interface unit as appropriate, the subject of the processing may be the processor unit (or a device such as a controller having that processor unit).

[0018] The program may be installed on a device such as a computer, or it may reside on a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0019] Furthermore, in the following description, "computer system" refers to a system that includes one or more physical computers. A physical computer may be a general-purpose computer or a dedicated computer. A physical computer may function as a computer that issues I / O (Input / Output) requests (e.g., a host computer) or as a computer that performs I / O of data in response to I / O requests (e.g., a storage device).

[0020] In other words, a computer system may consist of at least one of the following: a host system, which is one or more host computers that issue I / O requests; and a storage system, which is one or more storage devices that perform I / O of data in response to I / O requests. One or more virtual computers (e.g., VMs (Virtual Machines)) may be running on at least one physical computer. A virtual computer may be a computer that issues I / O requests, or a computer that performs I / O of data in response to I / O requests.

[0021] Furthermore, the computing system may be a distributed system consisting of one or more (typically more) physical node devices. A physical node device is a physical computer.

[0022] Furthermore, an SDx (Software-Defined anything) may be built on a physical computer (e.g., a node device) or on a computer system including that physical computer by having the computer execute predetermined software. As an SDx, for example, an SDS (Software Defined Storage) or an SDDC (Software-defined Datacenter) may be adopted.

[0023] For example, a storage system as SDS may be constructed by running software with storage capabilities on a physical general-purpose computer.

[0024] Furthermore, at least one physical computer (e.g., a storage device) may run one or more virtual computers as a host system and a virtual computer as a storage controller for the storage system (typically a device that inputs and outputs data to and from storage devices in response to I / O requests).

[0025] In other words, such at least one physical computer may have both the function of being at least part of a host system and the function of being at least part of a storage system.

[0026] Furthermore, a computer system (typically a storage system) may have redundant configuration groups. The redundant configuration may consist of multiple node devices, such as Erasure Coding, RAIN (Redundant Array of Independent Nodes), and inter-node mirroring, or it may consist of a single computer (e.g., a node device), such as one or more RAID (Redundant Array of Independent (or Inexpensive) Disks) groups as at least part of the storage devices.

[0027] Furthermore, in the following explanation, "dataset" refers to a single logical block of electronic data as viewed by a program such as an application program, and may be, for example, a record, a file, a key-value pair, or a tuple.

[0028] Furthermore, in the following explanation, identification numbers are used as identification information for various objects, but other types of identification information (for example, identifiers including letters or symbols) may also be used.

[0029] Furthermore, in the following explanations, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used, and when describing similar elements with distinction, the element's identification number (or reference code) may be used.

[0030] For example, when describing a "page," which is a unit of memory space, without making any particular distinction, it may be written as "page 312." When describing individual pages separately, the page number may be used, such as "page #0" or "page #1," or a reference number may be used, such as "page 312A" or "page 312B." In this embodiment, "#" means an identifier, such as a number.

[0031] (1) First Embodiment The first embodiment will be described below with reference to Figures 1 to 10. Figure 1 shows an example of a system configuration including a storage system 100 according to the first embodiment. The storage system 100 consists of two or more storage nodes 101. The host 102 sends an I / O request (write request or read request) to the storage node 101, specifying the I / O destination (for example, a logical volume number such as a LUN (Logical Unit Number) or a logical address such as an LBA (Logical Block Address)).

[0032] The storage system 100 and the host 102 do not need to be located in the same place; for example, the storage node 101 and the host 102 may be located in different data centers. Similarly, the storage node 101 and the host 102 may be physically composed of a single device. For example, a configuration such as HCI (Hyper Converged Infrastructure) may be used, where the functions of both the storage node 101 and the host 102 are housed in a single physical server. Furthermore, in this embodiment, a larger and more hierarchical storage system 100 may be configured by treating the storage system 100, which is configured by clustering the storage nodes 101, as a single storage node 101.

[0033] The storage node 101 and host 102 are computers, each including a CPU 104, memory 105, storage device 106, and communication port 107, respectively. However, the exact configuration is not required as long as they fulfill their respective roles as described later. The storage node 101 and host 102 may be equipped with multiple instances of each component, or with components other than those mentioned above. For example, multiple hard disk drives may be installed as storage devices 503 for redundancy, or an accelerator such as a GPU (Graphics Processing Unit) may be installed to take over some of the processing performed by the CPU 104. Each component may not be physical hardware, but virtual hardware based on virtualization technologies such as virtual machines or containers. Similarly, the system may be configured using services that execute programs without being aware of the actual computing environment, such as Function-as-a-Service or Serverless Computing.

[0034] The storage nodes 101 and hosts 102 that make up the storage system 100 are each connected to the network 103 and can communicate with each other. The network 103 is implemented by any communication line or combination thereof, such as Ethernet, InfiniBand, or optical fiber. In addition, it may include not only a LAN (Local Area Network) closed within the data center, but also a WAN (Wide Area Network) such as the Internet, or a virtual network inside a computer. Although not shown in the diagram, network devices such as network switches, routers, and gateways may be included as needed. In the illustrated example, all components are connected to a single network 103, but a dedicated network may be provided for use between specific components. For example, the storage system 100 may have a dedicated network to speed up communication between storage nodes 101.

[0035] Figure 2 is a block diagram showing an example of the logical configuration of the storage system 100 according to the first embodiment. The storage system 100 is a cluster of multiple storage nodes (two storage nodes in the illustrated example) 101A and 101B. In this embodiment, these multiple storage nodes 101A and 101B may also be collectively referred to as storage node 101. Each storage node 101 provides a logical storage area called a volume to the host 102. The host 102 reads and writes data to the volumes provided by the storage system 100 and the storage nodes 101 according to the various processes being executed on the host 102.

[0036] On the source storage node 101A, the source primary volume 202A is a normal volume. For example, at the user's instruction, source snapshot volumes 203A1 to 203A4 are taken as source snapshot volumes for the source primary volume 203A. On the destination storage node 101B, source snapshot volumes 203B1 to 203B4 are taken as source snapshot volumes for the destination primary volume 202B. Note that primary volumes 202A and 202B may also be collectively referred to as source primary volume 202, and snapshot volumes 203A1 to 203A4 and 203B1 to 203B4 may also be collectively referred to as source snapshot volume 203.

[0037] Snapshot volume 203 is a volume replicated using the snapshot function based on primary volume 202. In this embodiment, the primary volume may also be referred to as "P-VOL," and the snapshot volume may also be referred to as "SS-VOL." P-VOL202 and SS-VOL203 are provided to host 102 and are volumes that can be read from and written to by host 102. Note that some implementations of the snapshot function include a cascading function that allows further snapshots to be taken of SS-VOL203, but this is omitted in this embodiment.

[0038] In the illustrated example, four snapshot volumes, SS-VOL203A1 to A4, exist for P-VOL202, and each is receiving I / O from host 102. The illustrated example shows an example of the operation to migrate P-VOL202A and SS-VOL203A1 to A4 of snapshot group 201A1 of source storage node 101A to storage node 101B, for example, migrating SS-VOL203A4 from source storage node 101A to storage node 101B. This migration process will be described later. In this embodiment, P-VOL202A and P-VOL202B, described later, are also collectively referred to as "P-VOL202".

[0039] Snapshot group 201 is a group representing a collection of volumes where data differential management (i.e., management of data sharing relationships using mapping information) is performed using the snapshot function, and it includes one P-VOL202 or one or more SS-VOL203.

[0040] The differential extraction function unit 204 of storage node 101A is part of the snapshot function and is a function that extracts differential areas where the data differs between multiple volumes (P-VOL202 and SS-VOL203) within any volume in a single node snapshot group 201. In this embodiment, since differential management is performed by the snapshot function for volumes within a single node snapshot group 201, it is possible to extract differences at high speed without comparing the actual data. The differential management described above is, for example, the management of data sharing relationships using mapping information. This function will be used in the migration process described later.

[0041] In this embodiment, it is assumed that during the migration process, I / O requests from host 102 to SS-VOL203A4 will be temporarily stopped, and then resumed using SS-VOL203B4 after the migration process is complete. Therefore, it is not possible to accept I / O requests from host 102 while the migration process is running. For example, it is possible to maintain I / O reception from host 102 during the migration by replicating the I / O requests received during the migration process to SS-VOL203A4 and SS-VOL203B4. In this embodiment, as an example, the explanation will be given on the premise that I / O reception is temporarily stopped.

[0042] Figure 3 shows an overview of the process as a comparative example for migrating P-VOL202A and SS-VOL203A1~A3 from source storage node 101A to destination storage node 101B. Figure 3 shows a comparative example with the process described later.

[0043] In the illustrated example, the migration process progresses from time T1 to T2. Before the migration process, data sets A0, B0, and C0 are written to P-VOL202A from host 102. SS-VOL203A1 to A3 are created from this P-VOL202A for the purpose of deploying virtual machines. Of these SS-VOL203A1 to A3, the second generation, SS-VOL203A2, has data sets B1 and C1 written to it from host 102, and the third generation, SS-VOL203A3, has data set A1 written to it from host 102.

[0044] In the illustrated example, for instance, the oldest generation snapshot volume among the source snapshot volumes is fully copied to the destination storage node 101B, and then the intergenerational differential data is extracted from the oldest generation to the newest generation snapshot volume, and the differential data is transferred sequentially to the destination primary volume 202B.

[0045] Therefore, in the state at time T1, first, as procedure (1), the datasets A0, B0, and C0 of the oldest generation snapshot volume, SS-VOL203A1, are fully copied to P-VOL202B on the destination storage node 101B, and SS-VOL203B1 is created from P-VOL202B, thereby completing the migration of the oldest generation (migration from SS-VOL203A1 to SS-VOL203B1).

[0046] Next, as step (2), the datasets B1 and C1 of the difference area between the source SS-VOL203A1 and SS-VOL203A2 are differentially copied to P-VOL202B on the destination storage node 101B, and SS-VOL203B2 is created from P-VOL202B.

[0047] At time T2, the migration process in step (2) has been applied between each generation, and the migration up to P-VOL202A has been completed to the destination storage 101B. In step (3), the intergenerational differential datasets A1, B0, and C0, which show the difference between the source SS-VOL203A2 and SS-VOL203A3, are differentially copied to the destination storage node 101B. However, datasets B0 and C0 are datasets that were already copied in step (1). In step (4), the differential dataset showing the difference between SS-VOL203A3 and P-VOL202A is A0. However, dataset A0 is also a dataset that was already copied in step (1). In this comparative example, datasets A0, B0, and C0 are each transferred twice to the destination storage node 101B, which may increase the consumption of storage capacity at the destination.

[0048] Next, using Figures 4 to 10, this embodiment will be described to address the potential for increased memory capacity consumption at the destination system mentioned above. Figure 4 is a functional block diagram showing a logical configuration example of a storage node 101 according to the first embodiment. Each storage node 101 constituting the storage system 100 includes cluster management information 400, storage node control information 401, and a control program 402.

[0049] Cluster management information 400 is management information for clustering multiple storage nodes 101 and operating them as a single storage system 100. Cluster management information 400 includes a storage node management table 403 and a volume management table 404. The storage node management table 403 is a table for managing the storage nodes 101 that make up the storage system 100. The volume management table 404 is a table for managing the configuration of volumes and snapshots managed by the storage system 100. In practice, cluster management information 400 may be held by all storage nodes 101 that make up the storage system 100, or it may be held only by a representative source storage node 101B.

[0050] The storage node control information 401 is control information used by the control program 402 running on the storage node 101 to perform I / O processing to the volume and the operation of the snapshot function. The storage node control information 401 is unique to each storage node 101. The storage node control information 401 includes a volume control table 405 and a snapshot generation management table 406. Although omitted in the illustrated example, it may also include information for controlling the storage device 106 and information for controlling the communication port 107. The volume control table 405 and the snapshot generation management table 406 are control information necessary for I / O processing to the volume and the operation of the snapshot function.

[0051] The control program 402 is a program that enables I / O processing to the volume and the operation of the snapshot function by controlling the storage device 106 and communication port 107 of the storage node 101 using storage node control information 401. The control program 402 has an I / O function unit 407 and a snapshot function unit 408.

[0052] The I / O function unit 407 provides a volume to the host 102 and processes I / O requests made by the host 102 for that volume. Meanwhile, the snapshot function unit 408 implements snapshot functionality. The snapshot function unit 408 includes a snapshot creation function unit 409, a restore function unit 410, a snapshot migration function unit 411, and a difference extraction function unit 204.

[0053] The snapshot creation function unit 409 creates a new SS-VOL203 based on the specified P-VOL202. The restore function unit 410 performs a restore process to recover data from volumes within snapshot group 201 using the specified volume. The snapshot migration function unit 411 performs a process to migrate P-VOL202 and SS-VOL203 of the specified snapshot group 201 to another storage node 101.

[0054] It should be noted that the configuration of the control program 402 in this embodiment has been subdivided for ease of explanation, and the actual structure may differ significantly. For example, I / O processing on a volume to which the snapshot function is applied requires the I / O function unit 407 and the snapshot function unit 408 to work closely together, and in reality, it is not always possible to clearly divide it into two functions.

[0055] The storage system 100 has various functions in addition to the snapshot function described above, but these are omitted in this embodiment. Furthermore, regarding the IO function unit 407 and the snapshot function unit 408, functions that are not directly related to this embodiment are omitted from the description. In addition, functions for sending and receiving data between multiple storage nodes 101, functions for receiving instructions from users, and functions for keeping information such as the storage node management table 403 and the volume management table 404 up to date are also omitted. The cluster management information 400 and storage node control information 401 also contain various types of information, but these are omitted here.

[0056] The storage system 100 according to this embodiment includes a plurality of storage nodes, each having a processor CPU 104. A source storage node 101A and a destination storage node 101B, as examples of the plurality of storage nodes, each include a volume and a plurality of snapshot volumes, which are snapshots of the volume. Generally, during snapshot migration, transferring intergenerational differential data, which shows the generational differences between multiple source snapshot volumes, results in less data transfer than transferring parent-child differential data, which shows the differences between the source volume to be migrated and the plurality of source snapshot volumes. However, while it is possible to write to a snapshot volume after snapshot creation, in this case the snapshot changes by one generation. For example, the amount of data written and the amount of data to be restored in the next generation result in larger intergenerational differential data. Therefore, in this embodiment, taking advantage of the fact that parent-child differential data transfer amounts are smaller than intergenerational differential data depending on the write to the snapshot, the differential data transferred from the source storage node 101A to the destination storage node 101B is determined by using both intergenerational differential data and parent-child differential data.

[0057] Specifically, when migrating a volume and multiple snapshot volumes from the source storage node 101A to the destination storage node 101B, the source storage node 101A transfers the volume data to the destination storage node 101A. For example, it compares the total amount of difference in parent-child differential data, which shows the difference between the source P-VOL202A to be migrated and multiple source SS-VOL203A1~203A4, with the total amount of difference in intergenerational differential data, which shows the difference between generations among the multiple source SS-VOL203A1~203A4. If the difference in parent-child differential data (or its total amount) is smaller than the difference in intergenerational differential data (or its total amount), the parent-child differential data is transferred to the destination storage node 101B. Furthermore, if the difference in intergenerational differential data (or its total amount) is smaller than the difference in parent-child differential data (or its total amount), the source storage node 101A transfers the intergenerational differential data to the destination storage node 101B. The destination storage node 101B uses the received volume data and the received parent-child differential data to associate the destination P-VOL202B and the multiple destination SS-VOL203B1 to 203B4 with the destination storage node 101B. Normally, the host 102 reads and writes data to the source P-VOL202A, which is the primary volume of the source storage node 101A. However, in this embodiment, for example, it may read and write data to the source P-VOL202B, which is the primary volume of the destination storage node 101B.

[0058] Snapshot volumes are created by duplicating a volume, and it is possible to write to the snapshot volume after it is created. The source storage node 101A manages a snapshot group that includes source P-VOL202A and multiple source SS-VOL203A1~203A4. Source storage node 101A transfers source P-VOL202A and multiple source SS-VOL203A1~203A4 on a snapshot group basis, and creates destination P-VOL202B and multiple destination SS-VOL203B1~203B4 on destination storage node 101B.

[0059] The source storage node 101A compares source P-VOL202A with multiple source SS-VOL203A1~203A4 and extracts parent-child differential data and intergenerational differential data, respectively. The source storage node 101A then determines which parent-child differential data or intergenerational differential data to transfer to destination storage node 101B.

[0060] The source storage node 101A transfers the differential data between multiple source SS-VOL203A1 to 203A4, along with snapshot images of each generation of the volumes and multiple destination SS-VOL203A1 to 203A4, to the destination storage node 101B.

[0061] Figure 5 shows an example configuration of the storage node management table 403 shown in Figure 4. The storage node management table 403 is a table for managing the storage nodes 101 that make up the storage system 100. Each entry in the storage node management table 403 includes, for example, a node ID 500, an IP (Internet Protocol) address 501, capacity information 502, load information 503, and communication bandwidth information 504.

[0062] Node ID (IDentifier) ​​500 is identification information used to uniquely identify storage node 101 within storage system 100. In the illustrated example, node ID 500 is a sequential number, but it does not have to be a sequential number, and IDs (IDentifiers) using characters other than numbers are also acceptable. IP address 501 is connection information for communication between storage nodes 101. Furthermore, IP address 501 is connection information used by host 102 to access the volume provided by the storage node 101.

[0063] In the illustrated example, each storage node 101 has one IP address, but if a storage node 101 has multiple communication ports 107, the connection information for each port will be included. The capacity information 502 indicates the total amount of storage capacity that the storage node 101 can provide to the host 102 and the amount used. The load information 503 indicates the utilization status of computing resources (CPU 104 and memory 105) consumed by the storage node 101 for I / O processing to the volume, etc. The communication bandwidth information 504 indicates the utilization status of communication ports 107 consumed by the storage node 101 for I / O processing from the host to the volume, snapshot migration processing, etc.

[0064] In the illustrated example, storage node 101, whose node ID 500 is "0", has IP address 501 "1.1.1.1", a total storage capacity of "10 TB", of which "2 TB" is in use, "30%" of CPU 104 and "40%" of memory 105 are being used for I / O processing, etc., and communication port 107 is being used at "400 Mbps" in the transmission direction and "200 Mbps" in the reception direction.

[0065] Figure 6 shows an example configuration of the volume management table 404 shown in Figure 4. The volume management table 404 is a table for managing the configuration of volumes and snapshots managed by the storage node 101. Each entry in the volume management table 404 includes, for example, volume number (VOL#) 600, node ID 500, volume capacity 601, volume attribute 602, parent volume number (parent VOL#) 603, and snapshot group # 604. Note that # indicates a number.

[0066] Volume # is information used to uniquely identify a volume within the storage system 100. In the example shown, Volume # is a sequential number, but it does not have to be sequential, and an ID using characters other than numbers is also acceptable. Node ID 500 is information indicating the ID of the storage node 101 where the volume is stored. Volume capacity 601 is information indicating the storage capacity of the volume.

[0067] Volume attribute 602 indicates the purpose for which the volume is used. "P-VOL" indicates that the snapshot function is applied to the volume and that it is P-VOL202. "SS-VOL" indicates that the snapshot function is applied to the volume and that it is SS-VOL203. "-" indicates that the volume does not have any functions such as the snapshot function applied and is a normal volume. Volume attribute 602 may have multiple values. For example, "P-VOL / SS-VOL" indicates that the volume is a snapshot volume created using the snapshot function based on a certain P-VOL202, and that this volume is also a P-VOL that has a certain SS-VOL203.

[0068] Parent volume #603 is information indicating the ID of P-VOL202, the source of the snapshot, when the volume is a snapshot volume, i.e., when it has "SS-VOL" as volume attribute 602. If the volume is a primary volume, or a normal volume to which the snapshot function is not applied, "-" is stored.

[0069] Snapshot group #604 is information used to uniquely identify snapshot group 201 to which the volume belongs. In the example shown, snapshot group #604 is a sequential number, but it does not have to be sequential, and it can also be a number using characters other than numbers. If the volume is a normal volume to which the snapshot function is not applied, "-" is stored. Snapshot group #604 is newly assigned when the first snapshot is taken with a normal volume to which the snapshot function is not applied as the primary volume.

[0070] In the illustrated example, volume #600, which is "0", is stored on storage node 101, which has node ID 500 "0", has a storage capacity of "100 GB", is used as the primary volume in the snapshot function, and belongs to snapshot group 201, which has snapshot group #604 "0".

[0071] Figure 7 shows an example of the configuration of the volume control table 405 shown in Figure 4. The volume control table 405 is control information used by the control program 402 running on the storage node 101 to perform I / O processing to the volume and the operation of the snapshot function. The volume control table 405 is unique to each storage node 101. Each entry in the volume control table 405 includes volume #600, parent volume #603, logical address 700, and physical address 701.

[0072] For volume #600 and parent volume #603, the information is the same as in volume management table 404. Logical address 700 is information that specifies the location of the data to be read or written when host 102 performs I / O on the volume. Physical address 701 is mapping information that indicates where (physical address 701) in storage device 106 the data block at logical address 700 of the volume is stored. If the volume is a snapshot volume, that is, if it has "SS-VOL" as volume attribute 602, then physical address 701 may have parentheses like "(1000)". This indicates that the data at logical address 700 is shared with the primary volume indicated by parent volume #603.

[0073] Normally, immediately after a snapshot is taken, all areas of the snapshot volume are shared with the primary volume. Subsequently, the sharing is released when host 102 writes data to the snapshot volume and the primary volume. The mapping information and actual data management methods in such snapshot functions are implemented using control methods such as Copy on Write and Redirect on Write. Since these are all known technologies, in this embodiment, it is assumed that the control program 402 handles them appropriately as an extension of I / O processing and snapshot operations, and details are omitted.

[0074] In the illustrated example, the volume with volume #600 being "1" indicates that it was created as a snapshot of the primary volume where volume #600 is "0". The data block with logical address 700 being "0" shares data with the primary volume and is stored in the area where physical address 702 is "1000". The data block with logical address 700 being "1" is stored in the area where physical address 701 is "1100". The data block with logical address 700 being "2" indicates that it shares data with the primary volume and is stored in the area where physical address 701 is "1020".

[0075] Figure 8 shows an example configuration of the snapshot generation management table 406 shown in Figure 4. The snapshot generation management table 406 is control information used by the control program 402 running on the storage node 101 to operate the snapshot function. The snapshot generation management table 406 manages the latest generation #801 for each P-VOL#800. This is information unique to each storage node 101. Each entry in the snapshot generation management table 406 includes P-VOL#800, latest generation #801, generation #802, SS-VOL#803, and status 804. The snapshot generation management table 406 manages, for example, 1024 generations per P-VOL (generation #=0~1023).

[0076] In the snapshot generation management table 406, the latest generation #801 is incremented each time a snapshot is created for P-VOL202, and SS-VOL#803 and state 804 corresponding to the latest generation # are updated. State 804 has a COPY state, which indicates that a snapshot is being created. When the snapshot creation process is complete, the state of 804 changes to "-".

[0077] The various types of information handled by the storage system 100 in this embodiment have been described above. The above description includes, for example, information used for control in this embodiment and information that is often included in a general storage system 100. New information may be added to this information as needed, and some unnecessary information may be deleted.

[0078] In the following description, the processing content of the essential parts of the functions constituting the control program 402 in this embodiment will be explained using flowcharts. However, the processing described below mainly refers to the processing that should generally be performed in order to carry out the essential parts of the functions in this embodiment, and in this embodiment, some parts that are not important may be omitted from the explanation. In this embodiment, any processing may be added as needed, and some unnecessary processing may be deleted.

[0079] Figure 9 is a flowchart illustrating an example of the snapshot migration process. This snapshot migration process is primarily performed by the snapshot migration function unit 411 of the snapshot function unit 408. The snapshot migration function unit 411 is activated when the user instructs the storage system 100 to execute the migration process for the snapshot group 201. Alternatively, the control program 402 within the storage system 100 may be configured to periodically monitor the capacity information 502 and load information 503 of the storage node management table 403 and autonomously perform a migration to resolve any imbalances in storage capacity and computational load when detected.

[0080] In this snapshot migration process, first, the snapshot migration function unit 411 obtains information about the target snapshot group 201 from which the data is to be migrated (step S900). The snapshot migration function unit 411 refers to the volume management table 404 and obtains the volume information necessary for migration, such as volume #600, node ID 500, volume capacity 601, volume attribute 602, and parent volume #603 of the target snapshot group #604.

[0081] Next, in step S901, the snapshot migration function unit 411 creates a new volume with the same capacity as the source P-VOL202 and SS-VOL203 on the destination storage node 101, based on the volume capacity information 601 acquired in step S900.

[0082] The difference extraction function unit 204 extracts difference data showing the difference between P-VOL202A of the source storage node 101A and each SS-VOL203Ai (i=0, 1, ...1024). In order to transfer the difference data to the destination storage 101B, the difference extraction function unit 204 first copies all the data of the reference P-VOL202A to the volume created by the destination storage node 101B in step S901 (step S902).

[0083] Subsequently, in step S903, the snapshot migration function unit 411 creates SS-VOL203Bi (i=0, 1, ...1024) from P-VOL202B of the destination storage node 101B. In step S903, the snapshot migration function unit 411 uses the snapshot creation function unit 408 to create SS-VOL203 for each snapshot volume 203 in the snapshot group to be migrated.

[0084] In step S904, the snapshot migration function unit 411 checks whether any unmigrated SS-VOL203s still exist on the source storage node 101. Information on whether the migration is complete may be stored in the volume management table 404 or in the status 804 of the snapshot generation management table 406. If the snapshot migration function unit 411 determines that there are unmigrated SS-VOL203s (step S904: Yes), it executes step S905. However, if it determines that there are no unmigrated SS-VOL203s (step S904: No), that is, if the migration of all SS-VOL203s is complete, it terminates the snapshot migration process.

[0085] In step S905, the snapshot migration function unit 411 selects the newest generation of SS-VOL203 from among the unmigrated SS-VOL203s. The snapshot migration function unit 411 refers to the snapshot generation management table 406 and selects the SS-VOL#803 of the target P-VOL#800 that has the largest generation #802. If no SS-VOL203s have been migrated yet, the snapshot migration function unit 411 selects the source SS-VOL203A4 shown in Figure 2.

[0086] Next, in step S906, the snapshot migration function unit 411 uses the difference extraction function unit 204 to calculate parent-child difference data showing the difference between SS-VOL203 and the primary volume 202, which was selected in step S905. If the snapshot migration function unit 411 is SS-VOL203A3 as shown in Figure 2, it calculates parent-child difference data showing the difference between SS-VOL203A3 and P-VOL202A. Since the difference extraction function unit 204 is part of the basic functions of the snapshot function, a detailed explanation of the difference extraction function unit 204 will be omitted, but the difference extraction function unit 204 refers to the volume control table 405 and extracts difference data showing the difference between multiple volumes by comparing the physical address 701 for the same logical address 700 among multiple volumes that are the target of difference extraction.

[0087] Next, in step S907, the snapshot migration function unit 411 uses the difference extraction function unit 204 to calculate intergenerational difference data showing the difference between the SS-VOL203 selected in step S905 and the next generation of SS-VOL203. If the selected snapshot is SS-VOL203A3 shown in Figure 2, the snapshot migration function unit 411 calculates intergenerational difference data showing the difference between SS-VOL203A3 and SS-VOL203A4. On the other hand, if the selected snapshot is SS-VOL203A4 shown in Figure 2, the snapshot migration function unit 411 can omit some of the corresponding processing because there is no next generation of SS-VOL203. In this case, the snapshot migration function unit 411 executes the parent-child difference data copy processing in step S910.

[0088] The snapshot migration function unit 411 compares the parent-child difference data calculated in step S906 with the generation difference data calculated in step S907 (step S908). If the difference amount of the parent-child difference data is less than or equal to the difference amount of the generation difference data (step S908: Yes), the snapshot migration function unit 411 executes step S910 to transfer the parent-child difference data to the migration destination. On the other hand, if the difference amount of the parent-child difference data is greater than the difference amount of the generation difference data (step S908: No), it executes step S909 to transfer the generation difference data to the migration destination.

[0089] The snapshot migration function unit 411, when the difference amount of parent-child differential data is greater than the difference amount of intergenerational differential data (step S908: No), that is, when the difference amount of intergenerational differential data is less than the difference amount of parent-child differential data, transfers the intergenerational differential data to the migration destination to suppress the increase in the amount of data transferred and the amount of storage space consumed at the migration destination. If SS-VOL203A3 in Figure 2 is selected in step S905, the snapshot migration function unit 411 transfers the intergenerational differential data between SS-VOL203A3 and SS-VOL203A4 to SS-VOL203B3 on the migration destination storage node 101B. To reproduce the data of SS-VOL203A3 on SS-VOL203B3 by transferring only intergenerational differential data showing the difference between SS-VOL203A3 and SS-VOL203A4, it is necessary to reproduce the same data on SS-VOL203B3 as on SS-VOL203B4, which is on the destination storage 101B and has the same data as the already migrated SS-VOL203A4. For this reason, in step S909, the snapshot migration function unit 411 copies the image of the next newer SS-VOL203B4 to the selected generation of SS-VOL203B3. The copy process in step S909 may be a volume-to-volume copy function based on the snapshot function unit 408, or it may be a copy to a specified volume as part of the snapshot function unit 408.

[0090] After executing step S909 as described above, in step S910, the snapshot migration function unit 411 performs a differential copy of the intergenerational differential data to the destination SS-VOL203. Here, the process of calculating which logical address 700 contains the differential data using the differential extraction function unit 204 may be performed when step S910 is executed, or the differential data may be transferred using information saved in BitMap format or the like when calculating the difference amount in step S906. In step S905, if SS-VOL203A3 in Figure 2 is selected, the snapshot migration function unit 411 transfers the differential data showing the difference between SS-VOL203A3 and SS-VOL203A4 to SS-VOL203B3 on the destination storage node 101B.

[0091] The snapshot migration function unit 411, when the difference amount of parent-child differential data is less than or equal to the difference amount of intergenerational differential data (step S908: Yes), transfers the parent-child differential data to the migration destination, thereby suppressing the increase in the amount of data transferred and the amount of storage space consumed at the migration destination. In step S911, the snapshot migration function unit 411 differentially copies the parent-child differential data to the migration destination snapshot volume 203. If SS-VOL203A3 in Figure 2 is selected in step S905, the snapshot migration function unit 411 transfers the differential data showing the difference between SS-VOL203A3 and P-VOL202A to SS-VOL203B3 on the migration destination storage node 101B. Since SS-VOL203B4 is a snapshot volume created from P-VOL202B in step S903, it originally has the same image as P-VOL202B. Therefore, step S909 is unnecessary, for example, when transferring parent-child differential data.

[0092] Figure 10 is a diagram illustrating the effects of the storage system 100 according to the first embodiment. First, the data control method will be explained. This data control method is for the storage system 100 with the above-described configuration. If the parent-child difference data, which shows the difference between the source P-VOL202A and multiple source SS-VOL203A1~203A4, is smaller than the intergenerational difference data, which shows the difference between the multiple source SS-VOL203A1~203A4, the source storage node 101A transfers the parent-child difference data to the destination storage node 101B. Using this parent-child difference data, the destination storage node 101B creates the destination P-VOL202B and multiple destination SS-VOL203B1~203B4. A more detailed explanation follows.

[0093] The illustrated example shows the logical configuration when the snapshot migration function unit 411 migrates P-VOL202A and SS-VOL203A1~A3 from the source storage node 101A to the destination storage node 101B. An example of how the snapshot migration function unit 411 executes processing as time progresses from T1 to T2 is shown. The state of the data P-VOL202 and SS-VOL203A1~A3 before the migration process is almost the same as the state shown in Figure 3, so some explanation is omitted.

[0094] At time T1, as step (1), all data from P-VOL202A is fully copied to P-VOL202B on the destination storage node 101B (step S902). Subsequently, SS-VOL203B1~B3 are created from P-VOL202B on the destination storage node (step S903). In step (2), SS-VOL203A3 is selected as the SS-VOL to be migrated. Dataset A1, which is differential data showing the difference between SS-VOL203A3 and P-VOL202A, is differentially copied to SS-VOL203B3 (step S911).

[0095] At time T2, in step (3)-1, SS-VOL203A2 is selected as the SS-VOL to be migrated, and the difference amount of parent-child differential data showing the difference between SS-VOL203A2 and P-VOL202A is calculated, and it is found that the difference amount of parent-child differential data is 2 (step S906). In step (3)-2, the difference amount of intergenerational differential data between SS-VOL203A2 and the next generation SS-VOL203A3 is calculated, and it is found that the difference amount of intergenerational differential data is 3 (step S907). As a result, the difference amount of parent-child differential data is less than or equal to the difference amount of intergenerational differential data. Therefore, in step (3)-3, the parent-child differential data datasets B1 and C1 are differentially copied to SS-VOL203B2 in the destination storage 101B (step S911).

[0096] In step (4)-1, SS-VOL203A1 is selected as the SS-VOL to be migrated, and the difference amount of parent-child difference data showing the difference between SS-VOL203A1 and P-VOL202A is calculated, and it is found that the difference amount of parent-child difference data is 0 (step S906). In step (4)-2, the intergenerational difference data between SS-VOL203A1 and the next generation SS-VOL203A2 is calculated, and it is found that the difference amount of intergenerational difference data is 2 (step S907). As a result, since the difference amount of parent-child difference data is less than or equal to the difference amount of intergenerational difference data, the parent-child difference data is to be differentially copied to the migration destination. However, since the difference amount of parent-child difference data is 0 and no difference has occurred, copying the difference data is unnecessary.

[0097] As a result of migrating P-VOL202 and SS-VOL203 using the snapshot migration function unit 411, the only datasets transferred to the destination storage node 101B are datasets A0, B0, and C0 transferred by the full copy in procedure (1), and datasets A1, B1, and C1 transferred by the differential data copy. In the example shown in Figure 9, as in the example shown in Figure 3 above, the same dataset is not transferred twice, making it possible to suppress the increase in storage space consumption at the destination.

[0098] Figure 10 illustrates the effects of this embodiment, assuming that the snapshot volume is updated for the purpose of deploying virtual machines. However, even when the primary volume is updated for backup purposes, this embodiment can migrate while maintaining the same capacity consumption at the destination as at the source. In other words, this embodiment makes it possible to suppress the increase in storage space consumption of the destination storage node 101B whether P-VOL202 is updated as a primary volume for backup purposes or SS-VOL203 is updated as a snapshot volume for the purpose of deploying virtual machines. The snapshot migration function unit 411 calculates parent-child difference data showing the difference between P-VOL202 and SS-VOL203, and intergenerational difference data showing the difference between multiple SS-VOL203s. In step S908, it compares the difference amounts of the two and transfers the difference data with the smaller difference amount, thereby making it possible to suppress the increase in storage space consumption of the destination storage node 101B for both purposes.

[0099] The first embodiment has been described so far. In this embodiment, the difference amount of parent-child difference data showing the difference between P-VOL202 and SS-VOL203, and the difference amount of intergenerational difference data showing the difference between multiple SS-VOL203s are calculated, the difference amounts of the two are compared, and the difference data with the smaller difference amount is transferred. However, preferably, the difference amount of all difference data between P-VOL202 and each SS-VOL203 may be calculated, and the difference data of SS-VOL202 may be transferred in order of the smallest difference amount.

[0100] The storage system 100 according to the embodiment is a storage system 100 comprising a plurality of storage nodes, each having a CPU 104 as an example of a processor, wherein each storage node comprises a volume and a plurality of snapshot volumes, which are snapshots of the volume. When migrating a volume and a plurality of snapshot volumes from a source storage node 101A to a destination storage node 101B, the source storage node 101A transfers the volume data and parent-child difference data indicating the difference between the volume and the plurality of snapshot volumes to the destination storage node 101B, and the destination storage node 101B uses the received volume data and the received parent-child difference data to create a volume and a plurality of snapshot volumes on the destination storage node 101B. Snapshot volumes are created by duplicating the volume, and it is possible to write to the snapshot volumes after they are created.

[0101] More specifically, the storage system 100 according to this embodiment includes a plurality of storage nodes 101A and 101B, each having a CPU 104 as an example of a processor. In the storage system 100, each of the plurality of storage nodes 101A and 101B includes a volume and a plurality of snapshot volumes, which are snapshots of the volume. When migrating a volume and a plurality of snapshot volumes from the source storage node 101A to the destination storage node 101B, the source storage node 101A transfers the volume data to the destination storage node 101A, compares the total amount of difference in parent-child difference data, which shows the difference between the source P-VOL202A to be migrated and the plurality of source SS-VOL203A1 to 203A4, with the total amount of difference in intergenerational difference data, which shows the difference between generations among the plurality of source SS-VOL203A1 to 203A4. If the difference in parent-child difference data is smaller than the difference in intergenerational difference data, the parent-child difference data is transferred to the destination storage node 101B. Furthermore, if the difference in intergenerational differential data is smaller than the difference in parent-child differential data, the source storage node 101A transfers the intergenerational differential data to the destination storage node 101B. The destination storage node 101B uses the received volume data and the received parent-child differential data to associate the destination P-VOL202B with the multiple destination SS-VOL203B1~203B4.

[0102] With this configuration, whether the primary volume is updated for backup purposes or the snapshot volume is updated for deploying virtual machines, the increase in storage space consumption at the migration destination can be minimized.

[0103] In this embodiment, a snapshot volume is created by duplicating the volume, and it is possible to write to the snapshot volume after it has been created.

[0104] In this embodiment, the source storage node 101A compares source P-VOL202A with multiple source SS-VOL203A1~203A4 and extracts parent-child differential data and intergenerational differential data, respectively. The source storage node 101A determines which parent-child differential data or intergenerational differential data to transfer to the destination storage node 101B. In this way, the differential data with the smaller difference between parent-child differential data and intergenerational differential data is transferred to the destination storage node 101B. Therefore, even when the primary volume is updated for backup purposes, or when the snapshot volume is updated for virtual machine deployment purposes, the increase in storage space consumption at the destination can be suppressed.

[0105] In this embodiment, the source storage node 101A transfers intergenerational differential data, and the destination storage node 101B copies snapshot images between generations of the snapshot volumes on the destination storage node 101B. Specifically, when transferring intergenerational differential data between multiple source SS-VOL203A1~203A4 volumes on the source storage node 101A to the destination storage node 101B, snapshot images are copied between the target generations of the destination SS-VOL203B1~203B4 in order to reproduce the source SS-VOL203A1~203A4 volumes on the destination SS-VOL203B1~203B4 volumes using only the intergenerational differential data. In this way, even when the primary volume is updated for backup purposes, or when the snapshot volume is updated for the purpose of deploying virtual machines, the increase in storage space consumption on the destination can be suppressed, even when using snapshot images copied between the same generations of multiple source SS-VOL203A1~203A4 volumes.

[0106] (2) Second embodiment Since the storage system according to the second embodiment has substantially the same configuration and operation as the storage system 100 according to the first embodiment, the description of the similar configuration and operation will be omitted, and the following description will focus mainly on the differences. The second embodiment will be described below with reference to Figures 11 to 15.

[0107] In the first embodiment, parent-child difference data showing the difference between P-VOL202 and SS-VOL203, and intergenerational difference data showing the difference between multiple SS-VOL203s were extracted, and the difference data was transferred to the destination according to the difference amount of the two difference data. In contrast, the second embodiment differs from the first embodiment in that the data image of P-VOL202 is not only the latest, but also exists for each time each SS-VOL203 is created. In the second embodiment, a storage system having a function to manage the data image of P-VOL202 for each time each SS-VOL203 is created as generation information of SS-VOL203 will be described.

[0108] Figure 11 is a functional block diagram showing a logical configuration example of the storage node 101A according to the second embodiment. Compared to the storage node 101 according to the first embodiment shown in Figure 4, in the second embodiment, a mapping table 1101 is added as storage node control information 401, and the volume control table 1100 is partially updated. The snapshot migration function unit 1102 is partially updated as the control program 402.

[0109] The mapping table 1101 is a table that translates logical addresses that specify the data locations read and written by the host 102 to physical addresses that indicate where in the storage device 106 the data block at those logical addresses is stored. In other words, this table determines the data images of P-VOL202 and SS-VOL203. In the second embodiment, each SS-VOL203 has two images: the P-VOL data image at the time of SS-VOL203 creation and the latest image of the SS-VOL itself. Therefore, the table number of the mapping table 1101 can be managed from the volume control table 1100.

[0110] Figure 12 shows an example configuration of the volume control table 1100A according to the second embodiment. Compared to the volume control table 1100 shown in Figure 7, which corresponds to the first embodiment, the volume control table 1100A omits the logical address 700 and the physical address 701, and adds mapping table #1200.

[0111] Mapping table #1200 is information used to uniquely identify the table number of mapping table 1101. Mapping table #1200 manages two mapping tables # for every volume #600. P-VOL202 manages one P-VOL mapping information, while SS-VOL203 manages two mapping information entries, one for P-VOL and one for SS-VOL itself. P-VOL202, whose parent volume #603 is "-", has only the P-VOL mapping information, which is the latest image of P-VOL202, as mapping table #1100. On the other hand, SS-VOL203, for which parent volume #603 is a valid value, has two pieces of information: SS-VOL mapping information, which is the latest image of SS-VOL203, and P-VOL mapping information, which is the P-VOL data image at the time SS-VOL203 was created.

[0112] In the illustrated example, the volume with volume #600 being "1" is a volume created as a snapshot of the primary volume with volume #600 being "0". It also has mapping table #1200 with "2" and "3". "2" is P-VOL mapping information, which is the P-VOL data image of SS-VOL203 with volume #600 being "1" at the time it was created. "3" is SS-VOL mapping information, which is the latest image of SS-VOL203 with volume #600 being "1".

[0113] Figure 13 shows an example of the configuration of the mapping table 1101 according to the second embodiment. The mapping table 1101 is a table used to translate between a logical address that specifies the data location to be read or written by the host 102 and a physical address that indicates where in the storage device 106 the data block at that logical address is stored. The mapping table 1101 includes a logical address 700 and a physical address 701 as components of the mapping table 1101. These logical address 700 and physical address 701 are the same as the logical address 700 and physical address 701 shown in Figure 7 in the first embodiment, respectively.

[0114] In the illustrated example, the data block with logical address 700 being "0" shares data with the primary volume and is stored in the area with physical address 702 being "1000". The data block with logical address 700 being "1" is stored in the area with physical address 701 being "1100". The data block with logical address 700 being "2" shares data with the primary volume and is stored in the area with physical address 701 being "1020".

[0115] Figure 14 is a flowchart showing an example of the procedure for the snapshot migration process according to the second embodiment. The snapshot migration process is mainly performed by the snapshot migration function unit 1102. The snapshot migration function unit 1102 corresponds to the snapshot migration function unit 411 in the first embodiment. Similar to the flowchart shown in Figure 9 in the first embodiment, the snapshot migration function unit 1102 may be activated by instructing it to execute the migration process, or it may be configured to autonomously activate a migration to resolve an imbalance when the control program 402 in the storage system 100 detects an imbalance in storage capacity or computation load.

[0116] In this snapshot migration process, first, the snapshot migration function unit 1102 obtains information about the target snapshot group 201 from which the data is to be migrated (step S1400). The snapshot migration function unit 1102 refers to the volume management table 404 and obtains the volume information necessary for migration, such as volume #600, node ID 500, volume capacity 601, volume attribute 602, and parent volume #603 of the target snapshot group #604. This step S1400 is the same as step S900 in Figure 9.

[0117] Next, in step S1401, the snapshot migration function unit 1102 creates a new volume with the same capacity as the source P-VOL202 or SS-VOL203 on the destination storage node 101, based on the volume capacity information 601 acquired in step S1400. This process is the same as step S901 in Figure 9.

[0118] Next, in step S1402, the snapshot migration function unit 1102 copies all the data of the oldest generation SS-VOL203 of the source storage node 101A to the destination volume created in step S1401. In this embodiment, the snapshot migration function unit 1102 migrates the differential data sequentially starting from the oldest generation SS-VOL203, but the migration may also be performed starting from the latest generation P-VOL202, as in the first embodiment. However, as explained in step S909 of Figure 9, when transferring intergenerational differential data, additional processing is required at the destination storage node 101B to ensure that the data image is the same across generations. Therefore, in this embodiment, as an example, the method of migrating sequentially starting from the oldest generation SS-VOL203 will be described.

[0119] Next, in step S1403, the snapshot migration function unit 1102 creates SS-VOL203B1 from P-VOL202B on the destination storage node 101B. This completes the migration process of the last generation snapshot volume (for example, the migration from SS-VOL203A1 to SS-VOL203B1 in Figure 2).

[0120] In step S1404, the snapshot migration function unit 1102 checks whether the unmigrated SS-VOL203 and P-VOL202 still exist on the source storage node 101A. Information regarding whether the migration process is complete may be stored in the volume management table 404 or in the status 804 of the snapshot generation management table 406. If the snapshot migration function unit 1102 determines that there are unmigrated snapshots (step S1403: Yes), it proceeds to step S1404. If there are no unmigrated snapshots (step S1403: No), i.e., all migration processes have been completed, it terminates the process.

[0121] In step S1405, the snapshot migration function unit 1102 selects the oldest generation among the unmigrated SS-VOL203 and P-VOL202. If the copying of all data from the oldest generation SS-VOL203A1 has been completed, the snapshot migration function unit 1102 selects, for example, SS-VOL203A2 in Figure 2.

[0122] Next, in step S1406, the snapshot migration function unit 1102 extracts intergenerational difference data between the selected generation of P-VOL mapping information (SS-VOL203 or P-VOL202) and the SS-VOL203 of the previous generation. Specifically, if the snapshot migration function unit 1102 assumes that the snapshot volume #600 of the selected generation is "2" and the snapshot volume #600 of the previous generation is "1" in the volume control table 1100 shown in Figure 12, then it extracts difference data showing the difference between the P-VOL mapping information of mapping table #1200 "4" and the P-VOL mapping information of "2".

[0123] As explained in Figure 12, the P-VOL mapping information managed by SS-VOL203 is a data image of P-VOL202 at the time SS-VOL203 was created. In other words, by extracting differential data that shows the difference between the P-VOL data image at the time the selected generation SS-VOL203 was created and the P-VOL data image at the time the previous generation SS-VOL203 was created, it is possible to understand the P-VOL202 update data from the time the previous generation SS-VOL203 was created to the time the selected generation SS-VOL203 was created.

[0124] In step S1407, the snapshot migration function unit 1102 copies the differential data resulting from the update of P-VOL202, which was extracted in step S1406, to P-VOL202B on the destination storage node 101B. This makes it possible to reproduce the P-VOL data image at the time of creation of SS-VOL203, which is the selected generation of the source storage node 101A, on P-VOL202B on the destination storage node 101B.

[0125] Next, in step S1408, the snapshot migration function unit 1102 creates SS-VOL203B from P-VOL202B on the destination storage node 101B. This makes it possible to reproduce the P-VOL data image at the time of creation of SS-VOL203A, which is the selected generation of the source storage node 101A, that is, the data state at the time of creation of SS-VOL203A, which is the selected generation, on SS-VOL203B on the destination storage node 101B.

[0126] Next, in step S1409, the snapshot migration function unit 1102 extracts the P-VOL mapping information of the selected generation SS-VOL203 and the parent-child difference data of the SS-VOL mapping information. Specifically, in the volume control table 1100 in Figure 12, if the snapshot volume #600 of the selected generation is "2", the snapshot migration function unit 1102 extracts difference data showing the difference between the P-VOL mapping information of mapping table #1200 "4" and the SS-VOL mapping information of "5".

[0127] The P-VOL mapping information managed by SS-VOL203 is the data image of P-VOL202 at the time SS-VOL203 was created, that is, the data image of SS-VOL203 immediately after its creation. On the other hand, the SS-VOL mapping information is the latest image of SS-VOL itself, so in this process, data updated for SS-VOL after the creation of SS-VOL203 is extracted. Note that if the generation to be migrated is P-VOL202, it does not have SS-VOL mapping information, so steps S1409 and S1410 are unnecessary.

[0128] Finally, in step S1410, the snapshot migration function unit 1102 copies the parent-child differential data extracted in process 1409 to the SS-VOLB of the destination storage node 101B created in step S1408. The state of SS-VOL203B before copying the parent-child differential data is the data state at the time of creation of the selected generation, SS-VOL203A. Therefore, the snapshot migration function unit 1102 can migrate the data of the source SS-VOL203A to the destination SS-VOL203B by copying the parent-child differential data that was updated to SS-VOL after SS-VOL203A was created.

[0129] Figure 15 is a diagram illustrating the effects of the second embodiment. Figure 15 shows the logical configuration when migrating P-VOL202A and SS-VOL203A1~A3 from source storage node 101A to destination storage node 101B using the snapshot migration function unit 1102. The diagram shows the progress of processing by the snapshot migration function unit 1102 as time progresses from T1 to T2. In addition, the state of the data of P-VOL202 and SS-VOL203A1~A3 before migration is slightly different from the state shown in Figure 3 in order to make it easier to explain the effects of this embodiment. Specifically, the update to SS-VOL203 is the same as in Figures 3 and 10, but in addition, an example is shown in which some updates are also made to P-VOL202.

[0130] First, let's describe the state of the source storage node 101A before the snapshot migration. First, datasets A0, B0, and C0 are written to P-VOL202A from host 102, and SS-VOL203A1 is created from P-VOL202. Then, dataset B1 is written to P-VOL202A, and B0 is updated to B1. After that, SS-VOL203A2 is created from P-VOL202, and SS-VOLs for datasets A0, B1, and C0 are created. However, as in the first embodiment, SS-VOL203A2 is in a state where datasets B2 and C1 are written from host 102, and B1 and C0 are updated to B2 and C1. Also, SS-VOL203A3 is created from P-VOL202A, and SS-VOL203A3 for datasets A0, B1, and C0 is created. As in the first embodiment, dataset A1 is written from host 102, and dataset A0 is updated to dataset A1. Furthermore, the six or three squares shown to the right of each SS-VOL203 and P-VOL202 on the source storage node 101A correspond to the P-VOL mapping information ("P" in Figure 15) and SS-VOL mapping information ("SS" in Figure 15), as explained in Figure 12. In other words, the P-VOL mapping information is the data image of P-VOL202 at the time SS-VOL203 was created, and the SS-VOL mapping information is the latest data image of SS-VOL203 itself.

[0131] At time T1, as procedure (1), all data from the oldest generation SS-VOL203A1 is fully copied to P-VOL202B on the destination storage node 101B (step S1402). Subsequently, SS-VOL203B1 is created from P-VOL202B on the destination storage node (step S1403).

[0132] In step (2), SS-VOL203A2 is selected as the SS-VOL to be migrated, and dataset B1, which is the intergenerational difference data of P-VOL mapping information between SS-VOL203A2 and the older generation SS-VOL203A3, is differentially copied to P-VOL202B (step S1407). Dataset B1 is the data that was updated in P-VOL202 between the creation of SS-VOL203A1 and the creation of SS-VOL203A2.

[0133] Subsequently, in step (3), SS-VOL203B2 is created from P-VOL202B (processing 1408 of the snapshot migration function unit 1102). This process makes it possible to reproduce the datasets A0, B1, and C0 of SS-VOL203A2 immediately after its creation in SS-VOL203B2.

[0134] At time T2, as step (4), datasets B2 and C1, which are parent-child difference data showing the difference between the P-VOL mapping information and SS-VOL mapping information of SS-VOL203A2, are extracted and copied to SS-VOL203B2 on the destination storage node 101B (step S1410 executed by the snapshot migration function unit 1102). Through this process, datasets B2 and C1, which were updated in SS-VOL after SS-VOL203A2 was created, are copied to SS-VOL203B2, and the source SS-VOL203A2 is migrated to the destination SS-VOL203B2.

[0135] Next, in step (5), SS-VOL203A3 is selected as the SS-VOL to be migrated, and SS-VOL203B3 is created from the destination P-VOL202B. As in step (2), the intergenerational difference in P-VOL mapping information between SS-VOL203A3 and the older generation SS-VOL203A2 is not present because the data image (datasets A0, B1, C0) is the same, and therefore no difference data exists, and copying of intergenerational difference data is unnecessary. In other words, this indicates that P-VOL202 has not been updated between the time SS-VOL203A2 was created and the time SS-VOL203A3 was created.

[0136] Finally, in step (6), the snapshot migration function unit 1102 extracts dataset A1, which is parent-child difference data showing the difference between the P-VOL mapping information and SS-VOL mapping information of SS-VOL203A3, and copies it to SS-VOL203B3 on the destination storage node 101B. This copies dataset A1, which was updated in SS-VOL after the creation of SS-VOL203A3, to SS-VOL203B3, and the source SS-VOL203A3 is migrated to the destination SS-VOL203B3. In the next process, the snapshot migration function unit 1102 extracts intergenerational difference data showing the difference between P-VOL202 and SS-VOL203A3 in the P-VOL mapping information (step S1406). However, since all of these are identical datasets A0, B1, and C0, a difference copy is unnecessary, and the snapshot migration function unit 1102 terminates its processing.

[0137] As a result of migrating P-VOL202 and SS-VOL203, the only datasets transferred to the destination storage node 101B were datasets A0, B0, and C0, which were transferred by the full copy in procedure (1), dataset B1 (procedure (2)), which is intergenerational differential data generated by the P-VOL update, and datasets B2, C1, and A1 (procedures (4) and (6)), which are parent-child differential data generated by the SS-VOL update—a total of seven datasets. Compared to Figure 10 in the first embodiment, the transfer of dataset B1 has increased because, prior to the migration, dataset B1 was also updated in P-VOL202, increasing the amount of differential data in the source.

[0138] According to this embodiment, as shown in Figure 3 above, the same dataset is not transferred twice, making it possible to suppress the increase in storage space consumption at the destination. According to this embodiment, whether the update is performed on the primary volume or the snapshot volume, it is possible to suppress the increase in storage space consumption at the destination.

[0139] The storage system 100 according to this embodiment is a storage system comprising a plurality of storage nodes, each having a processor, wherein each storage node comprises a volume and a plurality of snapshot volumes, which are snapshots of the volume. A snapshot volume is created by duplicating a volume, is writable to the snapshot volume after creation, is a snapshot image at the time of snapshot volume creation, and has a snapshot image before writing to the snapshot volume. When migrating a volume and a plurality of snapshot volumes from a source storage node 101A to a destination storage node 101B, the source storage node 101A transfers the volume data, intergenerational difference data indicating the differences between the snapshot volumes at the time of creation of the plurality of snapshots, and parent-child difference data, which is the data written to the snapshot volumes after creation, to the destination storage node 101B. The destination storage node 101A uses the received volume data, the received intergenerational difference data, and the received parent-child difference data to associate the volume with the plurality of snapshot volumes.

[0140] (3) Third Embodiment The storage system according to the third embodiment has substantially the same configuration and operation as the storage system 100 according to the first embodiment and the storage system according to the second embodiment. Therefore, the description of the similar configuration and operation will be omitted, and the following description will focus mainly on the differences. The third embodiment will be described below with reference to Figures 16 to 20.

[0141] In the first and second embodiments, the storage system 100 was a cluster configuration consisting of two or more storage nodes 101. On the other hand, in the third embodiment, as will be described later, a configuration using VSSO (Virtual Storage Scale Out), a scale-out function that enables multiple storage systems to be treated as a single mass storage system by arranging two or more storage systems side by side and defining the same virtual storage machine (VSM) for multiple storage systems 100.

[0142] Figure 16 is a block diagram showing an example of the configuration of an overall system including a storage system according to the third embodiment. Compared to the configuration of the storage system 100 according to the first embodiment shown in Figure 1 in the first embodiment, the third embodiment has two or more storage systems 1600 arranged side by side. The storage system 1600 may be composed of multiple storage nodes 101, similar to the storage system 100 in the first embodiment. In addition, a management server 1601 for managing two or more storage systems 1600 is added.

[0143] In the storage system according to the third embodiment, the management server 1601 defines the same virtual storage machine (VSM) for multiple storage systems, thereby allowing multiple storage systems to be treated as a single mass storage system. The storage system 1600, host 102, and management server 1601 do not need to be located in the same place; for example, the storage system 1600 and host 102 may be located in different data centers. Similarly, the storage system 1600 and host 102 may be physically composed of a single device. For example, a configuration such as Hyper Converged Infrastructure (HCI) may be used, where the functions of the storage system 1600 and the functions of the host 102 are housed in a single physical server.

[0144] Each storage system 1600, host 102, and management server 1601 is connected to network 103 and can communicate with each other. Network 103 can be implemented using any communication line or combination thereof, such as Ethernet, InfiniBand, or optical fiber. In addition, network 103 may include not only a LAN (Local Area Network) confined within the data center, but also a WAN (Wide Area Network) such as the Internet, or a virtual network within a computer.

[0145] Figure 17 is a block diagram showing an example of the logical configuration of a storage system according to the third embodiment. Compared to Figure 2 in the first embodiment, in the third embodiment, storage systems 1600A and 1600B (hereinafter collectively referred to as "storage system 1600") are arranged side by side, and these multiple storage systems 1600 are managed by a management server 1601. The configuration of snapshot groups 201, primary volumes 202 and snapshot volumes 203, etc., within storage system 1600 is the same as in the first and second embodiments, so a description is omitted.

[0146] Figure 18 is a block diagram showing a logical configuration example of the storage system 1600 and management server 1601 according to the third embodiment. Compared to Figure 11 in the second embodiment, in the third embodiment, a storage system management table 1800 and a volume management table 1801 are added to the management server 1601, and the snapshot migration function unit 1102, which was in the control program 402 of the storage node 101, has been moved to the management server 1601. The storage node management table 403 and volume management table 404 of the cluster management information 400 in Figure 11 correspond to the storage system management table 1800 and volume management table 1801 in Figure 18, respectively.

[0147] The storage system management table 1800 is a management table for operating multiple storage systems as a single mass storage system. The volume management table 1801 is a table for managing the volume and snapshot configurations of multiple storage systems 1600 managed by the management server 1601. The snapshot migration function unit 1102 is the same program as the snapshot migration function in the second embodiment, but in this embodiment it is executed as a program of the management server 1601. However, the differential extraction function unit 204 still remains in the control program 402 of the storage system 1600.

[0148] The snapshot migration function unit 1102 of the management server 1601 identifies the P-VOL202 and SS-VOL203 to be migrated, manages the migration procedure and progress, and determines the volumes to be extracted for differential analysis. The differential extraction function unit 204 of the storage system 1600 extracts differential data showing the differences between multiple target volumes instructed by the snapshot migration function unit 1102 of the management server 1601, and exchanges the differential data with the management server 1601 using information such as BitMap. The transfer process of the differential data itself may be performed by the management server 1601 or by the storage system 1600.

[0149] Figure 19 is a block diagram showing an example configuration of the storage system management table 1800 according to the third embodiment. Compared to the example configuration shown in Figure 5 in the first and second embodiments, the entry that was node ID 500 has been replaced with device ID 1900 for each storage system 1600. The storage system management table 1800 is a table for managing the storage systems 1600 that constitute a single mass storage system in the VSSO described above. Each entry in the storage system management table 1800 includes device ID 1900, IP address 501, capacity information 502, load information 503, and communication bandwidth information 504. Device ID 1900 is information for uniquely identifying the storage system 1600 managed as VSSO. In the example shown, it is a sequential number, but it does not have to be a sequential number, and an ID using characters other than numbers is also acceptable.

[0150] Figure 20 shows an example configuration of the volume management table 1801 according to the third embodiment. Compared to the example configuration shown in Figure 6 in the first and second embodiments, the entry for node ID 500 has been replaced with the device ID 2000 for each storage system. The volume management table 1801 is a table for managing the configuration of volumes and snapshots of storage system 1600 managed by the management server 1601. Each entry in the volume management table 1801 includes volume number (VOL#) 600, device ID 2000, volume capacity 601, volume attribute 602, parent volume number (parent VOL#) 603, and snapshot group # 604.

[0151] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail for the purpose of clearly illustrating the present invention, and the present invention is not necessarily limited to having all the configurations described. In this embodiment, each element described in parallel may be configured such that at least one of the elements is connected in series with respect to the other elements. [Industrial applicability]

[0152] The present invention can be applied, for example, to a storage system that uses a snapshot function. [Explanation of Symbols]

[0153] 100...Storage system, 101A...Source storage device, 101B...Destination storage device, 102...Host device.

Claims

1. In a storage system comprising multiple storage nodes each having a processor, The storage node comprises a volume and a plurality of snapshot volumes which are snapshots of the volume, When migrating the aforementioned volume and the multiple snapshot volumes from the source storage node to the destination storage node, The aforementioned source storage node is The data of the volume is transferred to the destination storage node. The sum of the difference amounts of parent-child difference data, which shows the difference between the volume to be migrated and the multiple snapshot volumes, is compared with the sum of the difference amounts of intergenerational difference data, which shows the difference between generations of the multiple snapshot volumes. If the difference amount of the parent-child differential data is smaller than the difference amount of the intergenerational differential data, the parent-child differential data is transferred to the destination storage node, and if the difference amount of the intergenerational differential data is smaller than the difference amount of the parent-child differential data, the intergenerational differential data is transferred to the destination storage node. The destination storage node is, Using the received data from the volume and the received intergenerational differential data or parent-child differential data, the volume and the multiple snapshot volumes are associated with the destination storage node. A storage system characterized by the following features.

2. The aforementioned snapshot volume is created by duplicating the aforementioned volume. The snapshot volume created as described above can be written to. The storage system according to feature 1.

3. The aforementioned source storage node is The volume and the multiple snapshot volumes are compared to extract the parent-child differential data and the generational differential data, respectively, and the parent-child differential data or generational differential data to be transferred to the destination storage node is determined. The storage system according to claim 2, characterized by the features described above.

4. The source storage node transfers the intergenerational differential data, The destination storage node performs snapshot image copying between generations of the snapshot volume of the destination storage node. The storage system according to feature 1.

5. In a storage system comprising multiple storage nodes each having a processor, The storage node comprises a volume and a plurality of snapshot volumes which are snapshots of the volume, The aforementioned snapshot volume is created by duplicating the aforementioned volume. The snapshot volume created above is writable. This is a snapshot image at the time of creating the aforementioned snapshot volume, and it has a snapshot image before writing to the aforementioned snapshot volume. When migrating the aforementioned volume and the multiple snapshot volumes from the source storage node to the destination storage node, The source storage node transfers the data of the volume, the intergenerational difference data showing the differences between snapshot volumes at the time of creation of the multiple snapshots, and the parent-child difference data which is the data written to the snapshot volume after creation, to the destination storage node. The destination storage node uses the received volume data, the received intergenerational differential data, and the received parent-child differential data to map the volume and the multiple snapshot volumes on the destination storage node. A storage system characterized by the following features.

6. In a storage system comprising multiple storage nodes each having a processor, The storage node comprises a volume and a plurality of snapshot volumes which are snapshots of the volume, When migrating the aforementioned volume and the multiple snapshot volumes from the source storage node to the destination storage node, The source storage node transfers the data of the volume and parent-child differential data showing the difference between the volume and the multiple snapshot volumes to the destination storage node. The destination storage node uses the received volume data and the received parent-child differential data to create the volume and the multiple snapshot volumes on the destination storage node. A storage system characterized by the following features.

7. The aforementioned snapshot volume is created by duplicating the aforementioned volume. The snapshot volume created as described above can be written to. The storage system according to feature 6.

Images