Storage system and data placement method

Abstract

This invention provides a storage system and data placement method that can improve the performance of a storage system by efficiently operating multiple backend drives. [Solution] The storage system is constructed by having multiple storage nodes, each of which includes a storage control unit and is allocated multiple drives. The storage control unit reads and writes data to physical chunks in response to I / O requests from the host device. A drive has multiple physical chunks that are divided to store data. The storage control unit monitors the drive utilization rate for each of the multiple drives and, based on the drive utilization rates of the multiple drives, performs data relocation, moving data located in the physical chunks of the source drive to the physical chunks of the destination drive in order to equalize the drive utilization rates of the multiple drives.

Description

Technical Field

[0001] The present invention relates to a storage system and a data placement method.

Background Art

[0002] Software Defined Storage (SDS), which is a technology for managing storage devices distributed across multiple nodes with software and defining the whole as a large storage device to improve usage efficiency, has become widespread. In a storage system applying SDS, capacity leveling of multiple backend drives is an important technology for improving the efficiency and reliability of the storage system. The prior art performs chunk balancing and levels the used capacity among storage nodes when a failure occurs, maintenance is performed, or the number of storage nodes is changed.

[0003] Patent Document 1 discloses that in a storage system, when the number of drives in a node increases or decreases, the physical chunks assigned to parity groups in the node are changed, and the configuration is changed by moving data and parity within the node between drives (data movement).

[0004] Patent Document 2 discloses that when the number of storage nodes in a system is increased, the physical chunks assigned to a chunk group are rearranged, data is moved between components, and the used capacity is leveled among components.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In storage systems, user access to data is localized; rather than accessing the entire dataset evenly, users often access only a specific part of the data. Consequently, access tends to concentrate on certain data, and access tends to be biased towards drives (backend drives) where frequently accessed data is stored.

[0007] As a result, in storage systems, variations in the utilization rates of multiple drives can occur, leading to a situation where certain drives reach their performance limits while other drives have ample capacity. When this phenomenon occurs, the overall performance of the system deteriorates because the full performance of all drives cannot be utilized.

[0008] Conventional technologies perform chunk rebalancing to equalize the capacity usage of each storage node during failures or maintenance, but they do not perform chunk rebalancing to equalize the utilization rate of the drives, thus failing to maximize the performance of each drive.

[0009] The technologies described in Patent Documents 1 and 2 only rebalance physical chunks when adding or removing drives. Therefore, during operation, concentrated access to specific data can cause variations in the utilization rates of multiple drives, potentially leading to performance limitations on specific drives and a decrease in the overall performance of the storage system.

[0010] This invention was made in view of the above problems. Specifically, one of the objects of this invention is to provide a storage system and a data placement method that can improve the performance of a storage system by efficiently operating multiple backend drives as a whole. [Means for solving the problem]

[0011] To solve the above problems, the storage system of the present invention has a plurality of storage nodes, each of which includes a storage control unit and is allocated a plurality of drives, and is a virtual storage system constructed by the plurality of storage nodes, wherein each drive has a plurality of physical chunks divided for storing data, and the storage control unit is configured to read and write data to the physical chunks in response to I / O requests from a host device, monitor the drive utilization rate for each of the plurality of drives, and perform data relocation, which involves moving data located in the physical chunks of the source drive to the physical chunks of the destination drive in order to equalize the drive utilization rates of the plurality of drives based on the drive utilization rates of the plurality of drives.

[0012] The data placement method of the present invention is a data placement method in a virtual storage system constructed by a plurality of storage nodes, each of which includes a storage control unit and is allocated a plurality of drives, wherein the drives have a plurality of physical chunks divided for storing data, and the storage control unit reads and writes data to the physical chunks in response to I / O requests from a host device, monitors the drive utilization rate for each of the plurality of drives, and performs data relocation based on the drive utilization rates of the plurality of drives, in order to equalize the drive utilization rates of the plurality of drives, by placing the data located in the physical chunks of the source drive into the physical chunks of the destination drive. [Effects of the Invention]

[0013] According to the present invention, the performance of a storage system can be improved by efficiently operating multiple backend drives. The effects described herein are not necessarily limited to those described herein and may include any of the effects described herein. [Brief explanation of the drawing]

[0014] [Figure 1A]FIG. 1A is a diagram showing a configuration example of a system including a storage system according to a first embodiment of the present invention. [Figure 1B] FIG. 1B is a diagram showing a logical configuration of a storage node. [Figure 2] FIG. 2 is a diagram for explaining the correspondence relationship between a drive and a physical chunk. [Figure 3] FIG. 3 is a diagram for explaining the correspondence relationship between a physical chunk, a logical chunk, and a volume. [Figure 4] FIG. 4 is a diagram for explaining a drive operation rate management table stored in a memory. [Figure 5] FIG. 5 is a diagram for explaining a drive chunk mapping table stored in a memory. [Figure 6] FIG. 6 is a flowchart showing a processing flow executed by each storage node. [Figure 7] FIG. 7 is a flowchart showing a processing flow executed by each storage node. [Figure 8] FIG. 8 is a diagram showing a table example of a drive operation rate management table before and after execution of chunk balance. [Figure 9] FIG. 9 is a diagram showing a table example of a drive chunk mapping table before and after execution of chunk balance. [Figure 10] FIG. 10 is a diagram for explaining a node resource management table stored in the memory of a master node. [Figure 11] FIG. 11 is a flowchart showing a processing flow executed by a master node. [Figure 12] FIG. 12 is a flowchart showing a processing flow executed by a master node. [Figure 13] FIG. 13 is a flowchart showing a processing flow executed by a master node. [Figure 14] FIG. 14 is a flowchart showing a processing flow executed by a master node. [Figure 15]FIG. 15 is a sequence diagram for explaining an operation example of a storage system. [Figure 16] FIG. 16 is a sequence diagram for explaining an operation example of a storage system. [Figure 17] FIG. 17 is a diagram for explaining a node mode management table. [Figure 18] FIG. 18 is a flowchart showing a processing flow executed by a master node. [Figure 19] FIG. 19 is a diagram for explaining a drive chunk mapping table. [Figure 20] FIG. 20 is a flowchart showing a processing flow executed by each storage node. [Figure 21] FIG. 21 is a flowchart showing a processing flow executed by a master node. [Figure 22] FIG. 22 is a flowchart showing a processing flow executed by a master node. [Figure 23] FIG. 23 is a diagram for explaining a configuration example of a storage system according to a modification example.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. In all the drawings of the embodiments, the same or corresponding parts may be denoted by the same reference numerals.

[0016] In the following explanation, various types of information may be described using terms such as "table," "record," "row," "column," etc., but these types of information may also be represented using data structures other than these. When describing identification information, terms such as "ID" and "name" will be used, but these are interchangeable and can also be replaced with other expressions of identification information. In the following explanation, processing may be described with the device (storage node, master node, worker node) or storage system as the subject, but the subject of the processing may be the CPU or storage control unit instead of the device (storage node, master node, worker node).

[0017] <<First Embodiment>> Figure 1A is a diagram showing an example of the configuration of a system including a storage system according to the first embodiment of the present invention. As shown in Figure 1A, the system comprises a host device 100 and a virtual storage system including a plurality (three in this example) of storage nodes 200a to 200c. The virtual storage system is constructed by storage nodes 200a to 200c. Hereinafter, storage nodes 200a and 200c may be referred to as "storage node 200" when there is no need to distinguish between them. The virtual storage system may also be referred to as a "storage cluster". The storage cluster is realized by storage software running on storage node 200. Storage node 200a may also be referred to as "master node 200a". Storage node 200b may also be referred to as "worker node 200b". Storage node 200c may also be referred to as "worker node 200c".

[0018] The host device 100 and the storage nodes 200 are connected to each other via the compute network NW1, enabling them to communicate with one another. Each storage node 200 is connected to each other via the inter-storage node network NW2, enabling them to communicate with one another.

[0019] The host device 100 is a server device that performs various business processes by executing installed application programs. In response to requests from the running application programs, the host device 100 sends data read or write requests to volume 234 (see Figure 3) provided by storage node 200 via the compute network NW1. Note that there may be multiple host devices 100. Furthermore, the host device 100 may be a virtual server device provided by a cloud service.

[0020] The storage node 200 is a server device that provides volume 234 (see Figure 3), which is a storage area for reading and writing data to the host device 100. Multiple storage nodes 200 include one master node 200a and multiple worker nodes 200b and 200c. There is one master node 200a in the storage cluster, and it manages and controls the entire storage cluster. Worker nodes 200b and 200c are storage nodes within the storage cluster that do not have the role of managing the entire storage cluster.

[0021] The storage node 200 comprises a CPU 210, memory 220, multiple drives 230 allocated to the storage node 200, a drive adapter 240, a first network interface 250, and a second network interface 260. The CPU 210, memory 220, multiple drives 230, drive adapter 240, first network interface 250, and second network interface 260 are connected to each other via a bus, enabling them to send and receive information.

[0022] The CPU 210 is a control device that manages the operation of the entire storage node 200 and performs various processes by executing various programs stored in the memory 220. The memory 220 stores, for example, control information used by the storage node 200, programs executed by the CPU 210, data accessed by the host device 100, and various tables (Figures 4 and 5, etc.). The memory 220 is generally composed of DRAM (Dynamic RAM (Random Access Memory)), but it may also be composed of other storage media such as MRAM (Magneto-resistive RAM), ReRAM (Resistive RAM), PCM (Phase Change Memory), or NAND.

[0023] Figure 1B shows the logical configuration of the storage node 200. As shown in Figure 1B, the storage node 200 has a storage control unit 201. The CPU 210 implements the functions of the storage control unit 201 by executing a program (software) that controls various aspects of the volume 234 provided to the host device 100. The storage control unit 201 has the function of providing storage space to the host device 100 in volume units using the drive 230 and saving data to the drive 230 in response to I / O (Input / Output) requests from the host device 100. The volume 234 is managed by dividing the storage space into one or more blocks 202 of predetermined capacity from the beginning. Referring again to Figure 1A, the drive 230 is a device that physically has storage space and consists of a non-volatile storage device that can read and write data, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). Multiple drives 230 may be bundled together within the storage node 200 and a high-reliability technology such as RAID (Redundant Arrays of Independent Disks) may be used.

[0024] The drive adapter 240 is a hardware component for connecting the drive 230.

[0025] The first network interface 250 is an interface for connecting the storage node 200 to the host device 100 via the compute network NW1.

[0026] The second network interface 260 is an interface for connecting storage node 200 to other storage nodes 200 via the inter-storage node network NW2.

[0027] Figure 2 is a diagram illustrating the correspondence between drive 230 and physical chunk 232. Figure 3 is a diagram illustrating the correspondence between physical chunk 232, logical chunk 233, and volume 234.

[0028] As shown in Figure 2, the physical storage area 231 provided by the drive 230 is divided into one or more small areas of predetermined capacity, thereby dividing it into physical storage areas referred to as physical chunks 232.

[0029] As shown in Figure 3, one or more physical chunks 232 are assigned (associated) with logical chunks 233. The logical chunks 233 are associated with block 202 of volume 234 (see Figure 1B), which is a logical device mounted on the host device 100 for reading and writing data. The logical chunks 233 store the data written to the host device 100. Data redundancy may be achieved by associating one logical chunk 233 with two or more physical chunks 232, and storing the data written to the logical chunks 233 (or the data and parity calculated and generated based on the data) in all associated physical chunks 232.

[0030] Volume 234 is a storage area (logical device) provided by the storage system (storage control unit 201) to the host device 100, as explained with reference to Figure 1B, and the host device 100 requests data to be written to volume 234. Volume 234 is created when the administrator (user) of the storage system issues a volume creation instruction to the storage system via a management terminal (not shown).

[0031] The volume itself does not have physical storage space; instead, it allocates logical chunks 233 in response to write requests from the host device 100 and logically writes data to the logical chunks 233. As previously described, volume 234 is managed by dividing its storage space into one or more blocks 202 of predetermined capacity from the beginning. These blocks 202 are associated one-to-one with, for example, logical chunks 233. Immediately after volume creation, no logical chunk 233 is associated with any block 202. When the host device 100 writes data to volume 234, if a logical chunk 233 is not associated with the block 202 corresponding to the area where the data was written, the process of creating a logical chunk 233 and associating the block 202 with the logical chunk 233 is performed.

[0032] Figure 4 is a diagram illustrating the drive utilization management table 400 stored in memory 220. As shown in Figure 4, the drive utilization management table 400 includes the following columns for storing information (values): node ID 401, drive ID 402, drive status 403, response time 404, and utilization rate 405.

[0033] The drive utilization management table 400 stores information corresponding to each column of the drive 230 of each storage node 200, with the information being linked to each other and stored as row-level information (records).

[0034] Specifically, node ID 401 stores identification information (node ​​ID) to identify storage node 200. Drive ID 402 stores identification information (drive ID) to identify drive 230. Drive status 430 stores status information indicating the status of drive 230. The status information stored is one of the following: "Normal", "Error", or "Blockage". Normal indicates that drive 230 is in a normal state. Error indicates that drive 230 is in an abnormal state. Blockage indicates that drive 230 is blocked (I / O is not possible).

[0035] The response time 404 stores the time it takes to read and write data to drive 230. The availability 405 stores the drive availability. Drive availability is the ratio of the time drive 230 is performing I / O to the time drive 230 is not performing I / O (= time drive 230 is performing I / O / time drive 230 is not performing I / O). In this example, drive availability can be calculated by "monitoring IOPS (the actual number of read / write operations performed per second during monitoring)" / "drive specifications (the number of read / write operations that can be performed per second)". The specifications of drive 230 are, for example, unique specifications corresponding to the model number of drive 230. Drive availability can be obtained by copying the sar information obtained from the OS (operating system).

[0036] Figure 5 is a diagram illustrating the drive chunk mapping table 500 stored in memory 220. As shown in Figure 5, the drive chunk mapping table 500 includes the following columns for storing information (values): drive ID 501, P-chunk ID 502, P-chunk IOPS 503, L-chunk ID 504, and Volume ID 505.

[0037] The drive chunk mapping table 500 stores information corresponding to each column of each physical chunk 232, logical chunk 233, volume 234, and monitoring IOPS for each drive 230, with the information being mapped to each other and stored as row-level information (records).

[0038] Specifically, drive ID 501 stores identification information (drive ID) to identify each drive 230. P-chunk ID 502 stores identification information (physical chunk ID) to identify physical chunk 232. P-chunk IOPS 503 stores the monitoring IOPS (Input / Output Per Second) for physical chunk 232. Monitoring IOPS is the number of reads and writes per second and indicates the access frequency to physical chunk 232. Monitoring IOPS is sometimes referred to as "access frequency". Volume ID 505 stores identification information (volume ID) to identify volume 234.

[0039] <Specific operation> Figure 6 is a flowchart showing the processing flow executed by each storage node 200. The storage node 200 starts processing from step 600 and proceeds to step 605, where it refers to the drive utilization management table 400 to check the drive utilization of each drive 230 assigned to the storage node 200. Specifically, the storage node 200 calculates the difference between the maximum and minimum drive utilization (drive utilization difference) for multiple drives 230 on a per-storage node basis.

[0040] Subsequently, the storage node 200 proceeds to step 610, where it determines whether the difference in drive utilization is greater than or equal to a predetermined differential rebalancing threshold. The predetermined differential rebalancing threshold is, for example, 50%.

[0041] If the difference in drive utilization is less than a predetermined differential rebalancing threshold, the storage node 200 determines "NO" in step 610 and proceeds to step 695, terminating this processing flow.

[0042] In contrast, if the difference in drive utilization exceeds a predetermined differential rebalancing threshold, variations in the utilization of multiple drives 230 may occur, potentially leading to a situation where a specific drive 230 reaches its performance limit while other drives 230 have ample operational headroom. Therefore, if the difference in drive utilization exceeds a predetermined differential rebalancing threshold, a chunk rebalancing (rebalancing) is necessary to resolve this phenomenon. As a result, the storage node 200 determines "YES" in step 610 and proceeds to step 615. The chunk rebalancing is then performed according to the processing described later in step 615 and the processing flow shown in Figure 7.

[0043] Chunk rebalancing includes assigning the destination physical chunk 232, which is not assigned to the logical chunk 233, to the logical chunk 233 to which the source physical chunk 232 was assigned, in place of the source physical chunk 232, and migrating the data stored in the source physical chunk 232 from the source physical chunk 232 to the destination physical chunk 232 (relocating the data located in the source physical chunk 232 to the destination physical chunk 232).

[0044] As previously described, when a physical chunk 232 (source physical chunk 232) is assigned to a logical chunk 233, the read and write operations performed by the host device 100 on that logical chunk 233 are executed on the source physical chunk 232 (source drive 230) that was assigned to the logical chunk 233 by the backend drive (drive 230). When the physical chunk 232 assigned to the logical chunk 233 is changed from the source physical chunk 232 to the destination physical chunk 232, the read and write operations performed by the host device 100 on that logical chunk 233 are executed on the destination physical chunk 232 (destination drive 230) by the backend drive (drive 230). Therefore, if access to one drive 230 is concentrated, resulting in a large difference in drive utilization between that drive 230 and the other drives 230, the storage node 200 will equalize the drive utilization through chunk rebalancing to resolve the access concentration. Specifically, the storage node 200 identifies the source physical chunk 232 from among the drives 230 that are experiencing heavy access, and identifies the destination physical chunk 232 from among the other drives 230 that have available capacity. The storage node 200 assigns the destination physical chunk 232 to the logical chunk 233 to which the source physical chunk 232 is currently assigned, replacing the source physical chunk 232. The storage node 200 then migrates the data stored in the source physical chunk 232 from the source physical chunk 232 to the destination physical chunk 232 (relocating the data located in the source physical chunk 232 to the destination physical chunk 232).

[0045] After chunk rebalancing, data reads and writes that were previously performed on the source physical chunk 232 (source drive 230) will no longer occur on the source physical chunk 232 (source drive 230) but will instead be performed on the destination physical chunk 232 (destination drive 230). This eliminates the concentration of access on a single drive 230, thereby improving the performance of the storage system.

[0046] When the storage node 200 proceeds to step 615, it refers to the drive chunk mapping table 500 to identify the source physical chunk 232. Specifically, the storage node 200 refers to the drive chunk mapping table 500 to identify the drive 230 with the highest drive utilization among the multiple drives 230 assigned to the storage node 200. The storage node 200 identifies the physical chunk 232 with the highest monitoring IOPS among the multiple physical chunks 232 within the identified drive 230 as the source physical chunk 232.

[0047] When storage node 200 proceeds to step 620, it checks for unallocated physical chunks 232 in logical chunk 233. Storage node 200 refers to the drive chunk mapping table 500 to check whether there are any unallocated physical chunks 232 in any of the multiple physical chunks 232 contained in drives 230 other than the drive 230 containing the source physical chunk 232. In this example, the checks for unallocated physical chunks 232 in logical chunk 233 are performed starting with the drive 230 with the lowest drive utilization. Alternatively, the checks for unallocated physical chunks 232 in logical chunk 233 could be performed starting with the drive 230 with the lowest ID.

[0048] If it is not confirmed that there is an unallocated physical chunk 232 in logical chunk 233, the storage node 200 determines "NO" in step 620 and proceeds to step 695 to terminate this processing flow.

[0049] If it is confirmed that there is an unallocated physical chunk 232 in logical chunk 233, the storage node 200 determines "YES" in step 620 and proceeds to step 625, where it assigns the destination physical chunk 232 to the logical chunk 233 to which the source physical chunk 232 was previously assigned. This process is sometimes referred to as "L-Chunk migration". After that, the storage node 200 proceeds to step 695 and terminates this processing flow.

[0050] Figure 7 is a flowchart showing the processing flow executed by each storage node 200. After the storage node 200 assigns the destination physical chunk 232 to the logical chunk 233 to which the source physical chunk 232 was previously assigned (step 625 in Figure 6), it starts processing from step 700, and after sequentially executing the processes described below in steps 705 to 715, it proceeds to step 795 and terminates this processing flow.

[0051] Step 705: Storage node 200 initiates a copy process to copy the data stored in the source physical chunk 232 from the source physical chunk 232 to the destination physical chunk 232.

[0052] Step 710: Storage node 200 completes the copy process. After the copy process is complete, the data stored in the source physical chunk 232 will be deleted.

[0053] Step 715: The storage node 200 updates the information stored in the drive utilization management table 400 and the drive chunk mapping table 500 to correspond to the state after the copy process.

[0054] <Example of operation> An example of the storage system's operation will be described. Figure 8 shows an example of the drive utilization management table 400 before and after chunk rebalancing. Table example 400a shows the state before chunk rebalancing, and table example 400b shows the state after chunk rebalancing. Figure 9 shows an example of the drive chunk mapping table 500 before and after chunk rebalancing. Table example 500a shows the state before chunk rebalancing, and table example 500b shows the state after chunk rebalancing.

[0055] When the processing flow shown in Figure 6 is executed, as shown in Table Example 400a in Figure 8, among the multiple drives 230, including drive 230 corresponding to drive ID: drive 1-1 (hereinafter sometimes referred to as "drive 1-1") and drive 230 corresponding to drive ID: drive 1-2 (hereinafter sometimes referred to as "drive 1-2"), drive 1-1 has the highest drive utilization rate (90%), and drive 1-2 has the lowest drive utilization rate (20%).

[0056] In this case, the difference in drive utilization becomes 70%, and in step 610, it is determined that chunk rebalancing is required for multiple drives 230, including drives 1-1 and 1-2. In this example, chunk rebalancing is performed so that the drive utilization of multiple drives 230, including drives 1-1 and 1-2, is equalized.

[0057] As shown in the example table 500a in Figure 9, in step 615, drive 1-1, which shows the maximum drive utilization, is identified as the source drive 230. In step 615, from the identified drive 1-1, the physical chunk 232 (physical chunk ID "P#3") with the highest monitoring IOPS is identified as the source physical chunk 232.

[0058] In step 620, physical chunk 232 (physical chunk ID "P#6") is identified as an unassigned physical chunk 232 for logical chunk 233, which will become the destination physical chunk 232, from among the other drives 230 (drive 1-2) besides drive 1-1. In step 625, as shown in table example 500b in Figure 9, physical chunk 232 (physical chunk ID "P#3") is designated as the source physical chunk 232, and physical chunk 232 (physical chunk ID "P#6") is designated as the destination physical chunk 232, and the physical chunk 232 to be assigned to logical chunk 233 is changed from the source physical chunk 232 to the destination physical chunk 232. Subsequently, data migration is performed from the source physical chunk 232 to the destination physical chunk 232 according to the processing flow in Figure 7. From this point forward, data reads and writes that were previously performed on source physical chunk 232 (physical chunk ID "P#3") (drive 1-1) will no longer occur on source physical chunk 232 (drive 1-1) after the data migration, and will instead be performed on destination physical chunk 232 (physical chunk ID "P#6") (drive 1-2).

[0059] As a result, as shown in Table Example 400b in Figure 8, the drive utilization of the multiple drives 230 is leveled out compared to before chunk rebalancing, and the response time of drive 1-1 improves. In other words, it can be seen that the performance of the storage system is improved by performing chunk rebalancing when there is variation in the utilization of the multiple drives 230.

[0060] <Effects> As described above, the storage system according to the first embodiment of the present invention can improve the performance of the storage system by efficiently operating the entire group of drives 230 (backend drives). In the case of variations in the utilization rates of the multiple drives 230 (backend drives), the storage system according to the first embodiment can improve the performance of the storage system by performing chunk rebalancing to equalize the drive utilization rates of the multiple drives 230 (backend drives), thereby eliminating concentrated access to the drives 230.

[0061] <<Second Embodiment>> A storage system according to a second embodiment of the present invention will now be described. The storage system according to the second embodiment differs from the storage system according to the first embodiment only in the following respects.

[0062] The storage system performs chunk rebalancing across 200 different storage nodes.

[0063] The following explanation will focus on these differences.

[0064] Figure 10 is a diagram illustrating the node resource management table 1000 stored in the memory 220 of the master node 200a. The master node 200a periodically monitors itself, the worker nodes 200b and 200c (every predetermined time interval (e.g., every minute)) and updates the node resource management table 1000 according to the monitoring results.

[0065] As shown in Figure 10, the node resource management table 1000 includes the following columns for storing information (values): node ID 1001, status 1002, CPU usage 1003, bandwidth usage 1004, drive capacity usage 1005, and migration processing status 1006.

[0066] The node resource management table 1000 stores information corresponding to each column of resources for each storage node 200, with each column associated with the others, as row-level information (records). Specifically, node ID 1001 stores an identification ID for identifying the storage node 200. Status 1002 stores the status information of the corresponding storage node 200. CPU usage 1003 stores the CPU usage of the corresponding storage node 200. Bandwidth usage 1004 stores the bandwidth usage of the corresponding storage node 200. Drive capacity usage 1005 stores the drive capacity usage of the corresponding storage node 200. Migration processing status 1006 stores the migration processing status information of the corresponding storage node 200. The migration processing status information can be either "P" or "S". P indicates that the storage node 200 is performing the migration process. S indicates that the storage node 200 has stopped the migration process.

[0067] <Specific operation> Figure 11 is a flowchart showing the processing flow executed by the master node 200a. The master node 200a starts processing from step 1100 and proceeds to step 1105, where it refers to the drive utilization management table 400 for each storage node 200 (master node 200a and worker nodes 200b and 200c) to check the drive utilization of each drive 230 assigned to each storage node 200. Specifically, the master node 200a calculates the difference between the maximum and minimum drive utilization (drive utilization difference) for multiple drives 230 on a per-storage node basis.

[0068] Subsequently, the master node 200a proceeds to step 1110, where it determines whether the difference in drive utilization is greater than or equal to a predetermined differential rebalancing threshold.

[0069] If there are no storage nodes 200 whose drive utilization difference is greater than or equal to a predetermined differential rebalancing threshold, the master node 200a determines "NO" in step 1110 and proceeds to step 1195 to terminate this processing flow. If there are storage nodes 200 whose drive utilization difference is greater than or equal to a predetermined differential rebalancing threshold, the master node 200a determines "YES" in step 1110 and proceeds to step 1115.

[0070] When the master node 200a proceeds to step 1115, it refers to the drive chunk mapping table 500 of the storage node 200 that was determined to require rebalancing in step 1110, identifies the source drive 230 from among the multiple drives 230 assigned to that storage node 200, and then identifies the source physical chunk 232 from among the source drives 230. Specifically, the master node 200a refers to the drive chunk mapping table 500 and identifies the drive 230 with the highest drive utilization as the source drive 230 for the storage node 200 that was determined to require chunk rebalancing (rebalancing) in step 1110, and then identifies the physical chunk 232 showing the highest monitoring IOPS among the multiple physical chunks 232 of the identified source drive 230 as the source physical chunk 232.

[0071] Figure 12 is a flowchart showing the processing flow executed by the master node 200a. The master node 200a processes each storage node 200, excluding the source storage node 200 (hereinafter sometimes referred to as the "source node 200"), one by one, and repeatedly executes the processing flow in Figure 12 until the destination storage node 200 (hereinafter sometimes referred to as the "destination node 200") is identified, or until processing is performed for all target storage nodes 200.

[0072] Master node 200a starts processing from step 1200 and proceeds to step 1205, where it refers to the node resource management table 1000 to check the resource status of storage nodes 200 other than the source node 200.

[0073] Subsequently, the master node 200a proceeds to step 1210, where it determines whether the node status is "Normal" or "Error".

[0074] If the node status is "Error", it is unsuitable as the destination node 200. Therefore, in this case, master node 200a proceeds to step 1295 and terminates this processing flow. If the node status is "Normal", master node 200a proceeds to step 1215 and determines whether the drive capacity utilization is less than 80%.

[0075] If the drive capacity utilization is 80% or higher, it is unsuitable as the migration destination node 200 from the standpoint of excessive load. Therefore, in this case, master node 200a determines "NO" in step 1220 and proceeds to step 1225 to determine whether the bandwidth utilization is less than 70%.

[0076] If the bandwidth utilization rate is 70% or higher, the destination node 200 is unsuitable from the perspective of bandwidth congestion. Therefore, in this case, master node 200a determines "NO" in step 1225 and proceeds to step 1295 to terminate this processing flow.

[0077] If the bandwidth utilization is less than 70%, the master node 200a determines "YES" in step 1225 and proceeds to step 1230 to determine whether the migration processing status is "P" or "S".

[0078] If the migration processing status is "P", it is unsuitable as the destination node 200 from the perspective of processing delay. Therefore, in this case, master node 200a proceeds to step 1295 and terminates this processing flow. If the migration processing status is "S", master node 200a identifies the storage node 200 that was processed in this process as the destination node 200.

[0079] Subsequently, the master node 200a proceeds to step 1295 and terminates this processing flow.

[0080] The master node 200a may repeatedly execute the processing flow shown in Figure 12 until the processing is performed on all storage nodes 200 to be processed. In this case, if multiple destination nodes 200 (candidates) are identified, the master node 200a may select one destination node 200 from among the identified multiple destination nodes 200 (candidates) based on predetermined criteria. For example, the predetermined criteria may be the storage node 200 with the lowest drive capacity utilization.

[0081] Figure 13 is a flowchart showing the processing flow executed by the master node 200a. After the destination node 200 is identified in step 1235 of Figure 12, the master node 200a starts processing from step 1300 and proceeds to step 1305, where it refers to the drive utilization management table 400 of the destination node 200 to check whether the drive status of each of the multiple drives 230 of the destination node 200 is either "Normal" or "Error".

[0082] If all drives 230 are "Error", the master node 200a cannot identify the destination drive 230 (destination physical chunk 232) from the multiple drives 230 assigned to the destination node 200, so the master node 200a proceeds to step 1395 and terminates this processing flow. If at least one drive 230 is "Normal", the master node 200a proceeds to step 1310 and checks whether the drive utilization rate of the "Normal" drive 230 is less than 60%.

[0083] If there are no drives 230 with less than 60% capacity, the destination drive 230 (destination physical chunk 232) cannot be identified, so the master node 200a determines "NO" in step 1310 and proceeds to step 1395 to terminate this processing flow. If there are drives 230 with less than 60% capacity, the master node 200a determines "YES" in step 1310 and proceeds to step 1315.

[0084] When master node 200a proceeds to step 1315, it refers to the drive chunk mapping table 500 to identify drive 230 on the destination node 200 with a drive utilization rate of less than 60%, and then proceeds to step 1320.

[0085] When the master node 200a proceeds to step 1320, it checks whether there is a physical chunk 232 in the identified drive 230 that is not allocated to logical chunk 233.

[0086] If there is no unassigned physical chunk 232 in logical chunk 233, the destination physical chunk 232 cannot be identified, so the master node 200a determines "NO" in step 1320 and proceeds to step 1395 to terminate this processing flow.

[0087] If there is an unassigned physical chunk 232 in logical chunk 233, the master node 200a determines "YES" in step 1320 and proceeds to step 1325. The master node 200a identifies the destination physical chunk 232 from among the unassigned physical chunks 232 in logical chunk 233.

[0088] Subsequently, the master node 200a proceeds to step 1330, where it assigns the destination physical chunk 232 of the drive 230, which is assigned to a different storage node 200 than the source storage node 200, to the logical chunk 233 to which the source physical chunk 232 was previously assigned. After that, the master node 200a proceeds to step 1395, terminating this processing flow.

[0089] Figure 14 is a flowchart showing the processing flow executed by the master node 200a. After changing the physical chunk 232 to be assigned to the logical chunk 233 in step 1320 of Figure 13, the master node 200a starts processing from step 1400 and executes steps 1405 to 1430 described below in order, then proceeds to step 1495 to terminate this processing flow.

[0090] Step 1405: Master node 200a inputs the destination information for logical chunk 233 to source node 200. That is, master node 200a inputs the information for the destination physical chunk 232 to source node 200.

[0091] Step 1410: The master node 200a inputs the source information of logical chunk 233 to the destination node 200. That is, the master node 200a inputs information about the source physical chunk 232 and the logical chunk 233 to which the source physical chunk 232 is assigned to the destination node 200.

[0092] Step 1415: The master node 200a instructs the source node 200 to start the copy process. Upon receiving the copy process instruction, the source node 200 starts a copy process to copy the data stored in the source physical chunk 232 from the source physical chunk 232 of the source node 200 to the destination physical chunk 232 of the destination node 200. The source node 200 notifies the master node 200a when the copy process is complete.

[0093] Step 1420: Master node 200a receives notification that the copy process is complete. Note that after the copy process is complete, the data stored in the source physical chunk 232 is deleted.

[0094] Step 1425: The master node 200a updates the node resource management table 1000 to reflect the state after the copy process and instructs the source node 200 and the destination node 200 to update the information stored in the drive utilization management table 400 and the drive chunk mapping table 500 to reflect the state after the copy process. The source node 200 and the destination node 200, upon receiving the instruction, update the information stored in the drive utilization management table 400 and the drive chunk mapping table 500 to reflect the state after the copy process.

[0095] Figures 15 and 16 are sequence diagrams illustrating an example of operation of the storage system according to the second embodiment.

[0096] S1501: The master node 200a monitors the drive utilization rate of the multiple drives 230 of the worker node 200c by referring to the drive utilization rate management table 400 (configuration information) of the worker node 200c at predetermined intervals (1 hour in this example).

[0097] S1502: The master node 200a monitors the drive utilization rate of the multiple drives 230 of the worker node 200b by referring to the drive utilization rate management table 400 (configuration information) of the worker node 200b at predetermined intervals (1 hour in this example).

[0098] S1503: Master node 200a performs threshold exceedance checks on worker nodes 200c and 200b. Specifically, master node 200a determines whether the difference in drive utilization of worker node 200c is greater than or equal to the threshold (differential rebalancing threshold). Master node 200a also determines whether the difference in drive utilization of worker node 200b is greater than or equal to the threshold (differential rebalancing threshold). In this example, a threshold exceedance is confirmed in worker node 200c.

[0099] S1504: The master node 200a checks the status of the physical chunks 232 of each drive 230 of the worker node 200c by referring to the drive chunk mapping table 500 of the worker node 200c.

[0100] S1505: Master node 200a checks the status of the physical chunks 232 of each drive 230 on worker node 200b by referring to the drive chunk mapping table 500 of worker node 200b.

[0101] S1506: The master node 200a identifies the source physical chunk 232. In this example, the physical chunk 232 of drive 230 assigned to worker node 200c is identified as the source physical chunk 232.

[0102] S1507: Master node 200a identifies the destination node 200. In this example, worker node 200b is identified as the destination node 200.

[0103] S1508: The master node 200a identifies the target physical chunk 232 from the multiple drives 230 assigned to the worker node 200b, which has been identified as the target node 200.

[0104] S1509: Master node 200a initiates the copy process between physical chunks 232.

[0105] S1510: Master node 200a inputs the destination (destination physical chunk 232) of logical chunk 233 to worker node 200c.

[0106] S1511: Master node 200a inputs the source (source physical chunk 232) of logical chunk 233 to worker node 200b.

[0107] S1512: The master node 200a sends a command to the worker node 200b to start the copy process.

[0108] S1513: Worker node 200b sends a copy request to worker node 200c.

[0109] S1514: When worker node 200c receives a copy request, it sends the data to be copied to worker node 200b.

[0110] S1515: When worker node 200b completes the copy process, it sends a copy completion message to master node 200a.

[0111] S1516: Master node 200a updates the drive utilization management table 400 and the drive chunk mapping table 500 of master node 200a to correspond to the state after chunk rebalancing.

[0112] S1517: Master node 200a sends a command to worker node 200c to update the drive utilization management table 400 and the drive chunk mapping table 500. Worker node 200c updates the drive utilization management table 400 and the drive chunk mapping table 500 to correspond to the state after chunk rebalancing.

[0113] S1518: Master node 200a sends a command to worker node 200b to update the drive utilization management table 400 and the drive chunk mapping table 500. Worker node 200b updates the drive utilization management table 400 and the drive chunk mapping table 500 to correspond to the state after chunk rebalancing.

[0114] With the above steps, chunk rebalancing is completed between different storage nodes 200 (in this example, between worker node 200c and worker node 200b).

[0115] <Effects> As described above, the storage system according to the second embodiment of the present invention has the same effects as the first embodiment. In the storage system according to the second embodiment, when there is variation in the utilization rate of multiple drives 230 (backend drives), the storage system can eliminate concentrated access to the drives 230 and improve the performance of the storage system by performing chunk rebalancing between different storage nodes 200 to equalize the drive utilization rate.

[0116] <<Third Embodiment>> A storage system according to the third embodiment of the present invention will now be described. The storage system according to the third embodiment differs from the storage system according to the second embodiment only in the following respects.

[0117] The storage system identifies the destination node 200, taking into account the mode of the destination node 200. The mode of the storage node 200 is either active or standby. Active means the storage node 200 is in use by the storage system (storage cluster). Standby means the storage node 200 is not in use by the storage system (storage cluster). The administrator (user) can set the mode of each storage node 200 to either active or standby via a management device (not shown). By setting unnecessary storage nodes 200 to standby, power savings can be achieved in the storage system.

[0118] The following explanation will focus on these differences.

[0119] Figure 17 is a diagram illustrating the node mode management table 1700. As shown in Figure 17, the node mode management table 1700 includes node ID 1701 and mode 1702 as columns for storing information (values). The node mode management table 1700 stores information corresponding to each column regarding the mode of each storage node 200, with each column associated with the others, as row-level information (records).

[0120] Specifically, node ID 1701 stores the identification information (node ​​ID) of storage node 200. Mode 1702 stores the mode information of the corresponding storage node 200. Mode 1702 stores either "Active" or "Standby" as mode information. Active indicates that the mode of storage node 200 is active. Standby indicates that the mode of storage node 200 is standby.

[0121] Figure 18 is a flowchart showing the processing flow executed by the master node 200a. In the third embodiment, the processing flow of Figure 18 is executed instead of the processing flow of Figure 12. The master node 200a processes each storage node 200 one by one, except for the source node 200, until the destination node 200 is identified, or until processing is completed for all storage nodes 200.

[0122] Master node 200a starts processing from step 1800 and proceeds to step 1801, where it refers to the node mode management table 1700 of the unprocessed storage node 200 to determine whether the mode of that storage node 200 is "Active" or "Standby".

[0123] If the mode of storage node 200 is "Standby", then the destination node 200 is not being used by the storage system and is therefore unsuitable as a destination. In this case, master node 200a proceeds to step 1895 and terminates this processing flow.

[0124] If the storage node 200 is in "Active" mode, the master node 200a proceeds to step 1205. After performing the appropriate processing described in steps 1205 through 1235, it proceeds to step 1895 to terminate this processing flow. The processing flow shown in Figure 18 is executed to identify the destination node 200, taking into account the mode of the storage node 200.

[0125] In addition, similar to the second embodiment, the master node 200a may repeatedly execute the processing flow shown in Figure 12 until the processing is performed for all storage nodes 200 to be processed. In this case, if multiple destination nodes 200 (candidates) are identified, the master node 200a may select one destination node 200 from among the identified multiple destination nodes 200 (candidates) based on predetermined criteria.

[0126] <Effects> As described above, the storage system according to the third embodiment of the present invention has the same effects as the second embodiment. The storage system according to the third embodiment of the present invention can efficiently identify the migration destination node 200 by avoiding the selection of a storage node 200 in standby mode as the migration destination node 200.

[0127] <<Fourth Embodiment>> A storage system according to the fourth embodiment of the present invention will now be described. The storage system according to the fourth embodiment differs from the storage system according to the second embodiment only in the following respects.

[0128] The storage system identifies drive 230 as the source drive, taking into account its QoS (Quality of Service) settings.

[0129] The following explanation will focus on these differences.

[0130] The storage system is configured to provide QoS functionality by allowing QoS settings to be configured on a per-volume basis. By controlling I / O processing on a per-volume basis through QoS settings, the storage system minimizes performance interference between applications and provides consistent performance and quality. For example, QoS settings can include a function to control the upper limit of host I / O to a specific volume 234, and a function to monitor alert thresholds for QoS-targeted volumes 234. Administrators (users) can configure QoS settings for each volume 234 via a management device (not shown).

[0131] Physical chunk 232, which corresponds to volume 234 with the QoS function configured, should prioritize the QoS function and therefore should not be included in the migration target. Accordingly, the storage system according to the fourth embodiment controls the system so that physical chunk 232 with the QoS function configured is not included in the migration target.

[0132] Figure 19 is a diagram illustrating the drive chunk mapping table 1900. As shown in Figure 19, the drive chunk mapping table 1900 includes the following columns for storing information (values): drive ID 1901, P-chunk ID 1902, P-chunk IOPS 1903, L-chunk ID 1904, Volume ID 1905, and migration target mode 1906. Except for the migration target mode 1906, it is the same as the drive chunk mapping table 500 shown in Figure 5. The migration target mode 1906 stores migration target mode information. The migration target mode information stores either "N" or "Y". N indicates that it is not possible to select the corresponding physical chunk 232 as the migration target (source physical chunk 232). Y indicates that it is possible to select the corresponding physical chunk 232 as the migration target (source physical chunk 232). As will be explained in detail in the processing flow shown in Figure 20 below, for physical chunk 232 corresponding to volume 234 with QoS set, the migration target mode will be "N", and for physical chunk 232 corresponding to volume 234 without QoS set, the migration target mode will be "Y". In addition, for physical chunk 232 corresponding to drive 230 with a drive status of "Normal", the migration target mode will be "Y", and for physical chunk 232 corresponding to drive 230 with a drive status of "Error", the migration target mode will be "N".

[0133] Figure 20 is a flowchart showing the processing flow performed by each storage node 200. The storage node 200 starts processing from step 2000 and proceeds to step 2005, where it selects one drive 230 from among its multiple drives 230 that has not yet been configured, and determines whether the drive status of the selected drive 230 is "Error" or "Normal".

[0134] If the drive status is "Error", storage node 200 proceeds to step 2010, sets the migration target mode of physical chunk 232 (target P-chunk) corresponding to drive 230 with a drive status of "Error" to "N", and proceeds to step 2015. If the drive status is "Normal", storage node 200 proceeds to step 2020, determines whether VolumeQoS is configured or not. If VolumeQoS is configured, storage node 200 proceeds to step 2010, sets the migration target mode of physical chunk 232 (target P-chunk) corresponding to volume 234 with VolumeQoS configured to "N", and proceeds to step 2015.

[0135] If VolumeQoS is not configured, storage node 200 proceeds to step 2025, sets the migration target mode of physical chunk 232 (target P-chunk) corresponding to volume 234 with VolumeQoS configured to "Y", and then proceeds to step 2015.

[0136] When storage node 200 proceeds to step 2015, it determines whether it has completed setting the migration target mode for all drives 230 present on storage node 200. If it has not completed setting the migration target mode for all drives 230, storage node 200 determines "NO" in step 2015 and returns to step 2005, where it executes the process of step 2005 again. If it has completed setting the migration target mode for all drives 230, storage node 200 determines "YES" in step 2015 and proceeds to step 2095, terminating this processing flow for the time being.

[0137] Figure 21 is a flowchart showing the processing flow executed by the master node 200a. The master node 200a starts processing from step 2100 and executes the processes of steps 1105 and 1110 described above.

[0138] If master node 200a determines "YES" in step 1110, it proceeds to step 2110. In step 2110, master node 200a refers to the drive chunk mapping table 1900 to identify the source physical chunk 232 whose migration target mode is "Y".

[0139] Subsequently, the master node 200a proceeds to step 2115, where it refers to the drive chunk mapping table 1900 to identify the source physical chunk 232 from among the source physical chunks 232 whose migration target mode is "Y". After that, the master node 200a proceeds to step 2195 and terminates this processing flow.

[0140] After the source physical chunk 232 is identified, the storage node 200 identifies the destination node 200 by executing the flowchart in Figure 12 described above. After the destination node 200 is identified, the storage node 200 assigns the destination physical chunk 232 to the logical chunk 233 to which the source physical chunk 232 is currently assigned, by executing the flowchart in Figure 13 described above.

[0141] <Effects> As described above, the storage system according to the fourth embodiment provides the same effects as the second embodiment. The storage system according to the fourth embodiment of the present invention can improve the performance of the storage system while prioritizing the QoS function by not using the physical chunk 232 corresponding to the volume 234 on which QoS settings are configured as the source physical chunk 232.

[0142] <<Fifth Embodiment>> A storage system according to the fifth embodiment of the present invention will now be described. The storage system according to the fifth embodiment differs from the storage system according to the second embodiment only in the following respects.

[0143] The storage system according to the fifth embodiment prioritizes chunk rebalancing within the same storage node 200, and if chunk rebalancing within the same storage node 200 is not possible, it performs chunk rebalancing between different storage nodes. Chunk rebalancing within the same storage node 200 involves data migration between drives assigned to the same storage node 200, whereas chunk rebalancing between different storage nodes involves data migration between drives on different storage nodes 200. Therefore, chunk rebalancing within the same storage node 200 can be completed relatively faster than chunk rebalancing between different storage nodes. Accordingly, the storage system according to the fifth embodiment can quickly resolve variations in the drive utilization rates of multiple drives 230 by prioritizing chunk rebalancing within the same storage node 200. Furthermore, since the storage system according to the fifth embodiment performs chunk rebalancing between different storage nodes when chunk rebalancing within the same storage node 200 is not possible, it can more reliably resolve variations in the drive utilization rates of multiple drives 230.

[0144] The following explanation will focus on these differences.

[0145] The master node 200a executes the processing flow shown in Figure 11, and if it determines "YES" (chunk rebalancing is required) in step 1110, it proceeds to step 1115, where it refers to the drive chunk mapping table 500 to identify the source physical chunk 232.

[0146] Figure 22 is a flowchart showing the processing flow executed by the master node 200a. The master node 200a starts processing from step 2200 and proceeds to step 2205, identifies the drive 230 where the source physical chunk 232 resides, and then proceeds to step 2210.

[0147] When the master node 200a proceeds to step 2210, it refers to the drive utilization management table 400 to confirm (identify) the storage node 200 to which the drive 230 identified in step 2205 is assigned, and then proceeds to step 2215.

[0148] When the master node 200a proceeds to step 2215, it refers to the node resource management table 1000 and executes the processing flow (not shown) consisting of steps 1210 to 1230 in Figure 12 to check whether there is sufficient resources in the identified storage node 200, thereby determining whether chunk rebalancing is possible in the identified storage node 200 from a resource perspective.

[0149] If the node status is determined to be "Normal" in step 1205 of Figure 12, and "YES" is determined in all of steps 1210 through 1225, and the migration process status is determined to be "Stopped (S)" in step 1230, and there is sufficient resources available on the identified storage node 200, the master node 200a will determine "YES" in step 2215 and proceed to step 2220.

[0150] When the master node 200a proceeds to step 2220, it checks the drive chunk mapping table 500 to see if there is a physical chunk 232 that is not allocated to logical chunk 233 in the identified storage node 200.

[0151] If the identified storage node 200 has an unassigned physical chunk 232 in logical chunk 233, the master node 200a determines "YES" in step 2220 and proceeds to step 2225, identifying the unassigned physical chunk 232 in logical chunk 233 as the destination physical chunk 232. The master node 200a then assigns the identified destination physical chunk 232 to the logical chunk 233 to which the source physical chunk 232 is currently allocated, replacing the source physical chunk 232. After that, the master node 200a proceeds to step 2295 and terminates this processing flow.

[0152] If the identified storage node 200 does not have any unallocated physical chunks 232 in logical chunk 233, the master node 200a determines "NO" in step 2215 and proceeds to step 2230 to perform the inter-node migration process. The inter-node migration process is performed by executing the processing flow shown in Figures 12 to 14. After that, the master node 200a proceeds to step 2295 to terminate this processing flow.

[0153] Furthermore, if in step 2215 the result is "NO" in any of steps 1210 to 1230 in Figure 12, the node status is determined to be "Error" in step 1205 in Figure 12, or "NO" is determined in any of steps 1210 to 1225, or the migration process status is determined to be "Running (P)" in step 1230, and there is insufficient resources on the identified storage node 200, the master node 200a will determine "NO" in step 2215 and proceed to step 2230, execute the process in step 2230 (inter-node migration process) as described above, and then proceed to step 2295 to terminate this process flow.

[0154] <Effects> As described above, the storage system according to the fifth embodiment of the present invention can quickly resolve variations in the drive utilization of multiple drives 230 by prioritizing chunk rebalancing within the same storage node 200, thereby improving the performance of the storage system. Furthermore, if chunk rebalancing within the same storage node 200 is not possible, the storage system according to the fifth embodiment performs chunk rebalancing between different storage nodes, thereby more reliably resolving variations in the drive utilization of multiple drives 230 and improving the performance of the storage system.

[0155] <<Variation>> The present invention is not limited to the embodiments described above, and various modifications can be adopted within the scope of the invention. Furthermore, the embodiments described above can be combined with each other as long as they do not depart from the scope of the invention.

[0156] In each of the above embodiments, the storage system may be constructed using some or all of the storage nodes 200 that make up the storage system as devices (resources) provided by a cloud service. Figure 23 is a diagram illustrating an example of the configuration of a modified storage system. In this modified example, the storage system is constructed using devices provided by a cloud service, with all of the storage nodes 200 that make up the storage system being constructed from devices provided by a cloud service.

[0157] As shown in Figure 23, the system comprises a host device 100 and a virtual storage system constructed from multiple (three in this example) storage nodes 200a to 200c.

[0158] The host device 100 and the storage nodes 200 are connected to each other via the compute network NW1, enabling them to communicate with one another. Each storage node 200 is connected to each other via the inter-storage node network NW2, enabling them to communicate with one another.

[0159] Storage node 200 is composed of a virtual machine comprising a CPU 210, memory 220, network adapter 2340, first virtual network interface 2350, and second virtual network interface 2360. Multiple virtual drives 2330 are allocated to storage node 200. The virtual machine and the multiple virtual drives 2330 are provided by a cloud service. The CPU 210, memory 220, network adapter 2340, first network interface 250, and second network interface 260 are connected to each other via a bus, enabling them to send and receive information. Multiple virtual drives 2330 are connected to network adapter 2340.

[0160] In each of the above embodiments, the drive utilization rate of each drive 230 is compared with a threshold drive utilization rate set higher than the differential rebalancing threshold, and it may be determined that chunk rebalancing is necessary if the drive utilization rate is equal to or greater than the threshold drive utilization rate. In this case, the drive 230 whose drive utilization rate is equal to or greater than the threshold drive utilization rate is designated as the source drive 230, and the source physical chunk 232 may be identified from among the source drives 230.

[0161] In each of the above embodiments, a plurality of destination physical chunks 232 and a plurality of source physical chunks 232 may be identified. [Explanation of Symbols]

[0162] 100…Host device, 200…Storage node, 200a…Storage node, 200b…Storage node, 200c…Storage node, 201…Storage control unit, 210…CPU, 220…Memory, 230…Drive, 232…Physical chunk, 233…Logical chunk, 234…Volume, 400…Drive utilization management table, 500…Drive chunk mapping table

Claims

1. A virtual storage system comprising a storage control unit and a plurality of storage nodes to which a plurality of drives are allocated, and constructed by a plurality of said storage nodes, The drive has multiple physical chunks that are divided for storing data. The storage control unit, In response to I / O requests from the host device, data is read from and written to the physical chunk. The system monitors the drive utilization rate for each of the multiple drives and, based on the drive utilization rates of the multiple drives, performs data relocation by moving the data located in the physical chunk of the source drive to the physical chunk of the destination drive in order to equalize the drive utilization rates of the multiple drives. It is configured in such a way. Storage system.

2. In the storage system according to claim 1, The storage control unit, Based on the drive utilization rate of the multiple drives, the source drive is identified, and based on the access frequency of each physical chunk within the identified source drive, the source physical chunk is identified from among the identified source drives. It is configured in such a way. Storage system.

3. In the storage system according to claim 2, The storage control unit, Based on the drive utilization rate of the multiple drives, the destination drive is identified, and from the identified destination drives, the destination physical chunk is identified. It is configured in such a way. Storage system.

4. In the storage system according to claim 1, The storage control unit, Using the aforementioned drive, a volume, which is a storage area composed of logical chunks to which the physical chunks are assigned, is provided to the host device, and in response to I / O requests from the host device to the logical chunks, data is read from and written to the physical chunks assigned to the logical chunks. After assigning the physical chunk from the destination, which is not yet assigned to the logical chunk to which the physical chunk from the source is assigned, in place of the physical chunk from the source, the data rearrangement is performed. It is configured in such a way. Storage system.

5. In the storage system according to claim 1, The storage control unit, The system determines whether the difference between the maximum and minimum drive utilization rates of the multiple drives is greater than or equal to a threshold, and if the difference between the maximum and minimum drive utilization rates of the multiple drives is greater than or equal to a threshold, it performs the data rearrangement. It is configured in such a way. Storage system.

6. In the storage system according to claim 2, The storage control unit, From among the identified source drives, the physical chunk with the highest access frequency is identified as the source physical chunk. It is configured in such a way. Storage system.

7. In the storage system according to claim 1, The multiple storage nodes consist of a master storage node that manages and controls the entire storage system, and worker storage nodes other than the master storage node. The storage control unit of the master storage node is For each of the master storage node and the worker storage node, the drive utilization rate is monitored for each of the multiple drives on a per-storage node basis. Based on the utilization rates of multiple drives per storage node, the source storage node is identified, and from among the multiple drives assigned to the identified source storage node, the source physical chunk is identified based on the utilization rates of the multiple drives of the source storage node. It is configured in such a way. Storage system.

8. In the storage system according to claim 7, The storage control unit of the master storage node is From among the multiple drives assigned to the identified source storage node, the source physical chunk is identified based on the drive utilization rate of the multiple drives on the source storage node. It is configured in such a way. Storage system.

9. In the storage system according to claim 7, The storage control unit of the master storage node is From among the storage nodes other than the identified source storage node, the destination storage node is identified, and based on the drive utilization rate of the multiple drives of the destination storage node, the destination physical chunk is identified from among the multiple drives assigned to the destination storage node. It is configured in such a way. Storage system.

10. In the storage system according to claim 7, The storage control unit of the master storage node is From among the storage nodes other than the identified source storage node, a storage node that satisfies predetermined conditions regarding the state of the storage node is identified as the destination storage node. It is configured in such a way. Storage system.

11. In the storage system according to claim 10, The predetermined conditions are at least one of the following: condition 1, the drive capacity utilization rate of the storage node is below a predetermined threshold capacity utilization rate; condition 2, the CPU utilization rate of the storage node is below a predetermined threshold CPU utilization rate; condition 3, the bandwidth utilization rate of the storage node is below a predetermined threshold utilization rate; and condition 4, the state of the storage node is not in the process of performing a migration. Storage system.

12. In the storage system according to claim 7, The storage control unit of the master storage node is It is determined whether the identified source storage node satisfies predetermined conditions regarding the state of the storage node, and if the predetermined conditions are met, the destination physical chunk is identified from among the multiple drives assigned to the identified source storage node based on the drive utilization rate of the multiple drives of the source storage node. If the predetermined conditions are not met, the system identifies the destination storage node from among the storage nodes other than the identified source storage node, and identifies the destination physical chunk from among the multiple drives assigned to the destination storage node based on the drive utilization rate of the multiple drives of the destination storage node. It is configured in such a way. Storage system.

13. In the storage system according to claim 7, The storage control unit of the master storage node is When identifying a destination storage node from among the storage nodes other than the identified source storage node, it is checked whether the mode of the storage node is active or standby, and the destination storage node is identified from among the storage nodes whose mode is active. It is configured in such a way. Storage system.

14. In the storage system according to claim 4, The storage control unit, The physical chunk corresponding to the volume with QoS (Quality of Service) settings is excluded from the list of candidate physical chunks to be migrated, thereby identifying the physical chunk to be migrated. It is configured in such a way. Storage system.

15. A method for arranging data in a virtual storage system constructed by a plurality of storage nodes, each of which includes a storage control unit and is allocated multiple drives, The drive has multiple physical chunks that are divided for storing data. The storage control unit, In response to I / O requests from the host device, data is read from and written to the physical chunk. The system monitors the drive utilization rate for each of the multiple drives and, based on the drive utilization rates of the multiple drives, performs data relocation by moving the data located in the physical chunk of the source drive to the physical chunk of the destination drive in order to equalize the drive utilization rates of the multiple drives. Data placement method.

Images