Storage system and node management method
By specifying node attributes and splitting I/O processing, the problem of increasing CPU and memory volume when adding nodes in the storage cluster is solved, achieving more efficient resource utilization and performance improvement.
Patent Information
- Application Number
- CN202010110317.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-02-21
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-02-21
AI Technical Summary
When adding nodes to a storage cluster, the prior art leads to an increase in the amount of CPU and memory, affecting system performance and efficiency.
The user specifies the node attribute to select the started program, divides the I/O processing into front-end and back-end processing, and allocates the storage area to different levels according to the node attributes to reduce the CPU and memory requirements of the additional nodes.
Effectively reduces the amount of CPU and memory required for additional nodes, improves the performance and efficiency of the storage system, especially in large capacity situations, which can suppress throughput reduction and use low-cost physical disks.
Smart Images

Figure CN112306389B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a storage system and a node management method, and is applicable to a storage system and a node management method thereof which are composed of a plurality of storage devices (storage nodes) to form a storage cluster. Background Art
[0002] At present, there is a known technology for a storage system that imports multiple storage devices (storage nodes) as data storage areas and treats them as one storage device (storage cluster). For example, Patent Document 1 discloses a system in which a front-end storage device controls the hierarchy of data configuration based on the structure of a back-end storage device. In addition, when adding a node to a storage cluster, if it has been specified by a user, in the node, a method of using the computer that does not start as a host and instructs I / O processing is usually adopted, but only a storage action that receives instructions from other devices to perform I / O processing is used.
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2012-043407 Summary of the invention
[0006] Technical problem to be solved by the invention
[0007] However, in the case of the above-mentioned conventional technology, since all the processes of the storage control program must be activated when adding a node to the storage cluster, there is a problem that the amount of CPU and memory consumed increases.
[0008] The present invention is completed in consideration of the above aspects, and its purpose is to provide a storage system and a node management method, which, when adding a node to a storage cluster, can select a program to be started by the node based on the attributes of the node specified by the user, thereby reducing the amount of CPU and memory required for the forced-added node.
[0009] Technical solutions for solving technical problems
[0010] In order to solve the above technical problems, the present invention provides a storage system, which uses multiple nodes to form a storage cluster. The storage system includes: a storage device for storing data; a memory; and a processor for running a program for performing I / O processing on the storage device. Furthermore, the node includes a node with a first attribute and a node with a second attribute whose attributes are specified, and the I / O processing in the node is executed as a plurality of processes including front-end I / O processing and back-end I / O processing, wherein the front-end I / O processing is executed by any one of the nodes with the first attribute, and the back-end I / O processing is executed by the node with the first attribute and the node with the second attribute of the storage device having the data involved in the storage I / O processing.
[0011] In addition, in order to solve the above technical problems, the present invention provides a node management method performed by a storage system, wherein the storage system uses multiple nodes to form a storage cluster. In the node management method, the storage system has: a storage device for storing data; a memory; and a processor for running a program for performing I / O processing on the storage device, the node includes a node with a first attribute and a node with a second attribute whose attributes are specified, and the I / O processing in the node is executed as a plurality of processing including front-end I / O processing and back-end I / O processing, wherein the front-end I / O processing is executed by any node with the first attribute, and the back-end I / O processing is executed by the node with the first attribute and the node with the second attribute of the storage device having the data involved in the storage I / O processing.
[0012] Effects of the Invention
[0013] According to the present invention, when adding a node to a storage cluster, it is possible to reduce the amount of CPU and memory required for the added node. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is a diagram showing a configuration example of a storage system according to the first embodiment of the present invention.
[0015] Figure 2 Yes means Figure 1 FIG. 1 is a diagram of an example hardware configuration of a storage system shown in FIG.
[0016] Figure 3 This is a diagram showing an example of a computer program executed by a management node.
[0017] Figure 4 This is a diagram showing an example of a computer program executed by a normal node.
[0018] Figure 5 This is a diagram showing an example of a computer program executed by a lower-level node.
[0019] Figure 6 This is a diagram showing a structural example of a cluster structure management table.
[0020] Figure 7 It is a diagram showing a structural example of a pool management table.
[0021] Figure 8 This is a diagram showing a structural example of a volume management table.
[0022] Fig. 9 It is a diagram showing a structural example of a storage area management table.
[0023] Fig.10 It is a diagram showing a structural example of a storage device management table.
[0024] Fig.11 This is a diagram showing an example of the processing sequence of the node addition process.
[0025] Fig.12 This is a diagram showing an example of the processing sequence of the storage device addition process.
[0026] Fig.13 This is a diagram showing an example of the processing sequence of volume creation processing.
[0027] Fig.14 This is a diagram showing an example of the processing sequence of I / O processing.
[0028] Fig.15 This is a diagram showing an example of the processing sequence of the storage area creation process.
[0029] Fig.16 This is a diagram showing an example of the processing sequence of the structure propagation processing.
[0030] Fig.17 It is a diagram showing an example of the processing sequence of the storage device information update process.
[0031] Fig.18 This is a diagram showing an example of the processing sequence of the back-end I / O processing.
[0032] Fig.19 This is a diagram showing an example of the processing sequence of the hierarchy control process.
[0033] Fig. 20 This is a diagram showing an example of the processing sequence of storage area deletion processing.
[0034] Fig.21 This is a diagram showing an example of a computer program executed by a management node in the second embodiment.
[0035] Fig. 22 Yes means Fig.21 The diagram shows an example of the structure of the cluster structure management table. DETAILED DESCRIPTION
[0036] In the following description, an "interface unit" may be more than one interface. The one or more interfaces may also be more than one communication interface device of the same type (e.g., more than one NIC (Network Interface Card)), or more than two different types of communication interface devices (e.g., NIC and HBA (Host Bus Adapter)).
[0037] In the following description, a "memory unit" refers to one or more memories, and typically may be a main storage device. At least one memory in the memory unit may be a volatile memory or a nonvolatile memory.
[0038] In the following description, a "storage device unit" is one or more storage devices, typically an auxiliary storage device. A "storage device" refers to a physical storage device, typically a non-volatile storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive).
[0039] In addition, in the following description, a "storage section" is at least one of a memory section and a storage device section (typically, at least the memory section).
[0040] In addition, in the following description, a "processor unit" refers to more than one processor. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may also be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be a single-core or multi-core. At least one processor may be a generalized processor such as a hardware circuit that performs part or all of the processing (for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).
[0041] In addition, in the following description, information is sometimes described by expressions such as "xxx table", but the information can be expressed in any data structure. That is, in order to express that the information does not depend on the data structure, "xxx table" can be said to be "xxx information". In addition, in the following description, the structure of each table is just an example, and a table can also be divided into two or more tables, and all or part of two or more tables can also be one table.
[0042] In addition, in the following description, sometimes "program" is used as the subject to describe the processing, but because the program is executed by the processor unit, the memory unit and / or the interface unit can be appropriately used to perform the prescribed processing, so the subject of the processing can also be the processor unit (or a device such as a controller having the processor unit). The program can also be installed from a program source to a device such as a computer. The program source can also be, for example, a program distribution server or a computer-readable (e.g., non-temporary) recording medium. In addition, in the following description, two or more programs can also be implemented as one program, and one program can also be implemented as two or more programs.
[0043] In the following description, "volume" is an abbreviation of logical volume, which can be a logical storage device. A volume can also be a physical volume or a virtual volume. A "physical volume" can be a volume based on physical storage resources (for example, one or more RAID groups) of a storage system that provides the physical volume. A "virtual volume" can be any of a capacity expansion volume and a snapshot volume. A capacity expansion volume is composed of multiple virtual areas (virtual storage areas) and can be a volume that complies with capacity virtualization technology (typically Thin Provisioning). A snapshot volume can be a volume provided as a snapshot of an original volume. A snapshot can also be a physical volume. Typically, a snapshot volume locates the original volume as a primary volume in a secondary volume. A "storage pool (also referred to as a pool)" is a logical storage area (for example, a collection of multiple pool volumes) that can be prepared for each purpose. For example, as a pool, it can be at least one of a capacity expansion pool and a snapshot pool. A snapshot pool can be a storage area that stores data exited from a primary volume. A pool can be used as a capacity expansion pool or as a snapshot pool. A "pool volume" can be a volume that is a component of a pool. A pool volume can also be a physical volume.
[0044] In the following description, a "host system" refers to one or more physical or virtual host computers.
[0045] In addition, in the following description, a "storage cluster (also referred to as a cluster)" is equivalent to a storage device recognized by the host system, and the entity is equivalent to a storage system. A "storage system" includes one or more physical storage devices. A "storage device" can be any device with a storage unit, and can also be a general-purpose computer. At least one physical storage device can also execute a virtual computer (for example, a VM (Virtual Machine)) and can also execute SDx (Software-Definedanything). As SDx, for example, SDS (Software Defined Storage) (an example of a virtual storage device) or SDDC (Software-Defined Data Center) can be used. For example, in the same storage device, a virtual computer as a host system and a virtual computer as a storage device (storage controller) that receives input and output (I / O) requests from the host system for processing can also be executed. In addition, the storage system can also have a redundant structure group. As an example of a redundant structure, it can also be a structure including multiple nodes such as Erasure Coding, RAIN (Redundant Array of Independent Nodes) and inter-node mirroring, or it can be a structure configured in a single node such as one or more RAID (Redundant Array of Independent (or Inexpensive) Disks) groups composed of storage device units.
[0046] Hereinafter, embodiments of the present invention will be described based on the drawings. However, the present invention is not limited to the embodiments described below.
[0047] (1) First Embodiment
[0048] (1-1) Structure and Overview
[0049] Figure 1 1 is a diagram showing an example of the structure of a storage system according to the first embodiment of the present invention. The storage system 100 according to the first embodiment is a storage system having one or more management nodes 101, one or more ordinary nodes 102 and / or lower-level nodes 103, and may be a distributed storage system such as a scale-out storage system. The management node 101, ordinary node 102, and lower-level node 103 are all storage nodes (also referred to as nodes for short) having storage devices 215, and are classified according to the role (attribute) they play in the storage system 100. In the following description, Figure 1As shown, the storage system 100 has a single management node 101 and one or more common nodes 102 or subordinate nodes 103 .
[0050] In the storage system 100 of this embodiment, the computer program can cause the computer to execute the following processing A, processing B, and processing C. The "computer" mentioned here can be any management node 101, or a computer (for example, a management computer or a host computer) connected to the storage system 100. For example, the "computer" can be any management node 101, and the computer program that executes processing A can be the cluster configuration management program 111.
[0051] [Processing A] In the storage system 100 as a storage cluster that is recognized as "one storage device" relative to the host system (hereinafter, referred to as the host), when a storage node (node) is added to the storage cluster (cluster), based on the attribute information of the node specified by the user, the program started by the node is switched. The attribute specified to the node can be classified into at least superior / subordinate. In this embodiment, "superior" is further classified into "management" and "normal".
[0052] [Processing B] When a write instruction in input / output (I / O) is received from the host, data corresponding to the above write instruction is stored in a storage area of an upper level (higher level) in a storage area that is redundant across nodes and whose hierarchy is determined according to the attributes of the nodes.
[0053] [Processing C] The number of I / Os to the storage area within a certain period of time is checked, and the storage area that does not satisfy the application conditions of the I / O specified for each tier is moved to a tier that satisfies the application conditions.
[0054] In the storage system 100 of the present embodiment, by executing the above-mentioned processing A, in the node (lower-level node 103) with lower-level attributes specified by the user, the number of programs to be started is less than that in the node with upper-level attributes (for example, the ordinary node 102), so the amount of CPU and memory required by the node (lower-level node 103) can be reduced.
[0055] In addition, in the storage system 100 of the present embodiment, by executing the above-mentioned processing B and processing C, with respect to the storage area composed of the physical area (storage device 215) possessed by the node (lower-level node 103) to which the lower-level attribute is assigned, even if the input / output amount that can be processed per certain period of time, that is, the throughput, is low, data of the area with less necessary input / output volume within the cluster is stored. Therefore, data can be stored in the entire cluster without reducing the throughput requested by the host.
[0056] In addition, in the storage system 100 of the present embodiment, by executing the above-mentioned processing A, processing B and processing C, when the cluster requires a large capacity, by assigning lower-level attributes to the nodes added to increase the capacity, the throughput reduction can be suppressed and the amount of CPU and memory required for the node can be reduced. Therefore, a physical disk (HDD, etc.) with low throughput but relatively cheap can be used as an additional node.
[0057] Next, continue to refer to Figure 1 The storage system 100 will be described.
[0058] The management node 101 is a node responsible for management in the storage system 100 and is responsible for the host 201 (see Figure 2 ) provides a view 104. Figure 1 As shown, the view 104 displays the volume 105 provided to the host 201 and the storage pool (pool) 106. The storage pool (pool) 106 is a storage area to which the volume 105 belongs, and provides information on the capacity actually used and the capacity that can be allocated. The I / O to the volume 105 is actually performed on the storage area 107, which is a part of the area in the storage pool 106 (reference numeral 131). The storage area 107 is redundantly provided in a plurality of nodes (storage devices 215) by the backend (BE) I / O processing program 115. Thus, a virtual volume is realized.
[0059] Figure 1 132 to 136 are given to the processing flow, indicating a part of the program started by each node 101 to 103 of the storage system 100. Figure 1 In this article, due to space limitations, the word "program" is omitted. Figure 3 to Figure 5 The entire contents of the program started by each node 101 to 103 are described. Figure 1 The cluster structure management table 121 only shows a part of the table, which will be referred to later. Figure 6 Describe the overall structure.
[0060] When using Figure 1 In the configuration of the storage system 100 shown in FIG. 1 , the above-mentioned process A, process B, and process C are as follows.
[0061] First, regarding processing A, when adding a node to the storage system 100, the user (host 201) instructs to add a node whose attributes are variables (parameters), and the cluster structure management program 111 that manages the node 101 receives the instruction. When receiving the instruction to add a node, the cluster structure management program 111 refers to the cluster structure management table 121 and gives the start instruction of the program corresponding to the above-mentioned attribute to the node management program 112 of each node (reference numeral 132). Then, the node management program 112 that receives the program start instruction expands the programs such as the front-end (FE) I / O processing program 113, the hierarchy control program 114, and the back-end (BE) I / O processing program 115 on the storage in the node and starts them (reference numeral 133).
[0062] Next, in terms of process B, when the host 201 requests an I / O process from the storage system 100, the process is performed as follows. The only nodes that receive the I / O process from the host 201 are the management node 101 or the common node 102 where the front-end I / O processing program 113 exists (is started). When the host 201 requests an I / O process (write instruction) for instructing the storage of data to the volume 105, the front-end I / O processing program 113 specifies the ID of the storage area 107 and instructs the back-end I / O processing program 115 to perform the data storage process (reference numeral 134). Thereafter, in order to perform data redundancy, the back-end I / O processing program 115 performs the data storage process on the plurality of storage devices 215 across the nodes (reference numeral 135).
[0063] The storage system 100 is configured to be able to execute the process A and the process B, and the process required for the I / O process is divided into a plurality of processes (front-end I / O process and back-end I / O process) according to the function, and the lower-level node 103 having the lower-level attribute does not execute the front-end I / O process. Therefore, in the lower-level node 103, the number of programs to be started is small compared with the management node 101 or the ordinary node 102, and further, the amount of data such as tables required when executing the program is also small, so the necessary CPU and memory amount can be reduced.
[0064] Next, processing C is described as follows. In the storage system 100, the hierarchy control program 114 sets a plurality of hierarchies according to each storage area 107 constituting the storage pool 106. In the present embodiment, as an example, "higher hierarchy" and "lower hierarchy" are set as a plurality of hierarchies. In addition, applicable conditions are set for each hierarchy. Specifically, for example, "receiving more than 100 I / Os within a certain period of time" is set as the applicable condition for the higher hierarchy, and "receiving less than 100 I / Os within a certain period of time" is set as the applicable condition for the lower hierarchy. The hierarchy control program 114 periodically confirms whether the applicable conditions of the above-mentioned hierarchy are met, and for the storage area 107 that does not meet the applicable conditions, the data allocated to the volume 105 is copied to other hierarchies (reference numeral 136).
[0065] The storage system 100 is configured to execute processing C, for example, by treating a node having a low-function storage device such as an HDD as a lower-level node 103 and placing the storage area 107 created by the storage device 215 of the lower-level node 103 at a "low level", thereby being able to suppress the degradation of performance as a storage system when using a low-function storage device.
[0066] In addition, the above-mentioned process A is equivalent to the node addition process (refer to Fig.11 ), process B is equivalent to I / O processing (refer to Fig.14 etc.), process C is equivalent to hierarchical control process (refer to Fig.19 wait).
[0067] Figure 2 Yes means Figure 1 FIG. 1 is a diagram showing an example of a hardware configuration of a storage system. Figure 2 FIG. 2 illustrates an example of a hardware configuration of a system including a host 201 connected to the storage system 100 via a network.
[0068] like Figure 2 As shown, the storage system 100 includes a plurality of nodes 101 to 103 connected to a network, and further connected to a host 201. Figure 2 LAN (Local Area Network) 221, SAN (Storage Area Network) 222, and LAN 223 are shown as networks connecting the storage system 100. Among them, LAN 221 and SAN 222 are examples of front-end networks, which connect the host 201, the management node 101 of the storage system 100, and the ordinary nodes 102 of the storage system 100. In addition, LAN 223 is an example of a back-end network, which connects multiple nodes (i.e., the management node 101, the ordinary node 102, and the lower-level node 103) constituting the storage system 100. In addition, Figure 2 In the figure, only one host 201, management node 101, common node 102, and subordinate node 103 are shown respectively, but in fact, there can be multiple structures.
[0069] The host 201 is, for example, a server, and one or more hosts 201 are connected to the front-end network (LAN 221, SAN 222). The host 201 receives input operations from the user, requests I / O to the storage system 100, or receives a response notification from the storage system 100. For example, the host 201 issues an I / O request specifying an address belonging to the volume 105 to the storage system 100 via the front-end network. In response, the storage system 100 performs I / O processing on the volume 105 based on the I / O request from the host 201.
[0070] Each node 101 to 103 of the storage system 100 includes a memory 212, a SAN port 213, LAN ports 214 and 216, a storage device 215, and a CPU 211 connected thereto. The CPU 211 is an example of a processor unit, and the memory 212 is an example of a memory unit. In addition, the storage device 215 is an example of a storage device unit. In addition, the SAN port 213 and the LAN ports 214 and 216 are examples of interface units.
[0071] In addition, Figure 2 , the lower-level node 103 does not have the SAN port 213 and the LAN port 214. In the storage system 100 of this embodiment, the management node 101 and the ordinary node 102 need a communication interface for communicating with the host 201, that is, the SAN port 213 or the LAN port 214, in order to perform front-end I / O processing, while the lower-level node 103 does not necessarily need a SAN port 213 or a LAN port 214 because it does not perform front-end I / O processing. In other words, in order to perform back-end I / O processing, the lower-level node 103 only needs to have at least a communication interface for communicating with each node 101 to 103 in the storage system 100, that is, the LAN port 216.
[0072] (1-2) Programs and tables
[0073] Next, the computer program and table used in the storage system 100 of the present embodiment will be described in detail.
[0074] Figure 3 This is a diagram showing an example of a computer program executed by a management node. Figure 3 The computer programs shown are stored in the memory 212 of the management node 101, and the CPU 211 of the management node 101 reads out and executes each computer program to perform processing based on each computer program. Figure 3It also shows an example of a table stored in the memory 212 of the management node 101. When the CPU 211 of the management node 101 executes the above-mentioned computer program, it refers to or updates these tables.
[0075] like Figure 3 As shown, the memory 212 of the management node 101 stores: a cluster structure management program 111, a node management program 112, a front-end (FE) I / O processing program 113, a hierarchy control program 114, a back-end (BE) I / O processing program 115, a storage device management program 116, a storage area management program 117, a storage area creation / deletion program 118, a volume management program 119, and a pool management program 120.
[0076] The cluster configuration management program 111 is a program that receives instructions from a user or a program and issues a configuration change instruction to a program of any node in the cluster.
[0077] The node management program 112 is a program that expands the designated programs and tables on the memory 212 and starts them up.
[0078] The front-end I / O processing program 113 is a program that performs front-end I / O processing including information exchange with the host 201 (user) in the I / O processing. The front-end I / O processing program 113 receives an I / O request for the volume 105 from the host 201, determines the storage area 107 to be the storage destination of the data, issues a storage instruction to the back-end I / O processing program 115, and sends a completion response back to the host 201 when the completion is completed.
[0079] The hierarchy control program 114 is a program for controlling the hierarchy of the storage area 107 , and when data is stored in a storage area 107 of a hierarchy that does not meet the application conditions, the data can be moved to a storage area 107 of a hierarchy that meets the application conditions.
[0080] The back-end I / O processing program 115 is a program that performs back-end I / O processing in I / O processing and exchanges information between nodes, and performs processing based on input data and instructions from the front end. For example, when the back-end I / O processing program 115 receives a storage instruction from the front-end I / O processing program 113, it forwards the data to the node having the storage destination (storage area 107) of the data and instructs the writing of the data to the storage device 215.
[0081] The storage device management program 116 is a program that processes I / O requests to the storage device 215 .
[0082] The storage area management program 117 is a program that updates the storage area management table 124 in the node based on the received instruction.
[0083] The storage area creation / deletion program 118 is a program that determines the correspondence between the storage area 107 and the address of the storage device 215 which is the entity of the storage destination, and instructs the storage area management program 117 of the related node to update the storage area management table 124.
[0084] The volume management program 119 is a program for creating the volume 105 .
[0085] The pool management program 120 is a program that collects information on the capacity within the cluster and displays the information to the user.
[0086] In addition, if Figure 3 As shown, the memory 212 of the management node 101 stores: a cluster structure management table 121, a pool management table 122, a volume management table 123, a storage area management table 124, and a storage device management table 125. In the management node 101, the storage device management table 125 stores information related to all storage devices 215 existing in the cluster, the volume management table 123 stores information related to all volumes 105 existing in the cluster, and the storage area management table 124 stores information related to all storage areas 107 existing in the cluster. In addition, the pool management table 122 stores information indicating the relationship between the corresponding volume 105 and its capacity information (used capacity, unused capacity) for all pools 106 in the cluster. The information stored in the pool management table 122 can be displayed to the user. The structure examples of each table will be described later using Figure 6 to Figure 10 Give a description.
[0087] Figure 4 This is a diagram showing an example of a computer program executed by a normal node. Figure 4 The computer programs shown are stored in the memory 212 of the general node 102, and the CPU 211 of the general node 102 reads out and executes each computer program to perform processing based on each computer program. Figure 4 It also shows an example of a table stored in the memory 212 of the normal node 102. When the CPU 211 of the normal node 102 executes the above-mentioned computer program, it refers to or updates these tables.
[0088] like Figure 4 As shown, the memory 212 of the general node 102 stores: a node management program 112, a front-end (FE) I / O processing program 113, a hierarchy control program 114, a back-end (BE) I / O processing program 115, a storage device management program 116, a storage area management program 117, and a volume management program 119. The description of each program is as follows Figure 3 The same is omitted.
[0089] In addition, if Figure 4As shown, the memory 212 of the general node 102 stores: a volume management table 123, a storage area management table 124, and a storage device management table 125. In the general node 102, the volume management table 123 and the storage device management table 125 store only information related to the volume 105 and the storage device 215 of the node as records. In addition, the storage area management table 124 only needs to store information related to the storage area 107 that becomes the storage destination of the storage device 215 of the node and information related to the storage area 107 allocated to the volume 105 of the node.
[0090] Figure 5 This is a diagram showing an example of a computer program executed by a lower-level node. Figure 5 The computer programs shown are stored in the memory 212 of the lower node 103, and the CPU 211 of the lower node 103 reads out and executes each computer program to perform processing based on each computer program. Figure 5 It also shows an example of a table stored in the memory 212 of the lower node 103. When the CPU 211 of the lower node 103 executes the above-mentioned computer program, it refers to or updates these tables.
[0091] like Figure 5 As shown, the memory 212 of the lower node 103 stores: a node management program 112, a back-end (BE) I / O processing program 115, a storage device management program 116, and a storage area management program 117. The description of each program is as follows Figure 3 The same is omitted.
[0092] In addition, if Figure 5 As shown, the memory 212 of the lower node 103 stores: a storage area management table 124 and a storage device management table 125. In the lower node 103, the storage area management table 124 and the storage device management table 125 only need to store information related to the storage device 215 of the node and the storage area 107 where the storage device 215 becomes the storage destination as records.
[0093] Figure 6 The cluster configuration management table 121 is stored in the management node 101 (see Figure 3 ), each node included in the storage system 100 has a record.
[0094] like Figure 6 As shown, each record of the cluster configuration management table 121 includes information such as a node ID 1211 , an attribute 1212 , and a startup program 1213 .
[0095] The node ID 1211 indicates an identifier (ID) that can uniquely identify each node (storage nodes 101 to 103) in the storage system 100. The attribute 1212 indicates an attribute associated with each node, and the attribute can be specified by the user when adding a node.
[0096] Figure 6 The cluster structure management table 121 is based on Figure 1 An example of the storage system 100 shown in FIG. 100. Specifically, the management node 101 is associated with the attribute of "management" with the node ID "1", the normal node 102 is associated with the attribute of "normal" with the node ID "2", and the two subordinate nodes 103 are associated with the attribute of "subordinate" with the node IDs "3" and "4" respectively. Moreover, the startup program 1213 indicates that the startup program 1213 is associated with the node ID "1". Figure 3 to Figure 5 The corresponding computer program is the detailed content of the program started by each node. In this example, the attribute of the management node 101 is set to "management" and the attribute of the ordinary node 102 is set to "ordinary" to distinguish the attributes of the two, but this embodiment is not limited to this. For example, the attributes of the management node 101 and the ordinary node 102 can also be set to "superior" without distinction.
[0097] In this embodiment, a table (see Figure 3 to Figure 5 ), so only the program is recorded in the startup program 1213, but in the case of a structure such as expanding the program and the table separately, the table can also be recorded in the startup program 1213.
[0098] Figure 7 The pool management table 122 is stored in the management node 101 (see Figure 3 ), each pool 106 of the storage system 100 has a record.
[0099] like Figure 7 As shown, each record of the pool management table 122 includes (i.e., stores) information such as pool ID 1221, belonging volume ID 1222, high-level used capacity 1223, high-level unused capacity 1224, low-level used capacity 1225, and low-level unused capacity 1226.
[0100] The pool ID 1221 indicates an identifier (ID) that can uniquely identify the pool 106 in the storage system 100. The belonging volume ID 1222 indicates a list of IDs of the volumes 105 belonging to the pool 106 of the record. The high-level used capacity 1223 indicates the capacity actually used in the high-level pool among all the volumes 105 belonging to the pool 106 of the record. The high-level unused capacity 1224 indicates the capacity of the unused area in the high-level pool among all the volumes 105 belonging to the pool 106 of the record. The low-level used capacity 1225 indicates the capacity actually used in the low-level pool among all the volumes 105 belonging to the pool 106 of the record. The low-level unused capacity 1226 indicates the capacity of the unused area in the low-level pool among all the volumes 105 belonging to the pool 106 of the record.
[0101] Figure 7 The pool management table 122 is based on Figure 1 An example of the storage system 100 is shown. Specifically, the following structure is shown, that is, in the storage system 100, there is a storage pool 106, and two volumes 105 exist in the storage pool 106, and three storage areas 107 are allocated. Figure 7 In the case of , the capacity is expressed in blocks (Block), but the present embodiment is not limited to this, and units other than blocks such as "Byte" may also be used.
[0102] Figure 8 The volume management table 123 is stored in the management node 101 and the normal node 102 (see Figure 3 , Figure 4 ). The volume management table 123 in the management node 101 has a record for each storage area 107 allocated to all volumes 105 in the storage system 100. In addition, the volume management table 123 in the normal node 102 has a record for each storage area 107 allocated to the volume 105 created by the normal node 102. Figure 8 As an example, Figure 1 Specific values of the volume management table 123 stored in the management node 101 are shown as an example.
[0103] like Figure 8 As shown, each record of the volume management table 123 includes (ie, stores) information such as a volume ID 1231 , an address 1232 , a storage area ID 1233 , a tier 1234 , and an I / O counter 1235 .
[0104] The volume ID 1231 represents an identifier (ID) that can uniquely identify the volume 105 in the storage system 100 and can be used with Figure 7The address 1232 indicates the address of the volume 105 of the record. The storage area ID 1233 indicates the identifier (ID) of the unique storage area in the storage system 100 assigned to the address 1232 of the volume 105 of the record. The hierarchy 1234 indicates the hierarchy to which the storage area indicated by the storage area ID 1233 of the record belongs. In this example, there are two types of hierarchies, the high hierarchy and the low hierarchy. The I / O counter 1235 indicates the number of I / O counters after the last hierarchy control process (see below) was executed. Fig.19 ) and thereafter, the number of I / Os issued to the storage area indicated by the storage area ID 1233 of the record.
[0105] exist Figure 8 In the example, the volume management table 123 stored in the management node 101 is shown. However, in the case of the volume management table 123 stored in the general node 102 or the lower node 103, the data amount can be further reduced as in the case of the management node 101. In the case of reducing the data amount of the volume management table 123 stored in the general node 102 or the lower node 103, it is sufficient to store at least the information related to the entry.
[0106] Fig. 9 The storage area management table 124 is stored in the management node 101, the normal node 102, and the lower node 103 (see Figure 3 to Figure 5 The storage area management table 124 in the management node 101 has a record for each storage area 107 in the storage system 100. The storage area management table 124 in the general node 102 and the lower node 103 has a record for each storage area 107 that the storage device 215 of the node becomes a storage destination. Fig. 9 As an example, Figure 1 Specific values of the storage area management table 124 held (stored) in the management node 101 are illustrated.
[0107] like Fig. 9 As shown, each record of the storage area management table 124 includes (i.e., stores): information such as storage area ID 1241, hierarchy 1242, primary node ID 1243, primary drive ID 1244, primary address 1245, secondary node (Secondary node, also called "secondary node" or "sub-node") ID 1246, secondary drive ID 1247, and secondary address 1248.
[0108] The storage area ID 1241 represents an identifier (ID) that can uniquely identify the storage area 107 in the storage system 100 and can be used with Figure 8The same ID as the storage area ID 1233 of the record. The level 1242 indicates the level to which the storage area 107 of the record belongs, and can be used with Figure 8 The master node ID 1243 indicates the ID of the "master" node that has the sovereignty to process the storage area 107 of the record and is processed as a normal system. The IDs used by the master node ID 1243 and the auxiliary node ID 1246 are the same as Figure 6 The primary drive ID 1244 indicates the ID of the primary storage device 215 that is actually stored when an I / O is issued to the record storage area 107. The primary address 1245 indicates the storage destination address of the primary side that is actually stored when an I / O is issued to the record storage area 107. The secondary node ID 1246 indicates the ID of the "secondary" node that is requested to process the storage area 107 of the record as a redundant destination. The secondary drive ID 1247 indicates the ID of the storage device 215 of the redundant destination that is actually stored when an I / O is issued to the storage area 107 of the record. The secondary address 1248 indicates the address of the redundant destination that is actually stored when an I / O is issued to the storage area 107 of the record.
[0109] Fig.10 The storage device management table 125 is stored in the management node 101, the normal node 102, and the lower node 103 (see Figure 3 to Figure 5 The storage device management table 125 in the management node 101 has a record for each certain area (the same capacity as one storage area 107) of all storage devices 215 in the storage system 100. In addition, the storage device management table 125 in the ordinary node 102 and the lower node 103 has a record for each certain area (the same capacity as one storage area 107) of all storage devices 215 in the node. Fig.10 As an example, Figure 1 Specific values of the storage device management table 125 stored in the management node 101 are illustrated.
[0110] like Fig.10 As shown, each record of the storage device management table 125 includes (ie, stores) information such as a node ID 1251 , a drive ID 1252 , an address 1253 , and a status 1254 .
[0111] The node ID 1251 represents an identifier (ID) that can uniquely identify a node in the storage system 100 and can be used with Figure 6The drive ID 1252 indicates the ID of the unique storage device 215 in the node indicated by the node ID 1251 of the record. In addition, a unique ID may be assigned to the ID used by the drive ID 1252 in the storage system 100. The address 1253 indicates the address of a certain area (the same capacity as one storage area 107) divided in the storage device 215 indicated by the drive ID 1252 of the record. The status 1254 indicates the usage status (in use / unused) of the storage area indicated by the address 1253 of the record as the storage area 107.
[0112] (1-3) Processing
[0113] Next, various processes executed by the storage system 100 according to the present embodiment will be described in detail.
[0114] Fig.11 This is a diagram showing an example of the processing sequence of the node addition process. Fig.11 The node addition process shown is a process executed when adding a node to the storage system 100, and is started when a node addition instruction is received. The node addition instruction specifies a node to be added (additional node) and has the attributes of the node as variables. The attributes of the node are specified by the user, for example.
[0115] according to Fig.11 First, when the cluster configuration management program 111 of the management node 101 receives a node addition instruction with the attribute of the added node as a variable (step S1101), it adds a new record about the added node to the cluster configuration management table 121 stored in the node (step S1102). For example, when the attribute of the node addition instruction for adding the node 103 is designated as "subordinate", the cluster configuration management program 111 adds the ID of the node 103 to the node ID 1211, adds "subordinate" to the attribute 1212, and adds the information of the node management program 112 to the startup program 1213 in the new record of the cluster configuration management table 121 (refer to Figure 6 ). At this time, the program (in this example, the node management program 112) that adds information to the startup program 1213 can be predetermined based on the attribute 1212.
[0116] Thereafter, the cluster configuration management program 111 of the management node 101 instructs the node management program 112 of the node added to the storage system 100 to start other programs (step S1103 ).
[0117] Here, in the additional node (for example, the lower node 103), as an initial state, only the node management program 112 for starting other programs in the node exists (starts up). In this state, when the node management program 112 of the additional node receives the instruction to start the program from the cluster configuration management program 111 of the management node 101 in step S1103 (step S1104), it starts the other program specified as the variable (step S1105).
[0118] Furthermore, in step S1105, the node management program 112 of the additional node does not start all the programs stored in the node (for example, in the lower-level node 103, the same plurality of programs as those in the ordinary node 102 are stored in the memory 212), but only starts the program specified by the instruction of step S1103, and only the area of the memory 212 is secured corresponding to the table required by the program. Specifically, for example, in the lower-level node 103, the back-end I / O processing program 115, the storage device management program 116, and the storage area management program 117 are started (also referred to as "started") by the processing of step S1105, and the storage device management table 125 associated with the storage device management program 116 and the storage area management table 124 associated with the storage area management program 117 are expanded on the memory 212. In this embodiment, since the area of the memory 212 secured for the program to be started and the table associated with the program can be reduced in the node to which the "subordinate" attribute is specified and added in this way, compared with the management node 101 or the ordinary node 102, the amount of CPU 211 and memory 212 required can be reduced compared with the management node 101 and the ordinary node 102.
[0119] After the activation and expansion of the program in step S1105 are completed, the node management program 112 of the added node notifies the cluster configuration management program 111 of the management node 101 of a completion response (step S1106 ).
[0120] Next, the cluster configuration management program 111 of the management node 101 receives the completion response of the program startup instruction from the additional node (step S1107). Based on the completion response of the program startup instruction received, the cluster configuration management program 111 of the management node 101 starts the necessary program in the additional node, determines that the additional node has become capable of executing the function corresponding to the specified attribute, and sends a node addition completion response to the user (step S1108), ending the node addition process.
[0121] Fig.12 This is a diagram showing an example of the processing sequence of the storage device addition process. Fig.12The storage device addition processing shown is the following processing, namely, confirming whether there is an unregistered storage device 215 that is not registered (registered) in the management table (storage device management table 125), and if there is an unregistered storage device 215, taking prescribed measures. The storage device addition processing is executed periodically (for example, every 10 seconds) at each node.
[0122] according to Fig.12 First, in each node (for example, the lower-level node 103 with the least functions), the storage device management program 116 finds the storage device 215 in this node and compares the information of the storage device 215 with the information registered in the storage device management table 125 (step S1201).
[0123] Next, the storage device management program 116 determines whether the storage device 215 compared in step S1201 is not registered in the storage device management table 125 (step S1202), and if it is not registered, the process proceeds to step S1203. On the other hand, if there is no unregistered storage device 215 in step S1202 ("No" in step S1202), the storage device addition process ends.
[0124] In step S1203, the storage device management program 116 adds the information of the unregistered storage device 215 to the storage device management table 125. Specifically, for the unregistered storage device 215, its address is divided into intervals of a certain area (the same capacity as one storage area 107) together with a unique ID equivalent to the node ID 1251 and the drive ID 1252, and new records equal to the number of divisions are created, and the address 1253 is registered, and the status 1254 of each record is registered as "unused".
[0125] Next, the storage management program 116 issues a free capacity update instruction to the pool management program 120 operating in the management node 101, using the number of new records added to the storage management table 125 in step S1203 as a variable (step S1204).
[0126] When the pool management program 120 of the management node 101 receives the free capacity update instruction (step S1205), it converts the number of blocks according to the number of records received as a variable, increases the free capacity of the pool 106, and updates the pool management table 122 (step S1206). After that, the pool management program 120 sends a completion response to the storage device management program 116, which is the source of the free capacity update instruction in step S1204 (step S1207).
[0127] The storage management program 116 receives a completion response to the free capacity update instruction (step S1208), and ends the storage addition process.
[0128] Fig.13 This is a diagram showing an example of the processing sequence of volume creation processing. Fig.13 The volume creation process shown is a process of creating the volume 105 in response to a volume creation instruction from the user.
[0129] according to Fig.13 First, the cluster configuration management program 111 of the management node 101 receives a volume creation instruction from the user to instruct the creation of the volume 105 (step S1301). Next, the cluster configuration management program 111 searches the cluster configuration management table 121 (step S1302), searches for the node started by the volume management program 119 according to the startup program 1213, selects one of the nodes, and issues a volume creation instruction to the selected node (step S1303).
[0130] Next, in the node to which the volume creation instruction is sent, when the volume management program 119 receives the volume creation instruction issued in step S1303 (step S1304), it creates a virtual volume (volume 105) and updates the volume management table 123 (step S1305). In the stage of step S1305, the volume management program 119 registers only the volume ID 1231 in the volume management table 123, and in the subsequent I / O processing, only when data is written to an unallocated address in the storage area 107, the storage area 107 is allocated (address 1232, storage area ID 1233, etc. are registered), thereby realizing automatic thin provisioning. After that, the volume management program 119 sends a completion response to the cluster configuration management program 111, which is the source of the volume creation instruction in step S1303 (step S1306).
[0131] The cluster configuration management program 111 receives the completion response to the volume creation instruction (step S1307). Next, the cluster configuration management program 111 instructs the pool management program 120 of the own node (management node 101) to add volume information using the ID of the created volume 105 (volume ID 1231 registered in the volume management table 123 in step S1305) as an argument (step S1308).
[0132] When the pool management program 120 receives the volume information addition instruction from the cluster configuration management program 111 in step S1308 (step S1309), it records the ID of the volume 105 transmitted as the variable of the volume information addition instruction in the belonging volume ID 1222 of the pool management table 122 (step S1310). After that, the pool management program 120 sends a completion response to the cluster configuration management program 111, which is the source of the volume information addition instruction in step S1308 (step S1311).
[0133] When receiving the completion response to the volume information addition instruction (step S1312), the cluster configuration management program 111 determines that the creation of the volume 105 is completed, responds to the user that the volume creation is completed (step S1313), and ends the volume creation process.
[0134] In addition, in the above-mentioned step S1303, one of the nodes started by the volume management program 119 is selected as the destination for issuing the volume creation instruction, but as the basis for selecting the node at this time, for example, the node with the least volume 105 created among the nodes started by the volume management program 119 can also be selected.
[0135] Fig.14 This is a diagram showing an example of the processing sequence of I / O processing. Fig.14 The I / O processing shown is a processing for inputting and outputting data. Hereinafter, a case of writing processing for storing data will be described as an example.
[0136] according to Fig.14 First, the front-end I / O processing program 113 receives the I / O instruction and data (in this example, the write instruction and write data) sent by the host 201 via the front-end network (LAN221, SAN222) (step S1401). Among them, the 6 nodes started by the front-end I / O processing program 113 are management nodes 101 or ordinary nodes 102.
[0137] Next, the front-end I / O processing program 113 refers to the volume management table 123 (step S1402) to determine whether the storage area 107 is allocated to the address of the volume 105 specified by the I / O instruction from the host 201 (step S1403). If the storage area 107 is allocated in step S1403 ("Yes" in step S1403), the process proceeds to step S1409 described later.
[0138] On the other hand, if the storage area 107 is not allocated in step S1403 ("No" in step S1403), since the storage area 107 is not allocated to the address of the volume 105 specified by the I / O instruction (write instruction), the front-end I / O processing program 113 needs a storage area 107 for storing the specified data (write data). Therefore, in this case, the front-end I / O processing program 113 issues a storage area creation instruction to the storage area creation / deletion program 118 operating in the management node 101 using the hierarchy as an argument (step S1404). In this example, the storage area 107 allocated in response to the write instruction from the host 201 is set from the high hierarchy.
[0139] When the storage area creation instruction is issued in step S1404, the storage area creation / deletion program 118 of the management node 101 receives the instruction, executes the storage area creation process of creating the storage area 107, and after the storage area 107 is created, notifies the completion response (step S1405). Fig.15 Give a description.
[0140] The front end I / O processing program 113 receives a completion response from the storage area creation / deletion program 118 with the storage area ID 1241 and the master node ID 1243 regarding the storage area 107 created in step S1405 as return values (step S1406).
[0141] Next, the front-end I / O processing program 113 adds the information of the storage area ID 1233 and the layer 1234 to the record corresponding to the storage area 107 created in step S1405 in the volume management table 123 (step S1407). In addition, the front-end I / O processing program 113 adds the information of the storage area ID 1241 and the master node ID 1243 to the record corresponding to the storage area 107 created in step S1405 in the storage area management table 124 (step S1408). When the processing of step S1408 is completed, the process proceeds to step S1409.
[0142] In step S1409, the front-end I / O processing program 113 issues a data storage instruction to the back-end I / O processing program 115 operating in the current node, using the ID of the storage area 107 created in step S1405 and the data (write data) received in step S1401 as variables. In addition, if the processing of step S1409 is performed after "Yes" in step S1403, the front-end I / O processing program 113 can replace the ID of the storage area 107 created in step S1405 with the ID of the storage area 107 confirmed to be allocated in step S1403 as a variable.
[0143] When a data storage instruction is issued in step S1409, the back-end I / O processing program 115 of the node receives the instruction, executes the back-end (BE) I / O processing for storing data in the storage area 107 based on the data storage instruction, and notifies the completion response (step S1410). Fig.18 Give a description.
[0144] The front-end I / O processing program 113 receives the completion response from the back-end I / O processing program 115 (step S1411). Next, the front-end I / O processing program 113 updates the I / O counter 1235 in the volume management table 123 (step S1412). Then, the front-end I / O processing program 113 determines that the I / O instruction received from the host 201 has been completed, and returns the completion response to the host 201 (step S1413), and ends the I / O processing.
[0145] In the above description, a high level is assigned to the level of the storage area 107 storing the write data (step S1404 ), but the present embodiment is not limited thereto, and a low level may be assigned to the write data.
[0146] In the above description, the storage area 107 is created in the I / O processing (step S1405), but the present embodiment is not limited to this. For example, the storage area 107 may be created as Fig.11 The node addition process shown is extended. In this case, allocation can also be performed within the I / O process instead of the storage process.
[0147] Fig.15 This is a diagram showing an example of the processing sequence of the storage area creation process. Fig.15 The storage area creation processing shown is a processing for creating a new storage area 107 based on the storage area creation instruction, which is equivalent to Fig.14 Step S1405, Fig.19 Processing of step S1904.
[0148] according to Fig.15 First, the storage area creation / deletion program 118 of the management node 101 receives the storage area creation instruction (step S1501). Next, the storage area creation / deletion program 118 searches the cluster structure management table 121 and selects two nodes having the same level as the level specified by the storage area creation instruction (step S1502). In this example, it is assumed that the "high level" is specified, in which case the management node 101 and the ordinary node 102 are selected.
[0149] Next, the storage area creation / deletion program 118 searches the storage device management table 125 of the management node 101, and selects unused addresses of the storage devices 215 of the two nodes (management node 101, normal node 102) selected in step S1502 (step S1503).
[0150] Then, the storage area creation / deletion program 118 creates a new record in the storage area management table 124 of the management node 101, registering the information of the node selected in step S1502 and the information of the storage device having the address selected in step S1503 in the record (step S1504).
[0151] Next, the storage area creation / deletion program 118 performs structure propagation processing (step S1505). The details of the structure propagation processing will be described later. Fig.16 Next, the storage area creation / deletion program 118 performs a capacity information update process of issuing an instruction to update the capacity information indicated in the pool management table 122 to the pool management program 120 of the management node 101 (step S1506).
[0152] When the pool management program 120 of the management node 101 receives the capacity information update process (step S1507), it reduces the unused capacity (in this example, the high-level unused capacity 1224) by the amount corresponding to the newly created storage area 107 and increases the used capacity (in this example, the high-level used capacity 1223) in the pool management table 122 (step S1507). Thereafter, the pool management program 120 notifies the storage area creation / deletion program 118 of the completion response (step S1509).
[0153] Then, when receiving the completion response of the capacity information update instruction from the pool management program 120 (step S1510), the storage area creation / deletion program 118 determines that the creation of the storage area 107 has been completed, and sends a notification to the source of the storage area creation instruction (for example, Fig.14 In the case of the storage area creation processing of step S1405, the front end I / O processing program 113) notifies the completion response (step S1511) and ends the storage area creation processing.
[0154] In addition, because the above description takes the case of implementing redundancy of the storage area 107 through mirroring as an example, two nodes and storage devices 215 are selected in steps S1502 and S1503 respectively. However, when other RAID or erasure coding is used in the storage system 100, a number greater than 2 can also be selected, and multiple storage areas 107 can also be created at the same time.
[0155] Fig.16 This is a diagram showing an example of the processing sequence of the structure propagation processing. Fig.16 The structure propagation process shown is to propagate the structure information determined by the management node 101 to the associated nodes (normal nodes 102 or lower-level nodes 103), which is equivalent to Fig.15 Step S1505 or Fig. 20 Processing of step S2004.
[0156] according to Fig.16 First, the storage area creation / deletion program 118 of the management node 101 issues an information update instruction to the storage area management program 117 started by the master node constituting the storage area 107 (step S1601).
[0157] Furthermore, the combination of the primary node and the secondary node constituting the storage area 107 can be identified by referring to the storage area management table 124. Specifically, when referring to Fig. 9 , when the level of the storage area 107 is a high level, the master node is the management node 101 (node ID "1"), and the auxiliary node is the ordinary node 102 (node ID "2"); when the level of the storage area 107 is a low level, the master node is the lower node 103 (node ID "3"), and the auxiliary node is the lower node 103 (node ID "4").
[0158] Next, when the storage area management program 117 of the master node receives the information update instruction from the storage area creation / deletion program 118 (step S1602), it updates the storage area management table 124 of the node based on the information (variable) specified by the information update instruction (step S1603). After that, the storage area management program 117 of the master node notifies the storage area creation / deletion program 118 of the management node 101 of the completion response (step S1604).
[0159] Then, the storage area creation / deletion program 118 of the management node 101 receives a completion response from the storage area management program 117 of the master node (step S1605).
[0160] Through the processing of steps S1601 to S1605, the information update of the storage area management table 124 for the primary node is completed. Then, the same processing as steps S1601 to S1605 is performed on the secondary node (steps S1606 to S1610). In order to avoid repetition, detailed description is omitted. Through the processing of steps S1606 to S1610, the information update of the storage area management table 124 for the secondary node is completed.
[0161] After step S1610 is completed, the storage area creation / deletion program 118 performs storage device information update processing (step S1611). The details of the storage device information update processing will be described later. Fig.17 Give a description.
[0162] Fig.17 It is a diagram showing an example of the processing sequence of the storage device information update process. Fig.17The storage device information update process shown is to transmit the structure information determined by the management node 101 to the primary node and the secondary node, and update the information of the storage device management table 125, which is equivalent to Fig.16 Processing of step S1611.
[0163] according to Fig.17 First, the storage area creation / deletion program 118 of the management node 101 issues an information update instruction to the storage device management program 116 started by the master node constituting the storage area 107 (step S1701).
[0164] Next, when the storage device management program 116 of the master node receives the information update instruction from the storage area creation / deletion program 118 (step S1702), the state 1254 of the drive ID 1252 and the address 1253 specified by the information update instruction is updated for the storage device management table 125 of the node (step S1703). Regarding the update of the state 1254 in step S1703, specifically, when the storage area 107 is deleted, "in use (i.e., in use)" is changed to "unused", and when the storage area 107 is created, "unused" is changed to "in use". After that, the storage device management program 116 of the master node notifies the storage area creation / deletion program 118 of the management node 101 of the completion response (step S1704).
[0165] Then, the storage area creation / deletion program 118 of the management node 101 receives a completion response from the storage device management program 116 of the master node (step S1705).
[0166] Through the processing of steps S1701 to S1705, the information update of the storage device management table 125 for the primary node is completed. Then, the same processing as steps S1701 to S1705 is performed on the secondary node (steps S1706 to S1710). Because the information update of the storage device management table 125 for the secondary node is completed through the processing of steps S1706 to S1710, the storage device information update processing ends.
[0167] Fig.18 This is a diagram showing an example of the processing sequence of the back-end I / O processing. Fig.18 The backend processing shown is in Fig.14 The process executed in the I / O processing shown (see step S1410 ) receives and executes a data storage instruction from the front-end I / O processing program 113 .
[0168] according to Fig.18First, the back-end I / O processing program 115 receives the data storage instruction (step S1801). Next, the back-end I / O processing program 115 uses the ID of the storage area 107 assigned as a variable by the data storage instruction to search the storage area management table 124 in the current node (step S1802), and determines whether the master node of the above storage area 107 is the current node (step S1803). In step S1803, if it is determined that the master node of the storage area 107 is the current node ("Yes" in step S1803), the process proceeds to step S1804. If the master node of the storage area 107 is another node ("No" in step S1803), the process proceeds to step S1811.
[0169] First, the case where the master node is the own node in step S1803 is described. In step S1804, the back-end I / O processing program 115 performs data writing processing on the storage device 215 that becomes the master drive in the own node. Thereafter, the back-end I / O processing program 115 instructs the back-end I / O processing program 115 of the auxiliary node to perform data storage processing (step S1805).
[0170] When receiving the data storage instruction issued in step S1805 (step S1806), the backend I / O processing program 115 of the auxiliary node writes the data to the storage device 215 that becomes the auxiliary drive in the auxiliary node (step S1807). After that, the backend I / O processing program 115 of the auxiliary node notifies the backend I / O processing program 115 of the primary node of the completion response (step S1808).
[0171] Then, the backend I / O processing program 115 of the master node receives the completion response notified in step S1808 (step S1809). Since the processing up to this point enables data to be written to the master node and the auxiliary node in the storage area 107 assigned as a variable by the data storage instruction, the backend I / O processing program 115 of the master node sends a data storage instruction to the source (in step S1808). Fig.14 In the case of step S1410, it is the front-end I / O processing program 113, and in the case of step S1812 described later, it is the back-end I / O processing program 115) notifies the completion response (step S1810), and ends the back-end I / O processing.
[0172] Next, the case where the master node is another node in step S1803 is described. In this case, in step S1811, the backend I / O processing program 115 instructs the backend I / O processing program 115 of the master node that exists in a node different from the present node to perform data storage processing. The backend I / O processing program 115 of the master node receives the instruction of step S1811 and executes the data storage processing from step S1801. Fig.18However, since the own node of the forwarding destination is the master node in the forwarding destination, the processing is executed in the branch where the judgment of step S1803 is "yes".
[0173] When the backend I / O processing of the forwarding destination is completed, the backend I / O processing program 115 of the meta node receives the completion response from the backend I / O processing program 115 of the master node of the forwarding destination (step S1813). Then, the backend I / O processing program 115 waits for the action of step S1813, notifies the frontend I / O processing program 113, which is the source of the data storage instruction, of the completion response (step S1814), and ends the backend I / O processing.
[0174] As described above, by executing the backend I / O processing, the backend I / O processing program 115 can write data to the master node and the slave node of the storage area 107 assigned as a variable by the data storage instruction.
[0175] Fig.19 This is a diagram showing an example of the processing sequence of the hierarchy control process. Fig.19 The hierarchical control process shown is a process in which the hierarchical control program 114 checks whether the storage area 107 satisfies the applicable conditions specified by the hierarchy based on the number of I / Os to the storage area 107 within a certain period of time, and changes to a hierarchy that satisfies the applicable conditions if the applicable conditions are not satisfied. The hierarchical control process is executed periodically (for example, every one day) in the management node 101 and the ordinary node 102 started by the hierarchical control program 114. Fig.19 The hierarchy shown controls the processing.
[0176] according to Fig.19 First, the hierarchy control program 114 refers to the volume management table 123 to confirm the hierarchy 1234 and the I / O counter 1235 (step S1901). The hierarchy control program 114 determines whether there is a storage area 107 that does not meet the applicable conditions of the hierarchy by searching in step S1901 (step S1902).
[0177] Next, the judgment criteria of step S1902 are supplemented. In the present embodiment, the applicable range of the number of I / Os within a certain period of time is specified for each level (high level, low level), which is referred to as the "level applicable condition". Specifically, in this example, the I / O counter 1235 being "100" or more within a certain period of time is set as the applicable condition for the high level, and the I / O counter 1235 being less than "100" within a certain period of time is set as the applicable condition for the low level. At this time, in the judgment of step S1902, for each record of the volume management table 123, by distinguishing based on the level 1234, the I / O counter 1235 within a certain period of time (i.e., the count of the I / O counter 1235) is compared with the threshold value "100", and the existence of the storage area 107 that does not meet the applicable condition of the level to which it belongs can be confirmed.
[0178] If it is determined in step S1902 that there is a storage area 107 that does not satisfy the applicable conditions of the hierarchy ("Yes" in step S1902), the process proceeds to step S1903. On the other hand, if it is determined in step S1902 that all created storage areas 107 satisfy the applicable conditions of the hierarchy ("No" in step S1902), no special processing is required, so the hierarchy control process ends.
[0179] In step S1903, regarding the storage area 107 determined to be inapplicable in step S1902, the hierarchy control program 114 issues a storage area creation instruction to the storage area creation / deletion program 118 of the management node 101 using the hierarchy satisfying the applicability condition as a variable.
[0180] The storage area creation / deletion program 118 receives the storage area creation instruction of step S1903, performs storage area creation processing, and newly creates a storage area 107 in a hierarchy that meets the applicable conditions (step S1904). Fig.15 There are instructions in .
[0181] After that, when the hierarchy control program 114 receives the completion response of the storage area creation process (step S1905), it copies the data of the storage area 107 determined not to satisfy the applicable condition in step S1902 to the storage area 107 of the hierarchy newly created in step S1904 (step S1906). Then, the hierarchy control program 114 issues a deletion instruction to delete the storage area 107 that has become the copy source to the storage area creation / deletion program 118 of the management node 101 (step S1907).
[0182] The storage area creation / deletion program 118 receives the deletion instruction of step S1907 and performs storage area deletion processing (step S1908). The details of the storage area deletion processing will be described later. Fig. 20 Give a description.
[0183] Thereafter, when receiving a completion response of the storage area deletion process (step S1909), the hierarchy control program 114 ends the hierarchy control process.
[0184] As described above, by executing the hierarchical control processing, in the storage system 100 of this embodiment, the storage area 107 can be reconfigured according to the applicable conditions of the hierarchy, so that the resources of the entire storage system 100 can be effectively utilized and the processing performance can be improved.
[0185] The following is a more detailed description. As a premise of the hierarchy, a node having a storage device 215 with higher functions such as an SSD is a common node 102, and a storage area created by the storage device 215 is placed at a high level. On the other hand, a node having a storage device 215 with lower functions such as an HDD is a lower node 103, and a storage area 107 created by the storage device 215 is placed at a low level. By performing the above-mentioned hierarchical control processing with this structure, when the I / O counter in the storage area 107 of the high level is less than 100 (small number of accesses), it is not necessary to store it at the high level, so the storage area can be moved to a low-function low level. As a result, it is possible to prevent the overall performance of the storage system from being reduced when a low-function storage device is used. In addition, when the I / O counter in the storage area 107 of the low level is more than 100 (large number of accesses), by storing it at a high-function high level, the processing speed of I / O is accelerated, and the overall processing capacity of the system is increased.
[0186] In addition, in this example, as the applicable condition of the hierarchy, the hierarchy control based on the I / O counter within a certain period of time is described as an example, but the present embodiment is not limited to this, and the hierarchy control may be performed using an index such as the ratio of write processing and read processing in I / O processing, or an index such as whether the volume 105 and the data exist on the same node. In addition, after the data is copied, the process of deleting the storage area 107 of the copy source (steps S1907 to S1908) may not be performed.
[0187] Fig. 20 This is a diagram showing an example of the processing sequence of storage area deletion processing. Fig. 20 The storage area deletion process shown is a process for deleting the storage area 107 based on the storage area deletion instruction, which is equivalent to Fig.19 Processing of step S1908.
[0188] according to Fig. 20First, the storage area creation / deletion program 118 of the management node 101 receives a storage area deletion instruction from the hierarchy control program 114 (step S2001). Next, the storage area creation / deletion program 118 refers to the storage area management table 124 to determine the primary node and the secondary node associated with the storage area ID specified by the storage area deletion instruction (step S2002). Then, the storage area creation / deletion program 118 deletes the record corresponding to the content determined in step S2002 from the storage area management table 124 of the management node 101 (step S2003). Thereafter, the storage area creation / deletion program 118 performs the above-mentioned deletion on the primary node and the secondary node. Fig.16 The structure propagation process shown (step S2004).
[0189] Next, the storage area creation / deletion program 118 issues a capacity information update process to the pool management program 120 to instruct update of the capacity information in the pool management table 122 (step S2005).
[0190] When the pool management program 120 of the management node 101 receives the capacity information update process (step S2006), it increases the unused capacity by the amount corresponding to the deleted storage area 107 and decreases the used capacity in the pool management table 122 (step S2007). Thereafter, the pool management program 120 notifies the storage area creation / deletion program 118 of the completion response (step S2008).
[0191] Then, when the storage area creation / deletion program 118 receives the completion response of the capacity information update instruction from the pool management program 120 (step S2009), it determines that the deletion of the storage area 107 has been completed, and notifies the source of the storage area deletion instruction, i.e., the hierarchy control program 114, of the completion response (step S2010), thereby ending the storage area deletion processing.
[0192] As described above, according to the storage system 100 of this embodiment, each node can switch the program started by each node according to the attribute specified by the user, so when adding a node to the storage cluster, the amount of CPU and memory required for the added node can be reduced.
[0193] In more detail, in the storage system 100 of the present embodiment, since the I / O processing is divided into front-end processing (front-end I / O processing) for exchanging information with the host 201 and back-end processing (back-end I / O processing) for accessing the storage device 215 (data storage), when I / O processing is performed in the storage system, in the node to which the attribute representing the subordinate (in this case, "subordinate") is assigned, the program for performing the front-end I / O processing is not started, and only the program for performing the back-end I / O processing is started, so that the CPU and memory for performing the front-end I / O processing are not required.
[0194] Furthermore, in the storage system 100 of the present embodiment, the storage pool 106 provided by clustering the storage devices 215 of each node is classified into a plurality of hierarchies (in this example, "higher level / lower level") with different allowable frequencies of I / O processing, and the following hierarchical control is performed: the storage area 107 created by the storage device 215 of the node to which the upper attribute (in this example, "management" or "normal") is assigned is placed in the upper level of the storage pool 106 for treatment (i.e., to be managed), and on the other hand, the storage area 107 created by the storage device 215 of the node to which the attribute indicating the lower level (in this example, "lower level") is assigned is placed in the lower level of the storage pool 106 for treatment (i.e., to be managed). By performing such hierarchical control, in the storage system 100 of the present embodiment, the following effect can be expected: by specifying the attribute of the node according to the performance of the storage device 215, it is possible to prevent the overall performance of the storage system 100 from being reduced, and improve the overall processing capacity of the system. Specifically, for example, by specifying the "management" or "normal" attribute to a node of a storage device 215 having a relatively high performance, such as an SSD, and specifying the "lower level" attribute to a node of a storage device 215 having a relatively low performance, such as an HDD, it is possible to use the storage area 107 based on the high-performance storage device 215 at a high level with a high I / O frequency, and use the storage area 107 based on the low-performance storage device 215 at a low level with a relatively low I / O frequency. In order to distinguish between low-performance and high-performance storage devices 215, for example, a prescribed benchmark for processing performance may be set, and if the performance is lower than the prescribed benchmark, the storage device 215 is determined to be a low-performance storage device.
[0195] In addition, in the storage system 100 of the present embodiment, even if Figure 2 A storage node that does not have a SAN port 213 or a LAN port 214, such as the illustrated lower-level node 103, can also be assembled and operated in the storage system 100 by specifying a "lower-level" attribute. That is, even a node with a low-function structure can be added to the storage system 100, so the effect of suppressing the cost of adding nodes can be expected.
[0196] (2) Second Embodiment
[0197] Next, a storage system 300 according to a second embodiment of the present invention will be described. In the description of this embodiment, parts not particularly described have the same configuration as those of the first embodiment and perform the same processing.
[0198] In this embodiment, how to treat the functions other than I / O processing of the storage system of the present invention is described by taking snapshot as an example. The snapshot has the following functions: creating a snapshot volume for saving the data of a primary volume at a certain point in time, and storing the difference with the primary volume in the snapshot volume.
[0199] Fig.21 1 is a diagram showing an example of a computer program executed by a management node in the second embodiment. The storage system 300 of the second embodiment has a management node 301 instead of the management node 101 in the first embodiment. Fig.21 As shown, the memory 310 of the management node 301 stores a plurality of computer programs stored in the memory 212 of the management node 101 (see Figure 3 ) In addition, a snapshot management program 311 for managing the snapshot function is stored. In addition, the table stored in the memory 312 is the same as the table stored in the memory 212, but the program to be started in the cluster configuration management table 321 is different from that in the cluster configuration management table 121.
[0200] The difference from the storage system 100 of the first embodiment is that in the storage system 300 of the present embodiment, the user can specify not only the attributes of the node related to I / O processing (upper / lower), but also the presence or absence of ancillary functions (in this example, the snapshot function is exemplified as an ancillary function related to the management of volume 105) using its attributes as variables.
[0201] Fig. 22 Yes means Fig.21 The cluster configuration management table 321 is stored in the management node 301 and has records for each node in the storage system 300 . Fig.21 The structure of the cluster configuration management table 321 (node ID 3211, attribute 3212, startup program 3213) is similar to that of the first embodiment. Figure 6 The structure of the cluster structure management table 121 shown is the same. Fig. 22 As shown in the record of the node ID “1” in FIG. 3 , the snapshot management program 311 is deployed only in the management node 301 .
[0202] In the storage system 300 configured as described above, when the addition of a node is instructed, the cluster configuration management program 111 determines the program to be deployed in the added node based on the attribute of the designated node, similarly to the first embodiment. Fig.11 Step S1303).
[0203] In addition, as described above, in the storage system 300, when the user instructs to create a volume, whether to create a snapshot can be given as a variable. Furthermore, when the variable of the volume creation instruction indicates that a snapshot needs to be created, the cluster configuration management program 111 Fig.13 In the process of step S1303, a volume creation instruction is issued to the node (in this example, the management node 301) started by the snapshot management program 311. By performing such a process, in the storage system 300, only the amount of memory required according to the function being deployed can be used to operate it.
[0204] As described above, in addition to the effects obtained by the first embodiment, the storage system 300 of the second embodiment has the following effects: an auxiliary function (e.g., snapshot function) related to the management of the volume 105 can also be specified as a node attribute. Furthermore, by dividing the volume 105 management function according to the attribute of the auxiliary function and switching the program to be started for each node, the storage system 300 can be configured in a manner that consumes only the minimum amount of CPU and memory required to realize the auxiliary function.
[0205] The above describes a plurality of embodiments of the present invention, but these are merely examples for explaining the present invention and do not mean that the scope of the present invention is limited to these embodiments. The present invention can also be implemented in various other forms. For example, the startup of a replication program synchronized with other storage systems can be switched instead of the startup of a snapshot program. The properties of a new node such as a snapshot node can also be set for a node with a snapshot program, and the program to be started can be switched according to the properties.
[0206] Description of Reference Numerals
[0207] 100, 300 storage systems
[0208] 101, 301 management nodes (storage nodes, nodes)
[0209] 102 Ordinary nodes (storage nodes, nodes)
[0210] 103 Lower-level nodes (storage nodes, nodes)
[0211] 104 Views
[0212] 105 volumes
[0213] 106 Storage Pool (Pool)
[0214] 107 Storage Area
[0215] 111 Cluster Structure Manager
[0216] 112 Node Manager
[0217] 113 Front-end I / O handler
[0218] 114 Level Control Program
[0219] 115 Backend I / O Handler
[0220] 116 Storage Device Manager
[0221] 117 Storage Area Manager
[0222] 118 Storage Area Creation / Deletion Procedure
[0223] 119 Volume Manager
[0224] 120 Pool Manager
[0225] 121 Cluster structure management table
[0226] 122 Pool Management Table
[0227] 123 Volume Management Table
[0228] 124 Storage Area Management Table
[0229] 125 Storage Device Management Table
[0230] Reference numerals 131 to 136
[0231] 201 Host
[0232] 211 CPU
[0233] 212, 310 Memory
[0234] 213 SAN port
[0235] 214, 216 LAN ports
[0236] 215 Storage Devices
[0237] 221, 223 LAN
[0238] 222 SAN
[0239] 311 Snapshot Manager
Claims
1. A storage system, which uses multiple nodes to form a storage cluster, wherein the storage system is characterized by comprising: Storage devices that store data; Memory; and A processor that runs a program that performs I / O processing on a storage device. The nodes include a node of a first attribute and a node of a second attribute, The I / O processing in the node is performed as a plurality of processing including a front-end I / O processing and a back-end I / O processing, wherein the front-end I / O processing is performed by any one of the nodes of the first attribute, and the back-end I / O processing is performed by a node of the first attribute having the storage device storing data involved in the I / O processing and a node of the second attribute, As the attributes of each of the nodes, a first attribute related to I / O processing and a second attribute related to an auxiliary function for managing volumes created in a storage pool can be specified. The processor of each of the nodes switches the program started by the node according to the first attribute and the second attribute specified for the node.
2. The storage system according to claim 1, characterized in that: The node of the first attribute is a parent node of the attribute designated as a parent, the node of the second attribute is a child node of the attribute designated as a child, the parent node includes a management node for management, The management node designates the upper-level node responsible for executing the front-end I / O processing in the I / O processing performed on the storage device of the lower-level node.
3. The storage system according to claim 1, characterized in that: The attribute of the node is specified by a user when the node is added to the storage cluster.
4. The storage system according to claim 2, characterized in that: The storage pool provided by the storage cluster is divided into multiple levels for management. The processor of the upper node executes hierarchical control for controlling, according to the attribute of each node, at which level of the storage pool a storage area provided by the storage device of each node is managed.
5. The storage system according to claim 4, characterized in that: The levels of the storage pool include: a high level for storing data with a relatively high I / O frequency; and a low level for storing data with a relatively low I / O frequency.
6. The storage system according to claim 5, characterized in that: In the hierarchical control, the processor of the upper node manages the storage area provided by the storage device of the lower node only at the lower level.
7. The storage system according to claim 4, characterized in that: Setting applicable conditions based on I / O frequency for each of the tiers of the storage pool, In the hierarchical control, the processor of the upper node confirms whether the storage area meets the applicability conditions of the current level. If there is a storage area that does not meet the applicability conditions of the current level, the data of the storage area is copied to other levels that meet the applicability conditions.
8. The storage system according to claim 7, characterized in that: In the hierarchical control, the processor of the upper node deletes the data of the copy source after copying the data of the storage area that does not satisfy the application condition of the current hierarchical level to the other hierarchical level that satisfies the application condition.
9. The storage system according to claim 6, characterized in that: When the node added to the storage cluster has the storage device with low performance below a prescribed standard, the attribute of the subordinate is assigned to the node.
10. The storage system according to claim 6, characterized in that: When the node added to the storage cluster does not have an interface required for executing the front-end I / O processing but has an interface required for executing the back-end I / O processing, the attribute of the subordinate is specified for the node.
11. The storage system according to claim 1, characterized in that: In the node to which the highest-level attribute for management is assigned, the memory stores management information on resources of all the nodes of the plurality of nodes, In the node to which the attribute of the upper level or the lower level is specified, the memory stores management information on resources of the node.
12. A node management method for a storage system, wherein the storage system uses a plurality of nodes to form a storage cluster, and the node management method is characterized by: The storage system comprises: a storage device for storing data; a memory; and a processor for executing a program for performing I / O processing on the storage device. The nodes include a node of a first attribute and a node of a second attribute, The I / O processing in the node is performed as a plurality of processes including a front-end I / O processing and a back-end I / O processing, wherein The front-end I / O processing is performed by any node of the first attribute, and the back-end I / O processing is performed by a node of the first attribute and a node of the second attribute of the storage device having data involved in the I / O processing. As the attributes of each of the nodes, a first attribute related to I / O processing and a second attribute related to an auxiliary function for managing volumes created in a storage pool can be specified. The processor of each of the nodes switches the program started by the node according to the first attribute and the second attribute specified for the node.
13. The node management method according to claim 12, characterized in that: The node of the first attribute is a parent node of the attribute designated as a parent, the node of the second attribute is a child node of the attribute designated as a child, the parent node includes a management node for management, The management node designates the upper-level node responsible for executing the front-end I / O processing in the I / O processing performed on the storage device of the lower-level node.
14. The node management method according to claim 13, characterized in that: The storage pool provided by the storage cluster is divided into multiple levels including a high level and a low level for management. The high level is used to store data with a relatively high I / O frequency, and the low level is used to store data with a relatively low I / O frequency. The processor of the upper node executes hierarchical control for controlling, according to the attribute of each of the nodes, at which level of the storage pool a storage area provided by the storage device of each of the nodes to the storage pool is managed.
15. The node management method according to claim 14, characterized in that: Setting applicable conditions based on I / O frequency for each of the tiers of the storage pool, In the hierarchical control, the processor of the upper node confirms whether the storage area meets the applicability conditions of the current level. If there is a storage area that does not meet the applicability conditions of the current level, the data of the storage area is copied to other levels that meet the applicability conditions.
Citation Information
Patent Citations
Hierarchical storage pool management and control for environment of a plurality of loosely-coupled storages
JP2012043407A
Hybrid memory control method and hybrid memory system
CN104679661A
Continuous media system
US20030161302A1
Tiered storage pool management and control for loosely coupled multiple storage environment
US20120047346A1
