Storage system and memory management method

The storage system optimizes memory allocation by distinguishing between active and standby states of storage control units, addressing inefficiencies in redundant configurations to enhance performance and reliability.

JP2024162442APending Publication Date: 2024-11-21HITACHI VANTARA LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023077946
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Existing storage systems with redundant configurations face inefficiencies in memory capacity utilization due to making all cache data redundant without distinction, leading to deteriorated capacity efficiency.

Method used

A storage system with multiple storage nodes, each having a processor and memory, where active and standby storage control units are allocated different memory capacities based on their states, with the system dynamically adjusting memory allocation during state switches to optimize memory usage.

Benefits of technology

The system achieves high performance and reliability by optimizing memory capacity allocation, ensuring efficient use of memory resources while maintaining data redundancy and system availability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162442000001_ABST
    Figure 2024162442000001_ABST
Patent Text Reader

Abstract

To implement a storage system with high performance and high reliability.SOLUTION: A storage system includes a plurality of storage nodes each including a processor and a memory, and a storage device. Each of the plurality of storage nodes includes a storage controller configured to run on the processor. The plurality of storage controllers include an active storage controller configured to process data input to / output from the storage device, and a standby storage controller configured to take over the processing of the data from the active storage controller. Each of the active storage controller and the standby storage controller has a storage area of the memory. The storage node changes the amount of memory capacity allocated for the storage controller of the storage node when a state of the storage controller is switched between a standby state and an active state.SELECTED DRAWING: Figure 15
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a storage system and a memory management method. [Background technology]

[0002] Conventionally, a redundant configuration has been adopted in storage systems to improve availability and reliability. For example, Patent Document 1 proposes the following storage system. In a storage system having a plurality of storage nodes, each storage node is provided with one or more storage devices that provide storage areas, and one or more storage control units that read and write requested data to the corresponding storage devices in response to a request from a host device. Each storage control unit holds predetermined configuration information required to read and write requested data to the corresponding storage device in response to a request from the host device, a plurality of control software are managed as a redundancy group, the configuration information held by each control software belonging to the same redundancy group is updated synchronously, and the plurality of control software that constitute the redundancy group are arranged on different storage nodes so as to distribute the load on each storage node. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-101703 A Summary of the Invention [Problem to be solved by the invention]

[0004] According to Patent Document 1, a storage system that can continue reading and writing even in the event of a node failure can be constructed using a technology for constructing a storage system by software (Software Defined Storage: SDS). Patent Document 1 discloses a technology for increasing the availability of storage control software while efficiently using the server by arranging storage control software for an active system that belongs to a certain redundancy group and storage control software for a standby system that belongs to another redundancy group on the same server. Patent Document 1 also discloses that information in a memory required for controlling storage control software is made redundant between active storage control software and standby storage control software. The information in the memory includes cache data. However, cache data includes data that requires redundancy and data that does not, and if all of it were made redundant without distinguishing between the two and the capacity required for this were also reserved for standby control software, the efficiency of memory capacity would decrease. [Means for solving the problem]

[0005] In order to achieve the above-mentioned object, one representative storage system of the present invention comprises a plurality of storage nodes each having a processor and a memory, and a storage device, each of the plurality of storage nodes having a storage control unit running on the processor, and the plurality of storage control units including an active storage control unit which processes data input / output to the storage device, and a standby storage control unit which takes over processing of the data from the active storage control unit, the active storage control unit and the standby storage control unit each being assigned a storage area of ​​the memory, and the storage node is characterized in that when the state of the storage control unit switches between standby and active, the storage node changes the amount of memory capacity assigned to the storage control unit of its own node. Furthermore, one representative memory management method of the present invention is a memory management method for a storage system comprising a plurality of storage nodes each having a processor and a memory, and a storage device, each of the plurality of storage nodes having a storage control unit running on the processor, wherein the plurality of storage control units include an active storage control unit that processes data input / output to the storage device, and a standby storage control unit that takes over processing of the data from the active storage control unit, wherein the active storage control unit and the standby storage control unit are each assigned a storage area of ​​the memory, and the method includes the steps of: the storage node switching the state of the storage control unit between standby and active; and the storage node changing the allocation of memory capacity to the plurality of storage control units possessed by the own node. Effect of the Invention

[0006] According to the present invention, it is possible to realize a storage system that combines high performance and high reliability. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0007] [Figure 1] Overall configuration of the storage system according to the first embodiment [Diagram 2] Physical configuration of the storage node [Diagram 3] Logical configuration of storage node [Figure 4] Memory configuration diagram [Diagram 5] Software module structure of storage control unit [Figure 6] Cache directory example [Figure 7] Example of a cache allocation management table [Figure 8] Example of free cache capacity management table [Figure 9] Read Processing Flowchart [Figure 10] Write process flowchart [Figure 11] Asynchronous destage processing flowchart [Figure 12] Flowchart of control information update process [Figure 13] Flowchart of cache data update process [Figure 14] Flowchart of failover processing [Figure 15] Flowchart of cache capacity change process during failover [Figure 16] Example of updating the cache capacity management table [Figure 17] Flowchart of cache capacity adjustment process [Figure 18] Software module structure of the second embodiment [Figure 19] Example of cache allocation capacity management table for triplexing [Figure 20] Cache capacity change processing during failover in triplex configuration [Figure 21] FIG. 3 is an explanatory diagram of the third embodiment. [Figure 22] Example of a cache quota management table for managing metadata [Diagram 23] Flowchart of cache capacity change process considering metadata [Figure 24] Example of a cache allocation capacity management table for managing the cache hit rate [Diagram 25] Flowchart of cache capacity change process considering cache hit rate [Figure 26] Failback process flowchart [Figure 27] Flowchart of cache capacity change process during failback DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described in detail with reference to the drawings. The embodiment relates to a storage system including a plurality of storage nodes, each having one or more SDSs implemented therein.

[0009] In the disclosed embodiment, the storage node stores control information and cache data in memory. The storage node includes a non-volatile device. When updating the control information and data in response to a write request from the host, the updated data is stored in the non-volatile device in a log format and responded to the host. At this time, the control information and data are stored in an append format. EXAMPLES

[0010] (1) Example 1 (1-1) Configuration of Storage System of Example 1 FIG. 1 shows the storage system of the first embodiment as a whole. The storage system, for example, comprises a plurality of host devices (Host) 100, a plurality of storage nodes (Storage Node) 300, and a management node (Management Node) 200. The host devices 100, the storage nodes 300, and the management node 200 are connected to each other via a network configured from a Fibre Channel, Ethernet (registered trademark), a LAN (Local Area Network), or the like.

[0011] The host device 100 is a general-purpose computer device that transmits read requests or write requests (hereinafter, these are collectively referred to as I / O (Input / Output) requests) to the storage node 300 in response to requests from user operations, implemented application programs, etc. The host device 100 may be a virtual computer device such as a virtual machine.

[0012] The storage node 300 is a computer device that provides a storage area for reading and writing data to the host device 100. The storage node 300 is, for example, a general-purpose server device.

[0013] The management node 200 is a computer device used by a system administrator to manage the entire storage system. The management node 200 manages a plurality of storage nodes 300 as a group called a cluster. Although an example in which only one cluster is provided is shown in FIG. 1, a plurality of clusters may be provided within the storage system.

[0014] FIG. 2 is a diagram showing an example of a physical configuration of the storage node 300. As shown in FIG. The storage node 300 includes a CPU (Central Processing Unit) 303, a memory 302, a plurality of storage devices (Drives) 301, and a communication device (NIC: Network Interface Card) 304. The CPU 303 is a processor that controls the operation of the entire storage node 300. The memory 302 is composed of semiconductor memories such as a static random access memory (SRAM) and a dynamic RAM (DRAM). The memory 302 is used to temporarily store various programs and necessary data. The CPU 303 executes the programs stored in the memory 302, thereby executing various processes of the entire storage node 300 as described below.

[0015] The storage device 301 is composed of one or more types of large-capacity non-volatile storage devices such as an SSD (Solid State Drive), a SAS (Serial Attached SCSI (Small Computer System Interface)) hard disk drive, a SATA (Serial ATA (Advanced Technology Attachment)) hard disk drive, etc. The storage device 301 provides a physical storage area for reading or writing data in response to an I / O request from the host device 100.

[0016] The communication device 304 is an interface for the storage node 300 to communicate with the host device 100, other storage nodes 300, or the management node 200 via a network. The communication device 304 is configured, for example, with a NIC, an FC card, etc. The communication device 304 performs protocol control during communication with the host device 100, other storage nodes 300, or the management node 200.

[0017] FIG. 3 is a diagram illustrating an example of a logical configuration of a storage node. The storage node 300 includes a front-end driver 510, a back-end driver 540, one or more storage controllers 520, and a data protection controller 540.

[0018] The front-end driver 510 is software having a function of controlling the communication device 304 and providing the CPU 303 with an abstracted interface for the storage control unit 520 when communicating with the host device 100, other storage nodes 300, or the management node 200.

[0019] The back-end driver 540 is software having a function of controlling each storage device in its own storage node, and providing the CPU 303 with an abstracted interface during communication with each storage device.

[0020] The storage control unit 520 is software that functions as a controller of the SDS, that is, control software. The storage control unit 520 accepts an I / O request from the host device 100, and issues an I / O command corresponding to the I / O request to the data protection control unit. The storage control unit 520 also has a logical volume configuration function. The logical volume configuration function associates the logical chunks configured by the data protection control unit 530 with the logical volumes provided to the host device 100. For example, a straight mapping method (a logical chunk and a logical volume are associated with each other on a 1:1 basis, and the address of the logical chunk is the same as the address of the logical volume) may be used, or a virtual volume function (Thin Provisioning) method (a method in which the logical volume and the logical chunk are divided into small-sized areas (pages), and the addresses of the logical volume and the logical chunk are associated with each other on a page basis) may be used.

[0021] In the case of the first embodiment, each storage control unit 520 implemented in the storage node 300 is managed as a pair constituting a redundant configuration together with another storage control unit 520 arranged in another storage node 300. Hereinafter, this pair will be referred to as a storage control unit group. FIG. 3 shows a case where one storage control unit group is made up of two storage control units 520.

[0022] In the storage control unit group, one storage control unit 520 is set to a state (active system state, hereinafter referred to as active mode) in which it can accept I / O requests from the host device 100. In the storage control unit group, the other storage control unit 520 is set to a state (standby system state, hereinafter referred to as standby mode) in which it does not accept I / O requests from the host device 100.

[0023] In the storage control unit group, when a storage control unit set to active mode (hereinafter referred to as active storage control unit) or a storage node in which the active storage control unit is arranged fails, the state of the storage control unit that was set to standby mode up until that point (hereinafter referred to as standby storage control unit) is switched to active mode. This allows the standby storage control unit to take over the I / O processing that the active storage control unit had been executing when the active storage control unit becomes unable to operate.

[0024] The data protection control unit 530 is software that has the function of allocating physical storage areas provided by storage devices in its own storage node or in other storage nodes to each storage control unit group, and reading or writing specified data to the corresponding storage device in accordance with the above-mentioned I / O commands provided by the storage control unit. In this case, when the data protection control unit 530 allocates a physical storage area provided by a storage device in another storage node to a storage control unit group, it cooperates with the data protection control unit implemented in that other storage node to exchange data with that data protection control unit 530 via a network, thereby reading or writing the data to that storage area in accordance with an I / O command provided by the active storage control unit of that storage control unit group.

[0025] 4 is an example of a memory configuration diagram. Storage control information 410 and a cache data area 420 are stored in the memory. The storage control information 410 is an area in which control information for implementing various storage functions is stored, examples of which include a cache directory 411, a cache allocation capacity management table 412, and a free cache capacity management table 413. The cache directory 411 will be explained in Fig. 6, the cache allocation capacity management table 412 in Fig. 7, and the free cache capacity management table 413 in Fig. 8.

[0026] FIG. 5 is a diagram showing an example of a software module structure of the storage control unit 520. As shown in FIG. The storage control unit 520 executes a control information update process 524, a cache data update process 525, a read process 521, a write process 522, and an asynchronous destage process 523. Each process will be described in detail later.

[0027] The control information will be described below. 6 is a diagram showing an example of the cache directory 411. The cache directory 411 is management information for areas (cache segments) obtained by dividing a cache area, and has entries corresponding to each cache segment.

[0028] Each entry has a cache address, a logical volume number, a logical volume address, and an attribute entry. The cache address indicates the address in memory of the cache segment to which each entry corresponds, and the logical volume number and logical volume address indicate which logical volume and which address the data stored in the cache segment belongs to. If no data is stored in the cache segment, a "-" is stored to indicate "no value." If data is stored in the cache segment, the attribute field has a value of Dirty or Clean. Dirty indicates that the data has not yet been destaged and written to the storage device, and Clean indicates that the data has been destaged and matches the value written to the storage device.

[0029] 7 shows the cache allocation capacity management table 412. The cache allocation capacity management table 412 has an entry for each storage control unit 520 constituting each redundancy group of all nodes, and each entry has fields for node ID, redundancy group ID, active / standby state, allocated cache capacity, target cache capacity, clean cache usage, dirty cache usage, and unused cache capacity. The node ID indicates the node in which the storage control unit 520 is located, the redundancy group ID indicates the redundancy group to which the storage control unit 520 belongs, and the active / standby state indicates the active / standby state of each storage control unit 520. The allocated cache capacity indicates the logical capacity allocated to the storage control unit 520 from the memory in the node, and the clean cache usage and dirty cache usage indicate the capacity of the allocated cache capacity that is used for clean data and dirty data, respectively. The unused cache capacity is the free cache capacity of the allocated cache capacity that is not used for either the clean cache or the dirty cache. The dirty cache is a cache that stores data to be written to the storage device (storage device 301). When writing, the storage control unit 520 temporarily saves the write data in the cache before storing it in the storage device. Therefore, there is no identical data in the storage device for dirty data (=dirty). Because redundancy is implemented, the dirty cache is stored in the same way in both the active and standby modes. A clean cache is a cache that stores data read from a storage device (storage device 301). When reading, the storage control unit 520 reads the read data from the storage device, stores it in the cache, and transfers it to the source of the read request. The data is stored in the cache for a prescribed period of time, and if it is read again, it is read from the cache and transferred. Therefore, a clean cache is one in which the same data exists in the storage device (=clean). Since a read is in principle a task performed by an active storage controller, it is preferable for the active storage controller to have more clean cache.

[0030] The active storage control unit and standby storage control unit belonging to the same redundancy group have the same dirty cache usage because they make the dirty cache redundant. On the other hand, the clean cache is held to improve the response performance of the active storage control unit, so the clean cache usage of the standby storage control unit can be less than the clean cache usage of the active storage control unit belonging to the same redundancy group. For example, the dirty cache usage of redundancy group ID0 is 20GB for both active and standby. On the other hand, the clean cache usage of redundancy group ID0 is 40GB for active and 5GB for standby. The dirty cache usage and clean cache usage are adjusted based on the target cache capacity, which will be explained later in the cache capacity adjustment process (FIG. 17).

[0031] The cache capacity allocated to the active storage control unit may be smaller than that allocated to the standby storage control unit that is part of the same redundancy group. This is because, as will be described later, there is no need to make the clean cache redundant, and there is no need to reserve that capacity on the standby storage control unit side. This improves memory capacity efficiency.

[0032] 8 shows a free cache capacity management table 413. The free cache capacity management table 413 has an entry corresponding to each node, and each entry has fields for a node ID, a total cache capacity, an allocated cache capacity, and a free cache capacity.

[0033] 9 is an example of a flowchart of the read process. The read process 521 is called when a read I / O command is received from the host, and is executed by the storage control unit 520. First, the read command sent from the host device 100 is received via the front-end driver 510, and the read command is interpreted to obtain the logical volume number and logical volume address of the read target (step 4001). Next, it is determined whether or not there is a cache hit (step 4002). Specifically, the cache directory 411 is referenced, and an entry corresponding to the logical volume number and logical volume address is searched for in the cache directory 411, and if there is an entry (hit), the cache address is referenced from the entry, and the branch is taken to Yes to read the data from the cache area (step 4006), and the data is returned to the host device 100 (step 4007).

[0034] If there is no entry corresponding to the logical volume number and logical volume address in the cache directory 411 (miss), the process branches to No and calls staging processing (step 4003). The staging processing is a process executed by the data protection control unit 530, and reads data corresponding to the logical volume number / logical volume address from the permanent area on the storage device. The read data is stored in the cache data area 420 on the memory 302 (step 4004). At this time, the cache data update process described later is called with the redundancy necessity set to "unnecessary". The cache directory 411 is updated (step 4005). At this time, since the cache directory is one of the control information, the control information update process described later is called. At this time, the redundancy necessity is set to "unnecessary" as in the cache data update and called. In addition, the cache allocation capacity management table 412 is updated (increasing the clean cache usage due to the read). Then, as in the case of a cache hit, data is read from the cache (step 4006), and the data is responded to the host (step 4007).

[0035] FIG. 10 is an example of a flowchart showing the write process. The write process 522 is executed by the storage control unit 520. First, the write command sent from the host device 100 is received via the front-end driver 510, and the write command is interpreted to obtain the logical volume number and logical volume address of the write target (step 4101). Next, it is determined whether or not there is a cache hit (step 4102). Specifically, the cache directory 411 is referenced, and an entry corresponding to the logical volume number and logical volume address is searched for in the cache directory 411. If there is an entry (hit), the cache address is referenced from the entry, the branch proceeds to Yes, and the data is stored in the cache (step 4104). At this time, the cache data update process described later is called with the redundancy necessity set to "necessary". Next, the corresponding cache directory 411 is updated (step 4105). At this time, since the cache directory 411 is one of the control information, the control information update process described later is called. At this time, the redundancy necessity is set to "necessary" as in the cache data update and called. Also, the cache allocation capacity management table is updated (the dirty cache usage is increased due to the write). Next, the control information confirmation process, which will be described later, is called (step 4106). Finally, a response of write success is sent to the host (step 4107).

[0036] In the case of a cache miss, the process branches to No and a cache segment is reserved (4103). The cache segment reservation is also performed by calling the control information update process. At this time, the redundancy necessity is set to Yes. After that, the process proceeds to step 4104, and thereafter, the same process as in the case of a hit is performed.

[0037] FIG. 11 is a flowchart of the asynchronous destage process. The asynchronous destage process 523 searches for an entry whose attribute is "dirty" from the cache directory 411 (step 4201), and ends if there is no such entry (step 4202, branch No). If there is an entry that is dirty (branch Yes), destage process is executed for that cache segment (steps 4202 and 4203). The destage process is executed by the data protection control unit 530, and writes out data corresponding to the logical volume number / logical volume address in a permanent area on the storage device. At this time, the data may be stored in a redundant manner using a technique such as mirroring or EC (Erasure Coding). After that, the dirty cache is deleted from the cache directory 411 (step 4204). At this time, since this is a cache directory update, the control information update process is called and updated with the redundancy necessity set to "necessary." In addition, the cache allocation capacity management table 412 is updated.

[0038] 12 is a flowchart of the control information update process. The control information update process 524 is called when updating the control information in the memory 302. When called, the memory address and size for identifying the control information to be updated, the update value, and information indicating whether redundancy is required are passed. First, the control information in the memory is updated (step 4301). Next, the passed information on whether redundancy is necessary is referenced to determine whether it is necessary (step 4302). Only if it is necessary, the control information in the memory is made redundant (4303).

[0039] 13 is a flowchart of the cache data update process. Steps 4401 to 4403 are the same as steps 4301 to 4303, except that the update target is cache data rather than control information.

[0040] FIG. 14 is a flowchart of the process during failover. This process is called when a node failure occurs. When a node failure is detected (step 4501), first, Active / Standby switching process is executed (step 4502). This process changes one of the standby storage controllers that belongs to the same redundancy group as the active storage controller arranged in the failed node to the new active storage controller. Then, cache capacity change process during failover is executed for the node that includes the newly active storage controller (step 4503). The cache capacity change process during failover will be described later with reference to FIG. 15.

[0041] 15 is a flowchart of the cache capacity change process at the time of failover. First, a node including a storage control unit 520 that has newly become Active is selected as an adjustment target (step 4601). Next, the total allocated cache capacity (A) of all storage control units 520 in the node is calculated (step 4602), and the total dirty cache usage (B) of all storage control units 520 in the node is calculated (step 4603). Next, the target clean cache usage is calculated by dividing (AB) by 2 (step 4604). For each storage control unit 520 in the node, the sum of its own dirty cache usage and the target clean cache usage is set as the target cache capacity (step 4605). Finally, the cache capacity adjustment process is executed by all storage control units 520 in this node (step 4606). The cache capacity adjustment process will be described with reference to FIG.

[0042] 16 is an example showing how to update the cache capacity management table in the above cache capacity adjustment process. The target cache capacity field is updated and set to 50 GB for both the storage control unit 520 of redundancy group ID 0 arranged in node ID 1 that has newly become Active, and the storage control unit 520 of redundancy group ID 1 that was originally arranged in node ID 1 and is in Active state.

[0043] 17 is a flowchart of the cache capacity adjustment process. When there is a difference between the target cache capacity and the allocated cache capacity, this process actually adjusts the cache allocation amount to fill the difference, and is performed by each storage control unit 520. This process is executed as an extension of the above-mentioned failover, and is executed periodically.

[0044] First, the cache allocation capacity management table 412 is referenced to compare the target cache capacity with the cache capacity allocated to the own storage control unit (step 4701). If the target is smaller, the target reduction amount is set as (allocated cache capacity-target cache capacity) (step 4702), and the process proceeds to step 4703.

[0045] In step 4703, it is checked whether the unused cache amount is smaller than the target reduction amount and whether there is a clean cache usage amount. If YES, since reducing the unused cache will not reduce the target cache capacity, the clean cache usage amount of the own storage control unit is reduced (step 4704). Next, if the unused cache amount has not yet reached the target reduction amount (step 4705), the dirty cache usage amount of the own storage control unit is reduced (step 4706). Then, the unused cache amount increased in steps 4702 to 4706 is released, and the cache allocation amount of the own storage control unit 520 is reduced (step 4707). Then, the process returns to step 4701 again.

[0046] If the branch at step 4701 is NO, it is determined whether the target cache capacity exceeds the allocated cache capacity (step 4708). If it does, and if the free cache capacity of the node is greater than zero, the cache capacity allocated to its own storage control unit is increased (step 4709).

[0047] The method for reducing the clean cache usage in step 4704 is to delete the entries with the clean attribute from the cache directory table. Even if the clean cache data is lost, there is no problem because the same data has already been destaged and it is guaranteed that it can be read by staging it again from the drive.

[0048] As a method for reducing the dirty cache usage in step 4706, for example, there is a method of destaging the dirty data and then deleting the entry from the cache directory, or alternatively, the data may be destaged to change it to clean data and then released as clean cache again in step 4704.

[0049] In this embodiment, when a node failure causes a change from Standby to Active (failover), the cache allocation capacity is increased for the storage control unit 520 that has newly become Active. This maximizes the cache utilization efficiency during normal times when there is no node failure, while at the same time suppressing a decrease in the hit rate in the event of a node failure, thereby improving performance.

[0050] In addition, in this embodiment, when updating cache data or the cache directory, the redundancy requirement is set to "not required" in read processing, and to "required" in write processing. Since the same data is stored in the storage device as the data to be stored in the cache data in read processing, even if it is lost due to a node failure, it can be staged again, so there is no problem. Furthermore, since there is no overhead for memory redundancy in read processing, performance is improved. EXAMPLES

[0051] Next, a second embodiment will be described. This embodiment is an example in which a redundancy group is made up of three storage control units 520 (triple redundancy).

[0052] As shown in FIG. 18, there are three storage control units 520 that make up each redundancy group, which are arranged across three nodes and are in the active, standby 1, and standby 2 states, respectively. If a failure occurs in the placement node of the active storage control unit, it fails over to standby 1, and if a failure occurs in the placement node of standby 1 at that time, it fails over to standby 2. Therefore, the storage control unit 520 can continue to accept IO even if two nodes fail. The data itself may be protected by a data protection unit with triple redundancy or two parities such as RAID6.

[0053] 19 is an example of a cache allocation capacity management table in the case of triplexing. Each node has three storage controllers 520. Fig. 20 shows the cache capacity change process at the time of failover in triplexing. The difference from Fig. 15 is that step 4607 is used instead of step 4604, and when calculating the target cache capacity, the number of storage control units 520 is taken into consideration and the process is divided by 3 instead of 2. In this embodiment, it has been shown that the present invention can be applied to the case of triplexing, but the same is true for quadruple or more. EXAMPLES

[0054] FIG. 21 shows a third embodiment of the present invention. The third embodiment is an example in which a compression function is supported. The compression function is an example of various functions supported by the storage control unit 520. The compression function is a function that reduces capacity consumption, saves disk capacity, and reduces disk costs by compressing and storing data received from the host device 100 using a data compression algorithm such as LZ4.

[0055] As shown in Fig. 21, data received from the host device 100 is stored in the cache data area. The process at this time is similar to the write process shown in Fig. 10, but differs in that the data to be written is compressed and that metadata indicating the compressed data and its position is also written.

[0056] At this time, the compressed data and metadata are written to the cache data area in an append format because, in the compression function, the size of the compressed data received from the host is indefinite (depending on the compression effect), and even if you try to overwrite the old data with compressed data, it may be larger than the old data and not be possible to replace it. Therefore, an append method is suitable, which always allocates new space for writing and releases the space for old data as free space. Also, because it is an append format, information (metadata) that indicates the storage location of the compressed data is required. This metadata is also stored in the cache data area.

[0057] The compressed data and metadata stored in the cache data area are stored in the cache data area of ​​the standby storage control unit arranged in another node in synchronization with the host IO (i.e., before returning a response to the host). In other words, the data and metadata are made redundant.

[0058] As explained in Fig. 11, the destage process executed by the data protection unit during the process can make data redundant by mirroring or erasure coding (EC). Fig. 21 shows an example of mirroring, where a copy of data is sent to other data to make it redundant.

[0059] In this embodiment, metadata is also stored in the cache, but since metadata has a higher degree of locality than user data, meaning that the same data tends to be accessed repeatedly, it can be said that storing metadata in the cache has a high performance improvement effect. Therefore, it is also useful to take metadata into consideration in the cache capacity change process as shown in Fig. 15. For example, when calculating the target clean cache usage in step 4604 of Fig. 15, the target cache capacity can be increased for the storage control unit 520 in which the ratio of metadata to the cache usage is high. In this case, the cache capacity is managed by managing the ratio of metadata to cache data. An example of a cache allocation capacity management table in this embodiment is shown in FIG.

[0060] Unlike this table in the first embodiment, the clean cache usage and dirty cache usage fields are differentiated into "metadata" and "user data", and the clean data amount and dirty data amount fields in the cache allocation capacity management table are further managed by dividing them into "clean data amount (metadata)", "clean data amount (user data)", "dirty data amount (metadata)", and "dirty data amount (user data)".

[0061] 23 shows the cache capacity change process at the time of failover in this embodiment. The difference is that step 4610 is added after step 4603, and metadata is taken into consideration here. In calculating the target cache capacity, for example, the clean cache (metadata) capacity of each storage control unit 520 may also be added, so that a larger cache capacity can be allocated to a node with a larger cache amount of metadata. EXAMPLES

[0062] In this embodiment, the cache hit rate is also managed in the management of the cache capacity. Fig. 24 shows an example of a cache allocation capacity management table in this embodiment. It has a "cache hit rate" field. This field indicates the percentage of cache hits for each I / O, and has a value obtained by aggregating the percentage of cache hits in I / O processing (the percentage of cases where "Yes" was returned in step 4002 in Fig. 9 or 4102 in Fig. 10).

[0063] An example of cache capacity change processing at the time of failover considering the hit rate is shown in Fig. 25. After step 4603, the cache hit rate (C) for each storage control unit 520 of each node is collected (step 4611), and the total cache hit rate (D) of the storage control units 520 of each node is collected (step 4612), and a target clean cache usage is calculated by multiplying (AB) by C÷D so that more is allocated to the storage control unit 520 with the higher hit rate (step 4613). EXAMPLES

[0064] In this embodiment, the cache capacity is adjusted when a failback occurs. The process during failback is shown in Fig. 26. Failback is a process in which the storage control unit 520 is started again (started in standby state) in a node after the node has recovered from a node failure (by removing the failure in the node or replacing the node), and the storage control unit 520 arranged in that node is returned to active mode. In step 4511, the recovery of the node is detected, and in 4512, the storage control unit 520 of the node is switched from standby mode to active mode. After that, a cache capacity change process during failback is called (step 4513).

[0065] Fig. 27 shows a flowchart of the cache capacity change process during failback. This flowchart differs from Fig. 15 in that the recovered node is the target in step 4601, and in that in the target cache capacity calculation in step 4607, AB is set as the target cache amount, taking into account that the node has one active storage control unit 520. In the failback shown in Fig. 27, the active node is moved from the failover destination to the recovery node. With the failback, in the failover destination node, the storage control unit 520 that changed from standby to active due to the failover is returned to standby. In other words, since the memory allocation amount of the storage control unit 520 that became active due to the failover had increased, the memory allocation amount is reduced by the failback to return to the original amount.

[0066] In the flow of Figure 15, Figure 23, Figure 25, and Figure 27, when subtracting the total dirty amount (B) from the total cache allocation amount (A) in the step of calculating the target cache capacity, by subtracting more than that, it is possible to give the standby storage control unit a margin rather than the exact dirty amount. This makes it possible to meet further cache demands for IO in the active storage control unit that constitutes the redundancy group.

[0067] As described above, the disclosed storage system comprises a plurality of storage nodes 300 each having a processor (CPU 303) and a memory 302, and a storage device (storage device 301), and each of the plurality of storage nodes 300 has a storage control unit 520 running on the processor, and the plurality of storage control units 520 include an active storage control unit 520 that processes data input / output to the storage device, and a standby storage control unit 520 that takes over processing of the data from the active storage control unit 520, and the active storage control unit 520 and the standby storage control unit 520 are each assigned a storage area of ​​the memory, and the storage node 300 changes the amount of memory capacity assigned to the storage control unit of its own node when the state of the storage control unit 520 switches between standby and active. This makes it possible for the storage system to effectively utilize the memory of the nodes and to realize a storage system that combines high performance with high reliability.

[0068] As an example, when the state of the storage control unit 520 is switched from standby to active, the storage system increases the memory capacity allocated to the storage control unit 520. Furthermore, when the storage system increases the memory capacity, it reduces the memory allocation amount to other storage control units 520 of the same storage node 300 . Also, as an example, when the state of the storage control unit is switched from active to standby, the storage system reduces the memory capacity allocated to the storage control unit. In this way, the storage system can flexibly change the amount of memory allocated in response to the switching of the state of the storage control unit accompanying failover or failback, thereby enabling effective use of the memory.

[0069] In addition, the storage system forms a redundancy group with the active storage control unit 520 and the standby storage control unit 520 that takes over its processing, and multiple storage control units 520 in the same redundancy group are each placed in different nodes, and multiple storage control units 520 belonging to different redundancy groups are placed in the same node, and the data is made redundant in the multiple storage systems in the redundancy group and stored in the storage device. The cache assigned to the storage control unit 520 stores a dirty cache that stores data to be written to the storage device, and a clean cache that stores data read from the storage device, and the active storage control unit has a larger amount of clean cache stored therein than the standby storage control unit. This makes it possible to maintain performance by leaving a large amount of clean cache in the active system, while reducing the amount of clean cache in the standby system, thereby realizing effective use of memory.

[0070] In addition, the storage node 300 calculates the target cache capacity of each storage control unit 520 by subtracting the total capacity of the dirty cache held by all storage control units 520 of the node from the total capacity of the cache held by all storage control units 520 of the node, dividing it by the number of storage control units 520 of the node, and adding the dirty cache capacity of each storage control unit 520 to the obtained value. This makes it possible to reduce the clean cache while maintaining a dirty cache, thereby achieving memory optimization.

[0071] The storage node 300 is characterized in that it is capable of compressing user data and includes metadata relating to compression in cache data. In addition, the storage node 300 subtracts the total dirty cache capacity held by all storage control units 520 of its own node from the total cache capacity held by all storage control units 520 of its own node, subtracts the total clean cache capacity of metadata held by all storage control units 520 of its own node, divides this by the number of storage control units 520 of its own node, and calculates the target cache capacity of each storage control unit 520 by adding the dirty cache capacity and clean metadata cache capacity of each storage control unit 520 to the obtained value. This allows for improved memory efficiency when compression is used.

[0072] Furthermore, the storage node 300 sets a target cache capacity for each storage control unit 520 of the node itself, and if the cache capacity of any storage control unit 520 is larger than the target cache capacity, the cache capacity of that storage control unit is reduced. At this time, the storage node 300 reduces the cache capacity by reducing the clean cache first, and if further cache capacity reduction is necessary after the clean cache has been completely reduced, the storage node 300 destages the dirty cache to reduce the cache capacity. This allows for efficient memory usage while deleting dirty caches as needed.

[0073] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment is described in detail to easily explain the present invention, and is not necessarily limited to the embodiment having all the described configurations. Moreover, the present invention is not limited to the deletion of the configurations, and it is also possible to replace or add the configurations. [Explanation of symbols]

[0074] 100: host device, 200: management node, 300: storage node, 301: storage device, 302: memory, 303: CPU, 304: communication device, 410: storage control information, 411: cache directory, 412: cache allocation capacity management table, 413: free cache capacity management table, 420: cache data area, 510: front-end driver, 520: storage control unit, 521: read processing, 522: write processing, 523: asynchronous destage processing, 524: control information update processing, 525: cache data update processing, 530: data protection control unit, 540: back-end driver

Claims

1. A storage device and a plurality of storage nodes each having a processor and a memory, Each of the plurality of storage nodes has a storage control unit that operates on the processor, the plurality of storage control units include an active storage control unit that processes data input / output to the storage device, and a standby storage control unit that takes over processing of the data from the active storage control unit; the active storage control unit and the standby storage control unit are each assigned a storage area of ​​the memory; The storage node changes the amount of memory capacity allocated to the storage control unit of the storage node when the state of the storage control unit switches between standby and active. A storage system comprising:

2. 2. The storage system according to claim 1, When the state of the storage control unit is switched from standby to active, the memory capacity allocated to the storage control unit is increased. A storage system comprising:

3. 3. The storage system according to claim 2, When the memory capacity is increased, the memory allocation amount to other storage control units of the same storage node is decreased. A storage system comprising:

4. 2. The storage system according to claim 1, When the state of the storage control unit is switched from active to standby, the memory capacity allocated to the storage control unit is reduced. A storage system comprising:

5. 2. The storage system according to claim 1, a redundancy group is formed by the active storage control unit and the standby storage control unit which takes over the processing of the active storage control unit; A plurality of storage control units in the same redundancy group are arranged in different nodes, and a plurality of storage control units belonging to different redundancy groups are arranged in the same node; The data is made redundant and stored in the storage device in the plurality of storage systems in the redundancy group. A storage system comprising:

6. 6. The storage system according to claim 5, The cache allocated to the storage control unit stores a dirty cache for storing data to be written to the storage device, and a clean cache for storing data read from the storage device; The active storage controller has a larger amount of clean cache stored than the standby storage controller. A storage system comprising:

7. 7. The storage system according to claim 6, The storage node has one active storage control unit and two standby storage control units, and when one of the standby storage control units switches to active, the other standby storage control unit reduces the memory capacity used to hold clean cache.

8. 7. The storage system according to claim 6, The storage system is characterized in that the storage node calculates the target cache capacity of each storage control unit by subtracting the total capacity of the dirty cache held by all storage control units of the node from the total capacity of the cache held by all storage control units of the node, dividing the result by the number of storage control units of the node, and adding the dirty cache capacity of each storage control unit to the obtained value.

9. 2. The storage system according to claim 1, A storage system, wherein the storage node is capable of compressing user data, and cache data includes metadata relating to compression.

10. 7. The storage system according to claim 6, The storage node calculates a target cache capacity for each storage control unit by subtracting the total dirty cache capacity held by all storage control units of the node from the total cache capacity held by all storage control units of the node, subtracting the total clean cache capacity of metadata held by all storage control units of the node, dividing the result by the number of storage control units of the node, and adding the dirty cache capacity and clean cache capacity of metadata of each storage control unit to the obtained value.

11. 7. The storage system according to claim 6, A storage system characterized in that the storage node sets a target cache capacity for each storage control unit of the node, and if the cache capacity of any storage control unit is larger than the target cache capacity, reduces the cache capacity of that storage control unit.

12. 12. The storage system according to claim 11, The storage node reduces the cache capacity by preferentially reducing the clean cache, and if further cache capacity reduction is necessary after the clean cache has been completely reduced, destages the dirty cache to reduce the cache capacity.

13. A memory management method for a storage system comprising a plurality of storage nodes each having a processor and a memory, and a storage device, wherein each of the plurality of storage nodes has a storage control unit that operates on the processor, comprising: the plurality of storage control units include an active storage control unit that processes data input / output to the storage device, and a standby storage control unit that takes over processing of the data from the active storage control unit; the active storage control unit and the standby storage control unit are each assigned a storage area of ​​the memory; the storage node switching the state of the storage control unit between standby and active; the storage node changing the allocation of memory capacity to a plurality of storage control units included in the node itself; 13. A memory management method comprising:

Citation Information

Patent Citations

  • Storage system and control software arrangement method

    JP2019101703A