Cache allocation method and device, equipment and medium

Abstract

The invention discloses a cache allocation method and device, equipment and a medium in the technical field of computers. The method comprises the following steps of: after determining the total amount of mirror image pairs in the multi-control storage system, averagely dividing a total cache space in a node into a plurality of partitions according to the total amount of the mirror image pairs; in the running process of the multi-control storage system, space use information of all the partitions is monitored; and if the cache redistribution condition is met, redistributing the residual available space of each partition according to the space use information of each partition, so that the residual available space of each partition is relatively balanced. According to the scheme, after the cache space is equally divided, the cache space of each partition is adjusted in real time according to the space use information of each partition, cache redistribution and flexible adjustment of the multi-control storage system are achieved, the remaining available space of each partition can be relatively balanced, and the utilization rate of the cache space is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a cache allocation method, device, equipment and medium. Background Art

[0002] Currently, the cache space in a multi-controller storage system is fixed after being allocated to each mirror pair (domain). After a period of use, the cache space of some mirror pairs in the multi-controller storage system is almost full, while the cache space of other mirror pairs is relatively idle and has low utilization. The mirror pairs whose cache space is almost full may cause front-end business interruption due to exhaustion of dirty page resources in the cache space.

[0003] Therefore, how to flexibly adjust the cache allocation scheme of a multi-controller storage system is a problem that those skilled in the art need to solve. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a cache allocation method, device, equipment and medium to flexibly adjust the cache allocation scheme of a multi-controller storage system. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a cache allocation method, which is applied to any node in a multi-controller storage system, comprising:

[0006] Obtaining the total amount of mirror pairs in the multi-controller storage system;

[0007] Dividing the total cache space in the node into a plurality of partitions evenly according to the total amount of the mirror pairs;

[0008] During the operation of the multi-control storage system, space usage information of each partition is monitored respectively;

[0009] If the cache reallocation condition is met, the remaining available space of each partition is reallocated according to the space usage information of each partition, so that the remaining available space of each partition is relatively balanced.

[0010] Optionally, monitor the space usage information of each partition separately, including:

[0011] According to a preset first cycle, the used water level information of each partition is detected as the space usage information.

[0012] Optionally, the remaining available space of each partition is reallocated according to the space usage information of each partition, including:

[0013] Determine, in each partition, a first partition whose used water level information is greater than a preset threshold, and a second partition whose used water level information is the smallest;

[0014] Migrate part of the remaining available space in the second partition to the first partition, and keep other partitions unchanged.

[0015] Optionally, migrating part of the remaining available space in the second partition to the first partition includes:

[0016] Determine the migration ratio;

[0017] Calculating the product of the remaining available space in the second partition and the migration ratio, and using the product as the part of the remaining available space;

[0018] Migrate the portion of remaining available space to the first partition.

[0019] Optionally, monitor the space usage information of each partition separately, including:

[0020] According to the preset second cycle, the following steps are performed for each partition:

[0021] Check the used water level information and used cache space of the current partition;

[0022] Calculate the difference between the used water level information and the historical average water level information, and use the quotient of the difference and the historical average water level information as the used space change rate of the current partition;

[0023] Calculate the difference between the used cache space and the historical average used space, and use the quotient of the difference and the historical average used space as the water level change rate of the current partition;

[0024] The used space change rate and load change rate of the current partition are used as the space usage information of the current partition.

[0025] Optionally, the remaining available space of each partition is reallocated according to the space usage information of each partition, including:

[0026] For each partition, respectively calculate the product of the load change rate and the preset first coefficient, and use the sum of the product and the used space change rate of the corresponding partition as the adjustment coefficient of the corresponding partition;

[0027] For each partition, respectively calculate the product of the adjustment coefficient and the preset second coefficient, and use the sum of the product and the total current cache space of the corresponding partition as the pre-selected space of the corresponding partition;

[0028] Select the maximum value between the pre-selected space and the current cache space of the same partition as the pre-allocated space of the partition;

[0029] The allocation ratio of each partition is calculated based on the pre-allocated space of each partition;

[0030] Calculate the product of the allocation ratio of each partition and the total cache space in the node respectively to obtain the size of the cache space occupied by each partition;

[0031] The total cache space in the node is reallocated according to the size of the cache space occupied by each partition.

[0032] Optionally, it also includes:

[0033] During the operation of the multi-control storage system, generating operation adjustment instructions for each partition according to the space usage information of each partition;

[0034] The number of operations to be executed received by each partition is adjusted according to the operation adjustment instruction.

[0035] In a second aspect, the present application provides a cache allocation device, which is applied to any node in a multi-controller storage system, including:

[0036] An acquisition module, used to acquire the total amount of mirror pairs in the multi-controller storage system;

[0037] An equal distribution module, used for evenly dividing the total cache space in the node into a plurality of partitions according to the total amount of the mirror pairs;

[0038] A monitoring module, used to monitor the space usage information of each partition during the operation of the multi-control storage system;

[0039] The reallocation module is used to reallocate the remaining available space of each partition according to the space usage information of each partition if the cache reallocation condition is met, so that the remaining available space of each partition is relatively balanced.

[0040] Optionally, the monitoring module is specifically used for:

[0041] According to a preset first cycle, the used water level information of each partition is detected as the space usage information.

[0042] Optionally, the reallocation module is specifically used for:

[0043] Determine, in each partition, a first partition whose used water level information is greater than a preset threshold, and a second partition whose used water level information is the smallest;

[0044] Migrate part of the remaining available space in the second partition to the first partition, and keep other partitions unchanged.

[0045] Optionally, the reallocation module is specifically used for:

[0046] Determine the migration ratio;

[0047] Calculating the product of the remaining available space in the second partition and the migration ratio, and using the product as the part of the remaining available space;

[0048] Migrate the portion of remaining available space to the first partition.

[0049] Optionally, the monitoring module is specifically used for:

[0050] According to the preset second cycle, the following steps are performed for each partition:

[0051] Check the used water level information and used cache space of the current partition;

[0052] Calculate the difference between the used water level information and the historical average water level information, and use the quotient of the difference and the historical average water level information as the used space change rate of the current partition;

[0053] Calculate the difference between the used cache space and the historical average used space, and use the quotient of the difference and the historical average used space as the water level change rate of the current partition;

[0054] The used space change rate and load change rate of the current partition are used as the space usage information of the current partition.

[0055] Optionally, the reallocation module is specifically used for:

[0056] For each partition, respectively calculate the product of the load change rate and the preset first coefficient, and use the sum of the product and the used space change rate of the corresponding partition as the adjustment coefficient of the corresponding partition;

[0057] For each partition, respectively calculate the product of the adjustment coefficient and the preset second coefficient, and use the sum of the product and the total current cache space of the corresponding partition as the pre-selected space of the corresponding partition;

[0058] Select the maximum value between the pre-selected space and the current cache space of the same partition as the pre-allocated space of the partition;

[0059] The allocation ratio of each partition is calculated based on the pre-allocated space of each partition;

[0060] Calculate the product of the allocation ratio of each partition and the total cache space in the node respectively to obtain the size of the cache space occupied by each partition;

[0061] The total cache space in the node is reallocated according to the size of the cache space occupied by each partition.

[0062] Optionally, it also includes:

[0063] The adjustment module is used to generate operation adjustment instructions for each partition according to the space usage information of each partition during the operation of the multi-control storage system; and adjust the number of operations to be executed received by each partition according to the operation adjustment instructions.

[0064] In a third aspect, the present application provides an electronic device, including:

[0065] Memory for storing computer programs;

[0066] The processor is used to execute the computer program to implement the cache allocation method disclosed above.

[0067] In a fourth aspect, the present application provides a non-volatile storage medium for storing a computer program, wherein the computer program implements the aforementioned disclosed cache allocation method when executed by a processor.

[0068] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which implements the steps of the aforementioned disclosed cache allocation method when executed by a processor.

[0069] It can be seen from the above scheme that the present application provides a cache allocation method, which is applied to any node in a multi-control storage system, including: obtaining the total amount of mirror pairs in the multi-control storage system; dividing the total cache space in the node into multiple partitions according to the total amount of mirror pairs; during the operation of the multi-control storage system, monitoring the space usage information of each partition respectively; if the cache reallocation conditions are met, reallocating the remaining available space of each partition according to the space usage information of each partition, so that the remaining available space of each partition is relatively balanced.

[0070] It can be seen that after determining the total amount of mirror pairs in the multi-control storage system, the present application evenly divides the total cache space in the node into multiple partitions according to the total amount of mirror pairs; during the operation of the multi-control storage system, the space usage information of each partition is monitored separately; if the cache reallocation conditions are met, the remaining available space of each partition is reallocated according to the space usage information of each partition, so that the remaining available space of each partition is relatively balanced. After the cache space is evenly divided, this solution instantly adjusts the cache space of each partition according to the space usage information of each partition, realizes the cache redistribution and flexible adjustment of the multi-control storage system, can make the remaining available space of each partition relatively balanced, and improves the cache space utilization rate.

[0071] Correspondingly, a cache allocation device, equipment and medium provided by the present application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0073] Figure 1 This is a flow chart of the first cache allocation method disclosed in this application;

[0074] Figure 2 A matching schematic diagram of an IO group and a mirror pair disclosed in this application;

[0075] Figure 3 This is a flow chart of the second cache allocation method disclosed in this application;

[0076] Figure 4 This is a flow chart of the third cache allocation method disclosed in this application;

[0077] Figure 5 A schematic diagram of an electronic device disclosed in this application;

[0078] Figure 6 A server structure diagram provided for this application;

[0079] Figure 7 A terminal structure diagram provided for this application. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other examples obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] At present, the cache space in a multi-control storage system is fixed after being allocated to each mirror pair (domain). After a period of use, the cache space of some mirror pairs of the multi-control storage system is almost full, but the cache space of some mirror pairs is relatively idle and has a low utilization rate. The mirror pairs whose cache space is almost full may cause front-end business interruption due to exhaustion of dirty page resources in the cache space. To this end, the present application provides a cache allocation scheme that can evenly divide the cache space and instantly adjust the cache space of each partition according to the space usage information of each partition, thereby realizing cache redistribution and flexible adjustment of the multi-control storage system, making the remaining available space of each partition relatively balanced, and improving the cache space utilization rate.

[0082] See also Figure 1 As shown, an embodiment of the present application discloses a cache allocation method, which is applied to any node in a multi-controller storage system, including:

[0083] S101: Obtain the total number of mirror pairs in a multi-controller storage system.

[0084] A multi-controller storage system includes multiple controllers, which act as nodes to provide cache space. Taking a four-controller storage system as an example, domain0 corresponds to node0 and node1, domain1 corresponds to node and node2, domain2 corresponds to node2 and node3, and domain3 corresponds to node3 and node0. It can be seen that the total number of mirror pairs in this example is 4, and each can correspond to a different mirror pair at the same time. Then the dirty page resources in the cache space provided by the node will also belong to different mirror pairs.

[0085] S102: Divide the total cache space in the node into multiple partitions evenly according to the total amount of mirror pairs.

[0086] In this embodiment, the total cache space in each node is evenly divided into multiple partitions according to the total amount of mirror pairs. Referring to the above four-controller storage system, for the four nodes, the total cache space provided by each node is evenly divided into four partitions, so that one partition in a single node corresponds to one mirror pair.

[0087] S103: During the operation of the multi-controller storage system, the space usage information of each partition is monitored respectively.

[0088] S104: If the cache reallocation condition is met, the remaining available space of each partition is reallocated according to the space usage information of each partition, so that the remaining available space of each partition is relatively balanced.

[0089] This embodiment provides two reallocation methods. Method 1: In a single node, a portion of the available space in the lowest water level partition is allocated to the highest water level partition in the node. Method 2: Calculate the used space change rate and load change rate of each partition in a single node, and recalculate the size of the space allocated to each partition based on the used space change rate and load change rate. Among them, the water level of a single partition is: the ratio of the number of used dirty pages in the partition to the total number of available dirty page resources, that is, the used water level information.

[0090] In one embodiment, the space usage information of each partition is monitored separately, including: according to a preset first cycle, the used water level information of each partition is detected as the space usage information. Accordingly, the remaining available space of each partition is reallocated according to the space usage information of each partition, including: determining the first partition whose used water level information is greater than a preset threshold (such as a value of 95%) and the second partition whose used water level information is the smallest in each partition; migrating part of the remaining available space in the second partition to the first partition, and keeping other partitions unchanged. Wherein, migrating part of the remaining available space in the second partition to the first partition includes: determining a migration ratio (which can be a preset value, such as a value of 10%); calculating the product of the remaining available space in the second partition and the migration ratio, and using the product as part of the remaining available space; migrating part of the remaining available space to the first partition. For example: if the remaining available space in the second partition is 10,000 pages, then 10,000×10%=1,000 pages are used as part of the remaining available space to be migrated.

[0091] In one embodiment, the space usage information of each partition is monitored separately, including: according to the preset second cycle, the following steps are performed for each partition: detecting the used water level information and used cache space of the current partition; calculating the difference between the used water level information and the historical average water level information, and taking the quotient of the difference and the historical average water level information as the used space change rate of the current partition; calculating the difference between the used cache space (i.e., the number of used dirty pages) and the historical average used space, and taking the quotient of the difference and the historical average used space as the water level change rate of the current partition; taking the used space change rate and load change rate of the current partition as the space usage information of the current partition. The historical average water level information is: the average of several used water level information in the recent period, for example: calculating the average of multiple used water level information measured every 5 seconds within 3 minutes as the historical average water level information, wherein 5 seconds is the second cycle. Accordingly, the historical average used space is: the average of several used cache spaces in a recent period of time, for example: the average of multiple used cache spaces measured every 5 seconds within 3 minutes is calculated as the historical average used space, where 5 seconds is the second period.

[0092] Accordingly, the remaining available space of each partition is reallocated according to the space usage information of each partition, including: for each partition, respectively calculating the product of the load change rate and a preset first coefficient, and taking the sum of the product and the used space change rate of the corresponding partition as the adjustment coefficient of the corresponding partition; for each partition, respectively calculating the product of the adjustment coefficient and a preset second coefficient, and taking the sum of the product and the current total cache space of the corresponding partition as the pre-selected space of the corresponding partition; selecting the maximum value between the pre-selected space and the current total cache space of the same partition as the pre-allocated space of the partition; calculating the allocation ratio of each partition according to the pre-allocated space of each partition; respectively calculating the product of the allocation ratio of each partition and the total cache space in the node to obtain the size of the cache space occupied by each partition; and reallocating the total cache space in the node according to the size of the cache space occupied by each partition.

[0093] In one example, the following parameters are set: CurrentLevel[i] represents the current water level of partition i, where i is the partition ID; HistoricalAverageLevel[i] represents the historical average water level of partition i; CurrentLoad[i] represents the number of dirty pages currently used by partition i; HistoricalLoad[i] represents the historical average number of dirty pages used by partition i; OriginalPages[i] represents the number of dirty pages evenly allocated to partition i; TotalDirtyPages represents the total number of dirty pages available for all partitions, that is, the total cache space provided by a single node. Based on this, the water level change rate of each partition in a single node is first calculated, and the difference between the current water level and the historical average water level of each partition is calculated; the difference is divided by the historical average water level to obtain the water level change rate of each partition. Among them, the water level change rate of partition i is expressed by the formula: LevelChangeRate[i]=(CurrentLevel[i]–HistoricalAverageLevel[i]) / HistoricalAverageLevel[i]. Next, the service load change rate is calculated by calculating the difference between the current service load and the historical average service load, and dividing the difference by the historical average service load to obtain the service load change rate. The service load change rate of partition i is expressed by the formula: LoadChangeRate[i]=(CurrentLoad[i]-HistoricalLoad[i]) / HistoricalLoad[i]. Then, the adjustment coefficient of partition i is calculated as AdjustmentFactor[i]=LevelChangeRate[i]+γ×LoadChangeRate[i], where γ is the preset first coefficient and can be adjusted according to the specific memory specifications. Then, the pre-allocated space of partition i is determined, that is, the maximum value is selected from the pre-selected space OriginalPages[i]+σ×AdjustmentFactor[i] and the current total cache space CurrentLoad[i] of partition i as the pre-allocated space of partition i, which is expressed as follows: the pre-allocated space of partition i NewOutflow[i]=MAX(OriginalPages[i]+σ×AdjustmentFactor[i], CurrentLoad[i]), where σ is the preset second coefficient and σ is a weight coefficient adjusted according to the front-end business characteristics and is used to amplify or reduce the impact of AdjustmentFactor[i].The allocation ratio of each partition is calculated based on the pre-allocated space of each partition. Assuming i=0,1,2,3, NewOutflow[0]=A, NewOutflow[1]=B, NewOutflow[2]=C, NewOutflow[3]=D, then the allocation ratio of partition 0 = A / (A+B+C+D), the allocation ratio of partition 1 = B / (A+B+C+D), the allocation ratio of partition 2 = C / (A+B+C+D), and the allocation ratio of partition 3 = D / (A+B+C+D); accordingly, the cache of partition 0 Space size = TotalDirtyPages×A / (A+B+C+D), cache space size of partition 1 = TotalDirtyPages×B / (A+B+C+D), cache space size of partition 2 = TotalDirtyPages×C / (A+B+C+D), cache space size of partition 3 = TotalDirtyPages×D / (A+B+C+D); the sum of cache space sizes of all partitions is equal to TotalDirtyPages; based on this, the total cache space in the node can be redistributed.

[0094] It should be noted that if both the water level change rate and the business load change rate are very high, then the dirty page allocation may need to be increased. If AdjustmentFactor[i] is positive, the partition dirty page allocation may need to be increased; if it is negative, the allocation may need to be reduced. In the present embodiment, during the operation of the multi-control storage system, operation adjustment instructions for each partition are generated according to the space usage information of each partition; the number of operations to be executed received by each partition is adjusted according to the operation adjustment instructions to adjust the number of read and write operations on each partition.

[0095] It can be seen that after determining the total amount of mirror pairs in the multi-control storage system, this embodiment evenly divides the total cache space in the node into multiple partitions according to the total amount of mirror pairs; during the operation of the multi-control storage system, the space usage information of each partition is monitored separately; if the cache reallocation condition is met, the remaining available space of each partition is reallocated according to the space usage information of each partition, so that the remaining available space of each partition is relatively balanced. After the cache space is evenly divided, this solution instantly adjusts the cache space of each partition according to the space usage information of each partition, realizes the cache redistribution and flexible adjustment of the multi-control storage system, can make the remaining available space of each partition relatively balanced, and improves the cache space utilization rate.

[0096] It should be noted that in order to solve the problem of high write latency caused by insufficient dirty page resources in the partition, the present application can also control the number of IOs sent from the upper layer to the corresponding partition according to the partition water level (the ratio of used pages to total available pages). This is called the water level current limiting technology for the cache partition.

[0097] In a general dual-control storage system, the space of an IO volume belongs to only one node in a mirror pair, and the cache is cached on these two nodes for dual copy redundancy. The available dirty page resources of a cache partition belong entirely to a mirror pair. Based on this, if it is determined that the total water level of the cache partition on a node in the mirror pair exceeds the threshold, the partition is reported to the upper layer, and the upper layer initiates current limiting for the corresponding partition.

[0098] See also Figure 2 , the space of the unowned volume is divided into blocks to all mirror pairs in the IO volume, that is, mirror pairs are formed in pairs in a circular mirroring manner. Take four controllers as an example, domain0 (node0, node1), domain1 (node1, node2), domain2 (node2, node3), domain3 (node3, node0). Each node belongs to a different mirror pair at the same time, and the dirty page resources of the cache partition on the node will be competed for by the write IO belonging to different mirror pairs. In order to avoid deadlock problems caused by resource competition between different domains, the cache space in a node is evenly divided according to the number of domains, which can be used for write IO requests to access different domains.

[0099] See also Figure 3 ,The dynamically adjusted partition methods that support the high-end circular mirror architecture include:

[0100] (1) During the initialization phase, the dirty page resources provided by each node are evenly pre-allocated to the sub-partitions corresponding to the four domains (i.e., the partitions described above).

[0101] (2) Set up periodic checks. Every 5 seconds, the timer will initiate water level calculation and resource reallocation for each sub-partition.

[0102] Specifically, for each node in the system, the water level of each sub-partition is counted. If the water level of each partition is found to exceed the set threshold of 95%, the partition is reported to have exceeded the water level limit, i.e., full, and the upper layer is notified to limit the IO sent to the sub-partition; for example, the IO is converted to write-through; and / or the number of write IOs sent to the partition is reduced.

[0103] (3) After receiving the flow-limiting notification, the upper layer reduces the amount of write IO sent to the corresponding sub-partition according to the flow-limiting requirements to reduce the usage of dirty page resources and alleviate cache pressure.

[0104] (4) The resource reallocation mechanism is triggered by a timer, that is, within a single node, the sub-partition space allocation within the node is adjusted according to the water level of each sub-partition.

[0105] For example, if there is a sub-partition A with a water level higher than 80%, then the sub-partition B with the lowest water level is determined from the same node, and 10% of the number of dirty pages available in sub-partition B is allocated to the sub-partition A with a water level higher than 80% to complete the redistribution of dirty page resources.

[0106] (5) Repeat the above steps, continuously monitor the water level of the sub-partitions in each node, and dynamically adjust the resource allocation ratio according to the situation to ensure the resource utilization and performance stability of the entire system.

[0107] See also Figure 4 ,Another method of dynamically adjusting partitions to support high-end loop mirroring architectures includes:

[0108] (1) During the initialization phase, the dirty page resources provided by each node are evenly pre-allocated to the sub-partitions corresponding to the four domains (i.e., the partitions described above).

[0109] (2) Set up periodic checks. Every 5 seconds, the timer will initiate water level calculation and resource reallocation for each sub-partition.

[0110] Specifically, for each node in the system, the water level of each sub-partition is counted. If the water level of each partition is found to exceed the set threshold of 95%, the partition is reported to have exceeded the water level limit, i.e., full, and the upper layer is notified to limit the IO sent to the sub-partition; for example, the IO is converted to write-through; and / or the number of write IOs sent to the partition is reduced.

[0111] (3) After receiving the flow-limiting notification, the upper layer reduces the amount of write IO sent to the corresponding sub-partition according to the flow-limiting requirements to reduce the usage of dirty page resources and alleviate cache pressure.

[0112] (4) The resource reallocation mechanism is triggered by a timer, that is, within a single node, the sub-partition space allocation within the node is adjusted according to the water level and load conditions of each sub-partition.

[0113] Refer to the following process to implement the adjustment: Set the following parameters: CurrentLevel[i] represents the current water level of partition i, i is the partition ID; HistoricalAverageLevel[i] represents the historical average water level of partition i; CurrentLoad[i] represents the number of dirty pages currently used by partition i; HistoricalLoad[i] represents the historical average number of dirty pages used by partition i; OriginalPages[i] represents the average number of dirty pages allocated to partition i; TotalDirtyPages represents the total number of dirty pages available for all partitions, that is, the total cache space provided by a single node. Based on this, first calculate the water level change rate of each partition in a single node, calculate the difference between the current water level and the historical average water level of each partition; divide the difference by the historical average water level to obtain the water level change rate of each partition. Among them, the water level change rate of partition i is expressed by the formula: LevelChangeRate[i]=(CurrentLevel[i]-HistoricalAverageLevel[i]) / HistoricalAverageLevel[i].

[0114] Next, the service load change rate is calculated by calculating the difference between the current service load and the historical average service load, and dividing the difference by the historical average service load to obtain the service load change rate. The service load change rate of partition i is expressed by the formula: LoadChangeRate[i]=(CurrentLoad[i]-HistoricalLoad[i]) / HistoricalLoad[i]. Then, the adjustment coefficient of partition i is calculated as AdjustmentFactor[i]=LevelChangeRate[i]+γ×LoadChangeRate[i], where γ is the preset first coefficient and can be adjusted according to the specific memory specifications. Then, the pre-allocated space of partition i is determined, that is, the maximum value is selected from the pre-selected space OriginalPages[i]+σ×AdjustmentFactor[i] and the current total cache space CurrentLoad[i] of partition i as the pre-allocated space of partition i, which is expressed as follows: the pre-allocated space of partition i NewOutflow[i]=MAX(OriginalPages[i]+σ×AdjustmentFactor[i], CurrentLoad[i]), where σ is the preset second coefficient and σ is a weight coefficient adjusted according to the front-end business characteristics and is used to amplify or reduce the impact of AdjustmentFactor[i]. The allocation ratio of each partition is calculated based on the pre-allocated space of each partition. Assuming i=0,1,2,3, NewOutflow[0]=A, NewOutflow[1]=B, NewOutflow[2]=C, NewOutflow[3]=D, then the allocation ratio of partition 0 = A / (A+B+C+D), the allocation ratio of partition 1 = B / (A+B+C+D), the allocation ratio of partition 2 = C / (A+B+C+D), and the allocation ratio of partition 3 = D / (A+B+C+D); accordingly, the cache of partition 0 Space size = TotalDirtyPages×A / (A+B+C+D), cache space size of partition 1 = TotalDirtyPages×B / (A+B+C+D), cache space size of partition 2 = TotalDirtyPages×C / (A+B+C+D), cache space size of partition 3 = TotalDirtyPages×D / (A+B+C+D); the sum of cache space sizes of all partitions is equal to TotalDirtyPages; based on this, the total cache space in the node can be redistributed.

[0115] It can be seen that this embodiment can effectively solve the problem of partition resource adjustment under the high-end circular mirror architecture, avoid the occurrence of resource deadlock to a certain extent, and flexibly allocate resources according to actual conditions, so that the performance of the entire system is optimized, reducing the write latency of the cache partition and the problem of excessive IO issuance, ensuring the stable operation of the host-side business. At the same time, through the current limiting notification and resource reallocation mechanism, it can prevent the cache partition from being full and IO interruption, thereby improving the reliability and stability of the system.

[0116] A cache allocation device provided in an embodiment of the present application is introduced below. The cache allocation device described below can be referenced to other embodiments described in this document.

[0117] The embodiment of the present application discloses a cache allocation device, which is applied to any node in a multi-controller storage system, including:

[0118] An acquisition module is used to acquire the total amount of mirror pairs in the multi-controller storage system;

[0119] An equal distribution module is used to evenly divide the total cache space in the node into multiple partitions according to the total amount of mirror pairs;

[0120] The monitoring module is used to monitor the space usage information of each partition during the operation of the multi-control storage system;

[0121] The reallocation module is used to reallocate the remaining available space of each partition according to the space usage information of each partition if the cache reallocation condition is met, so that the remaining available space of each partition is relatively balanced.

[0122] In one embodiment, the monitoring module is specifically used to:

[0123] According to the preset first cycle, the used water level information of each partition is detected as the space usage information.

[0124] In one embodiment, the reallocation module is specifically used to:

[0125] Determine, in each partition, a first partition whose used water level information is greater than a preset threshold, and a second partition whose used water level information is the smallest;

[0126] Migrate some of the remaining free space in the second partition to the first partition and keep the other partitions unchanged.

[0127] In one embodiment, the reallocation module is specifically used to:

[0128] Determine the migration ratio;

[0129] Calculate the product of the remaining available space in the second partition and the migration ratio, and use the product as part of the remaining available space;

[0130] Migrate some of the remaining free space to the first partition.

[0131] In one embodiment, the monitoring module is specifically used to:

[0132] According to the preset second cycle, the following steps are performed for each partition:

[0133] Check the used water level information and used cache space of the current partition;

[0134] Calculate the difference between the used water level information and the historical average water level information, and use the quotient of the difference and the historical average water level information as the used space change rate of the current partition;

[0135] Calculate the difference between the used cache space and the historical average used space, and use the quotient of the difference and the historical average used space as the water level change rate of the current partition;

[0136] The used space change rate and load change rate of the current partition are used as the space usage information of the current partition.

[0137] In one embodiment, the reallocation module is specifically used to:

[0138] For each partition, respectively calculate the product of the load change rate and the preset first coefficient, and use the sum of the product and the used space change rate of the corresponding partition as the adjustment coefficient of the corresponding partition;

[0139] For each partition, respectively calculate the product of the adjustment coefficient and the preset second coefficient, and use the sum of the product and the total current cache space of the corresponding partition as the pre-selected space of the corresponding partition;

[0140] Select the maximum value between the pre-selected space and the current cache space of the same partition as the pre-allocated space of the partition;

[0141] The allocation ratio of each partition is calculated based on the pre-allocated space of each partition;

[0142] Calculate the product of the allocation ratio of each partition and the total cache space in the node to obtain the cache space occupied by each partition;

[0143] The total cache space in the node is reallocated according to the size of the cache space occupied by each partition.

[0144] In one embodiment, it further includes:

[0145] The adjustment module is used to generate operation adjustment instructions for each partition according to the space usage information of each partition during the operation of the multi-control storage system; and adjust the number of operations to be executed received by each partition according to the operation adjustment instructions.

[0146] Among them, for more specific working processes of each module and unit in this embodiment, reference can be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0147] It can be seen that this embodiment provides a cache allocation device, which can evenly divide the cache space and then instantly adjust the cache space of each partition according to the space usage information of each partition, thereby realizing cache redistribution and flexible adjustment of the multi-control storage system, making the remaining available space of each partition relatively balanced, and improving the cache space utilization.

[0148] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below can be referenced to other embodiments described in this document.

[0149] See also Figure 5 As shown, the embodiment of the present application discloses an electronic device, including:

[0150] Memory 501, used for storing computer programs;

[0151] The processor 502 is used to execute the computer program to implement the method disclosed in any of the above embodiments.

[0152] Furthermore, the present application also provides an electronic device. The electronic device can be Figure 6 The server shown can also be Figure 7 Terminal shown. Figure 6 and Figure 7 All of them are structural diagrams of electronic devices according to an exemplary embodiment, and the contents in the diagrams cannot be regarded as any limitation on the scope of use of the present application.

[0153] Figure 6 A schematic diagram of the structure of a server provided in an embodiment of the present application. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the relevant steps in the cache allocation disclosed in any of the aforementioned embodiments.

[0154] In this embodiment, the power supply is used to provide working voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0155] In addition, the memory as a carrier for resource storage can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include operating system, computer programs and data, etc. The storage method can be temporary storage or permanent storage.

[0156] The operating system is used to manage and control the hardware devices and computer programs on the server to realize the operation and processing of the data in the memory by the processor, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the cache allocation method disclosed in any of the above embodiments, the computer program can also further include computer programs that can be used to complete other specific tasks. In addition to data such as application update information, the data can also include data such as application developer information.

[0157] Figure 7 A schematic diagram of the structure of a terminal provided in an embodiment of the present application, the terminal may specifically include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer, etc.

[0158] Generally, the terminal in this embodiment includes: a processor and a memory.

[0159] Among them, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0160] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory is at least used to store the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps in the cache allocation method performed by the terminal side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, update information of the application.

[0161] In some embodiments, the terminal may also include a display screen, an input and output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0162] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or fewer components than those shown in the figure.

[0163] A non-volatile storage medium provided in an embodiment of the present application is introduced below. The non-volatile storage medium described below can be cross-referenced with other embodiments described herein.

[0164] A non-volatile storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the cache allocation method disclosed in the above embodiment. The non-volatile storage medium is a computer-readable non-volatile storage medium, which, as a carrier for storing resources, may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system, a computer program and data, etc., and the storage method may be temporary storage or permanent storage.

[0165] A computer program product provided in an embodiment of the present application is introduced below. The computer program product described below can be cross-referenced with other embodiments described in this document.

[0166] A computer program product includes a computer program / instruction, which implements the steps of the cache allocation method disclosed above when executed by a processor.

[0167] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0168] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile storage medium known in the art.

[0169] This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope.

Claims

1. A cache allocation method, characterized in that: Applicable to any node in a multi-controller storage system, including: Obtaining the total amount of mirror pairs in the multi-controller storage system; Dividing the total cache space in the node into a plurality of partitions evenly according to the total amount of the mirror pairs; During the operation of the multi-control storage system, space usage information of each partition is monitored respectively; If the cache reallocation condition is met, the remaining available space of each partition is reallocated according to the space usage information of each partition, so that the remaining available space of each partition is relatively balanced.

2. The method according to claim 1, characterized in that Monitor the space usage information of each partition separately, including: According to a preset first cycle, the used water level information of each partition is detected as the space usage information.

3. The method according to claim 2, characterized in that Redistribute the remaining available space of each partition based on the space usage information of each partition, including: Determine, in each partition, a first partition whose used water level information is greater than a preset threshold, and a second partition whose used water level information is the smallest; Migrate part of the remaining available space in the second partition to the first partition, and keep other partitions unchanged.

4. The method according to claim 3, characterized in that Migrating part of the remaining available space in the second partition to the first partition includes: Determine the migration ratio; Calculating the product of the remaining available space in the second partition and the migration ratio, and using the product as the part of the remaining available space; Migrate the portion of remaining available space to the first partition.

5. The method according to claim 1, characterized in that Monitor the space usage information of each partition separately, including: According to the preset second cycle, the following steps are performed for each partition: Check the used water level information and used cache space of the current partition; Calculate the difference between the used water level information and the historical average water level information, and use the quotient of the difference and the historical average water level information as the used space change rate of the current partition; Calculate the difference between the used cache space and the historical average used space, and use the quotient of the difference and the historical average used space as the water level change rate of the current partition; The used space change rate and load change rate of the current partition are used as the space usage information of the current partition.

6. The method according to claim 5, characterized in that Redistribute the remaining available space of each partition based on the space usage information of each partition, including: For each partition, respectively calculate the product of the load change rate and the preset first coefficient, and use the sum of the product and the used space change rate of the corresponding partition as the adjustment coefficient of the corresponding partition; For each partition, respectively calculate the product of the adjustment coefficient and the preset second coefficient, and use the sum of the product and the total current cache space of the corresponding partition as the pre-selected space of the corresponding partition; Select the maximum value between the pre-selected space and the current cache space of the same partition as the pre-allocated space of the partition; The allocation ratio of each partition is calculated based on the pre-allocated space of each partition; Calculate the product of the allocation ratio of each partition and the total cache space in the node respectively to obtain the size of the cache space occupied by each partition; The total cache space in the node is reallocated according to the size of the cache space occupied by each partition.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: During the operation of the multi-control storage system, generating operation adjustment instructions for each partition according to the space usage information of each partition; The number of operations to be executed received by each partition is adjusted according to the operation adjustment instruction.

8. A cache allocation device, characterized in that: Applicable to any node in a multi-controller storage system, including: An acquisition module, used to acquire the total amount of mirror pairs in the multi-controller storage system; An equal distribution module, used for evenly dividing the total cache space in the node into a plurality of partitions according to the total amount of the mirror pairs; A monitoring module, used to monitor the space usage information of each partition during the operation of the multi-control storage system; The reallocation module is used to reallocate the remaining available space of each partition according to the space usage information of each partition if the cache reallocation condition is met, so that the remaining available space of each partition is relatively balanced.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images