A hard disk pool selection method, device, equipment, system and readable storage medium

Abstract

The present invention discloses a hard disk pool selection method, device, equipment, system and readable storage medium, which are applied to the field of computer technology. In order to solve the problem that the existing hard disk pool selection method affects the cluster performance and user business performance, the method comprises: determining the weight of each hard disk pool based on the storage status of all hard disk pools; sending the weight of each hard disk pool to a client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein: the storage status of the hard disk pool at least includes the storage type of the hard disk pool and the capacity information of the hard disk pool, and the storage type of the hard disk pool includes a performance balancing group and a capacity balancing group; the client is used to write data through the target hard disk pool; in the embodiment of the present invention, the respective weights of all hard disk pools are dynamically adjusted according to the storage status, which is conducive to improving the cluster performance and user business performance.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a hard disk pool selection method, device, electronic device, distributed storage system and computer-readable storage medium. Background Art

[0002] When there are many cluster nodes in distributed storage, in order to improve cluster reliability and reduce the risk of data loss, cluster nodes are usually divided into multiple physically isolated hard disk pools, and the multi-hard disk pool mechanism is used to be compatible with heterogeneous hard disk pools, such as Figure 1 shown.

[0003] When writing data to multiple hard disk pools, the relevant data writing algorithm usually adopts a fixed weight polling algorithm. When users deploy hard disk pools, the fixed weight algorithm sets a fixed weight value according to the number of disks in the hard disk pool. The client selects the hard disk pool for data writing operations according to the hard disk pool weight value. However, when a hard disk pool fails, the hard disk pool selection is still carried out according to the fixed weight value, which will cause some hard disk pools to have instantaneous high loads, seriously affecting the availability of user services. In a heterogeneous cluster, because the hard disk pool with a higher weight will continuously write data for a period of time, the hard disk pool with a lower weight is in a low-load state, affecting the cluster performance and user service performance.

[0004] In view of this, how to improve cluster performance and user service performance becomes a problem that needs to be solved by those skilled in the art. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a hard disk pool selection method, device, electronic device, distributed storage system and computer-readable storage medium, which are conducive to improving cluster performance and user service performance.

[0006] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0007] In one aspect, the present invention provides a method for selecting a hard disk pool, comprising:

[0008] Based on the storage status of all hard disk pools, determine the weight of each hard disk pool;

[0009] The weight of each hard disk pool is sent to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein:

[0010] The storage status of the hard disk pool includes at least the storage type of the hard disk pool and the capacity information of the hard disk pool. The storage type of the hard disk pool includes a performance balancing group and a capacity balancing group. The client is used to write data through the target hard disk pool.

[0011] In one implementation, the remaining capacity ratio of the hard disk pool in the capacity balancing group is greater than or equal to a preset ratio threshold;

[0012] The remaining capacity ratio of the hard disk pool in the performance balancing group is less than the preset ratio threshold; wherein:

[0013] The remaining capacity ratio of the hard disk pool is obtained based on the total capacity and the remaining capacity of the hard disk pool.

[0014] In one implementation, sending the weight of each hard disk pool to the client so that the client determines the target hard disk pool according to the weight of each hard disk pool includes:

[0015] The weight of each hard disk pool is sent to the client to update the weight of each hard disk pool in the smooth polling weight list of the client, so that the client determines the target hard disk pool according to the updated smooth polling weight list.

[0016] In one embodiment, it further includes:

[0017] Monitor the status of each hard disk pool to obtain status information of each hard disk pool;

[0018] For each hard disk pool, when it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed, the weights of the hard disk pools are recalculated.

[0019] In one implementation, when it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed, recalculating the weights of the hard disk pools includes:

[0020] In the case where it is determined according to the status information of the hard disk pool that there is a faulty hard disk or a newly added hard disk in the hard disk pool, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is returned to be executed;

[0021] When it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool is stuck or slow, the weight of the hard disk pool is reduced by a preset value based on the current weight to obtain a new weight;

[0022] When it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has been replaced, the weight of the hard disk pool is kept unchanged.

[0023] In one embodiment, when it is determined according to the status information of the hard disk pool that a faulty hard disk or a newly added hard disk exists in the hard disk pool, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is returned to be executed, including:

[0024] In the case where it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has a faulty hard disk, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is directly returned to be executed;

[0025] When it is determined that there is a new hard disk in the hard disk pool based on the status information of the hard disk pool, the storage status of the hard disk pool is re-determined, and when the remaining capacity ratio of the new hard disk is consistent with the remaining capacity ratio of each original hard disk, return to execute the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools.

[0026] In one implementation, the capacity balancing group includes at least one first hard disk pool, the performance balancing group includes at least one second hard disk pool, and the method further includes:

[0027] Determine the total number of available hard disks in the performance balancing group based on the number of available hard disks in each of the second hard disk pools in the performance balancing group;

[0028] Then, determining the weight of each hard disk pool based on the storage status of all hard disk pools includes:

[0029] Determine, according to the capacity information of all the first hard disk pools and all the second hard disk pools, an initial weight of each of the first hard disk pools and an initial weight of each of the second hard disk pools;

[0030] Using the initial weight as the hard disk pool weight of the capacity balancing group of the first hard disk pool;

[0031] For each of the second hard disk pools, the performance balancing group hard disk pool weight of the second hard disk pool is determined according to the initial weights of all the second hard disk pools, the number of available hard disks in the second hard disk pool, and the total number of available hard disks in the performance balancing group.

[0032] In one implementation, determining the initial weight of each of the first hard disk pools and the initial weight of each of the second hard disk pools according to the capacity information of all the first hard disk pools and all the second hard disk pools includes:

[0033] Calculating a sum of current remaining capacities according to the current remaining capacities of each of the first hard disk pools and the current remaining capacities of each of the second hard disk pools;

[0034] For each of the first hard disk pools or each of the second hard disk pools, a ratio of the current remaining capacity of the first hard disk pool or the second hard disk pool to the sum of the current remaining capacities is calculated to obtain an initial weight of the first hard disk pool or the second hard disk pool.

[0035] In one implementation, for each of the second hard disk pools, determining the hard disk pool weight of the performance balancing group of the second hard disk pool according to the initial weights of all the second hard disk pools, the number of available hard disks in the second hard disk pools, and the total number of available hard disks in the performance balancing group includes:

[0036] Sum the initial weights corresponding to each of the second hard disk pools to obtain the sum of the initial weights;

[0037] Sum the number of available hard disks corresponding to each of the second hard disk pools to obtain the total number of available hard disks in the performance balancing group;

[0038] For each of the second hard disk pools, the performance balancing group hard disk pool weight of the second hard disk pool is determined according to the number of available hard disks in the second hard disk pool, the sum of the initial weights, and the total number of available hard disks in the performance balancing group.

[0039] In one aspect, the present invention provides a hard disk pool selection device, comprising:

[0040] A determination module, used for determining the weight of each hard disk pool based on the storage status of all hard disk pools;

[0041] A sending module is used to send the weight of each hard disk pool to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein:

[0042] The storage status of the hard disk pool includes at least the storage type of the hard disk pool and the capacity information of the hard disk pool. The storage type of the hard disk pool includes a performance balancing group and a capacity balancing group. The client is used to write data through the target hard disk pool.

[0043] In one aspect, the present invention provides an electronic device, comprising:

[0044] Memory for storing computer programs;

[0045] A processor is used to implement the steps of the hard disk pool selection method as described above when executing the computer program.

[0046] In one aspect, the present invention provides a distributed storage system, comprising:

[0047] A communication interface, a control module, a management module and a plurality of hard disk pools, wherein the control module is connected to the communication interface and the management module respectively, the management module is connected to the plurality of hard disk pools, and the hard disk pool is a hard disk management unit and is mapped with a plurality of corresponding hard disks;

[0048] The communication interface is used to connect to multiple clients and receive data written by the clients into the target hard disk pool;

[0049] The control module is used to write the data into the target hard disk pool;

[0050] The management module is used to implement the steps of the hard disk pool selection method as described above.

[0051] On one hand, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the hard disk pool selection method as described above are implemented.

[0052] It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

[0053] A hard disk pool selection method is provided in an embodiment of the present invention, including: determining the weight of each hard disk pool based on the storage status of all hard disk pools; sending the weight of each hard disk pool to a client, so that the client determines a target hard disk pool according to the weight of each hard disk pool; wherein: the storage status of the hard disk pool includes at least the storage type of the hard disk pool and the capacity information of the hard disk pool, and the storage type of the hard disk pool includes a performance balancing group and a capacity balancing group; the client is used to write data through the target hard disk pool.

[0054] It can be seen that in the embodiment of the present invention, the storage status of all hard disk pools is obtained to determine the weight of each hard disk pool, and the weight of each hard disk pool is sent to the client, so that the client can select a hard disk pool for writing data according to the weight of each hard disk pool. Since each hard disk pool in the embodiment of the present invention corresponds to a corresponding storage status, and the storage status can indicate the storage type and capacity information of the hard disk pool, where the storage type can indicate that the hard disk pool is a performance balancing group or a capacity balancing group, the embodiment of the present invention dynamically adjusts the respective weights according to the storage status of all hard disk pools, which is beneficial to improving cluster performance and user business performance.

[0055] In addition, the present invention also provides corresponding implementation devices, electronic devices, distributed storage systems and computer-readable storage media for the hard disk pool selection method, further making the method more practical, and the devices, electronic devices and computer-readable storage media have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the prior art and the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 This is a schematic diagram of a traditional multi-disk pool architecture;

[0058] Figure 2 A schematic diagram of a hard disk pool selection architecture provided by an embodiment of the present invention;

[0059] Figure 3 A flowchart of a hard disk pool selection method provided by an embodiment of the present invention;

[0060] Figure 4 A schematic diagram of another architecture for selecting a hard disk pool provided by an embodiment of the present invention;

[0061] Figure 5 A schematic diagram of a hard disk pool grouping provided by an embodiment of the present invention;

[0062] Figure 6 A schematic diagram of a hard disk pool with a faulty hard disk provided by an embodiment of the present invention;

[0063] Figure 7 A schematic diagram of a hard disk pool after expansion provided by an embodiment of the present invention;

[0064] Figure 8 A schematic diagram of a hard disk pool after a hard disk is replaced provided by an embodiment of the present invention;

[0065] Fig. 9 A schematic diagram of the structure of a hard disk pool selection device provided by an embodiment of the present invention;

[0066] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0067] Fig.11 A schematic diagram of the structure of a distributed storage system provided by an embodiment of the present invention;

[0068] Fig.12 A schematic diagram of another distributed storage system provided by an embodiment of the present invention

[0069] Fig.13 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The embodiments of the present invention provide a hard disk pool selection method, device, electronic device, distributed storage system and computer-readable storage medium, which are beneficial to improving cluster performance and user service performance.

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] It should be noted that storage pool (SP) is the primary concept of data storage in a distributed storage cluster, and disk pool (DP) is the secondary concept of data storage in a distributed storage cluster. A disk pool is an independent data storage unit composed of multiple hard disks that meets the requirements of redundant copies. Different disk pools may be heterogeneous, and the number of disks and disk capacity may be different; a storage pool can contain multiple disk pools.

[0073] When writing data to multiple hard disk pools, the traditional data writing algorithm uses a fixed weight polling algorithm. The weight of the hard disk pool is a fixed value. The client selects a hard disk pool for data writing operations based on the weight value of the hard disk pool. However, when a hard disk pool fails, this selection method will affect cluster performance and user service performance. In view of this, an embodiment of the present invention provides a hard disk pool selection method, such as Figure 2 As shown, the method can be applied to a management server, and the storage pool includes multiple hard disk pools, namely DP1, DP2, DP3, DP4, and DP5. The management server can obtain the current remaining capacity of each hard disk pool, dynamically calculate the weight of each hard disk pool, and update the hard disk pool weight list (that is, the DP weight list). Client A, client B or client C can request the management server to obtain the DP weight list as needed. The management server sends the DP weight list to the corresponding client, so that the client can select the hard disk pool according to the dynamically adjusted hard disk pool weight, which is beneficial to improving cluster performance and user business performance.

[0074] In the following embodiments of the present invention, a storage pool in a cluster is used as an example to illustrate the technical solution. Figure 3 , Figure 3 A flowchart of a method for selecting a hard disk pool is provided in an embodiment of the present invention.

[0075] S110: Determine a weight of each hard disk pool based on the storage status of all hard disk pools;

[0076] It should be noted that in the embodiment of the present invention, the storage status of all hard disk pools in the storage pool can be obtained, wherein the storage status can at least include the storage type of the hard disk pool and the capacity information of the hard disk pool, and the storage type can include a performance group and a capacity balancing group, that is, the storage type of the hard disk pool can determine whether the hard disk pool belongs to a performance balancing group or a capacity balancing group; the capacity information of the hard disk pool can at least include the number of hard disks in the hard disk pool and the remaining capacity. Therefore, in the embodiment of the present invention, the weight of each hard disk pool can be determined by obtaining the storage status of all hard disk pools. In other words, the weight of the hard disk pool in the embodiment of the present invention can be dynamically adjusted according to the actual situation of the storage status of the hard disk pool, and is not fixed.

[0077] In actual applications, the storage status of each hard disk pool can be obtained in real time or periodically, or, for example Figure 4 As shown, the hard disk pool nodes corresponding to each hard disk pool can report the capacity information of the hard disk pool to the management server in real time or periodically, so that the management server can determine the storage type of each hard disk pool according to the capacity information of each hard disk pool and calculate the weight of each hard disk pool.

[0078] S120: Send the weight of each hard disk pool to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein:

[0079] The storage status of the hard disk pool includes at least the storage type and capacity information of the hard disk pool. The storage type of the hard disk pool includes a performance balancing group and a capacity balancing group. The client is used to write data through the target hard disk pool.

[0080] It is understandable that in the embodiment of the present invention, after obtaining the weight of each hard disk pool, the weight of each hard disk pool can be sent to the client, and the client can determine the target hard disk pool according to the weight of each hard disk pool, and the client writes data through the determined hard disk pool. Since the weight of each hard disk pool can be dynamically adjusted according to the storage status of all hard disk pools in the embodiment of the present invention, the client can also determine the target hard disk pool for writing data according to the dynamically adjusted weight, which is conducive to improving cluster performance and user service performance.

[0081] In one implementation, the remaining capacity ratio of the hard disk pool in the capacity balancing group is greater than or equal to a preset ratio threshold;

[0082] The remaining capacity of the disk pool in the performance balancing group is less than the preset capacity threshold; where:

[0083] The remaining capacity ratio of the disk pool is based on the total capacity and remaining capacity of the disk pool.

[0084] It should be noted that, in the embodiment of the present invention, each hard disk pool in the storage pool can be divided into two groups, one is a capacity balancing group, and the other is a performance balancing group. The remaining capacity and the number of hard disks of the hard disk pool can be determined based on the capacity information of each hard disk pool, and the total capacity of the hard disk pool can be determined based on the number of hard disks. Further, the remaining capacity ratio of the hard disk pool can be determined based on the remaining capacity and the total capacity of the hard disk pool. Among them, the remaining capacity ratio of the hard disk pool is the ratio of the remaining capacity to the total capacity.

[0085] In actual applications, since the remaining capacity of each hard disk pool will not change suddenly, the capacity information of each hard disk pool can be obtained periodically (for example, at intervals of T hours) to periodically determine the remaining capacity ratio of each hard disk pool.

[0086] It is understandable that in the embodiment of the present invention, after obtaining the remaining capacity ratio of each hard disk pool, the storage type of the hard disk pool can be determined according to the remaining capacity ratio of the hard disk pool. For example, each remaining capacity ratio can be compared with a preset ratio threshold (for example, 80%). If the remaining capacity ratio of the hard disk pool reaches the preset ratio threshold (that is, greater than or equal to the preset ratio threshold), the storage type of the hard disk pool is determined to be a capacity balancing group. Figure 5 As shown, for example, the preset percentage threshold is 80%, the hard disk pool water level of DP1 (that is, the remaining capacity percentage of the hard disk pool) is 90% (greater than 80%), and the hard disk pool water level of DP2 (that is, the remaining capacity percentage of the hard disk pool) is 80% (equal to 80%), then the storage type of DP1 and DP2 is determined to be a capacity balancing group.

[0087] It is understandable that if the remaining capacity of the hard disk pool is less than the preset capacity threshold, the storage type of the hard disk pool is determined to be a capacity balancing group. Figure 5 As shown, for example, the preset percentage threshold is 80%, the hard disk pool water level of DP3 (that is, the remaining capacity percentage of the hard disk pool) is 60% (less than 80%), the hard disk pool water level of DP4 is 40% (less than 80%), and the hard disk pool water level of DP5 is 0% (less than 80%), then the DP3, DP4 and DP5 storage types are determined to be a performance balanced group.

[0088] In actual applications, the hard disk pool nodes corresponding to each hard disk pool can report the remaining capacity of the hard disk pool to the management server in real time or periodically, and can also report the total capacity of each hard disk pool. Of course, the hard disk pool nodes can also report the hard disk status information of the hard disk pool. The management server can calculate the total capacity of the hard disk pool based on the hard disk status information of the hard disk pool and determine the specific implementation method of the total capacity of the hard disk pool. The embodiments of the present invention do not make special limitations on this.

[0089] While each hard disk pool node reports the current remaining capacity of the corresponding hard disk pool, each hard disk pool node may also report the total capacity of the hard disk pool.

[0090] In one implementation, the process of sending the weight of each hard disk pool to the client in S120 so that the client determines the target hard disk pool according to the weight of each hard disk pool may include:

[0091] The weight of each hard disk pool is sent to the client to update the weight of each hard disk pool in the smooth polling weight list of the client, so that the client determines the target hard disk pool according to the updated smooth polling weight list.

[0092] It should be noted that in the embodiment of the present invention, the smooth polling weight list of the client stores the weight corresponding to each hard disk pool. Therefore, the determined weight of each hard disk pool can be sent to the client to update the weight of each hard disk pool in the smooth polling list of the client. The client can determine the target hard disk pool for data writing based on the updated smooth polling weight list.

[0093] In actual applications, the management server can actively send the weight of each hard disk pool to the client to update the weight of each hard disk pool in the client's smooth polling list, or the client can send a hard disk pool weight request to the management server when data needs to be written (such as Figure 4 As shown), the management server can return the weight of each hard disk pool to the corresponding client according to the hard disk pool weight acquisition request, so as to update the weight of each hard disk pool in the client's smooth polling weight list, so that the client can determine the target hard disk pool according to the updated smooth polling weight list. The hard disk pool weight acquisition request may include information such as the client IP address.

[0094] The management server can manage the weight of each hard disk pool through the hard disk pool weight list. For example, after determining the weight of each hard disk pool, the weight of each hard disk pool in the hard disk pool weight list can be updated, and the latest hard disk pool weight list can be actively sent to the client, or upon receiving a hard disk pool weight acquisition request sent by the client, the latest hard disk pool weight list can be sent to the client, so that the client's smooth polling weight list can be updated according to the hard disk pool weight list, so that the client can reasonably select the target hard disk pool according to the smooth polling weight list when writing data. In this process, the client can dynamically adapt to changes in the hard disk pool weights in the cluster storage pool, thereby achieving load balancing and optimal data writing performance.

[0095] It should be noted that the smooth polling weight list is to smooth the hard disk pool selection, so that within a hard disk pool weight update cycle, the order of hard disk pools obtained by the client is cross-proportional, rather than always selecting hard disk pools with high weights, nor polling to select hard disk pools. Cross-proportional acquisition of hard disk pools avoids hot spots in a short period of time. In the long run, allocating hard disk pools according to the weight ratio of each hard disk pool is more conducive to load balancing and ensuring data writing performance.

[0096] In one embodiment, it further includes:

[0097] Monitor the status of each hard disk pool and obtain the status information of each hard disk pool;

[0098] For each hard disk pool, when it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed, the weights of the hard disk pools are recalculated.

[0099] It should be noted that when the hard disks in one or more hard disk pools in each hard disk pool change, it means that the storage status of the corresponding hard disk pool has changed. For example, the total capacity or remaining capacity of the hard disk pool will also change accordingly. At this time, the weight of each hard disk pool can be recalculated. In actual applications, the status of the hard disk pool nodes of each hard disk pool can be monitored to obtain the status information of the corresponding hard disk pool nodes. According to the status information of the hard disk pool nodes, it can be further determined whether the hard disks in the hard disk pool have changed, and specifically whether the number of hard disks in the hard disk pool or the status of the hard disks has changed.

[0100] That is, on the one hand, the weight of each hard disk pool can be calculated periodically and the hard disk pool weight list can be updated. On the other hand, when it is determined that the hard disks in the hard disk pool have changed according to the status information of the hard disk pool, the weight of each hard disk pool in the storage pool can be recalculated, so that the load and performance capabilities of each hard disk pool can be timely reflected through the weight, so that the target hard disk pool selected by the client based on the latest weight of each hard disk pool can better meet the cluster performance requirements and user business performance requirements. In addition, by monitoring the status of the hard disk pool to determine whether the weight of the hard disk pool needs to be recalculated, it is beneficial to better respond to cluster operation and maintenance and failure scenarios, improve the availability of user services, and also reduce the high load of certain hard disk pools.

[0101] In one implementation, when it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed, recalculating the weights of the hard disk pools includes:

[0102] When it is determined according to the status information of the hard disk pool that there is a faulty hard disk or a newly added hard disk in the hard disk pool, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is returned;

[0103] When it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool is stuck and slow, the weight of the hard disk pool is reduced by a preset value based on the current weight to obtain a new weight;

[0104] When it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has been replaced, the weight of the hard disk pool is kept unchanged.

[0105] It should be noted that when it is determined based on the status information of the hard disk pool that there is a faulty hard disk or a new hard disk in the hard disk pool, it means that the storage status of the hard disk pool may have changed. At this time, it is necessary to redetermine the storage status of the hard disk pool. For example, one or more of the total capacity, remaining capacity and storage type of the hard disk pool may have changed. At this time, the storage status of the hard disk pool can be redetermined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools can be returned.

[0106] In actual applications, when it is determined that a hard disk in the hard disk pool has a faulty hard disk according to the hard disk pool status information, the storage status of the hard disk pool can be re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools can be directly returned to execute;

[0107] When it is determined based on the status information of the hard disk pool that a new hard disk exists in the hard disk pool, the storage status of the hard disk pool is redetermined, and when the remaining capacity ratio of the new hard disk is consistent with the remaining capacity ratio of each original hard disk, return to execute the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools.

[0108] It should be noted that, when it is determined based on the status information of the hard disk pool (for example, the status information of the hard disk pool node) that there is a faulty hard disk in the hard disk pool (for example, Figure 6 In the case of a change in the storage status of the hard disk pool (such as the total capacity and the remaining capacity), the storage status of the hard disk pool (such as the total capacity and the remaining capacity) may be changed. In this case, the storage status of the hard disk pool may be obtained again, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools may be returned. That is, steps S110 to S120 may be re-executed, wherein the storage type of the hard disk pool may also be re-determined based on the total capacity of the hard disk pool and the proportion of the remaining capacity, so as to re-group the hard disk pool. In order to improve the calculation efficiency, the number of failed disks in the hard disk pool may be obtained, and the total capacity of the hard disk pool may be obtained by subtracting the capacity of each failed hard disk from the total capacity calculated last time to obtain the current total capacity of the hard disk pool.

[0109] After determining the existence of a new hard disk in the hard disk pool (such as the status information of the hard disk pool node) based on the status information of the hard disk pool (such as the status information of the hard disk pool node), Figure 7 As shown in the figure, it indicates that the hard disk pool is in the state of expansion. At this time, the total capacity of the hard disks will also change. The number of expanded hard disks and the capacity of each expanded hard disk can be obtained. Due to the existence of newly added hard disks, there is no need to recalculate the weights of each hard disk pool in the initial stage of expansion. That is, the weights of each hard disk pool are kept unchanged in the initial stage of expansion. When the remaining capacity ratio of the old hard disks in the hard disk pool (calculated based on the total capacity of the old hard disks and the current remaining capacity) is consistent with the remaining capacity ratio of the new hard disks (calculated based on the total capacity of the new hard disks and the current remaining capacity), that is, when the remaining capacity ratio of the hard disk pool (which can be calculated based on the current remaining capacity and the current total capacity of the hard disk pool) reaches the preset threshold, the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is performed to redetermine the weight of each hard disk pool.

[0110] In addition, in an embodiment of the present invention, when it is determined based on the status information of the hard disk pool that there is a slow hard disk in the hard disk pool, the weight of the hard disk pool corresponding to the slow hard disk can be directly reduced by a preset value based on the current weight to obtain a new weight corresponding to the hard disk, so as to better ensure data writing performance.

[0111] Of course, if the hard disk in the hard disk pool is determined to have been replaced based on the status information of the hard disk pool (such as Figure 8 In the case of the disk pool being unavailable, the disk pool weight can be kept unchanged and there is no need to recalculate the disk pool weight.

[0112] In one implementation, the capacity balancing group includes at least one first hard disk pool, the performance balancing group includes at least one second hard disk pool, and the method further includes:

[0113] Determine the total number of available hard disks in the performance balancing group based on the number of available hard disks in each second hard disk pool in the performance balancing group;

[0114] Then, based on the storage status of all hard disk pools, determine the weight of each hard disk pool, including:

[0115] Determine, according to the capacity information of all the first hard disk pools and all the second hard disk pools, an initial weight of each first hard disk pool and an initial weight of each second hard disk pool;

[0116] The initial weight is used as the hard disk pool weight of the capacity balancing group of the first hard disk pool;

[0117] For each second hard disk pool, the hard disk pool weight of the performance balancing group of the second hard disk pool is determined according to the initial weights of all the second hard disk pools, the number of available hard disks in the second hard disk pool, and the total number of available hard disks in the performance balancing group.

[0118] It should be noted that in the embodiment of the present invention, the hard disk pool whose storage type is a capacity balancing group is determined as the first hard disk pool, and the hard disk pool whose storage type is a performance balancing group is determined as the second hard disk pool. The capacity balancing group includes at least one first hard disk pool, and the performance balancing group includes at least one second hard disk pool.

[0119] It is understandable that since the remaining capacity of each second hard disk pool in the performance balancing group is relatively small and the used capacity is relatively large, in order to better ensure the performance of each second hard disk pool in the performance balancing group, the number of available hard disks of each second hard disk pool in the performance balancing group can be obtained, and based on the number of available hard disks in each second hard disk pool, the total number of available hard disks in the performance balancing group can be determined.

[0120] In practical applications, the initial weight of each first hard disk pool and the initial weight of each second hard disk pool can be determined based on the capacity information of all first hard disk pools and all second hard disk pools (for example, the number of hard disks and the remaining capacity). For example, the total remaining capacity can be determined based on the remaining capacity of each first hard disk pool and the remaining capacity of each second hard disk pool, and then for each first hard disk pool or each second hard disk pool, the ratio of the remaining capacity of the first hard disk pool or the second hard disk pool to the total remaining capacity is calculated, so that the initial weight of the first hard disk pool or the second hard disk pool can be obtained.

[0121] That is, the remaining capacity of each first hard disk pool and the remaining capacity of each second hard disk pool can be summed, that is, the remaining capacity of all hard disk pools in the storage pool can be summed to obtain the total remaining capacity of all hard disk pools. Among them, C j Indicates the remaining capacity of the jth disk pool, and n indicates the total number of disk pools in the storage pool.

[0122] The first weight W of the i-th disk pool in the storage pool i for:

[0123] C i Indicates the current remaining capacity of the i-th hard disk pool, so that the initial weights corresponding to the n hard disk pools can be calculated.

[0124] It can be understood that after obtaining the initial weight of each hard disk pool, for each first hard disk pool whose storage type is a capacity balancing group, the initial weight corresponding to the first hard disk pool calculated by the above relationship can be directly used as the hard disk pool weight of the capacity balancing group of the first hard disk pool.

[0125] For each second hard disk pool, the hard disk pool weight of the performance balancing group of the second hard disk pool can be further determined based on the initial weights of all second hard disk pools, the number of available hard disks in the second hard disk pool, and the total number of available hard disks in the performance balancing group. This process can be achieved in the following ways:

[0126] Sum the initial weights corresponding to each second hard disk pool to obtain the sum of the initial weights;

[0127] Sum the current number of available hard disks corresponding to each second hard disk pool to obtain the total number of current available hard disks in the performance balancing group;

[0128] For each second hard disk pool, the hard disk pool weight of the performance balancing group of the second hard disk pool is determined according to the current number of available hard disks in the second hard disk pool, the sum of the initial weights, and the total number of currently available hard disks in the performance balancing group.

[0129] It should be noted that after the initial weight of each first hard disk pool and the initial weight of each second hard disk pool are obtained through the first weighted calculation, for each second hard disk pool, a second weighted calculation is required based on the initial weight corresponding to each second hard disk pool. According to the initial weight of each second hard disk pool, the number of available hard disks in the second hard disk pool and the total number of available hard disks in the performance balancing group, the final capacity balancing group hard disk pool weight corresponding to each second hard disk pool is calculated.

[0130] It can be understood that the initial weights corresponding to each second hard disk pool can be summed to obtain the sum of the initial weights of the performance balancing group, and then the number of available hard disks corresponding to each second hard disk pool can be summed to obtain the total number of available hard disks in the performance balancing group. Then, for each second hard disk pool, the hard disk pool weight of the capacity balancing group of the second hard disk pool can be obtained based on the number of available hard disks in the second hard disk pool, the sum of the initial weights and the total number of available hard disks in the performance balancing group.

[0131] For example, the storage pool includes n hard disk pools, among which there are m second hard disk pools in the performance balancing group, and the number of the first second hard disk pool is t. Then, the initial weights corresponding to the m second hard disk pools are obtained, which are W and t , W t+1 , …, W t+m Sum the initial weights of each second hard disk pool to get V = W t +W t+1+…+W t+m , V is the sum of the initial weights of the performance balancing group.

[0132] The number of available hard disks in each second hard disk pool is N. t 、N t+1 ,…,N t+m , sum the number of available hard disks in each second hard disk pool to get the total number of available hard disks in the performance balancing group Where, when k = 0, N k =N t , when k = 1, N k =N t+1 , ..., when k = m, N k =N t+m , the number of available hard disks in the second hard disk pool can be multiplied by the sum of the initial weights, and the ratio of the obtained product to the total number of available hard disks in the performance balancing group can be used as the hard disk pool weight of the capacity balancing group of the second hard disk pool.

[0133] That is, the capacity balancing group hard disk pool weight of the second hard disk pool is Among them, N g represents the number of available hard disks corresponding to the g-th second hard disk pool among the m second hard disk pools, W g Indicates the weight corresponding to the g-th second hard disk pool among the m second hard disk pools.

[0134] Below Figure 5 Take the hard disk pools shown in the figure as an example to illustrate weight calculation:

[0135] The weights of DP1 and DP2 in the capacity balancing group are calculated as follows:

[0136] DP1 weight = 10T / (10T+20T+40T+30T+100T);

[0137] DP2 weight = 20T / (10T+20T+40T+30T+100T);

[0138] The weights of DP3, DP4, and DP5 in the performance balancing group are calculated as follows:

[0139] DP3 weight = (40 / 110)*170T / (10T+20T+40T+30T+100T);

[0140] DP4 weight = (20 / 110)*170T / (10T+20T+40T+30T+100T);

[0141] DP5 weight = (50 / 110)*170T / (10T+20T+40T+30T+100T).

[0142] As can be seen from the above, the management server divides all hard disk pools into capacity balancing groups and performance balancing groups, and assigns corresponding weights to each hard disk pool through calculation to reflect the load and performance capabilities of the hard disk pool in the cluster. The overall weight of the performance balancing group is greater than the weight of the capacity balancing group, so that user performance can be guaranteed while taking into account the balance of capacity to better exert the overall performance of the cluster.

[0143] It can be seen that in the embodiment of the present invention, the storage status of all hard disk pools is obtained to determine the weight of each hard disk pool, and the weight of each hard disk pool is sent to the client, so that the client can select a hard disk pool for writing data according to the weight of each hard disk pool. Since each hard disk pool in the embodiment of the present invention corresponds to a corresponding storage status, and the storage status can indicate the storage type and capacity information of the hard disk pool, where the storage type can indicate that the hard disk pool is a performance balancing group or a capacity balancing group, the embodiment of the present invention dynamically adjusts the respective weights according to the storage status of all hard disk pools, which is beneficial to improving cluster performance and user business performance.

[0144] The present invention also provides a corresponding device for the hard disk pool selection method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The hard disk pool selection device provided by the present invention is introduced below. The device is used to implement the hard disk pool selection method provided by the present invention. In this embodiment, the hard disk pool selection device may include or be divided into one or more program modules. The one or more program modules are stored in a storage medium and executed by one or more processors to complete the hard disk pool selection method disclosed in the above embodiment. The program module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the hard disk pool selection device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The hard disk pool selection described below and the hard disk pool selection method described above can be referenced to each other.

[0145] From the perspective of functional modules, see Fig. 9 , Fig. 9 A structural diagram of a hard disk pool selection device provided by the present invention, the device may include:

[0146] A determination module 11, configured to determine a weight of each hard disk pool based on the storage status of all hard disk pools;

[0147] The sending module 12 is used to send the weight of each hard disk pool to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein:

[0148] The storage status of the hard disk pool includes at least the storage type and capacity information of the hard disk pool. The storage type of the hard disk pool includes a performance balancing group and a capacity balancing group. The client is used to write data through the target hard disk pool.

[0149] In one implementation, the remaining capacity ratio of the hard disk pool in the capacity balancing group is greater than or equal to a preset ratio threshold;

[0150] The remaining capacity of the disk pool in the performance balancing group is less than the preset capacity threshold; where:

[0151] The remaining capacity ratio of the disk pool is based on the total capacity and remaining capacity of the disk pool.

[0152] In one implementation, the sending module 12 is configured to:

[0153] The weight of each hard disk pool is sent to the client to update the weight of each hard disk pool in the smooth polling weight list of the client, so that the client determines the target hard disk pool according to the updated smooth polling weight list.

[0154] In one embodiment, it further includes:

[0155] A monitoring module is used to monitor the status of each hard disk pool and obtain the status information of each hard disk pool;

[0156] The calculation module is used for recalculating the weight of each hard disk pool for each hard disk pool when it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed.

[0157] In one embodiment, the computing module includes:

[0158] A first determination unit is used to re-determine the storage status of the hard disk pool when it is determined according to the status information of the hard disk pool that there is a faulty hard disk or a newly added hard disk in the hard disk pool, and return to execute the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools;

[0159] The second determining unit is used to reduce the weight of the hard disk pool by a preset value based on the current weight to obtain a new weight when it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool is stuck or slow;

[0160] The third determining unit is configured to keep the weight of the hard disk pool unchanged when it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has been replaced.

[0161] In one implementation, the first determining unit includes:

[0162] A first determination subunit is used to, when it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has a faulty hard disk, redetermine the storage status of the hard disk pool, and directly return to execute the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools;

[0163] The second determination sub-unit is used to redetermine the storage status of the hard disk pool when it is determined that a new hard disk exists in the hard disk pool based on the status information of the hard disk pool, and when the remaining capacity ratio of the new hard disk is consistent with the remaining capacity ratio of each original hard disk, return to execute the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools.

[0164] In one implementation, the capacity balancing group includes at least one first hard disk pool, the performance balancing group includes at least one second hard disk pool, and the device further includes:

[0165] A fourth determining unit, configured to determine the total number of available hard disks in the performance balancing group based on the number of available hard disks in each second hard disk pool in the performance balancing group;

[0166] Then, the determination module 11 includes:

[0167] a fifth determining unit, configured to determine an initial weight of each first hard disk pool and an initial weight of each second hard disk pool according to the capacity information of each of the first hard disk pools and the capacity information of each of the second hard disk pools;

[0168] A sixth determining unit, configured to use the initial weight as a hard disk pool weight of a capacity balancing group of the first hard disk pool;

[0169] The seventh determination unit is used to determine the performance balancing group hard disk pool weight of the second hard disk pool for each second hard disk pool according to the respective initial weights of all second hard disk pools, the current number of available hard disks in the second hard disk pool and the total number of currently available hard disks in the performance balancing group.

[0170] In one implementation, the fifth determining unit includes:

[0171] A third determining subunit is used to calculate the sum of the remaining capacities according to the remaining capacities of each first hard disk pool and the remaining capacities of each second hard disk pool;

[0172] The fourth determination subunit is used to calculate the ratio of the current remaining capacity of the first hard disk pool or the second hard disk pool to the total current remaining capacity for each first hard disk pool or each second hard disk pool, and obtain the initial weight of the first hard disk pool or the second hard disk pool.

[0173] In one implementation, the seventh determining unit includes:

[0174] A first summing subunit is used to sum the initial weights corresponding to each second hard disk pool to obtain the sum of the initial weights;

[0175] A second summing subunit is used to sum the number of currently available hard disks corresponding to each second hard disk pool to obtain the total number of currently available hard disks in the performance balancing group;

[0176] The fifth determination subunit is used to determine the hard disk pool weight of the performance balancing group of the second hard disk pool for each second hard disk pool according to the current number of available hard disks in the second hard disk pool, the sum of the initial weights and the total number of currently available hard disks in the performance balancing group.

[0177] It should be noted that the hard disk pool selection device provided in the embodiment of the present invention has the same beneficial effects as the hard disk pool selection method involved in the above embodiment, and for the specific introduction of the hard disk pool selection method involved in the embodiment of the present invention, please refer to the above embodiment, and this application will not repeat it here.

[0178] The hard disk pool selection device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Fig.10 A structural diagram of an electronic device provided in an embodiment of the present application, such as Fig.10 As shown, the electronic equipment includes:

[0179] A memory 20, used for storing computer programs;

[0180] The processor 21 is used to implement the steps of the hard disk pool selection method in the above embodiment when executing the computer program.

[0181] The electronic device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0182] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0183] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the memory 20 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 20 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. Further, the memory 20 may also include both an internal storage unit of an electronic device and an external storage device. The memory 20 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the hard disk pool selection method, etc., but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, the relevant steps of the hard disk pool selection method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary storage or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to (data corresponding to the hard disk pool selection result), etc.

[0184] In some embodiments, the electronic device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26. Among them, the display screen 22 and the input / output interface 23, such as a keyboard, belong to the user interface, and the optional user interface may also include a standard wired interface, a wireless interface, and the like. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, and the like. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 24 may optionally include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, and the like, which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 26 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, and the like. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0185] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure.

[0186] In addition, based on the above embodiments, Fig.11 and 12 As shown, the present invention can also provide a distributed storage system, which includes: a communication interface 31, a control module 32, a management module 33 and multiple hard disk pools 34, the control module 31 is connected to the communication interface 32 and the management module 33 respectively, the management module 33 is connected to the multiple hard disk pools 34, and the hard disk pool 34 is a hard disk management unit and is mapped with corresponding multiple hard disks;

[0187] The communication interface 31 is used to connect to multiple clients and receive data written by the clients into the target hard disk pool;

[0188] The control module 32 is used to write data into the target hard disk pool;

[0189] The management module 33 is used to implement the steps of the hard disk pool selection method as described above.

[0190] It is understandable that from the software level, if Fig.11As shown, the control module 32 in the embodiment of the present invention is connected to the communication interface 31 and the management module 33. The control module 32 can be connected to multiple clients through the communication interface 31, and receive the data written by the client to the target hard disk pool through the communication interface 31, and write the data to the target hard disk pool. The management module enables the client to determine the target hard disk pool by implementing the hard disk pool selection method in the embodiment of the present invention. The control module 32 in the embodiment of the present invention can be implemented by the client of the new distributed storage system, which is different from the client on the user side, and the specific implementation method is not limited. The management module 33 is the executor of the hard disk pool selection method in the embodiment of the present invention, and the management module 33 can be implemented by the minior monitoring module.

[0191] In addition, from the hardware level, Fig.12 As shown, the access layer (software layer) of the distributed storage system includes a communication interface 31 and a control module 32, the hardware layer of the distributed storage system includes multiple computer nodes, each computer node (such as CPU, GPU, memory, etc.) can provide computing power for the distributed storage system, the service layer of the distributed storage system includes a management module 33, and the storage layer includes multiple hard disks, and one or more hard disks can constitute a storage pool.

[0192] It should be noted that the distributed storage system in the embodiment of the present invention has the same beneficial effects as the above-mentioned hard disk pool selection method. For the specific introduction of the hard disk pool storage method involved in the embodiment of the present invention, please refer to the above-mentioned embodiment and the present invention will not be repeated here.

[0193] It is understandable that if the hard disk pool selection method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk and other media that can store program codes.

[0194] Based on this, Fig.13 As shown, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program 41 is stored. When the computer program 41 is executed by a processor, the steps of the hard disk pool selection method described above are implemented.

[0195] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0196] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0197] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0198] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0199] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for selecting a hard disk pool, characterized in that: include: Based on the storage status of all hard disk pools, determine the weight of each hard disk pool; The weight of each hard disk pool is sent to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein: The storage status of the hard disk pool includes at least the storage type of the hard disk pool and the capacity information of the hard disk pool. The storage type of the hard disk pool includes a performance balancing group and a capacity balancing group. The client is used to write data through the target hard disk pool.

2. The hard disk pool selection method according to claim 1, characterized in that: The remaining capacity ratio of the hard disk pool in the capacity balancing group is greater than or equal to a preset ratio threshold; The remaining capacity ratio of the hard disk pool in the performance balancing group is less than the preset ratio threshold; wherein: The remaining capacity ratio of the hard disk pool is obtained based on the total capacity and the remaining capacity of the hard disk pool.

3. The hard disk pool selection method according to claim 1, characterized in that: Sending the weight of each hard disk pool to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool, includes: The weight of each hard disk pool is sent to the client to update the weight of each hard disk pool in the smooth polling weight list of the client, so that the client determines the target hard disk pool according to the updated smooth polling weight list.

4. The method for selecting a hard disk pool according to claim 1, characterized in that: Also includes: Monitor the status of each hard disk pool to obtain status information of each hard disk pool; For each hard disk pool, when it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed, the weights of the hard disk pools are recalculated.

5. The method for selecting a hard disk pool according to claim 4, characterized in that: When it is determined according to the status information of the hard disk pool that the hard disks in the hard disk pool have changed, recalculating the weights of the hard disk pools includes: In the case where it is determined according to the status information of the hard disk pool that there is a faulty hard disk or a newly added hard disk in the hard disk pool, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is returned to be executed; When it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool is stuck or slow, the weight of the hard disk pool is reduced by a preset value based on the current weight to obtain a new weight; When it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has been replaced, the weight of the hard disk pool is kept unchanged.

6. The method for selecting a hard disk pool according to claim 5, characterized in that: In the case where it is determined according to the status information of the hard disk pool that there is a faulty hard disk or a newly added hard disk in the hard disk pool, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is returned to be executed, including: In the case where it is determined according to the status information of the hard disk pool that a hard disk in the hard disk pool has a faulty hard disk, the storage status of the hard disk pool is re-determined, and the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools is directly returned to be executed; When it is determined that there is a new hard disk in the hard disk pool based on the status information of the hard disk pool, the storage status of the hard disk pool is re-determined, and when the remaining capacity ratio of the new hard disk is consistent with the remaining capacity ratio of each original hard disk, return to execute the step of determining the weight of each hard disk pool based on the storage status of all hard disk pools.

7. The method for selecting a hard disk pool according to any one of claims 1 to 6, characterized in that: The capacity balancing group includes at least one first hard disk pool, the performance balancing group includes at least one second hard disk pool, and the method further includes: Determine the total number of available hard disks in the performance balancing group based on the number of available hard disks in each of the second hard disk pools in the performance balancing group; Then, determining the weight of each hard disk pool based on the storage status of all hard disk pools includes: Determine, according to the capacity information of all the first hard disk pools and all the second hard disk pools, an initial weight of each of the first hard disk pools and an initial weight of each of the second hard disk pools; Using the initial weight as the hard disk pool weight of the capacity balancing group of the first hard disk pool; For each of the second hard disk pools, the performance balancing group hard disk pool weight of the second hard disk pool is determined according to the initial weights of all the second hard disk pools, the number of available hard disks in the second hard disk pool, and the total number of available hard disks in the performance balancing group.

8. The method for selecting a hard disk pool according to claim 7, characterized in that: The determining, according to the capacity information of all the first hard disk pools and all the second hard disk pools, the initial weight of each of the first hard disk pools and the initial weight of each of the second hard disk pools includes: Calculating a total remaining capacity according to the remaining capacity of each of the first hard disk pools and the remaining capacity of each of the second hard disk pools; For each of the first hard disk pools or each of the second hard disk pools, a ratio of the remaining capacity of the first hard disk pool or the second hard disk pool to the total remaining capacity is calculated to obtain an initial weight of the first hard disk pool or the second hard disk pool.

9. The method for selecting a hard disk pool according to claim 8, characterized in that: For each of the second hard disk pools, determining the hard disk pool weight of the performance balancing group of the second hard disk pool according to the initial weights of all the second hard disk pools, the number of available hard disks in the second hard disk pools, and the total number of available hard disks in the performance balancing group includes: Sum the initial weights corresponding to each of the second hard disk pools to obtain the sum of the initial weights; Sum the number of available hard disks corresponding to each of the second hard disk pools to obtain the total number of available hard disks in the performance balancing group; For each of the second hard disk pools, the performance balancing group hard disk pool weight of the second hard disk pool is determined according to the number of available hard disks in the second hard disk pool, the sum of the initial weights, and the total number of available hard disks in the performance balancing group.

10. A hard disk pool selection device, characterized in that: include: A determination module, used for determining the weight of each hard disk pool based on the storage status of all hard disk pools; A sending module is used to send the weight of each hard disk pool to the client, so that the client determines the target hard disk pool according to the weight of each hard disk pool; wherein: The storage status of the hard disk pool includes at least the storage type of the hard disk pool and the capacity information of the hard disk pool. The storage type of the hard disk pool includes a performance balancing group and a capacity balancing group. The client is used to write data through the target hard disk pool.

11. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the hard disk pool selection method as described in any one of claims 1 to 9 when executing the computer program.

12. A distributed storage system, characterized in that: include: A communication interface, a control module, a management module and a plurality of hard disk pools, wherein the control module is connected to the communication interface and the management module respectively, the management module is connected to the plurality of hard disk pools, and the hard disk pool is a hard disk management unit and is mapped with a plurality of corresponding hard disks; The communication interface is used to connect to multiple clients and receive data written by the clients into the target hard disk pool; The control module is used to write the data into the target hard disk pool; The management module is used to implement the steps of the hard disk pool selection method as described in any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the hard disk pool selection method according to any one of claims 1 to 9 are implemented.

Images