Load balancing method and device for distributed storage link and storage medium

Abstract

This application discloses a load balancing method and related equipment for distributed storage links, which can effectively improve the overall load utilization of a distributed storage system. The method includes: obtaining initial link grouping information corresponding to the distributed storage, wherein the distributed storage includes M storage links; determining T data acquisition time points corresponding to the current processing cycle; acquiring load data corresponding to each of the M storage links based on the T data acquisition time points; determining a first maximum load value corresponding to the M storage links based on the initial link grouping information and the load data; determining a second maximum load value corresponding to target link grouping information based on the load data and the initial link grouping information; and adjusting the M storage links according to the target link grouping information if the difference between the second maximum load value and the first maximum load value satisfies an adjustment condition.

Description

Technical Field

[0001] This application relates to the field of cloud computing, and in particular to a load balancing method, apparatus and storage medium for a distributed storage link. Background Technology

[0002] In cloud computing scenarios, storage is mostly distributed. A distributed storage system consists of numerous storage nodes, ranging from dozens to hundreds. Distributed storage is simultaneously allocated to multiple compute servers, each hosting different services. When a compute server connects, it typically selects several storage nodes to establish storage links. One storage link serves as the primary link for data read and write, while the remaining links act as backups for disaster recovery in case of failure. Because the distributed storage architecture allocates multiple storage nodes to multiple compute nodes, and different compute nodes have different performance requirements, load balancing of the storage nodes and storage links is necessary to avoid time and space hotspots and improve the utilization rate of distributed storage.

[0003] Existing distributed storage software, such as CEPH, typically divides backend storage nodes into several storage node groups to achieve high availability. Load balancing within storage groups can be achieved using the built-in functions of the distributed storage software. However, the storage front-end link from the storage group to the compute node is manually configured, and there is currently no mature load balancing method. Generally, an equal distribution method is used, where each storage node group is assigned the same number of compute nodes.

[0004] Existing distributed storage software can generally achieve load balancing of nodes and links within small groups of backend storage nodes. However, the frontend link management method typically uses an even distribution approach, and once the allocation is completed, it is generally not adjusted again, resulting in various imbalances and preventing the overall capacity of distributed storage from being maximized. Summary of the Invention

[0005] This application provides a load balancing method, apparatus, and storage medium for distributed storage links, which can effectively improve the overall load utilization of distributed storage systems.

[0006] The first aspect of this application provides a load balancing method for distributed storage links, which may include:

[0007] Obtain the initial link group information corresponding to the distributed storage, wherein the distributed storage includes M storage links, where M is an integer greater than or equal to 1;

[0008] Determine the T data acquisition time points corresponding to the current processing cycle, where T is an integer greater than or equal to 1;

[0009] The load data corresponding to each of the M storage links is collected based on the T data collection time points;

[0010] The first maximum load value corresponding to the M storage links is determined based on the initial link grouping information and the load data corresponding to each storage link;

[0011] The second maximum load value corresponding to the target link group information is determined based on the load data corresponding to each storage link and the initial link group information. The second maximum load value is less than the first maximum load value. The target link group information is link group information that is different from the initial link group information.

[0012] If the difference between the second maximum load value and the first maximum load value meets the adjustment conditions, then the M storage links are adjusted according to the target link grouping information.

[0013] A second aspect of this application provides a load balancing device for a distributed storage link, comprising:

[0014] An acquisition unit is used to acquire the initial link group information corresponding to the distributed storage, wherein the distributed storage includes M storage links, where M is an integer greater than or equal to 1;

[0015] The first determining unit is used to determine the T data acquisition time points corresponding to the current processing cycle, where T is an integer greater than or equal to 1;

[0016] The acquisition unit is used to acquire load data corresponding to each of the M storage links according to the T data acquisition time points;

[0017] The second determining unit is used to determine the first maximum load value corresponding to the M storage links based on the initial link grouping information and the load data corresponding to each storage link;

[0018] The third determining unit is used to determine a second maximum load value corresponding to the target link group information based on the load data corresponding to each storage link and the initial link group information, wherein the second maximum load value is less than the first maximum load value, and the target link group information is link group information different from the initial link group information;

[0019] An adjustment unit is used to adjust the M storage links according to the target link grouping information when the difference between the second maximum load value and the first maximum load value meets the adjustment conditions.

[0020] In one possible design, the load data corresponding to each storage link includes IOPS load data (number of read / write operations per second) and bandwidth load data, and the second determining unit is specifically used for:

[0021] The first maximum load value corresponding to the M storage links is determined by the following formula:

[0022]

[0023] in, The first maximum load value, For the Nth storage node in the distributed storage, group the IOPS load data at data acquisition time point t. The bandwidth load data of the Nth storage node group in the distributed storage at data acquisition time point t, where N is the number of storage node groups included in the distributed storage, T is the number of data acquisition time points corresponding to the current processing cycle, and the M storage links correspond to the N storage node groups.

[0024] In one possible design, the third determining unit is specifically used for:

[0025] The second maximum load value is obtained by recursively calculating using the following formula:

[0026]

[0027] Where f[m][n] represents the minimum maximum load value when grouping the first n nodes in the distributed storage and the first m links in the M storage links. This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. The IOPS load data collected at the Tth data acquisition time point for the j-th storage link among the M storage links. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. f[m][n] is the bandwidth load data collected from the j-th storage link among the M storage links at the T-th data collection time point, where T is the number of data collection time points corresponding to the current processing cycle. When m is M and n is N, f[m][n] is the second maximum load value.

[0028] In one possible design, the third determining unit is further configured to perform the following steps:

[0029] Step 1: Calculate the grouping parameter value k′ corresponding to the second maximum load value using the following formula:

[0030]

[0031] in, This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. The IOPS load data collected at the Tth data acquisition time point for the j-th storage link among the M storage links. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. The bandwidth load data collected at the Tth data collection time point for the jth storage link among the M storage links, where T is the number of data collection time points corresponding to the current processing cycle, and N is the storage node grouping included in the distributed storage;

[0032] Step 2: Determine the storage links other than the first k′ storage links among the M storage links as link groups;

[0033] Step 3: Determine the grouping parameter value k′ as M and repeat steps 1 to 2 until all storage links in the M storage links are grouped to obtain the target grouping information.

[0034] In one possible design, the acquisition unit is specifically used for:

[0035] At each of the T data acquisition time points, IOPS load data and bandwidth load data of each of the M storage links are collected;

[0036] The IOPS load data and the bandwidth load data are recorded as percentages, with the percentages based on the maximum IOPS load capacity and the maximum bandwidth load capacity of a single storage node group.

[0037] In one possible design, the acquisition unit records the IOPS load data and the bandwidth load data as percentages, including:

[0038] The IOPS load data and the bandwidth load data are recorded using the following expressions:

[0039]

[0040]

[0041] in, The IOPS load data of the Mth storage link among the M storage links collected at data acquisition time point t. The bandwidth load data of the Mth storage link among the M storage links collected at the data collection time point t, where the collection time point t is any one of the T data collection time points, and m is any one of the M storage links.

[0042] In one possible design, the adjustment unit is further used for:

[0043] Determine whether the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold;

[0044] If the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold, then it is determined that the difference between the first maximum load value and the second maximum load value satisfies the adjustment condition.

[0045] If the difference between the first maximum load value and the second maximum load value is less than the adjustment threshold, then it is determined that the difference between the first maximum load value and the second maximum load value does not meet the adjustment condition.

[0046] A third aspect of this application provides a computer-readable storage medium storing at least one executable instruction that, when executed on a computing device, causes the computing device to perform a load balancing method for a distributed storage link as described in the first aspect of this application.

[0047] The fourth aspect of this application discloses a computer program product that, when run on a computer, causes the computer to execute the load balancing method for distributed storage links described in the first aspect of this application.

[0048] The fifth aspect of this application discloses an application publishing platform for publishing computer program products, wherein when the computer program products are run on a computer, the computer executes the load balancing method of the distributed storage link described in the first aspect of this application.

[0049] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0050] In the embodiments provided in this application, when the difference between the maximum load value of the current processing cycle and the maximum load value of the new group is greater than the adjustment threshold, the storage link group in the current processing cycle is adjusted. This can distribute the storage load of multiple computing nodes evenly across multiple storage node groups in the distributed storage, avoid time hotspots and space hotspots, and balance IOPS load and bandwidth load, which can effectively improve the overall load utilization of the distributed storage system. Attached Figure Description

[0051] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0052] Figure 1 This is a schematic diagram of the storage link structure provided in an embodiment of this application;

[0053] Figure 2 This application provides a system architecture diagram of a load balancing system for a distributed storage link, as illustrated in the embodiments of this application.

[0054] Figure 3 A flowchart illustrating the load balancing method for a distributed storage link provided in an embodiment of this application;

[0055] Figure 4 A virtual structure diagram of a load balancing device for a distributed storage link provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present application, the technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. All embodiments based on the present application should fall within the scope of protection of the present application.

[0058] Distributed storage boasts advantages such as high scalability and low cost, leading to its widespread use in cloud computing. A distributed storage system comprises multiple storage node groups, simultaneously providing storage services to multiple compute nodes. When a compute node connects, it typically selects several storage node groups through manual configuration to establish storage links. One storage link serves as the primary link for data read and write operations, while the remaining storage links act as backups for disaster recovery switching in case of failure.

[0059] Please see Figure 1 , Figure 1This is a schematic diagram of the storage link structure provided in an embodiment of this application. It illustrates the connection method of the primary storage link for a system with M compute nodes and N storage nodes grouped together. Figure 1 As shown, a compute node uses one primary link, and a storage node group may correspond to the primary links of multiple compute nodes.

[0060] Initial allocation of storage links typically uses an equal-sharing method, where each storage node is grouped and each node shares the primary storage link with all compute nodes. After a period of operation, storage link imbalances often occur, with the main imbalance issues being as follows:

[0061] 1. The load capacity of a storage node group is divided into two types: the number of read / write operations per second (IOPS) and the bandwidth capacity. A storage node group may have one capacity reaching its limit while the other has free capacity.

[0062] 2. Individual node groups may exhibit time hotspots. That is, at certain times, the load on a storage node exceeds its capacity; while at other times, the storage node's capacity is sufficient to meet the load demand.

[0063] 3. Local hotspots may exist in storage node groups. That is, some storage nodes are under high load, exceeding their capacity, while other storage node groups are under low load.

[0064] In view of this, this application provides a dynamic storage link adjustment method, which periodically adjusts the storage link according to the storage link load to achieve load balancing in three aspects: IOPS and bandwidth load balancing, time load balancing, and load balancing between node groups, thereby improving the load capacity utilization of distributed storage.

[0065] Please see Figure 2 , Figure 2 The diagram below shows the system architecture of a load balancing system for distributed storage links provided in this application. The system includes a data acquisition module, a recording module, and a processing module. The data acquisition module is used to acquire load data of the storage links, the recording module is used to record the load data and link group data of the storage links, and the processing module reads the load data of the storage links from the recording module, obtains the adjustment value of the storage links based on the adjustment algorithm, and issues an adjustment instruction to the storage system.

[0066] The load balancing method for distributed storage links provided in this application will be described in detail below from the perspective of the load balancing device for distributed storage links.

[0067] Please see Figure 3 , Figure 3A flowchart illustrating the load balancing method for distributed storage links provided in this application embodiment includes:

[0068] 301. Obtain the initial link group information corresponding to the distributed storage.

[0069] In this embodiment, the load balancing device for the distributed storage links can obtain the initial link group information corresponding to the distributed storage. Specifically, it is not limited to obtaining the initial link group information corresponding to the distributed storage; for example, it can receive input initial link group information. The distributed storage includes N storage node groups, and these N storage node groups are divided into M storage links, where M is an integer greater than or equal to 1. That is, the initial link group information divides the N storage node groups into M storage links.

[0070] 302. Determine the T data acquisition time points corresponding to the current processing cycle.

[0071] In this embodiment, the load balancing device of the distributed storage link can determine the T data collection time points corresponding to the current processing cycle, where T is an integer greater than or equal to 1. That is, the load balancing method of the distributed storage link can determine the data collection time points corresponding to the current processing cycle by using a fixed interval, or it can manually select representative data collection time points and record that a processing cycle contains T data collection time points.

[0072] It should be noted that the load balancing device of the distributed storage link can obtain the initial link group information through step 301 and determine the T data collection time points through step 302. However, there is no restriction on the order of execution between these two steps. Step 301 can be executed first, or step 302 can be executed first, or they can be executed simultaneously. There is no specific limitation.

[0073] 303. Collect load data for each of the M storage links based on T data collection time points.

[0074] In this embodiment, the load balancing device of the distributed storage links collects IOPS load data and bandwidth load data for each of the M storage links at each of the T data collection time points; and records the IOPS load data and bandwidth load data as percentages, wherein the percentage method is based on the upper limit of the IOPS load capacity and the upper limit of the bandwidth load capacity of a single storage node group. Specifically, the IOPS load data and bandwidth load data can be recorded using the following expressions:

[0075]

[0076]

[0077] in, This refers to the IOPS load data of the Mth storage link out of the M storage links collected at data acquisition time point t. Let m be the bandwidth load data of the Mth storage link among the M storage links collected at data acquisition time point t, where t is any one of the T data acquisition time points, and m is any one of the M storage links.

[0078] 304. Determine the first maximum load value corresponding to the M storage links based on the initial link grouping information and the load data corresponding to each storage link.

[0079] In this embodiment, after collecting the load data corresponding to each of the M storage links, the data processing stage begins. The load balancing device of the distributed storage links can read the stored link load data and existing link groups from the recording module, and calculate the load of each storage node group based on the existing link groups. Specifically, the first maximum load value corresponding to the M storage links can be determined by the following formula, which includes the maximum load value corresponding to IOPS load and the maximum load value corresponding to bandwidth load:

[0080]

[0081] in, The first maximum load value, For the nth storage node in the distributed storage, group the IOPS load data at data acquisition time t. This represents the bandwidth load data of the nth storage node group in the distributed storage at data acquisition time point t, where N is the number of storage node groups included in the distributed storage, and T is the number of data acquisition time points corresponding to the current processing cycle.

[0082] 305. Determine the second maximum load value corresponding to the target link group information based on the load data corresponding to each storage link and the initial link group information.

[0083] In this embodiment, the load balancing device of the distributed storage link can determine the second maximum load value corresponding to the target link group information based on the load data corresponding to each storage link and the initial link group information. The second maximum load value is less than the first maximum load value, and the target link group information is link group information different from the initial link group information. Specifically, the second maximum load value can be obtained by recursive calculation using the following formula:

[0084]

[0085] Where f[m][n] represents the minimum maximum load value when grouping the first n nodes and the first m links out of the M storage links in the distributed storage. This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. For the T-th data acquisition time point, the IOPS load data collected from the j-th storage link out of the M storage links is... The bandwidth load data collected at the j-th storage link out of M storage links at the first data acquisition time point. f[m][n] represents the bandwidth load data collected from the j-th storage link among M storage links at the T-th data collection time point, where T is the number of data collection time points corresponding to the current processing cycle. When m is M and n is N, f[m][n] is the second maximum load value.

[0086] In other words, the load balancing device for distributed storage links is based on a dynamic programming algorithm, which determines the minimum to maximum load value for storage node groups based on the collected storage link load data. And a new grouping method. The state transition expression involving the dynamic programming algorithm is as follows:

[0087]

[0088] Therefore, we can deduce step by step from the maximum load value f[1][1] under 1 storage node group and 1 storage link to the maximum load value f[M][N] under N storage node groups and M storage links, and f[M][N] is the second maximum load value.

[0089] In one embodiment, the load balancing device of the distributed storage link can determine the target packet information through the following steps:

[0090] Step 1: Calculate the grouping parameter value k′ corresponding to the second maximum load value using the following formula:

[0091]

[0092] in, This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. For the T-th data acquisition time point, the IOPS load data collected from the j-th storage link out of the M storage links is... The bandwidth load data collected at the j-th storage link out of M storage links at the first data acquisition time point. The bandwidth load data collected at the Tth data collection time point for the jth storage link among the M storage links, where T is the number of data collection time points corresponding to the current processing cycle, and N is the storage node grouping included in the distributed storage.

[0093] Step 2: Determine the storage links (excluding the first k′ storage links) out of the M storage links as link groups;

[0094] Step 3: Set the grouping parameter value k′ to M and repeat steps 1 to 2 until all storage links in the M storage links are grouped to obtain the target grouping information.

[0095] That is, when determining the second maximum load value, there will be a grouping parameter value k′. When the second maximum load value is to minimize the first maximum load value, the M storage links are divided by the grouping parameter value k′. The first k′ storage links in the M storage links are divided into N-1 groups, and k′+1 to M are divided into one storage link. Then, the above steps are repeated with k′ as M until all storage links in the M storage links are grouped, and the target grouping information is obtained.

[0096] 306. If the difference between the second maximum load value and the first maximum load value meets the adjustment conditions, then adjust the M storage links according to the target link grouping information.

[0097] In this embodiment, after obtaining the second maximum load value and the first maximum load value, the load balancing device of the distributed storage link can determine whether the difference between the first maximum load value and the second maximum load value is greater than an adjustment threshold. If the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold, it is determined that the difference between the first maximum load value and the second maximum load value meets the adjustment condition. Then, the M storage links are adjusted according to the target link group information, and the next processing cycle begins. If the difference between the first maximum load value and the second maximum load value is less than the adjustment threshold, it is determined that the difference between the first maximum load value and the second maximum load value does not meet the adjustment condition. Therefore, the M storage links are not adjusted, and the next processing cycle begins. That is, the load balancing device of the distributed storage link can set an adjustment threshold e to adjust the maximum load value under the new group. Maximum load value under the original group In comparison, if This indicates a significant decrease in the maximum load under the new group, prompting adjustments to the storage link groups, updating the link group data, and entering the next processing cycle. If This indicates that the maximum load reduction under the new group is not significant enough, the storage link will not be adjusted, and the process will proceed to the next processing cycle.

[0098] In summary, it can be seen that in the embodiments provided in this application, when the difference between the maximum load value of the current processing cycle and the maximum load value of the new group is greater than the adjustment threshold, the storage link group in the current processing cycle is adjusted. This can distribute the storage load of multiple computing nodes evenly across multiple storage node groups in the distributed storage, avoid time hotspots and space hotspots, and balance IOPS load and bandwidth load, which can effectively improve the overall load utilization of the distributed storage system.

[0099] The embodiments of this application have been described above from the perspective of load balancing methods for distributed storage links. The embodiments of this application will now be described from the perspective of load balancing devices for distributed storage links:

[0100] Please see Figure 4 , Figure 4 This is a virtual structural diagram of a load balancing device for a distributed storage link provided in an embodiment of this application. The load balancing device 400 for the distributed storage link includes:

[0101] The acquisition unit 401 is used to acquire the initial link group information corresponding to the distributed storage, wherein the distributed storage includes M storage links, and M is an integer greater than or equal to 1;

[0102] The first determining unit 402 is used to determine the T data acquisition time points corresponding to the current processing cycle, where T is an integer greater than or equal to 1;

[0103] The acquisition unit 403 is used to acquire load data corresponding to each of the M storage links according to the T data acquisition time points;

[0104] The second determining unit 404 is used to determine the first maximum load value corresponding to the M storage links based on the initial link grouping information and the load data corresponding to each storage link;

[0105] The third determining unit 405 is used to determine a second maximum load value corresponding to the target link group information based on the load data corresponding to each storage link and the initial link group information, wherein the second maximum load value is less than the first maximum load value, and the target link group information is link group information different from the initial link group information;

[0106] The adjustment unit 406 is used to adjust the M storage links according to the target link grouping information when the difference between the second maximum load value and the first maximum load value meets the adjustment conditions.

[0107] In one possible design, the load data corresponding to each storage link includes IOPS load data (number of read / write operations per second) and bandwidth load data, and the second determining unit 404 is specifically used for:

[0108] The first maximum load value corresponding to the M storage links is determined by the following formula:

[0109]

[0110] in, The first maximum load value, For the Nth storage node in the distributed storage, group the IOPS load data at data acquisition time point t. The bandwidth load data of the Nth storage node group in the distributed storage at data acquisition time point t, where N is the number of storage node groups included in the distributed storage, T is the number of data acquisition time points corresponding to the current processing cycle, and the M storage links correspond to the N storage node groups.

[0111] In one possible design, the third determining unit 405 is specifically used for:

[0112] The second maximum load value is obtained by recursively calculating using the following formula:

[0113]

[0114] Where f[m][n] represents the minimum maximum load value when grouping the first n nodes in the distributed storage and the first m links in the M storage links. This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. The IOPS load data collected at the Tth data acquisition time point for the j-th storage link among the M storage links. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. f[m][n] is the bandwidth load data collected from the j-th storage link among the M storage links at the T-th data collection time point, where T is the number of data collection time points corresponding to the current processing cycle. When m is M and n is N, f[m][n] is the second maximum load value.

[0115] In one possible design, the third determining unit 405 is further configured to perform the following steps:

[0116] Step 1: Calculate the grouping parameter value k′ corresponding to the second maximum load value using the following formula:

[0117]

[0118] in, This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. The IOPS load data collected at the Tth data acquisition time point for the j-th storage link among the M storage links. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. The bandwidth load data collected at the Tth data collection time point for the jth storage link among the M storage links, where T is the number of data collection time points corresponding to the current processing cycle, and N is the storage node grouping included in the distributed storage;

[0119] Step 2: Determine the storage links other than the first k′ storage links among the M storage links as link groups;

[0120] Step 3: Determine the grouping parameter value k′ as M and repeat steps 1 to 2 until all storage links in the M storage links are grouped to obtain the target grouping information.

[0121] In one possible design, the acquisition unit 403 is specifically used for:

[0122] At each of the T data acquisition time points, IOPS load data and bandwidth load data of each of the M storage links are collected;

[0123] The IOPS load data and the bandwidth load data are recorded as percentages, with the percentages based on the maximum IOPS load capacity and the maximum bandwidth load capacity of a single storage node group.

[0124] In one possible design, the acquisition unit 403 records the IOPS load data and the bandwidth load data as percentages, including:

[0125] The IOPS load data and the bandwidth load data are recorded using the following expressions:

[0126]

[0127]

[0128] in, The IOPS load data of the Mth storage link among the M storage links collected at data acquisition time point t. The bandwidth load data of the Mth storage link among the M storage links collected at the data collection time point t, where the collection time point t is any one of the T data collection time points, and m is any one of the M storage links.

[0129] In one possible design, the adjustment unit 406 is further used for:

[0130] Determine whether the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold;

[0131] If the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold, then it is determined that the difference between the first maximum load value and the second maximum load value satisfies the adjustment condition.

[0132] If the difference between the first maximum load value and the second maximum load value is less than the adjustment threshold, then it is determined that the difference between the first maximum load value and the second maximum load value does not meet the adjustment condition.

[0133] This application also provides another load balancing device for a distributed storage link, which is deployed on a server. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of a server structure provided in an embodiment of the present invention. The server 500 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 522 (e.g., one or more processors) and a memory 532, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 542 or data 544. The memory 532 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 522 may be configured to communicate with the storage media 530 and execute the series of instruction operations stored in the storage media 530 on the server 500.

[0134] Server 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input / output interfaces 558, and / or one or more operating systems 541, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0135] The steps performed by the load balancing device of the distributed storage link in the above embodiments can be based on this. Figure 5 The server structure shown.

[0136] This application also provides a computer-readable storage medium storing at least one executable instruction, which, when executed on a computing device, causes the computing device to perform the load balancing method of the distributed storage link described in any of the above embodiments.

[0137] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0138] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A load balancing method for distributed storage links, characterized in that, include: Obtain the initial link group information corresponding to the distributed storage, wherein the distributed storage includes M storage links, where M is an integer greater than or equal to 1; Determine the T data acquisition time points corresponding to the current processing cycle, where T is an integer greater than or equal to 1; The load data corresponding to each of the M storage links is collected based on the T data collection time points; The first maximum load value corresponding to the M storage links is determined based on the initial link grouping information and the load data corresponding to each storage link; The second maximum load value corresponding to the target link group information is determined based on the load data corresponding to each storage link and the initial link group information. The second maximum load value is less than the first maximum load value. The target link group information is link group information that is different from the initial link group information. If the difference between the second maximum load value and the first maximum load value meets the adjustment conditions, then the M storage links are adjusted according to the target link grouping information; The step of determining the second maximum load value corresponding to the target link packet information based on the load data corresponding to each storage link and the initial link packet information includes: The second maximum load value is obtained by recursively calculating using the following formula: ； in, To minimize the maximum load value when grouping the first n nodes in the distributed storage and the first m links out of the M storage links, This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. For the first The IOPS load data collected at each of the M storage links at the j-th storage link is analyzed. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. For the first The bandwidth load data collected at each of the M storage links at the j-th storage link. The number of data acquisition time points corresponding to the current processing cycle, where m is M and n is N. This is the second maximum load value.

2. The method according to claim 1, characterized in that, The load data corresponding to each storage link includes IOPS load data (number of read / write operations per second) and bandwidth load data. Determining the first maximum load value corresponding to the M storage links based on the initial link grouping information and the load data corresponding to each storage link includes: The first maximum load value corresponding to the M storage links is determined by the following formula: ； in, The first maximum load value, For the nth storage node group among the N storage node groups included in the distributed storage, at the data acquisition time point IOPS load data, For the nth storage node group among the N storage node groups included in the distributed storage, at the data acquisition time point Bandwidth load data, The number of storage node groups included in the distributed storage. The number of data acquisition time points corresponding to the current processing cycle, and the M storage links correspond to the N storage node groups.

3. The method according to claim 1, characterized in that, The method further includes: Step 1: Calculate the grouping parameter value corresponding to the second maximum load value using the following formula. : in, This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. For the first The IOPS load data collected at each of the M storage links at the j-th storage link is analyzed. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. For the first The bandwidth load data collected at each of the M storage links at the j-th storage link. This refers to the number of data acquisition time points corresponding to the current processing cycle. The distributed storage includes storage nodes grouped together; Step 2: Remove the first [item] from the M storage links. Storage links other than the specified storage link are identified as link groups; Step 3: Set the grouping parameter values Steps 1 to 2 are repeated for M until all storage link groups in the M storage links are completed, so as to obtain the target link group information.

4. The method according to claim 2, characterized in that, The step of collecting load data corresponding to each of the M storage links based on the T data collection time points includes: At each of the T data acquisition time points, IOPS load data and bandwidth load data of each of the M storage links are collected; The IOPS load data and the bandwidth load data are recorded as percentages, with the percentages based on the maximum IOPS load capacity and the maximum bandwidth load capacity of a single storage node group.

5. The method according to claim 4, characterized in that, The step of recording the IOPS load data and the bandwidth load data as percentages includes: The IOPS load data and the bandwidth load data are recorded using the following expressions: ； ； in, This refers to the IOPS load data of the m-th storage link out of the M storage links collected at data acquisition time point t. The bandwidth load data of the m-th storage link among the M storage links collected at the data collection time point t, where the collection time point t is any one of the T data collection time points, and m is any one of the M storage links.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Determine whether the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold; If the difference between the first maximum load value and the second maximum load value is greater than the adjustment threshold, then it is determined that the difference between the first maximum load value and the second maximum load value satisfies the adjustment condition. If the difference between the first maximum load value and the second maximum load value is less than the adjustment threshold, then it is determined that the difference between the first maximum load value and the second maximum load value does not meet the adjustment condition.

7. A load balancing device for a distributed storage link, characterized in that, include: An acquisition unit is used to acquire the initial link group information corresponding to the distributed storage, wherein the distributed storage includes M storage links, where M is an integer greater than or equal to 1; The first determining unit is used to determine the T data acquisition time points corresponding to the current processing cycle, where T is an integer greater than or equal to 1; The acquisition unit is used to acquire load data corresponding to each of the M storage links according to the T data acquisition time points; The second determining unit is used to determine the first maximum load value corresponding to the M storage links based on the initial link grouping information and the load data corresponding to each storage link; The third determining unit is used to determine a second maximum load value corresponding to the target link group information based on the load data corresponding to each storage link and the initial link group information, wherein the second maximum load value is less than the first maximum load value, and the target link group information is link group information different from the initial link group information; An adjustment unit is configured to adjust the M storage links according to the target link grouping information when the difference between the second maximum load value and the first maximum load value meets the adjustment conditions. The step of determining the second maximum load value corresponding to the target link packet information based on the load data corresponding to each storage link and the initial link packet information includes: The second maximum load value is obtained by recursively calculating using the following formula: ； in, To minimize the maximum load value when grouping the first n nodes in the distributed storage and the first m links out of the M storage links, This refers to the IOPS load data collected from the j-th storage link out of the M storage links at the first data acquisition time point. For the first The IOPS load data collected at each of the M storage links at the j-th storage link is analyzed. The bandwidth load data collected at the first data acquisition time point is the j-th storage link among the M storage links. For the first The bandwidth load data collected at each of the M storage links at the j-th storage link. The number of data acquisition time points corresponding to the current processing cycle, where m is M and n is N. This is the second maximum load value.

8. A computer program product, characterized in that, When the computer program product is run on a computer, the computer performs the load balancing method for the distributed storage link as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one executable instruction, which, when executed on a computing device, causes the computing device to perform a load balancing method for a distributed storage link as described in any one of claims 1 to 6.

Images