Traffic scheduling method and device for distributed storage cluster

Abstract

The disclosed embodiment provides a traffic scheduling method and device for a distributed storage cluster, including: obtaining the write traffic corresponding to each historical cycle of the distributed storage cluster in N historical cycles; obtaining the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical cycle; predicting the write traffic prediction value of the N+1 new cycle according to the write traffic corresponding to each historical cycle; generating a capacity increment prediction quantization model and a load prediction quantization model according to the write traffic corresponding to each historical cycle, the write traffic proportion, capacity increment and load corresponding to each storage pool in each cycle; calling the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1 new cycle, generating a target traffic scheduling strategy for the N+1 new cycle, and controlling the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy. This method improves the scheduling efficiency of the cluster.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of distributed storage clusters, and in particular to a traffic scheduling method and device for a distributed storage cluster. Background Art

[0002] Distributed storage clusters can store and call various unstructured data files including text, pictures, audio and video through the network, providing massive, secure, low-cost, reliable and highly available object storage services. Among them, a single distributed storage cluster mounts multiple storage pools. Over time, the writable capacity and load on multiple storage pools will gradually become unbalanced. Therefore, in a distributed storage cluster, a traffic scheduling method between multiple storage pools is required to ensure that the writable capacity and load of multiple storage pools are balanced.

[0003] In the prior art, technicians collect and analyze the writable capacity and load of multiple storage pools in the Tth cycle, and adjust the traffic write ratio of multiple storage pools in the T+1th cycle based on past manual experience, in the hope that the writable capacity and load of multiple storage pools are balanced.

[0004] However, according to past manual experience, the accuracy of adjusting the traffic write ratio of multiple storage pools in the T+1 period is poor, and multiple periods may need to be adjusted to balance the writable capacity and load of multiple storage pools. Therefore, the scheduling efficiency of the above traffic scheduling method is low. Summary of the invention

[0005] The embodiments of the present disclosure provide a traffic scheduling method and device for a distributed storage cluster, which can quickly balance the writable capacity and load of multiple storage pools and improve the scheduling efficiency of the distributed storage cluster.

[0006] In a first aspect, an embodiment of the present disclosure provides a traffic scheduling method for a distributed storage cluster, including:

[0007] Obtaining the write traffic corresponding to each historical period of a distributed storage cluster in N historical periods; the distributed storage cluster provides storage services through multiple storage pools;

[0008] Obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical period;

[0009] According to the write traffic corresponding to each historical cycle, predict the write traffic forecast value of the N+1 new cycle;

[0010] Generate a capacity increment prediction quantitative model and a load prediction quantitative model based on the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios;

[0011] The capacity increment prediction quantization model and the load prediction quantization model are called according to the write traffic prediction value of the N+1th new cycle to generate the target traffic scheduling strategy of the N+1 new cycle, and the traffic scheduling of each resource pool is controlled according to the target scheduling ratio indicated by the target traffic scheduling strategy.

[0012] In a second aspect, an embodiment of the present disclosure provides a traffic scheduling device for a distributed storage cluster, including:

[0013] An acquisition unit, used to acquire the write traffic corresponding to each historical period of the distributed storage cluster in N historical periods; the distributed storage cluster provides storage services through multiple storage pools;

[0014] The acquisition unit is further used to obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical period;

[0015] A prediction unit, used to predict a write traffic prediction value of the N+1th new cycle according to the write traffic corresponding to each historical cycle;

[0016] A model generation unit is used to generate a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios;

[0017] The scheduling unit is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, generate a target traffic scheduling strategy for the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy.

[0018] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0019] The memory stores computer-executable instructions;

[0020] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the traffic scheduling method for the distributed storage cluster as described in the first aspect and various possible designs of the first aspect.

[0021] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the traffic scheduling method of the distributed storage cluster as described in the first aspect and various possible designs of the first aspect is implemented.

[0022] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the traffic scheduling method for a distributed storage cluster as described in the first aspect and various possible designs of the first aspect.

[0023] The traffic scheduling method and device of the distributed storage cluster provided in this embodiment include: obtaining the write traffic corresponding to each historical cycle of the distributed storage cluster in N historical cycles; the distributed storage cluster provides storage services through multiple storage pools; obtaining the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical cycle; predicting the write traffic prediction value of the N+1 new cycle according to the write traffic corresponding to each historical cycle; generating a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical cycle, the write traffic proportion, capacity increment and load corresponding to each storage pool in each cycle; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios; calling the capacity increment prediction quantitative model and the load prediction quantitative model according to the write traffic prediction value of the N+1 new cycle, generating a target traffic scheduling strategy for the N+1 new cycle, and controlling the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increment meet the balance index. In this technical solution, the write traffic forecast value of the N+1 new cycle is first predicted based on the write traffic corresponding to each historical cycle, and then the capacity increment prediction quantification model and the load prediction quantification model are used to predict the capacity increment and load of the resource pool brought about by different scheduling ratios. Then, according to the impact of different scheduling ratios on capacity and load balance, the target traffic scheduling strategy for the N+1 new cycle is generated, and the capacity and load of multiple storage pools are quickly balanced through the target traffic scheduling scheme, thereby improving the scheduling efficiency of the distributed storage cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 A schematic diagram of an application scenario of a traffic scheduling method for a distributed storage cluster provided in an embodiment of the present disclosure;

[0026] Figure 2 The process of the traffic scheduling method of the distributed storage cluster provided in the embodiment of the present disclosure is Figure 1 ;

[0027] Figure 3 Schematic diagram of the traffic scheduling method of the distributed storage cluster provided in the embodiment of the present disclosure Figure 1 ;

[0028] Figure 4 The process of the traffic scheduling method of the distributed storage cluster provided in the embodiment of the present disclosure is Figure 2 ;

[0029] Figure 5 A schematic diagram of the structure of a traffic scheduling device for a distributed storage cluster provided in an embodiment of the present disclosure;

[0030] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] First, the background technology and some terms involved in this application are explained:

[0033] Distributed storage cluster: Distributed storage cluster can store and call various unstructured data files including text, pictures, audio and video through the network, providing massive, secure, low-cost, reliable and highly available object storage services. Distributed storage cluster can include multiple storage pools.

[0034] Storage pool stock capacity: the capacity already used in the storage pool, that is, the total size of all files in the storage pool.

[0035] The writable capacity of a storage pool: the remaining available space in the storage pool, that is, the capacity to which users can add files. The writable capacity is usually calculated by subtracting the stock capacity from the total capacity. For example, if the total capacity of a storage pool is 100GB and the stock capacity is 50GB, the writable capacity is 50GB.

[0036] Capacity increment of a storage pool: the capacity of files added to the storage pool. The stock capacity of the storage pool in the N+1th new cycle is equal to the sum of the historical stock capacity of the storage pool in the Nth cycle and the capacity increment of the storage pool in the N+1th new cycle. For example, if the total capacity of the storage pool is 100GB, the historical stock capacity of the Nth cycle is 50GB, and the capacity increment is 5GB, then the stock capacity of the N+1th new cycle is 55GB.

[0037] Distributed storage clusters can store and call various unstructured data files including text, pictures, audio and video through the network, providing massive, secure, low-cost, reliable and highly available object storage services. A single distributed storage cluster mounts multiple storage pools. As the writable capacity and load on multiple storage pools become unbalanced over time, a traffic scheduling solution between multiple storage pools is required in the distributed storage cluster to ensure that the writable capacity and load of different storage pools are balanced.

[0038] Among them, the reasons why the writable capacity and load on multiple storage pools become unbalanced over time include:

[0039] 1. Expand new storage pools: When the overall capacity of the distributed storage cluster is insufficient, you can expand new storage pools. The writable capacity of the newly expanded storage pool is much larger than that of other storage pools.

[0040] 2. Take old storage pools offline: Some storage pools need to be taken offline for various reasons, which will cause the data in the storage pool to be moved to other storage pools, and may also cause read and write imbalance in other storage pools.

[0041] 3. Unbalanced user writes: After users write data, the stock capacity or writable capacity of different storage pools may be unbalanced due to data scheduling / differences in data lifecycles.

[0042] In the prior art, technicians collect and analyze the writable capacity and load of multiple storage pools in the Tth cycle, and adjust the flow write ratio of multiple storage pools in the T+1th cycle (that is, Figure 2 The write traffic ratio in the embodiment is expected to balance the writable capacity and load of multiple storage pools.

[0043] However, according to past manual experience, the accuracy of adjusting the traffic write ratio of multiple storage pools in the T+1 period is poor, and multiple periods may need to be adjusted to balance the writable capacity and load of multiple storage pools. Therefore, the scheduling efficiency of the above traffic scheduling method is low.

[0044] It can be seen that how to quickly balance the writable capacity and load of multiple storage pools to improve the scheduling efficiency of distributed storage clusters is a technical problem that needs to be solved urgently.

[0045] In response to the technical problems in the prior art, the inventor's technical conception is as follows: based on the relationship between read-write load / capacity and traffic scheduling ratio, a certain AI (Artificial Intelligence) model is used to intelligently predict the impact of different scheduling ratios on capacity / load balancing, thereby intelligently selecting the optimal solution among multiple traffic scheduling ratios.

[0046] Accordingly, the specific steps may include: first, obtaining the write traffic corresponding to each historical cycle of the distributed storage cluster in N historical cycles; the distributed storage cluster provides storage services through multiple storage pools; obtaining the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical cycle. Then, based on the write traffic corresponding to each historical cycle, predict the write traffic prediction value of the N+1 new cycle; based on the write traffic corresponding to each historical cycle, the write traffic proportion, capacity increment and load corresponding to each storage pool in each cycle, generate a capacity increment prediction quantitative model and a load prediction quantitative model; wherein, the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios. Finally, according to the write traffic prediction value of the N+1 new cycle, call the capacity increment prediction quantitative model and the load prediction quantitative model, generate the target traffic scheduling strategy of the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increment meet the balance index.

[0047] In this technical solution, the write traffic forecast value of the N+1 new cycle is first predicted based on the write traffic corresponding to each historical cycle, and then the capacity increment prediction quantification model and the load prediction quantification model are used to predict the capacity increment and load of the resource pool brought about by different scheduling ratios. Then, according to the impact of different scheduling ratios on capacity and load balance, the target traffic scheduling strategy for the N+1 new cycle is generated, and the capacity and load of multiple storage pools are quickly balanced through the target traffic scheduling scheme, thereby improving the scheduling efficiency of the distributed storage cluster.

[0048] The application scenarios of the embodiments of the present disclosure are explained below:

[0049] The traffic scheduling method of a distributed storage cluster provided in the embodiments of the present disclosure can be applied to traffic scheduling scenarios of various distributed storage clusters. Figure 1 A schematic diagram of an application scenario of a traffic scheduling method for a distributed storage cluster provided by an embodiment of the present disclosure. Figure 1 As shown, the API (Application Programming Interface) of the distributed storage cluster can apply the traffic scheduling strategy to write the write traffic of the distributed storage cluster to each storage pool according to the write ratio of each storage pool in the corresponding strategy.

[0050] The storage pool is the application entity of the write traffic scheduling strategy. Different storage pools can take over write traffic according to different write ratios. The digital service can obtain the capacity and load indicators of each storage pool and automatically build the capacity / load indicators of the storage pool dimension. The intelligent model module can use various AI models ( Figure 1 The traffic prediction model and traffic scheduling model in the system can intelligently predict future write traffic and intelligently calculate the optimal scheduling method. Among them, the central control service module can call the distributed storage cluster API interface according to the optimal strategy recommended by the intelligent model, and the distributed storage cluster API application can recommend the optimal strategy.

[0051] Figure 2 The flow chart of the distributed storage cluster traffic scheduling method provided by the embodiment of the present disclosure Figure 1 In the embodiment of the present disclosure, the execution subject of the scheduling method may be a terminal or a server. Figure 2 As shown, the traffic scheduling method may include:

[0052] S201. Obtain the write traffic corresponding to each historical period of the distributed storage cluster in N historical periods; the distributed storage cluster provides storage services through multiple storage pools.

[0053] In an embodiment of the present disclosure, a distributed storage cluster may include multiple storage pools; wherein the number of the multiple storage pools may be set as needed. wherein N is a positive integer. in the embodiment of the present disclosure, the value of N and the cycle duration are not specifically limited. optionally, the value of N may be 10, 20, 30, etc., and the cycle duration may be 6 hours, 12 hours, one day, etc.

[0054] S202: Obtain the write traffic proportion, capacity increment, and load corresponding to each storage pool in the distributed storage cluster in each historical period.

[0055] The write traffic ratio is used to represent the write traffic allocated to each storage pool. The capacity increment is used to represent the newly added capacity of each storage pool. The load may include one or more of a read request load, a write request load, and a query request load. Optionally, the load represents the number of requests to be processed by the storage pool. Accordingly, the load of the storage pool includes: the sum of the number of pending read requests, the number of pending write requests, and the number of pending query requests.

[0056] For example, a distributed storage cluster includes storage pool A, storage pool B, and storage pool C. The total write traffic of the distributed storage cluster in the Nth cycle is 10 GB. The write traffic ratio can be: storage pool A (30%), storage pool B (30%), storage pool C (40%). In this case, the write traffic allocated to storage pool A is 3 GB, and the capacity increment is 3 GB. The write traffic allocated to storage pool B is 3 GB, and the capacity increment is 3 GB. The write traffic allocated to storage pool C is 4 GB, and the capacity increment is 4 GB.

[0057] S203: predict the write traffic prediction value of the N+1th new cycle according to the write traffic corresponding to each historical cycle.

[0058] In some embodiments, the write traffic of the N+1th new cycle has a strong relationship with the write traffic of N historical cycles, and the write traffic prediction value of the N+1th new cycle can be predicted by the trend prediction model. Accordingly, this step may include: predicting the write traffic prediction value of the N+1th new cycle by the trend prediction model according to the write traffic corresponding to each historical cycle.

[0059] The trend prediction model includes a prophet model or a long short-term memory (LSTM) model. For example, Figure 3 As shown, the trend prediction model includes writing a prediction quantization relationship function.

[0060] Here, since the write traffic of the N+1th new cycle has a strong relationship with the write traffic of N historical cycles, the accuracy of the write traffic prediction value of the N+1th new cycle can be improved by using the trend prediction model.

[0061] S204. Generate a capacity increment prediction quantitative model and a load prediction quantitative model based on the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; and the load prediction quantitative model is used to predict the load brought about by different scheduling ratios. The scheduling ratio is the write traffic proportion corresponding to each storage pool.

[0062] In some embodiments, this step may include: generating a capacity increment prediction quantitative model based on the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period, and the capacity increment; and generating a load prediction quantitative model based on the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period, and the load.

[0063] For example, Figure 3 As shown, the capacity increment prediction quantitative model includes a capacity increment prediction quantitative relationship function.

[0064] Optionally, the load includes one or more of a read request load, a write request load, and a query request load. Figure 3 As shown, the load includes a read request load (which can be expressed as read IOPS), a write request load (which can be expressed as write IOPS) and a query request load (which can be expressed as QPS).

[0065] In this case, the load prediction quantification model may include a write prediction quantification relationship function between write traffic, write traffic proportion and write request load; as well as a read prediction quantification relationship function between write traffic, write traffic proportion and read request load; and a query prediction quantification relationship function between write traffic, write traffic proportion and query request load.

[0066] S205. Call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, generate the target traffic scheduling strategy of the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increments meet the balance index.

[0067] In some embodiments, the capacity increment prediction quantization model and the load prediction quantization model are called according to the write traffic prediction value of the N+1 new cycle to generate a target traffic scheduling strategy for the N+1 new cycle, including: using a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1 new cycle to generate a target traffic scheduling strategy for the N+1 new cycle.

[0068] It should be noted that the capacity increment of the storage pool refers to the capacity of files added to the storage pool. The stock capacity of the storage pool in the N+1th new cycle is equal to the sum of the historical stock capacity of the storage pool in the Nth cycle and the capacity increment of the storage pool in the N+1th new cycle.

[0069] For example, the total capacity of storage pool A is 100 GB, the historical stock capacity of the Nth cycle is 50 GB, the capacity increment is 5 GB, the stock capacity of the N+1th new cycle is 55 GB, and the writable capacity is 45 GB.

[0070] Herein, making the capacity increment satisfy the balance index means: making the difference in stock capacity between storage pools smaller than a preset capacity threshold, or making the difference in writable capacity between storage pools smaller than a preset capacity threshold.

[0071] In some embodiments, making the load and capacity increments meet the balance index means: making the load difference between storage pools less than a preset load threshold, and making the stock capacity difference between storage pools less than a preset capacity threshold. The load represents the number of requests to be processed by the storage pool. The load difference between storage pools represents the difference in the number of requests to be processed.

[0072] In other embodiments, making the load and capacity increments satisfy the balance index means: making the load difference between storage pools smaller than a preset load threshold, and making the writable capacity difference between storage pools smaller than a preset capacity threshold.

[0073] In the embodiment of the present disclosure, the values ​​of the preset load threshold and the preset capacity threshold are not specifically limited and can be set and modified as needed. Optionally, the preset load threshold (ie, the difference in the number of pending requests) is 100. The preset capacity threshold is 10 GB.

[0074] The disclosed embodiment provides a traffic scheduling method for a distributed storage cluster: first, obtain the write traffic corresponding to each historical cycle of the distributed storage cluster in N historical cycles; the distributed storage cluster provides storage services through multiple storage pools; obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical cycle. Then, according to the write traffic corresponding to each historical cycle, predict the write traffic prediction value of the N+1 new cycle; according to the write traffic corresponding to each historical cycle, the write traffic proportion, capacity increment and load corresponding to each storage pool in each cycle, generate a capacity increment prediction quantitative model and a load prediction quantitative model; wherein, the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought by different scheduling ratios; the load prediction quantitative model is used to predict the load brought by different scheduling ratios. Finally, according to the write traffic prediction value of the N+1 new cycle, call the capacity increment prediction quantitative model and the load prediction quantitative model, generate the target traffic scheduling strategy of the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increment meet the balance index. In this technical solution, the write traffic forecast value of the N+1 new cycle is first predicted based on the write traffic corresponding to each historical cycle, and then the capacity increment prediction quantification model and the load prediction quantification model are used to predict the capacity increment and load of the resource pool brought about by different scheduling ratios. Then, according to the impact of different scheduling ratios on capacity and load balance, the target traffic scheduling strategy for the N+1 new cycle is generated, and the capacity and load of multiple storage pools are quickly balanced through the target traffic scheduling scheme, thereby improving the scheduling efficiency of the distributed storage cluster.

[0075] Figure 4 The flow chart of the distributed storage cluster traffic scheduling method provided by the embodiment of the present disclosure Figure 2 In the embodiment of the present disclosure, the method of calling the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1 new cycle in step S205 to generate the target traffic scheduling strategy of the N+1 new cycle is described in detail. Figure 4 As shown, the method includes:

[0076] S401. Determine multiple initial traffic scheduling schemes, where the initial traffic scheduling schemes include initial write traffic proportions of each storage pool in the N+1th cycle.

[0077] In the embodiment of the present disclosure, there is no specific limitation on the number of multiple initial traffic scheduling schemes.

[0078] Optionally, the initial write traffic proportions of various storage pools in the N+1th cycle vary greatly in different initial traffic scheduling schemes.

[0079] In some embodiments, there is at least one storage pool whose initial write traffic share difference in multiple initial traffic scheduling schemes is greater than a preset share threshold. In the embodiments of the present disclosure, the value of the preset share threshold is not specifically limited. Exemplarily, the preset share threshold may be 10%, 20%, 30%, etc.

[0080] Exemplarily, the distributed storage cluster includes storage pool A, storage pool B and storage pool C. The number of multiple initial traffic scheduling schemes is 2: the initial write traffic proportion of each storage pool in the N+1 cycle in the initial traffic scheduling scheme 1 is: storage pool A (10%), storage pool B (20%), storage pool C (70%). The initial write traffic proportion of each storage pool in the N+1 cycle in the initial traffic scheduling scheme 2 is: storage pool A (50%), storage pool B (20%), storage pool C (30%). The preset proportion threshold can be 30%. At this time, the difference between the initial write traffic proportions of storage pool A in the initial traffic scheduling scheme 1 and the initial traffic scheduling scheme 2 (40%) is greater than the preset proportion threshold (30%).

[0081] S402. For each initial traffic scheduling scheme, a simulated annealing model is used to call a capacity increment prediction quantization model and a load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a local traffic scheduling scheme for the N+1 new cycle, wherein the local scheduling ratio indicated by the local traffic scheduling scheme is used to make the load and capacity increment of each storage pool meet the local balance index.

[0082] In some embodiments, this step may include the following steps (1) to (3):

[0083] (1) For each initial traffic scheduling scheme, the simulated annealing model is used to adjust the initial write traffic ratio of each storage pool in the initial traffic scheduling scheme in the N+1th cycle, and multiple simulated write traffic ratios of each storage pool in the N+1th cycle are obtained.

[0084] In this step, the adjustment parameter of each initial write traffic ratio is not specifically limited. For example, the adjustment parameter is 1%, 2%, 3%, etc. Among them, the number of multiple simulated write traffic ratios is a preset number. For example, 10, 20, 30, etc.

[0085] Exemplarily, the distributed storage cluster includes storage pool A and storage pool B. The initial write traffic ratio is: storage pool A (70%), storage pool B (30%), the adjustment parameter is 1%, and the preset number is 10. At this time, multiple simulated write traffic ratios are: storage pool A (71%), storage pool B (29%); storage pool A (72%), storage pool B (28%); ...; storage storage pool A (80%), storage pool B (20%).

[0086] (2) For each simulated write traffic ratio, based on the simulated write traffic ratio and the write traffic prediction value, the capacity increment prediction quantitative model is called to predict the capacity increment of each storage pool in the N+1th cycle. Also, based on the simulated write traffic ratio and the write traffic prediction value, the load prediction quantitative model is called to predict the load of each storage pool in the N+1th cycle.

[0087] Exemplarily, the distributed storage cluster includes storage pool A and storage pool B. The predicted value of the write traffic of the N+1th new cycle is 10 GB. The simulated write traffic ratio is: storage pool A (70%), storage pool B (30%).

[0088] At this time, based on the simulated write traffic ratio and the predicted write traffic value, the capacity increment prediction quantitative model is called to predict that the capacity increment of storage pool A in the N+1th cycle is 7GB, and the capacity increment of storage pool B in the N+1th cycle is 3GB.

[0089] Based on the simulated write traffic ratio and the predicted write traffic value, the load prediction quantization model is called to predict that the load of storage pool A in the N+1th cycle is: 70 read and write requests per second, and the load of storage pool B in the N+1th cycle is: 30 read and write requests per second.

[0090] (3) Determine a first target write traffic ratio from the multiple simulated write traffic ratios, where the first target write traffic ratio is the simulated write traffic ratio corresponding to the capacity increment and load of each storage pool in the N+1th cycle when reaching a local equilibrium state.

[0091] In the embodiment of the present disclosure, the capacity increment and load of each storage pool reaching a local equilibrium state means: the capacity difference of each storage pool is minimal, and the load difference of each storage pool is minimal. Optionally, the capacity difference of each storage pool is minimal means: the stock capacity difference between the storage pools is minimal, or the writable capacity difference between the storage pools is minimal.

[0092] Alternatively, if Figure 3 As shown, the smallest capacity difference among the storage pools indicates that the difference in stock capacity between the storage pools is smallest. In this case, the stock capacity of the storage pool in the N+1th cycle is equal to the sum of the historical stock capacity and the capacity increment in the N+1th cycle. At this time, for each storage pool, the capacity increment of the storage pool is determined by step (2), and the stock capacity of the storage pool can be further determined based on the sum of the historical stock capacity and the capacity increment.

[0093] It should be noted that the values ​​of the multiple simulated write traffic ratios are obtained by adjusting the initial write traffic ratio, so the values ​​of the multiple simulated write traffic ratios are close to the initial write traffic ratios. In this case, a local optimal balance state can be determined by adjusting the multiple simulated write traffic ratios.

[0094] S403: Select a local traffic scheduling scheme with the highest local balance index from the local traffic scheduling schemes corresponding to the multiple initial traffic scheduling schemes as the target traffic scheduling strategy.

[0095] In the embodiment of the present disclosure, the local balance index includes: the stock capacity difference between each storage pool and / or the load difference between each storage pool. It should be noted that the smaller the stock capacity difference and / or the load difference between each storage pool, the higher the local balance index.

[0096] In some embodiments, this step may include: selecting, from the local optimal traffic scheduling schemes corresponding to multiple initial traffic scheduling schemes, a local traffic scheduling scheme with the smallest stock capacity difference and the smallest load difference between storage pools as the target traffic scheduling strategy.

[0097] Figure 5 A schematic diagram of the structure of a traffic scheduling device for a distributed storage cluster provided in an embodiment of the present disclosure, such as Figure 5 As shown, the traffic scheduling device of the distributed storage cluster includes:

[0098] An acquisition unit 501 is used to acquire the write traffic corresponding to each historical period in N historical periods of a distributed storage cluster; the distributed storage cluster provides storage services through multiple storage pools;

[0099] The acquisition unit 501 is further used to obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical period;

[0100] A prediction unit 502 is used to predict a write flow prediction value of the N+1th new cycle according to the write flow corresponding to each historical cycle;

[0101] The model generation unit 503 is used to generate a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios;

[0102] The scheduling unit 504 is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, generate a target traffic scheduling strategy for the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increments meet the balance index.

[0103] According to one or more embodiments of the present disclosure, the prediction unit 502 predicts the write traffic prediction value of the N+1th new cycle based on the write traffic corresponding to each historical cycle, including: predicting the write traffic prediction value of the N+1th new cycle through a trend prediction model based on the write traffic corresponding to each historical cycle; wherein the trend prediction model includes a prophet model or a long short-term memory LSTM model.

[0104] According to one or more embodiments of the present disclosure, the scheduling unit 504 calls the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle, specifically including: using a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle.

[0105] According to one or more embodiments of the present disclosure, the scheduling unit 504 uses a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, and generates a target traffic scheduling strategy for the N+1 new cycle, specifically including: determining multiple initial traffic scheduling schemes, the initial traffic scheduling scheme includes the initial write traffic proportion of each storage pool in the N+1th cycle; for each initial traffic scheduling scheme, using a simulated annealing model, calling the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, and generating a local traffic scheduling scheme for the N+1 new cycle, wherein the local scheduling ratio indicated by the local traffic scheduling scheme is used to make the load and capacity increment of each storage pool meet the local balance index; from the local traffic scheduling schemes corresponding to the multiple initial traffic scheduling schemes, selecting the local traffic scheduling scheme with the highest local balance index as the target traffic scheduling strategy.

[0106] According to one or more embodiments of the present disclosure, the scheduling unit 504, for each initial traffic scheduling scheme, uses a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, and generates a local traffic scheduling scheme for the N+1 new cycle, specifically including: for each initial traffic scheduling scheme, using a simulated annealing model to adjust the initial write traffic proportion of each storage pool in the initial traffic scheduling scheme in the N+1th cycle, and obtain multiple simulated write traffic proportions of each storage pool in the N+1th cycle; for each simulated write flow According to the simulated write traffic ratio and the predicted write traffic value of the N+1th new cycle, the capacity increment prediction quantization model is called to predict the capacity increment of each storage pool in the N+1th cycle, and according to the simulated write traffic ratio and the predicted write traffic value of the N+1th new cycle, the load prediction quantization model is called to predict the load of each storage pool in the N+1th cycle; a first target write traffic ratio is determined from multiple simulated write traffic ratios, wherein the first target write traffic ratio is the simulated write traffic ratio corresponding to the capacity increment and load of each storage pool in the N+1th cycle when they reach a local equilibrium state.

[0107] According to one or more embodiments of the present disclosure, the model generation unit 503 generates a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period, the capacity increment and the load, specifically including: generating a capacity increment prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period and the capacity increment; and generating a load prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period and the load.

[0108] According to one or more embodiments of the present disclosure, the load includes one or more of a read request load, a write request load, and a query request load.

[0109] refer to Figure 6 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure, and the electronic device 600 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0110] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0111] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0112] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0113] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0114] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0115] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0116] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0119] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0120] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] In a first aspect, according to one or more embodiments of the present disclosure, a traffic scheduling method for a distributed storage cluster is provided, comprising: obtaining the write traffic corresponding to each historical cycle of the distributed storage cluster in N historical cycles; the distributed storage cluster provides storage services through multiple storage pools; obtaining the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical cycle; predicting the write traffic prediction value of the N+1th new cycle according to the write traffic corresponding to each historical cycle; generating a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical cycle, the write traffic proportion, capacity increment and load corresponding to each storage pool in each cycle; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios; calling the capacity increment prediction quantitative model and the load prediction quantitative model according to the write traffic prediction value of the N+1th new cycle, generating a target traffic scheduling strategy for the N+1 new cycle, and controlling the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increment meet the balance index.

[0122] According to one or more embodiments of the present disclosure, predicting the write traffic prediction value of the N+1th new cycle based on the write traffic corresponding to each historical cycle includes: predicting the write traffic prediction value of the N+1th new cycle through a trend prediction model based on the write traffic corresponding to each historical cycle; wherein the trend prediction model includes a prophet model or a long short-term memory (LSTM) model.

[0123] According to one or more embodiments of the present disclosure, calling the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle includes: using a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle.

[0124] According to one or more embodiments of the present disclosure, the use of a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle includes: determining multiple initial traffic scheduling schemes, the initial traffic scheduling scheme including the initial write traffic proportion of each storage pool in the N+1th cycle; for each initial traffic scheduling scheme, using a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a local traffic scheduling scheme for the N+1 new cycle, wherein the local scheduling ratio indicated by the local traffic scheduling scheme is used to make the load and capacity increment of each storage pool meet the local balance index; from the local traffic scheduling schemes corresponding to the multiple initial traffic scheduling schemes, selecting the local traffic scheduling scheme with the highest local balance index as the target traffic scheduling strategy.

[0125] According to one or more embodiments of the present disclosure, for each initial traffic scheduling scheme, the simulated annealing model is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1 new cycle to generate a local traffic scheduling scheme for the N+1 new cycle, including: for each initial traffic scheduling scheme, the simulated annealing model is used to adjust the initial write traffic proportion of each storage pool in the initial traffic scheduling scheme in the N+1 cycle to obtain multiple simulated write traffic proportions of each storage pool in the N+1 cycle; for each simulated write traffic proportion, based on According to the simulated write traffic ratio and the predicted write traffic value of the N+1th new cycle, the capacity increment prediction quantitative model is called to predict the capacity increment of each storage pool in the N+1th cycle, and, according to the simulated write traffic ratio and the predicted write traffic value of the N+1th new cycle, the load prediction quantitative model is called to predict the load of each storage pool in the N+1th cycle; a first target write traffic ratio is determined from multiple simulated write traffic ratios, wherein the first target write traffic ratio is the simulated write traffic ratio corresponding to the capacity increment and load of each storage pool in the N+1th cycle when they reach a local equilibrium state.

[0126] According to one or more embodiments of the present disclosure, the method of generating a capacity increment prediction quantitative model and a load prediction quantitative model based on the write traffic corresponding to each historical period, the write traffic ratio corresponding to each storage pool in each period, the capacity increment and the load includes: generating a capacity increment prediction quantitative model based on the write traffic corresponding to each historical period, the write traffic ratio corresponding to each storage pool in each period and the capacity increment; and generating a load prediction quantitative model based on the write traffic corresponding to each historical period, the write traffic ratio corresponding to each storage pool in each period and the load.

[0127] According to one or more embodiments of the present disclosure, the load includes one or more of a read request load, a write request load, and a query request load.

[0128] In a second aspect, according to one or more embodiments of the present disclosure, a traffic scheduling device for a distributed storage cluster is provided, including:

[0129] An acquisition unit, used to acquire the write traffic corresponding to each historical period of the distributed storage cluster in N historical periods; the distributed storage cluster provides storage services through multiple storage pools;

[0130] The acquisition unit is further used to obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical period;

[0131] A prediction unit, used to predict a write traffic prediction value of the N+1th new cycle according to the write traffic corresponding to each historical cycle;

[0132] A model generation unit is used to generate a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios;

[0133] The scheduling unit is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, generate a target traffic scheduling strategy for the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy, so that the load and capacity increments meet the balance index.

[0134] According to one or more embodiments of the present disclosure, the prediction unit predicts the write traffic prediction value of the N+1th new cycle based on the write traffic corresponding to each historical cycle, including: predicting the write traffic prediction value of the N+1th new cycle through a trend prediction model based on the write traffic corresponding to each historical cycle; wherein the trend prediction model includes a prophet model or a long short-term memory (LSTM) model.

[0135] According to one or more embodiments of the present disclosure, the scheduling unit calls the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle, which specifically includes: using a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle.

[0136] According to one or more embodiments of the present disclosure, the scheduling unit uses a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, and generates a target traffic scheduling strategy for the N+1 new cycle, which specifically includes: determining multiple initial traffic scheduling schemes, the initial traffic scheduling scheme includes the initial write traffic proportion of each storage pool in the N+1th cycle; for each initial traffic scheduling scheme, using a simulated annealing model, calling the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, and generating a local traffic scheduling scheme for the N+1 new cycle, wherein the local scheduling ratio indicated by the local traffic scheduling scheme is used to make the load and capacity increment of each storage pool meet the local balance index; from the local traffic scheduling schemes corresponding to the multiple initial traffic scheduling schemes, selecting the local traffic scheduling scheme with the highest local balance index as the target traffic scheduling strategy.

[0137] According to one or more embodiments of the present disclosure, the scheduling unit, for each initial traffic scheduling scheme, uses a simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, and generates a local traffic scheduling scheme for the N+1 new cycle, specifically including: for each initial traffic scheduling scheme, using a simulated annealing model to adjust the initial write traffic proportion of each storage pool in the initial traffic scheduling scheme in the N+1th cycle, and obtain multiple simulated write traffic proportions of each storage pool in the N+1th cycle; for each simulated write traffic According to the simulated write traffic ratio and the predicted write traffic value of the N+1th new cycle, the capacity increment prediction quantization model is called to predict the capacity increment of each storage pool in the N+1th cycle, and according to the simulated write traffic ratio and the predicted write traffic value of the N+1th new cycle, the load prediction quantization model is called to predict the load of each storage pool in the N+1th cycle; a first target write traffic ratio is determined from multiple simulated write traffic ratios, wherein the first target write traffic ratio is the simulated write traffic ratio corresponding to the capacity increment and load of each storage pool in the N+1th cycle when they reach a local equilibrium state.

[0138] According to one or more embodiments of the present disclosure, the model generation unit generates a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period, the capacity increment and the load, specifically including: generating a capacity increment prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period and the capacity increment; and generating a load prediction quantitative model according to the write traffic corresponding to each historical period, the write traffic proportion corresponding to each storage pool in each period and the load.

[0139] According to one or more embodiments of the present disclosure, the load includes one or more of a read request load, a write request load, and a query request load.

[0140] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0141] The memory stores computer-executable instructions;

[0142] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the traffic scheduling method for the distributed storage cluster as described in the first aspect and various possible designs of the first aspect.

[0143] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the traffic scheduling method of the distributed storage cluster as described in the first aspect and various possible designs of the first aspect is implemented.

[0144] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the traffic scheduling method for a distributed storage cluster as described in the first aspect and various possible designs of the first aspect.

[0145] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0146] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0147] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A traffic scheduling method for a distributed storage cluster, characterized in that: The method comprises: Obtaining the write traffic corresponding to each historical period of a distributed storage cluster in N historical periods; the distributed storage cluster provides storage services through multiple storage pools; Obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical period; According to the write traffic corresponding to each historical cycle, predict the write traffic forecast value of the N+1 new cycle; Generate a capacity increment prediction quantitative model and a load prediction quantitative model based on the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios; The capacity increment prediction quantization model and the load prediction quantization model are called according to the write traffic prediction value of the N+1th new cycle to generate the target traffic scheduling strategy of the N+1 new cycle, and the traffic scheduling of each resource pool is controlled according to the target scheduling ratio indicated by the target traffic scheduling strategy.

2. The method according to claim 1, characterized in that The predicting of the write traffic prediction value of the N+1th new cycle according to the write traffic corresponding to each historical cycle includes: According to the write traffic corresponding to each historical cycle, the write traffic forecast value of the N+1 new cycle is predicted through the trend prediction model; Wherein, the trend prediction model includes a prophet model or a long short-term memory (LSTM) model.

3. The method according to claim 1, characterized in that The calling the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate the target traffic scheduling strategy of the N+1 new cycle includes: The simulated annealing model is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a target traffic scheduling strategy for the N+1 new cycle.

4. The method according to claim 3, characterized in that The using of the simulated annealing model to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate the target traffic scheduling strategy of the N+1 new cycle includes: Determine a plurality of initial traffic scheduling schemes, wherein the initial traffic scheduling schemes include initial write traffic proportions of each storage pool in the N+1th cycle; For each initial traffic scheduling scheme, the capacity increment prediction quantization model and the load prediction quantization model are called according to the write traffic prediction value of the N+1th new cycle by using a simulated annealing model to generate a local traffic scheduling scheme for the N+1th new cycle, wherein the local scheduling ratio indicated by the local traffic scheduling scheme is used to make the load and capacity increment of each storage pool meet the local balance index; From the local traffic scheduling schemes corresponding to the multiple initial traffic scheduling schemes, a local traffic scheduling scheme with the highest local balance index is selected as the target traffic scheduling strategy.

5. The method according to claim 4, characterized in that For each initial traffic scheduling scheme, the simulated annealing model is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle to generate a local traffic scheduling scheme of the N+1th new cycle, including: For each initial traffic scheduling scheme, the simulated annealing model is used to adjust the initial write traffic proportion of each storage pool in the initial traffic scheduling scheme in the N+1th cycle to obtain multiple simulated write traffic proportions of each storage pool in the N+1th cycle; For each simulated write traffic proportion, according to the simulated write traffic proportion and the write traffic prediction value of the N+1th new cycle, the capacity increment prediction quantitative model is called to predict the capacity increment of each storage pool in the N+1th cycle, and, according to the simulated write traffic proportion and the write traffic prediction value of the N+1th new cycle, the load prediction quantitative model is called to predict the load of each storage pool in the N+1th cycle; A first target write traffic ratio is determined from multiple simulated write traffic ratios, wherein the first target write traffic ratio is a simulated write traffic ratio corresponding to when the capacity increment and load of each storage pool in the N+1th cycle reach a local equilibrium state.

6. The method according to claim 1, characterized in that The method of generating a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load includes: Generate a quantitative model for predicting capacity increment based on the write traffic corresponding to each historical period, the write traffic proportion and capacity increment of each storage pool in each period; and A load prediction quantitative model is generated based on the write traffic corresponding to each historical period, the write traffic proportion and load corresponding to each storage pool in each period.

7. The method according to claim 6, characterized in that The load includes one or more of a read request load, a write request load, and a query request load.

8. A traffic scheduling device for a distributed storage cluster, characterized in that: include: An acquisition unit, used to acquire the write traffic corresponding to each historical period of the distributed storage cluster in N historical periods; The distributed storage cluster provides storage services through multiple storage pools; The acquisition unit is further used to obtain the write traffic proportion, capacity increment and load corresponding to each storage pool in the distributed storage cluster in each historical period; A prediction unit, used to predict a write traffic prediction value of the N+1th new cycle according to the write traffic corresponding to each historical cycle; A model generation unit is used to generate a capacity increment prediction quantitative model and a load prediction quantitative model according to the write traffic corresponding to each historical cycle, the write traffic proportion corresponding to each storage pool in each cycle, the capacity increment and the load; wherein the capacity increment prediction quantitative model is used to predict the capacity increment of the resource pool brought about by different scheduling ratios; the load prediction quantitative model is used to predict the load brought about by different scheduling ratios; The scheduling unit is used to call the capacity increment prediction quantization model and the load prediction quantization model according to the write traffic prediction value of the N+1th new cycle, generate a target traffic scheduling strategy for the N+1 new cycle, and control the traffic scheduling of each resource pool according to the target scheduling ratio indicated by the target traffic scheduling strategy.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the traffic scheduling method for the distributed storage cluster as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the traffic scheduling method for the distributed storage cluster according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the traffic scheduling method for a distributed storage cluster as described in any one of claims 1 to 7 is implemented.

Images