A data scheduling management method to maintain balanced computing power of multiple clouds
By calculating the task allocation coefficient and abnormal identification of the cloud platform, data calculation tasks are reassigned, and the problem of failure to schedule according to the operating status of the cloud platform in the existing technology is solved, and the consistency and load balancing of the cloud platform data processing time are achieved.
Patent Information
- Application Number
- CN202411295741.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-18
AI Technical Summary
The existing data scheduling technology fails to schedule in real time according to the operating conditions of the cloud platform, resulting in different data processing volumes of different cloud platforms, and there may be unreasonable situations, affecting the efficiency of data processing and resource consumption.
By obtaining the maximum computing power and current computing power of the cloud platform, calculating the task allocation coefficient, decomposing the data calculation tasks, and classifying and redistributing tasks according to the operation of the cloud platform, ensuring that the processing time of tasks on each cloud platform is consistent. Adjust the allocation strategy to deal with abnormal situations.
It realizes task allocation based on the remaining computing power of different cloud platforms, ensures the time consistency of each cloud platform during data processing, and makes appropriate adjustments when abnormalities occur, ensuring the rationality of data scheduling and load balancing.
Smart Images

Figure CN118819865B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cloud computing technology, and in particular to a data scheduling management method for maintaining a balanced computing power of multiple clouds. Background Art
[0002] With the rapid development of information technology, data has become an important resource that cannot be ignored in modern society. However, the rapid growth of data has also brought a series of management and scheduling challenges. Data scheduling technology refers to the reasonable scheduling and allocation of data processing tasks to achieve efficient execution of tasks. Data scheduling technology can help improve the efficiency of data processing and reduce resource consumption.
[0003] Existing scheduling technology uses multi-task parallel processing and uniform load to evenly process data, but it does not take into account the operating conditions of the cloud platform used for data processing. Real-time data scheduling is performed based on the operating conditions of the cloud platform. Different cloud platforms process different amounts of data and have different abnormal situations, which may lead to unreasonable data scheduling. Summary of the invention
[0004] In order to solve the above technical problems, a data scheduling management method for maintaining balanced computing power of multiple clouds is provided. This technical solution solves the problem that the existing scheduling technology proposed in the above background technology uses multi-task parallel processing and uniform load to evenly process data, but it does not take into account the operating status of the cloud platform used for data processing. Real-time scheduling of data is performed according to the operating status of the cloud platform. Different cloud platforms process different amounts of data and different abnormal situations, which leads to the problem that the data scheduling adopted may be unreasonable.
[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:
[0006] A data scheduling management method for maintaining a balanced computing power of multiple clouds, comprising:
[0007] Obtain at least one cloud platform for computing, and based on historical data, obtain the maximum computing power of the cloud platform as the characteristic computing power, where the characteristic computing power corresponds to the cloud platform;
[0008] Obtain the current computing power used by the cloud platform and calculate the first task allocation coefficient of the cloud platform;
[0009] receiving at least one data computing task, and decomposing the data computing task according to the first task allocation coefficient to obtain a sequence to be calculated, wherein the sequence to be calculated is composed of at least one data computing subtask, and the data computing subtask corresponds to the first task allocation coefficient;
[0010] Allocate the data computing subtask to the cloud platform corresponding to the paired first task allocation coefficient;
[0011] Perform real-time monitoring on the operation status of the cloud platform, and classify the cloud platform based on the monitoring results to obtain at least one abnormal cloud platform and a normal cloud platform;
[0012] Identify the operation status of abnormal cloud platforms and obtain local fault cloud platforms and global fault cloud platforms;
[0013] Re-obtain the maximum computing power of the partially faulty cloud platform, and recalculate the second task allocation coefficients of the normal cloud platform and the partially faulty cloud platform;
[0014] According to the second task allocation coefficient, the data calculation subtask of the global fault cloud platform is allocated to the normal cloud platform and the local fault cloud platform;
[0015] According to the second task allocation coefficient, the data computing subtask of the local fault cloud platform is partially allocated to the normal cloud platform;
[0016] The newly acquired data calculation tasks are allocated according to the second task allocation coefficient.
[0017] Preferably, the step of obtaining the current computing power used by the cloud platform and calculating the first task allocation coefficient of the cloud platform comprises the following steps:
[0018] The maximum computing power of the cloud platform minus the current computing power of the cloud platform is used to obtain the remaining computing power of the cloud platform;
[0019] Add the remaining computing power of at least one cloud platform to get the comprehensive computing power of the cloud platform;
[0020] The remaining computing power of the cloud platform is divided by the comprehensive computing power of the cloud platform to obtain the first task allocation coefficient of the cloud platform.
[0021] Preferably, decomposing the data computing task according to the first task allocation coefficient comprises the following steps:
[0022] The data volume of the data calculation task is multiplied by the first task allocation coefficient to obtain the subtask data volume;
[0023] The data computing task is divided according to the data volume of the subtasks to obtain at least one data computing subtask, and the data volume of the subtasks is respectively consistent with the data volume of one of the data computing subtasks;
[0024] The data computing subtask is paired with the subtask data volume, the data volume of the data computing subtask is consistent with the paired subtask data volume, and the first task allocation coefficients corresponding to the data volumes of the data computing subtask and the paired subtask are corresponding.
[0025] Preferably, the classifying of the cloud platforms to obtain at least one abnormal cloud platform and a normal cloud platform comprises the following steps:
[0026] Based on historical data, the processing speed of the cloud platform's current computing power for processing data under normal cloud platform conditions is obtained as the benchmark speed;
[0027] Monitor the actual processing speed of the cloud platform's current computing power to process data and obtain the actual speed;
[0028] When the actual speed is less than the reference speed, the cloud platform is regarded as an abnormal cloud platform. When the actual speed is consistent with the reference speed, the cloud platform is regarded as a normal cloud platform.
[0029] Preferably, the identifying the operation status of the abnormal cloud platform to obtain the local fault cloud platform and the global fault cloud platform comprises the following steps:
[0030] When the actual speed of the abnormal cloud platform is 0, the abnormal cloud platform is regarded as the global fault cloud platform;
[0031] When the actual speed of the abnormal cloud platform is not 0, the abnormal cloud platform is regarded as a local fault cloud platform.
[0032] Preferably, the re-obtaining the maximum computing power of the partially faulty cloud platform and recalculating the second task allocation coefficients of the normal cloud platform and the partially faulty cloud platform comprises the following steps:
[0033] The maximum computing power of the local fault cloud platform minus the current computing power of the local fault cloud platform is used to obtain the remaining computing power of the local fault cloud platform;
[0034] The maximum computing power of the normal cloud platform minus the current computing power of the normal cloud platform is used to obtain the remaining computing power of the normal cloud platform;
[0035] The remaining computing power of at least one partially faulty cloud platform and the remaining computing power of a normal cloud platform are added together to obtain the comprehensive updated computing power of the cloud platform.
[0036] The remaining computing power of the local fault cloud platform is divided by the comprehensive update computing power of the cloud platform to obtain the second task allocation coefficient of the local fault cloud platform;
[0037] The remaining computing power of the normal cloud platform is divided by the comprehensive update computing power of the cloud platform to obtain the second task allocation coefficient of the normal cloud platform.
[0038] Preferably, allocating the data calculation subtask of the global fault cloud platform to the normal cloud platform and the local fault cloud platform according to the second task allocation coefficient comprises the following steps:
[0039] Aggregate the data calculation subtasks of the global fault cloud platform to obtain the total fault data task;
[0040] The data volume of the total fault data task is multiplied by the second task allocation coefficient to obtain the fault allocation data volume;
[0041] According to the fault allocation data volume, the total fault data task is divided to obtain at least one data computing and allocation task, and the fault allocation data volume is respectively consistent with the data volume of one of the data computing and allocation tasks;
[0042] Pair the data computing deployment task with the fault allocation data volume, so that the data volume of the data computing deployment task is consistent with the paired fault allocation data volume;
[0043] The data computing allocation task is allocated to the local fault cloud platform or the normal cloud platform corresponding to the second task allocation coefficient corresponding to the paired fault allocation data volume.
[0044] Preferably, allocating the data computing subtask portion of the local fault cloud platform to the normal cloud platform according to the second task allocation coefficient comprises the following steps:
[0045] The first task allocation coefficient of the local fault cloud platform is subtracted from the second task allocation coefficient to obtain a third task allocation coefficient;
[0046] According to the ratio of the second task allocation coefficient to the third task allocation coefficient, the data computing subtask of the local fault cloud platform is divided into a first task and a second task, wherein the ratio of the data volume of the first task and the second task is equal to the ratio of the second task allocation coefficient to the third task allocation coefficient;
[0047] Summarize the second tasks of the local fault cloud platform to obtain the total data tasks to be allocated;
[0048] The data amount of the total data task to be allocated is multiplied by the second task allocation coefficient to obtain the data amount to be allocated;
[0049] According to the amount of data to be allocated, the total data task to be allocated is divided to obtain at least one transfer task, and the amount of data to be allocated is respectively consistent with the amount of data of one of the transfer tasks;
[0050] Pair the transfer task with the amount of data to be allocated, so that the amount of data in the transfer task is consistent with the paired amount of data to be allocated;
[0051] The transfer task is allocated to the normal cloud platform corresponding to the second task allocation coefficient corresponding to the paired amount of data to be allocated.
[0052] Preferably, allocating the newly acquired data calculation tasks according to the second task allocation coefficient comprises the following steps:
[0053] The data volume of the newly acquired data calculation task is multiplied by the second task allocation coefficient to obtain the sub-task data volume;
[0054] According to the data volume of the sub-tasks, the newly acquired data computing task is divided to obtain at least one data computing sub-task, and the data volume of the sub-tasks is respectively consistent with the data volume of one of the data computing sub-tasks;
[0055] The data computing subtask is paired with the subtask data volume, the data volume of the data computing subtask is consistent with the paired subtask data volume, and the second task allocation coefficients corresponding to the data volumes of the data computing subtask and the paired subtask data volume correspond to each other;
[0056] The data computing subtask is allocated to the local fault cloud platform or the normal cloud platform corresponding to the corresponding second task allocation coefficient.
[0057] Compared with the prior art, the present invention has the following beneficial effects:
[0058] By decomposing the data computing tasks according to the first task allocation coefficient, identifying anomalies on the cloud platform, and reallocating the computing tasks based on the identification results, tasks can be allocated based on the remaining computing power of different cloud platforms, ensuring that the time spent on data processing on each cloud platform is roughly the same. At the same time, when an anomaly occurs on the cloud platform, different task adjustment allocation strategies are implemented based on the different anomalies. By adjusting the allocation strategy, the rationality of data scheduling is guaranteed, thereby ensuring that the processing load of the cloud platform remains consistent. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 A flow chart of a data scheduling management method for maintaining a balanced computing power of multiple cloud computing systems according to the present invention;
[0060] Figure 2 A schematic diagram of a process of obtaining the current computing power used by a cloud platform and calculating the first task allocation coefficient of the cloud platform according to the present invention;
[0061] Figure 3 It is a flow chart of decomposing data calculation tasks according to the first task allocation coefficient of the present invention;
[0062] Figure 4 A schematic diagram of a process for classifying cloud platforms to obtain at least one abnormal cloud platform and a normal cloud platform according to the present invention;
[0063] Figure 5 The present invention identifies the operation status of an abnormal cloud platform and obtains a flow chart of a local fault cloud platform and a global fault cloud platform;
[0064] Figure 6A schematic diagram of a process of re-obtaining the maximum computing power of a partially faulty cloud platform and recalculating the second task allocation coefficients of a normal cloud platform and a partially faulty cloud platform according to the present invention;
[0065] Figure 7 It is a flow chart of allocating the data calculation subtask of the global fault cloud platform to the normal cloud platform and the local fault cloud platform according to the second task allocation coefficient of the present invention;
[0066] Figure 8 It is a flow chart of allocating part of the data calculation subtask of the local fault cloud platform to the normal cloud platform according to the second task allocation coefficient of the present invention;
[0067] Fig. 9 It is a schematic diagram of a flow chart of allocating newly acquired data calculation tasks according to the second task allocation coefficient of the present invention. DETAILED DESCRIPTION
[0068] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.
[0069] Reference Figure 1 As shown, a data scheduling management method for maintaining a balanced computing power of multiple clouds comprises:
[0070] Obtain at least one cloud platform for computing, and based on historical data, obtain the maximum computing power of the cloud platform as the characteristic computing power, where the characteristic computing power corresponds to the cloud platform;
[0071] Obtain the current computing power used by the cloud platform and calculate the first task allocation coefficient of the cloud platform;
[0072] receiving at least one data computing task, and decomposing the data computing task according to the first task allocation coefficient to obtain a sequence to be calculated, wherein the sequence to be calculated is composed of at least one data computing subtask, and the data computing subtask corresponds to the first task allocation coefficient;
[0073] Allocate the data computing subtask to the cloud platform corresponding to the paired first task allocation coefficient;
[0074] Perform real-time monitoring on the operation status of the cloud platform, and classify the cloud platform based on the monitoring results to obtain at least one abnormal cloud platform and a normal cloud platform;
[0075] Identify the operation status of abnormal cloud platforms and obtain local fault cloud platforms and global fault cloud platforms;
[0076] Re-obtain the maximum computing power of the partially faulty cloud platform, and recalculate the second task allocation coefficients of the normal cloud platform and the partially faulty cloud platform;
[0077] According to the second task allocation coefficient, the data calculation subtask of the global fault cloud platform is allocated to the normal cloud platform and the local fault cloud platform;
[0078] According to the second task allocation coefficient, the data computing subtask of the local fault cloud platform is partially allocated to the normal cloud platform;
[0079] The newly acquired data calculation tasks are allocated according to the second task allocation coefficient.
[0080] In this solution, the remaining computing power of the cloud platform is estimated based on the maximum computing power of the cloud platform and the currently used computing power of the cloud platform, and then the data tasks are allocated according to the remaining computing power of the cloud platform, so as to ensure that the processing time of the newly added data processing tasks on each cloud platform remains consistent, and the increased load on the cloud platform is also consistent. At the same time, when an abnormality occurs on a cloud platform, the tasks of the cloud platform are reallocated, and the tasks of the cloud platform with the abnormality are deployed and processed, so as to ensure that the data processing of all cloud platforms can be balanced.
[0081] Reference Figure 2 As shown, obtaining the current computing power used by the cloud platform and calculating the first task allocation coefficient of the cloud platform includes the following steps:
[0082] The maximum computing power of the cloud platform minus the current computing power of the cloud platform is used to obtain the remaining computing power of the cloud platform;
[0083] Add the remaining computing power of at least one cloud platform to get the comprehensive computing power of the cloud platform;
[0084] The remaining computing power of the cloud platform is divided by the comprehensive computing power of the cloud platform to obtain the first task allocation coefficient of the cloud platform.
[0085] The first task allocation coefficient is used to determine the amount of data allocated to the cloud platform. The first task allocation coefficient is determined by the remaining computing power of the cloud platform, which can ensure that the amount of data allocated to the cloud platform is proportional to the remaining computing power of the cloud platform, so that the time spent by each cloud platform in processing data can be kept consistent, and the new load of the cloud platform can be kept relatively consistent.
[0086] Reference Figure 3 As shown, decomposing the data computing task according to the first task allocation coefficient includes the following steps:
[0087] The data volume of the data calculation task is multiplied by the first task allocation coefficient to obtain the subtask data volume;
[0088] The data computing task is divided according to the data volume of the subtasks to obtain at least one data computing subtask, and the data volume of the subtasks is respectively consistent with the data volume of one of the data computing subtasks;
[0089] The data computing subtask is paired with the subtask data volume, the data volume of the data computing subtask is consistent with the paired subtask data volume, and the first task allocation coefficients corresponding to the data volumes of the data computing subtask and the paired subtask are corresponding.
[0090] Each first task allocation coefficient corresponds to a unique cloud platform, and therefore, the data computing subtask calculated by the first task allocation coefficient is allocated to the cloud platform corresponding to the first task allocation coefficient.
[0091] Reference Figure 4 As shown, classifying the cloud platforms to obtain at least one abnormal cloud platform and a normal cloud platform includes the following steps:
[0092] Based on historical data, the processing speed of the cloud platform's current computing power for processing data under normal cloud platform conditions is obtained as the benchmark speed;
[0093] Monitor the actual processing speed of the cloud platform's current computing power to process data and obtain the actual speed;
[0094] When the actual speed is less than the reference speed, the cloud platform is regarded as an abnormal cloud platform. When the actual speed is consistent with the reference speed, the cloud platform is regarded as a normal cloud platform.
[0095] The cloud platform may have abnormalities during operation, but the abnormalities vary. It may be completely unusable or partially available. Therefore, in order to evenly distribute the load of data tasks, it is necessary to evaluate the abnormalities of the cloud platform and obtain its data processing capacity, and then recalculate the distribution coefficient and make overall distribution adjustments.
[0096] Reference Figure 5 As shown, identifying the operation status of the abnormal cloud platform and obtaining the local fault cloud platform and the global fault cloud platform includes the following steps:
[0097] When the actual speed of the abnormal cloud platform is 0, the abnormal cloud platform is regarded as the global fault cloud platform;
[0098] When the actual speed of the abnormal cloud platform is not 0, the abnormal cloud platform is regarded as a local fault cloud platform.
[0099] Reference Figure 6 As shown, re-obtaining the maximum computing power of the local fault cloud platform and recalculating the second task allocation coefficients of the normal cloud platform and the local fault cloud platform includes the following steps:
[0100] The maximum computing power of the local fault cloud platform minus the current computing power of the local fault cloud platform is used to obtain the remaining computing power of the local fault cloud platform;
[0101] The maximum computing power of the normal cloud platform minus the current computing power of the normal cloud platform is used to obtain the remaining computing power of the normal cloud platform;
[0102] The remaining computing power of at least one partially faulty cloud platform and the remaining computing power of a normal cloud platform are added together to obtain the comprehensive updated computing power of the cloud platform.
[0103] The remaining computing power of the local fault cloud platform is divided by the comprehensive update computing power of the cloud platform to obtain the second task allocation coefficient of the local fault cloud platform;
[0104] The remaining computing power of the normal cloud platform is divided by the comprehensive update computing power of the cloud platform to obtain the second task allocation coefficient of the normal cloud platform.
[0105] Since the global fault cloud platform cannot process data, data allocation is only carried out in the normal cloud platform and the local fault cloud platform. However, since the maximum computing power of the local fault cloud platform is affected by the abnormality and changes, it is necessary to recalculate the allocation coefficients of the normal cloud platform and the local fault cloud platform to obtain the second task allocation coefficient.
[0106] Reference Figure 7 As shown, according to the second task allocation coefficient, allocating the data computing subtask of the global fault cloud platform to the normal cloud platform and the local fault cloud platform includes the following steps:
[0107] Aggregate the data calculation subtasks of the global fault cloud platform to obtain the total fault data task;
[0108] The data volume of the total fault data task is multiplied by the second task allocation coefficient to obtain the fault allocation data volume;
[0109] According to the fault allocation data volume, the total fault data task is divided to obtain at least one data computing and allocation task, and the fault allocation data volume is respectively consistent with the data volume of one of the data computing and allocation tasks;
[0110] Pair the data computing deployment task with the fault allocation data volume, so that the data volume of the data computing deployment task is consistent with the paired fault allocation data volume;
[0111] The data computing allocation task is allocated to the local fault cloud platform or the normal cloud platform corresponding to the second task allocation coefficient corresponding to the paired fault allocation data volume.
[0112] Since the global fault cloud platform cannot process data, the data calculation subtasks it processes need to be allocated to the normal cloud platform and the local fault cloud platform, otherwise it will affect the normal processing of the data. When allocating data, in order to ensure uniform load, it is necessary to allocate data tasks according to the second task allocation coefficient.
[0113] Reference Figure 8 As shown, according to the second task allocation coefficient, allocating part of the data computing subtask of the local fault cloud platform to the normal cloud platform includes the following steps:
[0114] The first task allocation coefficient of the local fault cloud platform is subtracted from the second task allocation coefficient to obtain a third task allocation coefficient;
[0115] According to the ratio of the second task allocation coefficient to the third task allocation coefficient, the data computing subtask of the local fault cloud platform is divided into a first task and a second task, wherein the ratio of the data volume of the first task and the second task is equal to the ratio of the second task allocation coefficient to the third task allocation coefficient;
[0116] Summarize the second tasks of the local fault cloud platform to obtain the total data tasks to be allocated;
[0117] The data amount of the total data task to be allocated is multiplied by the second task allocation coefficient to obtain the data amount to be allocated;
[0118] According to the amount of data to be allocated, the total data task to be allocated is divided to obtain at least one transfer task, and the amount of data to be allocated is respectively consistent with the amount of data of one of the transfer tasks;
[0119] Pair the transfer task with the amount of data to be allocated, so that the amount of data in the transfer task is consistent with the paired amount of data to be allocated;
[0120] The transfer task is allocated to the normal cloud platform corresponding to the second task allocation coefficient corresponding to the paired amount of data to be allocated.
[0121] Since the computing power of the local fault cloud platform is reduced, if the tasks it is currently processing are not adjusted, its load will be larger than that of the normal cloud platform. Therefore, in order to balance the load, the tasks of the local fault cloud platform are adjusted. During the adjustment, since the first task allocation coefficient and the second task allocation coefficient respectively evaluate the processing capacity of the local fault cloud platform before and after the abnormality, the tasks that exceed the load of the local fault cloud platform are determined according to the first task allocation coefficient and the second task allocation coefficient, and the tasks are summarized and evenly distributed to the normal cloud platform according to the processing capacity of the normal cloud platform. In this way, the existing tasks of the abnormal cloud platform can be allocated.
[0122] Reference Fig. 9As shown, allocating the newly acquired data calculation task according to the second task allocation coefficient includes the following steps:
[0123] The data volume of the newly acquired data calculation task is multiplied by the second task allocation coefficient to obtain the sub-task data volume;
[0124] According to the data volume of the sub-tasks, the newly acquired data computing task is divided to obtain at least one data computing sub-task, and the data volume of the sub-tasks is respectively consistent with the data volume of one of the data computing sub-tasks;
[0125] The data computing subtask is paired with the subtask data volume, the data volume of the data computing subtask is consistent with the paired subtask data volume, and the second task allocation coefficients corresponding to the data volumes of the data computing subtask and the paired subtask data volume correspond to each other;
[0126] The data computing subtask is allocated to the local fault cloud platform or the normal cloud platform corresponding to the corresponding second task allocation coefficient.
[0127] The newly acquired data computing tasks are allocated according to the second task allocation coefficient of the local fault cloud platform or the normal cloud platform. The allocation principle is consistent with the allocation principle according to the first task allocation coefficient of the cloud platform. The tasks are evenly distributed according to the remaining computing power of the local fault cloud platform or the normal cloud platform to ensure that the time spent on task processing is roughly the same.
[0128] Furthermore, the present solution also proposes a storage medium on which a computer-readable program is stored. When the computer-readable program is called, the above-mentioned data scheduling management method for maintaining balanced computing power of multiple clouds is executed.
[0129] It is understandable that the storage medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid state drive (SSD).
[0130] To sum up, the advantages of the present invention are: by decomposing the data computing tasks according to the first task allocation coefficient, identifying the anomalies of the cloud platform, and reallocating the computing tasks according to the identification results, the tasks can be allocated according to the remaining computing power of different cloud platforms, ensuring that the data processing time of each cloud platform is roughly the same when performing data processing. At the same time, when an abnormality occurs in the cloud platform, different task adjustment allocation strategies are performed according to the different abnormalities, and the rationality of data scheduling is guaranteed by adjusting the allocation strategy, thereby ensuring that the processing load of the cloud platform remains consistent.
[0131] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.
Claims
1. A data scheduling management method for maintaining a balanced computing power of multiple clouds, characterized in that: include: Obtain at least one cloud platform for computing, and based on historical data, obtain the maximum computing power of the cloud platform as the characteristic computing power, where the characteristic computing power corresponds to the cloud platform; Obtain the current computing power used by the cloud platform and calculate the first task allocation coefficient of the cloud platform; receiving at least one data computing task, and decomposing the data computing task according to the first task allocation coefficient to obtain a sequence to be calculated, wherein the sequence to be calculated is composed of at least one data computing subtask, and the data computing subtask corresponds to the first task allocation coefficient; Allocate the data computing subtask to the cloud platform corresponding to the paired first task allocation coefficient; Perform real-time monitoring on the operation status of the cloud platform, and classify the cloud platform based on the monitoring results to obtain at least one abnormal cloud platform and a normal cloud platform; Identify the operation status of abnormal cloud platforms and obtain local fault cloud platforms and global fault cloud platforms; Re-obtain the maximum computing power of the partially faulty cloud platform, and recalculate the second task allocation coefficients of the normal cloud platform and the partially faulty cloud platform; According to the second task allocation coefficient, the data calculation subtask of the global fault cloud platform is allocated to the normal cloud platform and the local fault cloud platform; According to the second task allocation coefficient, the data computing subtask of the local fault cloud platform is partially allocated to the normal cloud platform; Allocate the newly acquired data calculation tasks according to the second task allocation coefficient; The current computing power of the cloud platform is obtained, and the first task allocation coefficient of the cloud platform is calculated. The following steps are involved: The maximum computing power of the cloud platform minus the current computing power of the cloud platform is used to obtain the remaining computing power of the cloud platform; Add the remaining computing power of at least one cloud platform to get the comprehensive computing power of the cloud platform; The remaining computing power of the cloud platform is divided by the comprehensive computing power of the cloud platform to obtain the first task allocation coefficient of the cloud platform; Decomposing the data computing task according to the first task allocation coefficient comprises the following steps: The data volume of the data calculation task is multiplied by the first task allocation coefficient to obtain the subtask data volume; The data computing task is divided according to the data volume of the subtasks to obtain at least one data computing subtask, and the data volume of the subtasks is respectively consistent with the data volume of one of the data computing subtasks; The data computing subtask is paired with the subtask data volume, the data volume of the data computing subtask is consistent with the paired subtask data volume, and the first task allocation coefficients corresponding to the data volumes of the data computing subtask and the paired subtask are corresponding.
2. According to claim 1, a data scheduling management method for maintaining a balance in computing power among multiple clouds is characterized in that: The classifying of the cloud platforms to obtain at least one abnormal cloud platform and a normal cloud platform comprises the following steps: Based on historical data, the processing speed of the cloud platform's current computing power for processing data under normal cloud platform conditions is obtained as the benchmark speed; Monitor the actual processing speed of the cloud platform's current computing power to process data and obtain the actual speed; When the actual speed is less than the reference speed, the cloud platform is regarded as an abnormal cloud platform. When the actual speed is consistent with the reference speed, the cloud platform is regarded as a normal cloud platform.
3. The data scheduling management method for maintaining a balanced computing power of multiple clouds according to claim 2 is characterized in that: The step of identifying the operation status of the abnormal cloud platform and obtaining the local fault cloud platform and the global fault cloud platform includes the following steps: When the actual speed of the abnormal cloud platform is 0, the abnormal cloud platform is regarded as the global fault cloud platform; When the actual speed of the abnormal cloud platform is not 0, the abnormal cloud platform is regarded as a local fault cloud platform.
4. A data scheduling management method for maintaining a balance of computing power among multiple clouds according to claim 3, characterized in that: The method of re-obtaining the maximum computing power of the partially faulty cloud platform and re-calculating the second task allocation coefficients of the normal cloud platform and the partially faulty cloud platform comprises the following steps: The maximum computing power of the local fault cloud platform minus the current computing power of the local fault cloud platform is used to obtain the remaining computing power of the local fault cloud platform; The maximum computing power of the normal cloud platform minus the current computing power of the normal cloud platform is used to obtain the remaining computing power of the normal cloud platform; The remaining computing power of at least one partially faulty cloud platform and the remaining computing power of a normal cloud platform are added together to obtain the comprehensive updated computing power of the cloud platform. The remaining computing power of the local fault cloud platform is divided by the comprehensive update computing power of the cloud platform to obtain the second task allocation coefficient of the local fault cloud platform; The remaining computing power of the normal cloud platform is divided by the comprehensive update computing power of the cloud platform to obtain the second task allocation coefficient of the normal cloud platform.
5. A data scheduling management method for maintaining a balance of computing power among multiple clouds according to claim 4, characterized in that: The method of allocating the data computing subtask of the global fault cloud platform to the normal cloud platform and the local fault cloud platform according to the second task allocation coefficient includes the following steps: Aggregate the data calculation subtasks of the global fault cloud platform to obtain the total fault data task; The data volume of the total fault data task is multiplied by the second task allocation coefficient to obtain the fault allocation data volume; According to the fault allocation data volume, the total fault data task is divided to obtain at least one data computing and allocation task, and the fault allocation data volume is respectively consistent with the data volume of one of the data computing and allocation tasks; Pair the data computing deployment task with the fault allocation data volume, so that the data volume of the data computing deployment task is consistent with the paired fault allocation data volume; The data computing allocation task is allocated to the local fault cloud platform or the normal cloud platform corresponding to the second task allocation coefficient corresponding to the paired fault allocation data volume.
6. A data scheduling management method for maintaining a balance in computing power among multiple clouds according to claim 5, characterized in that: The method of allocating the data computing subtask portion of the local fault cloud platform to the normal cloud platform according to the second task allocation coefficient includes the following steps: The first task allocation coefficient of the local fault cloud platform is subtracted from the second task allocation coefficient to obtain a third task allocation coefficient; According to the ratio of the second task allocation coefficient to the third task allocation coefficient, the data computing subtask of the local fault cloud platform is divided into a first task and a second task, wherein the ratio of the data volume of the first task and the second task is equal to the ratio of the second task allocation coefficient to the third task allocation coefficient; Summarize the second tasks of the local fault cloud platform to obtain the total data tasks to be allocated; The data amount of the total data task to be allocated is multiplied by the second task allocation coefficient to obtain the data amount to be allocated; According to the amount of data to be allocated, the total data task to be allocated is divided to obtain at least one transfer task, and the amount of data to be allocated is respectively consistent with the amount of data of one of the transfer tasks; Pair the transfer task with the amount of data to be allocated, so that the amount of data in the transfer task is consistent with the paired amount of data to be allocated; The transfer task is allocated to the normal cloud platform corresponding to the second task allocation coefficient corresponding to the paired amount of data to be allocated.
7. A data scheduling management method for maintaining a balance of computing power among multiple clouds according to claim 6, characterized in that: The allocating of the newly acquired data computing tasks according to the second task allocation coefficient comprises the following steps: The data volume of the newly acquired data calculation task is multiplied by the second task allocation coefficient to obtain the sub-task data volume; According to the data volume of the sub-tasks, the newly acquired data computing task is divided to obtain at least one data computing sub-task, and the data volume of the sub-tasks is respectively consistent with the data volume of one of the data computing sub-tasks; The data computing subtask is paired with the subtask data volume, the data volume of the data computing subtask is consistent with the paired subtask data volume, and the second task allocation coefficients corresponding to the data volumes of the data computing subtask and the paired subtask data volume correspond to each other; The data computing subtask is allocated to the local fault cloud platform or the normal cloud platform corresponding to the corresponding second task allocation coefficient.
Citation Information
Patent Citations
Computing power network resource scheduling system and method
CN117331700A
Online service computing power optimization method and device based on cloud computing
CN117667602A
Dynamically generating solutions for updating plans and task allocation strategies
US20220048186A1
