Resource and task cooperative scheduling method, system and device based on computing power alliance

Abstract

The present application relates to a kind of resource and task collaborative scheduling method, system and equipment based on computing power alliance.The method comprises: obtaining the data computing information of all tasks in job.According to data computing information, the proportion weight of CPU and memory capacity in each task area is calculated respectively, and the dynamic contribution value is determined according to the proportion weight.According to the difference between the deadline and the path end time of the task, the dynamic contribution value, the task transmission time information and the network link attribute are evaluated.The total criticality of transmission path is calculated, and the task migration path of each task is obtained according to the total criticality and the difference evaluation result.The total load of each task area is determined by using load prediction algorithm according to the task migration path and historical experience data, and the resource storage and transmission of the task are collaboratively scheduled based on computing power alliance.The resource allocation efficiency of complex communication task under cross-regional condition can be improved by using the method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource and task collaborative scheduling of computing power alliance, and particularly relates to a resource and task collaborative scheduling method, system and device based on computing power alliance. BACKGROUND

[0002] The rapid progress of artificial intelligence methods has driven the emergence of transformative intelligent applications, including autonomous driving systems, precision medicine platforms, and large language models such as ChatGPT, Sora, and DeepSeek. According to IDC, the global data production is expected to reach 393.9 zettabytes by 2028. The rapid development of artificial intelligence urgently needs unified computing capacity scheduling and interconnection. Cloud computing provides on-demand computing services through a shared resource pool, but it faces limitations such as high latency, network congestion, and insufficient support for real-time applications. To address these issues, edge computing shifts computing resources from the cloud to the edge. Based on this trend, researchers explore collaborative computing in cloud-edge-end vertical architectures to support delay-sensitive and compute-intensive tasks. In addition, sky computing opens up new possibilities for cross-regional collaboration through workload migration across multiple clouds.

[0003] Task scheduling is a core challenge in distributed computing systems, traditionally focusing on computation offloading and resource allocation to optimize energy consumption, latency, throughput, reliability, and cost. Common methods include game theory, heuristic algorithms, and machine learning, but they usually limit task allocation to nearby edge nodes or cloud centers, ignoring large-scale computing collaboration. Some people propose a hybrid algorithm to minimize task processing delay while ensuring queue stability; some use a logic-based Benders decomposition method to reduce latency; and some develop a multi-agent actor-critic algorithm to improve Kubernetes throughput.

[0004] Cloud computing excels in large-scale data analysis and complex tasks, but it has high latency, network congestion, and limited real-time support capabilities. Edge computing achieves low latency by bringing computing closer to data sources, but it faces challenges due to limited capacity. Cloud-edge-end collaborative architecture focuses on vertical resource integration to improve efficiency and scalability, but lacks cross-layer and cross-domain horizontal collaboration to achieve unified resource scheduling. Traditional methods do not support cross-regional task migration and resource sharing, limiting their widespread application. SUMMARY

[0005] Therefore, it is necessary to provide a resource and task collaborative scheduling method, system and device based on computing power alliance to improve the resource allocation efficiency of complex communication tasks under cross-regional conditions.

[0006] A resource and task collaborative scheduling method based on computing power alliance, the method comprising:

[0007] Obtaining data calculation information of all tasks in a job.

[0008] According to the data calculation information, the proportion weight of CPU and memory capacity in each task area is calculated respectively, and the dynamic contribution value is determined according to the proportion weight.

[0009] According to the dynamic contribution value, task transmission time information and network link attribute, the difference between the deadline and path end time of the task is evaluated.

[0010] The total criticality of the transmission path is calculated, and the task migration path of each task is obtained according to the evaluation result of the total criticality and the difference.

[0011] According to the task migration path and historical experience data, the total load of each task area is determined by using a load prediction algorithm, and the resource storage and transmission of the task are collaboratively scheduled based on the computing power alliance.

[0012] A resource and task collaborative scheduling system based on computing power alliance, the system comprising:

[0013] An information acquisition module for obtaining data calculation information of all tasks in a job.

[0014] A contribution value calculation module for calculating the proportion weight of CPU and memory capacity in each task area according to the data calculation information, and determining the dynamic contribution value according to the proportion weight.

[0015] A time difference evaluation module for evaluating the difference between the deadline and path end time of the task according to the dynamic contribution value, task transmission time information and network link attribute.

[0016] A migration path acquisition module for calculating the total criticality of the transmission path, and obtaining the task migration path of each task according to the evaluation result of the total criticality and the difference.

[0017] A collaborative scheduling module for determining the total load of each task area by using a load prediction algorithm according to the task migration path and historical experience data, and collaboratively scheduling the resource storage and transmission of the task based on the computing power alliance.

[0018] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0019] Obtaining data calculation information of all tasks in a job.

[0020] According to the data computing information, the proportion weight of CPU and memory capacity in each task area is calculated, and the dynamic contribution value is determined according to the proportion weight.

[0021] According to the dynamic contribution value, the task transmission time information and the network link attribute, the difference between the deadline of the task and the end time of the path is evaluated.

[0022] The total criticality of the transmission path is calculated, and the task migration path of each task is obtained according to the evaluation result of the total criticality and the difference.

[0023] According to the task migration path and the historical experience data, the total load of each task area is determined by using the load prediction algorithm, and the resource storage and transmission of the task are cooperatively scheduled based on the total load.

[0024] The above-mentioned resource and task cooperative scheduling method, system and device based on the computing power alliance first calculates the proportion weight of CPU and memory capacity in the area, generates a dynamic contribution value in combination with the task attribute, establishes a unified priority quantization standard for tasks of different levels (cloud, edge and end), breaks the problem of fragmented scheduling logic of each layer in vertical resource integration, enables cross-layer tasks to participate in resource competition based on a unified value dimension, and lays a foundation for horizontal cooperation. Then, by evaluating the difference between the deadline of the task and the end time of the path, the migration path selection is optimized in combination with the total criticality of the transmission path, which not only guarantees the timeliness of the task, but also balances the network resource consumption through criticality quantization, so that the task can be flexibly migrated between different regions and different level nodes. At the same time, based on the load prediction algorithm of the migration path and the historical data, the total load of each area can be accurately calculated to avoid resource overload and provide a basis for dynamic allocation of cross-domain resources. Finally, relying on the cooperative scheduling mechanism of the computing power alliance, the scheme integrates the dispersed cloud, edge and end resources into a unified management framework, realizes on-demand allocation of resources through the dynamic contribution value, guarantees the cross-regional transmission efficiency through path optimization, realizes the elastic supply of resources through load prediction, forms a closed loop of task value quantization-path intelligent selection-load accurate regulation-resource global cooperation, effectively solves the pain points of lack of horizontal cooperation and not supporting cross-regional migration in traditional architecture, and significantly improves the resource allocation efficiency of cross-regional complex communication tasks. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A collaborative scheduling system framework in a layered computing power alliance in one embodiment;

[0026] Figure 2 A flowchart of a resource and task cooperative scheduling method based on a computing power alliance in one embodiment;

[0027] Figure 3 A diagram of the time interval and rate change point of the task arrival in one embodiment;

[0028] Figure 4 A structural block diagram of a resource and task collaborative scheduling system based on a computing power alliance in an embodiment is shown in the figure.

[0029] Figure 5 An internal structure diagram of a computer device in an embodiment is shown in the figure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0031] The resource and task collaborative scheduling method based on a computing power alliance provided by the present application can be applied in a system framework as shown in the figure. Figure 1 The scheduling system includes two levels, a resource layer: each CP node has multiple resources, and the router responsible for data forwarding simultaneously acts as a gateway. A control layer: the agent is responsible for unified resource management; the resource directory contains resource state information; the global scheduler integrates various optimization algorithms; and the policy executor is responsible for deploying the scheduling strategy. In the interaction process, each arriving task reports its resource requirements to the local agent, and the local agent is responsible for unified resource management within the region. The global scheduler decides whether to migrate and selects a specific migration path. Finally, a computing power alliance integrating computing and network resource selection is established, aiming to maximize the overall benefit.

[0032] In an embodiment, as shown in the figure, a resource and task collaborative scheduling method based on a computing power alliance is provided. Figure 2 The system to which the method is applied is taken as an example for illustration, including the following steps: Figure 1

[0033] Step 202: Obtain data computing information of all tasks in a job.

[0034] Step 204: Calculate the proportion weight of CPU and memory capacity in each task area according to the data computing information, respectively, and determine the dynamic contribution value according to the proportion weight.

[0035] Step 206: Evaluate the difference between the deadline and the path end time of the task according to the dynamic contribution value, the task transmission time information and the network link attribute.

[0036] Step 208: Calculate the total criticality of the transmission path, and obtain the task migration path of each task according to the evaluation result of the total criticality and the difference.

[0037] Step 210: Determine the total load of each task area by using a load prediction algorithm according to the task migration path and historical experience data, and collaboratively schedule the resource storage and transmission of the task based on the computing power alliance according to the total load.​

[0038] In the above resource and task collaborative scheduling method based on computing power alliance, first, the CPU and memory capacity proportion weight in the calculation area is calculated, and a dynamic contribution value is generated combined with the task attribute, a unified priority quantization standard is established for tasks of different levels (cloud, edge, and end), breaking the problem of fragmented scheduling logic in vertical resource integration, enabling cross-layer tasks to participate in resource competition based on a unified value dimension, and laying the foundation for horizontal collaboration. Then, by evaluating the difference between the task deadline and the path end time, and optimizing the migration path selection combined with the total criticality of the transmission path, the timeliness of the task is ensured, and the network resource consumption is balanced through criticality quantization, so that the task can be flexibly migrated between different regions and different level nodes. At the same time, based on the load prediction algorithm of the migration path and historical data, the total load of each region can be accurately calculated to avoid resource overload and provide a basis for dynamic allocation of cross-domain resources. Finally, relying on the collaborative scheduling mechanism of the computing power alliance, the scheme integrates the dispersed cloud, edge, and end resources into a unified management framework, realizes on-demand allocation of resources through dynamic contribution value, ensures cross-regional transmission efficiency through path optimization, and realizes elastic supply of resources through load prediction, forming a closed loop of task value quantization-path intelligent selection-load precise regulation-resource global collaboration. It effectively solves the pain points of lack of horizontal collaboration and lack of support for cross-regional migration in traditional architecture, and significantly improves the resource allocation efficiency of cross-regional complex communication tasks.

[0039] In one embodiment, the data computing information includes: task arrival time, task required CPU core number, task required memory capacity, task required GPU type, task arrival time, task duration, task deadline, data size, task completion benefit, and overtime penalty.

[0040] In one embodiment, the proportion weight of CPU and memory resources in each task area is calculated according to the data computing information:

[0041] ;

[0042] ;

[0043] wherein, CPU proportion weight, memory capacity proportion weight, total required CPU core number in the region, total memory capacity in the region. The resource efficiency is weighted and integrated according to the CPU proportion weight, the memory capacity proportion weight, the task completion benefit, and the overtime penalty contribution value to obtain the dynamic contribution value:

[0044] ;

[0045] ;

[0046] ;

[0047] wherein, is a dynamic contribution value, is a contribution efficiency of a unit CPU, is a contribution efficiency of a unit memory capacity, is a contribution value of allocating a benefit to an unfinished task, is a required CPU core number of a task, is a required memory capacity of a task.

[0048] In one of the embodiments, the task transmission time information comprises a transmission start time, a transmission end time and a transmission time. The difference between the deadline of the scheduled task and the end time of the path is evaluated according to the dynamic contribution value, the task transmission time information and the network link attribute, and if the transmission end time of the task on the candidate path satisfies the difference greater than the delay tolerance, the evaluation result of the difference is that the candidate path satisfies the time constraint; otherwise, the evaluation result of the difference is that no candidate path satisfies the time constraint.

[0049] In one of the embodiments, the total criticality of the transmission path is calculated as:

[0050] ;

[0051] wherein, is the total criticality of the candidate path, is the criticality of the network link e, is the candidate path, is a certain network link contained in the candidate path . According to the evaluation result of the total criticality and the difference, the path optimization strategy is selected, if the evaluation result of the difference is that the candidate path satisfies the time constraint, the candidate path corresponding to the minimum value of the total criticality is selected as the task migration path of the task:

[0052] ;

[0053] wherein, is the task migration path, is the set of all candidate paths from the source node to the destination node, is the delay tolerance, is the task deadline, is the transmission end time. If the evaluation result of the difference is that no candidate path satisfies the time constraint, the candidate path corresponding to the earliest time of the transmission end time is selected as the task migration path of the task:

[0054] ;​

[0055] wherein, is a task migration path, is a candidate path.

[0056] In one of the embodiments, the total load of each task area is determined according to the task migration path and the historical experience data by using a load prediction algorithm, and the specific steps are as follows: the sliding window of the historical experience data is weighted to obtain the predicted transmission time:

[0057] ;

[0058] wherein, is the predicted transmission time, is the size of the sliding window, is the weight of the first data, is the transmission time of the first historical migration. According to the total load of each task area of the task migration path:

[0059]

[0060] ;

[0061] wherein, is the total load of the required CPU cores in the area, is the load of the required CPU cores of the new task, is the load of the required CPU cores of the running task, is the load of the required CPU cores of the migrating task, is the total load of the memory capacity in the area, is the load of the required memory capacity of the new task, is the load of the required memory capacity of the running task, is the load of the required memory capacity of the migrating task, is the total load of the required GPU cores in the area, is the total load of the required GPU cores of the new task, is the total load of the required GPU cores of the running task, is the load of the required GPU cores of the migrating task. According to the total load, the resource consumption time is obtained, and the candidate migration area corresponding to the minimum value of the resource consumption time is taken as the target migration area of the task, and the collaborative scheduling is completed;

[0062] ;

[0063] ;

[0064] ;​​

[0065] where, is the CPU core consumption time, is the memory capacity consumption time, is the GPU core consumption time, is the task completion time, is the target migration region, is the candidate migration region set, is the GPU core number required by the task.

[0066] In one embodiment, let be a set of regions, where each region is associated with a set of CP nodes. Assume that these regions are interconnected through a wide area network. Each CP node is equipped with a CPU, memory, and various types of GPUs, and the optimization goal is to maximize the total revenue of all regions while meeting resource constraints.

[0067] Binary variable indicates whether the task requires GPU type x, while indicates whether GPU type x is selected to execute the task. Tasks that require GPU type x can be replaced with higher-performance GPU type y as needed. Each task is limited to using only one GPU during execution. These constraints are as follows:

[0068] (1)

[0069] (2)

[0070] At any time slot t, the CPU, memory, and GPU usage of the computing node in region i cannot exceed its available capacity. Where indicates whether the task is allocated resources from the computing node in region m at time slot t. Next:

[0071] (3)

[0072] (4)

[0073] (5)

[0074] and the relationship between them: if task needs to be migrated, only one path can be selected; if task does not need to be migrated, it does not need to select any path.

[0075] (6)

[0076] In addition, each task can only select one path at most, and can only be allocated resources on one computing node (CP node). These constraints must be satisfied as follows:

[0077] (7)

[0078] (8)

[0079] and the relationship: if obtains resources in , .

[0080] (8)

[0081] and the relationship: if in a time slot , obtains resources in , .

[0082] (10)

[0083] (11)

[0084] To complete the time constraint, the total execution time required is .

[0085] (12)

[0086] Regarding the network constraints, when transmitting, bandwidth needs to be occupied, and when not transmitting, bandwidth is not consumed.

[0087] When , , otherwise :

[0088] (13)

[0089] In any time slot , the link bandwidth constraint in the network is:

[0090] (14) ​​

[0091] If migration is needed, task needs to wait when path bandwidth is not enough, task needs to wait when computation and memory resource is not enough, waiting is also needed:

[0092] (15)

[0093] (16)

[0094] The completion time of

[0095] (17)

[0096] The network transmission time of

[0097] (18)

[0098] The start and end time of network transmission of

[0099] (19)

[0100] Where: (Head-tail constraint): If , for any : .

[0101] (20)

[0102] If , and , then for any , : (link in any path h, if not in path k, then );

[0103] (21)

[0104] If , then , otherwise 0, converted to linear constraints, where is a large enough constant to relax the condition.

[0105] (22)

[0106] (23)

[0107] , then , otherwise 0, converted to linear constraints, the first constraint ensures that if there is a certain such that ， then , the second constraint ensures that only when all , .

[0108] (24)

[0109] (25)

[0110] Objective function: maximize total revenue (revenue of Task - penalty for overtime + bonus of job)

[0111] (26a)

[0112] s.t. (1) - (25) (26b)

[0113] A. Dynamic contribution based scheduling algorithm:

[0114] In the computing power alliance problem, the optimization goal is to maximize the total revenue of all tasks. However, greedily scheduling the highest value task at each step may result in some tasks being unable to complete, resulting in the loss of additional rewards. In addition, resource availability and dominant resources vary in different regions, so resource ratios need to be calculated to address these challenges. To solve these problems, a dynamic solution algorithm, DCS (Dynamic Contribution Based Scheduling), is proposed to achieve efficient scheduling in the local computing power alliance environment.

[0115] Algorithm 1 DCS algorithm:

[0116] Input: resource types and quantities in each region, and attributes of each task;

[0117] Output: dynamic contribution value of each task;

[0118] 1. Calculate the total proportion of CPU and memory resources in the region;

[0119] 2. For each task in the waiting task list, execute;

[0120] 3. Calculate the initial contribution value:

[0121] c1 ← v 任务 - max(t 当前 + t 持续 - deadline, 0) × penalty value;

[0122] 4. Compute additional rewards for task groups:

[0123] b <- reward x bool(status!= failed);

[0124] 5. Distribute additional rewards to unfinished tasks in the job:

[0125] c2 <- cl + b / n;

[0126] 6. Compute contribution per unit of CPU and memory resources:

[0127] c3 <- c2 / (r cpu x t 持续 );

[0128] c4 <- c2 / (r 内存 x t 持续 );

[0129] 7. Compute final dynamic contribution value for a task:

[0130] c5 <- c3 x ml + c4 x m2.

[0131] The inputs of this algorithm include total CPU and memory resources of each zone , resource requirements, value, duration, and deadline of tasks , penalty for task timeout, and reward for completing a job. The output is the final dynamic contribution value of each task, which is used to schedule tasks in descending order of contribution value. The algorithm first computes the proportion of CPU and memory resources in the zone as the basis for determining the final contribution value. Lines 2-7 of Algorithm 1 detail the process of computing the contribution value. First, initialize the contribution value of each task. If a task exceeds its deadline, its contribution value will be reduced. In addition, if all tasks in the current job do not exceed the deadline, the reward of the job will be evenly distributed to the unfinished tasks. In lines 6-7, the revenue obtained per unit of resource consumption of the current task is calculated, and the final dynamic contribution value is obtained using the resource proportion.

[0132] B. Key-based path selection algorithm:

[0133] Migration is needed when a task cannot be completed in time in the current region. In a network with multiple tasks, the bandwidth competition among tasks can significantly affect performance. To solve this problem, a path selection algorithm based on criticality is proposed. In the network topology, the importance of a link is affected by two key factors: bandwidth and the number of paths passing through the link. A link with smaller bandwidth is essentially more critical. In addition, a link passing through more paths exhibits higher criticality, similar to a high-traffic road in a traffic network that needs congestion management. The criticality of a link is defined by the following formula:

[0134] (27)

[0135] Here, denotes the criticality of a link, denotes the number of paths passing through the link, denotes the bandwidth of the link. In actual calculation, the criticality value is normalized. Algorithm 2 shows the pseudo code of the CPS (Criticality-based Path Selection) algorithm.

[0136] Algorithm 2 CPS algorithm:

[0137] Input: the criticality and load of each link in the network, and the properties of tasks;

[0138] Output: the migration path of the task, and the allocated bandwidth and time;

[0139] 1. For each task in pending_migration_tasks;

[0140] 2. Initialize , , , , ;

[0141] 3. Get all possible paths between s and d: ;

[0142] 4. For each path in P;

[0143] 5. Calculate the earliest start time and the allocated bandwidth b;

[0144] 6. Calculate the transmission time and end time: ;

[0145] 7. If , then:

[0146] 8. Calculate the criticality of the path:

[0147] 9. If criticality_path < criticality_min, then:

[0148] 10. Update criticality_min ← criticality_path;

[0149] 11. Update path_choose ← that path;

[0150] 12. Update ;

[0151] 13. Select the path with the minimum criticality, starting at t_start and ending at t_end, with bandwidth b.

[0152] 14. Update the load of the links in the selected path: link.append(t_start, t_end, b_alloc);

[0153] 15. If no path satisfies the condition, then:

[0154] 16. Select the path with the minimum t2.

[0155] The input includes the criticality and load of each network link, as well as the attributes of the task. The load of a link is represented as a set of tuples , where represents the allocated bandwidth within the time range . The output is the migration path of the task, as well as its allocated bandwidth and time. The algorithm starts from line 2 to initialize parameters for filtering paths that meet the requirements and recording the final time and bandwidth. Line 3 identifies all possible paths between the source and destination of the task. Lines 5-6 calculate the earliest start time , allocated bandwidth , transmission time , and end time . Lines 7-12 evaluate the difference between the deadline of the task and the end time of the path. If the difference exceeds the tolerance value , it is considered that the task is far from its deadline, and the subsequent success probability is higher. Then, the algorithm calculates the criticality of the path, defined as the sum of the criticality values of all edges in the path. In the case of meeting the tolerance requirement, the path with lower criticality is preferred to avoid overloading critical resources. Lines 13-14 return the path, time, and bandwidth that meet the conditions, and update the load of each link accordingly. Lines 15-16 handle the case where no path meets the requirements, returning the path with the shortest end time.

[0156] C. Regional selection algorithm based on load prediction:

[0157] For tasks that need to be migrated, multiple candidate regions provide different types and amounts of resources, while region load fluctuates over time. For example, some regions reach peak resource usage in the morning, while others reach peak usage at night. These time characteristics can be used to optimize task migration. In addition, network bandwidth usage also varies over time, and historical data can be used to estimate the transmission time of the current task.

[0158] To solve the problem of region load prediction, it is assumed that the task arrival rate of a specific region exhibits periodic changes. Through historical data, the task arrival rate at a specific time can be predicted. As shown in Figure 3 , the load change is simulated using a Poisson distribution. In the time period [0, 30], the task arrival rate is 1; in [30, 70], it is 0.5; and in [70, 100], it is 2. The horizontal axis of Figure 3 represents the interval index, and the vertical axis represents the difference in continuous task arrival times. Based on these differences, the time period is segmented and the change points of the arrival rate are identified. Figure 3 The dashed line indicates that the arrival rate changes near the 30th interval (t = 30) and the 50th interval (t = 70). The adaptive segmentation algorithm aims to divide the interval sequence into multiple segments, minimizing the error within each segment while controlling the total number of segments. By introducing a penalty term β , the algorithm avoids over-segmentation and balances between segmentation error and segment number. The objective function is defined as follows:

[0159] (28)

[0160] where, represents a change point, represents the variance of the i-th segment, and m represents the total number of segments, is the penalty value of the new segment, used to control the number of change points.

[0161] Next, the number of segments m and the corresponding change point positions are automatically determined using dynamic programming, thereby minimizing C(m). The following recursive formula is constructed:

[0162] ;

[0163] Here, represents the minimum cost from the starting point to the change point , such that . The set Sis the feasible set containing potential change points. By applying a pruning strategy, points that do not satisfy the condition are removed. Starting from the initial point, the algorithm iteratively computes the optimal position of each change point. Finally, the best number and position of change points are determined by backtracking. The method and steps of the LPRS (Load Prediction based Region Selection) algorithm are outlined in Algorithm 3.

[0164] Algorithm 3 LPRS algorithm:

[0165] Input: resource types and quantities of each region, information of running tasks, information of migrating tasks, and stored historical data;

[0166] Output: index of the task migration region;

[0167] 1. Initialization ;

[0168] 2. For each region r in regions;

[0169] 3. Estimate the average resource consumption and duration based on the tasks that have arrived at region r: (CPU, memory, GPU, and duration), respectively;

[0170] 4. Calculate the estimated transfer time using a weighted sliding window ;

[0171] 5. Use the adaptive segmentation algorithm to predict the number of new tasks expected to arrive within the time interval ;

[0172] 6. Calculate the new task load: , ;

[0173] 7. Calculate the load of running tasks and migrating tasks, denoted as load2 and load3, respectively;

[0174] 8. Calculate the total load: ;

[0175] 9. Estimate the total resource consumption time for each resource;

[0176] 10. Calculate the estimated time: ;

[0177] 11. If , then:

[0178] 12. Update .

[0179] The algorithm takes as input the type and amount of resources available in each region, running task information, migration task information, and historical data, and outputs the index of the selected region. The algorithm starts with parameter initialization. In line 3, the average resource consumption and execution time of each region are calculated using historical task arrival data. Line 4 estimates the task transfer time by weighted average of historical data. Line 5 predicts the number of new tasks expected to arrive within the time interval using an adaptive partitioning algorithm. Line 6 calculates the future potential load based on the predicted number of task arrivals, average resource consumption, and execution time. Line 7 calculates and by aggregating the information of running tasks and migration tasks. Line 8 adds these components to determine the total load of the region. Lines 9-10 estimate the total consumption time of each resource and select the maximum value as the final completion time. Finally, lines 11-12 select the region with the smallest estimated completion time tmin and return its index.

[0180] It should be understood that, although Figure 2 the steps in the flowchart are shown in order according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order limitation on the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 at least part of the steps in may include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0181] Figure 4 In one embodiment, as shown in , a resource and task collaborative scheduling system based on computing power alliance is provided, comprising: an information acquisition module 402, a contribution value calculation module 404, a time difference evaluation module 406, a migration path acquisition module 408, and a collaborative scheduling module 410, wherein:

[0182] The information acquisition module 402 is configured to acquire data calculation information of all tasks in the job.

[0183] The contribution value calculation module 404 is configured to calculate the proportion weight of CPU and memory capacity in each task region according to the data calculation information, and determine the dynamic contribution value according to the proportion weight.

[0184]

[0185] The migration path acquisition module 408 is configured to calculate the total criticality of the transmission path, and acquire the task migration path of each task according to the total criticality and the evaluation result of the difference.

[0186] The cooperative scheduling module 410 is configured to determine the total load of each task area according to the task migration path and the load prediction algorithm based on historical experience data, and cooperatively schedule the resource storage and transmission of the task based on the total load and the computing power alliance.

[0187] For specific definitions of the resource and task cooperative scheduling system based on the computing power alliance, please refer to the definitions of the resource and task cooperative scheduling method based on the computing power alliance in the foregoing, which will not be repeated here. Each module in the above-mentioned resource and task cooperative scheduling system based on the computing power alliance can be realized by software, hardware and combinations thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0188] In one embodiment, a computer device is provided, which can be a terminal, and its internal structure diagram can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface, a display screen and an input system connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a resource and task cooperative scheduling method based on a computing power alliance. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input system of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0189] Those skilled in the art can understand that Figures 4-5 the structure shown in the above-mentioned figure is only a block diagram of part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0190] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0191] Obtain data calculation information of all tasks in a job.

[0192] Calculate the proportion weight of CPU and memory capacity in each task area according to the data calculation information, and determine the dynamic contribution value according to the proportion weight.

[0193] According to the dynamic contribution value, task transmission time information and network link attribute, evaluate the difference between the deadline and the path end time of the task.

[0194] Calculate the total criticality of the transmission path, and obtain the task migration path of each task according to the evaluation result of the total criticality and the difference.

[0195] According to the task migration path and historical experience data, determine the total load of each task area by using a load prediction algorithm, and based on the total load, cooperatively schedule the resource storage and transmission of the task by the computing power alliance.

[0196] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0197] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described above, however, as long as the combination of the technical features does not exist in contradiction, it shall be considered within the scope of the present disclosure.

[0198] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it shall not be understood as a limitation on the scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A resource and task collaborative scheduling method based on computing power alliances, characterized in that, The method includes: Obtain data calculation information for all tasks in the job; The data calculation information includes: task arrival time, number of CPU cores required for the task, memory capacity required for the task, GPU type required for the task, task duration, task deadline, data size, reward for completing the task, and timeout penalty; Based on the data calculation information, the proportion weight of CPU and memory capacity in each task area is calculated, and the dynamic contribution value is determined based on the proportion weight. Based on the data calculation information, calculate the proportion and weight of CPU and memory resources in each task region: in, As a weight for CPU usage, As a percentage of memory capacity, This represents the total number of CPU cores required in the region. This represents the total memory capacity within the region. Based on the CPU usage weight, memory capacity weight, task completion rewards, and timeout penalty contribution value, resource efficiency is weighted and integrated to obtain a dynamic contribution value: in, For dynamic contribution value, Efficiency per unit of CPU. Efficiency contribution per unit of memory capacity To allocate profits to the contribution value of unfinished tasks, The number of CPU cores required for the task. The memory required for the task; The difference between the task's deadline and the path's end time is evaluated based on the dynamic contribution value, task transmission time information, and network link attributes. Calculate the total criticality of the transmission path, and obtain the task migration path for each task based on the evaluation result of the total criticality and the difference. Calculate the total criticality of the transmission path: in, Candidate paths Total criticality, For network links criticality Candidate paths, Candidate paths It includes a certain network link; The optimal path strategy is selected based on the evaluation results of the total criticality and the difference. If the evaluation result of the difference indicates that the candidate path meets the time constraint, then the candidate path corresponding to the minimum total criticality is selected as the task migration path. in, For the task migration path, This is the set of all candidate paths from the source node to the destination node. For delay tolerance, The deadline for the task. This is the transmission end time; If the evaluation result of the difference is that no candidate path satisfies the time constraint, then the candidate path corresponding to the earliest transmission end time is selected as the task migration path: in, For the task migration path, Candidate paths; Based on the task migration path and historical experience data, a load prediction algorithm is used to determine the total load of each task region. Based on the total load, the resource storage and transmission of the task are coordinated and scheduled by the computing power alliance.

2. The method according to claim 1, characterized in that, The task transmission time information includes: transmission start time, transmission end time, and transmission time; The difference between the task's deadline and the path's end time is evaluated based on the dynamic contribution value, task transmission time information, and network link attributes, including: The difference between the deadline and the path end time of the scheduled task is evaluated based on the dynamic contribution value, task transmission time information, and network link attributes. If the transmission end time of the task on the candidate path satisfies the difference being greater than the delay tolerance, the difference evaluation result is that the candidate path meets the time constraint; otherwise, the difference evaluation result is that no candidate path meets the time constraint.

3. The method according to claim 2, characterized in that, Based on the task migration path and historical experience data, a load prediction algorithm is used to determine the total load of each task region. Based on this total load, resource storage and transmission for the tasks are collaboratively scheduled by the computing power alliance, including: Based on the task migration path and historical experience data, a load prediction algorithm is used to determine the total load of each task region. The specific steps are as follows: Weighted calculation of the sliding window of the historical experience data is performed to obtain the predicted transmission time. in, To predict transmission time, To adjust the sliding window size, For the first The weight of the data. For the first Transmission time of each historical migration; Based on the total load of each task region along the task migration path: in, The total load representing the number of CPU cores required within the region. The load of CPU cores required for the new task. The load factor representing the number of CPU cores required by the running task. The number of CPU cores required for the tasks during migration. The total load on the memory capacity within the region. The load is the memory capacity required for the new task. The load is the memory capacity required by the running tasks. The load of memory required for tasks during migration. The total load representing the number of GPU cores required within the region. The total load representing the number of GPU cores required for the new task. The total load is the number of GPU cores required by the running tasks. The workload of the task requiring GPU cores during migration; The resource consumption time is obtained based on the total load, and the candidate migration region corresponding to the minimum value of the resource consumption time is taken as the target migration region of the task to complete the collaborative scheduling. in, The time consumed by the number of CPU cores. Time consumed for memory capacity Time consumed by the number of GPU cores For task completion time, For the target migration area, For the set of candidate migration regions, The number of GPU cores required for the task.

4. A resource and task collaborative scheduling system based on a computing power alliance, characterized in that, The system for implementing the method according to any one of claims 1 to 3 comprises: The information acquisition module is used to acquire data calculation information for all tasks in the job. The contribution value calculation module is used to calculate the proportion weight of CPU and memory capacity in each task area based on the data calculation information, and determine the dynamic contribution value based on the proportion weight. The time difference assessment module is used to assess the difference between the task's deadline and the path's end time based on the dynamic contribution value, task transmission time information, and network link attributes. The migration path acquisition module is used to calculate the total criticality of the transmission path and obtain the task migration path for each task based on the evaluation result of the difference between the total criticality and the total criticality. The collaborative scheduling module is used to determine the total load of each task region based on the task migration path and historical experience data using a load prediction algorithm, and to collaboratively schedule the resource storage and transmission of the task based on the total load and the computing power alliance.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

Images