A resource task scheduling method and device, electronic equipment and program product

Abstract

This application discloses a resource task scheduling method, apparatus, electronic device, and program product, belonging to the field of cloud-edge collaboration technology. The method includes: dividing available cloud resources into a first type of virtual container and a second type of virtual container; predicting the target available resources based on the historical available resources of the edge cluster; combining the target available resources, the resources of the first type of virtual container, and the amount of task data to obtain the predicted task completion time for both; with the goal of having the same unit task processing time, splitting the task data and allocating it to the edge cluster and the first type of virtual container for parallel execution; and dynamically adjusting resource allocation or task splitting boundaries with the goal of minimizing the maximum task completion time, until a preset scheduling stop condition is met. In the event of an edge cluster failure, unfinished tasks are scheduled to the second type of virtual container for continued processing. This application achieves efficient collaboration between cloud and edge resources, effectively improving resource utilization and ensuring the stability and timeliness of task execution.

Description

Technical Field

[0001] This application relates to the field of cloud-edge collaboration technology, and in particular to a resource task scheduling method, apparatus, electronic device, and program product. Background Technology

[0002] In modern computing network environments, cloud-edge collaborative resource management aims to uniformly allocate and schedule computing, storage, and network resources on the cloud and edge sides to improve overall resource utilization and task processing efficiency. Current mainstream resource scheduling strategies primarily rely on task submission order, fixed priorities, or pre-defined resource quotas. While these methods are simple to implement, they lack flexibility, easily leading to uneven resource allocation and difficulty in meeting dynamically changing business load demands, potentially resulting in task interruptions or service degradation.

[0003] Therefore, there is an urgent need for a resource task scheduling method that can achieve cloud-edge collaboration and efficient resource utilization. Summary of the Invention

[0004] This application provides a resource task scheduling method, apparatus, electronic device, and program product.

[0005] Firstly, this application provides a resource task scheduling method, including: The available resources in the cloud are divided into Category I virtual containers and Category II virtual containers; Based on the historical available resources of the edge cluster, predict the target available resources of the edge cluster; Based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of task data, the task completion time prediction results of the edge cluster and the first type of virtual container are obtained. Based on the task completion time prediction results of the edge cluster and the first type of virtual container, with the goal of the edge cluster and the first type of virtual container taking the same time to process a unit of task data, the task data splitting boundary between the edge cluster and the first type of virtual container is calculated, and the task data is distributed to the edge cluster and the first type of virtual container for parallel execution according to the task data splitting boundary. With the goal of minimizing the maximum task completion time of the edge cluster and the first type of virtual container, adjust resource allocation or task data partitioning boundaries until the preset scheduling stop conditions are met.

[0006] In some embodiments, the first type of virtual container and the edge cluster work together to process the assigned task data; when the edge cluster fails, the unfinished task data assigned to the edge cluster is rescheduled to the second type of virtual container for execution.

[0007] In some embodiments, available cloud resources are allocated according to a preset first ratio to create a first type of virtual container; available cloud resources are allocated according to a preset second ratio to create a second type of virtual container; wherein the preset first ratio is greater than the preset second ratio, and the sum of the two is 1.

[0008] In some embodiments, predicting the target available resources of an edge cluster based on the historical available resources of the edge cluster includes: Perform correlation analysis on the parameters of available resources for historical tasks, and remove redundant resource parameters whose correlation is higher than the preset correlation threshold. Based on the parameters of available resources for historical tasks after removing redundant resource parameters, a resource prediction matrix is ​​constructed, and the parameters of the resource prediction matrix are determined to calculate the target available resources.

[0009] In some embodiments, based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of task data, a task completion time prediction result is obtained, including: A task completion time prediction model is constructed by inputting the target available resources, the available resources of the first type of virtual container, and the amount of task data into the task completion time prediction model to obtain the task completion time prediction result.

[0010] In some embodiments, adjusting resource allocation or task data partitioning boundaries with the goal of minimizing the maximum task completion time of the edge cluster and the first type of virtual container includes: Locate the target scheduling object with the longest current task completion time. The target scheduling object is either an edge cluster or a first-type virtual container. If the target scheduling object is a first-class virtual container, then resources are scheduled from the other first-class virtual container with the shortest current task completion time to the target scheduling object; If the target scheduling object is an edge cluster, then a portion of the task data allocated to the edge cluster is transferred to its corresponding first-type virtual container.

[0011] Secondly, this application provides a resource task scheduling device, comprising: The resource partitioning module is configured to divide available cloud resources into first-class virtual containers and second-class virtual containers. The resource prediction module is configured to predict the target available resources of the edge cluster based on the historical available resources of the edge cluster. The completion time prediction module is configured to obtain the task completion time prediction results for the edge cluster and the first type of virtual container based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of task data. The task splitting module is configured to calculate the task data splitting boundary between the edge cluster and the first type of virtual container based on the predicted task completion time of the edge cluster and the first type of virtual container, with the goal that the time taken by the edge cluster and the first type of virtual container to process unit task data is the same, and then distribute the task data to the edge cluster and the first type of virtual container according to the task data splitting boundary for parallel execution. The dynamic adjustment module is configured to adjust resource allocation or task data splitting boundaries with the goal of minimizing the maximum task completion time of the edge cluster and the first type of virtual container, until the preset scheduling stop condition is met.

[0012] In some embodiments, the task segmentation module includes a boundary calculation submodule and a task data allocation submodule; wherein, the boundary calculation submodule is configured to calculate the task data segmentation boundary based on the task completion time prediction results of the edge cluster and the first type of virtual container, with the goal that the time taken by the edge cluster and the first type of virtual container to process unit task data is the same. The task data allocation submodule is configured to allocate task data to the edge cluster and the first type of virtual container for parallel execution according to the task data partitioning boundary.

[0013] Thirdly, this application provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement any one of the resource task scheduling methods.

[0014] Fourthly, this application provides a computer program product, including a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements any one of the resource task scheduling methods.

[0015] The at least one technical solution adopted in this application can achieve the following beneficial effects: by dividing cloud resources into a first type of virtual container and a second type of virtual container, and combining the correlation analysis of the parameter data of available resources for historical tasks in the edge cluster to predict the target available resources, a basis for task allocation is provided; with the goal of the edge cluster and the first type of virtual container having the same processing unit task time, task data is segmented to avoid inefficient waiting; and by locating the target scheduling object with the longest task completion time, resources and tasks are dynamically optimized to minimize the maximum value of the overall task completion time. When the edge cluster fails, the unfinished task data of the edge cluster is migrated and unloaded to the second type of virtual container, which continues to process the data, ensuring the continuity and reliability of task processing. Through the technical solution of this application, efficient collaboration between cloud and edge resources is achieved, which can effectively improve resource utilization and ensure the stability and timeliness of task execution.

[0016] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This illustration schematically shows a resource task scheduling process diagram of a resource task scheduling method according to an embodiment of this application; Figure 2 This illustration schematically shows a process diagram of edge cluster target available resources prediction according to an embodiment of the resource task scheduling method of this application; Figure 3 This illustration schematically shows a task completion time prediction flowchart of a resource task scheduling method according to an embodiment of this application; Figure 4 This illustration schematically shows a task splitting process diagram of a resource task scheduling method according to an embodiment of this application; Figure 5 This illustration schematically shows a first type of virtual container resource allocation and scheduling process according to an embodiment of the resource task scheduling method of this application; Figure 6 This illustration schematically shows a flowchart of an edge cluster task data reallocation method according to an embodiment of the present application; Figure 7 This illustration schematically shows a process diagram of edge cluster fault task rescheduling according to an embodiment of the resource task scheduling method of this application; Figure 8 This schematic diagram illustrates the overall structure of a resource task scheduling device according to an embodiment of this application. Figure 9 This illustration schematically shows a task segmentation module structure of a resource task scheduling device according to an embodiment of this application; Figure 10 An exemplary block diagram of a computer program product of a resource task scheduling method according to an embodiment of this application is shown schematically.

[0019] In the diagram: 801, Resource Allocation Module; 802, Resource Prediction Module; 803, Completion Time Prediction Module; 804, Task Splitting Module; 805, Dynamic Adjustment Module; 806, Boundary Calculation Submodule; 807, Task Data Allocation Submodule; 1001, Computer Program. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0021] Figure 1 The illustration shows a schematic diagram of the resource task scheduling process of a resource task scheduling method according to an embodiment of the present application.

[0022] like Figure 1 As shown, the steps include: S101 divides available cloud resources into two types of virtual containers: Type I and Type II. Specifically, Type I virtual containers establish mapping relationships with various edge clusters to collaboratively process task data; Type II virtual containers serve as a backup resource pool, handling unfinished tasks when an edge cluster fails, ensuring the continuity of task execution. This division of cloud resources into Type I and Type II containers improves cloud resource utilization efficiency while ensuring fault tolerance.

[0023] The available cloud resources are allocated according to a preset first ratio to create a first type of virtual container; the available cloud resources are allocated according to a preset second ratio to create a second type of virtual container; the preset first ratio is greater than the preset second ratio, and the sum of the two is 1.

[0024] S102, based on the historical available resources of the edge clusters, predict the target available resources for the edge clusters to execute the tasks to be processed. Specifically, collect the actual resource usage data of each edge cluster during the execution of historical tasks, such as the resource availability sequence of CPU, memory, disk I / O, and network bandwidth. Based on the parameter data of historical available resources, construct an available resource prediction model, and combine it with the current resource status of the edge clusters to predict the availability of various resources that it can provide when executing the target task, i.e., the target available resources.

[0025] S103, based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of data of tasks to be processed, obtain the task completion time prediction results for the edge cluster and the first type of virtual container. By constructing a task completion time prediction model, taking the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of data of tasks to be processed as input, the model outputs the corresponding task completion time prediction results. This task completion time prediction model quantitatively evaluates the processing time of tasks under different resource conditions, providing data support for task segmentation and resource scheduling.

[0026] S104: Based on the task completion time prediction results, and aiming for the edge cluster and the first type of virtual container to have the same processing time for unit task data, calculate the task data partitioning boundary, and allocate the task data to the edge cluster and the first type of virtual container for parallel execution according to the task data partitioning boundary. This achieves a reasonable partitioning of task data between the cloud and the edge, matching resources with the corresponding task data load, thereby improving the overall task processing efficiency.

[0027] S105, with the goal of minimizing the maximum task completion time in the edge cluster and the first type of virtual containers, adjusts resource allocation or task data partitioning boundaries until a preset scheduling stop condition is met. During scheduling, the predicted task completion times of all first type of virtual containers and edge clusters are monitored. The container with the largest predicted task completion time in either the first type of virtual container or the edge cluster is selected as the target scheduling object. Load balancing is achieved by scheduling resources or reallocating task data. For example, if a first type of virtual container becomes the target scheduling object with the largest predicted task completion time, resources are scheduled from the first type of virtual container with the smallest predicted task completion time to that target scheduling object; if an edge cluster becomes the target scheduling object with the largest predicted task completion time, a portion of its allocated task data is scheduled to its corresponding first type of virtual container. Through iterative adjustments to resources and tasks, a balance in task completion time is ultimately achieved between the edge cluster and the first type of virtual containers, resulting in efficient resource utilization.

[0028] Figure 2 The illustration shows a schematic diagram of the edge cluster target available resource prediction process of a resource task scheduling method according to an embodiment of the present application.

[0029] like Figure 2 As shown, the steps include: S201, obtain resource parameters of the edge cluster during the execution of historical tasks. Specifically, this includes the availability of resources such as computing resources, memory resources, storage I / O resources, and network bandwidth resources during the execution of historical tasks.

[0030] S202 involves performing correlation analysis on the parameters of available resources from historical tasks, removing redundant resource parameters whose correlation exceeds a preset correlation threshold. Since different resource parameters may have strong correlations, using all of them to train the available resource prediction model could easily lead to overfitting and reduce its generalization ability. Therefore, by calculating the correlation coefficients between resource parameters and comparing them with a preset correlation threshold, highly correlated redundant resource parameters are removed, thereby simplifying the model input and improving the stability and accuracy of target available resource prediction.

[0031] (Formula 1) Formula 1 is used to calculate the Pearson correlation coefficient of available resources for any two historical tasks, such as CPU and memory, disk and network bandwidth. ,in Indicates parameters p and g covariance, and Representing parameters respectively p and g The variance of the Pearson correlation coefficient. The value range of is [0,1]. When When the correlation value exceeds the preset correlation threshold, it indicates that the two resource parameters are strongly correlated. If they are used as inputs to the available resource prediction model at the same time, it is easy to cause overfitting. Therefore, one of the redundant resource parameters needs to be randomly removed to improve the generalization ability of the model.

[0032] S203. Based on the parameters of available resources for historical tasks after filtering, construct a resource prediction matrix. This resource prediction matrix establishes a mapping relationship between the parameter sequence of available resources for historical tasks and the amount of resources available when executing the target task.

[0033] S204 aims to minimize the loss between the predicted and actual resource parameter values. An optimization algorithm is used to determine the parameters of the resource prediction matrix. By inputting the parameter data of available resources from historical tasks into the available resource prediction model, the optimization algorithm iteratively adjusts the parameters of the resource prediction matrix to make the model's available resource prediction results as close as possible to the actual situation of historical tasks, thereby obtaining a resource prediction matrix with determined parameters.

[0034] (Formula 2) (Formula 3) (Formula 4) The resource prediction matrix is ​​analyzed using formulas 2, 3, and 4. Optimization was carried out, including Indicates the first iThe deviation between the predicted and actual resource values ​​for each task. 1× n Resource prediction matrix, For the front n Historical resource matrix for each task For the first i The actual available resource vector for each task; total loss The goal is to minimize the total loss by summing the quadratic forms of the deviations from all tasks; this is achieved by applying the total loss... Regarding the prediction matrix Taking the partial derivative and setting it to zero will give us the optimal solution. Parameters are used to ensure that the error between the predicted and actual resource values ​​is minimized.

[0035] S205, using the resource prediction matrix with determined parameters and the parameters of available resources for historical tasks, calculate the predicted value of the target available resources for the edge cluster to execute the target task. Specifically, input the parameter sequence of the most recent historical task available resources within a preset time window length of the edge cluster into the resource prediction matrix with determined parameters, and obtain the predicted value of the amount of available resources that it can provide for the target task through matrix operations.

[0036] (Formula 5) Formula 5 is used to calculate the target available resources for the tasks to be processed by the edge cluster. ,in This is the resource prediction matrix optimized through the above steps. This is the front of the edge cluster n This is a matrix of available resources for each historical task. By performing matrix multiplication, the availability of various resources during the execution of the target task can be quantitatively predicted.

[0037] Figure 3 The illustration shows a schematic diagram of a task completion time prediction process for a resource task scheduling method according to an embodiment of this application.

[0038] like Figure 3 As shown, the steps include: S301. Construct a training dataset based on the parameters of available resources for historical tasks, the amount of task data, and the corresponding task completion time. Specifically, the parameters of available resources for historical tasks include the amount of available resources such as computing, memory, storage, and network; the amount of task data is the data scale of the task being processed; and the task completion time is the actual time consumed by the corresponding task during execution.

[0039] S302, Construct a task completion time prediction model based on a neural network model. This model takes resource parameters and task data volume as input, performs feature transformation and nonlinear mapping through one or more hidden layers, and finally provides a predicted task completion time at the output layer. The neural network model structure enables the task completion time prediction model to learn and represent the complex nonlinear relationship between resource parameters, task data volume, and task processing time.

[0040] S303 uses the constructed training dataset to train a task completion time prediction model using the backpropagation gradient descent algorithm until the loss function converges, resulting in a well-trained task completion time prediction model. During training, the task completion time prediction model calculates the loss based on the difference between its predicted task completion time and the actual task completion time, and iteratively adjusts the weights and bias parameters of each layer within the model using the backpropagation gradient descent algorithm, causing the loss function value to continuously decrease and eventually stabilize. At this point, the task completion time prediction model has learned the mapping pattern from input features to task completion time.

[0041] (Formula 6) Formula 6 defines the loss function of the task completion time prediction model. ,in m The number of historical training samples. For the first j The actual task completion time for each sample This refers to the task completion time predicted by the model. The loss function is the mean squared error between the predicted and actual values. The model training process iteratively adjusts the weights and bias parameters of the neural network using backpropagation gradient descent until... Convergence occurs when the task completion time prediction model can fit the nonlinear relationship between resource parameters, task data volume, and task completion time.

[0042] S304, the target available resources, the available resources of the first type of virtual container, and the current task data volume are input into the trained task completion time prediction model, and the corresponding task completion time prediction results are output. Specifically, the predicted target available resources of the edge cluster, the available resources of the first type of virtual container, and the data volume of the task to be processed are used as input features and input into the trained task completion time prediction model, and the task completion time prediction results required for the edge cluster and the first type of virtual container to process the task are output respectively.

[0043] Figure 4 The illustration shows a task splitting process diagram of a resource task scheduling method according to an embodiment of this application.

[0044] like Figure 4 As shown, the steps include: S401. Based on the task completion time prediction results of the edge cluster and the corresponding first-type virtual container, a task data splitting ratio model is established with the goal of ensuring that the time taken by the edge cluster and the first-type virtual container to process a unit of task data is the same. To ensure that the tasks allocated to the edge cluster and its corresponding first-type virtual container can be completed approximately simultaneously, thereby avoiding the situation where resources on one side become idle too early while waiting for the other side, a task data splitting ratio model is established. This model correlates the task completion time prediction results of both sides with the amount of task data to be processed, with the constraint that both sides take the same amount of time to process each unit of task data, thereby calculating the optimal task data allocation ratio.

[0045] (Formula 7) Formula 7 is used to calculate the task data splitting boundary, where For edge clusters j The predicted task completion time. This is the predicted task completion time for the first type of virtual container. To be allocated to edge clusters j The amount of task data, For edge clusters j The corresponding task data volume. By solving... That is, the amount of task data that the edge cluster should process, and the amount of remaining task data. It is then allocated to the corresponding first-type virtual container to ensure that the edge cluster and the first-type virtual container process in parallel and complete synchronously, thus avoiding resource idleness.

[0046] S402, Solve the task data partitioning ratio model to obtain the partitioning boundary of task data between the edge cluster and the first type of virtual container. By solving the task data partitioning ratio model, the partitioning boundary of task data is calculated to divide the part that should be processed by the edge cluster and the part that should be processed by its corresponding first type of virtual container, so that under the current predicted resource parameters of both parties, the time required to complete the allocated task data volume of each party meets the optimization objective of equal unit task data processing time.

[0047] S403, based on the calculated task data partitioning boundaries, distributes the task data to be processed to the edge cluster and the first type of virtual container for parallel execution. Based on the determined task data partitioning boundaries, the task data is divided into two subsets and distributed to the edge cluster and its corresponding first type of virtual container, respectively. The edge cluster and the first type of virtual container process their respective allocated task data in parallel, thereby achieving cloud-edge collaborative computing, effectively shortening the overall task completion time and improving resource utilization efficiency.

[0048] Figure 5The illustration shows a schematic diagram of a first type of virtual container resource allocation and scheduling process according to an embodiment of the present application.

[0049] like Figure 5 As shown, the steps include: S501, when adjusting and scheduling resources and tasks with the goal of minimizing the maximum completion time, if the target scheduling object with the longest current task completion time is located and identified as a first-type virtual container, the first-type virtual container is determined to be the bottleneck restricting the overall task processing progress and is used as the target scheduling object for this adjustment.

[0050] S502, from all first-class virtual containers, determine the first-class virtual container with the shortest current task completion time. To alleviate the bottleneck in task processing progress, select the first-class virtual container with the shortest predicted task completion time from all first-class virtual containers, and regard it as a relatively idle or efficient first-class virtual container. Its surplus resources can be used to support the target scheduling object.

[0051] S503: Allocate resources from the first type of virtual container with the shortest current task completion time to the target scheduling object. Resources from the first type of virtual container with the shortest task completion time are allocated to the target scheduling object, providing computing, memory, storage, network, and other resources to enhance the target scheduling object's task processing capabilities, thereby shortening its task completion time.

[0052] S504: After completing resource scheduling, re-acquire and predict the task completion time forecasts for all edge clusters and all Type 1 virtual containers. Resource scheduling and allocation change the state of available resources for Type 1 virtual containers; therefore, based on the new resource distribution, the task completion time prediction process needs to be re-executed to obtain updated task completion time forecasts for all edge clusters and all Type 1 virtual containers that reflect the effectiveness of resource scheduling.

[0053] S505, determine whether the task completion time dispersion between the edge cluster and the first type of virtual container has fallen below a preset dispersion threshold. Based on the re-acquired task completion time prediction results, calculate the dispersion value of the task completion time for all edge clusters and the first type of virtual container, using standard deviation or coefficient of variation, and compare it with the preset dispersion threshold. If the calculated dispersion value is lower than the preset dispersion threshold, it indicates that the edge cluster and the first type of virtual container have reached or are close to a load-balanced state, and the overall maximum completion time has been effectively minimized, and this adjustment ends; otherwise, the next round of adjustment will be carried out based on the new task completion time prediction results.

[0054] Figure 6The diagram illustrates an edge cluster task data reallocation flowchart of a resource task scheduling method according to an embodiment of this application.

[0055] like Figure 6 As shown, the steps include: S601, when adjusting resources and tasks with the goal of minimizing the maximum completion time, if the target scheduling object with the longest current task completion time is located and identified as the edge cluster, then the edge cluster is determined to be the bottleneck restricting the overall task processing progress, and it is used as the target scheduling object for this task data reallocation.

[0056] S602, a portion of the task data allocated to the target scheduling object is reallocated to its corresponding first-type virtual container. To alleviate the bottleneck in task processing progress, some incomplete task data is transferred from the task data already allocated to the edge cluster to the first-type virtual container corresponding to the edge cluster. The relatively abundant or stable resources of the first-type virtual container in the cloud are used to share the task processing pressure of the edge cluster, thereby shortening the task completion time of the edge cluster.

[0057] S603, After the task data is redistributed, the task completion time prediction results for all edge clusters and all Class I virtual containers are reacquired and predicted. Due to the redistribution of task data, the distribution of task data between edge clusters and Class I virtual containers has changed. Therefore, it is necessary to re-execute the task completion time prediction process based on the latest task data distribution to obtain updated task completion time prediction results for all edge clusters and Class I virtual containers that reflect the effect of task data redistribution.

[0058] S604, determine whether the task completion time dispersion between the edge cluster and the first type of virtual container has fallen below a preset dispersion threshold. Based on the re-acquired task completion time prediction results, calculate the dispersion value of the task completion time for all edge clusters and the first type of virtual container, using standard deviation or coefficient of variation, and compare it with the preset dispersion threshold. If the calculated dispersion value is lower than the preset dispersion threshold, it indicates that the edge cluster and the first type of virtual container have reached or are close to a load-balanced state, and the overall maximum completion time has been effectively minimized, and this adjustment ends; otherwise, the next round of adjustment will be carried out based on the new task completion time prediction results.

[0059] Figure 7 The illustration shows a schematic diagram of the edge cluster fault task rescheduling process of a resource task scheduling method according to an embodiment of this application.

[0060] like Figure 7 As shown, the steps include: S701 monitors the operational status of each edge cluster. It performs status monitoring on all edge clusters, obtaining availability and operational status information through heartbeat detection, performance indicator probes, or fault alarm mechanisms to ensure timely detection of any anomalies or fault events occurring in the edge clusters.

[0061] S702, when a failure is detected in an edge cluster, reschedules unfinished tasks assigned to that edge cluster to the second type of virtual container. When an edge cluster is confirmed to have failed or become unresponsive, its assigned unfinished tasks are reassigned to a reserved second type of virtual container in the cloud. This second type of virtual container acts as a flexible backup computing resource, receiving and processing these tasks to ensure that tasks are not interrupted or lost due to a single point of failure, achieving seamless task succession and high availability.

[0062] Figure 8 The schematic diagram illustrates the overall structure of a resource task scheduling device according to an embodiment of this application.

[0063] like Figure 8 As shown, this device includes a resource allocation module 801, a resource prediction module 802, a completion time prediction module 803, a task segmentation module 804, and a dynamic adjustment module 805.

[0064] The resource partitioning module 801 is configured to divide the available resources in the cloud into a first type of virtual container and a second type of virtual container. The first type of virtual container is used to establish a mapping relationship with each edge cluster to collaboratively process task data, and the second type of virtual container is used to take over the unfinished tasks of the edge cluster when it fails.

[0065] The resource prediction module 802 is configured to predict the target available resources for the edge cluster to execute the target task based on the parameters of the historical available resources of the edge cluster.

[0066] The completion time prediction module 803 is configured to obtain the task completion time prediction results for the edge cluster and the first type of virtual container based on the target available resources, the available resources of the first type of virtual container, and the amount of data to be processed.

[0067] The task splitting module 804 is configured to calculate the task data splitting boundary based on the task completion time prediction result, with the goal of the edge cluster and the first type of virtual container taking the same time to process unit task data, and then distribute the task data to the edge cluster and the first type of virtual container for parallel execution according to the task data splitting boundary.

[0068] The dynamic adjustment module 805 is configured to adjust resource allocation or task data partitioning boundaries with the goal of minimizing the maximum task completion time in the edge cluster and the first type of virtual container, until the preset scheduling stop condition is met.

[0069] Figure 9 The illustration shows a schematic diagram of the task segmentation module structure of a resource task scheduling device according to an embodiment of this application.

[0070] like Figure 9 As shown, the task segmentation module 804 includes a boundary calculation submodule 806 and a task data allocation submodule 807.

[0071] Among them, the boundary calculation submodule 806 is configured to establish and solve the task data segmentation ratio model based on the task completion time prediction results of the edge cluster and the corresponding task completion time prediction results of the first type of virtual container, with the goal that the time taken by the edge cluster and the first type of virtual container to process unit task data is the same, so as to obtain the segmentation boundary of task data between the edge cluster and the first type of virtual container.

[0072] The task data allocation submodule 807 is configured to allocate the task data to the edge cluster and the first type of virtual container for parallel execution based on the task data segmentation boundary calculated by the boundary calculation submodule 806.

[0073] Figure 10 An exemplary block diagram of a computer program product of a resource task scheduling method according to an embodiment of this application is shown schematically.

[0074] like Figure 10 As shown, the computer program product stores a computer program 1001, which, when executed by a processor, implements the method provided in any embodiment of this application.

[0075] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0076] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0077] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0078] It should also be noted that in the system and method of this application, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of this application.

[0079] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this application is not limited to the specific aspects of the processes, machines, manufacturing, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described above can be utilized. Therefore, the appended claims include such processes, machines, manufacturing, events, means, methods, or actions within their scope.

[0080] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0081] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A resource task scheduling method, characterized in that, include: The available resources in the cloud are divided into Category I virtual containers and Category II virtual containers; Based on the historical available resources of the edge cluster, predict the target available resources of the edge cluster; Based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of task data, the task completion time prediction results of the edge cluster and the first type of virtual container are obtained. Based on the task completion time prediction results of the edge cluster and the first type of virtual container, with the goal of the edge cluster and the first type of virtual container taking the same time to process unit task data, the task data splitting boundary between the edge cluster and the first type of virtual container is calculated, and the task data is distributed to the edge cluster and the first type of virtual container for parallel execution according to the task data splitting boundary. With the goal of minimizing the maximum task completion time of the edge cluster and the first type of virtual container, adjust resource allocation or task data splitting boundaries until the preset scheduling stop condition is met.

2. The resource task scheduling method according to claim 1, characterized in that, The first type of virtual container works in conjunction with the edge cluster to process task data; when the edge cluster fails, the unfinished task data allocated to the edge cluster is rescheduled to the second type of virtual container for execution.

3. The resource task scheduling method according to claim 1, characterized in that, The available cloud resources are allocated according to a preset first ratio to create the first type of virtual container; the available cloud resources are allocated according to a preset second ratio to create the second type of virtual container; wherein the preset first ratio is greater than the preset second ratio, and the sum of the two is 1.

4. The resource task scheduling method according to claim 1, characterized in that, Predicting the target available resources of the edge cluster based on the historical available resources of the edge cluster includes: A correlation analysis is performed on the parameters of the available resources for the historical tasks, and redundant resource parameters with a correlation higher than a preset correlation threshold are removed. Based on the parameters of the available resources for the historical task after removing the redundant resource parameters, a resource prediction matrix is ​​constructed, and the parameters of the resource prediction matrix are determined to calculate the target available resources.

5. The resource task scheduling method according to claim 1, characterized in that, Based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the task data volume, the task completion time prediction result is obtained, including: A task completion time prediction model is constructed by inputting the target available resources, the available resources of the first type of virtual container, and the amount of task data into the task completion time prediction model to obtain the task completion time prediction result.

6. The resource task scheduling method according to claim 1, characterized in that, With the goal of minimizing the maximum task completion time of the edge cluster and the first type of virtual container, the resource allocation or the task data partitioning boundary is adjusted, including: Locate the target scheduling object with the longest current task completion time, where the target scheduling object is the edge cluster or the first type of virtual container; If the target scheduling object is the first type of virtual container, then resources are scheduled from the other first type of virtual container with the shortest current task completion time to the target scheduling object; If the target scheduling object is the edge cluster, then a portion of the task data allocated to the edge cluster is transferred to its corresponding first type of virtual container.

7. A resource task scheduling device, characterized in that, include: The resource partitioning module is configured to divide available cloud resources into first-class virtual containers and second-class virtual containers. The resource prediction module is configured to predict the target available resources of the edge cluster based on the historical available resources of the edge cluster. The completion time prediction module is configured to obtain the task completion time prediction results of the edge cluster and the first type of virtual container based on the target available resources of the edge cluster, the available resources of the first type of virtual container, and the amount of task data. The task splitting module is configured to calculate the task data splitting boundary between the edge cluster and the first type of virtual container based on the task completion time prediction results of the edge cluster and the first type of virtual container, with the goal that the time taken by the edge cluster and the first type of virtual container to process unit task data is the same, and to distribute the task data to the edge cluster and the first type of virtual container according to the task data splitting boundary for parallel execution; The dynamic adjustment module is configured to adjust resource allocation or task data splitting boundaries with the goal of minimizing the maximum task completion time of the edge cluster and the first type of virtual container, until a preset scheduling stop condition is met.

8. The resource task scheduling device according to claim 7, characterized in that, The task segmentation module includes a boundary calculation submodule and a task data allocation submodule; wherein, The boundary calculation submodule is configured to calculate the task data segmentation boundary based on the task completion time prediction results of the edge cluster and the first type of virtual container, with the goal that the edge cluster and the first type of virtual container take the same time to process unit task data. The task data allocation submodule is configured to allocate task data according to the task data partitioning boundary to the edge cluster and the first type of virtual container for parallel execution.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program implements the resource task scheduling method according to any one of claims 1 to 6.

10. A computer program product comprising a computer-readable storage medium on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the resource task scheduling method according to any one of claims 1 to 6.

Images