Services-free computing dynamic workflow scheduling method, storage medium and electronic equipment

Abstract

The invention provides a service-free computing dynamic workflow scheduling method, a storage medium and electronic equipment, and the method comprises the steps: periodically merging and decomposing function containers on computing nodes based on a resource access frequency; in response to a received task scheduling request, determining a deployment node of a task according to an intermediate data volume and a task parallelism degree generated by the task scheduling request; and distributing all tasks into a plurality of task groups, and distributing the tasks to containers or container combinations with proper degrees for the task groups where the tasks scheduled to the current node are located. The resource utilization rate of the computing node can be effectively improved, the load throughput is effectively improved, and the end-to-end delay is reduced.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to the field of cloud native and serverless computing technology. Background Art

[0002] With the development of cloud computing, serverless computing has gradually become a hot spot of development. Its main features are pay-as-you-go and on-demand expansion, which have significant performance and price advantages when dealing with load fluctuations. In order to expand the application scenarios of serverless computing, cloud providers have launched workflow-oriented serverless computing implementations. From the perspective of control flow, complex functions are implemented through the topological relationship between multiple stage functions, which can generally be represented by a directed acyclic graph (DAG). However, the current common serverless computing workflow engines are mainly oriented towards static workflows, that is, they assume that the computing resources required for each request are relatively consistent, ignoring the impact of the dynamic characteristics of the request on the service quality:

[0003] (1) The impact of the size of intermediate data on end-to-end latency. There is a data transmission process between adjacent steps in the workflow. The stateless nature of serverless computing requires that containers pulled up on any node can access intermediate data. To this end, it is usually necessary to use remote storage to access intermediate data. This method has too high a transmission overhead when the intermediate data is large.

[0004] (2) The impact of the number of parallel branches on system resource allocation. Workflows provide parallel statements to support large-scale data processing. However, the scalability of serverless computing requires that each task corresponds to a container instance. Therefore, over-scaling containers under high parallelism will occupy a large amount of computing resources. In addition, a large number of containers have low access frequencies, resulting in resource waste and a negative impact on the deployment of cloud providers and the service quality of users.

[0005] Using local memory storage as the data transmission medium between upstream and downstream functions can effectively reduce data transmission overhead. This method requires that upstream and downstream functions are deployed on the same computing node. The resource demand is relatively constant under static workflows, and the workflow graph can be segmented in advance by pre-collecting data. However, for dynamic workflows, due to the high volatility of resource demand, the graph segmentation effect is not good, and it is difficult to meet the deployment conditions under certain load characteristics, which will lead to queuing and serious degradation of user service quality. In addition, previous work has also proposed to reduce memory overhead under high parallelism by sharing the runtime, but it is mainly aimed at multiple requests for a single function, ignoring issues such as security and fairness; in the workflow scenario, parallel tasks are easily interfered with each other, and no effective sharing strategy has been proposed yet.

[0006] Compared with static workflows, task scheduling for dynamic workflows faces two new challenges: First, the problem of over-resource allocation is prone to occur in dynamic scenarios. Under certain load conditions, the expanded container instances occupy a large amount of system resources, but their utilization rate is extremely low for a long time, which has an adverse impact on the overall performance of the system. Therefore, how to use system resources more efficiently has become a key issue that needs to be solved urgently. Second, different requests may show significant feature differences in the same workflow. General deployment strategies cannot adapt to this difference and may cause the performance of some requests to drop significantly. Therefore, how to achieve an effective balance between resource overhead and data transmission overhead and formulate the optimal scheduling plan is another important challenge in dynamic workflow scheduling. Summary of the invention

[0007] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a serverless computing dynamic workflow scheduling method, storage medium and electronic device, which are used to improve the resource utilization of computing nodes, effectively increase load throughput, and reduce end-to-end latency.

[0008] To achieve the above-mentioned purpose and other related purposes, the present application provides a method for dynamic workflow scheduling of serverless computing, the method comprising: based on the frequency of resource access, periodically merging and decomposing function containers on computing nodes; in response to receiving a task scheduling request, determining the deployment node of the task according to the amount of intermediate data generated by the task scheduling request and the task parallelism; distributing all tasks into multiple task groups, and distributing the tasks to containers or container combinations of appropriate degrees for the task group to which the tasks scheduled to the current node belong.

[0009] In one embodiment of the present application, the merging and decomposing of function containers on the computing nodes includes: controlling a single container to process a group of tasks in a multi-process multi-request parallel processing mode; making containers of different degrees form a heterogeneous container pool, and allocating corresponding container groups to the current task parallelism based on the current task parallelism requirement and the heterogeneous container pool; merging and decomposing the function containers in each container group on the computing node according to the container degree and access frequency.

[0010] In one embodiment of the present application, the method of allocating a corresponding container group for the current task parallelism based on the current task parallelism requirement and the heterogeneous container pool is:

[0011]

[0012] Among them, N is the current parallelism requirement, D i is the number of tasks that container i can support for parallel execution, that is, the container degree, and K is the minimum number of containers in the heterogeneous container pool for the current parallelism requirement N.

[0013] In one embodiment of the present application, the merging of function containers in each container group on the computing node according to the container degree and the access frequency includes: if there is a container whose container degree matches the current parallelism requirement, the container is allocated to the current task parallelism; if there is a container whose degree is less than the current parallelism requirement, a container combination consisting of at least two containers is searched from the heterogeneous container pool based on the access frequency, the total container degree of the container combination matches the current parallelism requirement, and the container combination is allocated to the current task parallelism.

[0014] In one embodiment of the present application, decomposing the function containers in each container group on the computing node according to the container degree and the access frequency includes: in response to the failure of the container combination search, obtaining the container to be decomposed from the heterogeneous container pool based on the access frequency and the container degree; decomposing the container to be decomposed into at least two containers, and combining one of the containers with other containers in the heterogeneous container pool to form the container combination; in response to the access frequency of the container being lower than the access frequency threshold, decomposing the container.

[0015] In one embodiment of the present application, determining the deployment node of the task based on the intermediate data volume and task parallelism generated by the task scheduling request includes: calculating the intermediate data volume corresponding to the current task parallelism, and judging whether the resources of the local computing node meet the resource computing requirements of the intermediate data volume; if the resources of the local computing node meet the resource computing requirements of the intermediate data volume, deploying the task on the local computing node; if the resources of the local computing node do not meet the resource computing requirements of the intermediate data volume, splitting the task, and deploying the split part of the task on the remote computing node.

[0016] In one embodiment of the present application, splitting the task and deploying part of the split tasks on the remote computing node includes: respectively obtaining the completion time of the task on the local computing node and the completion time of the task on the remote computing node; enumerating all binary results of the task to form a local task set and a remote task set; respectively obtaining each time combination of the completion time of the task on the local computing node and the completion time of the task on the remote computing node according to the remote task set; obtaining the minimum value of each of the time combinations, splitting the task based on the minimum value, and deploying part of the split tasks on the remote computing node.

[0017] In one embodiment of the present application, for the task group to which the task scheduled to the current node belongs, distributing the task to a container or container combination of appropriate degree includes: sorting all tasks to be processed from small to large according to the amount of data to obtain a double-ended task queue; for the container degree D i , respectively take D i / 2 team leader tasks and Di / 2 tail tasks are distributed as container tasks to the corresponding container or container group; repeat the above operation until all tasks are distributed.

[0018] To achieve the above-mentioned purpose and other related purposes, the present application also provides a computer storage medium storing program instructions, which, when executed, implement the steps of the serverless computing dynamic workflow scheduling method as described above.

[0019] To achieve the above-mentioned purpose and other related purposes, the present application also provides an electronic device, including a memory for storing a computer program; and a processor for running the computer program to implement the steps of the serverless computing dynamic workflow scheduling method as described above.

[0020] As described above, the serverless computing dynamic workflow scheduling method, storage medium and electronic device of the present application have the following beneficial effects:

[0021] This application can solve the problem of service quality degradation caused by high variability of computing resources and intermediate data volume in dynamic workflow scenarios, achieve the goal of minimizing latency, while minimizing resource waste, effectively improving computing node resource utilization, effectively improving load throughput, and reducing end-to-end latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 Shown is a schematic diagram of an implementation principle architecture diagram of a method for dynamic workflow scheduling of serverless computing in an embodiment of the present application;

[0024] Figure 2 Shown is a schematic diagram of the overall process of a method for dynamic workflow scheduling of serverless computing in an embodiment of the present application;

[0025] Figure 3 Shown is a schematic diagram of the process of merging and decomposing containers in a method for dynamic workflow scheduling of serverless computing in an embodiment of the present application;

[0026] Figure 4 Shown is a flow chart of determining the deployment nodes of tasks in a method for dynamic workflow scheduling of serverless computing in an embodiment of the present application;

[0027] Figure 5Shown is a flowchart of task segmentation in a method for dynamic workflow scheduling of serverless computing in an embodiment of the present application;

[0028] Figure 6 Shown is a schematic diagram of the overall execution process of a serverless computing dynamic workflow scheduling method in an embodiment of the present application;

[0029] Figures 7a to 7c Shown is an average delay curve diagram of a serverless computing dynamic workflow scheduling method under various benchmark tests in an embodiment of the present application;

[0030] Figures 8a to 8c Shown is a throughput curve chart of a serverless computing dynamic workflow scheduling method under various benchmark tests in an embodiment of the present application;

[0031] Fig. 9 It is a principle block diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The following describes the implementation of this embodiment through specific examples, and those skilled in the art can easily understand other advantages and effects of this embodiment from the content disclosed in this specification. This embodiment can also be implemented or applied through other different specific implementations, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of this embodiment. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0033] How to achieve an effective balance between resource overhead and data transmission overhead and formulate the optimal scheduling plan is another important challenge in dynamic workflow scheduling. One of the existing methods is to adopt a load balancing priority method, by setting a global scheduler to monitor the resource usage of each node in the cluster in real time, and allocate working nodes with sufficient resources to all tasks to be executed, so as to effectively avoid problems such as node overload. Although this solution has alleviated the problem of node overload to a certain extent, it has the following shortcomings: high scheduling overhead. All scheduling requests need to be decided by the global scheduler, which not only increases the system scheduling overhead, but also easily causes the global scheduler to become the performance bottleneck of the system; low data transmission efficiency: due to the lack of location information of downstream functions, the system needs to frequently use remote databases as storage media, resulting in high data transmission overhead, which has an adverse impact on overall performance.

[0034] Another existing approach is to adopt a locality-first approach, deploying the entire workflow on the same node and transferring data through local high-speed memory cache, thereby significantly reducing the data transmission overhead between upstream and downstream functions. Although this approach effectively reduces data transmission overhead, it does not fully consider the current node status and load resource requirements. This neglect may cause node overheating and cause queuing (i.e., node resources are exhausted and function execution needs to wait for idle resources to be released), thereby limiting the full utilization of cluster resources and making it difficult to meet high service quality requirements.

[0035] Another method in the prior art is to use a static graph splitting scheduling algorithm. This method uses statistical analysis to identify the average computing resources required for each step in the serverless computing workflow and the average data transmission volume between upstream and downstream functions. Then, the upstream and downstream functions with the largest data transmission volume are merged and deployed to the same node using a greedy algorithm to reduce data transmission overhead. The above process is repeated until the node resources cannot meet the further merging requirements, and finally a fixed workflow scheduling path is generated. The main drawback of this method is that the characteristics of the workflow are simplified to the average value of task parallelism and intermediate data size. However, in dynamic workflow scenarios, the task parallelism and intermediate data size of different requests have a large variance. The fixed scheduling path based on average value splitting cannot always meet the optimal scheduling requirements of all requests, limiting its application effect in dynamic scenarios.

[0036] This application proposes a serverless computing dynamic workflow scheduling method for latency optimization, which mainly includes control: a resource manager based on a container merging / decomposition mechanism, identifying container instances that can share runtime, reallocating system resources by merging / decomposing containers, reducing memory resource waste while avoiding repeated cold start overhead; a resource-load collaboratively aware task scheduler, obtaining the computing resource requirements of the current request, combining the capacity of local resources and cluster resources, planning appropriate scheduling paths and scheduling schemes, and minimizing end-to-end delays by taking advantage of locality. This embodiment effectively improves the throughput of the system by improving the utilization of node resources; at the same time, it can efficiently process requests with diverse data characteristics and parallelism characteristics, provide the optimal execution path and deployment scheme, and minimize end-to-end delays. Therefore, this embodiment can solve the problem of service quality degradation caused by the high variability of computing resources and intermediate data volume in dynamic workflow scenarios, achieve the goal of minimizing delays, and at the same time minimize resource waste and improve load throughput.

[0037] The principles and implementation methods of the server-free computing dynamic workflow scheduling method, storage medium and electronic device of this embodiment will be explained in detail below, so that those skilled in the art can understand the server-free computing dynamic workflow scheduling method, storage medium and electronic device of this embodiment without creative work.

[0038] Figure 1 It is a schematic diagram of the implementation principle architecture of the serverless computing dynamic workflow scheduling method in one embodiment of the present application; Figure 1 As shown, the dynamic workflow scheduling method for serverless computing of this embodiment is applied to the resource manager and task scheduler to form a resource manager based on the container merging / decomposition mechanism and a resource-load collaboratively aware task scheduler, which is used to achieve accurate resource allocation and optimal execution path generation. The resource manager uses a reasonable and effective algorithm to merge multiple container instances of the same function on the node to reduce the overall memory usage. When it is observed that the merged container has not been used for a period of time, it will shrink or decompose it to release over-allocated computing resources in time. The task scheduler pulls up the corresponding container on a single or multiple nodes according to the characteristics of the load, and minimizes the request execution delay through optimization methods such as data locality, load splitting, and task distribution. Specifically, the dynamic workflow scheduling method for serverless computing of this embodiment is implemented through three steps:

[0039] 1) Regularly monitor system resources and the number and access frequency of containers of various functions on the node. For containers with high access frequency, periodically merge them and use shared runtime to reduce memory redundancy and reduce overall system resource usage. For containers with low access frequency, decompose them into other containers with high access frequency or directly recycle them to improve the effective utilization of resources.

[0040] 2) Track the execution status of each request. The task scheduler determines the deployment location of the downstream function based on the size of the intermediate data generated by the current request and the task parallelism of the downstream function. When the data volume is large and the parallelism is low, deploying on the same node can take advantage of locality to speed up data access and avoid the data transmission overhead caused by remote storage; conversely, when the data volume is small and the parallelism is high, deploying on multiple nodes can make full use of cluster resources to ensure execution efficiency and avoid queuing caused by node overheating. At this time, the task scheduler divides the load deployed to the remote node through the load splitting algorithm to minimize the overall delay.

[0041] 3) For tasks scheduled to the current node, the task scheduler uses a heuristic algorithm to distribute all tasks into multiple task groups, and the resource manager matches an appropriate number of containers or container combinations for each task group. Due to the unbalanced nature of the task load, the distribution method can fully utilize computing resources and reduce the total execution delay of multiple container combinations.

[0042] Figure 2 The overall flow diagram of the serverless computing dynamic workflow scheduling method in one embodiment of the present application is shown; Figure 2 As shown, the serverless computing dynamic workflow scheduling method includes the following steps S100 to S300.

[0043] Step S100, periodically merging and decomposing function containers on computing nodes based on resource access frequency;

[0044] Step S200, in response to receiving a task scheduling request, determining a deployment node for the task according to the amount of intermediate data generated by the task scheduling request and the task parallelism;

[0045] Step S300 , distribute all tasks into multiple task groups, and distribute the tasks in the task group to which the tasks scheduled to the current node belong to to containers or container combinations of appropriate degrees.

[0046] The present application can efficiently utilize cluster resources to support higher load throughput and achieve the lowest end-to-end latency under various request characteristics of dynamic workflows.

[0047] The above steps S100 and S300 in the serverless computing dynamic workflow scheduling method of this embodiment are described in detail below.

[0048] Step S100: periodically merge and decompose function containers on computing nodes based on resource access frequency.

[0049] There are often multiple container instances of the same function on the same node. The runtime and dependencies of these containers are the same, and the number of containers required by the current request is equal to the parallelism of the task. In the dynamic workflow scenario, due to the difference in parallelism, the extended containers are used unevenly, and some will not be used for a long time, wasting system resources and even occupying the expansion space of other functions / workflows. This embodiment provides a resource manager based on a container merging / decomposition mechanism to alleviate the phenomenon of over-allocation of resources caused by dynamic workflows by sharing runtime. The container merging / decomposition mechanism of this embodiment follows the following principles: First, it does not undermine security and fairness, that is, tasks from different requests are not allowed to be scheduled to the same container to ensure isolation across requests; second, it does not violate the current billing model of serverless computing, that is, the corresponding operating resources must be allocated according to the basic specifications set by the user, and it is consistent with the original billing model; third, it does not cause serious performance loss, that is, it is necessary to ensure that the non-extensible resources (such as hard disk, network bandwidth, etc.) of the merged container do not exceed the upper limit.

[0050] Figure 3It is a schematic diagram of the process of merging and decomposing containers in the serverless computing dynamic workflow scheduling method in one embodiment of the present application; Figure 3 As shown, in a specific implementation of this embodiment, the merging and decomposing of function containers on computing nodes includes the following steps S110 to S130.

[0051] Step S110, the multi-process multi-request parallel processing mode controls a single container to process a group of tasks;

[0052] Step S120, containers of different degrees are composed of a heterogeneous container pool, and a corresponding container group is allocated to the current task parallelism based on the current task parallelism requirement and the heterogeneous container pool;

[0053] Step S130 , merging and decomposing the function containers in each container group on the computing node according to the container degree and the access frequency.

[0054] First, this embodiment improves the existing processing mode to adapt to the dynamic parallelism characteristics of the workflow. In the existing serverless computing framework, usually a single container can only process one task in the same time period. However, this embodiment has observed that multiple container instances of the same function have a high degree of similarity, and their underlying dependencies (such as imported function libraries, loaded neural network models, etc.) can be shared. Based on this feature, this embodiment improves the request processing mechanism of the container, adopts the process replication method, and realizes the processing mode of multi-process service multi-request. At the same time, by introducing a signal mechanism to achieve synchronization and coordination between processes, a group of tasks can be centrally processed by a single container, and the results are returned uniformly after all tasks are executed.

[0055] In a specific implementation of this embodiment, the method of allocating a corresponding container group for the current task parallelism based on the current task parallelism requirement and the heterogeneous container pool is:

[0056]

[0057] Among them, N is the current parallelism requirement, D i is the number of tasks that container i can support for parallel execution, that is, the container degree, K is the current parallel degree requirement N, and the minimum number of containers is obtained in the heterogeneous container pool.

[0058] This embodiment proposes a conversion mechanism between heterogeneous container pools and containers. Defining the degree D iis the number of tasks that container i can support for parallel execution. After the multi-process parallel processing mode is introduced, the degrees of each container are different due to different allocated resources. The higher the degree of the container, the more tasks it shares at runtime, and the less memory resources a single task requires on average. The heterogeneous container pool is composed of containers of different degrees. For ease of management, this embodiment sets the degree to a power of 2 (1, 2, 4, etc.) and specifies the upper bound as the maximum degree under non-scalable resource constraints (Principle 3). After the introduction of the heterogeneous container pool, this embodiment uses the minimum covering method to allocate corresponding container groups to the current task parallelism to reduce scheduling overhead (cold start, etc.). That is, for the current parallelism requirement N, the minimum number of containers K in the container pool is obtained so that:

[0059]

[0060] This embodiment combines frequently used containers into containers with higher degrees, decomposes infrequently used containers into frequently used containers with lower degrees or directly cleans them up according to the actual usage of the containers during the window period, and recycles idle system resources in time to meet the needs of the current load.

[0061] Specifically, in the specific implementation of this embodiment, the merging of function containers in each container group on the computing node according to the container degree and the access frequency includes: if there is a container whose container degree matches the current parallelism requirement, the container is allocated to the current task parallelism; if there is a container whose degree is less than the current parallelism requirement, a container combination consisting of at least two containers is searched from the heterogeneous container pool based on the access frequency, the total container degree of the container combination matches the current parallelism requirement, and the container combination is allocated to the current task parallelism.

[0062] In a specific implementation of this embodiment, decomposing the function containers in each container group on the computing node according to the container degree and the access frequency includes: in response to a failure in the container combination search, obtaining a container to be decomposed from the heterogeneous container pool based on the access frequency and the container degree; decomposing the container to be decomposed into at least two containers, and combining one of the containers with other containers in the heterogeneous container pool to form the container combination; in response to the access frequency of a container being lower than an access frequency threshold, decomposing the container.

[0063] That is, in this embodiment, there are four allocation methods for the degree of demand, with the priority from high to low:

[0064] 1) Directly allocate containers of corresponding specifications, which requires that there are idle containers of the corresponding specifications in the system;

[0065] 2) Combining other containers with smaller degrees. In order to make the best use of existing containers, this embodiment attempts to search for a container combination so that the sum of the parallelism of the container combination is equal to the specified degree, avoiding adjustments to existing containers.

[0066] 3) Decomposition of containers with a larger degree. Since a container with a larger degree can be split into several containers with smaller degrees, in the case of container combination failure, this embodiment attempts to decompose a container with a larger degree. Since the decomposition process is asynchronous, it will not increase the scheduling overhead of the current request. In addition, in order to avoid jitter caused by frequent decomposition / merge, this embodiment requires that the number of containers with this degree must be no less than two, or the container is the container with the maximum degree on the current node.

[0067] 4) Cold start a container with a specified degree of cold start. When the existing container cannot meet the demand, the container can only be pulled up by cold start.

[0068] As can be seen from the above, in this embodiment, there are two opportunities for container merging: one is during cold start, when all the pulled containers are the result of merging the basic containers; the other is when the container combination is frequently triggered, by merging the accessed containers, the probability of direct allocation can be increased, which can not only reduce the scheduling overhead, but also save more memory resources. There are also two opportunities for container decomposition: one is when the container decomposition is triggered, the large-degree container is decomposed into small-degree containers; the other is that the container is cleaned up by the system at the end of the keep-alive time because it has not been accessed for a long time.

[0069] Step S200, in response to receiving a task scheduling request, determining a deployment node for the task according to the amount of intermediate data generated by the task scheduling request and the task parallelism.

[0070] Considering the limited resources of a single node, the goal of the task scheduler is to maximize the use of local computing resources while avoiding queuing. By deploying upstream and downstream functions to the same node, the data transmission process through remote storage can be avoided. When the local resource requirements of the current task parallelism are not met, some tasks are deployed to remote nodes through the load splitting algorithm, so that the total execution delay of the local and remote parts is minimized. In addition, after introducing the container merging / decomposition mechanism, since the container supports the parallel execution of multiple tasks, a suitable scheduling strategy is required to select which loads to route to the same container to improve execution efficiency. Therefore, in this embodiment, the key points provided by the task scheduler are: (1) calculating the resource requirements of the downstream function and deciding the deployment plan based on the local resource status; (2) when remote execution is required, deciding which loads are handed over to the remote node to minimize the total delay of this stage; (3) for the load assigned to the current node, deciding which tasks each container undertakes to efficiently utilize computing resources.

[0071] Figure 4 It is a flow chart showing the process of determining the deployment nodes of tasks in the method for dynamic workflow scheduling of serverless computing in one embodiment of the present application; Figure 4 As shown, in a specific implementation of this embodiment, the step of determining the deployment node of the task according to the amount of intermediate data generated by the task scheduling request and the task parallelism includes the following steps S210 to S240.

[0072] Step S210, calculating the amount of intermediate data corresponding to the current task parallelism;

[0073] Step S220, determines whether the resources of the local computing node meet the resource computing requirements of the intermediate data volume. If the resources of the local computing node meet the resource computing requirements of the intermediate data volume, executes step S230 to deploy the task on the local computing node. If the resources of the local computing node do not meet the resource computing requirements of the intermediate data volume, executes step S240 to split the task and deploy the split part of the task on the remote computing node.

[0074] In serverless computing scenarios, the upper limit of expansion is mainly restricted by various resources, such as CPU, memory, network IO, etc. Taking memory as an example, by pre-collecting the memory required for different task parallelisms, a curve of memory occupancy changes with respect to parallelism is obtained. Since this embodiment introduces a container merging mechanism, the curve is approximately a linear function that does not pass through the origin. Based on this, the corresponding resource requirements under the current parallelism can be calculated, and then the task scheduler compares the local memory capacity with this value. If the requirements are met, all can be deployed locally; otherwise, part of the load is split to remote nodes.

[0075] Figure 5 It is a flowchart showing the task segmentation in the serverless computing dynamic workflow scheduling method in one embodiment of the present application; Figure 5 As shown, in a specific implementation of this embodiment, dividing the task and deploying the divided parts of the task on the remote computing node includes the following steps S241 to S2440.

[0076] Step S241, respectively obtaining the completion time of the task on the local computing node and the completion time of the task on the remote computing node;

[0077] Step S242, enumerate all binary results of the task to form a local task set and a remote task set;

[0078] Step S243, obtaining, according to the remote task set, respective time combinations of the completion time of the task on the local computing node and the completion time of the task on the remote computing node;

[0079] Step S244, obtaining the minimum value of each of the time combinations, dividing the task based on the minimum value, and deploying the divided part of the task on the remote computing node.

[0080] When locality is not satisfied, part of the load needs to be distributed to remote nodes to fully utilize cluster resources to ensure QoS. This embodiment defines the completion time of the local part as T local , the completion time of the remote part is T remote , then the optimization goal of this embodiment is to find MAX(T local ,T remote ). The calculation formulas for both are:

[0081] T local =T exec (AVG(data_size))

[0082] T remote =T exec (AVG(data_size))+T trans (MAX(data_size))

[0083] Where T exec It represents the time required for task processing; T trans It represents the time required to transmit data, and data transmission is only required on remote nodes. Due to the introduction of the container merging mechanism, the computing resources within the same container are shared, and the tasks that are completed first can release computing resources for other unfinished tasks to use. Therefore, the average value of the task group scheduled to the container can be used to estimate the execution time of the task group; and data transmission is generally related to factors such as network bandwidth. In parallel transmission, only the transmission time of the maximum amount of data needs to be considered. Experiments have found that T exec and T trans The values ​​of are strongly correlated with the data size and can be predicted by the linear regression model.

[0084] In the local scheduling feasibility determination, this embodiment can obtain the maximum task parallelism K of the function on the local node. For the total parallelism N, there are NK tasks that need to be migrated to remote execution. In order to find the best splitting method, MAX(T local ,T remote ) is the smallest, this embodiment provides the following algorithm: Enumerate all binary results S of the task set local and S remote , corresponding to the local and remote task sets respectively, and then predict the local part T according to the above regression model local and the execution time T of the remote part remote Finally, take the MAX(Tlocal ,T remote ). The time complexity of this algorithm is O(2 N ), when N is large, the Monte Carlo simulation algorithm is used to approximate it to avoid excessive search overhead.

[0085] Step S300 , distribute all tasks into multiple task groups, and distribute the tasks in the task group to which the tasks scheduled to the current node belong to to containers or container combinations of appropriate degrees.

[0086] In a specific implementation of this embodiment, for the task group to which the task scheduled to the current node belongs, distributing the task to a container or container combination of appropriate degree includes: sorting all tasks to be processed from small to large according to the amount of data to obtain a double-ended task queue; for the container degree D i , respectively take D i / 2 team leader tasks and D i / 2 tail tasks are distributed as container tasks to the corresponding container or container group; repeat the above operation until all tasks are distributed.

[0087] After obtaining the task set that the current node needs to process, it is also necessary to distribute it to several containers. Similarly, this embodiment needs to ensure that the task completion time of each container is as close as possible. Therefore, the distribution of tasks needs to satisfy the load and the degree of the container. For example, the load processed by a container with a degree of 4 is about 4 times that of a container with a degree of 1; in addition, due to the sharing of computing resources within the container, the resources released after the completion of the current task can be used by other tasks. Therefore, deploying long-term tasks and short-term tasks to the same container is beneficial to utilize this feature to speed up the execution process of the entire task set. In order to reduce the complexity of the search, this embodiment provides a heuristic algorithm to approximately achieve the above effect: sort the tasks to be processed from small to large according to the amount of data to obtain a double-ended queue. For container i and its degree D i , respectively take D i / 2 team leader tasks and D i / 2 tail tasks as its load, and repeat the above operation until all tasks are distributed.

[0088] Figure 6 It is a schematic diagram showing the overall execution process of the serverless computing dynamic workflow scheduling method in one embodiment of the present application; Figure 6 As shown, the execution process is as follows:

[0089] 1) User request arrives, triggering workflow execution;

[0090] 2) The task scheduler evaluates the resources required for the current step in the workflow. If the local resource capacity is greater than the resource demand, all tasks are allowed to be scheduled locally, using locality to reduce data transmission overhead. Otherwise, some tasks need to be scheduled to remote nodes, using the computing power of multiple nodes to avoid uneven loads. At the same time, the load splitting algorithm is triggered to decide the load to be allocated to the local / remote node.

[0091] 3) For the nodes assigned tasks, the corresponding containers are first allocated through the resource manager. During this process, the merging / decomposition mechanism may be triggered. Then, the task scheduler triggers the task distribution algorithm to balance the load of each container so that the overall execution delay is minimized.

[0092] 4) After the distribution is completed, the corresponding container executes the task and triggers the execution of the function of the next step of the workflow.

[0093] Repeat 2) to 4) until all steps of the workflow are executed and the user request is completed.

[0094] As can be seen from the above, the serverless computing dynamic workflow scheduling method in this embodiment has the following technical features:

[0095] 1. Form a resource manager based on container merging / decomposition mechanism:

[0096] a. A management method for protecting heterogeneous container pools, maintaining a number of containers of different degrees, and selecting container groups by the minimum cover method when assigning tasks to them, so as to efficiently utilize existing container resources and reduce scheduling overhead;

[0097] b. Protect the merging / decomposition mechanism of the resource manager module based on the container merging / decomposition mechanism. In the process of allocating containers, low-specification containers with high frequency of access will be merged into high-specification containers to improve the effective utilization of memory resources. Or, when there is a lack of containers of corresponding specifications, high-specification containers will be decomposed into low-specification containers to meet the current parallelism requirements;

[0098] 2. Form a resource-load collaboratively aware task scheduler:

[0099] a. Protect the load splitting algorithm of the resource-load collaboratively aware task scheduler module, determine the feasibility of local scheduling of the load based on the local resource capacity and task requirements, and if complete local scheduling cannot be achieved, predict the transmission delay and execution delay based on the delay prediction model, and search for the best way to split the local load and remote load to minimize the overall delay;

[0100] b. Task distribution algorithm of the task scheduler module that protects resources and load coordination. To ensure that the task processing time of each container is as consistent as possible, the task distribution algorithm distributes the load in proportion to the container computing power (container degree). In addition, by adopting a heuristic algorithm, long-term tasks and short-term tasks are reasonably allocated to the same container, thereby achieving task load balance and optimization to a certain extent.

[0101] The serverless computing dynamic workflow scheduling method in this embodiment includes the following steps during specific implementation:

[0102] 1) Request characteristics and resource manager interaction based on container merging / decomposition mechanism: According to the parallelism requirements of the current request, the appropriate combination is selected through container merging / decomposition;

[0103] 2) Interaction between the resource manager and the resource-load collaboratively aware task scheduler based on the container merging / decomposition mechanism:

[0104] a) The resource manager can save memory resources and increase the probability of the task scheduler triggering fully local scheduling;

[0105] b) The container allocation results of the task scheduler affect the resource manager's selection and adjustment of existing containers;

[0106] 3) Task scheduler interaction based on request characteristics and resource-load collaboration awareness:

[0107] a) Based on the data transmission overhead and resource requirements of the current request, the task scheduler will choose a deployment solution that minimizes latency.

[0108] This embodiment has been verified through experiments that this embodiment can significantly improve the overall throughput of the system while optimizing the end-to-end delay of requests. Latency refers to the time it takes for a request to be sent and completed. The lower the latency, the better the user experience. Throughput refers to the maximum number of requests that can be completed per unit time in the system. The higher the throughput, the more stable the system performance under high load.

[0109] The comparison objects of the experiment include load balancing first, locality first, static graph segmentation (FaaSFlow), and the dynamic graph segmentation scheduling system (FaaStep) proposed in this embodiment. The experimental applications selected video analysis tasks, image processing tasks, and data extraction tasks from the best practice cases. In terms of hardware configuration, the experiment used a machine equipped with an Intel(R) Xeon(R) Platinum 8369B CPU@2.70GHz, 8 cores, and 16GB of memory as a management node, and equipped with 4 machines of the same specification as working nodes.

[0110] For each application, this embodiment tests at low, medium, and high parallelism to simulate the dynamic nature of requests. Low parallelism has the lowest resource requirements, followed by medium parallelism, and high parallelism has the highest resource requirements. The arrival time distribution of requests refers to the sample distribution in Azure Functions Trace 2019, and requests are generated evenly at the above three parallelisms, and their performance on different scheduling systems is tested.

[0111] The experimental results are as follows Figures 7a to 7c As shown, compared with the three scheduling systems of load balancing priority, locality priority and static graph segmentation, the scheduling system of this embodiment performs better in the following tasks:

[0112] Video analysis task: average latency was reduced by 42%, 20%, and 27% respectively;

[0113] Image processing tasks: average latency was reduced by 25%, 17%, and 15% respectively;

[0114] Data extraction task: The average latency was reduced by 45%, 35%, and 38% respectively.

[0115] The main reason for this performance improvement is that the system of this embodiment can better balance local and remote execution. Dynamic scheduling through real-time state perception not only adapts to the dynamic characteristics of the workflow, but also effectively avoids the problem of excessive data transmission overhead and local node overload.

[0116] This embodiment measures the maximum throughput (RPM, i.e., the number of requests completed per minute) of the system by starting multiple client threads and sending requests to the scheduling system concurrently. Specifically, each thread sends the next request immediately after completing the current request. By gradually increasing the number of client threads, the performance of the system under different load conditions can be observed, and its maximum throughput capacity can be ultimately determined.

[0117] The experimental results are as follows Figures 8a to 8c As shown in the figure, compared with the three scheduling systems of load balancing priority, locality priority and static graph segmentation, this system shows significant throughput improvement in the following tasks:

[0118] Video analysis task: throughput increased by 182%, 41%, and 33% respectively;

[0119] Image processing tasks: throughput increased by 15%, 23%, and 10% respectively;

[0120] Data extraction task: throughput increased by 37%, 30%, and 24% respectively.

[0121] This performance improvement is due to the container decomposition / merge management mechanism introduced in this system. This mechanism can effectively reduce resource waste and significantly improve the parallel processing capabilities of nodes, thus better adapting to high-load scenarios.

[0122] The serverless computing dynamic workflow scheduling method of this embodiment does not need to invade user code or obtain user data. The core of the system consists of two key modules: a resource manager based on the container merging and decomposition mechanism and a task scheduler with resource-load collaborative perception capabilities. This embodiment provides a practical solution for the processing technology of dynamic workflow tasks in serverless computing scenarios, which not only effectively improves the user service quality, but also optimizes the resource deployment cost of cloud service providers to a certain extent, laying the foundation for the practical application of related technologies.

[0123] The protection scope of the server-less computing dynamic workflow scheduling method described in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing, or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0124] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the server-less computing dynamic workflow scheduling method provided in any embodiment of the present application is implemented.

[0125] In the embodiment of the present application, any combination of one or more storage media can be used. The storage medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0126] An embodiment of the present application also provides an electronic device. Fig. 9Shown is a structural diagram of an electronic device 100 provided in an embodiment of the present application. In some embodiments, the electronic device may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) and other terminal devices. In addition, the service-free computing dynamic workflow scheduling method provided in the present application can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. The embodiment of the present application does not impose any restrictions on the specific application scenarios of the service-free computing dynamic workflow scheduling method.

[0127] like Fig. 9 As shown, the electronic device 100 provided in the embodiment of the present application includes a memory 101 and a processor 102 .

[0128] The memory 101 is used to store computer programs; preferably, the memory 101 includes: ROM, RAM, disk, USB flash drive, memory card or CD-ROM and other media that can store program codes.

[0129] Specifically, the memory 101 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 100 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 101 may include at least one program product having a set (e.g., at least one) of program modules, which are configured to perform the functions of the various embodiments of the present application.

[0130] The processor 102 is connected to the memory 101 and is used to execute the computer program stored in the memory 101 so that the electronic device 100 executes the serverless computing dynamic workflow scheduling method provided in any embodiment of the present application.

[0131] Optionally, the processor 102 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0132] Optionally, the electronic device 100 in this embodiment may further include a display 103. The display 103 is communicatively connected with the memory 101 and the processor 102, and is used to display a GUI interaction interface related to the serverless computing dynamic workflow scheduling method.

[0133] In summary, the present application can solve the problem of service quality degradation caused by high variability of computing resources and intermediate data volume in dynamic workflow scenarios, achieve the goal of minimizing delay, and at the same time minimize resource waste, effectively improve computing node resource utilization, effectively increase load throughput, and reduce end-to-end delay. Therefore, this embodiment effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0134] The above embodiments are merely illustrative of the principles and effects of the present embodiment, and are not intended to limit the present embodiment. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present embodiment. Therefore, all equivalent modifications or changes made by those of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present embodiment should still be covered by the claims of the present embodiment.

Claims

1. A serverless computing dynamic workflow scheduling method, characterized by: The method comprises: Based on the frequency of resource access, the function containers on the computing nodes are periodically merged and decomposed; In response to receiving a task scheduling request, determining a deployment node for the task according to an amount of intermediate data generated by the task scheduling request and a task parallelism; All tasks are distributed into multiple task groups, and for the task group to which the task scheduled to the current node belongs, the task is distributed to a container or container combination of a suitable degree.

2. The method for dynamic workflow scheduling of serverless computing according to claim 1, characterized in that: The merging and decomposing of the function containers on the computing nodes includes: Control a single container to process a set of tasks in multi-process multi-request parallel processing mode; Containers of different degrees are composed of a heterogeneous container pool, and a corresponding container group is allocated to the current task parallelism based on the current task parallelism requirement and the heterogeneous container pool; The function containers in each container group on the computing node are merged and decomposed according to the container degree and access frequency.

3. The method for dynamic workflow scheduling of serverless computing according to claim 2, characterized in that: The method of allocating the corresponding container group for the current task parallelism based on the current task parallelism requirement and the heterogeneous container pool is: Among them, N is the current parallelism requirement, D i is the number of tasks that container i can support for parallel execution, that is, the container degree, and K is the minimum number of containers in the heterogeneous container pool for the current parallelism requirement N.

4. The method for dynamic workflow scheduling of serverless computing according to claim 2, characterized in that: The merging of the function containers in each container group on the computing node according to the container degree and the access frequency includes: If there is a container whose degree matches the current parallelism requirement, the container is allocated for the current task parallelism; If there is a container whose degree is less than the current parallelism requirement, a container combination consisting of at least two containers is searched from the heterogeneous container pool based on the access frequency, and the total container degree of the container combination matches the current parallelism requirement, and the container combination is allocated for the current task parallelism.

5. The method for dynamic workflow scheduling of serverless computing according to claim 4, characterized in that: Decomposing the function containers in each container group on the computing node according to the container degree and the access frequency includes: In response to a failure in the container combination search, obtaining a container to be decomposed from the heterogeneous container pool based on access frequency and container degree; Decomposing the container to be decomposed into at least two containers, and combining one of the containers with other containers in the heterogeneous container pool to form the container combination; In response to the access frequency of the container being lower than an access frequency threshold, the container is decomposed.

6. The method for dynamic workflow scheduling of serverless computing according to claim 1, characterized in that: The step of determining the deployment node of the task according to the amount of intermediate data generated by the task scheduling request and the task parallelism comprises: Calculate the amount of intermediate data corresponding to the current task parallelism, and determine whether the resources of the local computing node can meet the intermediate data volume. If the resources of the local computing node can meet the resource computing requirements of the intermediate data volume, deploy the task on the local computing node. If the resources of the local computing node cannot meet the resource computing requirements of the intermediate data volume, split the task and deploy the split parts on the remote computing node.

7. The method for dynamic workflow scheduling of serverless computing according to claim 6, characterized in that: Splitting the task and deploying the split parts of the task on the remote computing node includes: Get the task completion time on the local computing node and the task completion time on the remote computing node respectively; Enumerate all binary results of the task to form a local task set and a remote task set; According to the remote task set, respectively obtain various time combinations of the completion time of the task on the local computing node and the completion time of the task on the remote computing node; The minimum value of each time combination is obtained, the task is divided based on the minimum value, and the divided part of the task is deployed on the remote computing node.

8. The method for dynamic workflow scheduling of serverless computing according to claim 1 or 7, characterized in that: For the task group to which the task scheduled to the current node belongs, the task is distributed to a container or container combination of appropriate degree, including: Sort all tasks to be processed from small to large according to the amount of data to obtain a double-ended task queue; For container degree D i , respectively take D i / 2 team leader tasks and D i / 2 tail tasks are distributed as container tasks to the corresponding container or container group; Repeat the above steps until all tasks are distributed.

9. An electronic device, characterized in that: include: comprising a memory for storing a computer program; A processor is used to run the computer program to implement the steps of the serverless computing dynamic workflow scheduling method as described in any one of claims 1 to 8.

10. A computer storage medium storing program instructions, characterized in that: When the program instructions are executed, the steps of the serverless computing dynamic workflow scheduling method as described in any one of claims 1 to 8 are implemented.

Images