Dynamic and static combined service-free calculation function workflow flexible deployment method and system

Abstract

The invention discloses a dynamic and static combined service-free computing function workflow flexible deployment method and system, and relates to the field of service-free computing. The problems that in the prior art, resource requirements of different function types in the service-free calculation function workflow are not fully considered, and the sensitivities to resource fluctuation in a dynamic cluster environment are different are solved. The method comprises the following steps: extrapolating task execution time by adopting reference node operation data and a random forest regression model; the predicted task execution time is input into an SLO-based heterogeneous cluster task scheduling algorithm for task scheduling; a scheduling result is input into a task serial algorithm on a node, and a plurality of small functions are combined into fewer larger function execution units through serialization; and monitoring in real time and dynamically implementing dynamic merging and optimization re-scheduling of a plurality of DAG tasks based on dynamic task scheduling of the shortest stage completion time. The method is suitable for the field of no-service computing, no-service edge computing and server cluster resource scheduling and optimization.

Description

Technical Field

[0001] The present invention relates to the field of serverless computing, and in particular to a method for flexible deployment of serverless computing function workflows that combines dynamic and static operations. Background Art

[0002] With the continuous evolution of cloud computing technology and the increasing popularity of containerized deployment models, cloud platform service architectures are gradually evolving from traditional virtualized infrastructure to higher-level computing abstractions. As a core implementation paradigm for serverless computing, Function-as-a-Service (FaaS) has garnered widespread attention due to its advantages, including on-demand billing, elastic scalability, and event-driven nature. By decomposing applications into independent functions, FaaS reduces system coupling and simplifies operations and maintenance processes. It has been applied in scenarios such as log analysis, the Internet of Things, edge computing, and real-time computing.

[0003] In complex businesses, developers typically organize multiple functions into directed graphs to build function workflows, implementing multi-stage processing logic. Compared to single function execution, function workflows involve more resource scheduling and task dependency management, making their performance more susceptible to fluctuations in underlying resources. Currently, mainstream platforms mostly adopt static deployment strategies. Although research has introduced process sharing, thread isolation, and container reuse mechanisms to alleviate resource waste, these scheduling strategies still face problems such as poor adaptability and insufficient flexibility in scenarios with heterogeneous resources and high concurrent loads.

[0004] To address the problem of lack of adaptability of static deployment strategies, the present invention designs a dynamic and static combined function workflow scheduling method and constructs a resource-aware scheduling method. This method completes the initial function combination division based on static analysis, and dynamically reconstructs the task graph in combination with the node resource status during the function running phase, thereby enhancing the flexibility of the scheduling strategy and the resource matching capability. Summary of the Invention

[0005] This invention aims to address the problem that existing technologies fail to fully consider the resource requirements of different function types in serverless computing function workflows and their varying sensitivities to resource fluctuations in dynamic cluster environments. To achieve this objective, the invention provides the following technical solutions: Solution 1: The present invention provides a method for flexible deployment of serverless computing function workflows that combines dynamic and static scheduling. This method is a heuristic static scheduling method, thereby providing an efficient initial layout for the entire workflow. The method includes the following steps: Step 1: Use the benchmark node running data and the random forest regression model to extrapolate the task execution time; Step 2: Input the predicted task execution time into the SLO-based heterogeneous cluster task scheduling algorithm for task scheduling; Step 3: Input the scheduling results from step 2 into the task serialization algorithm on the node. Through serialization, multiple small functions are combined into fewer larger function execution units. This can reduce the number of function instance creations and cross-function call overhead, improve overall resource utilization, reduce latency, and increase throughput, thereby better meeting the performance requirements of actual applications. Step 4: Dynamic task scheduling based on the shortest stage completion time monitors and dynamically implements dynamic merging and optimized rescheduling of multiple DAG tasks in real time.

[0006] Furthermore, in another preferred embodiment, the method for extrapolating task execution time using the benchmark node operating data and the random forest regression model in step 1 is specifically as follows: Step 101: Collect the features of the function running on the standard machine and build a joint feature set based on the hardware resources of the node; Step 102: Use the joint feature set to train a random forest regression model; Step 103: Predict the execution time of the function on machines with different performance based on the random forest regression model.

[0007] Furthermore, there is a preferred embodiment in which the construction of the above-mentioned joint feature set includes two dimensions: computing resource parameters (such as CPU type, main frequency, memory size) and function operation characteristics (computing load, I / O times). The multi-dimensional construction of features facilitates efficient training of subsequent models.

[0008] Furthermore, in another preferred embodiment, the task scheduling reasoning method in step 2 is specifically as follows: Step 201: Group the sequential functions in the serverless computing function workflow into parallel stages, define the maximum distance of the path from the starting node to any node, and divide the nodes into several parallel execution stages according to the maximum distance; Step 202: For each divided parallel stage, ensure that the function tasks in all stages are completed before executing the function tasks in the next stage. The tasks in each stage can be safely executed in parallel. Step 203: Based on the predicted execution time and node cost, find the default deployment node that meets the execution requirements and has the lowest cost. Repeatedly optimize node selection for key tasks to ensure that the task allocation of each stage reaches the optimal state. Calculate the difference between the actual maximum execution time of the stage and the SLO, and allocate this redundant time to the next stage to balance the execution time of each stage.

[0009] Further, there is a preferred embodiment of the above stage division method, the maximum distance definition is clear, which ensures the effectiveness of stage division and the safety of task execution sequence, and provides a reliable basis for subsequent resource allocation and task scheduling.

[0010] Further, there is a preferred embodiment of the above step 3, which is specifically as follows: Step 301, for the task set, the maximum execution time of a single task is calculated; Step 302, the total execution time of all tasks is determined, and the appropriate scheduling length is calculated based on this; Step 303, initialize the processor load, and sequentially allocate and combine the tasks until all tasks are scheduled.

[0011] Further, there is a preferred embodiment of the above task serialization process, which can dynamically monitor the load of each processor, and according to the real-time monitoring result, flexibly adjust the task allocation strategy to better balance the load between processors; at the same time, historical scheduling data can be introduced to learn the task execution mode, so as to further optimize the scheduling strategy of subsequent tasks and improve the overall task execution efficiency and resource utilization.

[0012] Further, there is a preferred embodiment of the above step 3, which is specifically as follows: Step 4, dynamic task scheduling based on the shortest completion time of the stage, real-time monitoring and dynamic implementation of the dynamic merging and optimization rescheduling of multiple DAG tasks: Step 401, detect the resource inadaptation condition of the static scheduling scheme in the execution process, redefine and adjust the task execution plan.

[0013] Step 402, identify and recombine the timeout tasks into a new DAG graph for rescheduling to ensure real-time performance and resource adaptability. Multiple concurrent DAG tasks are merged through virtual start and end nodes to unify task management. Step 403, predict the task execution time and communication cost to determine the task priority. According to the priority, the task is rescheduled to optimize system performance and resource utilization efficiency.

[0014] Further, the above dynamic task scheduling method can realize real-time sensing of resource changes and load state, significantly improve the cluster resource utilization efficiency and task execution stability, reduce the overall delay and enhance the system throughput, especially suitable for heterogeneous and dynamically fluctuating resource environment Solution 2: The method for flexible deployment of a serverless computing function workflow combining dynamic and static functions described in the present invention can be fully implemented using computer software. Therefore, correspondingly, the present invention also provides a flexible deployment system for a serverless computing function workflow combining dynamic and static functions, the system comprising: The task execution time prediction module is used to collect function operation characteristics and node hardware resource information, build a feature set and train a random forest model to predict the execution time of the function on nodes with different performance; The task scheduling optimization module inputs the predicted function execution time into the task scheduling algorithm based on the service level objective (SLO), divides the task into parallel stages, and determines the initial deployment node for task execution; The task serialization optimization module adopts a serialization strategy to combine multiple small functions into larger execution units, optimize resource allocation and task execution efficiency, reduce latency and improve overall throughput to achieve efficient task execution.

[0015] Solution 3. The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes any one of the above-mentioned methods for flexible deployment of dynamic and static combined server-less computing function workflows.

[0016] Solution 4. The present invention also provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes any one of the above-mentioned methods for flexible deployment of dynamic and static combined server-less computing function workflows.

[0017] The beneficial effects of the present invention are as follows: This paper proposes a combined dynamic and static task deployment method. This method addresses the performance deficiencies of existing resource scheduling strategies under conditions of resource heterogeneity and load fluctuation. By combining random forest prediction, task phase partitioning, and serialized optimization, it dynamically adapts to node resource changes, achieving efficient task deployment and execution. Through dynamic monitoring and load adjustment strategies, it further improves resource utilization and task execution efficiency, significantly reducing execution latency and service level objective (SLO) violations.

[0018] Furthermore, compared with existing methods, the present invention can better adapt to dynamic changes in computing resource environments in real time, not only improving the accuracy of task execution, but also building a more efficient task execution system, reducing resource waste, and improving the overall system operation efficiency.

[0019] The present invention is applicable to the fields of server-less computing, server-less edge computing, and server cluster resource scheduling and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the specific embodiments or prior art of the present application, the drawings required to be used in the description of the specific embodiments or prior art will be briefly introduced. Obviously, the drawings described below are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 The flow chart of the dynamic and static combined service-free computing function workflow flexible deployment method of the present application; Figure 2 The overall framework entity diagram of the dynamic and static combined service-free computing function workflow flexible deployment method of the present application; Figure 3 The static scheduling method flow chart of the present application; Figure 4 The new DAG composition diagram of residual tasks of the present application; Figure 5 The rescheduling diagram of merging multiple DAGs of the present application; Figure 6 The comparison diagram of the scheduling effect of the service-free workflow of the present application. DETAILED DESCRIPTION

[0022] The specific embodiments of the present application will be further described in detail below with reference to the drawings. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application, and these changes and improvements are within the scope of protection of the present application.

[0023] Embodiment one, see Figure 1 and Figure 2 This embodiment describes a dynamic and static combined service-free computing function workflow flexible deployment method, which is used to solve the problem that the traditional deployment strategy in the prior art cannot dynamically adapt to resource changes, resulting in performance fluctuations and low resource utilization efficiency.

[0024] As Figure 1 shown, the method specifically comprises the following steps: Step 1, using benchmark node running data and random forest regression model to extrapolate task execution time; Step 2, inputting the predicted task execution time into a heterogeneous cluster task scheduling algorithm based on service level objective (SLO) for task scheduling; Step 3, input the scheduling result in step 2 into the task serial algorithm on the node, optimize by serialization, reduce the frequency of function instance creation and cross-function call overhead; Step 4, based on the dynamic task scheduling of the shortest completion time of the stage, real-time monitoring and dynamic implementation of the dynamic merging and optimization rescheduling of multiple DAG tasks to ensure the continuous optimization of the overall performance. In actual application, according to the overall framework of the dynamic and static combined service-free computing function workflow flexible deployment method as shown in Figure 2 The embodiment solves the problem that the existing resource scheduling strategy cannot dynamically adapt to resource changes, resulting in low task execution efficiency and insufficient resource utilization. The embodiment accurately predicts the function execution time through a random forest regression model, and reasonably divides and optimally schedules tasks in combination with service level objectives (SLO). At the same time, the embodiment uses real-time resource monitoring tools to dynamically monitor the cluster state, adjusts the task execution plan in real time through effective task serialization and dynamic scheduling optimization strategies, significantly improves the execution efficiency of tasks and the overall utilization of resources, and thus realizes efficient management of resources and stable execution of tasks in a dynamically changing environment.

[0025] Embodiment two, the embodiment is a specific description of the dynamic and static combined service-free computing function workflow flexible deployment method described in embodiment one: Step 1, use benchmark node running data and random forest regression model to extrapolate task execution time; specifically: Step 101, collect function running data on standard nodes, combine node computing time, I / O time, I / O operation times, CPU architecture and memory capacity and other hardware resource information to construct a joint feature set.

[0026] Step 102, train the random forest regression model using the constructed joint feature set.

[0027] Step 103, use the trained random forest regression model to predict the execution time of the function on different nodes.

[0028] Further, the feature set includes computing resource parameters (CPU type, frequency, memory size) and function running features (computing load, I / O times).

[0029] Step 2, input the predicted task execution time into the heterogeneous cluster task scheduling algorithm based on service level objectives (SLO) for task scheduling; specifically: Step 201, divide the tasks in the function workflow into stages according to the maximum distance from the starting node to any node path, forming a stage structure for parallel execution.

[0030] Step 202, combine the predicted execution time and node cost, select the node with the lowest cost and performance meeting the requirements as the initial deployment node of the task.

[0031] Step 203, calculate and adjust the redundancy time of the difference between the longest task executed in the stage and the SLO, realize the optimization and balance of task execution.

[0032] Further, the above node selection and adjustment process dynamically adapts to the real-time monitored resource state.

[0033] Step 3, task serialization optimization method; specifically: Step 301, calculate the maximum execution time of a single task for the task set.

[0034] Step 302, determine the total execution time of all tasks, and calculate the appropriate scheduling length based on this.

[0035] Step 303, initialize the processor load, and sequentially allocate and serially combine tasks until all tasks are scheduled.

[0036] Further, dynamically monitor the processor load, and real-time adjust the task serial combination strategy to improve resource utilization.

[0037] Step 4, dynamic task scheduling based on the shortest completion time of the stage, real-time monitor and dynamically implement the dynamic merging and optimization rescheduling of multiple DAG tasks; specifically: Step 401, detect the resource inadaptation of the static scheduling scheme in the execution process, redefine and adjust the task execution plan.

[0038] Step 402, identify and recombine the timeout tasks into a new DAG graph, merge multiple concurrent DAG tasks through virtual start and end nodes, unified task management, and ensure real-time and resource adaptability.

[0039] Step 403, reschedule tasks according to priority, optimize system performance and resource utilization efficiency.

[0040] Further, the above embodiment improves resource utilization efficiency, reduces execution delay and task execution cost, and improves the stability and adaptability of overall system performance, especially suitable for service-free computing scenarios in heterogeneous resources and dynamic environments.

[0041] Embodiment three, the embodiment proposes a dynamic and static combined service-free computing function workflow deployment system, the system comprises: A task execution time prediction module is responsible for collecting function running data on the benchmark node, and extrapolating the execution time of the task on various performance nodes using a random forest regression model. The heterogeneous cluster task scheduling module combines the predicted task execution time with the service level objective (SLO) to perform task phase division and node deployment optimization to ensure the order and security of task execution; The task serialization optimization module combines task collections in series, effectively reducing the frequency of function instance creation and the overhead of cross-function calls, thereby improving resource utilization efficiency. The multi-task dynamic merging and rescheduling module monitors cluster resource status in real time, quickly responds to resource fluctuations and task execution anomalies, redefines task execution plans, merges multiple concurrent DAG tasks, optimizes task execution sequence and node resource allocation, and achieves continuous optimization of overall performance.

[0042] Implementation method 4: This implementation method proposes a computer device, including a memory and a processor, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the flexible deployment method of the dynamic and static combined server-less computing function workflow in the above implementation method.

[0043] Implementation method five: This implementation method proposes a computer-readable storage medium, which stores a computer program. When the processor runs the computer program, the processor executes any one of the dynamic and static combined server-less computing function workflow flexible deployment methods in the above implementation methods.

[0044] Implementation 6: This implementation provides a specific example, which is used to explain the above implementation. Specifically, the example is as follows: Combine Figures 1 to 5 This implementation proposes a flexible deployment method for serverless computing function workflows that combines dynamic and static methods, aiming to improve the execution efficiency and resource utilization of serverless computing function workflows. Specifically, the method includes the following steps: This implementation adapts to dynamic changes in cluster resources in real time, enabling efficient deployment and execution of serverless computing function workflows. First, a random forest regression model is used to accurately predict task execution times from benchmark node performance data, effectively reducing data collection costs and time. Second, a static task scheduling method is employed to divide function tasks into multiple parallel stages to meet the service level objectives (SLOs) for task execution. Task deployment is then performed by selecting the most cost-effective nodes.

[0045] Furthermore, this implementation utilizes a task serialization optimization strategy to reduce the number of function instance creations and cross-function call overhead, thereby improving overall resource utilization efficiency. Furthermore, dynamic task scheduling based on real-time resource status monitoring can quickly identify and address resource adaptation issues in static solutions, dynamically reorganizing task execution plans.

[0046] The dynamic and static combined flexible deployment method of the service-free computing function workflow realizes efficient use of resources and stability of task execution through three key links of static prediction, task serialization and dynamic monitoring and rescheduling, significantly improves the task execution performance and the overall response ability of the system, and ensures the scientificity and rationality of the task deployment scheme.

[0047] In combination Figures 1 to 5 , the embodiment proposes a dynamic and static combined flexible deployment method of the service-free computing function workflow, which comprises the following steps: Step 1, using benchmark node running data and random forest regression model to extrapolate task execution time; The benchmark node running data and random forest regression model to extrapolate task execution time specifically are: Step 101, collect the running data of the function on the standard node, and combine the computing time, I / O time, I / O operation times, CPU architecture and memory capacity and other hardware resource information of the node to construct a joint feature set.

[0048] Step 102, training the random forest regression model by using the constructed joint feature set.

[0049] Step 103, predicting the execution time of the function on different performance nodes.

[0050]

[0051] Further, the above feature set includes computing resource parameters (CPU type, frequency, memory size) and function running features (computing load, I / O times).

[0052] Step 2, input the predicted task execution time into the service level objective (SLO) based heterogeneous cluster task scheduling algorithm for task scheduling; The service level objective (SLO) based heterogeneous cluster task scheduling algorithm for task scheduling specifically is: Step 201, divide the tasks in the service-free computing function workflow into stages according to the maximum distance from the starting node to any node path, form a stage structure for parallel execution. Let represent all stages of the service-free computing function workflow, a total of stages, for any , all function tasks in must be completed before the function tasks in can be executed, thus obtaining the composition of the service-free computing function workflow as Ensure that tasks at each stage are completed in sequence, ensuring the order and security of task execution. According to the definition, it is divided into parallel stages, each stage Contains a certain maximum distance According to the definition of stage division, for any stage For a node in the stage, all its predecessor nodes must be in the previous stage set This is due to the definition is the maximum distance of the path from the starting node to the current node. Therefore, the distance of the predecessor node of the node in the same stage must be smaller than the current node, thereby ensuring that the predecessor node must be located in a more advanced stage.

[0053] Step 202: Combine the predicted execution time and node cost to select the node with the lowest cost and satisfactory performance as the initial deployment node for the task. For function In the machine The predicted execution time on For machines First, find all the The machine with the lowest cost is selected as the default deployment machine.

[0054] Step 203: Repeat the above steps for all key tasks in the current phase until all tasks are assigned to the machine with the best execution time. Calculate the actual maximum execution time of the current phase. The difference between the SLO and the project time is allocated to the next phase.

[0055] Step 3: Task serialization optimization method; The above task serialization optimization method is specifically as follows: Step 301: Calculate the maximum execution time of a single task for a task set.

[0056]

[0057] Step 302: Determine the total execution time of all tasks and calculate the appropriate scheduling length based on the total execution time of all tasks.

[0058]

[0059] in is the total execution cost of the stage, and are the current stage and target number of processors, respectively.

[0060] Step 303: Initialize the processor load, and sequentially distribute and serially combine tasks until all tasks are scheduled.

[0061] Step 4: Dynamic task scheduling based on the shortest stage completion time, real-time monitoring and dynamic implementation of dynamic merging and optimized rescheduling of multiple DAG tasks; The above dynamic task scheduling based on the shortest stage completion time monitors and dynamically implements the dynamic merging and optimized rescheduling of multiple DAG tasks in real time as follows: Step 401: Detect resource mismatch during the execution of the static scheduling scheme, and redefine and adjust the task execution plan. Initially planned as a serial scheduling strategy, where tasks At the node The execution time is recorded as If some functions are combined The following conditions are met:

[0062] It means the node The schedule for this part is expected to exceed the SLO constraint. If the original plan is continued, the SLO will be violated and other node resources will be idle and not fully utilized. Step 402: Identify and reorganize the timed tasks into a new DAG graph and reschedule to ensure real-time performance and resource adaptability. Figure 4 As shown in the figure, the task set of the timeout part is separated from the original serialization scheme and reorganized into a new DAG. The reorganized DAG is defined as ,in v A node set represents a function task. e The edge set represents the dependency between tasks and performs dynamic scheduling. Through virtual start and end nodes, multiple concurrent DAG tasks are merged and task management is unified. By introducing virtual start nodes and virtual termination nodes , construct the merged DAG as .

[0063] Step 403: Predict task execution time, communication cost, and SLO, and determine task priority. Define the stage SLO as the latest completion time of each task, denoted as The priority of a task is calculated based on the following factors: the variance of each function's execution time on different nodes, communication latency, and the latest completion time. The larger the variance, the tighter (smaller) the latest completion time, and the longer the communication latency, the higher the priority. The formula for calculating task priority is as follows:

[0064] Reschedule tasks according to priority to optimize system performance and resource utilization efficiency.

[0065] This implementation method targets the field of dynamic serverless computing function workflow deployment. By integrating random forest regression model prediction, static task scheduling, and dynamic task monitoring and rescheduling strategies, it uses intelligent and quantitative methods to significantly improve task deployment efficiency and resource utilization accuracy, optimize system responsiveness, promote the development of serverless computing resource management technology, and promote the refinement and intelligent advancement of resource scheduling in heterogeneous computing environments.

[0066] The specific implementation is as follows: (1) Implementation methods

[0067] Combine Figures 1 to 5 This embodiment proposes a flexible deployment method for serverless computing function workflows that combines dynamic and static operations. The method includes the following steps: In step 101, run the function task on the standard machine ,collect its operation characteristics such as computing time, I / O time, and number of I / O operations, and build a joint feature set based on the node’s hardware resources such as CPU architecture, CPU frequency, and memory size.

[0068]

[0069] in Indicates characteristics such as I / O times, computing time and data volume, Indicates hardware parameters such as CPU architecture type, L3 cache size, CPU frequency, memory capacity and memory type. Then train the random forest model M Input feature vector Execution time Make predictions.

[0070] In step 102, the random forest regression model is trained using the constructed joint feature set.

[0071] In step 103, use the trained random forest model M Predict the execution time of a function on different performance nodes.

[0072]

[0073] In step 201, the tasks in the serverless computing function workflow are divided into stages according to the maximum distance from the starting node to any node path, forming a stage structure for parallel execution. Represents all stages of the serverless computing function workflow, totaling stage, for any , mandatory regulations All function tasks in the function must be completed before execution can begin The function task of the serverless computing function workflow is composed of Ensure that tasks at each stage are completed in sequence, ensuring the order and security of task execution. According to the definition, it is divided into parallel stages, each stage Contains a certain maximum distance According to the definition of stage division, for any stage For a node in the stage, all its predecessor nodes must be in the previous stage set This is due to the definition is the maximum distance of the path from the starting node to the current node. Therefore, the distance of the predecessor node of the node in the same stage must be smaller than the current node, thereby ensuring that the predecessor node must be located in a more advanced stage.

[0074] In step 203, the node with the lowest cost and satisfactory performance is selected as the initial deployment node of the task based on the predicted execution time and node cost. For function In the machine The predicted execution time on For machines First, find all the The machine with the lowest cost is selected as the default deployment machine. .

[0075] Optimize the execution of critical tasks, For the default machine The longest execution time of the task. For each function f ,have:

[0076] Repeat the above steps for all critical tasks in the current phase until all tasks are assigned to the machine with the best execution time. Calculate the actual maximum execution time of the current phase Difference from SLO , and allocate this time to the next stage.

[0077] In step 301 , the maximum execution time of a single task is calculated for a task set.

[0078]

[0079] In step 302 , the total execution time of all tasks is determined, and based on this, a suitable schedule length is calculated.

[0080]

[0081] in is the total execution cost of the stage, and are the current stage and target number of processors, respectively.

[0082] Initialize the calculation amount in step 303 , processor index p = L . Schedule the tasks in sequence: The remaining length of the current processor is: r=L- , if the current task execution time , in the current processor p Upper assigned task interval , update the calculation amount ; If the current task execution time ; Switch to the next processor and reset the computation , and update the processor index p = p + 1 , in the processor Upper assigned task interval , update the calculation amount , repeat the above process until all tasks are scheduled.

[0083] Step 4: Dynamic task scheduling based on the shortest stage completion time, real-time monitoring and dynamic implementation of dynamic merging and optimized rescheduling of multiple DAG tasks; The above-mentioned dynamic task scheduling based on the shortest stage completion time, real-time monitoring and dynamic implementation of the dynamic merging and optimized rescheduling method of multiple DAG tasks are as follows: Step 401: Detect resource mismatch during the execution of the static scheduling scheme, and redefine and adjust the task execution plan. Initially planned as a serial scheduling strategy, where tasks At the node The execution time is recorded as If some functions are combined The following conditions are met:

[0084] It means the node The schedule for this part is expected to exceed the SLO constraint. If the original plan is continued, the SLO will be violated and other node resources will be idle and not fully utilized. Step 402: Identify and reorganize the timed tasks into a new DAG graph and reschedule to ensure real-time performance and resource adaptability. Figure 4As shown in the figure, the task set of the timeout part is separated from the original serialization scheme and reorganized into a new DAG. The reorganized DAG is defined as ,in v A node set represents a function task. e The edge set represents the dependency between tasks and performs dynamic scheduling. Through virtual start and end nodes, multiple concurrent DAG tasks are merged and task management is unified. By introducing virtual start nodes and virtual termination nodes , construct the merged DAG as .

[0085] Step 403: Predict task execution time, communication cost, and SLO, and determine task priority. Define the stage SLO as the latest completion time of each task, denoted as The priority of a task is calculated based on the following factors: the variance of each function's execution time on different nodes, communication latency, and the latest completion time. The larger the variance, the tighter (smaller) the latest completion time, and the longer the communication latency, the higher the priority. The formula for calculating task priority is as follows:

[0086] Reschedule tasks according to priority to optimize system performance and resource utilization efficiency.

[0087] This implementation method targets the field of dynamic serverless computing function workflow deployment. By integrating random forest regression model prediction, static task scheduling, and dynamic task monitoring and rescheduling strategies, it uses intelligent and quantitative methods to significantly improve task deployment efficiency and resource utilization accuracy, optimize system responsiveness, promote the development of serverless computing resource management technology, and promote the refinement and intelligent advancement of resource scheduling in heterogeneous computing environments.

[0088] (2) Experimental verification.

[0089] See also Figure 6 shown.

[0090] In this implementation, we focus on comprehensively validating and evaluating the performance of the proposed serverless computing function workflow deployment method. Since there is no widely accepted standard comparison method in the serverless computing field, we employed a phased experimental design based on standardized historical benchmark data and real-world application scenarios to fully verify the effectiveness and practicality of our method.

[0091] The cloud cluster is built on the cloud platform for experiments. The standardized service-free computing benchmark test sets such as SLapp, finra-5, finra-50 and finra-100 are selected for comparison and verification. The two typical static scheduling methods, Faastlane and Chiron, are deployed to ensure the comparability of the experimental environment and results. Each group of experiments runs for 30 minutes. The mean value is obtained by multiple measurements to reduce the error caused by system fluctuations. The measurement results include the entire process from request initiation to task completion.

[0092] As shown in Figure 6 Specifically, the SLApp benchmark includes 2 stages and 7 functions. Each stage shows similar delay characteristics, but the workload types are divided into CPU-intensive, disk I / O-intensive and network I / O-intensive. The financial industry supervision (FINRA) benchmark includes 2 stages and is configured with different numbers of parallel function tasks, 5, 50, 100 and 200 respectively.

[0093] To solve the problems of excessive resource consumption and insufficient deployment flexibility in dynamic, heterogeneous and high-concurrency execution environments, the embodiment proposes a dynamic and static combined service-free computing function workflow flexible deployment method. By distinguishing and grouping tasks with different functional characteristics, the adaptability in a heterogeneous resource environment is improved. In the dynamic scheduling stage, the function combination and deployment strategy are dynamically adjusted according to the real-time cluster resource state and the level of concurrent requests, which effectively optimizes the overall performance and enhances the system flexibility. The experimental results show that the method improves the task execution performance by an average of 38%, and reduces the CPU resource consumption by an average of 25%, which fully verifies the actual deployment effect and performance advantage of the method In the future, we will consider the platform adaptability and study how to realize the cross-platform reuse and function landing of the core idea of the method without modifying the underlying platform architecture through intermediate layer encapsulation, deployment strategy reconstruction or log feedback mechanism.

[0094] In the above description, it should be understood that Figure 1Any process or method description in the flowcharts described in the flowcharts or otherwise described herein can be understood to represent a module, segment or portion of code including one or more executable instructions for implementing the steps of a custom logic function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed in a manner other than the order shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention relate. The logic and / or steps represented in the flowcharts or otherwise described herein illustrate the possible architecture, functions and operations of the apparatus and methods according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the figures. For example, two boxes shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in a reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using a combination of dedicated hardware and computer instructions. For example, it may be considered as a sequenced list of executable instructions for implementing a logical function, which may be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or used in conjunction with such instruction execution system, apparatus, or device.

[0095] Those skilled in the art will understand that the above description is only a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of the present disclosure may be combined or coupled in various ways, even if such a combination or coupling is not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

[0096] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such changes and modifications as fall within the scope of the claims and their equivalents.

Claims

1. A flexible deployment method for serverless computing function workflows that combines dynamic and static features, characterized by: The method is: Step 1: Use the benchmark node running data and the random forest regression model to extrapolate the task execution time; Step 2: Input the predicted task execution time into the SLO-based heterogeneous cluster task scheduling algorithm for task scheduling; Step 3: Input the scheduling result in step 2 into the task serialization algorithm on the node. Through serialization, multiple small functions are combined into fewer larger function execution units. Step 4: Dynamic task scheduling based on the shortest stage completion time monitors and dynamically implements dynamic merging and optimized rescheduling of multiple DAG tasks in real time.

2. The method for flexible deployment of serverless computing function workflows combining dynamic and static functions according to claim 1 is characterized in that: Step 1 is as follows: Step 101: Collect the features of the function running on the standard machine and build a joint feature set based on the hardware resources of the node; Step 102: Use the joint feature set to train a random forest regression model; Step 103: Use the trained random forest regression model to predict the execution time of the function on machines with different performance.

3. The method for flexible deployment of serverless computing function workflows combining dynamic and static functions according to claim 2 is characterized in that: The construction of the joint feature set includes computing resource parameters and function running characteristics.

4. The method for flexible deployment of serverless computing function workflows combining dynamic and static functions according to claim 1 is characterized in that: Step 2 is as follows: Step 201: Group the sequential functions in the serverless computing function workflow into parallel stages, define the maximum distance of the path from the starting node to any node, and divide the nodes into several parallel execution stages according to the maximum distance; Step 202: For each parallel execution stage after division, ensure that the function tasks in all stages are completed before executing the function tasks in the next stage. The tasks in each stage can be safely executed in parallel. Step 203: Based on the predicted execution time and node cost, find the default deployment node that meets the execution requirements and has the lowest cost. Repeatedly optimize node selection for key tasks to ensure that the task allocation of each stage reaches the optimal state. Calculate the difference between the actual maximum execution time of the stage and the SLO, and allocate this redundant time to the next stage to balance the execution time of each stage.

5. The method for flexible deployment of serverless computing function workflow combining dynamic and static functions according to claim 1 is characterized in that: Step 3 is as follows: Step 301: Calculate the maximum execution time of a single task for a task set. Step 302: Determine the total execution time of all tasks and calculate the optimal scheduling length based on the total execution time of all tasks. Step 303: Initialize the processor load, and sequentially distribute and serially combine tasks until all tasks are scheduled.

6. The method for flexible deployment of serverless computing function workflow combining dynamic and static functions according to claim 5 is characterized in that: During the task serialization process, historical scheduling data can be input to learn the task execution mode.

7. The method for flexible deployment of serverless computing function workflow combining dynamic and static functions according to claim 1 is characterized in that: Step 4 is as follows: Step 401: Detect resource mismatch during the execution of the static scheduling solution, and redefine and adjust the task execution plan. Step 402: Identify and reorganize the timed-out tasks into a new DAG graph for rescheduling to ensure real-time performance and resource adaptability. Through virtual start and end nodes, multiple concurrent DAG tasks are merged for unified task management. Step 403: predict task execution time and communication cost, determine task priority, and reschedule tasks based on the priority.

8. A flexible deployment system for serverless computing function workflows that combines dynamic and static features, characterized by: The system includes a storage device, which is used to execute the method and steps described in claim 1.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the method for flexible deployment of dynamic and static combined serverless computing function workflows according to any one of claims 1 to 7.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the dynamic and static combined service-free computing function workflow flexible deployment method described in any one of claims 1-7.

Images