An intelligent workflow scheduling system for data-intensive applications

Abstract

The present invention provides an intelligent workflow scheduling system for data-intensive applications, which calculates the total time overhead of executing tasks through a time estimation module set on a computing power node in a computing power network, calculates the total energy consumption through an energy prediction module, calculates the utility of all functions of intelligent workflow scheduling through a UE utility analysis module, determines the priority of tasks through a priority strategy module to adjust the execution order of tasks, and forms an intelligent workflow for scheduling data-intensive applications through an application scheduling module; compared with the prior art, it can achieve accurate prediction and optimization of task time, energy consumption and priority, improve resource utilization, reduce delay and energy consumption, and use intelligent workflow scheduling at the brain layer of the computing network to optimize time consumption, energy consumption and cost in data-intensive applications, so as to improve the overall performance of the computing power network.

Description

Technical Field

[0001] The present invention relates to the technical field of network resource allocation, and in particular to an intelligent workflow scheduling system for data-intensive applications. Background Art

[0002] In today's digital transformation, the development of data-intensive applications has become one of the important forces driving scientific and technological progress and social development. With the rapid development of artificial intelligence technology, the demand for processing large-scale data sets in various industries is increasing day by day. As the core link of data-intensive applications, large language models not only cover multiple fields such as large-scale data analysis, machine learning model training, and real-time data processing systems, but also have an increasing demand for computing resources. At the same time, they also put forward higher requirements for the flexibility, efficiency, and intelligence level of resource scheduling. In recent years, data has become a key factor driving the development of various industries. Data-intensive applications, such as large-scale data analysis, machine learning model training, and real-time data processing systems, have an increasing demand for computing resources, and at the same time, they have higher and higher requirements for the flexibility, efficiency, and intelligence level of resource scheduling.

[0003] In the era of big data, the complexity of data processing requires higher computing power, and traditional single data centers can hardly meet this demand. The computing network integrates computing resources scattered around the world to form a huge virtual computing pool, which can provide sufficient computing power to support large-scale data processing tasks. In addition, the computing network can dynamically allocate resources according to the specific needs of the task, thereby improving resource utilization and reducing energy consumption and costs.

[0004] Data-intensive applications refer to software systems and tools that focus on efficient processing and analysis of large data sets in design and implementation. They optimize data access, processing processes and algorithms to ensure that valuable information can be quickly extracted in scenarios such as data analysis, machine learning, and large-scale data storage, support decision-making and business innovation, and thus cope with the challenges brought by the explosive growth of modern data. Data-intensive applications are characterized by processing huge amounts of data, emphasizing efficient data storage, fast access and complex analysis capabilities, requiring highly scalable architectures to cope with data growth, and focusing on the timeliness and cost efficiency of data processing. With the help of intelligent algorithms, resource utilization is optimized to ensure that valuable insights can be extracted from the data torrent to drive business decisions and innovation. As the intelligent center of the computing network, the computing network brain is an intelligent system that integrates advanced computing power and high-speed network technology. It is a core component in the development of the computing network and represents a new generation of deeply integrated computing models. In this model, the computing network brain can connect and coordinate diverse discrete computing resources to achieve their efficient flow. It uses advanced technology to optimize resource allocation to ensure that computing power and network demands can be responded to immediately and intelligently scheduled. However, few existing scheduling solutions use intelligent workflow scheduling for optimization at the brain layer of the computing network, nor do they have better optimization of the time consumption, energy consumption and cost of scheduling functions in data-intensive applications. Summary of the invention

[0005] The present invention provides an intelligent workflow scheduling system for data-intensive applications, the purpose of which is to use intelligent workflow scheduling at the brain layer of the computing network to optimize the time consumption, energy consumption and cost in data-intensive applications, so as to improve the overall performance of the computing network.

[0006] In order to achieve the above-mentioned object, the present invention provides an intelligent workflow scheduling system for data-intensive applications, including a time estimation module, an energy prediction module, a UE utility analysis module, a priority strategy module and an application scheduling module arranged on a computing power node in a computing power network;

[0007] The time estimation module is used to calculate the total time cost of executing tasks on the computing power node. The total time cost includes scheduling time, calculation time, execution time, transmission time and delay time.

[0008] Energy prediction module, used to predict the total energy consumption of computing nodes in the process of executing tasks based on execution time, calculation time, scheduling time and delay time;

[0009] UE utility analysis module, which is used to calculate the utility of all functions of intelligent workflow scheduling based on the total energy consumption of computing nodes during task execution. The utility includes the time consumption, energy consumption and user cost of task execution.

[0010] The priority strategy module is used to allocate tasks to each computing node based on the total time cost, the total energy consumption of the computing node in the process of executing the task, and the frequency of the utility computing network brain, and determine the priority of the task based on the frequency to dynamically adjust the order of task execution;

[0011] The application scheduling module is used to input the output results of the priority strategy module into the deepening priority Q network to form an intelligent workflow for scheduling data-intensive applications.

[0012] Furthermore, the total time cost for:

[0013] ;

[0014] in, Represents computing power node The time to complete the task, Indicates the number of computing nodes.

[0015] Furthermore, computing nodes Time to complete the task for:

[0016]

[0017] in, Indicates that the computing network brain dispatches tasks to computing power nodes The scheduling time, Represents computing power node The execution time required to complete the task, Represents computing power node The delay time consumed in completing the task. Represents computing power node The time to start completing the task.

[0018] Furthermore, the computing network brain dispatches tasks to computing nodes. Scheduling time The calculation expression is:

[0019]

[0020] in, Indicates that the computing network brain is dispatched to the computing power node The amount of task data on Indicates the transmission rate of tasks scheduled by the computing network brain to different computing power nodes;

[0021] Computing Node Execution time required to complete the task The calculation expression is:

[0022]

[0023] in, It represents the amount of computation required by the computing network brain to transfer tasks to computing power nodes. Indicates Computing nodes The computing power of executing functions in Indicates Computing nodes The execution time to complete the task;

[0024] Computing Node The delay time consumed in completing the task for:

[0025]

[0026] in, Indicates the processing delay time of the task. It represents a constant and has no specific physical meaning;

[0027] The transmission time from the computing network brain to the computing power node for:

[0028]

[0029] in, Indicates the transmission rate;

[0030] Transmission time between computing nodes for:

[0031]

[0032] in, Indicates the amount of task data sent by the computing power node to the idle computing power node. Indicates the ambient noise floor. Indicates the interference intensity;

[0033] Computing Node Start time to complete the task for:

[0034] ;

[0035] in, Indicates the task The end time of Indicates the task A collection of .

[0036] To put it more concretely, the total energy consumption of a computing node during task execution is:

[0037]

[0038] in, Represents the scheduling energy consumption of computing nodes, Indicates the computing energy consumption of the computing power node, Indicates the transmission energy consumption from the computing network brain to the computing power node, Represents the transmission energy consumption between two computing nodes, Indicates the delayed energy consumption.

[0039] Furthermore, computing nodes Scheduling energy consumption for:

[0040]

[0041] Computing Node Computational energy consumption for:

[0042]

[0043] From the brain of the computing network to the computing power node Transmission energy consumption for:

[0044]

[0045] Two computing nodes , The transmission energy consumption between for:

[0046]

[0047] Computing Node Delayed energy consumption for:

[0048]

[0049] in, Indicates the active function consumption of the computing power node, Indicates the idle consumption of computing power nodes. Indicates the idle consumption of the computing network brain, Indicates the cost of delay.

[0050] Furthermore, the utility of all functions of intelligent workflow scheduling for:

[0051]

[0052] in, Indicates the sensitivity of the computing power node to time delay, Indicates the sensitivity of computing nodes to energy consumption, Indicates the calculation time of a single task execution process, Represents the energy consumption of a single task execution process, represents the memory cost of the task, Represents the memory requirement of the task.

[0053] Specifically, the priority strategy module is used to:

[0054] The fullness of the task is calculated based on the utility of all functions scheduled by the intelligent workflow. The fullness is used to characterize the proportion of the task resources to the total system resources. The calculation formula is:

[0055]

[0056] The urgency of the task is calculated based on the scheduling time. The urgency is used to characterize the relative proportion of the execution delay to the total system delay. The calculation formula is:

[0057]

[0058] Through the fullness and urgency Calculate the frequency distribution weight of the task for:

[0059]

[0060] Assigning weights to frequencies Normalize and get the normalized frequency allocation weight for:

[0061]

[0062] Assign weights based on normalized frequency Calculate the frequency of the network brain assigning tasks to each computing node for:

[0063]

[0064] in, Represents the total number of tasks, It is used to adjust the impact of utility on priority. Indicates the overall frequency of the server.

[0065] Further, the deepening priority Q network includes the evaluation network and the target network;

[0066] Use the evaluation network to select the best action in the next state , the expression is selected as:

[0067]

[0068] Use the target network to find the value of the best action in the next state To evaluate, the evaluation expression is:

[0069]

[0070] in, Indicates the next state, Indicates action, represents the parameters of the evaluation network, represents the state-action value function, Indicates immediate reward, represents the discount factor, represents the Q value of the optimal action in the next state evaluated by the target network, Represents the parameters of the target network.

[0071] Furthermore, the parameters of the evaluation network are updated according to the target Q value, as follows:

[0072]

[0073] The parameters of the target network are updated through the soft update rule:

[0074]

[0075] in, Indicates the current state. Indicates the current action. represents the learning rate, It is used to control the speed at which the target network parameters approach the evaluation network parameters.

[0076] The above scheme of the present invention has the following beneficial effects:

[0077] The present invention calculates the total time cost of executing tasks through a time estimation module, calculates the total energy consumption through an energy prediction module, calculates the utility of all functions of intelligent workflow scheduling through a UE utility analysis module, determines the priority of tasks through a priority strategy module to adjust the execution order of tasks, and forms an intelligent workflow for scheduling data-intensive applications through an application scheduling module. Compared with the prior art, it can achieve accurate prediction and optimization of task time, energy consumption and priority, improve resource utilization, reduce latency and energy consumption, and use intelligent workflow scheduling at the brain layer of the computing network to optimize time consumption, energy consumption and cost in data-intensive applications, so as to improve the overall performance of the computing power network.

[0078] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 is a principle block diagram of an embodiment of the present invention;

[0080] Figure 2 A scheduling architecture diagram of an embodiment of the present invention;

[0081] Figure 3 To deepen the structural diagram of the priority Q network;

[0082] Figure 4 It is a curve chart comparing the effects of the embodiment of the present invention with the existing scheduling method. DETAILED DESCRIPTION

[0083] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0085] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0086] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0087] In view of the existing problems, the present invention provides an intelligent workflow scheduling system for data-intensive applications.

[0088] like Figure 1 As shown, an embodiment of the present invention provides an intelligent workflow scheduling system for data-intensive applications, including a time estimation module, an energy prediction module, a UE utility analysis module, a priority strategy module and an application scheduling module arranged on a computing power node in a computing power network;

[0089] The time estimation module is used to calculate the total time cost of executing tasks on the computing power node. The total time cost includes scheduling time, calculation time, execution time, transmission time and delay time.

[0090] Energy prediction module, used to predict the total energy consumption of computing nodes in the process of executing tasks based on execution time, calculation time, scheduling time and delay time;

[0091] UE utility analysis module, which is used to calculate the utility of all functions of intelligent workflow scheduling based on the total energy consumption of computing nodes during task execution. The utility includes the time consumption, energy consumption and user cost of task execution.

[0092] The priority strategy module is used to allocate tasks to each computing node based on the total time cost, the total energy consumption of the computing node in the process of executing the task, and the frequency of the utility computing network brain, and determine the priority of the task based on the frequency to dynamically adjust the order of task execution;

[0093] The application scheduling module is used to input the output results of the priority strategy module into the deepening priority Q network to form an intelligent workflow for scheduling data-intensive applications.

[0094] It should be noted that the task can be a large-scale data analysis task, a machine learning model training task, or a real-time data processing task, etc. The computing network includes a computing network brain and multiple computing nodes. The computing network brain is equivalent to a server, and the computing nodes are equivalent to terminal devices.

[0095] Specific as Figure 2 As shown, in the embodiment of the present invention, first, a user submits a task of training a large language model to the computing network brain, and the computing network brain is responsible for preliminary processing and allocation of the task;

[0096] Subsequently, the task is sent to each computing node, which is a computing resource pool that can contain multiple modules to handle different aspects, such as time estimation module, energy prediction module and UE utility analysis module: these modules focus on the time required for task execution, energy consumption and utility respectively. They may evaluate the demand and performance of the task by analyzing historical data or real-time monitoring. Based on the information provided by the above three modules, they decide how to allocate tasks among available resources.

[0097] Subsequently, the task scheduling is optimized by applying the deepening priority Q network in the scheduling module to ensure the best performance and efficiency. During the task execution, the system monitors and dynamically adjusts resource allocation in real time, and automatically adjusts the learning rate curve to speed up the convergence of model training. For example, the learning rate is quickly increased at the beginning of training, and then gradually reduced until it stabilizes, ensuring that the model converges quickly and stably.

[0098] After completion, the system will perform a performance evaluation based on the training results and return it to the user, and adjust the scheduling method based on the execution of this task, in order to achieve better resource utilization and performance in similar tasks in the future.

[0099] Specifically, the embodiment of the present invention predicts the execution time, scheduling time, transmission time and delay time of tasks on different computing power nodes in the time estimation module, and the specific process is as follows:

[0100] First, we can find the upload rate of the task through the computing network brain. The calculation formula is:

[0101]

[0102] in, Indicates the current transmission bandwidth. represents the path loss constant, Indicates user The relative distance to the computing power node, Indicates user The actual distance to the computing power node, represents the path loss exponent, Indicates the mobile device uninstall task. represents the noise power spectral density, Indicates the interference intensity;

[0103] Next, we can find out how the computing network brain dispatches tasks to computing power nodes. Scheduling time The calculation expression is:

[0104]

[0105] in, Indicates that the computing network brain is dispatched to the computing power node The amount of task data on Indicates the transmission rate of tasks scheduled by the computing network brain to different computing power nodes;

[0106] After that, we can find the computing power node Execution time required to complete the task The calculation expression is:

[0107]

[0108] in, It represents the amount of computation required by the computing network brain to transfer tasks to computing power nodes. Indicates Computing nodes The computing power of executing functions in Indicates Computing nodes The execution time to complete the task;

[0109] Due to various factors in the task completion process, delay is an objective phenomenon and cannot be completely eliminated. Therefore, the embodiment of the present invention takes into account the existence of delay and calculates the computing power node. The delay time consumed in completing the task for:

[0110]

[0111] in, Indicates the processing delay time of the task. It represents a constant and has no specific physical meaning;

[0112] Since it is difficult to calculate the processing delay of a task even with accurate operation data, the embodiment of the present invention adds a constant, which is specifically given by the operator. The processing delay can be expressed by the following function:

[0113]

[0114] in, Indicates the performance of computing nodes, including computing resources, storage resources, and I / O resources. Represents the properties of a computational task;

[0115] Since the computing network brain needs to assign tasks to different computing nodes, it is necessary to calculate the transmission time from the computing network brain to the computing nodes. for:

[0116]

[0117] in, Indicates the transmission rate;

[0118] When a computing node has multiple tasks, you can consider transferring the tasks to other idle computing nodes to maximize the use of resources. Therefore, the transmission time between computing nodes can be obtained. for:

[0119]

[0120] in, Indicates the amount of task data sent by the computing power node to the idle computing power node. Indicates the ambient noise floor. Indicates the interference intensity;

[0121] The embodiment of the present invention assumes that after the task is scheduled by the computing network brain, there is a start time and a completion time of the task, which can be recursively calculated from the start time to the end time, as shown below:

[0122] Computing Node Start time to complete the task for:

[0123] ;

[0124] in, Indicates the task The end time of Indicates the task A collection of .

[0125] The completion time of each task depends on the start time and the execution time on the computing node. Time to complete the task for:

[0126] ;

[0127] in, Indicates that the computing network brain dispatches tasks to computing power nodes The scheduling time, Represents computing power node The execution time required to complete the task, Represents computing power node The delay time consumed in completing the task. Represents computing power node The time to start completing the task.

[0128] After all functions are allocated and mapped to different computing nodes, the total time cost is calculated for:

[0129]

[0130] in, Represents computing power node The time to complete the task, Indicates the number of computing nodes.

[0131] For the brain of the computing network, energy consumption may occur at any time during the scheduling process, execution process, and transmission process. When a task is scheduled to a computing node, the computing node Scheduling energy consumption for:

[0132]

[0133] Once the tasks are assigned to different computing nodes, the computing network brain is relatively idle, but the computing nodes still need to spend some computing energy to ensure that the tasks are completed normally. Computational energy consumption for:

[0134]

[0135] From the brain of the computing network to the computing power node Transmission energy consumption for:

[0136]

[0137] At the same time, considering the working status between different computing nodes, the tasks are transferred to the idle computing nodes to maximize the utilization of resources. , The transmission energy consumption between for:

[0138]

[0139] After that, we also need to consider the energy consumption caused by the delay in executing the task, the computing power node Delayed energy consumption for:

[0140]

[0141] in, Indicates the active function consumption of the computing power node, Indicates the idle consumption of computing power nodes. Indicates the idle consumption of the computing network brain, Indicates the cost of delay.

[0142] Specifically, the total energy consumption of the computing power node in the process of executing the task is:

[0143] ;

[0144] in, Represents the scheduling energy consumption of computing nodes, Indicates the computing energy consumption of the computing power node, Indicates the transmission energy consumption from the computing network brain to the computing power node, Represents the transmission energy consumption between two computing nodes, Indicates the delayed energy consumption.

[0145] Specifically, the utility of all functions of intelligent workflow scheduling for:

[0146]

[0147] in, , are non-negative weights, used to help maintain a balance between total time cost and total energy consumption. , Indicates the sensitivity of the computing power node to time delay, Indicates the sensitivity of computing nodes to energy consumption, Indicates the calculation time of a single task execution process, Represents the energy consumption of a single task execution process, represents the memory cost of the task, Represents the memory requirement of the task, which consists of time overhead, energy consumption, and user cost.

[0148] In order to meet the interdependencies between functions, it is crucial to execute these tasks in sequence. If a task is urgent and occupies less resources, but it will result in higher time and energy consumption during completion, and the computing nodes are very sensitive to these consumptions, then the priority of the task will be reduced due to its higher utility. On the contrary, if a task is not so urgent and occupies more resources, but its execution time and energy consumption are lower, and the computing nodes are not very sensitive to this, then the priority of the task will be improved. Therefore, the embodiment of the present invention provides a priority calculation method. The higher the calculated frequency value, the higher the priority of the task;

[0149] First, the fullness of the task is calculated based on the utility of all functions scheduled by the intelligent workflow. The fullness is used to characterize the proportion of the task resources to the total system resources. The calculation formula is:

[0150]

[0151] The urgency of the task is then calculated based on the scheduling time. The urgency is used to characterize the relative proportion of the execution delay to the total system delay. The calculation formula is:

[0152]

[0153] Next, through the filling and urgency The ratio of can be used to assign weights to the frequency of tasks. Calculate the expression:

[0154]

[0155] Then assign weights to the frequencies Normalize and get the normalized frequency allocation weight for:

[0156]

[0157] Finally, weights are assigned according to the normalized frequencies Calculate the frequency of the network brain assigning tasks to each computing node for:

[0158]

[0159] in, Indicates the total number of tasks, represents a positive coefficient, which is used to adjust the impact of utility on priority. Indicates the total frequency of the server. The higher the frequency, the higher the priority of the task. Therefore, the embodiment of the present invention can execute the tasks in sequence according to their urgency.

[0160] The embodiment of the present invention proposes deepening the priority Q network in the application scheduling module, aiming to overcome the overestimation problem of the existing reinforcement learning model when dealing with complex environment decisions and improve learning efficiency and stability. A multi-step learning mechanism is introduced for Q-learning. Compared with the single-step TD learning of the traditional priority Q network, multi-step learning considers the rewards of the next few steps and updates the Q value by calculating the cumulative rewards of n steps, which can reduce variance and accelerate learning. For Q-learning, the n-step update rule can be written as:

[0161]

[0162] in, Indicates the current state. Indicates the current action. Represents the learning rate.

[0163] Specifically, in order to enhance the efficiency and stability of learning, the deepening priority Q network is designed as a two-layer structure, including an evaluation network and a target network;

[0164] Use the evaluation network to select the best action in the next state , the expression is selected as:

[0165]

[0166] Use the target network to find the value of the best action in the next state To evaluate, the evaluation expression is:

[0167]

[0168] in, Indicates the next state, Indicates action, represents the parameters of the evaluation network, represents the state-action value function, Indicates immediate reward, represents the discount factor, represents the Q value of the optimal action in the next state evaluated by the target network, Represents the parameters of the target network.

[0169] Specifically, the evaluation network and the target network both contain an input layer, multiple hidden layers and an output layer, where the hidden layer usually uses the ReLU activation function, the input layer inputs the representation of the environment state, and the output layer outputs the Q value estimate of each possible action. The evaluation network and the target network have the same structure but the parameters are initialized independently.

[0170] The evaluation network and the target network adopt He initialization or similar strategies to ensure the reasonable distribution of network weights, which is conducive to rapid convergence.

[0171] Specific as Figure 3 As shown, the training process of deepening the priority Q network is as follows:

[0172] The computing network interacts with the environment, collects a set of experience fragments as training samples, and stores them in the created experience replay pool. The parameters of the evaluation network and the target network are updated through the dynamic priority experience replay mechanism. Then the loss value is calculated through the loss function, and the loss value is used to train the evaluation network. Finally, the weights of the evaluation network are copied to the target network until the end of the round and the training is completed.

[0173] The multi-step learning mechanism includes:

[0174] First, initialize the parameters of the evaluation network , and the parameters of the target network Initialized to 0;

[0175] Since the evaluation network is a neural network model used to evaluate the value of actions in the current state, its parameters are usually expressed as , which determines the weight and bias of the network, thus affecting the network's prediction of the Q value of a given state-action pair. The goal of the evaluation network is to approximate the true state-action value function by learning this parameter. , that is, a given state Take action The expected cumulative reward that can be obtained after

[0176] The parameters of the target network refer to the weights and biases in the neural network model used to calculate the target Q value. The parameters are usually expressed as The main purpose of the target network is to provide a stable baseline during the training process to evaluate the quality of the decisions made by the current evaluation network by using the same parameters as the evaluation network. Different parameters , the target network can provide a more stable target Q value, which helps to reduce fluctuations and over-estimation problems in the learning process;

[0177] Then, collect experience fragments: interact the computing network with the environment and collect a set of experience fragments as training samples;

[0178] Next, multi-step learning is performed: In multi-step learning, the target Q value is usually calculated using the following formula:

[0179]

[0180] in, Indicates the current state. Indicates the current action. It is used to control the speed at which the target network parameters approach the evaluation network parameters. represents the number of steps forward to consider, represents the Q-value estimate of the state-action pair after n steps;

[0181] The parameters of the target network are updated regularly through soft update rules to ensure smooth transition of parameters and avoid instability caused by drastic changes;

[0182] Assume that at time step In state , and selected the action , and got instant rewards through double Q learning , and it moves to a new state ;

[0183] Finally, update the network parameters as follows:

[0184] The parameters of the evaluation network are updated according to the target Q value:

[0185]

[0186] The parameters of the target network are updated through the soft update rule:

[0187]

[0188] in, Indicates the current state. Indicates the current action. represents the learning rate, It is a small positive number (such as 0.001) that controls the speed at which the target network parameters approach the evaluation network parameters.

[0189] In the embodiment of the present invention, the soft update rule is a method for progressively updating the target network parameters to ensure a smooth transition of the target network parameters, thereby maintaining the stability of the target network, reducing fluctuations in the learning process, and improving learning efficiency and stability.

[0190] The embodiment of the present invention also proposes an innovative mechanism of dynamic priority experience replay in the application scheduling module, which aims to optimize the efficiency of experience reuse under the deep reinforcement learning framework. The core of this mechanism is to assign a dynamically calculated priority value to each experience segment stored in the experience replay pool, which is closely related to the prediction error of the experience segment. This design enables experience segments with larger prediction errors, that is, those data points that are most critical to improving model performance, to obtain a higher probability of being sampled. Compared with traditional experience replay technology, the DPER mechanism implements a priority-based sampling strategy to ensure that the algorithm can learn lessons from the most challenging and learning-valuable experiences more frequently, thereby significantly accelerating the learning process.

[0191] Specifically, the dynamic priority experience playback mechanism includes:

[0192] Experience pool module: creates an experience replay pool. When each experience segment is added to the experience replay pool, it is given an initial priority based on its prediction error. The larger the prediction error, the higher the priority.

[0193] Weighted sampling module: During training, instead of randomly selecting experience segments, weighted sampling is performed based on their priorities, which ensures that experience segments with higher priorities are more likely to be selected;

[0194] Priority Update Module: As the network learns, the priorities of the experience segments are updated accordingly. Every time an experience segment is used for training and updating the network, its prediction error is calculated again to update its priority.

[0195] In order to improve the value function estimation capability in the deep reinforcement learning algorithm, the embodiment of the present invention introduces a separable value and advantage evaluation network architecture. Through the separable design, the separable value and advantage evaluation network can more accurately evaluate the inherent value of a given state and quantify the relative advantages of actions, thereby reducing the value estimation bias. At the same time, in the face of a complex decision-making environment with a high-dimensional observation space and a large number of action options, the separable value and advantage evaluation network can significantly improve the accuracy of decision-making and reduce the risk of over-estimation, thereby improving the stability of the learning process. Moreover, through more accurate value estimation, the separable value and advantage evaluation network can accelerate the convergence speed of the learning process.

[0196] In the embodiment of the present invention, when introducing a separate value and advantage evaluation network, the evaluation network and the target network are designed to be a network containing two branches, one branch for evaluating the inherent value of the state, and one branch for evaluating the relative advantage of the action. Such a design enables the evaluation network to more accurately estimate the value of the state and distinguish the relative advantages and disadvantages of different actions, thereby improving the accuracy of action selection. The target network is enabled to more accurately estimate the value of the state, thereby improving the accuracy of the target Q value and reducing the risk of overestimation.

[0197] In the embodiment of the present invention, when a separate value and advantage evaluation network is introduced, the state-action value function is further subdivided into two parts, namely, the state value and the action advantage;

[0198] Among them, the state value is specifically used to evaluate the inherent value of a given state, that is, the expected cumulative reward that the computing power network can obtain by acting according to the strategic method in this state is:

[0199]

[0200] in, is the discount factor, is at the time step Instant rewards after Indicates that in a given strategy The expected value of the behavior outcome of the agent;

[0201] Action advantage focuses on quantifying the relative advantage of an action relative to the current state s, that is, the additional benefit of taking a certain action relative to other actions is:

[0202]

[0203] in, Indicates that in a given state Take action The expected cumulative rewards that can be obtained in the future.

[0204] The embodiment of the present invention uses the Huber loss function or the mean square error loss function to measure the difference between the predicted Q value and the target Q value, thereby reducing the impact of extreme errors on learning. At the same time, through the optimization algorithm, an optimizer such as Adam or RMSprop is used to minimize the loss function and update the parameters of the evaluation network.

[0205] The embodiment of the present invention combines three types of intelligent workflow scheduling algorithms to train 300 rounds of average cumulative reward curves, such as Figure 4 As shown, Figure 4 The horizontal axis (x-axis) represents the rounds of training during the training process. Each round represents a complete process of interaction between the computing power network and the environment, starting from the initial state until the termination condition is reached. The vertical axis (y-axis) represents the average value of the rewards obtained by the computing power network in one or more rounds; the average convergence curves of the three algorithms show that the advantages of deepening the priority Q network are as follows:

[0206] 1. Faster convergence speed: It can be clearly seen that the traditional DQN algorithm gradually converges at about 230 steps, the DDQN algorithm gradually converges at about 200 steps, and the deepening priority Q network begins to converge at about 150 steps. Compared with the traditional DQN algorithm and DDQN algorithm, the deepening priority Q network has better convergence performance, which means that it can learn effective scheduling methods in a shorter time, thereby saving computing resources and time costs;

[0207] 2. Higher stability: Although all three curves show fluctuations, the fluctuations of the deepening priority Q network are relatively small, which indicates that its performance is more stable and it is easier to maintain good performance in different environments or when facing different challenges;

[0208] 3. Better long-term performance: Throughout the entire time period, the deepening priority Q network generally maintains a high performance level, while the other two algorithms may experience decline or instability, which shows that APQN is more adaptable and robust in the long run.

[0209] The embodiment of the present invention calculates the total time overhead of executing tasks through a time estimation module set on a computing power node in a computing power network, calculates the total energy consumption through an energy prediction module, calculates the utility of all functions of intelligent workflow scheduling through a UE utility analysis module, determines the priority of tasks through a priority strategy module to adjust the execution order of tasks, and forms an intelligent workflow for scheduling data-intensive applications through an application scheduling module; compared with the prior art, it can achieve accurate prediction and optimization of task time, energy consumption and priority, improve resource utilization, reduce latency and energy consumption, and use intelligent workflow scheduling at the brain layer of the computing network to optimize time consumption, energy consumption and cost in data-intensive applications, so as to improve the overall performance of the computing power network.

[0210] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent workflow scheduling system for data-intensive applications, characterized in that: It includes a time estimation module, an energy prediction module, a UE utility analysis module, a priority strategy module and an application scheduling module which are arranged on a computing power node in a computing power network; The time estimation module is used to calculate the total time cost of executing the task on the computing power node, and the total time cost includes scheduling time, calculation time, execution time, transmission time and delay time; The energy prediction module is used to predict the total energy consumption of the computing node in the process of executing the task according to the execution time, the calculation time, the scheduling time and the delay time; The UE utility analysis module is used to calculate the utility of all functions of intelligent workflow scheduling according to the total energy consumption of the computing power node in the process of executing the task, and the utility includes the time consumption, energy consumption and user cost of task execution; The priority strategy module is used to determine the priority of the task according to the total time cost, the total energy consumption of the computing node in the process of executing the task, the frequency of the utility computing network brain allocating tasks to each computing node, and according to the frequency, so as to dynamically adjust the order of task execution; The application scheduling module is used to input the output result of the priority strategy module into the deepening priority Q network to form an intelligent workflow for scheduling data-intensive applications.

2. The intelligent workflow scheduling system for data-intensive applications according to claim 1, characterized in that: The total time cost for: ； in, Represents computing power node The time to complete the task, Indicates the number of computing nodes.

3. The intelligent workflow scheduling system for data-intensive applications according to claim 2, characterized in that: Computing Node Time to complete the task for: in, Indicates that the computing network brain dispatches tasks to computing power nodes The scheduling time, Represents computing power node The execution time required to complete the task, Represents computing power node The delay time consumed in completing the task. Represents computing power node The time to start completing the task.

4. The intelligent workflow scheduling system for data-intensive applications according to claim 3, characterized in that: The computing network brain dispatches tasks to computing nodes Scheduling time The calculation expression is: in, Indicates that the computing network brain is dispatched to the computing power node The amount of task data on Indicates the transmission rate of tasks scheduled by the computing network brain to different computing power nodes; Computing Node Execution time required to complete the task The calculation expression is: in, It represents the amount of computation required by the computing network brain to transfer tasks to computing nodes. Indicates Computing nodes The computing power of executing functions in Indicates Computing nodes The execution time to complete the task; Computing Node The delay time consumed in completing the task for: in, Indicates the processing delay time of the task. It represents a constant and has no specific physical meaning; The transmission time from the computing network brain to the computing power node for: in, Indicates the transmission rate; Transmission time between computing nodes for: in, Indicates the amount of task data sent by the computing power node to the idle computing power node. Indicates the ambient noise floor. Indicates the interference intensity; Computing Node Start time to complete the task for: ； in, Indicates the task The end time of Indicates the task A collection of .

5. The intelligent workflow scheduling system for data-intensive applications according to claim 4, characterized in that: The total energy consumption of the computing node during task execution is: in, Represents the scheduling energy consumption of computing nodes, Indicates the computing energy consumption of the computing power node, Indicates the transmission energy consumption from the computing network brain to the computing power node, Represents the transmission energy consumption between two computing nodes, Indicates the delayed energy consumption.

6. The intelligent workflow scheduling system for data-intensive applications according to claim 5, characterized in that: The computing power node Scheduling energy consumption for: The computing power node Computational energy consumption for: The computing network brain to the computing power node Transmission energy consumption for: Two computing nodes , The transmission energy consumption between for: The computing power node Delayed energy consumption for: in, Indicates the active function consumption of the computing power node, Indicates the idle consumption of computing power nodes. Indicates the idle consumption of the computing network brain, Indicates the cost of delay.

7. The intelligent workflow scheduling system for data-intensive applications according to claim 6, characterized in that: The intelligent workflow scheduler is a utility for all functions for: in, Indicates the sensitivity of computing power nodes to time delays, Indicates the sensitivity of computing nodes to energy consumption, Indicates the calculation time of a single task execution process, Represents the energy consumption of a single task execution process, represents the memory cost of the task, Represents the memory requirement of the task.

8. The intelligent workflow scheduling system for data-intensive applications according to claim 7, characterized in that: The priority strategy module is specifically used for: The fullness of the utility calculation tasks of all functions scheduled by the intelligent workflow is used to characterize the proportion of the resources of the task to the total resources of the system. The calculation formula is: The urgency of the task is calculated based on the scheduling time. The urgency is used to characterize the relative proportion of the execution delay to the total system delay. The calculation formula is: By the degree of filling and the degree of urgency Calculate the frequency distribution weight of the task for: Assign weights to the frequencies Normalize and get the normalized frequency allocation weight for: Assign weights according to the normalized frequency Calculate the frequency of the network brain assigning tasks to each computing node for: in, Represents the total number of tasks, It is used to adjust the impact of utility on priority. Indicates the overall frequency of the server.

9. The intelligent workflow scheduling system for data-intensive applications according to claim 8, characterized in that: The deepening priority Q network includes an evaluation network and a target network; Use the evaluation network to select the best action in the next state , the expression is selected as: The value of the optimal action for the next state using the target network To evaluate, the evaluation expression is: in, Indicates the next state, Indicates action, represents the parameters of the evaluation network, represents the state-action value function, Indicates immediate reward, represents the discount factor, represents the Q value of the optimal action in the next state evaluated by the target network, Represents the parameters of the target network.

10. The intelligent workflow scheduling system for data-intensive applications according to claim 9, characterized in that: The parameters of the evaluation network are updated according to the target Q value, as follows: The parameters of the target network are updated by soft update rules as follows: in, Indicates the current state. Indicates the current action. represents the learning rate, It is used to control the speed at which the target network parameters approach the evaluation network parameters.

Images