Task scheduling and resource allocation method oriented to edge computing power node

Abstract

The invention relates to an edge computing power node-oriented task scheduling and resource allocation method, and belongs to the field of edge computing. The method comprises the following steps: S1, constructing a multi-objective optimization model in an edge computing environment; the model aims at minimizing the comprehensive operation cost of the system, and the comprehensive operation cost is a weighted combination of the average task completion time, the node load balance degree, the task achievement rate and the resource throughput efficiency; s2, modeling an optimization problem into a Markov decision process, and defining a state space, an action space and a reward function; s3, constructing and training a calculation task scheduling model based on DDQN; and S4, outputting a reasonable task allocation scheme by utilizing a trained strategy based on the current node state and the task information. The method can effectively cope with the dynamics and complexity of the edge computing environment.

Description

Technical Field

[0001] This invention belongs to the field of edge computing and relates to a method for task scheduling and resource allocation for edge computing nodes. Background Technology

[0002] With the rapid development of computing power-aware networks, how to efficiently match existing computing resources with dynamically changing user needs, and improve service quality through the collaborative cooperation of distributed computing nodes, has become a key research issue in related fields. As an important implementation of computing power-aware networks, edge computing offers significant advantages in reducing latency and saving bandwidth by pushing computing power to the network edge. However, compared to traditional centralized cloud computing, resource management in edge computing environments is more complex. This complexity mainly stems from the geographical distribution and high dynamism of edge nodes: computing nodes in each region exhibit significant heterogeneity in terms of computing power, storage capacity, and network bandwidth, and the availability and load status of these resources change continuously over time.

[0003] In practical applications of edge computing, task scheduling and resource allocation are the core factors determining system performance. Faced with diverse scenarios where tasks with high real-time requirements (such as autonomous driving and industrial control) coexist with computationally intensive tasks (such as AI inference and video processing), traditional resource scheduling methods often employ static allocation or centralized control strategies. These methods tend to assume that resources are static or tasks are fixed, making it difficult to cope with the complexity and dynamism of heterogeneous cluster environments, resulting in low resource utilization and high system response latency.

[0004] To address the aforementioned problems, various improvement schemes have emerged in the existing technology, but each still has its own limitations: Scheduling methods based on traditional machine learning: For example, Zhang Guilan et al. proposed a scheduling strategy based on machine learning algorithms, which dynamically evaluates resource utilization by combining load information from monitoring servers and virtual machines. Although this method optimizes system performance to some extent, it is inherently prone to getting trapped in local optima and struggles to maintain global load balancing in large-scale, highly dynamic network environments.

[0005] Scheduling methods based on utility functions: Chai Rong et al. established a fusion network model and constructed a utility function that comprehensively considers network throughput, user satisfaction, and fairness. However, this method is highly sensitive to the parameter settings of the utility function. If the parameters are not set properly (e.g., unreasonable weight allocation), it will significantly affect the final scheduling effect and lacks robustness.

[0006] Hybrid scheduling methods based on heuristic algorithms: Qi Rui et al. proposed a hybrid scheduling method for heterogeneous clusters based on edge computing, which utilizes resource similarity clustering and genetic algorithms for optimization. Although this method performs well in terms of load balancing and resource utilization, its computational complexity is too high and its convergence speed is slow in large-scale network scenarios, making it difficult to meet the stringent real-time requirements of edge computing.

[0007] In summary, existing technologies generally suffer from insufficient scheduling flexibility, weak global optimization capabilities, and slow response to dynamic and sudden tasks when facing edge computing environments. Existing scheduling algorithms fail to fully consider the time-variance of resources and the diversity of tasks in edge computing environments, making it difficult to achieve the optimal balance between latency, load balancing, and resource utilization.

[0008] In recent years, Deep Reinforcement Learning (DRL) has provided a new intelligent approach to solving the aforementioned multidimensional heterogeneous resource allocation problem. DRL can adapt to high-dimensional, nonlinear dynamic environments by interacting with the environment and perceiving task requirements and resource status. However, designing optimal scheduling schemes for specific edge computing scenarios remains a critical technical challenge that urgently needs to be addressed. Summary of the Invention

[0009] In view of this, the purpose of this invention is to provide a task scheduling and resource allocation method for edge computing nodes, employing a task scheduling and multi-dimensional heterogeneous resource allocation algorithm based on deep reinforcement learning. By modeling the task scheduling problem as a Markov Decision Process (MDP), including state, action, reward, and state transition probabilities, a policy is learned through the interaction between the agent and the environment to maximize the long-term cumulative reward. By continuously trying different task allocation decisions and adjusting the policy based on feedback (such as latency and load balancing), the optimal scheduling scheme is finally found. Furthermore, this algorithm can effectively cope with the dynamic and complex nature of edge computing environments.

[0010] To achieve the above objectives, the present invention provides the following technical solution: A task scheduling and resource allocation method for edge computing nodes specifically includes the following steps: S1: Construct a multi-objective optimization model in an edge computing environment; the model aims to minimize the overall system operating cost under the condition of satisfying constraints, and the overall operating cost is a weighted combination of average task completion time, node load balancing, task achievement rate and resource throughput efficiency; S2: Model the task scheduling and resource allocation optimization problem as a Markov decision process (MDP), defining a state space, an action space, and a reward function; wherein, the state space includes a node resource state vector and a task demand vector to be scheduled; the action represents the system's scheduling decision in a specific state, that is, assigning a task to a certain edge node; S3: Construct a computational task scheduling model based on Double Deep Q Network (DDQN). That is, adopt the DDQN framework and update the policy function stably through experience replay and target network mechanism. During the training process, the model continuously interacts with the environment and gradually improves the task scheduling ability in dynamic and changing environments by sampling historical experience and current state decisions. S4: Based on the current node status and task information, output a reasonable task allocation scheme using the trained strategy.

[0011] Furthermore, in step S1, the multi-objective optimization model constructed in the edge computing environment is expressed as follows:

[0012]

[0013]

[0014]

[0015]

[0016] in, Indicates the overall operating cost. Indicates task Completion time, Indicates task Maximum tolerable latency, Represents a set of tasks. Represents a set of nodes. Indicates task Assigned to nodes Decision variables, This indicates the amount of resources that need to be allocated. Represents a node The largest available resource, Indicates system time slot The amount of resources effectively allocated and actually executed within the scope. Indicates system time slot Total internal resource supply.

[0017] Furthermore, in step S1, the overall operating cost... The calculation formula is:

[0018] in, The average time to complete the task. For node load balancing, For task completion rate, For resource throughput efficiency; For the weight parameters, satisfying .

[0019] Furthermore, in step S1, the average task completion time The calculation formula is:

[0020] in, Let M be the completion time of task i, and M be the total number of tasks completed by the system within time slot t.

[0021] Furthermore, in step S1, the node load balancing degree The calculation formula is:

[0022] in, , , The number of computing nodes in the system. For the first Resource load of each node.

[0023] Furthermore, in step S1, the task completion rate... The calculation formula is:

[0024] in, For time slots The total number of tasks received internally. For time slots The number of tasks successfully assigned and fulfilled within the specified timeframe.

[0025] Furthermore, in step S1, resource throughput efficiency The calculation formula is:

[0026] in, For system time slots Total internal resource supply For system time slots The amount of resources that are effectively allocated and actually executed within the system.

[0027] Furthermore, in step S2, the state space includes: 1) Node resource state vector: Define the node resource state vector as... , used to characterize the Each node at the current moment The schedulable computing power capacity is expressed as follows:

[0028] in, This represents the number of available CPU cores. This represents the number of available GPU computing units. This represents the available video memory capacity. This refers to the available memory capacity. This refers to the available outbound bandwidth. Real-time load percentage of nodes; Queue tasks for estimated completion time; 2) Task Demand Vector to be Scheduled: Define the task demand vector to be scheduled as follows: This is used to describe the resource requirements and service quality requirements of the currently scheduled task, and its expression is as follows:

[0029] in, The amount of CPU resources required for the task; The amount of GPU resources required for the task; The memory required for the task; The amount of bandwidth resources required for the task; Enter the scale for the task data; Determine the time delay constraints for task execution; This is the task priority. The system state space is constructed using the above vectorized representation. ,in This represents the total number of nodes, thus enabling a unified mathematical description of the resource status and task requirements of each node.

[0030] Furthermore, in step S2, the reward function is a linearly weighted instantaneous reward function, expressed as:

[0031] in, For instant reward function, This represents the change in the overall system cost. This is a penalty coefficient used to balance the improvement in system benefits with the risk of future task backlog; Indicates the percentage of tasks not completed. For at any time The remaining number of tasks to be scheduled. This represents the initial total number of tasks.

[0032] Furthermore, in step S3, the training process specifically includes: constructing a state vector from the resource requirements of the current task to be scheduled and the available resources of each node in the system. The data is input into a Dual Deep Q Network (DDQN), and the network parameters are initialized. First, the network is evaluated... Select the candidate action with the largest Q value. Subsequently, by the target network Calculate the target Q value based on the action; execute the action. The task will be scheduled to the selected target node, and the system will receive a reward based on the task execution result. and transition to a new system state. ; Experience sample The data is stored in the experience pool, and during the training phase, a small batch of samples is randomly drawn from the experience pool to update the network. The target Q-value is estimated using a dual-network structure of a dual-depth Q-network (DDQN) to reduce estimation bias caused by maximization operations. Its update method is as follows: the evaluation network selects actions... The target value is calculated by the target network:

[0033] in, The discount factor is used to continuously update the evaluation network parameters by minimizing the mean square error between the estimated Q value and the target Q value, and to synchronize the evaluation network parameters to the target network in a fixed number of steps to improve the stability and convergence performance of training.

[0034] The beneficial effects of this invention are as follows: 1) This invention employs the Dual Deep Q-Network (DDQN) algorithm, which effectively overcomes the problem of overestimating Q-values ​​that traditional DQN algorithms tend to exhibit in complex edge computing environments by decoupling action selection (evaluation network) from target value calculation (target network). This mechanism enables the agent to more accurately evaluate the value of scheduling strategies, significantly improving the stability and convergence speed of model training, and allowing it to adapt to the needs of large-scale task scheduling more quickly.

[0035] 2) Addressing the highly heterogeneous nature of edge node resources (CPU, GPU, video memory, bandwidth, etc.), this invention constructs a high-dimensional state space containing fine-grained resource states and task requirement features. Through the feature extraction capabilities of deep reinforcement learning, the algorithm can accurately perceive the real-time load differences and capacity bottlenecks of different nodes, achieving efficient matching of heterogeneous resources. This not only avoids the bottleneck effect caused by scheduling a single resource (such as considering only CPU), but also significantly improves the overall resource throughput efficiency of the system.

[0036] 3) This invention constructs a comprehensive cost objective function that includes average task completion time, node load balancing, task achievement rate, and resource throughput efficiency. By adjusting the weight parameters, the algorithm can find the optimal balance between reducing user latency and maintaining network-wide load balancing. Compared to traditional methods that focus only on a single metric (such as optimizing latency to prevent a node from becoming overloaded), this invention effectively prevents task failures due to resource exhaustion caused by overheating of some nodes, thus improving the robustness of the system.

[0037] 4) This invention introduces a legality screening mechanism in the action selection phase, which preemptively filters out invalid nodes that do not meet resource constraints or latency constraints, avoiding invalid trials and improving exploration efficiency. Simultaneously, a penalty term for unfinished task backlog is introduced into the reward function, guiding the agent to focus not only on immediate gains but also on long-term task processing capabilities, thereby effectively reducing the risk of system crashes or severe task backlogs during sudden traffic spikes.

[0038] 5) The DRL agent based on the MDP model of this invention can dynamically adapt to topology changes, task traffic fluctuations, and node performance jitters in the edge network through continuous interaction with the environment. When the network environment changes, the model can automatically adjust the scheduling strategy without manual remodeling, maintaining optimal or near-optimal scheduling performance, which greatly reduces the complexity of operation and maintenance management.

[0039] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the task scheduling and resource allocation method for edge computing nodes according to the present invention; Figure 2 A diagram illustrating the reinforcement learning process. Detailed Implementation

[0041] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0042] Example 1: Please see Figure 1 This embodiment provides an edge computing task scheduling method based on deep reinforcement learning. Its core lies in utilizing the dynamic relationship between environmental state, action selection, and reward feedback to achieve an intelligent and adaptive task allocation strategy. The overall implementation includes the following steps: Step 1: State Representation The system constructs a state vector by collecting key information from each edge computing node and the requirements of the tasks to be scheduled. State information may include, but is not limited to: node computing power, available memory, network bandwidth, historical load data, and task computing requirements, priorities, and deadlines. The state vector serves as environmental input in the reinforcement learning model, providing a basis for task scheduling decisions.

[0043] Step 2: Motion Space Design An action represents a scheduling decision made by the system in a specific state, namely, assigning a task to a specific edge node. The action space covers all feasible task-node combinations, enabling the model to explore multiple scheduling schemes. In dynamic environments, the selection of actions considers not only the current node resources but also task completion latency and overall system load balancing.

[0044] Step 3: Reward Function Construction The reward function is used to evaluate the effectiveness of the scheduling strategy and is typically related to task completion efficiency, resource utilization, and system load balancing. The reward signal is fed back to the model through reinforcement learning, guiding it to optimize the strategy. During the design phase, multiple metrics can be combined to comprehensively evaluate task scheduling performance, thereby promoting the strategy to achieve optimal or near-optimal results in long-term scheduling.

[0045] Step 4: Strategy Optimization and Learning Mechanism Step 5: Method Application In real-world edge computing scenarios, the system can output a reasonable task allocation scheme based on the current node state and task information, utilizing trained strategies. This method is adaptive and scalable, continuously optimizing scheduling performance in multi-node, multi-task, and highly dynamic environments without manual intervention or predefined rules.

[0046] Through the above implementation method, this embodiment provides an edge computing task scheduling strategy that combines theory and practice. This strategy can achieve efficient, balanced, and adaptive task allocation under varying resource conditions, and is suitable for various edge computing architectures and complex application scenarios.

[0047] In multi-user, multi-node networks, this strategy addresses scenarios where computational tasks are scheduled and processed across different computing nodes. To resolve issues such as low task execution efficiency, uneven node load, and insufficient resource utilization, this embodiment provides a scheduling optimization method that comprehensively considers multiple indicators. This method constructs a multi-objective optimization model to jointly optimize average task completion time, node load balancing, task achievement rate, and resource throughput efficiency. Furthermore, it proposes utilizing deep reinforcement learning algorithms for computation to improve the overall computing power utilization efficiency of the system.

[0048] 1. Problem Modeling (1) Definition of system indicators The objective function of the system within time slot t is defined as follows:

[0049] in, For overall operating costs; The average time to complete the task; This refers to the load balancing performance of nodes. For task completion rate; For resource throughput efficiency; Let the weight parameters satisfy:

[0050] The weights can be set according to the actual deployment needs to achieve different optimization focuses.

[0051] (2) Calculation method of optimization index a. Average task completion time

[0052] Suppose the system completes M tasks within time slot t, and the completion time of task i is... , but:

[0053] Used to measure the overall task execution latency level of the system.

[0054] b. Node load balancing

[0055] Suppose the system includes The computing node, the first The resource load of each node is The degree of load balancing can then be represented by the coefficient of variation (CV).

[0056] in:

[0057]

[0058] The higher the value of this indicator, the more balanced the load.

[0059] c. Task completion rate

[0060] Task completion rate measures how well the scheduling strategy meets task requirements, including task latency and resource constraints. Let time slots be defined. Internal Communist Party Receiving Task There are [number] tasks, of which [number] tasks were successfully assigned and met the requirements. ,but:

[0061] If a task meets its maximum tolerable latency constraints and resource requirements, it is considered a successful task.

[0062] d. Resource throughput efficiency

[0063] Set system time slots The total internal resource supply is The amount of resources effectively allocated and actually executed is ,but:

[0064] Used to measure the effectiveness of system resource input and actual output.

[0065] (3) Optimization Objective Based on the above indicators, the scheduling and resource allocation optimization problem in this embodiment can be formally expressed as:

[0066] That is, minimizing the overall system cost while satisfying multiple constraints. Let the task set be... The node set is ,Task Assigned to nodes The decision variables are The constraints include, but are not limited to:

[0067] The task completion time must not exceed the maximum tolerable delay of the task;

[0068] Node resources cannot exceed their maximum available resources;

[0069] Each task can only be assigned to a single node for execution;

[0070] The actual resources used by the system must not exceed the total available resources.

[0071] 2. Markov Decision Process Modeling As part of the field of artificial intelligence, reinforcement learning maximizes benefits through the interaction and learning between intelligent agents and their environment. In recent years, the continuous development of deep learning has made it possible for computers to learn and represent high-dimensional features of data. Its main process is as follows: Figure 2 .

[0072] (1) State space In this embodiment, to achieve intelligent scheduling of multi-node computing resources, the system operating state of the computing network at any scheduling time t is defined as the state space. The state space It includes at least the node resource status vector and the task demand vector to be scheduled.

[0073] a. Node resource state vector Define the node resource state vector as , used to characterize the Each node at the current moment The schedulable computing power capacity is expressed as follows:

[0074] Each component represents: Number of available CPU cores; Number of available GPU computing units; Available video memory capacity; Available memory capacity; Outbound bandwidth availability; Real-time load percentage of nodes; : Estimated completion time for the task queue.

[0075] b. Task demand vector to be scheduled Define the task requirement vector to be scheduled as follows: This is used to describe the resource requirements and service quality requirements of the currently scheduled task, and its expression is as follows:

[0076] Each component represents: The amount of CPU resources required for the task; : The amount of GPU resources required for the task; : Memory required for the task; The amount of bandwidth resources required for the task; : Task data input scale; : Task execution latency constraints; Task priority. The system state space is constructed using the above vectorized representation. ,in This represents the total number of nodes, thus enabling a unified mathematical description of the resource status and task requirements of each node.

[0077] (2) Action space In this embodiment, for each task to be scheduled, the scheduling decision action is to select a target node from the candidate node set for task allocation. The action space is defined as:

[0078] in, This indicates that the currently scheduled task will be assigned to the task with the number [number]. The node is executed.

[0079] To ensure the feasibility of scheduling actions, the set of actions can be screened for legality based on node resource status information before action selection. Nodes that do not meet the task resource requirements or cannot meet service latency constraints can be blocked from the available actions to avoid unexecutable scheduling behavior.

[0080] (3) Design of reward function This embodiment introduces a deep reinforcement learning mechanism to achieve adaptive optimization of the scheduling strategy by constructing an immediate reward function. To ensure that the reward result can effectively reflect the degree of improvement of the current scheduling decision on the overall system operation, this embodiment adopts a linearly weighted immediate reward function, which is constructed as follows: a. Definition of Change in System Overall Cost Suppose the system is at time 10:00. The overall operating cost is At any moment Updated after executing the scheduling decision The immediate benefit brought by this scheduling is defined as:

[0081] Among them, comprehensive operating costs It can be composed of multiple performance indicators weighted by weight, such as the average task completion time. Node load balancing Task success rate Resource throughput efficiency Each indicator is normalized and then summed according to a preset weight.

[0082] b. Penalties for not completing the task To avoid scheduling strategies focusing solely on the current task's reward while ignoring the risk of backlogged tasks, this embodiment introduces the proportion of unprocessed tasks as a penalty. Let's assume at time... The number of remaining tasks to be scheduled is The initial total number of tasks is The percentage of uncompleted tasks is expressed as follows:

[0083] c. Instant reward function Based on the above-mentioned relationship between benefits and penalties, this embodiment defines the immediate reward function as follows:

[0084] in, This is a penalty coefficient used to balance the improvement in system benefits with the risk of future task backlog.

[0085] Based on the above reward function, a positive reward can be obtained when the scheduling behavior reduces the overall cost; if the scheduling leads to the deterioration of system performance or the risk of task backlog, the immediate reward will be reduced or become negative, thereby guiding the intelligent scheduling model to converge towards the direction of overall performance optimization.

[0086] 3. DDQN-based computational task scheduling strategy For computing network environments with multiple users and multiple computing nodes, traditional Q-learning-based task scheduling methods store the Q-values ​​corresponding to states and actions. While decision-making is feasible, the state space and action space expand exponentially with the increase in the number of system nodes and the scale of user tasks, leading to high storage overhead for Q-tables and difficulty in meeting real-time scheduling requirements. To address the issues of Q-table dimensionality explosion and unstable estimation in high-dimensional state spaces, deep Q-networks (DQNs) are typically used. These neural networks approximate Q-values ​​under different state-action combinations, handling continuous, high-dimensional state inputs. However, traditional DQNs directly update Q-values ​​using a maximization estimation method, which can easily cause deviations in target value estimation, thus affecting the network's convergence stability.

[0087] To address this, the present invention proposes a DDQN-based computational task scheduling method for multi-node computing network resource allocation, in order to reduce estimation bias and improve learning stability.

[0088] Specifically, the resource requirements of the currently scheduled tasks and the available resources of each node in the system are used to construct a state vector. The data is input into the DDQN network, and the network parameters are initialized. First, the network is evaluated. Select the candidate action with the largest Q value Subsequently, by the target network Calculate the target Q value based on this action. Execute the action. The task will be scheduled to the selected target node, and the system will receive a reward based on the task execution result. and transition to a new system state. Empirical Sample The data is stored in an experience pool, and during the training phase, mini-batch samples are randomly drawn from the experience pool to update the network. The target Q-value is estimated using a dual-network structure of DDQN to reduce estimation bias caused by maximization operations. The update method is as follows: actions are selected by the evaluation network. The target value is calculated by the target network:

[0089] in, The discount factor is used. The evaluation network parameters are continuously updated by minimizing the mean square error between the estimated Q-value and the target Q-value, and these parameters are synchronized to the target network at fixed steps to improve training stability and convergence performance. This invention introduces a task scheduling strategy based on the DDQN structure, enabling more accurate value assessment and convergence control in a high-dimensional resource state space, thus improving the intelligence, stability, and resource utilization efficiency of task scheduling in a computing power network environment.

[0090] The overall training and inference process of this method is as follows. The process includes steps such as task input, state extraction, action decision-making, execution scheduling, reward feedback, and network update. Step S1: Initialize the parameters of the online network and the target network, build the experience replay pool, and set hyperparameters such as learning rate, discount factor, and synchronization period. Step S2: Obtain the current system status This includes information on available resources at each node and characteristics of the queue of tasks to be scheduled. Step S3: Based on -greedy strategy selects actions, i.e., based on probability. Randomly select nodes, with probability Select the node with the best current valuation:

[0091] Step S4: Perform the action The corresponding task will be scheduled to the target node, and the system resource status will be updated.

[0092] Step S5: Calculate the reward And collect the next system status. .

[0093] Step S6: Combine the empirical quadruples The samples are stored in the experience replay pool and randomly sampled from it for network training.

[0094] Step S7: Calculate the target value using the DDQN update rule. The online network parameters are updated using a loss function.

[0095] Step S8: Periodically perform target network parameter synchronization operations.

[0096] Step S9: Repeat steps S2 to S8 until training converges or the set number of iterations is reached.

[0097] Example 2: This embodiment provides a task scheduling and resource allocation system based on deep reinforcement learning, including: The data acquisition module is used to obtain the resource status of each computing node in the edge computing network and the demand characteristics of the tasks to be scheduled. The model training module is used to store experience replay data and train an intelligent scheduling strategy based on the Dual Deep Q Network (DDQN) algorithm, updating model parameters by minimizing the error between the evaluation network and the target network. The scheduling decision module is used to load the trained model strategy and output the target node for task allocation based on the real-time status. The task execution module is used to distribute tasks to target nodes for execution and to provide feedback on execution results and system status changes.

[0098] Example 3: A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0099] This invention employs a Deep Dual-Q Network (DDQN) framework, using experience replay and a target network mechanism to stably update the policy function. During training, the model continuously interacts with the environment, gradually improving its task scheduling capabilities in dynamic and changing environments by sampling historical experience and current state decisions. This method can automatically learn how to efficiently allocate resources under conditions of changing resources and diverse tasks.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An edge-computing-node-oriented task scheduling and resource allocation method, characterized in that, The method specifically comprises the following steps: S1: constructing a multi-objective optimization model under an edge computing environment; the model is to minimize a system comprehensive operation cost as an objective under a constraint condition, and the comprehensive operation cost is a weighted combination of a task average completion time, a node load balancing degree, a task achievement rate, and a resource throughput efficiency; S2: modeling a task scheduling and resource allocation optimization problem as a Markov decision process, and defining a state space, an action space, and a reward function; the state space comprises a node resource state vector and a to-be-scheduled task demand vector; the action represents a scheduling decision of the system in a specific state, that is, assigning a certain task to a certain edge node; S3: constructing a computing task scheduling model based on a double-depth Q network, that is, adopting a deep double Q network framework, and stably updating a policy function through an experience replay and a target network mechanism; in a training process, the model constantly interacts with an environment, makes a decision through sampling historical experience and a current state, and gradually improves a task scheduling capability in a dynamic and changeable environment; S4: based on current node states and task information, outputting a reasonable task allocation scheme by using a trained policy.

2. The method of claim 1, wherein, In step S1, the multi-objective optimization model under the edge computing environment is expressed as: wherein, denotes the overall running cost, denotes the completion time of a task , denotes the maximum tolerable time delay of a task , denotes the set of tasks, denotes the set of nodes, denotes the decision variable of a task assigned to a node , denotes the amount of resource to be allocated, denotes the maximum available resource of a node , denotes the amount of resource effectively allocated and actually executed within a system time slot , denotes the total amount of resource supply within a system time slot .

3. The method of claim 2, wherein, In step S1, the formula for calculating the total operation cost C is: C = a + b * (P + Q) wherein, is the average task completion time, is the node load balancing degree, is the task achievement rate, is the resource throughput efficiency; is the weight parameter, satisfying .

4. The method of claim 3, wherein, In step S1, the task average completion time The formula for calculating the task average completion time is: wherein, is the completion time of task i, and M is the number of tasks completed by the system in time slot t.

5. The method of claim 3, wherein, In step S1, the node load balancing degree The calculation formula is: wherein, , , is the number of computing power nodes of the system, is the resource load of the th node.

6. The method of claim 3, wherein, In step S1, the task achievement rate The calculation formula is: wherein, the number of tasks successfully allocated and satisfied within the time slot the number of tasks successfully allocated and satisfied within the time slot the number of tasks successfully allocated and satisfied within the time slot the number of tasks successfully allocated and satisfied within the time slot 7. The method of claim 3, wherein, In step S1, the resource throughput efficiency is calculated by the following formula: wherein, is the system time slot is the total resource supply amount within the system time slot, is the system time slot is the amount of resources effectively allocated within the system time slot and actually executed.

8. The method of claim 1, wherein, In step S2, the state space comprises: 1) Node resource state vector: define the node resource state vector as , which is used to represent the schedulable computing power resource capability of the th node at the current time , and its expression is as follows: wherein, is the number of available CPU cores; is the number of available GPU compute units; is the available video memory capacity; is the available memory capacity; is the available outbound bandwidth; is the real-time load occupancy of the node; is the estimated completion time of the task queue; 2) The demand vector of the task to be scheduled: The demand vector of the task to be scheduled is defined as , which is used to describe the resource demand characteristics and quality of service requirements of the current task to be scheduled, and its expression is as follows: wherein, an amount of CPU resources required for the task; an amount of GPU resources required for the task; an amount of memory capacity required for the task; an amount of bandwidth resources required for the task; a size of data input for the task; a latency constraint for execution of the task; a priority of the task; By the above vectorization representation, the system state space is constructed wherein is the total number of nodes, thereby achieving a unified mathematical description of the resource status of each node and the task demand.

9. The method of claim 1, wherein, In step S2, the reward function is a linearly weighted immediate reward function, and an expression is as follows: wherein, is the immediate reward function, is the system integrated cost change amount, is the penalty coefficient, used to adjust the balance between system revenue improvement and future task accumulation risk; represents the proportion of incomplete tasks, is the initial total number of tasks at time is the number of remaining tasks to be scheduled, is the initial total number of tasks.

10. The method of claim 1, wherein, In step S3, the training process specifically includes: inputting the resource requirement of the current to-be-scheduled task and the available resource of each node of the system into a state vector , and initializing network parameters; first, evaluating the network , selecting a candidate action with the maximum Q value , then calculating a target Q value according to the action by a target network ; performing the action , that is, scheduling the task to the selected target node, obtaining a reward from the system according to the task execution result, and transferring to a new system state ; storing experience samples in an experience pool, and updating the network by randomly extracting a small batch of samples from the experience pool in the training stage; the target Q value is estimated by using a double-network structure of the double-depth Q network, so as to reduce the estimation deviation caused by the maximization operation; and the updating manner is: selecting an action by the evaluation network , and calculating a target value by the target network: wherein, is a discount factor; the evaluation network parameters are constantly updated by minimizing the mean square error between the estimated Q value and the target Q value, and the evaluation network parameters are synchronized to the target network according to a fixed number of steps, so as to improve the stability and convergence performance of the training.

Images