Computing power scheduling method and system applied to large-scale network simulation system

Abstract

The invention relates to the technical field of computing power scheduling, in particular to a computing power scheduling method and system applied to a large-scale network simulation system, and the method comprises the steps: obtaining a resource occupation state information set of all computing power nodes; constructing a task queue of all computing power nodes; setting deep reinforcement learning parameters and initializing a neural network; taking the computing power node as an intelligent agent to interact with the environment to generate an experience sample, and training a neural network by adopting a deep reinforcement learning method to obtain a preliminary task scheduling strategy; further optimizing the initial task scheduling by using an ant colony algorithm to obtain a final task scheduling strategy; and scheduling the simulation task by adopting the final task scheduling strategy. According to the method, during task scheduling, the real-time resource state of the computing power node is considered, the particularity of the tasks, the priority of the tasks and the dependency relationship between the tasks are also considered, the utilization efficiency of the computing power is improved, and the simulation time is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of computing power scheduling, and in particular to a computing power scheduling method and system applied to a large-scale network simulation system. Background Art

[0002] In today's digital age, the development of network communication technology has become one of the key driving forces of modern social life and industrial development. With the rapid popularization and development of emerging technologies such as the Internet of Things, 6G networks, and smart cities, more and more network devices are being deployed on a large scale to build a large and complex communication network. However, with the continuous expansion of the network scale, the construction and management of the network face many challenges. Large-scale deployed networks often have high heterogeneity, complex topology, and dynamic environmental changes, which leads to the instability and unpredictability of network performance. At the same time, different application scenarios have different requirements for the network. Therefore, before the actual network is built, it is often verified in advance on the network simulation system. Nowadays, the computing power of a single computing node in the network simulation system can no longer meet the requirements for the efficiency of simulation task processing. Large-scale network simulation systems have begun to use distributed clusters, including multiple computing nodes, to improve the efficiency of simulation task processing.

[0003] In a network simulation system, there are often multiple computing nodes, each of which has its own simulation task. However, different simulation tasks may consume computing resources in different ways, and the resources of each computing node may also vary greatly. Therefore, computing scheduling is usually achieved by scheduling simulation tasks.

[0004] Most of the existing task scheduling algorithms are based on heuristic algorithms to reallocate tasks. Although a task scheduling plan can be obtained directly after certain calculations, it is not suitable for large-scale network simulation systems. Large-scale network simulation systems contain multiple computing nodes, each of which has a large number of simulation tasks. These simulation tasks may have dependencies between tasks, and may also have some special features, such as some urgent tasks that need to be completed in advance. If heuristic algorithms are used for computing power scheduling, they cannot be well applied to network simulation systems to maximize the utilization of computing power resources.

[0005] In summary, it is necessary to develop an optimized scheduling strategy that can combine the computing power of computing nodes in large-scale network simulation systems, dynamically adapt to the resource requirements of different simulation tasks, maximize the use of the system's computing resources, and avoid resource waste and uneven load. At the same time, the scheduling strategy should also have a certain degree of flexibility and scalability, and be able to adjust the computing power allocation in real time in a complex and changeable network environment to ensure the efficiency, stability and response speed of the simulation system. In addition, the new scheduling scheme needs to take into account the particularity of the task, the dependency between tasks and the load status of the computing nodes, and reduce the processing delay of the simulation task by reasonably allocating resources, thereby improving the overall performance of the system and user experience. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a computing power scheduling method and system applied to a large-scale network simulation system.

[0007] On the one hand, the present invention provides a computing power scheduling method applied to a large-scale network simulation system, comprising:

[0008] S1: Obtain the resource occupancy status information set of all computing nodes as the original set, and normalize the original set to obtain a standard sample set;

[0009] S2: Build task queues for all computing nodes and pre-process the task queues;

[0010] S3: Set deep reinforcement learning parameters and initialize the neural network according to the standard sample set of computing power nodes;

[0011] S4: The computing nodes are used as intelligent agents to interact with the environment to generate experience samples. Based on the experience samples, the neural network is trained using deep reinforcement learning methods to obtain a preliminary task scheduling strategy.

[0012] S5: Use the ant colony algorithm to further optimize the preliminary task scheduling to obtain the final task scheduling strategy;

[0013] S6: Use the final task scheduling strategy to schedule the simulation tasks.

[0014] On the other hand, the present invention provides a large-scale network simulation system, including: a simulation node, a computing node, a business center, a monitoring center, and a training module;

[0015] The simulation node is a virtualized node used to simulate a real device in a network simulation system. It is mainly implemented through a docker container or a lightweight container based on a Linux namespace, and is mainly used as a carrier for executing simulation tasks.

[0016] The computing nodes are mainly simulation servers for building simulation nodes, as well as computing services that provide computing assistance, using real CPU and memory resources. Because simulation nodes still use the physical resources of real devices when performing simulation tasks, in order to better implement computing power scheduling, they are uniformly abstracted as computing power nodes, that is, nodes that use computing power resources to complete simulation tasks;

[0017] The business center is mainly a module for the network simulation system to send simulation tasks to simulation nodes, and is mainly responsible for interacting with the outside world to realize the simulation task requirements of the outside world;

[0018] The monitoring center is mainly a module used by the network simulation system to monitor the resource consumption and task load of computing nodes, and collects parameters for computing power scheduling using the computing power scheduling model;

[0019] The training module is used to set deep reinforcement learning parameters according to the state information of the computing power node and train the neural network according to the experience samples generated by the interaction between the computing power node and the environment;

[0020] The computing power node determines the target of task scheduling according to the trained neural network, and transfers the computing power of the computing power node according to the task scheduling to maximize the utilization of resources.

[0021] Beneficial effects of the present invention:

[0022] This application proposes a computing power scheduling method based on the combination of deep reinforcement learning and ant colony algorithm, which is suitable for large-scale network simulation systems to improve the utilization efficiency of computing power resources and shorten the simulation time. The core steps of this method include: obtaining the resource occupancy status of the computing power node through the monitoring center, and normalizing it to ensure that different resource characteristics are compared and calculated on the same scale; then, constructing a task queue for each computing power node, considering the dependencies and resource consumption between tasks, sorting them according to the resource consumption and priority of the tasks, and layering the tasks to ensure that tasks at the same level can be executed in parallel; based on the normalized standard sample set, a deep reinforcement learning model (using the DDPG algorithm) is constructed. The model generates a preliminary task scheduling strategy by simulating the interaction between the computing power node and the environment; the ant colony algorithm is used to optimize the preliminary scheduling strategy for the scheduling needs of sudden tasks. By setting the pheromone concentration and the heuristic factor, the ant colony algorithm can dynamically adjust the scheduling decision, optimize the task allocation of the computing power node, and finally obtain the global optimal task scheduling strategy. The final scheduling strategy is applied to the scheduling of simulation tasks. The system allocates computing power resources according to the generated strategy, and adjusts the resource usage in real time to ensure that the simulation tasks are efficiently processed according to the priority and dependency. This method can effectively solve the problem of computing resource scheduling in large-scale network simulation systems. Through the combination of deep reinforcement learning and ant colony algorithm, it significantly improves resource utilization, reduces system load imbalance and simulation delay, and improves the operating efficiency and stability of the simulation system.

[0023] The present invention not only considers the real-time resource status of the computing power node when scheduling tasks, but also considers the particularity of the task, the priority of the task, and the dependency between tasks. The present invention uses a reward mechanism to enable the intelligent agent to automatically select a better computing power node to complete the simulation task. Using a trained neural network to schedule tasks can improve the utilization efficiency of computing power and reduce simulation time. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a computing power scheduling method applied to a large-scale network simulation system in the present invention;

[0025] Figure 2 This is a neural network training block diagram in the present invention;

[0026] Figure 3 This is the flow chart of the ant colony algorithm. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The present invention proposes a computing power scheduling method applied to a large-scale network simulation system. Figure 1 As shown, the method comprises:

[0029] S1: Obtain the resource occupancy status information set of all computing power nodes as the original set, and normalize the set to obtain a standard sample set.

[0030] The monitoring center in the network simulation system will obtain the resource usage of each computing node in a predetermined order, including the CPU, memory, network bandwidth, and link latency of the computing node; based on the collected resource usage data of the computing node, normalization is performed to scale all data to the range of [0,1] to form a standard sample set. This normalization process ensures that feature data of different dimensions can participate in calculations at the same scale.

[0031] S2: Build task queues for all computing nodes and pre-process the task queues to provide reasonable input data for subsequent scheduling decisions.

[0032] Before the simulation starts, the business center will build a task queue for each computing node based on the network simulation requirements of the overall system and pre-process the task queue.

[0033] The preprocessing process includes:

[0034] Construct a set of task queues and use G(T,L,ω) to represent the task queues;

[0035] First, the tasks of the computing nodes are expressed using a set T, T = {T1, T1, ..., TN}, where Ti represents the i-th task. The computing resource consumption of task Ti is recorded as Li.

[0036] A two-dimensional matrix E is used to represent an N*N matrix, where E(i,j)=1 indicates that there is a dependency constraint between Ti and Tj, and task Tj can only be executed after Ti is completed. If E(i,j)=0, it means that there is no dependency relationship between the two parties.

[0037] According to E, find ω in the task queue set;

[0038] S3: Set the deep reinforcement learning parameters and initialize the weights of the neural network based on the standard sample set of computing power nodes.

[0039] Set up the state space, action space, and reward function according to the Markov decision process;

[0040] Initialize the neural network weights, specifically:

[0041] The present invention adopts DDPG algorithm, such as Figure 2 As shown, initialize the target policy network and target value network, as well as the parameters of the local policy network and local value network. Initialize the experience replay buffer, set the batch size, learning rate, reward discount factor, soft update parameters, etc.

[0042] S4: The computing nodes are used as intelligent agents to interact with the environment to generate experience samples. The neural network is trained using deep reinforcement learning methods based on the experience samples to obtain a preliminary task scheduling strategy.

[0043] First, the initial state of the computing power node is input into the deep reinforcement learning model built on the neural network. The deep reinforcement learning model randomly selects action a according to the current strategy in each iteration.

[0044] Execute action a and observe the next state S ′ and reward R;

[0045] The state S, action a, reward R, next state S ′ Stored in the experience replay pool;

[0046] A batch of samples are randomly drawn from the experience replay pool to update the parameters of the policy and value networks;

[0047] Calculate the target Q value, where Q refers to the long-term report of an action, starting from the current state S, taking a specific a under the characteristic strategy to represent the predicted value;

[0048] Update the value network, calculate the loss function of the value network, use the mean square error loss function to minimize the difference between the target Q value and the a value, and update the parameters of the value network. The loss function is:

[0049]

[0050] Update the policy network, calculate the loss function of the policy network, use the policy gradient to maximize the Q value of the current state, and update the parameters of the policy network, where the policy function is:

[0051]

[0052] In the DDPG algorithm, both the policy network and the value network are based on deep neural networks, and the gradient descent method is used to update parameters.

[0053] The update functions of the policy network and the value network are:

[0054] θ Q′ ←τθ Q +(1-τ)θ Q′ ,

[0055] θ μ′ ←τθ μ +(1-τ)θ μ′

[0056] Repeat the above steps until the optimal Q(s,a|θ) is reached. Q ) value and output the corresponding scheduling strategy as a preliminary scheduling strategy.

[0057] S5: Use the optimized ant colony algorithm to further optimize the preliminary task scheduling to obtain the final task scheduling strategy;

[0058] The steps of the ant colony algorithm are as follows: Figure 3 As shown, mainly:

[0059] Step 1. Get the computing power node status information;

[0060] Step 2. Calculate the profit matrix;

[0061] Step 3. Set the initial pheromone for the computing power node;

[0062] Step 4. Initialize the ant colony and randomly distribute it among the computing nodes;

[0063] Step 5. Calculate the transfer probability of ant k and select the next hop node;

[0064] Step 6. Update local pheromones after the ant completes the current search;

[0065] Step 7. After all ants have completed their search for this round, update the global pheromone;

[0066] Step 8. When the number of iterations is reached, output the scheduling strategy.

[0067] S6: Use the final task scheduling strategy to schedule the simulation tasks.

[0068] Using the final resource scheduling strategy, the computing power node will schedule tasks through the scheduling strategy before performing task simulation to reduce the simulation time.

[0069] In summary, the present invention not only considers the real-time resource status of the computing power node when scheduling tasks, but also considers the particularity of the task, the priority of the task, and the dependency between tasks. The present invention uses a reward mechanism to enable the intelligent agent to automatically select a better computing power node to complete the simulation task. Using a trained neural network to schedule tasks can improve the utilization efficiency of computing power and reduce simulation time.

[0070] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A computing power scheduling method applied to a large-scale network simulation system, characterized in that: include: S1: Obtain the resource occupancy status information set of all computing nodes as the original set, and normalize the original set to obtain a standard sample set; S2: Build task queues for all computing nodes and pre-process the task queues; S3: Set deep reinforcement learning parameters and initialize the neural network according to the standard sample set of computing power nodes; S4: The computing nodes are used as intelligent agents to interact with the environment to generate experience samples. Based on the experience samples, the neural network is trained using deep reinforcement learning methods to obtain a preliminary task scheduling strategy. S5: Use the ant colony algorithm to further optimize the preliminary task scheduling to obtain the final task scheduling strategy; S6: Use the final task scheduling strategy to schedule the simulation tasks.

2. The computing power scheduling method applied to a large-scale network simulation system according to claim 1, characterized in that: Obtain the resource occupancy status information set of all computing power nodes as the original set, and normalize the original set to obtain the standard sample set, including: The resource occupancy status information includes: the remaining CPU resources of the computing power node, the remaining memory resources, the remaining bandwidth flow of the network link, and the transmission delay between the computing power nodes; Based on the collected computing node resource occupancy status information, normalization processing is performed to obtain a standard sample set; The normalization process mainly scales all data to the range of [0,1] to obtain a standard sample set; The standard sample set includes in, Indicates the normalized CPU resource consumption of the target computing power node j at the beginning of the simulation; Indicates the normalized memory resource usage of the target computing power node j at the beginning of the simulation; represents the normalized bandwidth resource usage of the target computing power node j at the beginning of the simulation; D j Represents the set of transmission delays between computing node j and other computing nodes at the beginning of the simulation.

3. The computing power scheduling method applied to a large-scale network simulation system according to claim 1 is characterized in that: Build the task queues of all computing nodes and pre-process the task queues, including: Construct a task queue set G(T,L,ω); where G represents the task queue set of the computing node, T represents the task queue of the computing node, T={T1,T2,…Ti...,TN}, Ti represents the i-th task; L represents the estimated consumption of computing resources of the task, L={L1,L2,…Ti...,LN}, Li represents the estimated consumption of computing resources of task Ti; ω represents the priority of the task, ω={ω1,ω2,…ωi...,ωN}, ωi represents the priority of task Ti. When task Ti has no previous level task, it is considered a level 1 task. If task Ti has a preceding dependent task, the level of task Ti is set to the maximum level of its preceding task plus 1; the task hierarchy is used as the order of task execution, and tasks of one level are executed at a time. There is no dependency between tasks at the same level, and they can be executed in parallel.

4. The computing power scheduling method applied to a large-scale network simulation system according to claim 1, characterized in that: Set the deep reinforcement learning parameters according to the standard sample set of computing power nodes, including: According to the Markov decision process, set the state space S, action space a, and reward function R: The state space S is: Among them, S t represents the overall state of the simulation system after completing the task with task level t (t∈[0, max(ωi)]), ωi represents the priority of task Ti; Indicates the state of computing power node j at this time, represents the normalized CPU resource consumption of computing node j after completing task level t, represents the normalized memory resource usage of computing node j after completing task level t, represents the normalized bandwidth resource usage of computing power node j after completing task level t, represents the transmission delay set between computing node j and other computing nodes after completing task level t; Represents the task queue set of computing node j at this time, represents the remaining tasks of computing node j, L represents the estimated consumption of computing resources of the task, and ω represents the priority of the task; The action space is for the simulation system to select computing nodes to complete all simulation tasks of level t; The reward function is Among them, Rt represents the reward function after completing the task with task level t, and σt represents the time after completing the task with task level t; Among them, δi represents the start time of task Ti, satisfying δi' represents the latest end time of the predecessor tasks of task Ti, ω represents the priority of the task, ωi represents the task level of task Ti, λij represents the execution efficiency of task Ti on computing node j, Xj represents the resource usage of computing node j at this time; Li represents the estimated consumption of computing resources of task Ti; Dj represents the propagation delay of the task between computing node j and the node where the task is located, and M represents the number of tasks.

5. The computing power scheduling method applied to a large-scale network simulation system according to claim 1, characterized in that: Initialize the neural network, including: Using the critic network weights θ Q and the actor network weights θ μ Initialize the critic network Q and the actor network μ separately; at the same time, initialize the parameters in the target network, using θ Q′ Corresponding to the Q′ of the critic network, use θ μ′ μ′ corresponding to the actor network; where Q′ represents the objective function of the critic network Q, μ′ represents the objective function of the actor network μ, and θ Q′ represents the weight of the Q′ network, θ μ′ represents the weight of the μ′ network; Initialize the experience replay pool and set the capacity of the replay pool to B.

6. The computing power scheduling method applied to a large-scale network simulation system according to claim 1, characterized in that: The computing nodes are used as intelligent agents to interact with the environment to generate experience samples. Based on the experience samples, the neural network is trained using the deep reinforcement learning method to obtain a preliminary task scheduling strategy, including: Randomly select action a(t), execute action a(t), obtain reward R(t), and observe the updated state; Store the experience sample (St, at, Rt, St+1) in the experience replay pool; Calculate and update the loss function of the critic network: Among them, Loss represents the loss function of the critic network, B represents the size of the experience revisit pool, Rt represents the reward obtained at time step t, γ represents the decay factor, and its value is between [0,1]; Q′(St+1,a′|θ μ′ ) represents the value of Q′ in the next state St+1 and the target action a′, which is determined by the target strategy network θ μ′ Generated, that is, the optimal action of the next state St+1 is calculated through the μ′ network, Q′ represents the objective function of the critic network; Q(St,at|θ Q′ ) represents the Q value of the current critic network Q under the current state St and action at, θ Q′ is the weight of the Q′ network; Update the actor network using policy gradients, whose policy is: in, represents the update strategy of the actor network, θ μ represents the actor network weight, θ Q represents the critic network weight, represents the gradient of the actor network, which represents the gradient of the Q function at the current state st and through the policy network a = μ(s t |θ μ ) Derivative on the action a generated; represents the actor network with respect to its parameters θ μ The gradient of the policy network to the state s t Changes in output actions; Update target network parameters periodically: i Q′ ←tth Q +(1-τ)θ Q′ , i μ′ ←tth μ +(1-τ)θ μ′ Among them, θ Q′ represents the objective function of the critic network; θ μ′ represents the objective function of the actor network; τ represents the soft update coefficient of the target network update; θ Q represents the weight of the critic network, θ μ is the weight of the actor network; Regularly reduce the probability in represents the exploration rate; Update the state space S(t)←S(t+1); Repeat the above steps until the optimal Q(s,a|θ) is reached. Q ) value and output the corresponding scheduling strategy as a preliminary scheduling strategy.

7. The computing power scheduling method applied to a large-scale network simulation system according to claim 1, characterized in that: The ant colony algorithm is used to further optimize the preliminary task scheduling to obtain the final task scheduling strategy, including: Initialize the system, regard the performance parameters of the computing nodes as node pheromones, and randomly place k ants on k computing nodes. Set the number of ant iterations, as well as the heuristic factor and pheromone factor β to control the path selection weight of the ants. Set the path selection rule of the ant, that is, the probability of selecting the next computing power node as j is: in, It is expressed as the probability that the kth ant chooses the next computing node j at time t′; β represents the information inspiration factor; cj is the expected profit of the next task executed on computing node j; τj(t′) is the pheromone concentration of computing node j at time t′; ci is the expected profit of the next task executed on computing node i; τi(t′) is the pheromone concentration of computing node i at time t′, and n represents the number of computing nodes; Set the update rules for pheromones: τi(t′+1)=(1-e)τi(t′)+ψmax(M(Ik,i)) Among them, τi(t′+1) is the pheromone concentration of the computing power node at time t′+1; e is the volatility coefficient of pheromone; τi(t′) is the concentration of pheromone before; ψ is the pheromone intensity, and M(Ik,i) is the allocation method of the i-th ant in the k-th iteration.

8. A large-scale network simulation system, used to implement a computing power scheduling method applied to a large-scale network simulation system as described in claims 1-7, characterized in that: include: Simulation nodes, computing nodes, business centers, monitoring centers, and training modules; The simulation node is a virtualized node used to simulate a real device in a network simulation system. It is mainly implemented through a docker container or a lightweight container based on a Linux namespace, and is mainly used as a carrier for executing simulation tasks. The computing nodes are mainly simulation servers for building simulation nodes, as well as computing services that provide computing assistance, using real CPU and memory resources. Because simulation nodes still use the physical resources of real devices when performing simulation tasks, in order to better implement computing power scheduling, they are uniformly abstracted as computing power nodes, that is, nodes that use computing power resources to complete simulation tasks; The business center is mainly a module for the network simulation system to send simulation tasks to simulation nodes, and is mainly responsible for interacting with the outside world to realize the simulation task requirements of the outside world; The monitoring center is mainly a module used by the network simulation system to monitor the resource consumption and task load of computing nodes, and collects parameters for computing power scheduling using the computing power scheduling model; The training module is used to set deep reinforcement learning parameters according to the state information of the computing power node and train the neural network according to the experience samples generated by the interaction between the computing power node and the environment; The computing power node determines the target of task scheduling according to the trained neural network, and transfers the computing power of the computing power node according to the task scheduling to maximize the utilization of resources.

Images