A computing power scheduling method and system applied to a large-scale network simulation system

Abstract

The application relates to the technical field of computing power scheduling, in particular to a computing power scheduling method and system applied to a large-scale network simulation system.The method comprises the following steps: acquiring a resource occupation state information set of all computing power nodes; constructing a task queue of all computing power nodes; setting deep reinforcement learning parameters and initializing a neural network; interacting with the environment to generate experience samples by taking the computing power node as an agent; training the neural network by using a deep reinforcement learning method to obtain a preliminary task scheduling strategy; further optimizing the preliminary task scheduling by using an ant colony algorithm to obtain a final task scheduling strategy; and scheduling simulation tasks by using the final task scheduling strategy.In the task scheduling, not only the real-time resource state of the computing power node is considered, but also the particularity of the task, the priority of the task and the dependency relationship between the tasks are considered, so that the utilization efficiency of the computing power is improved, and the simulation time is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computing power scheduling, and particularly relates to a computing power scheduling method and system applied to a large-scale network simulation system. BACKGROUND

[0002] In the current digital era, the development of network communication technology has become one of the key driving forces of modern social life and industrial development. With the rapid popularization and development of emerging technologies such as the Internet of Things, 6G networks, and smart cities, more and more network devices are being deployed on a large scale, building a huge and complex communication network. However, as the network scale continues to expand, the construction and management of the network face many challenges. Large-scale deployed networks often have high heterogeneity, complex topological structures, and dynamic environmental changes, which lead to instability and unpredictability of network performance. At the same time, different application scenarios have different requirements for the network. Therefore, before the actual network is built, it is often verified in advance on the network simulation system. Nowadays, the computing power of a single computing power node in the network simulation system cannot fully meet the requirements of simulation task processing efficiency, and large network simulation systems begin to use distributed clusters containing multiple computing power nodes to improve the efficiency of simulation task processing.

[0003] In the network simulation system, there are often multiple computing power nodes, each of which has its own simulation task. However, the consumption of computing power resources by different simulation tasks may differ greatly, and the resource differences of each computing power node may also be great, so simulation task scheduling is usually used to achieve computing power scheduling.

[0004] Most existing task scheduling algorithms are based on heuristic algorithms to redistribute tasks, which can directly obtain a task scheduling scheme after a certain calculation, but are not suitable for large-scale network simulation systems; large-scale network simulation systems contain multiple computing power nodes, each of which has a large number of simulation tasks, and these simulation tasks may have dependencies between tasks, or may have some special properties, such as some urgent tasks that need to be completed in advance; if a heuristic algorithm is used for computing power scheduling, it cannot be well adapted to the network simulation system to maximize the utilization of computing power resources.

[0005] In summary, it is necessary to develop an optimized scheduling strategy that can jointly utilize the computing power of nodes in a large-scale network simulation system, dynamically adapt to the resource needs of different simulation tasks, maximize the use of system computing power resources, and avoid resource waste and uneven load. At the same time, the scheduling strategy should also have flexibility and scalability to adjust the allocation of computing power in real time in complex and changing network environments, ensuring the efficiency, stability and response speed of the simulation system. In addition, the new scheduling scheme needs to consider the particularity of tasks, the dependency between tasks and the load status of computing power nodes, and reduce the processing delay of simulation tasks by reasonably allocating resources to improve the overall performance and user experience of the system. SUMMARY

[0006] To solve the above technical problems, the application provides a computing power scheduling method and system applied to a large-scale network simulation system.

[0007] In one aspect, the application provides a computing power scheduling method applied to a large-scale network simulation system, comprising:

[0008] S1: Obtain the resource occupation state information set of all computing power nodes as the original set, and normalize the original set to obtain a standard sample set;

[0009] S2: Construct a task queue of all computing power nodes and preprocess the task queue;

[0010] S3: Set the deep reinforcement learning parameters according to the computing power node standard sample set and initialize the neural network;

[0011] S4: Interact with the environment to generate experience samples by taking the computing power node as an agent, and train the neural network using the deep reinforcement learning method according to the experience samples to obtain a preliminary task scheduling strategy;

[0012] S5: Further optimize the preliminary task scheduling using the ant colony algorithm to obtain the final task scheduling strategy;

[0013] S6: Schedule the simulation task using the final task scheduling strategy.

[0014] In another aspect, the application provides a large-scale network simulation system, comprising: simulation nodes, computing power nodes, a business center, a monitoring center and a training module;

[0015] The simulation node is a virtualization node used to simulate real devices in the network simulation system, and the main implementation is through a docker container or a lightweight container based on Linux namespace, mainly used as a carrier for executing simulation tasks;

[0016] The computing power node is mainly a simulation server for building a simulation node and a computing service for providing computing assistance, and uses real CPU and memory resources, because the simulation node still uses the physical resources of the real device when performing a simulation task, in order to better realize computing power scheduling, the computing power node is uniformly abstracted as a computing power resource node, i.e., a node using computing power resources to complete a simulation task;

[0017] The business center is mainly a module for the network simulation system to issue a simulation task to the simulation node, and is mainly responsible for interaction with the outside world to realize the simulation task demand of the outside world;

[0018] The monitoring center is mainly a module for the network simulation system to monitor the resource consumption and task load of the computing power node, and collects parameters for using a computing power scheduling model to perform computing power scheduling;

[0019] The training module is used for setting deep reinforcement learning parameters according to the state information of the computing power node and training a neural network according to experience samples generated by the interaction between the computing power node and the environment;

[0020] The computing power node determines a task scheduling target according to the trained neural network, and realizes the transfer of computing power of the computing power node according to the task scheduling, so as to realize the maximum utilization of resources.

[0021] The beneficial effects of the present application are as follows:

[0022] The application proposes a computing power scheduling method based on the combination of deep reinforcement learning and ant colony algorithm, which is suitable for large-scale network simulation systems to improve the utilization efficiency of computing power resources and shorten the simulation time. The core steps of the method include: obtaining the resource occupation state of the computing power node through the monitoring center, and normalizing it to ensure that different resource characteristics are compared and calculated on the same scale; then, a task queue is constructed for each computing power node, considering the dependency relationship and resource consumption between tasks, and the tasks are sorted according to the resource consumption and priority of the tasks, and the tasks are layered to ensure that the tasks in the same level can be executed in parallel; based on the normalized standard sample set, a deep reinforcement learning model (using DDPG algorithm) is constructed. The model simulates the interaction between the computing power node and the environment to generate a preliminary task scheduling strategy; for the scheduling requirements of sudden tasks, the ant colony algorithm is used to optimize the preliminary scheduling strategy. By setting the pheromone concentration and the heuristic factor, the ant colony algorithm can dynamically adjust the scheduling decision, optimize the task allocation of the computing power node, and finally obtain the globally optimal task scheduling strategy. The final scheduling strategy is applied to the scheduling of simulation tasks, and the system allocates computing power resources according to the generated strategy and adjusts the resource usage in real time to ensure that the simulation tasks are efficiently processed according to the priority and dependency relationship. The method can effectively solve the computing power resource scheduling problem in large-scale network simulation systems, and through the combination of deep reinforcement learning and ant colony algorithm, it significantly improves the resource utilization, reduces the system load imbalance and simulation delay, and improves the running efficiency and stability of the simulation system.

[0023] In the task scheduling of the application, not only the real-time resource state of the computing power node is considered, but also the particularity of the task, the priority of the task and the dependency relationship between the tasks are considered. The application uses a reward mechanism to enable the agent to automatically select a better computing power node to complete the simulation task. Using the trained neural network for task scheduling can improve the utilization efficiency of computing power and reduce simulation time. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 A flowchart of a computing power scheduling method applied to a large-scale network simulation system in the application;

[0025] Figure 2 A neural network training block diagram in the application;

[0026] Figure 3 An ant colony algorithm flowchart. DETAILED DESCRIPTION

[0027] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0028] The present application provides a computing power scheduling method applied to a large-scale network simulation system, as shown in Figure 1 The method comprises:

[0029] S1: Obtain the resource occupation state information set of all computing power nodes as an original set, and perform normalization processing on the set to obtain a standard sample set.

[0030] The monitoring center in the network simulation system will obtain the resource occupation of each computing power node in a predetermined order, including the CPU, memory, network bandwidth of the computing power node, and the link delay between nodes; according to the collected resource occupation data of the computing power node, normalization processing is performed to scale all data to the range of [0, 1] to form a standard sample set. This normalization process ensures that feature data of different dimensions can participate in calculation on the same scale.

[0031] S2: Construct a task queue of all computing power nodes, and pre-process the task queue to provide reasonable input data for subsequent scheduling decisions.

[0032] The business center will construct a task queue for each computing power node according to the network simulation requirements of the overall system before the simulation starts, and pre-process the task queue.

[0033] The pre-processing process includes:

[0034] A task queue set is constructed, and G(T, L, ω) is used to represent the task queue;

[0035] The tasks of the computing power node are first expressed using a set T, T={T1, T1,..., TN}, where Ti represents the i-th task. The computing resource consumption of task Ti is denoted as Li.

[0036] A two-dimensional matrix E is used to represent an N*N matrix, where E(i,j)=1 indicates that there is a dependency constraint between Ti and Tj, and task Tj can only start execution after Ti is completed. If E(i,j)=0, it indicates that there is no dependency relationship between the two.

[0037] According to E, ω in the task queue set is obtained;

[0038] S3: Set the deep reinforcement learning parameters according to the computing power node standard sample set and initialize the weights of the neural network.

[0039] Set the state space, action space and reward function according to the Markov decision process;

[0040] Initialize the neural network weights, specifically:

[0041] The DDPG algorithm is adopted in the present application, as shown in the figure, the parameters of the target policy network and the target value network, and the local policy network and the local value network are initialized. The experience replay buffer is initialized, and the batch size, learning rate, reward discount factor, soft update parameter, etc. are set. Figure 2

[0042] S4: The computing power node is used as an agent to interact with the environment to generate experience samples, and the neural network is trained using the deep reinforcement learning method according to the experience samples to obtain a preliminary task scheduling strategy.

[0043] First, the initial state of the computing power node is input into the deep reinforcement learning model constructed based on the neural network, and the deep reinforcement learning model randomly selects an action a according to the current policy in each iteration;

[0044] Execute the action a, and observe the next state S ′ and the reward R;

[0045] Store the state S, action a, reward R, next state S ′ into the experience replay pool;

[0046] Randomly sample a batch of samples from the experience replay pool for updating the parameters of the policy and value networks;

[0047] Calculate the target Q value, where Q refers to an action long report, starting from the current state S, taking a specific a under the characteristic policy to represent the predicted value;

[0048] Update the value network, calculate the loss function of the value network, use the mean square error loss function to minimize the difference between the target Q value and the a value, and update the parameters of the value network. The loss function is:

[0049]

[0050] Update the policy network, calculate the loss function of the policy network, use the policy gradient to maximize the Q value of the current state, and update the parameters of the policy network, where the policy function is:

[0051]

[0052] ​In the DDPG algorithm, the policy network and the value network are both based on deep neural networks, and the gradient descent method is used for parameter updating.

[0053] where the update functions of the policy network and the value network are:

[0054] θ Q′ ←τθ Q +(1-τ)θ Q′ ,

[0055] θ μ′ ←τθ μ +(1-τ)θ μ′

[0056] Loop the above steps until the optimal Q(s, a|θ Q ) value is reached, and output the corresponding scheduling policy as a preliminary scheduling policy.

[0057] S5: Further optimize the preliminary task scheduling using the optimized ant colony algorithm to obtain the final task scheduling strategy;

[0058] The steps of the adopted ant colony algorithm are shown in Figure 3 , mainly including:

[0059] Step 1. Obtain the state information of the computing power nodes;

[0060] Step 2. Calculate the revenue matrix;

[0061] Step 3. Set the initial pheromone for the computing power nodes;

[0062] Step 4. Initialize the ant colony and randomly distribute it in the computing power nodes;

[0063] Step 5. Calculate the transition probability of ant k and select the next hop node;

[0064] Step 6. Update the local pheromone after the ant completes the current search;

[0065] Step 7. Update the global pheromone after all ants complete the search in this round;

[0066] Step 8. Output the scheduling policy when the number of iterations is reached.

[0067] S6: Use the final task scheduling strategy to schedule the simulation tasks.

[0068] Using the final resource scheduling strategy, the computing power nodes will perform task scheduling through the scheduling strategy before performing task simulation, reducing the simulation time.

[0069] To sum up, when scheduling tasks, the application not only considers the real-time resource state of the computing power node, but also considers the particularity of the task, the priority of the task, and the dependency relationship between tasks. The application uses a reward mechanism to enable the agent to automatically select a better computing power node to complete the simulation task. Using the trained neural network for task scheduling can improve the utilization efficiency of computing power and reduce simulation time.

[0070] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A computing power scheduling method applied to a large-scale network simulation system, characterized in that, Comprise: S1: obtain the resource occupation state information set of all computing power nodes as the original set, normalize the original set to obtain the standard sample set; S2: build a task queue of all computing power nodes, and preprocess the task queue to provide reasonable input data for subsequent scheduling decisions; Build a task queue of all computing power nodes, and preprocess the task queue, including: Build a task queue set G(T, L, ω), and represent the task queue; First, express the task usage set T of the computing power node, T={T1, T1,..., TN}, where Ti represents the ith task, and the computing resource consumption of task Ti is denoted as Li; Use a two-dimensional matrix Ei to represent an N*N matrix, where E(i,j)=1 indicates that there is a dependency constraint between Ti and Tj, and task Tj can only start execution after Ti is completed, and if E(i,j)=0, it indicates that there is no dependency relationship between the two; S3: set the deep reinforcement learning parameters according to the standard sample set of computing power nodes and initialize the weights of the neural network; S4: interact with the environment as an agent to generate experience samples, train the neural network using deep reinforcement learning method according to the experience samples, and obtain a preliminary task scheduling strategy; Interact with the environment as an agent to generate experience samples, train the neural network using deep reinforcement learning method according to the experience samples, and obtain a preliminary task scheduling strategy, including: Input the initial state of the computing power node into the deep reinforcement learning model based on neural network, and the deep reinforcement learning model randomly selects an action a according to the current strategy in each iteration; Execute action a, observe the next state S' and reward R; Store state S, action a, reward R, and next state S' into the experience replay pool; Randomly draw a batch of samples from the experience replay pool to update the parameters of the policy and value networks; Calculate the target Q value, where Q represents a long-term report of an action, starting from the current state S, taking a specific a under the policy to predict the value; Update the value network, calculate the loss function of the value network, use the mean square error loss function to minimize the difference between the target Q value and the a value, and update the parameters of the value network, where the loss function is: wherein Loss denotes the loss function of the critic network, B denotes the size of the experience replay pool, Rt denotes the reward obtained at time step t, γ denotes a decay factor, the size of which is between [0, 1]; Q'(St+1, a'| θ μ′ ) denotes the value of Q' at the next state St+1 and the target action a', the target action a' being generated by the target policy network θ μ′ , i.e. the optimal action of the next state St+1 is calculated by the μ' network, Q' denotes the target function of the critic network; Q(St, at| θ Q′ ) denotes the Q value of the current critic network Q at the current state St and the action at, θ Q′ is the weight of the Q' network; Update the policy network, calculate the loss function of the policy network, use the policy gradient to maximize the Q value of the current state, and update the parameters of the policy network, where the policy function is: where, denotes the critic network, θ μ denotes the actor network weights, θ Q denotes the critic network weights, denotes the gradient of the actor network, denotes the derivative of the Q function with respect to the current state stand the action a generated by the policy network a = μ(s t |θ μ ); denotes the gradient of the actor network with respect to its parameters θ μ , i.e. the change in the output action of the policy network with respect to the state s t ; In the DDPG algorithm, the policy network and the value network are based on deep neural networks, and the gradient descent method is used for parameter update, where the update functions of the policy network and the value network are: θ Q′ ←τθ Q +(1-τ)θ Q′ , θ μ′ ←τθ μ +(1-τ)θ μ′ where θ Q′ represents the critic network's objective function; θ μ′ represents the actor network's objective function; τ represents the soft update coefficient for target network updates; θ Q represents the critic network's weights, θ μ are the actor network's weights; Circulate the above steps until the optimal θ(s, a | θ Q ) value is reached, and output the corresponding scheduling strategy as a preliminary scheduling strategy. S5: use the optimized ant colony algorithm to further optimize the preliminary task scheduling to obtain the final task scheduling strategy; Use the optimized ant colony algorithm to further optimize the preliminary task scheduling to obtain the final task scheduling strategy, including: Step 1. Obtain the computing power node state information; Step 2. Calculate the revenue matrix; Step 3. Set the initial pheromone for the computing power node; Step 4. Initialize the ant colony and randomly distribute it in the computing power node; Step 5. Calculate the transition probability of the ant k, and select the next hop node; Step 6. Update the local pheromone after the ant completes the current search; Step 7. Update the global pheromone after all ants complete the search for this round; Step 8. Output the scheduling strategy when the number of iterations is reached; S6: Adopt the final task scheduling strategy to schedule the simulation tasks.

2. The computing power scheduling method applied to a large-scale network simulation system according to claim 1, characterized in that, Obtain the resource occupation state information set of all computing power nodes as the original set, normalize the original set to obtain the standard sample set, including: The resource occupation state information includes: CPU, memory, network bandwidth of the computing power node, and link delay between nodes; According to the collected resource occupation data of the computing power nodes, normalize the data to scale all data to the range of [0, 1] to form a standard sample set.

3. The computing power scheduling method for a large-scale network simulation system according to claim 1, wherein, According to the standard sample set of the computing power nodes, set the deep reinforcement learning parameters, including: According to the Markov decision process, set the state space, action space and reward function.

4. The computing power scheduling method for a large-scale network simulation system according to claim 1, wherein, Initialize the neural network, including: Initialize the parameters of the target policy network and the target value network, as well as the local policy network and the local value network using the DDPG algorithm; initialize the experience replay buffer, set the batch size, learning rate, reward discount factor, and soft update parameter.

5. A large-scale network simulation system for implementing the computing power scheduling method applied to the large-scale network simulation system according to any one of claims 1-4, characterized in that, Including: Simulation nodes, computing power nodes, business centers, monitoring centers, and training modules; The simulation node is a virtualization node in the network simulation system used to simulate real devices, and the main implementation is through a docker container or a lightweight container based on Linux namespace, mainly used as a carrier for executing simulation tasks; The computing power node is mainly a simulation server for building simulation nodes and a computing service for providing computing assistance, using real CPU and memory resources, because the simulation node still uses the physical resources of real devices when executing simulation tasks, in order to better realize computing power scheduling, it is abstracted as a computing power node, i.e. a node that uses computing power resources to complete simulation tasks; The business center is mainly a module for the network simulation system to issue simulation tasks to the simulation node, mainly responsible for interacting with the outside world to realize the simulation task requirements of the outside world; The monitoring center is mainly a module used by the network simulation system to monitor the resource consumption and task load of the computing power node, and collects parameters for using the computing power scheduling model to schedule computing power; The training module is used to set the deep reinforcement learning parameters according to the state information of the computing power nodes and train the neural network according to the experience samples generated by the interaction between the computing power nodes and the environment; The computing power node determines the task scheduling target according to the trained neural network, and realizes the transfer of computing power of the computing power node according to the task scheduling to realize the maximum utilization of resources.

Images