Resource scheduling method based on spiking neural network

Abstract

The invention discloses a resource scheduling method based on a spiking neural network. The method comprises the following steps: acquiring a resource condition of a current environment and a task resource acquisition condition; judging according to a current scheduling scene and a scheduling scale, and setting values of parameters; recording an input signal received by each sub-neuron and an activation state of the neuron, namely whether a pulse is emitted or not; maintaining a replay buffer to store samples; selecting a small batch of samples from the replay buffer to calculate gradient update; and a task scheduling result is generated in an output layer, tasks are issued through a scheduling module, subsequent monitoring is maintained, communication with the scheduling module is established, and information samples in the tasks are continuously collected. According to the method, the spiking neural network is put forward to be used in the field of resource scheduling for the first time, the advantages of deep reinforcement learning in the field of resource scheduling are fused, the advantages of energy conservation and discretization calculation of the spiking neural network are fused by fully utilizing the characteristics of the spiking neural network and event driving, and energy-saving and efficient scheduling is achieved.

Description

Technical Field

[0001] The present invention relates to the field of resource scheduling, and in particular to a resource scheduling method based on a pulse neural network. Background Art

[0002] Cloud computing resource scheduling refers to the efficient allocation and optimization of computing, storage, and network resources in a cloud environment to meet user needs and maximize resource utilization. The goals of cloud computing resource scheduling include improving scheduling speed, improving resource utilization, reducing energy consumption, reducing costs, and ensuring service quality. Cloud computing resource scheduling algorithms include static scheduling, dynamic scheduling, and multi-objective scheduling. Static scheduling allocates resources before the task starts, while dynamic scheduling adjusts resource allocation according to demand changes while the task is running. Multi-objective scheduling considers multiple goals at the same time, such as response time, cost, and resource utilization.

[0003] DRL (Deep Reinforcement Learning) is gradually maturing in the field of cloud computing resource scheduling. It provides an effective solution for improving the utilization efficiency of cloud computing resources and system performance by learning optimized resource allocation and task scheduling strategies. With the deepening of research and the development of technology, the application of DRL in the field of resource scheduling will become more extensive and in-depth.

[0004] In order to improve the scheduling effect in the discrete resource scheduling scenario (with the goal of reducing the average running time of each scheduled task as much as possible), reduce the energy consumption used for task scheduling itself, reduce the load of the entire system when performing calculations in a centralized architecture, and improve the scheduling effect, the introduction of spiking neural network (SNN, Spiking Neural Net) can not only improve the intelligence level of scheduling, but also significantly reduce the energy consumption of the system, especially in cloud computing environments that need to process a large number of dynamic and real-time tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a resource scheduling method based on a pulse neural network in view of the deficiencies of the prior art.

[0006] To achieve the above object, the present invention provides a resource scheduling method based on a pulse neural network, comprising the following steps:

[0007] (1) Modeling cluster physical resources as quantifiable attributes, and defining their capacity, performance indicators and availability as feature inputs; defining state variables to represent the current usage of resources, including remaining quantity, current load and task allocation; determining the input layer resource quantity coefficient and scheduling scale according to the current cluster resource quantity; constructing an environment information module to describe the real-time system status and available resource information; constructing a task information module to save the information of the current task, including quantity, resource requirements, physical resources of the working nodes obtained by the tasks with allocated resources, and monitoring the survival of the tasks; the environment information module and the task information module generate multiple feature matrices as inputs of the resource scheduling method;

[0008] (2) Design a resource scheduling strategy module by combining a deep reinforcement learning network model with a pulse neural network; the network architecture of the resource scheduling strategy module includes a feature input layer, a feedforward network layer, a fully connected layer, a LIF activation function layer, an output layer, and a reward function layer; the feature input layer is used to receive the resource usage information of the environment information module and the input signal of the task information module, and is connected to the fully connected layer for activation through normalization processing; feature extraction, combination, and loss calculation are performed in the fully connected layer to generate potential information to act on subsequent neurons; the LIF activation function is used to record the activation state of each neuron; a reward function is designed, including reward functions for indicators such as average task completion time, energy consumption used by the scheduling scheme, resource utilization, and load balancing, and weights are set according to the scheduling requirements of different environments; the output layer generates scheduling decisions based on the neural network, uses the alternative gradient method to update the parameters of the resource scheduling strategy module, and stores samples in the replay buffer to calculate gradient updates; the trained resource scheduling strategy module is deployed to the resource scheduling system for real-time decision-making and resource management;

[0009] (3) Construct and distribute tasks based on the results of the scheduling strategy module.

[0010] Furthermore, in step (1), the input of the resource scheduling method includes a task type matrix, a timing information matrix of task execution, an allocated resource type matrix, and the tasks to which the resources are allocated.

[0011] Furthermore, the task type matrix represents the offline data task type used to train the model, for a specific type of task to be scheduled;

[0012] The timing information matrix of the task when it is running represents the time when the task is running in the current cluster;

[0013] The allocated resource type matrix represents the proportion of key resources allocated in the machine;

[0014] The task to which resources are allocated is used to record the quantity of resources allocated to the task.

[0015] Furthermore, in step (2), the LIF activation function is used to record the input signal received by each sub-neuron and the activation state of the neuron, that is, whether a pulse is emitted, in each time step; the membrane potential V(t) of the neuron accumulates the input signal over time, and the input signal is the postsynaptic potential caused by the pulse emitted by the presynaptic neuron; the dynamics of the membrane potential is described by the following differential equation:

[0016]

[0017] Among them, τ m is the membrane time constant, which determines the rate at which the membrane potential decays; V(t) is the membrane potential at time t; V rest is the resting membrane potential of the neuron; R m is the membrane resistance; I(t) is the input current.

[0018] Furthermore, in step (2), the reward function includes a reward function of indicators - average task completion time, energy consumption used by the scheduling scheme, resource utilization and load balancing degree; wherein:

[0019] The indicator - the reward function R of the average task completion time completion It is expressed as:

[0020]

[0021] In the formula, N represents the number of tasks, j represents the task number, and L j represents the load of task number j, T j Represents the task execution time;

[0022] The reward function R for the energy consumption of the scheduling scheme energy It is expressed as:

[0023] R energy =-E

[0024] In the formula, E represents the scheduling energy consumption;

[0025] The reward function R of the resource utilization util It is expressed as:

[0026]

[0027] In the formula, R cur represents the current resource usage, and R target represents the target utilization rate;

[0028] The reward function R of the load balancing degree balanceIt is expressed as:

[0029]

[0030] Where, L ij Represents the load level of resource i.

[0031] The beneficial effects of the present invention are:

[0032] (1) It fully exploits the discretization characteristics of SNN and its sparse activation characteristics to cope well with discrete events in cloud computing scenarios;

[0033] (2) Set different weighted reward functions according to different scenarios to improve adaptability;

[0034] (3) Combining the advantages of deep reinforcement learning in the field of resource scheduling, the effect of improving resource scheduling in cloud computing scenarios is achieved;

[0035] (4) At the same time, due to the integration of the energy-saving characteristics of SNN, it can reduce the energy consumption of the resource scheduling center for model training and reduce performance load. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the pulse neuron excitation process in the present invention;

[0037] Figure 2 It is a schematic diagram of the pulse deep reinforcement learning network in the present invention;

[0038] Figure 3 It is a schematic diagram of the scheduling platform architecture based on SNN in the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0040] The principle of the resource scheduling method based on pulse neural network can be summarized as the following steps:

[0041] (1) Feature input initialization: Obtain the resource status of the current environment and the resource status of the task through the environment information module;

[0042] (2) Define model parameters based on the environment: Based on the current scheduling scenario, mainly the current business scenario and scheduling scale are judged, and reasonable values ​​are set for relevant parameters;

[0043] (3) Input and neuron activity monitoring: At each time step, record the input signal received by each sub-neuron and the activation state of the neuron, that is, whether it emits a pulse;

[0044] (4) Maintain a replay buffer to store samples collected over a large time span. At the end of each time period, a small batch of samples is selected from the replay buffer to calculate the gradient update. The loss function is continuously reduced to fit the complex situation of the production environment;

[0045] (5) Generate scheduling results: Generate task scheduling results at the output layer, issue tasks through the scheduling module, maintain subsequent monitoring, establish communication with it, and continuously collect information samples.

[0046] See also Figure 3 The present invention provides a resource scheduling method based on a pulse neural network, comprising the following steps:

[0047] (1) Environment and task information module: The current cluster physical resources, including CPU, memory, disk storage, network bandwidth, and GPU, are modeled as quantifiable attributes, and the capacity, performance indicators, and availability of the cluster resources are defined as feature inputs. State variables are defined to represent the current usage of resources, including the remaining amount of resources, current load, task allocation, etc. Based on the current number of cluster resources, the input layer resource quantity coefficient k and the current cluster scheduling scale O(S num ). See Figure 3 ,Schematic diagram of the scheduling platform architecture based on SNN,First, an environment information module that can reflect the resource scheduling environment is built based on the need, which can describe the real-time system status, available resources and other information. The environment information module establishes communication with the information collection device on each working node by providing a registration center and a connection management center to save environmental characteristics. Secondly, a task information module is built to save various information of the current task, including the quantity, the demand for resources, the physical resources of the working node obtained by the task that has been allocated resources, and at the same time monitors the survival of tasks on each node in the environment. These modules will finally generate multiple feature matrices used as inputs to the resource scheduling algorithm, including the task type matrix, the timing information matrix when the task is running, the type matrix of the allocated resources, and the task to which the resources are allocated.

[0048] a. For the task type matrix, it represents the offline data task type used to train this model, which is used for specific types of tasks to be scheduled;

[0049] b. The timing information matrix of the task during runtime represents the time the task is running in the current cluster;

[0050] c. For the allocated resource type matrix, it represents the proportion of key resources that have been allocated in the machine;

[0051] d. For tasks that are assigned resources, record the number of resources assigned to the task.

[0052] (2) Resource scheduling strategy module design: Utilize existing deep reinforcement learning network models for resource scheduling, such as DeepRm, combined with the spiking neural network (SNN), and modify its activation function and reward function and other related information to change its network architecture, and design a spiking deep reinforcement learning network (i.e., resource scheduling strategy module). SNN can respond quickly to input changes, which is crucial for resource scheduling scenarios that require real-time decision-making. Design the topology of SNN, including the hierarchy of neurons, the connection pattern of neurons, and the synaptic model.

[0053] See also Figure 2 , Schematic diagram of the pulse deep reinforcement learning network. The entire network architecture is divided into: feature input layer, feedforward network layer, fully connected layer, LIF activation function layer, output layer and reward function layer.

[0054] Feature input: Receives resource usage information from the environment information module, as well as task information and resource requirements from the task information module. The input state matrix S is connected to the fully connected layer for activation through normalization. The number of spiking neurons in this layer is proportional to the size of the state matrix:

[0055] N=k×O(S num )

[0056] Where N represents the number of tasks, k is the proportional constant, O(S num ) is the order of magnitude of the input matrix. For the actual deployment environment, k is related to the computer indicators to be considered in the environment. For AI training platforms, it will involve more GPU information, and for big data platforms, it mainly considers CPU, memory, disk and other information. num ), which is related to the number of tasks to be deployed.

[0057] Fully connected layer: extracts, combines, and calculates the loss of feature input information. This method is based on SNN implementation and generates potential information to act on subsequent neurons.

[0058] LIF activation function: see Figure 1 , at each time step, record the input signal received by each sub-neuron and the activation state of the neuron, that is, whether it emits a pulse; the membrane potential V(t) of the neuron accumulates the input signal over time. The input signal is usually the postsynaptic potential caused by the pulse emitted by the presynaptic neuron. The dynamics of the membrane potential can be described by the following differential equation:

[0059]

[0060] Among them, τ mis the membrane time constant, which determines the rate at which the membrane potential decays; V(t) is the membrane potential at time t; V rest is the resting membrane potential of the neuron; R m is the membrane resistance; I(t) is the input current.

[0061] It should be noted that the pulse is emitted when the membrane potential V(t) reaches the preset threshold voltage V thresh , the neuron fires a pulse and the membrane potential is reset to a lower value, V reset (usually close to the resting potential V rest ). The membrane potential cannot increase for a brief period of time, this period of time is called the refractory period.

[0062] Reset phase: After the neuron fires a spike, the membrane potential is reset and the integration process starts again.

[0063] Leakage: During the integration phase, the membrane potential will gradually decay due to leakage, simulating the natural decay of the membrane potential of biological neurons over time.

[0064] Reward function:

[0065] R total =w1R completion +w2R util +w3R energy +w4R balance

[0066] Where R completion , R energy , R util , R balance Represents various indicators - average task completion time, energy consumption of scheduling scheme, resource utilization, and reward function of load balancing. total It is a combination of all reward functions based on factors such as task completion time, resource utilization, energy consumption, load balancing, etc.; w1, w2, w3, and w4 represent the rewards for R completion , R energy , R util , R balance The weights of each reward function are used to meet the scheduling requirements of different environments.

[0067] The following is the specific meaning of each reward function

[0068]

[0069] For the time part, the reward function R completion , N represents the number of tasks, j represents the task number, L j represents the load of task number j, T j Represents the task execution time.

[0070] R=-E

[0071] Energy

[0072] For the energy consumption part, the reward function R energy , E represents the scheduling energy consumption,

[0073]

[0074] For the usage part, the reward function R util , R cur represents the current resource usage, and R target Represents the target utilization rate, usually 0.5.

[0075]

[0076] For the reward function R about the load balancing part balance , L ij It represents the load level of resource i, and defines its average to obtain its standard deviation, which represents the balance level of the overall load.

[0077] Output layer: Generates scheduling decisions, such as task allocation plans, based on the neural network.

[0078] At the same time, the parameters of the pulse deep reinforcement learning network are updated through the surrogate gradient method (SGM), and a replay buffer is maintained to store samples collected from a large time span. At the end of each time period, a small batch of samples is selected from the replay buffer to calculate the gradient update. The loss function is continuously reduced to fit the complex situation of the production environment.

[0079] After designing the network architecture, the initial ICSD_Qos training set is used to train the pulse deep reinforcement learning network. The performance of the SNN model is tested in a simulated environment, and its performance in resource scheduling tasks is evaluated and compared with existing methods such as ant colony algorithm, genetic algorithm and other heuristic algorithms. The trained SNN model is deployed to the actual resource scheduling system for real-time decision-making and resource management. By recording the running time of all tasks and the resource load, an evaluation is made on the actual scheduling model, so as to achieve continuous optimization to improve the overall performance. The pre-trained model is deployed in the actual scenario.

[0080] (3) Scheduling behavior module: responsible for constructing and issuing tasks based on the results of the scheduling strategy module.

[0081] After the model of the present invention is deployed, new tracking data is continuously collected to ensure that the new data represents the current operating environment and user behavior of the model. The goal of the present invention is to optimize the scheduling energy consumption, average time consumption, load balancing and other indicators. The judgment of the quality of the results is achieved by setting a reward function, and the priority of related indicators can be set by related weight parameters.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0083] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A resource scheduling method based on pulse neural network, characterized in that: The following steps are involved: (1) Model the cluster physical resources as quantifiable attributes and define their capacity, performance indicators, and availability as feature inputs; Define state variables to represent the current usage of resources, including remaining amount, current load and task allocation; determine the input layer resource quantity coefficient and scheduling scale according to the current cluster resource quantity; construct an environment information module to describe the real-time system status and available resource information; construct a task information module to save the information of the current task, including quantity, resource requirements, physical resources of the working node obtained by the task with allocated resources, and monitor the survival of the task; the environment information module and the task information module generate multiple feature matrices as inputs of the resource scheduling method; (2) Design a resource scheduling strategy module by combining a deep reinforcement learning network model with a pulse neural network; the network architecture of the resource scheduling strategy module includes a feature input layer, a feedforward network layer, a fully connected layer, a LIF activation function layer, an output layer, and a reward function layer; the feature input layer is used to receive the resource usage information of the environment information module and the input signal of the task information module, and is connected to the fully connected layer for activation through normalization processing; feature extraction, combination, and loss calculation are performed in the fully connected layer to generate potential information to act on subsequent neurons; the LIF activation function is used to record the activation state of each neuron; a reward function is designed, including reward functions for indicators such as average task completion time, energy consumption used by the scheduling scheme, resource utilization, and load balancing, and weights are set according to the scheduling requirements of different environments; the output layer generates scheduling decisions based on the neural network, uses the alternative gradient method to update the parameters of the resource scheduling strategy module, and stores samples in the replay buffer to calculate gradient updates; the trained resource scheduling strategy module is deployed to the resource scheduling system for real-time decision-making and resource management; (3) Construct and distribute tasks based on the results of the scheduling strategy module.

2. The resource scheduling method based on pulse neural network according to claim 1 is characterized in that: In the step (1), the input of the resource scheduling method includes a task type matrix, a timing information matrix of task runtime, an allocated resource type matrix and the tasks to which resources are allocated.

3. The resource scheduling method based on pulse neural network according to claim 2 is characterized in that: The task type matrix represents the offline data task type used to train the model, targeting specific types of tasks to be scheduled; The timing information matrix of the task when it is running represents the time when the task is running in the current cluster; The allocated resource type matrix represents the proportion of key resources allocated in the machine; The task to which resources are allocated is used to record the quantity of resources allocated to the task.

4. The resource scheduling method based on pulse neural network according to claim 1 is characterized in that: In step (2), the LIF activation function is used to record the input signal received by each sub-neuron and the activation state of the neuron, that is, whether a pulse is emitted, in each time step; the membrane potential V(t) of the neuron accumulates the input signal over time, and the input signal is the postsynaptic potential caused by the pulse emitted by the presynaptic neuron; the dynamics of the membrane potential is described by the following differential equation: Among them, τ m is the membrane time constant, which determines the rate at which the membrane potential decays; V(t) is the membrane potential at time t; V rest is the resting membrane potential of the neuron; R m is the membrane resistance; I(t) is the input current.

5. The resource scheduling method based on pulse neural network according to claim 1 is characterized in that: In step (2), the reward function includes the reward function of the indicators - average task completion time, energy consumption used by the scheduling scheme, resource utilization and load balancing degree; wherein: The indicator - the reward function R of the average task completion time completion It is expressed as: In the formula, N represents the number of tasks, j represents the task number, and L j represents the load of task number j, T j Represents the task execution time; The reward function R for the energy consumption of the scheduling scheme energy It is expressed as: R=-E Energy Efficiency In the formula, E represents the scheduling energy consumption; The reward function R of the resource utilization util It is expressed as: In the formula, R cur represents the current resource usage, and R target represents the target utilization rate; The reward function R of the load balancing degree balance It is expressed as: Where, L ij Represents the load level of resource i.

Images