A resource scheduling method based on spiking neural network

Abstract

The present invention discloses a resource scheduling method based on a pulse neural network, comprising: obtaining the resource status of the current environment and the resource acquisition status of the task; making judgments based on the current scheduling scenario and scheduling scale, and setting parameters; recording the input signals received by each sub-neuron and the activation state of the neuron, that is, whether a pulse is emitted; maintaining a replay buffer to store samples; selecting a small batch of samples from the replay buffer to calculate gradient updates; generating task scheduling results at the output layer, issuing tasks through the scheduling module, and maintaining subsequent monitoring, while establishing communication with it and continuously collecting information samples therein. The present invention proposes for the first time the use of pulse neural networks in the field of resource scheduling, integrating the advantages of deep reinforcement learning in the field of resource scheduling, and integrating the energy-saving and discrete computing advantages of pulse neural networks by fully utilizing the characteristics of pulse neural networks and event-driven methods, thereby achieving energy-saving and efficient scheduling.

Description

Technical Field

[0001] The present invention relates to the field of resource scheduling, and in particular to a resource scheduling method based on a pulse neural network. Background Art

[0002] Cloud computing resource scheduling refers to the efficient allocation and optimization of computing, storage, and network resources in a cloud environment to meet user needs and maximize resource utilization. The goals of cloud computing resource scheduling include improving scheduling speed, increasing resource utilization, reducing energy consumption, lowering costs, and ensuring quality of service. Cloud computing resource scheduling algorithms include static scheduling, dynamic scheduling, and multi-objective scheduling. Static scheduling allocates resources before a task begins, while dynamic scheduling adjusts resource allocation during task execution based on changing demand. Multi-objective scheduling simultaneously considers multiple objectives, such as response time, cost, and resource utilization.

[0003] Deep reinforcement learning (DRL) is gradually maturing in the field of cloud computing resource scheduling. By learning optimized resource allocation and task scheduling strategies, it provides an effective solution for improving cloud computing resource utilization efficiency and system performance. With the deepening of research and technological development, the application of DRL in resource scheduling will become more extensive and in-depth.

[0004] In order to improve its scheduling effect in the discrete resource scheduling scenario (with the goal of reducing the average running time of each scheduled task as much as possible), reduce the energy consumption used for task scheduling itself, reduce the load of the entire system when performing calculations in a centralized architecture, and improve the scheduling effect, the introduction of spiking neural network (SNN, Spiking Neural Net) can not only improve the intelligence level of scheduling, but also significantly reduce the energy consumption of the system, especially in cloud computing environments that need to process a large number of dynamic and real-time tasks. Summary of the Invention

[0005] The purpose of the present invention is to provide a resource scheduling method based on a pulse neural network to address the deficiencies of the existing technology.

[0006] To achieve the above object, the present invention provides a resource scheduling method based on a spiking neural network, comprising the following steps:

[0007] (1) Modeling cluster physical resources as quantifiable attributes, and defining their capacity, performance indicators and availability as feature inputs; defining state variables to represent the current usage of resources, including remaining amount, current load and task allocation; determining the input layer resource quantity coefficient and scheduling scale according to the current number of cluster resources; constructing an environment information module to describe the real-time system status and available resource information; constructing a task information module to save the information of the current task, including the quantity, resource requirements, the physical resources of the working nodes obtained by the tasks with allocated resources, and monitoring the survival of the tasks; the environment information module and the task information module generate multiple feature matrices as inputs of the resource scheduling method;

[0008] (2) Design a resource scheduling strategy module by combining a deep reinforcement learning network model with a pulse neural network; the network architecture of the resource scheduling strategy module includes a feature input layer, a feedforward network layer, a fully connected layer, a LIF activation function layer, an output layer, and a reward function layer; the feature input layer is used to receive resource usage information from the environment information module and input signals from the task information module, and is connected to the fully connected layer for activation through normalization processing; feature extraction, combination, and loss calculation are performed in the fully connected layer to generate potential information to act on subsequent neurons; the LIF activation function is used to record the activation state of each neuron; a reward function is designed, including reward functions for indicators such as average task completion time, energy consumption used in the scheduling scheme, resource utilization, and load balancing, and weights are set according to the scheduling requirements of different environments; the output layer generates scheduling decisions based on the neural network, uses the alternative gradient method to update the parameters of the resource scheduling strategy module, and stores samples in the replay buffer to calculate gradient updates; the trained resource scheduling strategy module is deployed to the resource scheduling system for real-time decision-making and resource management;

[0009] (3) Construct and distribute tasks based on the results of the scheduling strategy module.

[0010] Furthermore, in step (1), the input of the resource scheduling method includes a task type matrix, a timing information matrix of task runtime, an allocated resource type matrix, and tasks to which resources are allocated.

[0011] Furthermore, the task type matrix represents the offline data task type used to train the model, targeting specific types of tasks to be scheduled;

[0012] The timing information matrix of the task during execution represents the time the task is executed in the current cluster;

[0013] The allocated resource type matrix represents the proportion of key resources allocated to the machine;

[0014] The task to which resources are allocated is used to record the amount of resources allocated to the task.

[0015] Furthermore, in step (2), the LIF activation function is used to record the input signal received by each sub-neuron and the activation state of the neuron, that is, whether a pulse is emitted, at each time step; the membrane potential V(t) of the neuron accumulates the input signal over time, and the input signal is the postsynaptic potential caused by the pulse emitted by the presynaptic neuron; the dynamics of the membrane potential is described by the following differential equation:

[0016]

[0017] Among them, τ m is the membrane time constant, which determines the rate of membrane potential decay; V(t) is the membrane potential at time t; V rest is the resting membrane potential of the neuron; R m is the membrane resistance; I(t) is the input current.

[0018] Furthermore, in step (2), the reward function includes a reward function of indicators - average task completion time, energy consumption used by the scheduling scheme, resource utilization and load balancing degree; wherein:

[0019] The indicator - the reward function R of the average task completion time completion Expressed as:

[0020]

[0021] Where N represents the number of tasks, j represents the task number, and L j represents the load of task number j, T j Represents the task execution time;

[0022] The reward function R for the energy consumption of the scheduling scheme energy Expressed as:

[0023] R energy =-E

[0024] Where, E represents the scheduling energy consumption;

[0025] The reward function R of the resource utilization util Expressed as:

[0026]

[0027] Where R cur represents the current resource usage, and R target represents target utilization rate;

[0028] The reward function R of the load balancing degree balanceExpressed as:

[0029]

[0030] Where, L ij Represents the load level of resource i.

[0031] The beneficial effects of the present invention are:

[0032] (1) It fully exploits the discretization characteristics of SNN and its sparse activation characteristics to effectively deal with discrete events in cloud computing scenarios;

[0033] (2) Setting different weighted reward functions according to different scenarios to improve adaptability;

[0034] (3) Combining the advantages of deep reinforcement learning in the field of resource scheduling, the effect of improving resource scheduling in cloud computing scenarios is achieved;

[0035] (4) At the same time, due to the integration of the energy-saving characteristics of SNN, it can reduce the energy consumption of the resource scheduling center for model training and reduce performance load. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the pulse neuron excitation process in the present invention;

[0037] Figure 2 Schematic diagram of the pulse deep reinforcement learning network in the present invention;

[0038] Figure 3 It is a schematic diagram of the SNN-based scheduling platform architecture in the present invention. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below with reference to the accompanying drawings. Unless there is any conflict, the features of the following embodiments and implementations may be combined with each other.

[0040] The principle of the resource scheduling method based on pulse neural network can be summarized as the following steps:

[0041] (1) Feature input initialization: Obtain the resource status of the current environment and the resource status of the task through the environment information module;

[0042] (2) Define model parameters based on the environment: Based on the current scheduling scenario, mainly judge the current business scenario and scheduling scale, and set reasonable values ​​for relevant parameters;

[0043] (3) Input and neuronal activity monitoring: At each time step, the input signal received by each sub-neuron and the activation state of the neuron, that is, whether it emits a pulse, are recorded;

[0044] (4) Maintain a replay buffer to store samples collected over a large time span. At the end of each time period, a small batch of samples is selected from the replay buffer to calculate the gradient update. The loss function is continuously reduced to fit the complex conditions of the production environment;

[0045] (5) Generate scheduling results: Generate task scheduling results at the output layer, issue tasks through the scheduling module, maintain subsequent monitoring, establish communication with it, and continuously collect information samples.

[0046] See also Figure 3 The present invention provides a resource scheduling method based on a pulse neural network, comprising the following steps:

[0047] (1) Environment and task information module: The current cluster physical resources, including CPU, memory, disk storage, network bandwidth, and GPU, are modeled as quantifiable attributes, and the capacity, performance indicators, and availability of the cluster resources are defined as feature inputs. State variables are defined to represent the current usage of resources, including the remaining amount of resources, current load, task allocation, etc. Based on the current number of cluster resources, the input layer resource quantity coefficient k and the current cluster scheduling scale O(S) are determined. num ). See Figure 3 ,Schematic diagram of the scheduling platform architecture based on SNN,First, an environmental information module that can reflect the resource scheduling environment is built based on the need, which can describe real-time system status, available resources and other information. The environmental information module establishes communication with the information collection device on each working node by providing a registration center and a connection management center to save environmental characteristics. Secondly, a task information module is built to save various information about the current task, including the quantity, resource requirements, and the physical resources of the working node obtained by the task that has been assigned resources. At the same time, it monitors the survival status of tasks on each node in the environment. These modules will eventually generate multiple feature matrices used as input to the resource scheduling algorithm, including the task type matrix, the timing information matrix of the task during runtime, the matrix of the type of resources that have been assigned, and the task to which the resources have been assigned.

[0048] a. The task type matrix represents the offline data task type used to train this model, which is used to target specific types of tasks to be scheduled.

[0049] b. The timing information matrix for the task during runtime represents the time the task is running in the current cluster;

[0050] c. For the allocated resource type matrix, it represents the proportion of key resources allocated to the machine;

[0051] d. For tasks that are assigned resources, record the number of resources assigned to the task.

[0052] (2) Resource Scheduling Strategy Module Design: Utilizing existing deep reinforcement learning network models for resource scheduling, such as DeepRm, combined with a spiking neural network (SNN), we modify its activation function, reward function, and other related information to change its network architecture and design a spiking deep reinforcement learning network (i.e., resource scheduling strategy module). SNNs are able to respond quickly to input changes, which is crucial for resource scheduling scenarios that require real-time decision-making. Design the topology of the SNN, including the neuron hierarchy, neuron connection patterns, and synaptic models.

[0053] See also Figure 2 , Schematic diagram of the pulse deep reinforcement learning network. The entire network architecture is divided into: feature input layer, feedforward network layer, fully connected layer, LIF activation function layer, output layer and reward function layer.

[0054] Feature Input: Receives resource usage information from the environment information module, as well as task information and resource requirements from the task information module. The input state matrix S is normalized and connected to the fully connected layer for activation. The number of spiking neurons in this layer is proportional to the size of the state matrix:

[0055] N=k×O(S num )

[0056] Where N represents the number of tasks, k is the proportional constant, O(S num ) is the order of magnitude of the input matrix. For the actual deployment environment, k is related to the computer indicators to be considered in the environment. For AI training platforms, it will involve more GPU information, and for big data platforms, it mainly considers CPU, memory, disk and other information. For O(S num ), which is related to the number of tasks to be deployed.

[0057] Fully connected layer: extracts, combines, and calculates the loss of feature input information. This method is based on SNN implementation and generates potential information to act on subsequent neurons.

[0058] LIF activation function: see Figure 1 At each time step, the input signal received by each daughter neuron and the neuron's activation state, that is, whether it fires a spike, are recorded; the neuron's membrane potential V(t) accumulates the input signal over time. The input signal is usually the postsynaptic potential caused by the spike fired by the presynaptic neuron. The dynamics of the membrane potential can be described by the following differential equation:

[0059]

[0060] Among them, τ mis the membrane time constant, which determines the rate of membrane potential decay; V(t) is the membrane potential at time t; V rest is the resting membrane potential of the neuron; R m is the membrane resistance; I(t) is the input current.

[0061] It should be noted that the pulse is emitted when the membrane potential V(t) reaches the preset threshold voltage V thresh When the neuron fires a pulse, the membrane potential is reset to a lower value V reset (usually close to the resting potential V rest ). The membrane potential cannot increase for a brief period of time, this period is called the refractory period.

[0062] Reset phase: After the neuron fires a spike, the membrane potential is reset and the integration process starts again.

[0063] Leakage: During the integration phase, the membrane potential gradually decays due to leakage, simulating the natural decay of the membrane potential of biological neurons over time.

[0064] Reward function:

[0065] R total =w1R completion +w2R util +w3R energy +w4R balance

[0066] where R completion 、R energy 、R util 、R balance Represents various indicators - average task completion time, energy consumption of scheduling scheme, resource utilization, and reward function of load balancing. total It is a combination of all reward functions based on factors such as task completion time, resource utilization, energy consumption, load balancing, etc.; w1, w2, w3, and w4 represent the rewards for R completion 、R energy 、R util 、R balance The weights of each reward function are used to meet the scheduling requirements of different environments.

[0067] The following is the specific meaning of each reward function

[0068]

[0069] For the time part of the reward function R completion , N represents the number of tasks, j represents the task number, L j represents the load of task number j, T j Represents the task execution time.

[0070] R=-E

[0071] energy

[0072] For the energy consumption part, the reward function R energy , E represents the scheduling energy consumption,

[0073]

[0074] For the reward function R of the utilization part util , R cur represents the current resource usage, and R target Represents target utilization, usually 0.5.

[0075]

[0076] For the reward function R about the load balancing part balance , L ij It represents the load level of resource i and defines its mean to obtain its standard deviation, which represents the balance level of the overall load.

[0077] Output layer: Generates scheduling decisions based on the neural network, such as task allocation plans.

[0078] Simultaneously, the surrogate gradient method (SGM) is used to update the parameters of the spiking deep reinforcement learning network. A replay buffer is maintained to store samples collected over a large time span. At the end of each time period, a mini-batch of samples is selected from the replay buffer to calculate the gradient update. This continuously reduces the loss function to adapt to the complexities of production environments.

[0079] After designing the network architecture, a spiking deep reinforcement learning network was trained using the initial ICSD_Qos training set. The SNN model's performance was tested in a simulated environment, evaluating its performance in resource scheduling tasks and comparing it with existing heuristics such as ant colony algorithms and genetic algorithms. The trained SNN model was deployed in a real-world resource scheduling system for real-time decision-making and resource management. The actual scheduling model was evaluated by recording the runtime and resource load of all tasks, enabling continuous optimization to improve overall performance. The pre-trained model was then deployed in real-world scenarios.

[0080] (3) Scheduling behavior module: responsible for constructing and issuing tasks based on the results of the scheduling strategy module.

[0081] After the model is deployed, new tracking data is continuously collected to ensure that the new data represents the model's current operating environment and user behavior. The goal of this invention is to optimize metrics such as scheduling energy consumption, average scheduling time, and load balancing. The quality of results is judged by setting a reward function, and the priority of related metrics can be set using relevant weight parameters.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0083] The above embodiments are intended only to illustrate the design concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. The scope of protection of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design concepts disclosed in the present invention are within the scope of protection of the present invention.

Claims

1. A resource scheduling method based on pulse neural network, characterized in that: The following steps are involved: (1) Modeling cluster physical resources as quantifiable attributes and defining their capacity, performance indicators, and availability as feature inputs; State variables are defined to represent the current usage of resources, including remaining quantity, current load, and task allocation. The input layer resource quantity coefficient and scheduling scale are determined based on the current number of cluster resources. An environment information module is constructed to describe the real-time system status and available resource information. A task information module is constructed to store information about the current task, including quantity, resource requirements, physical resources of the work nodes obtained by the tasks with allocated resources, and monitor the survival of the tasks. The environment information module and the task information module generate multiple feature matrices as input to the resource scheduling method. (2) Design a resource scheduling strategy module by combining a deep reinforcement learning network model with a pulse neural network; the network architecture of the resource scheduling strategy module includes a feature input layer, a feedforward network layer, a fully connected layer, a LIF activation function layer, an output layer, and a reward function layer; the feature input layer is used to receive resource usage information from the environment information module and input signals from the task information module, and is connected to the fully connected layer for activation through normalization processing; feature extraction, combination, and loss calculation are performed in the fully connected layer to generate potential information to act on subsequent neurons; the LIF activation function is used to record the activation state of each neuron; a reward function is designed, including reward functions for indicators such as average task completion time, energy consumption used in the scheduling scheme, resource utilization, and load balancing, and weights are set according to the scheduling requirements of different environments; the output layer generates scheduling decisions based on the neural network, uses the alternative gradient method to update the parameters of the resource scheduling strategy module, and stores samples in the replay buffer to calculate gradient updates; the trained resource scheduling strategy module is deployed to the resource scheduling system for real-time decision-making and resource management; (3) Construct and distribute tasks based on the results of the scheduling strategy module.

2. The resource scheduling method based on pulse neural network according to claim 1, characterized in that: In step (1), the input of the resource scheduling method includes a task type matrix, a timing information matrix of task runtime, a matrix of allocated resource types, and tasks to which resources are allocated.

3. The resource scheduling method based on pulse neural network according to claim 2, characterized in that: The task type matrix represents the offline data task types used to train this model, targeting specific types of tasks to be scheduled; The timing information matrix of the task during execution represents the time the task is executed in the current cluster; The allocated resource type matrix represents the proportion of key resources allocated in the machine; The task to which resources are allocated is used to record the amount of resources allocated to the task.

4. The resource scheduling method based on pulse neural network according to claim 1, characterized in that: In step (2), the LIF activation function is used to record the input signal received by each sub-neuron and the activation state of the neuron, that is, whether it emits a pulse, in each time step; the membrane potential V(t) of the neuron accumulates the input signal over time, and the input signal is the postsynaptic potential caused by the pulse emitted by the presynaptic neuron; the dynamics of the membrane potential is described by the following differential equation: Among them, τ m is the membrane time constant, which determines the rate of membrane potential decay; V(t) is the membrane potential at time t; V rest is the resting membrane potential of the neuron; R m is the membrane resistance; I(t) is the input current.

5. The resource scheduling method based on pulse neural network according to claim 1, characterized in that: In step (2), the reward function includes the reward function of the indicators - average task completion time, energy consumption used by the scheduling scheme, resource utilization and load balancing degree; wherein: The indicator - the reward function R of the average task completion time completion Expressed as: Where N represents the number of tasks, j represents the task number, and L j represents the load of task number j, T j Represents the task execution time; The reward function R for the energy consumption of the scheduling scheme energy Expressed as: R=-E energy Where, E represents the scheduling energy consumption; The reward function R of the resource utilization util Expressed as: Where R cur represents the current resource usage, and R target represents target utilization rate; The reward function R of the load balancing degree balance Expressed as: Where, L ij Represents the load level of resource i.

Images