Adaptive task scheduling method for batch processing task based on pulse reinforcement learning

Abstract

In order to solve the resource problem in the heterogeneous cloud computing environment, the application provides a batch processing task adaptive scheduling method based on pulse reinforcement learning. The technical path covers: multi-dimensional feature decoupling extraction is carried out on the input batch processing task, and clustering analysis is carried out; task classification is realized based on task characteristic attributes, and sorting is carried out according to priority; based on task analysis modeling, a pulse reinforcement learning resource scheduling model is established, and a double sparse attention mechanism is introduced to realize global state updating; the task scheduling environment of pulse reinforcement learning is constructed; the decision network and evaluation network of the pulse reinforcement learning model are constructed, and finally, in the simulation simulation scene, the trained model is used to realize the adaptive scheduling of batch processing tasks. The application fully considers the characteristic attributes of different tasks, overcomes the performance bottleneck of traditional scheduling algorithms in complex heterogeneous environment through accurate supply and demand matching strategy, and reduces the inference energy consumption during model training.

Description

Technical Field

[0001] This invention relates to the field of heterogeneous cloud computing environment technology, and in particular to an adaptive task scheduling method for batch processing tasks based on spurious reinforcement learning. Background Technology

[0002] In cloud computing environments, task scheduling is a core technology for improving computing power utilization efficiency, and its effectiveness directly impacts the task turnaround speed and energy consumption levels of data centers. However, in complex environments, different tasks have significantly different resource requirements in terms of CPU, memory, and runtime. Most current methods consider only one aspect and lack consideration for the characteristic attributes of tasks. Furthermore, traditional task scheduling methods suffer from low computational efficiency and slow response in complex dynamic environments. Machine learning-based task scheduling methods are currently a research hotspot, but the complexity of the algorithm models leads to high energy consumption for inference and training. In addition, current task scheduling methods lack a comprehensive consideration of cloud computing environment performance, typically using a single metric as the optimization objective. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an adaptive task scheduling method for batch processing tasks based on spiking reinforcement learning. First, feature analysis is performed on large batch processing tasks in task scheduling. A batch processing task clustering analysis method based on multi-dimensional decoupling is constructed using K-means clustering. Based on this, a task scheduling algorithm based on spiking reinforcement learning is proposed. By introducing a spiking neural network into the decision network, traditional intensive computation is transformed into sparse spiking accumulation to reduce inference energy consumption. Simultaneously, to accurately extract complex state information in cloud environments, a multimodal perception method is designed, and a reward-based dynamic threshold adjustment mechanism is proposed. This enables neurons to adaptively adjust the firing threshold based on scheduling feedback, providing a high-performance solution for intelligent scheduling in energy-constrained environments.

[0004] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0005] This invention discloses an adaptive task scheduling method for batch processing tasks based on spurious reinforcement learning. It is mainly implemented through two stages: batch task clustering analysis and spurious reinforcement learning model establishment. The overall method includes the following steps:

[0006] Step 1: Decouple batch processing tasks from multidimensional features, remove irrelevant features, and assign priorities.

[0007] Step 2: Perform cluster analysis on the CPU, memory and time characteristics of the tasks, package the tasks, and sort each task class. Tasks with high priority and low latency are placed at the front of the queue. The resulting task types serve as the basis for subsequent model design.

[0008] Step 3: Construct a spurious reinforcement learning multi-agent model. This step is based on the task packaging type in Step 2 and designs a multi-agent collaborative model, in which each agent is an independent Actor-Critic model.

[0009] Step 4: Construct the task scheduling environment for spurious reinforcement learning. This step is used to construct the task state and server resource state in the environment, the multimodal feature input of the batch task, the action space in the scheduling environment, the reward function after the action is completed, and the spurious firing threshold adjustment.

[0010] Step 5: Construct the decision network and evaluation network of the spiking reinforcement learning model. In this step, a spiking neural network is introduced into the decision network to realize the output of scheduling actions; the evaluation network uses a DNN network with a dual sparse attention mechanism to update the algorithm model with the global state until scheduling is completed.

[0011] Further, in step 1, the batch processing tasks input by the user are obtained, features are extracted based on the batch processing task information, and the priority of the tasks is marked; the resource requirement features of each task are obtained, the feature attributes are analyzed, irrelevant features are removed, and the retained feature information is... ,in:

[0012] vector This represents the resource requirement characteristics of the task, where, For the task The computing resource requirements For the task Storage resource requirements For the task Disk resource requirements,

[0013] use Indicates task Priority characteristics;

[0014] use This indicates the scheduling time characteristics of a task.

[0015] Furthermore, in step 2, the K-means algorithm is used to perform cluster analysis on the retained feature attributes. For the input batch processing task flow, cluster analysis is performed to obtain task clustering results with different features. The clustering results are then packaged according to the number of tasks, providing a foundation for subsequent model design for computationally intensive tasks, storage-intensive tasks, long-duration tasks, and balanced tasks.

[0016] To determine the effectiveness and rationality of the number of K-means clusters, the elbow rule and silhouette coefficient are introduced to determine the optimal number of clusters.

[0017] Furthermore, in step 3, based on the task packaging results, a spurious reinforcement learning multi-agent model is constructed. According to the task types obtained in step 2, four types of agents are designed: computationally intensive agents, storage-intensive agents, balanced agents, and long-duration agents. Each agent consists of an independent Actor-Critic network, wherein:

[0018] The decision network (Actor) learns local features and outputs the optimal scheduling action for the current type of agent;

[0019] The Critic network learns global features and updates the network. To reduce computational overhead, a dual sparse attention mechanism is introduced, employing both spatial proximity and task relevance as constraints.

[0020] In spatial proximity constraints, the Top-k selection strategy is used to retain only the K nearest neighbors, as shown in the formula: , here This represents a spatial proximity mask; 0 indicates that this information is retained. This indicates that it will be removed in Softmax;

[0021] In the task proximity constraint, cosine similarity is used to calculate the current query. With all keys The semantic relevance was determined, and the highest relevance was selected. Each node generates a task mask, using the following formula: , ,in, Represents a task-related mask, a vector This represents the retrieval demand vector of the current agent j, while This maps the attribute feature encoding of the neighboring agent j;

[0022] The two mask matrices are fused using a logical AND operation to obtain the final attention distribution. Numerically, this is equivalent to adding the masks: At this point, only agent j that simultaneously satisfies both the conditions of close spatial distance and matching task attributes can... Only if it is 0, otherwise it is 0. .

[0023] Furthermore, in step 4, based on the model established in step 3, the task scheduling environment is designed, mainly including the following steps:

[0024] Step 4-1: In the design of the state space, consider the state of the task and the state of the server;

[0025] Step 4-2: Based on the state space, consider the multimodal inputs of the decision network;

[0026] Step 4-3: Design the action space for each agent and the global action space;

[0027] Step 4-4: Design a differentiated reward function for each agent.

[0028] As a further design option,

[0029] Step 4-1: During each scheduling, the current conditions that meet the requirements are presented in the form of (machine, task) pairs, using... The concatenated representation represents the state space, where vectors This represents the current resource capacity of the machine, where Indicates machine CPU remaining capacity Indicates machine Remaining memory capacity Indicates machine The remaining disk space varies, and the state space also differs due to the different task characteristics of different agents.

[0030] Step 4-2: When designing multimodal inputs, three coding methods are included: frequency coding, delay coding, and group coding.

[0031] Frequency coding is used to represent the magnitude of values, which include two categories: machine resources and task resources. After normalizing the values, the larger the value, the more pulses per unit time, thus representing the importance of the task and the machine.

[0032] Delay coding is used to represent task priority, thus characterizing urgency. Task priority is used as the coding input; the higher the value, the earlier the transmission pulse.

[0033] To improve the robustness of the model, a small group of neurons is assigned to the task set of each agent class. Each neuron is sensitive to specific agent features, and a certain type of feature value will activate a portion of the neurons.

[0034] Step 4-3: In the model designed in step 3, there are four agents in total, using... Represents the joint action space, where, The system executes actions based on the separate scheduling decisions of the four types of intelligent agents. Determined by the priority judgment mechanism: .

[0035] Step 4-4: Design the reward function: Composite reward function The scheduling time efficiency reward is set as follows: if there are incomplete tasks, Otherwise, it is 0; the load balancing efficiency reward is set as follows: let the variance of cluster resource utilization at time t be... ,like ,but Next, the two rewards will be combined, resulting in a total reward of... The weights vary depending on the agent.

[0036] Furthermore, in step 5, a decision network and an evaluation network are constructed, and the update method during training iterations is as follows:

[0037] Each agent i maintains an independent policy network, denoted as . ,in The network parameters are given, and the input is the local observations of agent i. After feature extraction via a hidden layer with ReLU activation, the output layer uses the Softmax function to generate a value in the current action space. probability distribution on The gradient update formula is: ,in: It is the advantage function, which guides the direction of gradient updates; It is the entropy of the policy distribution, used to measure the randomness of the policy; It is the entropy coefficient, used to regulate the balance between exploration and utilization;

[0038] The strategy network is a spiking neural network. When designing the neuron threshold, the threshold of the spiking neuron in the l-th layer is set to be... Introducing the advantage function As a regulating factor, the advantage function is defined as follows: This reflects the quality of the current action relative to the average level. The threshold update formula is:

[0039]

[0040] in, The indicator function indicates that modulation is applied only to neurons that produce firing behavior, in the following way:

[0041] Evaluation Network Access to global state information during the training phase global state It contains the observation set of all agents. The Critic network evaluates the quality of the current joint state by fitting a state-value function, where the objective value is constructed using a temporal difference method. : The parameters of the Critic network are updated by minimizing the mean square error between the predicted value and the TD target value: , where B is the training batch size.

[0042] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0043] This invention provides an adaptive task scheduling method for batch processing tasks based on spiking reinforcement learning, comprising: decoupling multi-dimensional features of batch processing tasks, removing irrelevant features, and prioritizing them; clustering tasks based on CPU, memory, and time to obtain tasks of different feature types, packaging the tasks, and sorting each task class; constructing a spiking reinforcement learning multi-agent model and designing a multi-agent collaborative model; constructing a task scheduling environment for spiking reinforcement learning, including task states and server resource states in the environment, multimodal feature inputs of batch processing tasks, action space in the scheduling environment, reward function after action completion, and spiking threshold adjustment; constructing a decision network and an evaluation network for the spiking reinforcement learning model. In this step, a spiking neural network is introduced into the decision network to realize the output of scheduling actions; the evaluation network uses a DNN network with a dual sparse attention mechanism to update the algorithm model with global state until scheduling is completed. This invention fully analyzes the feature attributes of tasks and introduces a spiking neural network into the decision network, which can match scheduling nodes for different types of tasks while reducing the energy consumption of model inference and training. Attached Figure Description

[0044] To fully illustrate the technical concepts of the embodiments of the present invention or the prior art, the accompanying drawings will be briefly described below. It should be understood that the following drawings correspond only to some embodiments of the present invention, and those skilled in the art can still derive other technical illustrations based on these drawings without making any inventive contribution.

[0045] Figure 1 A flowchart of the method provided in an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of the impulse reinforcement learning algorithm model provided in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram illustrating multimodal input and fusion provided in an embodiment of the present invention;

[0048] Figure 4 The scheduling time difference of the present invention and single-modal impulse reinforcement learning is compared with that of deep reinforcement learning for different task lengths;

[0049] Figure 5 This paper compares the energy consumption differences generated by the present invention and different algorithms during each training session under different task lengths. Detailed Implementation

[0050] The technical solutions disclosed in this invention will be explained in detail with reference to the accompanying drawings. It should be understood that the embodiments listed below are for illustrative purposes only and do not represent all implementations of this invention. Within the scope of knowledge of those skilled in the art, any other solutions that can be derived by referring to these specific examples without substantial research and development innovation should be included within the scope of patent protection of this invention.

[0051] To further clarify the core technical aspects of this disclosure, this section will provide an explanation through specific implementation strategies and accompanying drawings.

[0052] Figure 1 The method flow architecture proposed in this invention is illustrated. For example... Figure 1 As shown, this study constructs an adaptive task scheduling method for batch processing tasks based on spurious reinforcement learning, which mainly includes:

[0053] Step 100: Obtain the batch processing task input by the user, decouple the multi-dimensional features of the task, remove irrelevant features, and mark the priority;

[0054] Step 200: Analyze and classify the CPU, memory and time of the tasks to obtain tasks with different feature types, package the tasks, sort them within each task class, and use the different task types as the basis for subsequent model design.

[0055] Step 300: Construct a spurious reinforcement learning multi-agent model and design a multi-agent collaborative model;

[0056] Step 400: Construct a task scheduling environment for spurious reinforcement learning, including the task state and server resource state in the environment, the multimodal feature input of batch tasks, the action space in the scheduling environment, the reward function after the action is completed, and the spurious firing threshold adjustment.

[0057] Step 500: Construct the decision network and evaluation network of the spurious reinforcement learning model, train the spurious reinforcement learning model in a simulated cloud computing environment, and use the trained model to implement batch processing task scheduling.

[0058] The specific implementation method of this method includes the following parts:

[0059] Step 1: Obtain the batch processing tasks input by the user; extract features based on the DAG information of the tasks; and complete the priority marking of the tasks. Obtain the resource requirement features of each task, analyze the feature attributes, remove irrelevant features, and retain the following feature information: ,in:

[0060] vector This represents the resource requirement characteristics of the task, where, For the task The computing resource requirements For the task Storage resource requirements For the task Disk resource requirements,

[0061] use Indicates task Priority characteristics;

[0062] use This indicates the scheduling time characteristics of a task.

[0063] Step 2: Use the K-means algorithm to perform cluster analysis on the retained feature attributes. For the input batch processing task flow, perform cluster analysis to obtain task clustering results with different features, and package them according to the number of tasks. The task type label is... These correspond to compute-intensive tasks, storage-intensive tasks, long-running tasks, and balanced tasks, respectively.

[0064] To determine the effectiveness and rationality of the number of K-means clusters, the elbow rule and silhouette coefficient are designed to determine the optimal number of clusters.

[0065] Step 3: According to Each task class is sorted, with tasks of high priority and low latency placed at the front of the queue. The resulting task types serve as the basis for subsequent model design.

[0066] Step 4: Based on the task packaging results, construct a spurious reinforcement learning multi-agent model. Design an agent set according to the task types obtained in Step 2. These correspond to computationally intensive task types (high CPU task agents), storage-intensive task types (high memory task agents), long-duration task types, and balanced task types, respectively. Each agent consists of an independent Actor-Critic network, where:

[0067] The decision network (Actor) learns local features and outputs the optimal scheduling action for the current type of agent;

[0068] The Critic network learns global features and updates the network. To reduce computational overhead, a dual sparse attention mechanism is introduced, employing both spatial proximity and task relevance as constraints.

[0069] First, the input features are preprocessed using a shared-state encoder. Assume the hidden features of the i-th agent are... Three learnable linear transformation matrices are introduced. This is divided into representing queries, keys, and values, and features are mapped to the latent semantic space of queries, keys, and values: ,in,

[0070] vector This represents the retrieval request vector of agent i at present, while This maps the attribute feature encoding of the neighboring agent j, and at the same time It carries the environmental state features to be extracted. Its core original dot product correlation degree is derived using the following formula: Next, dual constraints of spatial proximity and task relevance are applied:

[0071] In spatial proximity constraints, the Top-k selection strategy is used to retain only the K nearest neighbors, as shown in the formula: , here This represents a spatial proximity mask; 0 indicates that this information is retained. This indicates that it will be removed in Softmax;

[0072] In the task proximity constraint, cosine similarity is used to calculate the current query. With all keys The semantic relevance was determined, and the highest relevance was selected. Each node generates a task mask, using the following formula: , ,in, Represents a task-related mask, a vector This represents the retrieval demand vector of the current agent j, while This maps the attribute feature encoding of the neighboring agent j;

[0073] The two mask matrices are fused using a logical AND operation to obtain the final attention distribution. Numerically, this is equivalent to adding the masks: At this point, only agent j that simultaneously satisfies both the conditions of close spatial distance and matching task attributes can... Only if it is 0, otherwise it is 0. .

[0074] Step 5: Construct a task scheduling environment for spurious reinforcement learning;

[0075] Step 5-1: In the design of the state space, consider the characteristic states of the task and the state of the server;

[0076] During each scheduling, the current conditions that meet the requirements are presented in the form of (machine, task) pairs, using... The concatenated representation represents the state space, where vectors This represents the current resource capacity of the machine, where Indicates machine CPU remaining capacity Indicates machine Remaining memory capacity Indicates machine The remaining disk space, where the state space varies due to the different task characteristics of different agents, is shown in the table below:

[0077] intelligent agent state space Highly computationally intensive tasks Storage-intensive task Long-duration task Balanced task type

[0078] Step 5-2: Based on the state space, consider the multimodal inputs of the decision network;

[0079] Frequency coding is used to represent the magnitude of values, which include two categories: machine resources and task resources. After normalizing the values, the larger the value, the more pulses per unit time, thus representing the importance of the task and the machine.

[0080] Delay coding is used to represent task priority, thus characterizing urgency. Task priority is used as the coding input; the higher the value, the earlier the transmission pulse.

[0081] To improve the robustness of the model, a small group of neurons is assigned to the task set of each agent class. Each neuron is sensitive to specific agent features, and a certain type of feature value will activate a portion of the neurons.

[0082] Step 5-3: In the model designed in step 4, there are four agents in total, using... Represents the joint action space, where, The system executes actions based on the separate scheduling decisions of the four types of intelligent agents. Determined by the priority judgment mechanism: .

[0083] Step 5-4: Design the reward function: a composite reward function The scheduling time efficiency reward is set as follows: if there are incomplete tasks, Otherwise, it is 0; the load balancing efficiency reward is set as follows: let the variance of cluster resource utilization at time t be... ,like ,but Next, the two rewards will be combined, resulting in a total reward of... The weight design is shown in the table below:

[0084] intelligent agent High CPU task intelligent agent 0.3 0.7 High-memory task intelligent agents 0.3 0.7 Long-duration intelligent agents 1 / Balanced task agent 0.5 0.5

[0085] Step 6: Construct the decision network and evaluation network. During training iterations, update them as follows:

[0086] Each agent i maintains an independent policy network, denoted as . ,in The network parameters are given, and the input is the local observations of agent i. After feature extraction via a hidden layer with ReLU activation, the output layer uses the Softmax function to generate a value in the current action space. probability distribution on The gradient update formula is: ,in: It is the advantage function, which guides the direction of gradient updates; It is the entropy of the policy distribution, used to measure the randomness of the policy; It is the entropy coefficient, used to regulate the balance between exploration and utilization;

[0087] Evaluation Network Access to global state information during the training phase global state It contains the observation set of all agents. The Critic network evaluates the quality of the current joint state by fitting a state-value function, where the objective value is constructed using a temporal difference method. : The parameters of the Critic network are updated by minimizing the mean square error between the predicted value and the TD target value: , where B is the training batch size;

[0088] To reduce the variance of policy gradient estimation and accelerate algorithm convergence, this paper uses baseline subtraction to calculate the advantage function. The advantage function is represented in the state. Next, execute the action. The additional benefit compared to average performance is Utilizing TD error Approximation: Then the advantage function can be estimated as .

[0089] The trained model is used to implement task scheduling on batch processing tasks to obtain the optimal scheduling strategy.

[0090] Experimental comparison

[0091] The following comparison is based on deep reinforcement learning models and single-modal impulse reinforcement learning models.

[0092] The initial state of each node in the experiment was set as follows: CPU size 64, memory size 1, and disk size 1. Each task included CPU, memory, disk size, runtime, start time, and priority. In terms of scale, different load intensities were constructed by increasing the task length: the number of tasks to be scheduled was increased from 10 to 100, increasing by 10 each time, for both training and testing.

[0093] Figure 4This paper demonstrates the scheduling time deviation between the present invention and single-modal impulse reinforcement learning relative to deep reinforcement learning algorithms under different task lengths. Experimental results show that, across the entire task length range, the present invention consistently ranks below the single-modal network curve, indicating a significant advantage in scheduling time.

[0094] Figure 5 The data curves for this invention demonstrate energy consumption variations, showing that the curves remain at their lowest points throughout the entire task length, with the energy consumption difference consistently maintained at a low level. In contrast, single-modal impulsive reinforcement learning and deep reinforcement learning models generate significantly higher additional energy consumption during operation.

[0095] In summary, this invention addresses the challenges of complex resource types and unbalanced loads in heterogeneous cloud computing environments during batch task scheduling. It achieves an optimal scheduling strategy in complex environments by simultaneously optimizing total task completion time and cluster load stability. Based on spiking reinforcement learning and using an Actor-Critic network as its core architecture, this invention constructs a highly adaptive task scheduling model through a multi-agent collaborative mechanism. Furthermore, it deeply considers the multimodal attribute preferences of tasks, driving independent decision-making by corresponding dedicated agents through K-means clustering and feature labeling. Moreover, this invention fully leverages the biological characteristics of spiking neural networks, enhancing model robustness through frequency, delay, and population coding, and introducing a dual sparse attention mechanism to constrain and update interactions between agents, significantly reducing computational overhead while improving scheduling accuracy. Experiments demonstrate that this scheme exhibits excellent energy-saving performance and scheduling efficiency under varying load intensities, demonstrating significant technical gains.

[0096] The beneficial effects of this invention are as follows:

[0097] (1) This invention deeply explores the personalized needs and preferences of batch processing tasks for multidimensional resources, and combines the hardware resource advantages of heterogeneous computing nodes to achieve efficient mapping between tasks and nodes through feature decoupling and clustering methods. This precise matching strategy effectively avoids performance loss caused by resource mismatch.

[0098] (2) The impulse reinforcement learning model constructed in this invention has a strong adaptive adjustment capability and can perceive the dynamic fluctuations and task load changes in the simulated cloud computing environment in real time. Through the dynamic adjustment of the impulse firing threshold and the online evolution of the policy network, the system can quickly respond and optimize the scheduling instructions, while reducing the energy consumption of the model's inference and training.

Claims

1. An adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning, characterized in that, include: Decouple batch processing tasks using multi-dimensional features. Irrelevant feature removal and priority labeling; Cluster analysis is performed on the CPU, memory and time of the tasks to obtain tasks with different feature types. These tasks are then packaged and sorted within each task class, with tasks with high priority and low latency placed at the front of the queue. The resulting task types serve as the basis for subsequent model design. Constructing a spurious reinforcement learning multi-agent model; Construct a task scheduling environment for spurious reinforcement learning; Construct the decision network and evaluation network of the spurious reinforcement learning model.

2. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 1, characterized in that, The batch processing task information includes: CPU resource requirements, memory resource requirements, disk resource requirements, start time, duration, and DAG.

3. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 2, characterized in that, The batch processing task is extracted to obtain feature information, and the retained feature information is as follows: ,in: vector This represents the resource requirement characteristics of the task, where, For the task The computing resource requirements For the task Storage resource requirements For the task Disk resource requirements; use Indicates task Priority characteristics; use This indicates the scheduling time characteristics of a task.

4. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 3, characterized in that, The method for decoupling analysis and classification of task features is K-means clustering.

5. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 4, characterized in that, A spurious reinforcement learning multi-agent model is constructed, and four types of agents are designed according to task characteristics: computationally intensive agents, storage-intensive agents, balanced agents, and long-duration agents. Each agent consists of an independent Actor-Critic network, wherein: The decision network (Actor) learns local features and outputs the optimal scheduling action for the current type of agent; The evaluation network (Critic) learns global features and updates the network. To reduce computational overhead, a dual sparse attention mechanism is introduced.

6. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 5, characterized in that, The dual sparse attention mechanism employs dual constraints based on spatial proximity and task relevance, where: In spatial proximity constraints, the Top-k selection strategy is used to retain only the K nearest neighbors, as shown in the formula: , here This represents a spatial proximity mask; 0 indicates that this information is retained. This indicates that it will be removed in Softmax; In the task proximity constraint, cosine similarity is used to calculate the current query. With all keys The semantic relevance was determined, and the highest relevance was selected. Each node generates a task mask, using the following formula: , ,in, Represents a task-related mask, a vector This represents the retrieval demand vector of the current agent j, while This maps the attribute feature encoding of the neighboring agent j; The two mask matrices are fused using a logical AND operation to obtain the final attention distribution. Numerically, this is equivalent to adding the masks: At this point, only agent j that simultaneously satisfies both the conditions of close spatial distance and matching task attributes can... Only if it is 0, otherwise it is 0. .

7. The adaptive task scheduling method for batch processing tasks based on spurious reinforcement learning according to claim 6, characterized in that, Establish a compound reward function with dual objectives. ,in: The scheduling time efficiency reward is set as follows: if there are incomplete tasks... Otherwise, it is 0; The load balancing efficiency reward is set as follows: Let the variance of cluster resource utilization at time t be... ,like ,but Next, the two rewards will be combined, resulting in a total reward of...

8. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 7, characterized in that, Establish multimodal state input: Frequency coding is used to represent the magnitude of values, which include two categories: machine resources and task resources. After normalizing the values, the larger the value, the more pulses per unit time, thus representing the importance of the task and the machine. Delay coding is used to represent task priority, thus characterizing urgency. Task priority is used as the coding input; the higher the value, the earlier the transmission pulse. To improve the robustness of the model, a small group of neurons is assigned to the task set of each agent class. Each neuron is sensitive to specific agent features, and a certain type of feature value will activate a portion of the neurons.

9. The adaptive task scheduling method for batch processing tasks based on impulse reinforcement learning according to claim 8, characterized in that, The steps to update a spiking reinforcement learning model include: Each agent i maintains an independent policy network, denoted as . ,in The network parameters are given, and the input is the local observations of agent i. After passing through a hidden layer with ReLU activation for feature extraction, the output layer uses the Softmax function to generate the current action space. probability distribution on The gradient update formula is: ,in: It is the advantage function, which guides the direction of gradient updates; It is the entropy of the policy distribution, used to measure the randomness of the policy; It is the entropy coefficient, used to regulate the balance between exploration and utilization. The strategy network is a spiking neural network. When designing the neuron threshold, the threshold of the spiking neuron in the l-th layer is set to be... Introducing the advantage function As a regulating factor, the advantage function is defined as follows: This reflects the quality of the current action relative to the average level. The threshold update formula is: To ensure that the threshold is always positive and controllable, a parameterized form can be used: ,in, The indicator function indicates that modulation is applied only to neurons that produce firing behavior, in the following way: ; Evaluation Network Access to global state information during the training phase global state It contains the observation set of all agents. The Critic network evaluates the quality of the current joint state by fitting a state-value function, where the objective value is constructed using a temporal difference method. : The parameters of the Critic network are updated by minimizing the mean square error between the predicted value and the TD target value: , where B is the training batch size.

Images