Task scheduling method and system based on structured modeling and reinforcement learning, terminal and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a task scheduling method and system based on structured modeling and reinforcement learning, a terminal and a storage medium, and the method comprises the steps: obtaining resource state information and task information, carrying out the topological connection of the resource state information and the task information, and obtaining a heterogeneous graph; obtaining an attention weight, carrying out feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, carrying out relation aggregation on the initial representation vector according to the attention weight to obtain a plurality of representation vectors, and carrying out vector processing on all the representation vectors through the graph attention network to obtain vectorization representation; and performing task scheduling on the vectorized representation through a scheduling decision model to obtain a scheduling allocation result. The method is based on heterogeneous graph modeling and the graph attention network, reinforcement learning is combined, globally optimized task scheduling and resource allocation are achieved, and the task data allocation efficiency is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a task scheduling method, system, terminal, and computer-readable storage medium based on structured modeling and reinforcement learning. Background Technology

[0002] At the state modeling level, existing scheduling methods based on heuristics or deep reinforcement learning typically represent cluster states as a set of statistical vectors, such as CPU utilization, memory usage, and task queue length. While this flattened representation is computationally simple, it ignores the dynamic mapping relationship between tasks and Executors, as well as the resource contention structure among multiple tasks. In multi-task concurrent scenarios, different tasks may depend on the same Executor resource, or there may be execution priority and data dependencies. Traditional vectorized representations cannot explicitly express this topology, making it difficult for scheduling strategies to make reasonable allocation decisions from a global perspective.

[0003] Traditional scheduling methods based on heuristics or simple deep reinforcement learning (such as FIFO (FirstIn, FirstOut) schedulers, Fair (Fair Scheduling) schedulers, and reinforcement learning scheduling schemes using multilayer perceptrons or recurrent neural networks) cannot characterize the dynamic topological relationships, resource contention structures, and global dependencies of multitasking in Spark clusters because state modeling typically uses flat statistical vector representations. This makes it difficult to achieve globally optimized task scheduling and resource allocation in large-scale distributed computing scenarios with high task concurrency and strong resource heterogeneity, which has become an urgent problem to be solved.

[0004] For example, a platform needs to analyze user behavior log data daily. The cluster consists of a mix of server nodes with different configurations, including CPU-intensive, I / O-intensive, and general-purpose nodes. During peak periods, hundreds of Spark Tasks may be queued for scheduling simultaneously. Traditional FIFO schedulers allocate tasks based on queue order, regardless of task type or node status. As a result, CPU-intensive tasks are crammed onto nodes with large memory but few CPUs, while I / O-intensive tasks are squeezed onto CPU nodes. Task data, even when located on machine A, is scheduled to be executed on node B across a rack, requiring data to be transferred over the network, resulting in significant network I / O. Multiple tasks compete for resources without global coordination, leading to cluster resource utilization of less than 70%, yet jobs are delayed indefinitely.

[0005] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0006] The main objective of this invention is to provide a task scheduling method, system, terminal, and computer-readable storage medium based on structured modeling and reinforcement learning. This invention aims to solve the problem that in large-scale distributed computing scenarios with high task concurrency and strong resource heterogeneity, existing technologies cannot achieve globally optimized task scheduling and resource allocation, and thus cannot improve the efficient allocation of task data.

[0007] To achieve the above objectives, this invention provides a task scheduling method based on structured modeling and reinforcement learning, which includes the following steps: Obtain resource status information and task information, and perform topological connections between the resource status information and the task information to obtain a heterogeneous graph; Obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and aggregate the initial representation vector according to the attention weights to obtain multiple representation vectors. The graph attention network and activation function are used to concatenate all the representation vectors to obtain a first vectorized representation. The first vectorized representation is averaged to obtain a second vectorized representation. The first vectorized representation and the second vectorized representation are then residually concatenated to obtain a vectorized representation. The vectorized representation is used to schedule tasks through a scheduling decision model to obtain scheduling allocation results.

[0008] Optionally, in the task scheduling method based on structured modeling and reinforcement learning, the resource status information includes static resource characteristics, dynamic resource characteristics, and deployment location characteristics. The heterogeneous graph includes task preference edges, executor service edges, and executor co-location edges; The process of acquiring resource status information and task information, and then performing topological connections between the resource status information and the task information to obtain a heterogeneous graph, specifically includes: Obtain static resource characteristics, dynamic resource characteristics, deployment location characteristics, and task information; model the static resource characteristics, dynamic resource characteristics, and deployment location characteristics to obtain an executor set; Obtain feature construction rules, and perform topological connections between the executor nodes of the executor set and the task nodes of the task information according to the feature construction rules to obtain task preference edges, executor service edges and executor co-location edges.

[0009] Optionally, in the task scheduling method based on structured modeling and reinforcement learning, the initial representation vector includes task node representation vectors and executor node representation vectors. The process of obtaining attention weights involves learning features from the heterogeneous graph using a graph attention network to obtain an initial representation vector. Then, based on the attention weights, the initial representation vector is aggregated to obtain multiple representation vectors. Specifically, this includes: The task preference edge, the executor service edge, and the executor co-location edge are used to learn features through a graph attention network to obtain the task node representation vector and the executor node representation vector. Obtain attention weights, and aggregate the relationships between the task node representation vector and the executor node representation vector based on the attention weights to obtain multiple representation vectors.

[0010] Optionally, the task scheduling method based on structured modeling and reinforcement learning, wherein obtaining attention weights and aggregating the relationships between the task node representation vectors and the executor node representation vectors according to the attention weights to obtain multiple representation vectors specifically includes: Obtain attention weights, and perform relational aggregation on the task node representation vector and the executor node representation vector based on the attention weights to obtain an attention score; Obtain a normalization function, and normalize the attention score according to the normalization function to obtain the attention coefficient; The attention coefficients are transformed using a multi-head attention mechanism to obtain multiple representation vectors.

[0011] Optionally, the task scheduling method based on structured modeling and reinforcement learning, wherein the step of concatenating all the representation vectors through the graph attention network to obtain a first vectorized representation and a second vectorized representation, and performing a residual connection between the first vectorized representation and the second vectorized representation to obtain a vectorized representation, specifically includes: The activation function is obtained, and all the representation vectors are concatenated using the intermediate layer of the graph attention network and the activation function to obtain a first vectorized representation. The first vectorized representation is then averaged using the output layer of the graph attention network and the activation function to obtain a second vectorized representation. ; in, This represents the second vectorized representation. This represents the activation function. This indicates the total number of attention heads. Indicates the attention coefficient. Represents a node The set of neighboring nodes, Indicates the first The weight matrix of each attention head. Represents the eigenvector; By performing a residual concatenation between the first vectorized representation and the second vectorized representation, we obtain the vectorized representation: ; in, Represents a vectorized representation. This represents the first vectorized representation. This represents the transformation function of the second-layer GAT.

[0012] Optionally, in the task scheduling method based on structured modeling and reinforcement learning, the scheduling allocation result includes any one of process-local result, node-local result, rack-local result, and arbitrary local result; The step of scheduling tasks based on the vectorized representation using a scheduling decision model to obtain scheduling allocation results specifically includes: The initial allocation result is obtained by jointly deciding on the vectorized representation through a scheduling decision model; Obtain the constraints, and constrain the initial allocation result according to the constraints to obtain any one of the following: process local result, node local result, rack local result, and arbitrary local result.

[0013] Optionally, the task scheduling method based on structured modeling and reinforcement learning, wherein the step of scheduling the vectorized representation through a scheduling decision model to obtain the scheduling allocation result, further includes: Based on the scheduling allocation result, determine the scheduling feedback information, perform multi-objective reward calculation on the scheduling feedback information, and obtain multiple reward values; All the aforementioned reward values ​​are weighted and calculated to obtain the target reward value, which is then stored in the trajectory buffer. ; in, Indicates the target reward value. Indicates execution time information. Indicates data locality level information. Indicates load distribution information, This indicates that the execution was successful. Indicates energy consumption information. This represents the first weight matrix. This represents the second weight matrix. This represents the third weight matrix. This represents the fourth weight matrix. This represents the fifth weight matrix; If the trajectory buffer reaches a preset value, the scheduling decision model is optimized and updated according to the target reward value to obtain the optimal scheduling decision model. The scheduling feedback information includes execution time information, data locality level information, load distribution information, execution success information, and energy consumption information.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a task scheduling system based on structured modeling and reinforcement learning, wherein the task scheduling system based on structured modeling and reinforcement learning: The heterogeneous graph construction module is used to acquire resource status information and task information, and to perform topological connections between the resource status information and the task information to obtain a heterogeneous graph. The vector aggregation module is used to obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and perform relation aggregation on the initial representation vector according to the attention weights to obtain multiple representation vectors. The vectorized representation concatenation module is used to concatenate all the representation vectors through the graph attention network and activation function to obtain a first vectorized representation, to average the first vectorized representation to obtain a second vectorized representation, and to perform residual concatenation between the first vectorized representation and the second vectorized representation to obtain a vectorized representation. The task scheduling module is used to schedule tasks based on the vectorized representation through a scheduling decision model to obtain scheduling allocation results.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a task scheduler based on structured modeling and reinforcement learning, and the task scheduler based on structured modeling and reinforcement learning, when executed by a processor, implements the steps of the task scheduling method based on structured modeling and reinforcement learning as described above.

[0016] This invention acquires resource status information and task information, and topologically connects the resource status information and task information to obtain a heterogeneous graph. Attention weights are acquired, and feature learning is performed on the heterogeneous graph using a graph attention network to obtain initial representation vectors. Relationship aggregation is performed on the initial representation vectors according to the attention weights to obtain multiple representation vectors. All representation vectors are concatenated using the graph attention network and an activation function to obtain a first vectorized representation. The first vectorized representation is averaged to obtain a second vectorized representation. The first and second vectorized representations are residually connected to obtain a vectorized representation. A scheduling decision model is used to schedule tasks based on the vectorized representations to obtain a scheduling allocation result. This invention, based on heterogeneous graph modeling and graph attention networks, combined with reinforcement learning, achieves globally optimized task scheduling and resource allocation, significantly improving the efficiency of task data allocation. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the task scheduling method based on structured modeling and reinforcement learning of the present invention; Figure 2 This is a schematic diagram of heterogeneous graph information aggregation, representing a preferred embodiment of the task scheduling method based on structured modeling and reinforcement learning of the present invention. Figure 3 This is a schematic diagram of a preferred embodiment of the three-layer collaborative scheduling method of the present invention based on structured modeling and reinforcement learning. Figure 4 This is a schematic diagram of the intelligent scheduling online learning closed-loop structure of a preferred embodiment of the task scheduling method based on structured modeling and reinforcement learning of the present invention; Figure 5 This is a schematic diagram of the weighted aggregation mechanism of a preferred embodiment of the task scheduling method based on structured modeling and reinforcement learning of the present invention; Figure 6 This is a structural diagram of a preferred embodiment of the task scheduling system based on structured modeling and reinforcement learning of the present invention; Figure 7 This is a structural diagram of a preferred embodiment of the terminal of the device of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] Traditional scheduling methods based on heuristics or simple deep reinforcement learning (such as FIFO, Fair schedulers, and reinforcement learning scheduling schemes using multilayer perceptrons or recurrent neural networks) cannot characterize the dynamic topological relationships, resource contention structures, and global dependencies of multi-task concurrency between task nodes and executor nodes in a Spark cluster because state modeling typically uses flat statistical vector representations. This makes it difficult to achieve globally optimized task scheduling and resource allocation in large-scale distributed computing scenarios with high task concurrency and strong resource heterogeneity. Therefore, a task scheduling method based on structured modeling and reinforcement learning is needed. This method combines heterogeneous graph modeling and graph attention networks with reinforcement learning to achieve globally optimized task scheduling and resource allocation, greatly improving the efficiency of task data allocation.

[0020] The task scheduling method based on structured modeling and reinforcement learning described in the preferred embodiment of the present invention, such as... Figure 1 and Figure 2 As shown, the task scheduling method based on structured modeling and reinforcement learning includes the following steps: Step S10: Obtain resource status information and task information, and perform topological connection between the resource status information and the task information to obtain a heterogeneous graph.

[0021] Step S10 includes: Step S11: Obtain static resource characteristics, dynamic resource characteristics, deployment location characteristics, and task information; model the static resource characteristics, dynamic resource characteristics, and deployment location characteristics to obtain an executor set; Step S12: Obtain feature construction rules, and perform topological connections between the executor nodes of the executor set and the task nodes of the task information according to the feature construction rules to obtain task preference edges, executor service edges and executor co-location edges.

[0022] Specifically, static resource features, dynamic resource features, deployment location features, and task information are obtained. These features are then modeled to obtain an executor set (where task information describes the task's stage, data location, and execution attributes, and resource status information describes its resource availability and running status). Feature construction rules are obtained, and based on these rules, topological connections are made between the executor nodes of the executor set and the task nodes of the task information to obtain task preference edges. ), actuator service side ( ) and actuator co-positioned edge ( ).

[0023] In this embodiment, it is used to represent a task node. For actuator nodes The execution preference relationship is mainly determined by the task's data locality information, denoted as: ; Among them, the task preference edge represents the task node. Data partitions are stored in the executor The task is located on the same node or rack node. During scheduling, the executor It has a data locality affinity preference, reflecting the task's preference for specific actuators, and is used to support the optimization goals of reducing cross-node data transmission and reducing network I / O power consumption.

[0024] Furthermore, it is used to represent the actuator node. Able to serve as a task node The relationship that provides execution services, as a supplement to the task preference edge, is denoted as: ; By introducing this reverse association, the graph structure can simultaneously aggregate information from both the task side and the executor side during information propagation. Specifically: the executor service edge represents the executor node. Able to serve as a task node It provides execution services, which are generated in pairs with task preference edges in opposite directions. The service edges are designed as independent edge types rather than degenerating into simple undirected edges because the semantics of the information transmitted in the two directions are completely different: the executor node transmits dynamic resource status information such as the current number of available cores and memory utilization to the task node along the edge. This allows the task node's representation to simultaneously integrate two dimensions: data location preference and executor real-time availability, forming a complete bidirectional perception mechanism. This supports the scheduling strategy in reducing data transmission energy consumption while avoiding resource overload.

[0025] Furthermore, it is used to represent the physical or logical proximity relationship between actuator nodes, such as being located on the same host or in the same rack, and is denoted as: ; Among them, the co-location edge of actuators represents the proximity relationship between actuator nodes in terms of physical or logical topology. The proximity relationship is divided into two levels: co-location with the same host ( and Deployed on the same physical host, sharing CPU, memory, and local disk resources, with bidirectional connections between the same host executors via a complete graph (bidirectional edges) and co-located in the same rack ( and Located in the same rack, sharing the bandwidth of the switches within the rack. This type of edge (representing a task node) is constructed based on the host identifier and rack identifier fields in the executor node characteristics. This edge is used to characterize the potential resource contention and load correlation between executors. GAT transmits load status information between executor nodes through shared edges, enabling each executor node's representation to perceive the current load distribution of neighboring executors, providing a local load view for scheduling decisions and supporting load balancing optimization goals. In the implementation, undirected shared edges are converted into two directed edges in opposite directions to adapt to the computational requirements of the message passing framework.

[0026] Specifically, the resource status information is shown in Table 1: Table 1: Resource Status Information

[0027] In this embodiment, at the scheduling trigger moment, a heterogeneous graph structure needs to be constructed based on the task set and the executor set. This process includes: node feature construction, task-executor preference relationship construction, and co-location relationship construction between executors. The node construction complexity calculation formula is as follows: ; in, This indicates that the complexity of establishing executor co-location relationships is related to the executor deployment topology. This indicates the number of tasks currently awaiting scheduling. This indicates the number of currently available executors.

[0028] Furthermore, given a fixed graph structure, the complexity of each round of feature updates and the number of edges... Proportional. If proceeding If the feature is updated in round 1, then the complexity of this stage is: ; in, Indicates the GAT feature update round number. This represents the number of edges in the heterogeneous graph.

[0029] For example, if a full matching calculation is used, the complexity of this stage is O(n log n). The computational overhead is reduced by masking unavailable executors, limiting the set of candidate executors available for each task, and performing segmented decision-making at the task batch level. These methods ensure that the scheduling decision generation time maintains near-linear growth as the task size increases. The online learning phase mainly includes: reward calculation; experience storage; and batch policy updates. Both reward calculation and experience storage are constant-time operations, i.e. The complexity and batch size of the policy update phase. And related to the model parameter scale, approximately ,in, This represents the number of model parameters.

[0030] Specifically, the complexity characteristics of the key stages are shown in Table 2: Table 2: Complexity Characteristics of Key Stages

[0031] Step S20: Obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and perform relation aggregation on the initial representation vector according to the attention weights to obtain multiple representation vectors.

[0032] like Figure 3 As shown, step S20 includes: Step S21: Perform feature learning on the task preference edge, the executor service edge, and the executor co-location edge through a graph attention network to obtain the task node representation vector and the executor node representation vector; Step S22: Obtain attention weights, and perform relational aggregation on the task node representation vector and the executor node representation vector based on the attention weights to obtain multiple representation vectors.

[0033] Specifically, feature learning is performed on the task preference edge, the executor service edge, and the executor co-location edge using a Graph Attention Network (GAT) to obtain the task node representation vector and the executor node representation vector. That is, the Graph Attention Network is used to perform feature learning on the constructed heterogeneous graph to obtain attention weights. Based on the attention weights, the task node representation vector and the executor node representation vector are subjected to relation aggregation to obtain multiple representation vectors. In other words, through message passing with attention weights and relation aggregation, vectorized representations of task nodes and executor nodes in the current scheduling state are generated respectively. The initial representation vectors include the task node representation vector and the executor node representation vector.

[0034] In this embodiment, the input to the degree state representation generation method is the task-actuator heterogeneous graph constructed in the previous section. , where the set of nodes Includes a set of task nodes With the set of executor nodes edge set Containing various types of relational edges, the heterogeneous graph is subjected to feature encoding and information aggregation. The output includes: each task node. The corresponding task state representation vector, for each executor node The corresponding actuator state representation vector.

[0035] Step S22 includes: Step S221: Obtain attention weights, and perform relation aggregation on the task node representation vector and the executor node representation vector according to the attention weights to obtain attention scores; Step S222: Obtain the normalization function, and normalize the attention score according to the normalization function to obtain the attention coefficient; Step S223: Perform transformation feature calculation on the attention coefficients according to the multi-head attention mechanism to obtain multiple representation vectors.

[0036] Specifically, attention weights are obtained, and the relationship between the task node representation vector and the executor node representation vector is aggregated based on the attention weights to obtain an attention score. A normalization function is obtained, and the attention score is normalized based on the normalization function to obtain attention coefficients. The attention coefficients are then transformed using a multi-head attention mechanism to obtain multiple representation vectors.

[0037] In this embodiment, firstly, for any edge in the heterogeneous graph... Calculate the unnormalized attention score: ; in, This represents the unnormalized attention score. This represents the activation function. Represents the weight matrix. Represents a node The input feature vector, Represents a node. Represents a node The input feature vector, This indicates a vector concatenation operation.

[0038] Furthermore, for nodes All neighboring nodes Normalization yields the normalized attention coefficients: ; in, Indicates the attention coefficient. Represents the normalization function. This represents the natural exponential function. express The set of neighboring nodes, This indicates the input value.

[0039] Furthermore, nodes The updated representation is calculated by weighted aggregation of the transformation features of its neighboring nodes: ; in, This represents the first vectorized representation. Represents the eigenvector. This represents a non-linear activation function.

[0040] As an example, this application applies to the intermediate layer (first-layer GAT), where the outputs of each attention head are merged through splicing. This involves concatenating the outputs of k attention heads to preserve the heterogeneous semantic information captured by each head. Specifically, each attention head outputs a 32-dimensional vector, and concatenating the outputs of four heads yields a 128-dimensional output, thus expanding the dimensionality while preserving the semantic differences between the multiple heads. and These are the attention coefficients and weight matrices of the k-th attention head, respectively, with the parameters of each head being independent and optimized in parallel.

[0041] Step S30: Concatenate all the representation vectors using the graph attention network and activation function to obtain a first vectorized representation; average the first vectorized representation to obtain a second vectorized representation; and perform residual connection between the first vectorized representation and the second vectorized representation to obtain a vectorized representation.

[0042] Step S30 includes: Step S31: Obtain the activation function. Concatenate all the representation vectors through the intermediate layer of the graph attention network and the activation function to obtain the first vectorized representation. Average the first vectorized representation through the output layer of the graph attention network and the activation function to obtain the second vectorized representation. Step S32: Perform a residual connection between the first vectorized representation and the second vectorized representation to obtain a vectorized representation.

[0043] Specifically, the activation function is obtained, and all the representation vectors are concatenated through the intermediate layer of the graph attention network and the activation function to obtain the first vectorized representation (the first layer GAT, i.e., the intermediate layer: the input dimension is the original feature dimension of the node (8 dimensions for task nodes and 6 dimensions for executor nodes), and the output dimension is 128 dimensions (32 dimensions for each attention head, and 128 dimensions after concatenation of 4 heads)). The first vectorized representation is averaged through the output layer of the graph attention network and the activation function (both the first and second layers use ELU) to obtain the second vectorized representation (the second layer GAT, i.e., the output layer: the input dimension is 128 dimensions, the output dimension is 128 dimensions, and 4 attention heads are used (averaged)). The first vectorized representation and the second vectorized representation are residually connected to obtain the vectorized representation.

[0044] In this embodiment, the output layer (second-layer GAT) is modified by averaging the results from each head instead of concatenating them. This converges the representation dimension to a unified 128-dimensional hidden layer dimension, facilitating subsequent input processing by the Actor-Critic policy network. The reason for using averaging instead of concatenation is that the output layer needs to generate node embedding vectors of a fixed specification for the scheduling decision module. The averaging operation maintains the stability of the output dimension while fusing information from multiple heads. ; in, This represents the second vectorized representation. Indicates the attention coefficient. This represents the weight matrix of the k-th attention head.

[0045] Furthermore, residual joins: The second-layer GAT uses residual joins, as shown in the following formula: ; in, Represents a vectorized representation. This represents the first vectorized representation. This represents the transformation function of the second-layer GAT.

[0046] Step S40: Perform task scheduling on the vectorized representation using the scheduling decision model to obtain the scheduling allocation result.

[0047] Step S40 includes: Step S41: Perform joint decision-making on the vectorized representation through the scheduling decision model to obtain the initial allocation result; Step S42: Obtain constraints, and constrain the initial allocation result according to the constraints to obtain any one of the following: process local result, node local result, rack local result, and arbitrary local result.

[0048] Specifically, after step S30, the method further includes determining scheduling feedback information based on the scheduling allocation result, performing multi-objective reward calculation on the scheduling feedback information to obtain multiple reward values, weighting all the reward values ​​to obtain a target reward value, and storing the target reward value in the trajectory buffer. ; in, Indicates the target reward value. Indicates execution time information. Indicates data locality level information. Indicates load distribution information, This indicates that the execution was successful. Indicates energy consumption information. This represents the first weight matrix. This represents the second weight matrix. This represents the third weight matrix. This represents the fourth weight matrix. This represents the fifth weight matrix.

[0049] Furthermore, if the trajectory buffer reaches a preset value, the scheduling decision model is optimized and updated according to the target reward value to obtain the optimal scheduling decision model.

[0050] In this embodiment, a joint decision is made on the vectorized representation through a scheduling decision model to obtain an initial allocation result, obtain constraints, and constrain the initial allocation result according to the constraints to obtain any one of the following: process local result, node local result, rack local result, and arbitrary local result. The constraints include execution time information, data locality level information, load distribution information, execution success information, and energy consumption information.

[0051] For example, a new batch of TaskSets to be scheduled enters the Scheduling Queue; existing tasks are completed, releasing executor resources; the cluster resource status undergoes a significant change, satisfying preset scheduling triggering conditions (constraints). These significant changes specifically include the following: the CPU utilization or memory utilization of a certain Executor changes by more than a preset threshold (e.g., a change exceeding 20% ​​within a single data collection period); an Executor fails or goes offline, reducing the number of available executors; the number of tasks backlogged in the current scheduling queue exceeds a preset upper limit; or the time interval since the last scheduling trigger exceeds the maximum waiting time. When any of the above conditions are met, the system considers the cluster status to have changed significantly and triggers a new round of scheduling decisions. After scheduling is triggered, the scheduling management component sends the current scheduling status to the intelligent decision-making module. This scheduling status is composed of the task status representation and executor status representation generated by the aforementioned method. Triggering conditions include: 1. A new batch of tasks to be scheduled enters the scheduling queue. When a new TaskSet enters the scheduling queue, a scheduling decision process is triggered. 2. Existing tasks complete execution and release executor resources. When existing tasks complete execution and executor resources are released, a scheduling decision process is triggered. 3. A significant change in cluster resource status. The paper describes a significant change as: the CPU or memory utilization of an executor changes beyond a preset threshold. 4. The time interval since the last scheduling trigger exceeds the maximum waiting time.

[0052] In this embodiment, after obtaining the scheduling allocation result, the scheduling management component converts the result into a scheduling instruction recognizable by the Spark cluster (the Agent returns JSON format data through the HTTP server interface) and sends it to the corresponding executor management module (in the resourceOffers() method of TaskSchedulerImpl.scala). The executor management module starts the Task execution process on the corresponding Executor based on the received scheduling instruction. During Task execution, the cluster completes resource allocation, task execution, and intermediate result management according to existing mechanisms, without making intrusive modifications to the original execution logic, thus ensuring system compatibility. When generating scheduling decisions, the following hard constraints must be met: each task can only be assigned to one executor; the available resources of the executor must meet the minimum resource requirements of the task; if the task has data locality preferences, executors with higher locality are selected first, provided that resource constraints are met.

[0053] For example, such as Figure 4 As shown, the process of generating scheduling and allocation results is as follows: Step 1: GAT encoding, using heterogeneous graphs... The first step involves inputting the task and executor representations into a two-layer GAT encoder, generating 128-dimensional embedding vectors for both task and executor nodes. The second step is Actor network inference, where the task and executor representations generated by the GAT are input into the Actor network. The Actor network outputs the action probability distribution for each task across all available executors. The third step is a validity mask filtering, where the probability of executors that do not meet the following three hard constraints are set to zero: each task can only be assigned to a single executor; the executor's current available resources must meet the task's minimum resource requirements; and, under the premise of meeting resource constraints, executors with higher locality are prioritized. The fourth step is sampling to obtain the allocation results, based on the filtered validity probability distribution, to obtain the final task-executor allocation result. After task execution, the system collects the execution results to form scheduling feedback information. This feedback information includes whether the task execution was successful, the execution time, and changes in executor load. The scheduling feedback is sent back to the intelligent decision-making module to evaluate the effectiveness of the current scheduling decision. By continuously collecting scheduling feedback and updating internal policy parameters, the intelligent decision-making module can gradually adjust scheduling preferences based on the dynamically changing load of the Spark cluster, enabling subsequent scheduling decisions to continuously adapt to the current job type distribution and load status of each worker node in the cluster.

[0054] In this embodiment, the PPO policy update mechanism schedules the policy through online iterative updates using the PPO algorithm. Its core elements include the following: a pruning objective function, and an experience replay mechanism: the experience quadruple generated from each scheduling decision. The data is stored in a trajectory buffer. When the accumulated number of data points in the buffer reaches a threshold, a policy update is triggered. Each update iterates for four rounds. After the update is completed, the buffer is cleared. Indicates the action to be taken. This represents the advantage discount factor compared to the average level. Indicates the target reward value. Indicates the next state.

[0055] As an example, after receiving standardized task feedback, the system calculates the scheduling reward value based on a multi-objective evaluation mechanism. The reward consists of multiple sub-items, each reflecting a different scheduling optimization objective.

[0056] Specifically, the target reward values ​​are shown in Table 3: Table 3: Target Reward Values

[0057] Furthermore, after triggering a policy update, the system retrieves all accumulated experience data from the buffer and iterates in batches according to a preset mini-batch size (default set to 32 experience records), performing multiple rounds of gradient updates on the policy parameters (default update rounds are 4). In each update round, the system recalculates the action probability and state value based on the current policy, and combines the PPO's pruning objective function to constrain the update magnitude, thus completing the optimization of the policy parameters.

[0058] Specifically, the hyperparameter settings are shown in Table 4: Table 4: Hyperparameter Settings

[0059] In this embodiment, the policy update process, for example, the PPO algorithm, constrains the policy update magnitude by pruning the objective function, thus avoiding performance collapse caused by excessively large single updates. Its core objective function is defined as follows: ; in, This represents the pruning substitution objective function of the PPO algorithm. This represents the expected estimate for the experience batch. Indicates the current policy network parameters. Represents the probability ratio. Represents the dominance function. Indicates the restricted range.

[0060] The formula for calculating the probability ratio is as follows: ; in, Indicates the current policy in state Select action The probability, This represents the probability corresponding to the old strategy (the strategy used during data collection).

[0061] Furthermore, The advantage function is calculated using the Generalized Advantage Estimation (GAE): ; in, Indicates timing difference error. Indicates the discount factor. Indicates GAE parameters, Represents a value function network.

[0062] Furthermore, the complete PPO loss function also includes the value function loss and the entropy regularization term: ; in, Represents the total loss function. Indicates expected estimate, Represents the loss coefficient of the value function. Represents the entropy regularity coefficient. Represents the loss of the value function. Represents the entropy of the strategy.

[0063] The formula for calculating the value function loss is as follows: ; in, Indicates the estimated value. This represents the estimated true return.

[0064] Furthermore, the calculation formula is as follows: ; in, Represents the policy entropy value. Indicates the current policy network. Indicates the current scheduling status.

[0065] Specifically, the key data flow during the online learning phase is shown in Table 5: Table 5: Key Data Flow During the Online Learning Phase

[0066] In this embodiment, as Figure 5 As shown, the overall architecture of the three-layer collaborative scheduling system of this invention is as follows: The top layer is the Spark execution layer, responsible for task execution and triggering scheduling requests; the middle layer is a software-layer scheduling agent based on reinforcement learning, which takes scheduling state (heterogeneous graph embedding, task / executor features) as input and outputs task-executor scheduling decisions through an RL decision model; the bottom layer is a hardware-layer frequency regulation agent based on PPO and DVFS, which takes node load state as input and outputs CPU frequency / power control instructions. The three layers work together through synchronous scheduling main links, cross-layer collaborative bypasses, and hardware-layer feedback channels.

[0067] As an example, the communication interface layer is responsible for data interaction with the Spark cluster. Its main functions include: receiving scheduling decision requests, receiving task execution feedback, returning task allocation results, and providing interfaces for health checks and model status queries. In actual operation, the Spark cluster sends decision request data to the intelligent scheduling agent via a standard HTTP interface, including information about the tasks to be scheduled and executor status information. After generating the allocation results, the scheduling agent returns the task-executor matching results to the cluster scheduling component through the interface. By adopting a standard interface protocol, the scheduling agent can be deployed as an independent process or a separate server without affecting the original Spark scheduler's operating logic. The communication interface adopts a RESTful API design, based on the HTTP / 1.1 protocol, and uses JSON as the data format. On the Spark cluster side, the `resourceOffers()` method of `TaskSchedulerImpl.scala` performs the following operations when scheduling is triggered: It constructs a scheduling request message containing the current list of tasks to be scheduled and the status of available executors; sends a POST request to the ` / api / v1 / schedule / request` endpoint of the Agent; waits for the Agent's response (timeout period 500ms); parses the `assignments` field in the response to obtain the task allocation result to the executor; and applies the allocation result to the Spark scheduling process. After the task execution is complete, `TaskSetManager.scala` reports feedback in the following ways: it extracts execution metrics (duration, resource usage, locality, etc.) from the `TaskResult`, constructs a feedback message, and asynchronously sends a POST request to the ` / api / v1 / schedule / feedback` endpoint of the Agent. Feedback sending failure does not affect the scheduling process; eventual consistency is ensured through a retry mechanism. Through the above interface design, loosely coupled communication between the Spark cluster and the external intelligent scheduling Agent is achieved, ensuring the reliability and scalability of the system.

[0068] In this embodiment, when the available resources of all executors are insufficient to immediately support new tasks, the scheduling system does not force the generation of unexecutable allocation results. Instead, it adopts the following processing strategy: The currently scheduled tasks are kept in the waiting queue; the executor resource status is periodically monitored; and the scheduling decision is retried after resource release is detected. When a batch of tasks fails simultaneously within a short period, the system performs the following processing: Negative rewards are applied to failed tasks during the reward calculation phase; failure experience is included in the experience buffer; policy parameters are updated when the training trigger condition is met; and by introducing a failure penalty term, the scheduling policy can gradually reduce the probability of allocation patterns leading to failure. During the policy update process, if parameter update anomalies, training interruptions, or insufficient data occur, the system adopts the following mechanism: the most recently valid policy parameters are retained, the current update process is suspended, anomaly log information is recorded, and the existing policy continues to be used for subsequent scheduling. This mechanism ensures that even if anomalies occur during the training phase, the scheduling service can continue to operate without affecting the normal task execution of the cluster.

[0069] As an example, in a distributed environment, feedback information may be missing or fields may be incomplete. To address this, the system takes the following approach: missing fields are supplemented with default values ​​or historical statistics; the experience record is marked as low-reliability data; and severely abnormal feedback is not included in the policy update sample. By filtering abnormal feedback, erroneous data is prevented from interfering with the policy optimization process. Specific abnormal feedback information includes: completely missing feedback information; and failure to receive any feedback data after Task execution due to network failure, node downtime, or communication timeout, resulting in the inability to construct the corresponding experience record. Incomplete feedback fields, with some fields missing from the feedback data, such as: missing execution duration field (e.g., task termination leading to unrecorded end time), missing data locality level field (e.g., scheduling logs not being written correctly), missing resource usage field (e.g., abnormal monitoring module data collection), or severely abnormal feedback data (e.g., although complete, the values ​​deviate significantly from the normal range, such as: negative execution duration or abnormally large values, resource utilization exceeding 100%, or extreme abnormal values ​​in energy consumption data). When the scheduling decision generation time exceeds a preset threshold, the system triggers a timeout handling mechanism: terminating the current decision calculation, returning to the allocation rule generated by the most recently valid strategy, or reverting to the system's default scheduling rule. This mechanism ensures that scheduling delays do not affect the overall task execution efficiency.

[0070] For example, during each task scheduling, a heterogeneous graph is built to identify the relationships between all scheduled tasks and available nodes, such as "whose data is where, what can each run, and which machines share the same rack". The graph attention network automatically identifies which node is most suitable for each task to be executed, and reinforcement learning makes the final scheduling decision based on this analysis. After each task is executed, the scheduling strategy is continuously optimized based on the actual effect, and the scheduling becomes more and more accurate.

[0071] Furthermore, such as Figure 6 As shown, based on the above-described task scheduling method based on structured modeling and reinforcement learning, this invention also provides a task scheduling system based on structured modeling and reinforcement learning, wherein the task scheduling system based on structured modeling and reinforcement learning includes: Heterogeneous graph construction module 51 is used to obtain resource status information and task information, and to perform topological connection of the resource status information and the task information to obtain a heterogeneous graph. The vector aggregation module 52 is used to obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and perform relation aggregation on the initial representation vector according to the attention weights to obtain multiple representation vectors. The vectorized representation splicing module 53 is used to perform vector splicing on all the representation vectors through the graph attention network and activation function to obtain a first vectorized representation, to perform average calculation on the first vectorized representation to obtain a second vectorized representation, and to perform residual connection on the first vectorized representation and the second vectorized representation to obtain a vectorized representation. The task scheduling module 54 is used to schedule tasks based on the vectorized representation through a scheduling decision model to obtain scheduling allocation results.

[0072] Furthermore, such as Figure 7 As shown, based on the above-mentioned task scheduling method and system based on structured modeling and reinforcement learning, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0073] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a task scheduler 40 based on structured modeling and reinforcement learning, which can be executed by the processor 10 to implement the task scheduling method based on structured modeling and reinforcement learning in this application.

[0074] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the task scheduling method based on structured modeling and reinforcement learning.

[0075] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The terminals communicate with each other via a system bus.

[0076] In one embodiment, when the processor 10 executes the task scheduler 40 based on structured modeling and reinforcement learning in the memory 20, the following steps are performed: Obtain resource status information and task information, and perform topological connections between the resource status information and the task information to obtain a heterogeneous graph; Obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and aggregate the initial representation vector according to the attention weights to obtain multiple representation vectors. The graph attention network and activation function are used to concatenate all the representation vectors to obtain a first vectorized representation. The first vectorized representation is averaged to obtain a second vectorized representation. The first vectorized representation and the second vectorized representation are then residually concatenated to obtain a vectorized representation. The vectorized representation is used to schedule tasks through a scheduling decision model to obtain the scheduling allocation result; The resource status information includes static resource characteristics, dynamic resource characteristics, and deployment location characteristics; The heterogeneous graph includes task preference edges, executor service edges, and executor co-location edges; The process of acquiring resource status information and task information, and then performing topological connections between the resource status information and the task information to obtain a heterogeneous graph, specifically includes: Obtain static resource characteristics, dynamic resource characteristics, deployment location characteristics, and task information; model the static resource characteristics, dynamic resource characteristics, and deployment location characteristics to obtain an executor set; Obtain feature construction rules, and perform topological connections between the executor nodes of the executor set and the task nodes of the task information according to the feature construction rules to obtain task preference edges, executor service edges and executor co-location edges.

[0077] The initial representation vector includes the task node representation vector and the executor node representation vector; The process of obtaining attention weights involves learning features from the heterogeneous graph using a graph attention network to obtain an initial representation vector. Then, based on the attention weights, the initial representation vector is aggregated to obtain multiple representation vectors. Specifically, this includes: The task preference edge, the executor service edge, and the executor co-location edge are used to learn features through a graph attention network to obtain the task node representation vector and the executor node representation vector. Obtain attention weights, and aggregate the relationships between the task node representation vector and the executor node representation vector based on the attention weights to obtain multiple representation vectors.

[0078] The step of obtaining attention weights involves aggregating the relationships between the task node representation vector and the executor node representation vector based on the attention weights to obtain multiple representation vectors, specifically including: Obtain attention weights, and perform relational aggregation on the task node representation vector and the executor node representation vector based on the attention weights to obtain an attention score; Obtain a normalization function, and normalize the attention score according to the normalization function to obtain the attention coefficient; The attention coefficients are transformed using a multi-head attention mechanism to obtain multiple representation vectors.

[0079] Specifically, the process of concatenating all representation vectors using the graph attention network and activation function to obtain a first vectorized representation, averaging the first vectorized representation to obtain a second vectorized representation, and performing a residual concatenation between the first and second vectorized representations to obtain a final vectorized representation includes: The activation function is obtained, and all the representation vectors are concatenated using the intermediate layer of the graph attention network and the activation function to obtain a first vectorized representation. The first vectorized representation is then averaged using the output layer of the graph attention network and the activation function to obtain a second vectorized representation. ; in, This represents the second vectorized representation. This represents the activation function. This indicates the total number of attention heads. Indicates the attention coefficient. Represents a node The set of neighboring nodes, Indicates the first The weight matrix of each attention head. Represents the eigenvector; By performing a residual concatenation between the first vectorized representation and the second vectorized representation, we obtain the vectorized representation: ; in, Represents a vectorized representation. This represents the first vectorized representation. This represents the transformation function of the second-layer GAT.

[0080] The scheduling and allocation result includes any one of the following: process local result, node local result, rack local result, and arbitrary local result; The step of scheduling tasks based on the vectorized representation using a scheduling decision model to obtain scheduling allocation results specifically includes: The initial allocation result is obtained by jointly deciding on the vectorized representation through a scheduling decision model; Obtain the constraints, and constrain the initial allocation result according to the constraints to obtain any one of the following: process local result, node local result, rack local result, and arbitrary local result.

[0081] The step of scheduling the vectorized representation using a scheduling decision model to obtain the scheduling allocation result further includes: Based on the scheduling allocation result, determine the scheduling feedback information, perform multi-objective reward calculation on the scheduling feedback information, and obtain multiple reward values; All the aforementioned reward values ​​are weighted and calculated to obtain the target reward value, which is then stored in the trajectory buffer. ; in, Indicates the target reward value. Indicates execution time information. Indicates data locality level information. Indicates load distribution information, This indicates that the execution was successful. Indicates energy consumption information. This represents the first weight matrix. This represents the second weight matrix. This represents the third weight matrix. This represents the fourth weight matrix. This represents the fifth weight matrix; If the trajectory buffer reaches a preset value, the scheduling decision model is optimized and updated according to the target reward value to obtain the optimal scheduling decision model. The scheduling feedback information includes execution time information, data locality level information, load distribution information, execution success information, and energy consumption information.

[0082] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a task scheduler based on structured modeling and reinforcement learning, and the task scheduler based on structured modeling and reinforcement learning, when executed by a processor, implements the steps of the task scheduling method based on structured modeling and reinforcement learning as described above.

[0083] In summary, this invention provides a task scheduling method, system, terminal, and storage medium based on structured modeling and reinforcement learning. The method includes: acquiring resource state information and task information; performing topological connection between the resource state information and the task information to obtain a heterogeneous graph; acquiring attention weights; performing feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector; performing relation aggregation on the initial representation vector according to the attention weights to obtain multiple representation vectors; concatenating all the representation vectors through the graph attention network and an activation function to obtain a first vectorized representation; averaging the first vectorized representation to obtain a second vectorized representation; performing residual connection between the first and second vectorized representations to obtain a vectorized representation; and performing task scheduling on the vectorized representations through a scheduling decision model to obtain a scheduling allocation result. This invention, based on heterogeneous graph modeling and graph attention networks, combined with reinforcement learning, achieves globally optimized task scheduling and resource allocation, greatly improving the efficiency of task data allocation.

[0084] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal system that includes that element.

[0085] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0086] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A task scheduling method based on structured modeling and reinforcement learning, characterized in that, The task scheduling method based on structured modeling and reinforcement learning includes: Obtain resource status information and task information, and perform topological connections between the resource status information and the task information to obtain a heterogeneous graph; Obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and aggregate the initial representation vector according to the attention weights to obtain multiple representation vectors. The graph attention network and activation function are used to concatenate all the representation vectors to obtain a first vectorized representation. The first vectorized representation is averaged to obtain a second vectorized representation. The first vectorized representation and the second vectorized representation are then residually concatenated to obtain a vectorized representation. The vectorized representation is used to schedule tasks through a scheduling decision model to obtain scheduling allocation results.

2. The task scheduling method based on structured modeling and reinforcement learning according to claim 1, characterized in that, The resource status information includes static resource characteristics, dynamic resource characteristics, and deployment location characteristics; The heterogeneous graph includes task preference edges, executor service edges, and executor co-location edges; The process of acquiring resource status information and task information, and then performing topological connections between the resource status information and the task information to obtain a heterogeneous graph, specifically includes: Obtain static resource characteristics, dynamic resource characteristics, deployment location characteristics, and task information; model the static resource characteristics, dynamic resource characteristics, and deployment location characteristics to obtain an executor set; Obtain feature construction rules, and perform topological connections between the executor nodes of the executor set and the task nodes of the task information according to the feature construction rules to obtain task preference edges, executor service edges and executor co-location edges.

3. The task scheduling method based on structured modeling and reinforcement learning according to claim 2, characterized in that, The initial representation vector includes the task node representation vector and the executor node representation vector; The process of obtaining attention weights involves learning features from the heterogeneous graph using a graph attention network to obtain an initial representation vector. Then, based on the attention weights, the initial representation vector is aggregated to obtain multiple representation vectors. Specifically, this includes: The task preference edge, the executor service edge, and the executor co-location edge are used to learn features through a graph attention network to obtain the task node representation vector and the executor node representation vector. Obtain attention weights, and aggregate the relationships between the task node representation vector and the executor node representation vector based on the attention weights to obtain multiple representation vectors.

4. The task scheduling method based on structured modeling and reinforcement learning according to claim 3, characterized in that, The step of obtaining attention weights involves performing relational aggregation on the task node representation vector and the executor node representation vector based on the attention weights to obtain multiple representation vectors, specifically including: Obtain attention weights, and perform relational aggregation on the task node representation vector and the executor node representation vector based on the attention weights to obtain an attention score; Obtain a normalization function, and normalize the attention score according to the normalization function to obtain the attention coefficient; The attention coefficients are transformed using a multi-head attention mechanism to obtain multiple representation vectors.

5. The task scheduling method based on structured modeling and reinforcement learning according to claim 4, characterized in that, The process of concatenating all representation vectors using the graph attention network and activation function to obtain a first vectorized representation, averaging the first vectorized representation to obtain a second vectorized representation, and then performing a residual concatenation between the first and second vectorized representations to obtain a final vectorized representation, specifically includes: The activation function is obtained, and all the representation vectors are concatenated using the intermediate layer of the graph attention network and the activation function to obtain a first vectorized representation. The first vectorized representation is then averaged using the output layer of the graph attention network and the activation function to obtain a second vectorized representation. ； in, This represents the second vectorized representation. This represents the activation function. This indicates the total number of attention heads. Indicates the attention coefficient. Represents a node The set of neighboring nodes, Indicates the first The weight matrix of each attention head. Represents the eigenvector; By performing a residual concatenation between the first vectorized representation and the second vectorized representation, we obtain the vectorized representation: ； in, Represents a vectorized representation. This represents the first vectorized representation. This represents the transformation function of the second-layer GAT.

6. The task scheduling method based on structured modeling and reinforcement learning according to claim 5, characterized in that, The scheduling and allocation result includes any one of the following: process local result, node local result, rack local result, and arbitrary local result; The step of scheduling tasks based on the vectorized representation using a scheduling decision model to obtain scheduling allocation results specifically includes: The initial allocation result is obtained by jointly deciding on the vectorized representation through a scheduling decision model; Obtain the constraints, and constrain the initial allocation result according to the constraints to obtain any one of the following: process local result, node local result, rack local result, and arbitrary local result.

7. The task scheduling method based on structured modeling and reinforcement learning according to claim 6, characterized in that, The step of scheduling tasks based on the vectorized representation using a scheduling decision model to obtain scheduling allocation results further includes: Based on the scheduling allocation result, determine the scheduling feedback information, perform multi-objective reward calculation on the scheduling feedback information, and obtain multiple reward values; All the aforementioned reward values ​​are weighted and calculated to obtain the target reward value, which is then stored in the trajectory buffer. ； in, Indicates the target reward value. Indicates execution time information. Indicates data locality level information. Indicates load distribution information, This indicates that the execution was successful. Indicates energy consumption information. This represents the first weight matrix. This represents the second weight matrix. This represents the third weight matrix. This represents the fourth weight matrix. This represents the fifth weight matrix; If the trajectory buffer reaches a preset value, the scheduling decision model is optimized and updated according to the target reward value to obtain the optimal scheduling decision model. The scheduling feedback information includes execution time information, data locality level information, load distribution information, execution success information, and energy consumption information.

8. A task scheduling system based on structured modeling and reinforcement learning, characterized in that, The task scheduling system based on structured modeling and reinforcement learning includes: The heterogeneous graph construction module is used to acquire resource status information and task information, and to perform topological connections between the resource status information and the task information to obtain a heterogeneous graph. The vector aggregation module is used to obtain attention weights, perform feature learning on the heterogeneous graph through a graph attention network to obtain an initial representation vector, and perform relation aggregation on the initial representation vector according to the attention weights to obtain multiple representation vectors. The vectorized representation concatenation module is used to concatenate all the representation vectors through the graph attention network and activation function to obtain a first vectorized representation, to average the first vectorized representation to obtain a second vectorized representation, and to perform residual concatenation between the first vectorized representation and the second vectorized representation to obtain a vectorized representation. The task scheduling module is used to schedule tasks based on the vectorized representation through a scheduling decision model to obtain scheduling allocation results.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a task scheduler based on structured modeling and reinforcement learning stored in the memory and executable on the processor. When the task scheduler based on structured modeling and reinforcement learning is executed by the processor, it implements the steps of the task scheduling method based on structured modeling and reinforcement learning as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a task scheduler based on structured modeling and reinforcement learning, which, when executed by a processor, implements the steps of the task scheduling method based on structured modeling and reinforcement learning as described in any one of claims 1-7.

Images