Multi-agent reinforcement learning satellite task scheduling method and system based on graph attention network

Abstract

This invention discloses a multi-agent reinforcement learning satellite mission scheduling method and system based on graph attention networks, belonging to the field of artificial intelligence and satellite resource scheduling technology. The method includes: constructing a dynamic task relationship graph, wherein the dynamic task relationship graph uses satellite missions as nodes and inter-task relationships as edges, and evolves dynamically with the scheduling process; extracting features from the dynamic task relationship graph through a graph attention network encoder to obtain node embedding vectors and global graph embedding vectors; incrementally updating the node embedding vectors and global graph embedding vectors based on a dynamic task embedding mechanism to adapt to changes in the scheduling environment; constructing a multi-agent reinforcement learning decision framework based on graph representation, wherein the framework contains multiple heterogeneous agents, and each agent makes collaborative decisions based on the updated embedding vectors; introducing a task priority propagation mechanism to achieve graph-level diffusion of priority signals through a propagation network to guide agent decisions; training and optimizing the graph attention network encoder and the multi-agent reinforcement learning decision framework using a phased course learning strategy to obtain a satellite scheduling scheme; explicitly modeling complex relationships and physical constraints between tasks through graph structures, adapting to environmental changes using a dynamic embedding mechanism, and achieving efficient multi-agent collaborative decision-making based on this, comprehensively improving the performance, robustness, and interpretability of satellite mission scheduling.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and satellite scheduling, specifically relating to a multi-agent reinforcement learning satellite mission scheduling method and system based on graph attention networks. Background Technology

[0002] Imaging satellites are crucial national space infrastructure supporting disaster relief, emergency monitoring, and military applications. Their observation range is unrestricted by geographical boundaries, offering rapid response and wide-area coverage. Agile satellites, in particular, possess attitude maneuvering capabilities in pitch, roll, and yaw compared to traditional non-agile satellites. They can actively adjust their attitude to conduct observations even before the satellite passes directly over the target, significantly extending the effective observation window and enhancing observation flexibility and opportunities. However, the attitude maneuvering characteristics of agile satellites also make their scheduling problem significantly time-dependent. Specifically, the transition time between tasks is no longer fixed but depends on the change in attitude angle between them, greatly increasing the complexity of scheduling optimization. The agile satellite scheduling problem has been proven to be an NP-hard combinatorial optimization problem. Its core objective is to allocate execution resources and time to a series of observation or data transmission tasks with different priorities, benefits, and constraints under strictly limited satellite resources, in order to maximize the overall observation benefits or mission completion value.

[0003] Existing scheduling methods mainly include rule-based heuristic algorithms and metaheuristic optimization algorithms. Although these methods perform reasonably well on small- to medium-sized problems, as the task scale increases and constraints become more complex, heuristic methods, while offering strong interpretability and fast computation, often struggle to guarantee the quality and stability of their solutions, making them unsuitable for low-level scheduling requirements. Metaheuristic methods, while providing high solution quality, also incur significant time overhead.

[0004] To address the aforementioned challenges in agile satellite scheduling, it is urgently necessary to introduce an intelligent decision-making framework that integrates graph neural networks and multi-agent reinforcement learning. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks. This method aims to explicitly model the complex relationships and physical constraints between tasks through graph structures, adapt to environmental changes using dynamic embedding mechanisms, and achieve efficient multi-agent collaborative decision-making, thereby comprehensively improving the performance, robustness, and interpretability of satellite mission scheduling.

[0006] To achieve the above objectives, firstly, a multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks is provided, comprising the following steps: A dynamic task relationship graph is constructed, which uses satellite tasks as nodes and inter-task relationships as edges, and evolves dynamically with the scheduling process. The dynamic task relationship graph is feature extracted by a graph attention network encoder to obtain node embedding vectors and global graph embedding vectors. The node embedding vector and global graph embedding vector are incrementally updated based on the dynamic task embedding mechanism to adapt to changes in the scheduling environment. A multi-agent reinforcement learning decision-making framework based on graph representation is constructed. The framework includes multiple heterogeneous agents, and each agent makes collaborative decisions based on the updated embedding vectors. A task priority propagation mechanism is introduced to achieve graph-level diffusion of priority signals through a propagation network, guiding agent decision-making; A phased learning strategy was adopted to train and optimize the graph attention network encoder and the multi-agent reinforcement learning decision framework to obtain a satellite scheduling scheme.

[0007] Furthermore, the node characteristics of the dynamic task relationship graph include task benefits, execution duration, attitude angle requirements, resource consumption rate, visible time window information, and dynamic scheduling status; the edge relationships include temporal dependency edges with weights equal to attitude transition time, spatiotemporal conflict edges representing time window conflicts, competitive edges representing resource competition, and logical dependency edges. Dynamic graph structure updates include dynamically adding nodes and relational edges when a new task arrives, updating node state characteristics and adjusting associated edges after task scheduling, and periodically re-evaluating edge weights.

[0008] Furthermore, the benefit attributes in the node characteristics include normalized priority or benefit value; execution requirements include execution duration; attitude requirements include three-dimensional attitude angle requirements, namely pitch angle, yaw angle, and roll angle; resource requirements include data generation rate and energy consumption rate; spatiotemporal constraints include earliest start time, latest completion time, and visible time window list; dynamic status includes task scheduling status, assigned satellite ID, and planned start time.

[0009] Furthermore, the processing steps of the graph attention network encoder include: The input layer projects the original node features onto the initial hidden space through a linear transformation; The multi-head graph attention layer aggregates neighbor node information through an attention mechanism, and the attention calculation incorporates edge weights; The outputs of each attention head are spliced ​​or averaged, and multiple layers of networks are stacked with residual connections and layer normalization. Global graph embedding vectors are generated through attention pooling to capture the overall scheduling situation; Furthermore, in the graph attention network encoder, the first... Layer, First Attention head to node Attention coefficient The calculations include: Calculate the raw attention score:

[0010] Normalization yields the attention coefficient:

[0011] in: , For nodes and In the Hidden feature vectors of the layer; For the first Learnable weight matrix for each attention head; For the first Attention vectors of attention heads; For connecting nodes and The weight of the edge; A learnable function that maps scalar edge weights to feature vectors; This is a vector concatenation operation; :node The set of neighboring nodes; For a linear rectified activation function with leakage, The index of the neighboring node being traversed.

[0012] Furthermore, dynamic task embedding mechanisms include: Events trigger local updates. When the state of a task node changes, a local recoding of its Δ-hop neighbor subgraph is triggered, updating only the node embeddings within the subgraph. Periodic edge weight reassessment: After each preset scheduling step, the weights of resource contention edges are recalculated based on the remaining satellite resources. Embedded state fusion is achieved by fusing the historical embeddings of nodes with the candidate embeddings obtained from local updates through a gated loop unit, thus maintaining temporal smoothness.

[0013] Furthermore, the heterogeneous agents in the multi-agent reinforcement learning decision-making framework include task selection agents and resource allocation agents; The observation information of the task selection agent includes the node embedding vector of unscheduled tasks, the global graph embedding vector, and the satellite resource status summary, and outputs the probability distribution of unscheduled tasks; The observation information of the resource allocation agent includes the node embedding vector of the selected task, the detailed status of each satellite, and the global graph embedding vector, and outputs the probability distribution on the satellite set.

[0014] Furthermore, the task priority propagation mechanism includes: Calculate the base priority for each task, which takes into account the task's own priority, time urgency, and resource requirements. The basic priority is used as the initial node signal and iteratively diffused through a propagation network that shares attention weights with the encoder; Integrate the propagated priorities into the value estimation of the task selection agent to guide the globally optimal decision-making. Task priority propagation is iteratively calculated using the following formula:

[0015] in, For the first The priority vector of the next iteration For the transmission rate, For the first The attention weight matrix of the layered graph attention network, and the priorities after propagation, are used to bias the Q-value function of the agent selection task.

[0016] Furthermore, the training process of the method adopts a phased strategy: the first phase uses graph contrast learning to pre-train the graph encoder; the second phase fixes the encoder and uses a multi-agent reinforcement learning algorithm to train the agent policy; the third phase performs joint fine-tuning of the encoder and policy network, and uses course learning to gradually progress from simple scenarios to complex scenarios.

[0017] Secondly, the present invention also provides a multi-agent reinforcement learning satellite mission scheduling system based on graph attention networks, comprising: The graph construction module is used to construct a dynamic task relationship graph, which uses satellite tasks as nodes and inter-task relationships as edges, and evolves dynamically with the scheduling process. The graph encoding module is used to extract features from the dynamic task relationship graph through a graph attention network encoder to obtain node embedding vectors and global graph embedding vectors. The embedding update module is used to incrementally update the node embedding vector and the global graph embedding vector based on the dynamic task embedding mechanism to adapt to changes in the scheduling environment. A multi-agent decision-making module is used to construct a graph-representation-based multi-agent reinforcement learning decision-making framework, which includes multiple heterogeneous agents, each of which makes collaborative decisions based on updated embedding vectors. The priority propagation module is used to introduce a task priority propagation mechanism, which realizes the graph-level diffusion of priority signals through a propagation network to guide the agent's decision-making. The training optimization module is used to train and optimize the graph attention network encoder and the multi-agent reinforcement learning decision framework using a phased course learning strategy to obtain a satellite scheduling scheme.

[0018] Compared with existing technologies, this invention has at least the following beneficial effects: First, this invention constructs a task relationship graph, where nodes represent tasks to be scheduled, and edges are used to characterize the spatiotemporal relationships, resource competition relationships, and logical dependencies between tasks. Based on this, a graph attention network encoder is designed, aggregating neighbor node information through a multi-head attention mechanism to generate task node embeddings and a global graph representation rich in contextual information. To address the dynamic nature of the scheduling environment, this invention introduces a dynamic task embedding mechanism, which can efficiently update the embeddings of relevant nodes according to real-time changes in task states without recalculating the entire graph. Finally, a multi-agent reinforcement learning framework based on graph representation is used to encode the graph... Using structural information as the perceptual input of the agents, this invention designs multiple specialized agents for task selection, resource allocation, and other tasks. Efficient collaboration among these agents is achieved through a graph-level attention coordination mechanism. An innovative task priority propagation mechanism is proposed, leveraging the inherent message passing capabilities of graph neural networks to propagate and dynamically adjust task priorities within the task relationship network, thereby guiding the agents to make better global scheduling decisions. This invention significantly enhances the model's understanding and representation of complex task relationships through graph structure modeling, adaptively captures the spatiotemporal correlations and transformation dependencies between tasks using a graph attention mechanism, and improves the algorithm's adaptability to real-time changing scheduling environments through its dynamic embedding mechanism. Experimental results show that, compared with traditional methods, the method described in this invention improves both scheduling success rate and average scheduling benefit, while also exhibiting better interpretability and generalization ability. Attached Figure Description

[0019] Figure 1 A schematic diagram of the method flow is provided for this invention.

[0020] Figure 2 This is a schematic diagram of the multilayer graph attention network encoder described in this invention. Detailed Implementation

[0021] The following section will provide a detailed description of the proposed multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks, in conjunction with the accompanying drawings and specific embodiments. It should be noted that the implementation of this invention is not limited to the specific scenarios and parameter settings described below; its core ideas and framework have broad applicability and scalability. To systematically verify the performance, robustness, and adaptability of this method, three typical embodiments with progressively increasing mission size, satellite resources, and constraint complexity will be provided. These embodiments simulate three types of real-world scheduling scenarios: small-scale rapid response, medium-scale routine observation, and large-scale emergency coordination. Furthermore, the design of these embodiments reflects the design philosophy of this method, which deeply embeds physical concepts from the actual satellite scheduling domain (such as attitude angles, visibility time windows, and onboard resources) into the core of the model. For example, in the graph structure construction of the model, node features directly encode the mission's attitude requirements and resource consumption, while edge weights accurately express the physical transition time between missions. This makes the learned strategy have clear physical interpretability and naturally satisfy complex engineering constraints.

[0022] refer to Figure 1 The construction of dynamic task relationships described in this invention includes the following steps: The satellite mission scheduling problem is formally modeled as a dynamic graph. ,in The graph represents the scheduling time, and its structure evolves as the scheduling process progresses.

[0023] 1.1 Node Modeling Each node Corresponding to a task Its eigenvectors Includes the following dimensions: Task identifier: Task ID; Profitability Attribute: Normalization Priority Or earnings value; Execution requirements: Execution duration ; Attitude requirements: 3D attitude angle requirements , respectively representing pitch, yaw, and roll angles; Resource requirements: Data generation rate (MB / s, used to calculate storage consumption), energy efficiency The unit is power per second; Spatiotemporal constraints: Earliest possible start time The latest completion time The list of visible time windows This indicates that it was controlled by different satellites. The visible time interval; Dynamic status: Whether it has been scheduled If already scheduled, record the assigned satellite ID. and the planned start time .

[0024] 1.2 Edge Modeling and Weight Calculation side and their weights Used to quantify relationships between tasks.

[0025] Temporal dependency edges (directed edges) ): Definition: If task It is possible in the mission If the same satellite then executes the same operation (within the allowed time window and with matching resource types), then this edge is established.

[0026] Weight : Directly defined from the task Switch to task Required attitude transition time The unit is seconds.

[0027] Spacetime conflict edge (undirected edge) ): Definition: If task and The visible time windows overlap, and the tasks and If it can be executed by the same satellite, then establish this side.

[0028] Weight Its weight can be defined as the normalized overlap, with a high weight indicating strong conflict.

[0029] in , Tasks and The set of visible time windows This is a function for calculating the maximum temporal overlap.

[0030] Resource competition edge (undirected edge) ): Definition: If task and If they compete for the same on-board resources (such as a specific type of satellite, high memory usage, high energy consumption), then this edge is established.

[0031] Weight Total data volume of the integrated task Total energy consumption And resource scarcity calculation. This edge encodes the effects of storage and energy constraints, and its weight can be represented as a normalized value of the product of resource requirements:

[0032] in This is the maximum storage capacity of the satellite.

[0033] Logical dependency edges (directed edges) If task Must be in the task If the previous step was completed, then this edge is created with a weight of 1.

[0034] 1.3 Dynamic updates to the graph structure: When a new task arrives, nodes and their relationships with existing nodes are dynamically added. After a task is scheduled, its node state characteristics are updated. The system removes or weakens associated conflicting and competing edges. Edge weights are periodically reassessed to reflect the latest scheduling status and resource availability.

[0035] The graph attention network encoder provided by this invention is as follows: Design a multi-layer graph attention network (GAT) as the encoder. ,in This is an adjacency matrix (containing edge weight information). The encoder maps the original task relation graph to a high-level node embedding. and global graph embedding .like Figure 2 As shown.

[0036] Input layer: Linear transformation converts the original features Projected onto the initial hidden space.

[0037]

[0038] Multi-head graph attention layer: Each layer aggregates neighbor information through an attention mechanism. For nodes... and neighbors :

[0039] Attention calculation considers not only node features It also explicitly incorporates edge weights. (e.g., conversion time) or degree of conflict ). Such as the degree of conflict Represents a node and nodes The edge properties of "mutual exclusion" or "interference" between them. This allows the model to directly adjust the importance of information transmission based on the strength of physical constraints. , For nodes and In the Hidden feature vectors of the layer; For the first Learnable weight matrix for each attention head; For the first Attention vectors of attention heads; For connecting nodes and The weight of the edge; A learnable function that maps scalar edge weights to feature vectors; This is a vector concatenation operation; :node The set of neighboring nodes; For a linear rectified activation function with leakage, The index of the neighboring node being traversed; Each attention head Output: .

[0040] Multi-head output splicing or averaging: .

[0041] Hierarchical stacking: Stacking Each layer uses residual connections and layer normalization for stable training.

[0042] Graph-level readout: Generating global graph embeddings using attention pooling Used to capture the overall scheduling situation and summarize information from the entire map:

[0043]

[0044] This invention incrementally updates the node embedding vector and the global graph embedding vector to adapt to changes in the scheduling environment, including the following steps: To achieve efficient online scheduling, an incremental update strategy is designed to avoid full graph recoding after each state change.

[0045] 3.1 Event-triggered partial update: When a certain task node... When its state changes (e.g., it is scheduled), triggering its... - Jump to neighbor's sub-graph Local recoding. Only for Within the nodes, a lightweight 1-2 layer frozen GAT is used for forward propagation to update their embeddings, while - The embedding of nodes outside the skip neighbor subgraph remains unchanged.

[0046] 3.2 Periodic edge weight reassessment: every time... Each scheduling step is based on the current remaining satellite resources (storage). Battery ), recalculate the weights of all resource contention edges, for example, update to To reflect changes in the degree of resource scarcity, This is for boundary handling to prevent division by zero.

[0047] 3.3 Embedded State Fusion: For the updated node, its historical embedding is fused using a gated recurrent unit (GRU). Candidate embeddings obtained from local updates :

[0048] To maintain the temporal smoothness of the embedding.

[0049] Build a containing A collaborative system of heterogeneous intelligent agents, each agent The strategy based on graph embedding includes the following steps: 4.1 Agent Roles and Observations: Task selection agent :observe ,in A summary of the resource status of each satellite.

[0050] Resource allocation intelligent agent :observe ,in For satellite The detailed status includes location, resources, and next idle time.

[0051] 4.2 Policy Networks and Actions: Task selection agent Output a probability distribution across all unscheduled nodes:

[0052]

[0053] Resource allocation intelligent agent Output a probability distribution over the satellite set, satisfying a hard constraint: Action Must meet and ,in For satellite The next free time.

[0054] 4.3 Coordination and Training: A centralized training distributed execution (CTDE) framework is adopted. During training, a centralized critic network is used. To assess joint actions The value of, among which It is the set of all currently embedded nodes. Each agent's policy is implemented using a policy gradient method to maximize the team's cumulative discount reward. Update to the target. Rewards Includes: the rewards for successfully scheduling a task Penalties for violating storage / energy constraints And rewards for efficient use of time.

[0055] Introducing a task priority propagation mechanism, which uses a propagation network to achieve graph-level diffusion of priority signals to guide agent decision-making, includes the following steps: By utilizing a trained graph attention network as a propagation network, graph-level diffusion of priority signals is achieved to implicitly optimize multi-objectives.

[0056] 5.1 Initial Priority Calculation: Calculate a basic priority for each task. :

[0057] in These are the weighting coefficients. As a parameter for the scale of time urgency, The latest start time, This is the satellite's maximum storage capacity. For the task The required storage space or amount of data For the task priority, It has the highest priority among all tasks.

[0058] 5.2 Priority Propagation Based on Graph Attention: Treating basic priorities as initial node signals, through... The propagation network, which shares attention weights with the encoder, performs iterative diffusion:

[0059] in It is the first Attention weight matrix of layer GAT ( ), It is the transmission rate ( After propagation, the final priority of the task... It takes into account its own value and the degree of its "influence" and "being influenced" in the entire task relationship network.

[0060] 5.3 Prioritization-Guided Decision Making: Integrating the propagated priorities into the value estimation of the task selection agent guides it to balance immediate gains and long-term impacts.

[0061] in It is an adjustable bias coefficient. This is equivalent to adding an intrinsic reward related to the global importance of the task to the reward function, prompting the agent to make scheduling decisions with a more global perspective and multi-objective trade-offs.

[0062] A phased learning strategy is employed to train and optimize the graph attention network encoder and the multi-agent reinforcement learning decision framework to obtain a satellite scheduling scheme, including the following steps: Phase 1: Graph Encoder Pre-training. Graph contrastive learning is used to enable the encoder to extract effective task relation representations from the graph structure.

[0063] 1) Collect a large number of task relationship graph samples in the training environment using the basic scheduling strategy. The task relationship graph is based on tasks as nodes and the spatiotemporal constraints and resource competition relationships between tasks as edges.

[0064] 2) For each collected image Two random data augmentation operations are applied to generate two related views. and Enhanced operations include: (1) Random masking of node features: Randomly masking some feature attributes of task nodes with a preset probability; (2) Randomly discard relational edges: Randomly delete some connecting edges in the graph with a preset probability; After the graph encoder After processing, the node embedding set is obtained:

[0065] in This represents the number of nodes (tasks) in the graph. and Representing nodes respectively Embedded vectors in both views.

[0066] 3) Calculate the contrastive learning loss by inputting the enhanced view to the graph encoder with shared parameters.

[0067] First, define a similarity measurement function between node embeddings. As an example, cosine similarity can be used:

[0068] in and For embedding vectors, Represents the vector dot product. express Norm.

[0069] Node-level contrast loss: Minimizes the difference in the embedding representation of the same node across different views, and maximizes the difference in the embedding representation of different nodes. Its node-level contrastive loss is defined as:

[0070] in: This is a temperature hyperparameter used to control the sharpness of the distribution. For indicator functions, when The value is 1 if the condition is met, and 0 otherwise; the denominator contains two types of negative samples: 1. Cross-view negative samples: Different nodes in another view ( ); 2. Negative samples from the same view: Different nodes in the same view ( ); For a batch There are 1000 image samples, each image has 10000 images. For each node, the overall node-level loss is the average of the losses of all nodes:

[0071] Graph-level contrast loss: Minimizes the global representation difference between different views of the same graph, and maximizes the global representation difference between different graphs. This is achieved through graph pooling functions. Node embeddings are aggregated into a graph-level representation. As an example, pooling methods include:

[0072] For a batch A sample image is generated. Each graph is represented by two views. The graph-level contrast loss is defined as:

[0073] The symmetric loss term The calculation is as follows:

[0074] In the formula: For the image A hierarchical representation of two views; For all in the batch A set of graph-level representations; Temperature parameters for graph-level comparative learning (may differ from node-level temperatures); Exclude positive samples from themselves as negative samples; The final pre-training loss is a weighted sum of node-level loss and graph-level loss:

[0075] in and To balance the weight hyperparameters, the typical range of values ​​is [value range missing]. And satisfy .

[0076] 4) Optimize encoder parameters through gradient descent to enable the encoder to extract stable and robust feature representations from the task graph.

[0077] Phase Two: Multi-Agent Policy Training. Algorithms such as MADDPG are employed, utilizing graph augmentation for experience replay. Details are as follows: 1. Network architecture configuration: Each agent is configured with an independent policy network (Actor) to receive local observation information processed by the encoder; Configure a centralized value assessment network (Critic) to receive joint observations and joint actions from all agents; 2. Graph-enhanced experience replay mechanism: During the interaction between the agent and the environment, it not only stores the conventional state-action-reward tuples. ; Simultaneously store a snapshot of the complete task relationship graph at the corresponding moment. t Forming enhanced experience tuples ; Establish a graph augmentation experience playback buffer to store the aforementioned experience data; 3. Training process: Batch empirical data is sampled from the buffer, and the graph data is processed using a fixed encoder. The embedded representation of the encoder output is input to the policy network and the value network. Then, the parameters of the policy network and the value network are updated using a multi-agent deep deterministic policy gradient algorithm. The graph encoder parameters remain frozen during this stage.

[0078] Phase 3: Joint Fine-Tuning. This phase unfreezes encoder parameters and performs end-to-end optimization of the entire system, enabling the task representation to adaptively adjust for specific scheduling objectives. 1) Unfreeze the graph encoder parameters after the second phase of training, making them trainable; 2) Based on the graph augmentation experience replay buffer established in the second stage, joint training is performed: sample batch experience data and update the parameters of the encoder, policy network and value network at the same time; set a low learning rate for the encoder to prevent catastrophic forgetting of pre-trained knowledge; 3) Adopt a progressive unfreezing strategy: prioritize unfreezing the last few layers of the encoder parameters, and gradually unfreeze more layers of parameters as training progresses.

[0079] Throughout the three-stage training process, course learning is used. Instead of starting with the most complex target scenario, a sequence of scenarios is designed to increase in difficulty. The model is trained sequentially at different difficulty levels, from stage one to stage three (or at least stage two and stage three).

[0080] 1. Curriculum Design Dimension: Task size: 100-500 tasks → 600-1000 tasks → 1000+ tasks.

[0081] Number of satellites / agents: 1 satellite → : 2-3 satellites working together → Constellation (multiple stars) in synergy.

[0082] Constraint complexity: Time window: from a generous window to a tight, overlapping window.

[0083] Resource constraints: from a single resource (such as time) to multiple resources (storage, power, data transmission bandwidth).

[0084] Task dependency: A chain of tasks that have sequential dependencies from independent tasks.

[0085] Environmental dynamism: From a static task list to the arrival of dynamic tasks (online scheduling). 2. Integrated training process course learning: 1) Initialization: Starting from the simplest level start.

[0086] 2) Training and Evaluation: At the current level, train the model in stages one, two, and three (or start from stage two, reusing the pre-trained encoder from the previous level). Periodically evaluate the model's performance on the validation set at this level (e.g., task completion rate, reward).

[0087] 3) Advancement criteria: When the model's performance at the current level reaches a preset mastery threshold (e.g., task completion rate > 95% or profit exceeds 80% of the baseline algorithm), it is advanced to the next more difficult level. .

[0088] Phase Four: Knowledge Transfer 1) Model parameter transfer: Use all network parameters (encoder, actor, critic) trained at the current level as the initial values ​​for training the next level.

[0089] 2) Experience buffer: It can be cleared or a portion of representative old experience can be retained.

[0090] 3) Repeat: Repeat stages 2-4 until the model is trained at the final target difficulty level and achieves satisfactory performance.

[0091] Based on the above-mentioned concept, this invention also provides a multi-agent reinforcement learning satellite mission scheduling system based on graph attention networks, comprising: The graph construction module is used to construct a dynamic task relationship graph, which uses satellite tasks as nodes and inter-task relationships as edges, and evolves dynamically with the scheduling process. The graph encoding module is used to extract features from the dynamic task relationship graph through a graph attention network encoder to obtain node embedding vectors and global graph embedding vectors. The embedding update module is used to incrementally update the node embedding vector and the global graph embedding vector based on the dynamic task embedding mechanism to adapt to changes in the scheduling environment. A multi-agent decision-making module is used to construct a graph-representation-based multi-agent reinforcement learning decision-making framework, which includes multiple heterogeneous agents, each of which makes collaborative decisions based on updated embedding vectors. The priority propagation module is used to introduce a task priority propagation mechanism, which realizes the graph-level diffusion of priority signals through a propagation network to guide the agent's decision-making. The training optimization module is used to train and optimize the graph attention network encoder and the multi-agent reinforcement learning decision framework using a phased course learning strategy to obtain a satellite scheduling scheme.

[0092] Specific Embodiment 1 based on the method described in this invention: Task scheduling for a single agile Earth observation satellite The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks in this embodiment includes the following steps: I. Constructing a decision variable model for the single-satellite observation and scheduling problem The objective function of the single time-dependent agile satellite scheduling model is to maximize the total revenue, and the task... The benefits are determined by priority. pi ∈[1,10] is determined, and the model established according to the above requirements is shown below:

[0093] The constraints are as follows: Uniqueness constraint: Each task is executed at most once.

[0094] Time window validity: The task must be executed within the valid time window.

[0095] Dynamic memory constraints: Memory usage during execution does not exceed the upper limit. M

[0096] Dynamic power constraint: Total power consumption shall not exceed the satellite's maximum power limit. E

[0097] II. Establishing an observation task model Set of Observable Targets Each task contains the following attributes: Spatiotemporal constraint parameters: Target geographic coordinates: longitude ,latitude ; Visible time window set: ; Each time window corresponds to a different observation angle and quality factor; Task requirement parameters: Execution time: Second Data volume: GB Priority:

[0098] III. Scene Setting Satellite configuration: 1 agile observation satellite with three-axis attitude maneuverability; Mission scale: 400 Earth observation missions; Scheduling period: 24-hour track cycle; Constraints: attitude maneuver time, storage capacity, energy constraints, time window; IV. Construction of Graph Structure Model Node (task) feature vector (12 dimensions): task reward, execution time, target longitude, target latitude, roll angle requirement, pitch angle requirement, yaw angle requirement, data volume (MB), power consumption (kW), time window start, time window end, priority.

[0099] Edge relationship construction: 1. Temporal dependency edges: Establish directed edges between pairs of tasks that may be executed consecutively; Weight calculation: =

[0100] in Let be the change in attitude angle, and be Time interval. Adjust the weighting coefficients for the attitude. This is the weighting coefficient for the time interval.

[0101] 2. Resource contention edge: An undirected edge is established when the sum of the task storage requirements exceeds the satellite storage capacity; Weight calculation: =

[0102] In the formula: , :Task and The amount of data generated, in GB; Total satellite storage capacity, in GB; when The combined data volume of the two tasks exceeds the satellite's capacity, so they cannot be scheduled simultaneously. when Strong competitive relationship; placing them together carries high risk. when Weak competitive relationships are easier to coordinate. V. Agent Design and Training Single agent architecture: Graph encoder: 2-layer GAT, outputting 64-dimensional node embeddings; Policy network: 3-layer MLP (64→32→1), outputs task selection probability; Training strategy: Course learning phase: 1) Initial stage: 50 tasks, wide time window (±30 minutes), no storage constraints; 2) Intermediate stage: 200 tasks, normal time window (±15 minutes), medium storage constraints; 3) Advanced stage: 400 tasks, strict time window (±5 minutes), complete constraints; Each training phase consists of 10,000 steps, and once the performance target is met, the next phase begins.

[0103] Training phase: To ensure a rigorous and unbiased evaluation of the proposed method, the dataset was divided into three mutually exclusive subsets: a training set, a validation set, and a test set. Each subset consists of 15 independently generated scheduling scenarios, with the number of observation tasks in each instance ranging from 50 to 400 in increments of 25. While the structural features of the three subsets are consistent, the specific content of each instance is unique, ensuring no overlap in scheduling problems between training, validation, and testing. The three subsets are constructed as follows: Training set: 15 scenarios, task numbers from 50 to 400, used for policy learning and parameter optimization.

[0104] Validation set: 15 scenarios with the same task number distribution, used for hyperparameter tuning and early stopping.

[0105] Test set: 15 scenarios with the same task number distribution, specifically used for final performance evaluation and comparison with baseline methods.

[0106] The training process follows a strict three-phase protocol: training, validation, and testing. The model iteratively optimizes its parameters using the training set, and its performance is periodically evaluated during validation based on a cumulative reward setting. An early stopping strategy is employed: if the validation reward does not improve performance for more than five consecutive evaluations (i.e., patience parameter 5), training terminates, and the model retains the parameters corresponding to the highest validation reward. Finally, the selected model is evaluated on an independent test set to assess its generalization ability. This process ensures effective learning, prevents overfitting, and provides a robust and fair evaluation of the model's performance.

[0107] VI. Output Scheduling Results Once the maximum number of iterations is reached, the maximum return on each task size is output. This completes the multi-agent reinforcement learning-based satellite task scheduling using a graph attention network on a single satellite.

[0108] Example 2: Mission Scheduling for Multiple Agile Earth Observation Satellites The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks in this embodiment consists of the following steps: I. Establishing a satellite model Agile Observation Satellite Collection The number of satellites is Each satellite has a different payload configuration and maneuverability.

[0109] II. Establishing an observation task model Set of Observable Targets Each task contains the same attributes as in Example 1.

[0110] III. Establishing a decision variable model for the multi-satellite observation and scheduling problem The decision variables for the multi-star cooperative scheduling problem are .in, For binary decision variables: when the task Within the visible time window During internal execution, Otherwise, it is 0. Decision variable This indicates a task. In the window The specific start time of execution within the specified timeframe. Based on the above definition, the mathematical model for this problem is constructed as follows: As the objective function of the multi-star cooperative scheduling problem, it aims to maximize the total benefit of all scheduled tasks:

[0111] The following specifies a unique constraint for task execution, ensuring that each observation task is assigned to be executed at most once:

[0112] The visible time window constraint requires that the execution period of a task must be completely within a certain visible time window to ensure the effectiveness of the observation operation.

[0113]

[0114] The binary variables defined in the following formula Used to represent the temporal relationship between tasks; a value of 1 indicates a task. It is a task Direct precursor missions on the same satellite.

[0115]

[0116] The regulations specify transition time constraints between missions on the same satellite, requiring that the time interval between adjacent missions must meet the needs of satellite attitude adjustment. This constraint is one of the most critical characteristics of agile satellite scheduling, reflecting the time dependency in mission sequence arrangement.

[0117]

[0118] The following equations together constitute the calculation model for attitude transition time, where ~ This represents the attitude maneuver angular velocity in different angular ranges. This represents the total difference in attitude angles between roll, pitch, and yaw angles between missions.

[0119]

[0120]

[0121] in ~ This represents the attitude maneuver angular velocity in different angular ranges. This represents the total difference in attitude angles (roll, pitch, and yaw) between missions. For satellite At time The roll angle, For satellite At time The roll angle, For satellite At time pitch angle, For satellite At time pitch angle, For satellite At time Yaw angle, For satellite At time Yaw angle.

[0122] Image quality constraints are specified, requiring that the actual imaging quality of the mission must meet the minimum quality standard set by the user. This quality level is determined by the start time of the mission's observations. Observations that fail to meet the quality requirements will be considered invalid, resulting in a waste of satellite resources.

[0123]

[0124] IV. Establishing a large-scale multi-star scheduling graph model Establish a relationship diagram comprising 6 heterogeneous agile observation satellites and 1000 observation missions: Node set: Among them, task nodes Satellite nodes .

[0125] Task node feature vector (6 dimensions):

[0126] Total number of tasks; Number of uncompleted tasks; Priority average; : Mean of return weights; Number of available ground stations; : Estimated download time.

[0127] Satellite node feature vector (5-dimensional):

[0128] Average storage capacity; Observation / download average rate; Average resource utilization rate; Average number of tasks in the queue.

[0129] V. Constructing Extended Multi-Layer Graph Edge Relationships Spatiotemporal relation edge: for task pairs If satellites exist Make the task and tasks If the time windows overlap, then create a spatiotemporal relationship edge. Its weight is calculated as follows:

[0130] in, and Representing tasks respectively and tasks In satellite The time window on; Indicates the degree of overlap of time windows; and Representing tasks respectively and tasks The length of the time window. This represents the set of all satellites.

[0131] Resource contention edge: For task pairs If they have overlapping time windows on the same satellite and have the same resource requirements, then a resource contention edge is created. Its weight is calculated as follows:

[0132] in, and Representing tasks respectively and tasks The resource demand vector; The similarity of resource demand vectors; This is the balance coefficient, with a value of 0.6.

[0133] Dependency edges: For task pairs If a dependency exists (i.e., task) Must be in the task If previously completed, then a directed dependency edge will be created. Its weight is:

[0134] Similarity edges: for task pairs Calculate their feature similarity; if the similarity exceeds a set threshold... Then create similarity edges. Its weight is calculated as follows:

[0135] in, and Indicates task and tasks eigenvectors; The similarity of feature vectors is expressed using the reciprocal of the Euclidean distance.

[0136] VI. Multi-agent decision-making based on graph attention networks 1. Graph Augmentation State Construction Task node embedding: Each task node is encoded using a graph attention network (GAT) to generate a 64-dimensional embedding vector. To capture the complex relationships between tasks:

[0137] in, For the task The 12-dimensional original features, It is a task The set of neighboring nodes, It is a set of edges, including spatiotemporal relationship edges, resource competition edges, dependency relationship edges, and similarity edges.

[0138] Graph-level global representation: Utilizing the attention pooling mechanism, the embedding vectors of all task nodes are... Aggregated into a 64-dimensional graph-level representation The calculation method is as follows:

[0139] in, This represents the set of all task nodes. Attention pooling aggregates the embeddings of each node using weighted aggregation.

[0140] State fusion: By fusing graph-level representations with satellite state statistics, global information, decision history, and hierarchical information, the state input of the agent is constructed. :

[0141] in, It is a 64-dimensional graph-level representation, with other information including 12-dimensional satellite state features, 10-dimensional global information, 10-dimensional decision history, and 6-dimensional hierarchical information, resulting in a final state dimension of 102 dimensions. 2. Global Scheduling Agent Input State: 102-dimensional graph-enhanced state, including: Graph-level representation (64 dimensions): describing the global relationships between tasks; Satellite state statistical features (12 dimensions): such as mean, standard deviation, etc.; Global information (10 dimensions): such as the overall state of task scheduling; Decision history (10 dimensions): historical decisions and strategies; Hierarchical information (6 dimensions): hierarchical coordination information.

[0142] Action Space: An 8-dimensional continuous action space with a dual-head output structure: The first 4 dimensions: task allocation strategy selection (such as load balancing, time windows, etc.). The last 4 dimensions: metaheuristic algorithm selection (such as genetic algorithm, simulated annealing, etc.).

[0143] Actor network architecture: Graph-enhanced feature extraction: Graph-level branch: 64-dimensional → 128-dimensional (graph feature extraction); Statistical feature branch: 38-dimensional → 64-dimensional (traditional feature extraction); Feature fusion: 128-dimensional + 64-dimensional → 256-dimensional → 128-dimensional; Attention layer: Multi-head attention mechanism (8 attention heads, embed_dim=128) Dual-head output: Allocation strategy head: 128-dimensional → 32-dimensional → 4-dimensional (task allocation strategy probability distribution); Algorithm selection head: 128-dimensional → 32-dimensional → 4-dimensional (algorithm selection probability distribution).

[0144] 3. Locally optimized intelligent agents Input state: Shares the same 102-dimensional graph augmentation state as the global scheduling agent.

[0145] Action Space: An 8-dimensional continuous action space with a dual-head output structure: First 4 dimensions: Selection of disruptive operators (e.g., random, minimum reward, etc.). Last 4 dimensions: Selection of repair operators (e.g., greedy, earliest, etc.).

[0146] Actor network architecture: Graph-enhanced feature extraction: Graph-level branch: 64-dimensional → 128-dimensional (graph feature extraction); Statistical feature branch: 38-dimensional → 64-dimensional (traditional feature extraction); Feature fusion: 128-dimensional + 64-dimensional → 256-dimensional → 128-dimensional.

[0147] Attention layer: Multi-head attention mechanism (8 attention heads, embed_dim=128) Dual-head output: Disrupt operator head: 128-dimensional → 32-dimensional → 4-dimensional (disrupts operator probability distribution); Repair operator head: 128-dimensional → 32-dimensional → 4-dimensional (repairs operator probability distribution).

[0148] 4. Centralized Critic Network Input: A concatenation of graph-augmented states and actions of all agents, with the following dimensions:

[0149] Network architecture: Graph feature extraction: Graph level representation: 64-dimensional × 2 (two agents) → 128-dimensional; Graph attention layer: Multi-head attention mechanism (4 attention heads, embed_dim=128); Traditional feature extraction: Statistical features: 76-dimensional × 2 (two agents) → 152-dimensional; Action characteristics: Action splicing: 8-dimensional × 2 (two agents) → 16-dimensional.

[0150] Feature fusion: Fusion layer: 128-dimensional + 152-dimensional + 16-dimensional → 256-dimensional (LayerNorm); Hidden layer: 256-dimensional → 128-dimensional (LayerNorm); Attention layer: Multi-head attention mechanism (4 attention heads, embed_dim=128).

[0151] Output layer: 128 dimensions → 1 dimension (Q value).

[0152] 5. Graph-Agent Coordination Mechanism Graph-level information sharing: The global scheduling agent and the local optimization agent share the same graph-level representation, thereby enabling coordinated decision-making.

[0153] Task relationship perception and decision-making: The intelligent agent can identify the spatiotemporal relationships and resource competition between tasks, and make more reasonable scheduling decisions.

[0154] Dynamic graph update feedback: The agent's decision affects the task scheduling state, and the task relationship graph is updated accordingly, forming a closed-loop feedback mechanism: decision → graph update → new decision.

[0155] Layered diagram coordination: Global scheduling agent: Makes high-level decisions based on graph-level representations, focusing on global task relationships. Local optimization agent: Performs low-level execution based on graph-level representations, focusing on local task relationships. Graph attention mechanism: Through graph attention, the two agents can coordinate and handle task relationships at different levels.

[0156] Attention weight interpretability: The weights of the graph attention mechanism reveal the task relationships that the agent focuses on when making decisions, increasing the interpretability of the decisions.

[0157] VII. Training Process 1. Data preparation: Task generation: First, generate 100 scheduling scenarios of different scales, with the number of tasks ranging from 100 to 1000, forming scheduling task environments of different scales.

[0158] Task Relationship Graph: A task relationship graph is constructed for each scheduling scenario. Nodes in the graph represent tasks, and edges represent the dependencies between tasks.

[0159] 2. Graph encoder pre-training: Objective: To pre-train a graph encoder using a graph contrastive learning method.

[0160] Positive sample generation: Positive samples consist of different views under the same scheduling state, which are generated through data augmentation.

[0161] Negative sample generation: Negative samples are generated from task graphs under different scheduling states.

[0162] Training settings: Train for 100 epochs with a learning rate of 0.001 to optimize the encoder's ability to represent the task graph, enabling it to extract effective information from different views.

[0163] 3. Agent training: Algorithm selection: The MADDPG (Multi-Agent Deep Deterministic Policy Gradient) algorithm is used for agent training.

[0164] Experience replay: Set up an experience replay buffer to store up to 50,000 experiences for sample extraction during the training process.

[0165] Batch size: 64 samples are used for each training iteration.

[0166] Learning rate settings: Actor (policy network) learning rate: 0.0001; Critic (value network) learning rate: 0.0005.

[0167] Training cycle: 500 episodes are trained, and the agent learns how to perform scheduling effectively in multiple scheduling scenarios.

[0168] 4. Joint fine-tuning: Fixed graph encoder: In this stage, the graph encoder remains fixed and does not participate in training. The main focus is on fine-tuning the network part of the agent.

[0169] Training cycle: 200 episodes of training.

[0170] Learning rate adjustment: The learning rate is adjusted to 0.1 of the original learning rate to help the agent gradually stabilize and converge in the subsequent training.

[0171] 5. Course Learning: Phase division: Based on the size of the task, the training is divided into three phases.

[0172] Phase 1: The number of tasks is between 100 and 300, and 200 episodes are trained. The agent starts learning from smaller-scale tasks.

[0173] Phase 2: The number of tasks is between 300 and 600. Train 300 episodes and gradually increase the task scale to examine the performance of the agent in more complex scenarios.

[0174] Phase 3: The number of tasks is between 600 and 1000, and 400 episodes are trained. The agent will face the most complex scheduling scenarios and undergo final capability verification and improvement.

[0175] The above training process, through pre-training of the graph encoder and step-by-step training of the agent, gradually improves the ability to automate scheduling tasks, and through course learning, gradually enhances the ability to handle larger-scale tasks.

[0176] This invention addresses key issues in existing satellite mission scheduling technologies, such as limited state representation capabilities, insufficient modeling of complex inter-task relationships, poor adaptability to dynamic environments, and low efficiency in multi-agent coordination. The core of this method lies in modeling the satellite mission scheduling problem as a graph structure, dynamically learning the embedded representations of tasks using graph attention networks, and designing a complete dynamic update and multi-agent collaborative decision-making framework. This method achieves explicit modeling of complex constraints by constructing a dynamic task relationship graph, adaptively capturing the spatiotemporal correlations and transformation dependencies between tasks using a graph attention mechanism, and constructing a heterogeneous agent collaborative system to achieve distributed decision optimization. Through environment-feedback-driven policy iteration and dynamic graph structure updates, this framework can intelligently balance task gains and resource consumption, effectively avoid local optima, and exhibit superior generalization performance and decision quality when facing large-scale dynamic scheduling scenarios.

[0177] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks, characterized in that, Includes the following steps: A dynamic task relationship graph is constructed, which uses satellite tasks as nodes and inter-task relationships as edges, and evolves dynamically with the scheduling process. The dynamic task relationship graph is feature extracted by a graph attention network encoder to obtain node embedding vectors and global graph embedding vectors. The node embedding vector and global graph embedding vector are incrementally updated based on the dynamic task embedding mechanism to adapt to changes in the scheduling environment. A multi-agent reinforcement learning decision-making framework based on graph representation is constructed. The framework includes multiple heterogeneous agents, and each agent makes collaborative decisions based on the updated embedding vectors. A task priority propagation mechanism is introduced to achieve graph-level diffusion of priority signals through a propagation network, guiding agent decision-making; A phased learning strategy was adopted to train and optimize the graph attention network encoder and the multi-agent reinforcement learning decision framework to obtain a satellite scheduling scheme.

2. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, The node characteristics of the dynamic task relationship graph include task benefits, execution time, attitude angle requirements, resource consumption rate, visible time window information, and dynamic scheduling status; the edge relationships include temporal dependency edges with weights equal to attitude transition time, spatiotemporal conflict edges representing time window conflicts, competitive edges representing resource competition, and logical dependency edges. Dynamic graph structure updates include dynamically adding nodes and relational edges when a new task arrives, updating node state characteristics and adjusting associated edges after task scheduling, and periodically re-evaluating edge weights.

3. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 2, characterized in that, The revenue attributes in node features include normalized priority or revenue value; execution requirements include execution duration. Attitude requirements include three-dimensional attitude angle requirements, namely pitch angle, yaw angle, and roll angle; resource requirements include data generation rate and energy consumption rate; spatiotemporal constraints include the earliest start time, the latest completion time, and the list of visible time windows; dynamic status includes mission scheduling status, assigned satellite ID, and planned start time.

4. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, The processing steps of a graph attention network encoder include: The input layer projects the original node features onto the initial hidden space through a linear transformation; The multi-head graph attention layer aggregates neighbor node information through an attention mechanism, and the attention calculation incorporates edge weights; The outputs of each attention head are spliced ​​or averaged, and multiple layers of networks are stacked with residual connections and layer normalization. Global graph embedding vectors are generated by attention pooling to capture the overall scheduling situation.

5. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, In the graph attention network encoder, the first... Layer, First Attention head to node Attention coefficient The calculations include: Calculate the raw attention score: Normalization yields the attention coefficient: in: , For nodes and In the Hidden feature vectors of the layer; For the first Learnable weight matrix for each attention head; For the first Attention vectors of attention heads; For connecting nodes and The weight of the edge; A learnable function that maps scalar edge weights to feature vectors; This is a vector concatenation operation; :node The set of neighboring nodes; For a linear rectified activation function with leakage, The index of the neighboring node being traversed.

6. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, Dynamic task embedding mechanisms include: Events trigger local updates. When the state of a task node changes, a local recoding of its Δ-hop neighbor subgraph is triggered, updating only the node embeddings within the subgraph. Periodic edge weight reassessment: After each preset scheduling step, the weights of resource contention edges are recalculated based on the remaining satellite resources. Embedded state fusion is achieved by fusing the historical embeddings of nodes with the candidate embeddings obtained from local updates through a gated loop unit, thus maintaining temporal smoothness.

7. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, The heterogeneous agents in the multi-agent reinforcement learning decision-making framework include task selection agents and resource allocation agents. The observation information of the task selection agent includes the node embedding vector of unscheduled tasks, the global graph embedding vector, and the satellite resource status summary, and outputs the probability distribution of unscheduled tasks; The observation information of the resource allocation agent includes the node embedding vector of the selected task, the detailed status of each satellite, and the global graph embedding vector, and outputs the probability distribution on the satellite set.

8. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, Task priority propagation mechanisms include: Calculate the base priority for each task, which takes into account the task's own priority, time urgency, and resource requirements. The basic priority is used as the initial node signal and iteratively diffused through a propagation network that shares attention weights with the encoder; Integrate the propagated priorities into the value estimation of the task selection agent to guide the globally optimal decision-making. Task priority propagation is iteratively calculated using the following formula: in, For the first The priority vector of the next iteration For the transmission rate, For the first The attention weight matrix of the layered graph attention network, and the priorities after propagation, are used to bias the Q-value function of the agent selection task.

9. The multi-agent reinforcement learning satellite mission scheduling method based on graph attention networks according to claim 1, characterized in that, The training process of the method adopts a phased strategy: the first phase uses graph contrast learning to pre-train the graph encoder; the second phase fixes the encoder and uses a multi-agent reinforcement learning algorithm to train the agent policy. The third stage involves joint fine-tuning of the encoder and policy network, and uses course learning to gradually progress from simple to complex scenarios.

10. A multi-agent reinforcement learning satellite mission scheduling system based on graph attention networks, characterized in that, include: The graph construction module is used to construct a dynamic task relationship graph, which uses satellite tasks as nodes and inter-task relationships as edges, and evolves dynamically with the scheduling process. The graph encoding module is used to extract features from the dynamic task relationship graph through a graph attention network encoder to obtain node embedding vectors and global graph embedding vectors. The embedding update module is used to incrementally update the node embedding vector and the global graph embedding vector based on the dynamic task embedding mechanism to adapt to changes in the scheduling environment. A multi-agent decision-making module is used to construct a graph-representation-based multi-agent reinforcement learning decision-making framework, which includes multiple heterogeneous agents, each of which makes collaborative decisions based on updated embedding vectors. The priority propagation module is used to introduce a task priority propagation mechanism, which realizes the graph-level diffusion of priority signals through a propagation network to guide the agent's decision-making. The training optimization module is used to train and optimize the graph attention network encoder and the multi-agent reinforcement learning decision framework using a phased course learning strategy to obtain a satellite scheduling scheme.

Images