Multi-satellite cooperative computing and routing joint optimization method for low-orbit satellite network

Abstract

The invention relates to a multi-satellite cooperative computing and routing joint optimization method for a low-orbit satellite network, and belongs to the technical field of satellite communication. The method comprises the following steps: constructing a low earth orbit satellite network topology model; the satellite updates the satellite resource state representation through the time memory map attention network; establishing a multi-satellite cooperative computing and routing joint optimization model of the low-orbit satellite network, specifically, comprehensively considering storage resources, transmission queue states, computing queue states and task characteristics of satellites, and establishing an optimization model with the goal of minimizing the total time delay of earth observation tasks; and solving the optimization problem by adopting a deep reinforcement learning method, specifically, realizing off-line training of a deep reinforcement learning model based on satellite network data provided by a terrestrial digital twin system until algorithm convergence, and periodically sending a strategy obtained by training to an on-orbit satellite. According to the invention, support can be provided for timely and reliable transmission and processing of massive earth observation tasks.

Description

Technical Field

[0001] This invention belongs to the field of satellite communication technology and relates to a method for multi-satellite collaborative computing and routing joint optimization in low-Earth orbit satellite networks. Background Technology

[0002] With the continuous advancement of Earth observation and remote sensing technologies, Low Earth Orbit (LEO) satellite networks play a crucial role in important application areas such as environmental monitoring, agriculture, and disaster response. To achieve high-frequency, high-resolution Earth observation, commercial space companies such as Starlink and OneWeb have deployed LEO constellations consisting of thousands of satellites. Meanwhile, improvements in hyperspectral imaging (HSI) sensing technology have significantly increased observation accuracy, but also led to a dramatic increase in the amount of observation data. Currently, each observation satellite generates approximately 1TB of data per day, which typically needs to be transmitted to ground stations in its raw form for processing, consuming a significant amount of satellite bandwidth. However, due to the limited quality of satellite-to-ground links, traditional task offloading schemes relying on centralized processing at ground stations are insufficient to meet the high real-time and reliability requirements of observation missions, gradually becoming a bottleneck for the further development of Earth observation services. With the enhanced onboard computing capabilities of LEO satellite networks, utilizing satellite computing resources to process observation data in real time during mission routing offers a potential solution to overcome this development bottleneck.

[0003] The computational processing of observation tasks is typically complex. For example, image recognition often involves multiple stages such as image preprocessing, feature extraction, and target recognition. Individual satellite nodes, constrained by payload and energy limitations, lack sufficient onboard computing resources to handle the demands of massive observation tasks. Furthermore, the arrival patterns of tasks, satellite network topology, and onboard resources are all highly dynamic. Therefore, in routing observation tasks to ground stations via the LEO satellite network, it is necessary to jointly design the computational offloading location for each subtask and the routing path through the satellite network to achieve multi-satellite collaborative scheduling with "simultaneous transmission and computation." In low-Earth orbit (LEO) satellite networks characterized by rapid topology changes and frequent inter-satellite link switching, the prerequisite for multi-satellite collaborative computation and joint routing scheduling is the ability to timely and accurately perceive the topology and onboard resource status of the LEO satellite network. Graph Neural Networks (GNNs), a deep learning method based on graph structures, can utilize the topological relationships of satellites to extract resource and link status features from local nodes and their neighboring nodes, thus showing potential application in onboard resource perception and routing decisions. However, in large-scale LEO constellations, due to high node degrees and frequent topology changes, traditional multi-layer message-passing-based GNNs require multiple rounds of propagation to capture multi-hop information, which can easily lead to oversmoothing effects. This causes the features of multi-order neighbors to gradually become consistent during propagation, resulting in a decrease in the discriminative power of resource status representation. Therefore, existing GNN methods can usually only effectively perceive neighbor information within a limited range, making it difficult to accurately represent resource status within a multi-hop range, thus affecting the decision-making quality of multi-satellite collaborative computing and routing strategies. Furthermore, existing research lacks a systematic joint optimization design for multi-satellite collaborative computing and routing, making it still a significant challenge to simultaneously meet the computational and transmission requirements of large-scale observation missions in low-Earth orbit satellite networks. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for multi-satellite collaborative computing and routing joint optimization in low-Earth orbit satellite networks, which supports the timely and reliable transmission and processing needs of massive Earth observation missions.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for multi-satellite collaborative computing and joint route optimization in low-Earth orbit satellite networks specifically includes the following steps: S1: Construct a low-Earth orbit satellite network topology model. Specifically, the continuous time is discretized into time slots. The satellite network topology remains relatively stable within each time slot, mainly including satellite nodes, inter-satellite links, and the network topology state that changes over time.

[0006] S2: The satellite updates its resource status representation through a temporal memory graph attention network. Specifically, in the temporal memory graph attention network, each satellite maintains a multi-hop resource cache vector on-board based on its local resource status feature vector. By exchanging caches with one-hop neighbors, the resource status information gradually spreads outward in the time dimension, thereby gradually forming a resource status view of the multi-hop neighborhood. In each time slot, the current local resource status representation, the resource status representation of one-hop neighbors, and the historical multi-hop neighborhood resource information represented by the cache are used as three types of information sources. A multi-hop attention mechanism is introduced to weight and aggregate the three types of information, and the residual term of the local resource status representation is superimposed on the aggregation result to obtain the updated satellite resource status representation, thereby realizing multi-hop perception of on-board resources.

[0007] S3: Establish a multi-satellite collaborative computing and routing joint optimization model for low-Earth orbit satellite networks. Specifically, this involves comprehensively considering satellite storage resources, transmission queue status, computing queue status, and mission characteristics (such as data volume, computing power requirements, and mission type) to establish an optimization model with the goal of minimizing the total latency of Earth observation missions.

[0008] S4: The optimization problem in step S3 is solved using a deep reinforcement learning method. Specifically, the deep reinforcement learning model is trained offline based on satellite network data provided by the terrestrial digital twin system until the algorithm converges, and the trained strategy is periodically sent to the satellite in orbit.

[0009] Furthermore, step S1 specifically includes: S11: Usage Time Extension Chart To represent the dynamic network topology of low-Earth orbit satellite networks; where, , Represents the first in the system One satellite; Represents the set of all inter-satellite links, satellite , Links between them are connected by edges. express; Representing the time domain, in each time slot Internally, it updates link status and other parameters based on satellite orbital parameters, thus establishing the satellite network topology. Remain unchanged; S12: For One of the satellites in the constellation at that time Its resource status characteristics are represented by a vector. Indicates; among which, express Time Satellite The remaining storage space; Indicates the satellite at time t The length of the computation queue represents the computational resource load. express Time Satellite The transmission queue length represents the transmission resource load. S13: Regarding At this moment, satellite To satellite inter-satellite links Its state can be represented by a tuple. Indicates; among which, This indicates the link rate, which can be detected in real time by onboard equipment. The current link distance can be calculated using satellite orbit parameters; Indicates the link from the satellite to The current transmission queue length can be obtained from the link buffer statistics.

[0010] Furthermore, step S2 specifically includes: S21: For time... satellite nodes below Its resource status feature vector is defined by step S12 as To improve the representation capability of resource state features and facilitate subsequent attention mechanisms and message aggregation processing, the three-dimensional resource state feature vector is... Mapped to In the feature space, we obtain the local embedding vector; let... Let be the characteristic transformation matrix, where Given a preset positive integer feature dimension, the local embedding vector is defined as follows: (1) Where, vector For satellite At any moment of Dimensional resource embedding representation, The elements are updated during training via gradient descent to transform the original resource state into a high-dimensional representation suitable for subsequent attention and aggregation operations. In addition, in each satellite Maintain one Dimensional hop resource cache vector It is used to store the state information of multi-hop neighborhood resources obtained through hop-by-hop diffusion from neighbors; during system initialization, the cache vector is set to zero, i.e. ; S22: In each time slot The satellite performs a multi-hop cache update process; the satellite Set the current cache vector Send to all its one-hop neighbors Simultaneously receive neighbor cache vectors Subsequently, the cache itself and its neighbors' caches are averaged and aggregated to obtain the intermediate cache vector. : (2) in, Indicates satellite The set of direct neighbors, For satellite The number of direct neighbors; Next, the local embedding vector at the current time will be... Write to the cache to obtain the cache for the next time slot: (3) in, This is a forgetting factor used to strike a balance between emphasizing the current state of local resources and preserving historical multi-hop information; as time slots progress, Will contain The system provides statistical information on the resource status of multiple satellites within a central hop neighborhood at multiple historical moments. This information is propagated spatially through hop-by-hop exchange of neighbor caches and multi-hop fusion is achieved through time series analysis. S23: In time slot During the resource sensing phase, satellite With local embedding vectors One-hop neighbor embedding and multi-hop cache vector Based on this, and to facilitate a unified representation of resource information from different sources, the information source categories are numbered as follows: Construct three types of information sources: This represents the current local resource information of the satellite, and the formula is as follows: (4) This represents the satellite's one-hop neighbor resource information at the current moment, and the formula is as follows: (5) , indicating that it is a cache vector Representation of historical multi-hop neighborhood resource information: (6) To extract useful features for subsequent routing and computational decisions from different information sources and map them to a unified latent space, avoiding attention weight bias caused by differences in the scale of the original features, a linear transformation is performed on the three information sources respectively; specifically, for each hop count... Set the feature transformation matrix ,definition: (7) in, In order to target the Intermediate feature representation extracted from skip information sources; matrix These are learnable parameters that are automatically adjusted during training via gradient descent. They are used to convert resource information with different hop counts into feature representations suitable for attention mechanisms. S24: To adaptively characterize three types of hop count information for satellites The importance of current resource awareness is highlighted, leading to the introduction of an attention mechanism; firstly, a query vector is generated from the local embedding vector: (8) in, For learnable query transformation matrix, Indicates satellite The query characteristics at the current moment; For each jump number Construct a concatenated vector of the query vector and the corresponding information source features. : (9) in, This represents the vector concatenation operation; let... For the learnable attention parameter vector, the activation function is... LeakyReLU calculates attention score : (10) LeakyReLU is a linear rectified function with leakage, and its typical form is: (11) in Leakage coefficient; Then, the information sources of the three hop counts are normalized using the softmax function to obtain the multi-hop attention weights. : (12) in, ,and This indicates the importance allocation for 0-hop, 1-hop, and multi-hop information sources; Based on the aforementioned multi-hop attention weights, the information sources with the three hop counts are weighted and aggregated to obtain the aggregation result of single-layer message passing: (13) To preserve the distinctiveness of local resource states, a residual structure is introduced on top of the aggregation result; let the residual transformation identity matrix be... Activation function Using the ReLU activation function, the satellite At any moment The final resource state is characterized as The formula is as follows: (14) Furthermore, step S3 specifically includes: S31: Constructing an observation mission model; when an observation mission is generated on an Earth observation satellite. The task will be broken down into a chain of subtasks with numerical dependencies. Each subtask state is represented by a six-tuple. Indicates; among which, This represents the initial data size of the task, in units of... ; This indicates the computing power requirements at each stage of the task, in units of... ; This represents the amount of data processed after each stage of the task is completed. Indicates the current processing stage of the task; The landing satellite represents the data return journey; the on-board calculation results of the observation data will eventually be transmitted back to the ground station via the landing satellite. It is a binary variable (1 indicates that the task has completed all calculations and processing, and 0 indicates that it has not been completed). S32: Construct a transmission and processing delay model for the observation task; assume an observation task is performed on a satellite. Generate satellites within the coverage area of ​​the ground station. The destination satellite; the data transmission path is The satellite performing the calculation is ,and The times when the missions arrive at each satellite transmission queue are as follows: The arrival time of each satellite computing queue is The total latency of a task during transmission and processing. for: (19) in, To delay the transmission time, For transmission delay, To calculate the delay, To delay queuing; Propagation delay The distance is determined by the relay link, as shown in the following formula: (15) in, represent Satellites that land at all times Distance to the ground station It represents the speed of light.

[0011] Transmission delay The amount of data and the link speed before and after each stage of the task are determined by the amount of data before and after the calculation. Let's say the task is in the first stage. The data size after each stage is The task is in The transmission path after computing satellites is The formula is as follows: (16) in, This indicates the transmission path between the starting satellite and the first computing satellite. Indicates the transmission path The Middle to Inter-satellite link rate between satellites This represents the downlink speed from the landing satellite to the ground station. Representing a path Number of satellites included.

[0012] Calculate latency The computational power required for the mission and the computational power of the satellite are determined by the following formula: (17) in, For the first task Each calculation stage, For the first The computing power of a single satellite, in units of ; here the first The first computing satellite and mission Each calculation stage corresponds to another.

[0013] Queue delay This includes the queuing delay of tasks in the satellite's computation and transmission queues, which depends on the transmission queue length of each link and the computation queue length of each computation satellite, as shown in the following formula: (18) Part One This represents the sum of the queuing delays of each relay link. Indicates reaching the th The time of each relay satellite, pointing to the transmission queue backlog (transmission queue length), in units of... Part Two The waiting time is calculated for each satellite. Indicates at time At that time, the first The computation queue backlog (i.e., computation queue length) for each satellite, in units of The last part This represents the queuing delay of the downlink for the landing satellite. Indicates the landing of satellites At any moment The length of the downlink queue.

[0014] S33: In satellite observation missions, the objective is to find data transmission paths within the constraints of satellite storage capacity. and computational satellites that complete each stage of the mission To minimize the overall task latency, the optimization objective function is constructed as follows: (20) in, Indicates satellite At any moment Storage resource usage, This indicates the satellite's maximum storage capacity.

[0015] Furthermore, step S4 specifically includes: S41: Employing deep reinforcement learning methods D 3 QN This approach addresses the joint optimization of computation and routing in low-Earth orbit satellite networks to achieve distributed decision-making in dynamic environments. The agent takes the local satellite resource state representation, mission state, and inter-satellite link state perceived by the time memory graph attention network as input, and outputs joint computation and routing decisions, thereby achieving on-orbit distributed execution. S42: State space; when the mission arrives, mission information and currently perceived local resource information from the satellite will be provided as input. DRL Agent, among which, DRL Represents deep reinforcement learning; state space ,in, Indicates resource status. Indicates task status; resource status It provides satellite resource information for the nearby area at the current time. For the satellite's final resource status representation output by the time memory map attention network in step S2, The inter-satellite link status in step S13; mission status This provides information such as the computational requirements and transmission destination of the task; S43: Action space; the action space can be represented as... ,in This indicates that the task will be transferred to the next level. The transmission queue of each neighbor, This indicates that the task will be pushed to the current satellite's computing queue and the computing processing for the current task phase will be executed; S44: Reward function; Reward function for satellite network task allocation Different calculation methods are used under different conditions to optimize task performance while minimizing latency and resource consumption; each condition corresponds to a specific transmission or processing event and is associated with a corresponding reward or penalty.

[0016] Rewards upon successful mission landing at the satellite for: (twenty one) in, This indicates whether the task has been considered complete. To successfully transmit the reward, This is the penalty coefficient for failure; When data packets are lost during transmission, a reward is given. R for: (twenty two) in, This is the packet loss penalty coefficient. The delay penalty coefficient, The task start time. For the current decision-making moment; For a correctly completed task, a one-hop transfer or a one-stage calculation is rewarded. R for: (twenty three) in, To store reserved values, Indicates the last decision-making time. Penalty for exceeding storage limits.

[0017] S45: Training the reinforcement learning model; S46: Relying on the ground-based digital twin system, the dynamic scenario of the low-orbit satellite network is reproduced, and model training and parameter optimization are completed. The trained strategy is deployed to the on-orbit satellite to realize real-time joint decision-making, and the strategy iteration is completed through satellite-ground interaction.

[0018] The beneficial effects of this invention are as follows: (1) Regarding on-board resource perception, this invention designs a temporal memory graph attention network to achieve fine perception of resource status within a multi-hop range in a large-scale LEO satellite network, alleviating the oversmoothing problem of traditional graph neural networks under multi-layer message passing. The temporal memory graph attention network is based on an embedded multi-hop cache attention mechanism: a multi-hop resource cache vector that evolves over time is maintained on each satellite, and cache exchange and aggregation updates are performed with its one-hop neighbors in each time slot. The multi-hop neighborhood resource status view centered on the satellite is gradually reconstructed through "one-hop cache exchange + time series accumulation + forgetting update". Thus, without explicitly superimposing multi-layer graph neural networks, the network can explicitly model the computation, storage, and link resource status of neighboring satellites within a multi-hop range, effectively suppressing the oversmoothing phenomenon of feature convergence and reduced recognizability in traditional graph neural networks in large-scale LEO constellations. Furthermore, the temporal memory graph attention network integrates the local resource features at the current moment, the resource features of one-hop neighbors, and historical multi-hop neighborhood resource information represented by multi-hop resource cache vectors into the graph attention structure. It adaptively weights and fuses these three types of information through attention weights, outputting a multi-hop resource representation vector that simultaneously reflects the current state and historical evolution patterns. Through this design, this invention, while relying on one-hop neighbor information exchange, can dynamically and accurately capture changes in satellite resource status within a multi-hop neighborhood, providing fine and reliable onboard resource awareness support for multi-satellite collaborative computing and joint route optimization.

[0019] (2) Regarding multi-satellite collaborative computing and task routing, this invention introduces an on-board collaborative computing mechanism during the task routing process, tightly coupling the computational processing of satellite Earth observation tasks with the routing and forwarding process, thereby achieving distributed collaborative processing of tasks among multiple satellites. Specifically, this invention divides the observation task into several ordered dependent sub-task stages, and during the task routing process, dynamically selects the current relay satellite or satellites on the subsequent forwarding path to perform the corresponding computation stage based on the computing power, storage resources, and queue status of each satellite. This allows the satellite to perform real-time computational processing of the observation data during the forwarding process, and then forwards or transmits the processed data back, thereby reducing the amount of large-volume raw data transmitted back and significantly alleviating the bandwidth pressure and congestion risk of the satellite-to-ground link. Through this "forwarding while computing" processing mode, this invention can effectively reduce communication overhead under limited satellite-to-ground communication resources, provide more efficient on-board real-time data processing capabilities and better end-to-end transmission performance, and provide support for the timely processing and reliable transmission of massive observation tasks.

[0020] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the multi-satellite collaborative computing and routing joint optimization method for low-Earth orbit satellite networks of the present invention; Figure 2 This is an architecture diagram for multi-satellite collaborative computing and joint routing optimization in low-Earth orbit satellite networks. Detailed Implementation

[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0023] Please see Figures 1-2 This invention provides a method for multi-satellite collaborative computing and routing joint optimization in low-Earth orbit satellite networks, which includes the following steps: S1: Construct a low-Earth orbit (LEO) satellite network topology model. Discretize continuous time into time slots, maintaining a relatively stable satellite network topology within each time slot. This mainly includes satellite nodes, inter-satellite links, and the network topology state changing over time. Specifically, this includes the following steps: S11: Usage Time Extension Chart To represent the dynamic network topology of low-Earth orbit satellite networks; where, , Represents the first in the system One satellite; Represents the set of all inter-satellite links, satellite , Links between them are connected by edges. express; Representing the time domain, in each time slot Internally, it updates link status and other parameters based on satellite orbital parameters, thus establishing the satellite network topology. It remains unchanged.

[0024] S12: For One of the satellites in the constellation at that time Its resource status characteristics are represented by a vector. Indicates; among which, express Time Satellite The remaining storage space; Indicates the satellite at time t The length of the computation queue represents the computational resource load. express Time Satellite The length of the transmission queue represents the load on transmission resources.

[0025] S13: Regarding At this moment, satellite To satellite inter-satellite links Its state can be represented by a tuple. Indicates; among which, This indicates the link rate, which can be detected in real time by onboard equipment. The current link distance can be calculated using satellite orbit parameters; Indicates the link from the satellite to The current transmission queue length can be obtained from the link buffer statistics.

[0026] S2: The satellite updates its resource status representation through a temporal memory graph attention network. In this network, each satellite maintains a multi-hop resource cache vector on-board based on its local resource status feature vector. By exchanging caches with its one-hop neighbors, resource status information gradually diffuses outward over time, thus gradually forming a resource status view of the multi-hop neighborhood. Within each time slot, the current local resource status representation, the resource status representation of one-hop neighbors, and the historical multi-hop neighborhood resource information represented by the cache are used as three types of information sources. A multi-hop attention mechanism is introduced to weight and aggregate these three types of information, and the residual term of the local resource status representation is superimposed on the aggregation result to obtain the updated satellite resource status representation, achieving multi-hop perception of on-board resources. Specifically, the following steps are included: S21: For time... satellite nodes below Its resource status feature vector is defined by step S12 as To improve the representation capability of resource state features and facilitate subsequent attention mechanisms and message aggregation processing, the three-dimensional resource state feature vector is... Mapped to In the feature space, we obtain the local embedding vector; let... Let be the characteristic transformation matrix, where Given a preset positive integer feature dimension, the local embedding vector is defined as follows: (1) Where, vector For satellite At any moment of Dimensional resource embedding representation, The elements are updated during training via gradient descent to transform the original resource state into a high-dimensional representation suitable for subsequent attention and aggregation operations.

[0027] In addition, in each satellite Maintain one Dimensional hop resource cache vector It is used to store the state information of multi-hop neighborhood resources obtained through hop-by-hop diffusion from neighbors; during system initialization, the cache vector is set to zero, i.e. .

[0028] S22: In each time slot The satellite performs a multi-hop cache update process; the satellite Set the current cache vector Send to all its one-hop neighbors Simultaneously receive neighbor cache vectors Subsequently, the cache itself and its neighbors' caches are averaged and aggregated to obtain the intermediate cache vector. : (2) in, Indicates satellite The set of direct neighbors, For satellite The number of direct neighbors.

[0029] Next, the local embedding vector at the current time will be... Write to the cache to obtain the cache for the next time slot: (3) in, This is a forgetting factor used to strike a trade-off between emphasizing the current state of local resources and preserving historical multi-hop information. As time slots progress, Will contain The system provides statistical information on the resource status of multiple satellites within a central hop neighborhood at multiple historical moments. This information is propagated spatially through hop-by-hop exchange of neighbor caches and multi-hop fusion is achieved through time series analysis.

[0030] S23: In time slot During the resource sensing phase, satellite With local embedding vectors One-hop neighbor embedding and multi-hop cache vector Based on this, and to facilitate a unified representation of resource information from different sources, the information source categories are numbered as follows: Construct three types of information sources: This represents the current local resource information of the satellite, and the formula is as follows: (4) This represents the satellite's one-hop neighbor resource information at the current moment, and the formula is as follows: (5) , indicating that it is a cache vector Representation of historical multi-hop neighborhood resource information: (6) To extract useful features for subsequent routing and computational decisions from different information sources and map them to a unified latent space, avoiding attention weight bias caused by differences in the scale of the original features, a linear transformation is performed on the three information sources respectively; specifically, for each hop count... Set the feature transformation matrix ,definition: (7) in, In order to target the Intermediate feature representation extracted from skip information sources; matrix These are learnable parameters that are automatically adjusted during training via gradient descent. They are used to convert resource information with different hop counts into feature representations suitable for attention mechanisms.

[0031] S24: To adaptively characterize three types of hop count information for satellites The importance of current resource awareness is highlighted, leading to the introduction of an attention mechanism; firstly, a query vector is generated from the local embedding vector: (8) in, For learnable query transformation matrix, Indicates satellite The query characteristics at the current moment; For each jump number Construct a concatenated vector of the query vector and the corresponding information source features. : (9) in, This represents the vector concatenation operation; let... For the learnable attention parameter vector, the activation function is... LeakyReLU calculates attention score : (10) LeakyReLU is a linear rectified function with leakage, and its typical form is: (11) in This represents the leakage coefficient.

[0032] Then, the information sources of the three hop counts are normalized using the softmax function to obtain the multi-hop attention weights. : (12) in, ,and This indicates the importance allocation for 0-hop, 1-hop, and multi-hop information sources.

[0033] Based on the aforementioned multi-hop attention weights, the information sources with the three hop counts are weighted and aggregated to obtain the aggregation result of single-layer message passing: (13) To preserve the distinctiveness of local resource states, a residual structure is introduced on top of the aggregation result; let the residual transformation identity matrix be... Activation function Using the ReLU activation function, the satellite At any moment The final resource state is characterized as The formula is as follows: (14) S3: Establish a joint optimization model for multi-satellite collaborative computing and routing in a low-Earth orbit satellite network. Taking into account satellite storage resources, transmission queue status, computing queue status, and mission characteristics (such as data volume, computing power requirements, and mission type), an optimization model is established with the objective of minimizing the overall latency of Earth observation missions. Specifically, this includes the following steps: S31: Constructing an observation mission model; when an observation mission is generated on an Earth observation satellite. The task will be broken down into a chain of subtasks with numerical dependencies. Each subtask state is represented by a six-tuple. It indicates. Among them, This represents the initial data size of the task, in units of... ; This indicates the computing power requirements at each stage of the task, in units of... ; This represents the amount of data processed after each stage of the task is completed. Indicates the current processing stage of the task; The landing satellite represents the data return journey; the on-board calculation results of the observation data will eventually be transmitted back to the ground station via the landing satellite. It is a binary variable (1 indicates that the task has completed all calculations and processing, and 0 indicates that it has not been completed).

[0034] S32: Construct a transmission and processing delay model for the observation task; assume an observation task is performed on a satellite. Generate satellites within the coverage area of ​​the ground station. The destination satellite; the data transmission path is The satellite performing the calculation is ,and The times when the missions arrive at each satellite transmission queue are as follows: The arrival time of each satellite computing queue is Let the speed of light be... Therefore, there are four main types of delays during the transmission and processing of a task: Propagation delay The distance is determined by the relay link, as shown in the following formula: (15) in, represent Satellites that land at all times Distance to the ground station.

[0035] Transmission delay The amount of data and the link speed before and after each stage of the task are determined by the amount of data before and after the calculation. Let's say the task is in the first stage. The data size after each stage is The task is in The transmission path after computing satellites is The formula is as follows: (16) in, This indicates the transmission path between the starting satellite and the first computing satellite. Indicates the transmission path The Middle to Inter-satellite link rate between satellites This represents the downlink speed from the landing satellite to the ground station. Representing a path Number of satellites included.

[0036] Calculate latency The computational power required for the mission and the computational power of the satellite are determined by the following formula: (17) in, For the first task Each calculation stage, For the first The computing power of a single satellite, in units of .

[0037] Queue delay This includes the queuing delay of tasks in the satellite's computation and transmission queues, which depends on the transmission queue length of each link and the computation queue length of each computation satellite, as shown in the following formula: (18) Part One This represents the sum of the queuing delays of each relay link. Indicates reaching the th The time of each relay satellite, pointing to the transmission queue backlog (transmission queue length), in units of... Part Two The waiting time is calculated for each satellite. Indicates at time At that time, the first The computation queue backlog (i.e., computation queue length) for each satellite, in units of The last part This represents the queuing delay of the downlink for the landing satellite. Indicates the landing of satellites At any moment The length of the downlink queue.

[0038] Finally, the total latency of the task It is the sum of the four types of delay: (19) S33: In satellite observation missions, the objective is to find data transmission paths within the constraints of satellite storage capacity. and computational satellites that complete each stage of the mission To minimize the overall task latency, the optimization objective function is constructed as follows: (20) in, Indicates satellite At any moment Storage resource usage, This indicates the satellite's maximum storage capacity.

[0039] S4: Solve the optimization problem in S3 using deep reinforcement learning. Based on satellite network data provided by the terrestrial digital twin system, the deep reinforcement learning model is trained offline until the algorithm converges, and the trained policy is periodically sent to the in-orbit satellite. Specifically, this includes the following steps: S41: Employing deep reinforcement learning methods D 3 QN This approach addresses the joint optimization of computation and routing in low-Earth orbit satellite networks to achieve distributed decision-making in dynamic environments. The agent takes the local satellite resource state representation, mission state, and inter-satellite link state perceived by the time memory graph attention network as input, and outputs joint computation and routing decisions, thereby achieving on-orbit distributed execution. S42: State space. When the mission arrives, mission information and the local resource information currently sensed by the satellite will be provided as input. DRL Agent, among which, DRL Represents deep reinforcement learning; state space ,in, Indicates resource status. Indicates task status. Resource status. It provides satellite resource information for the nearby area at the current time. For the satellite's final resource status representation output by the time memory map attention network in step S2, This refers to the inter-satellite link status in step S13. (Task Status) This provides information such as the computational requirements and transmission destination of the task.

[0040] S43: Action Space. Action space can be represented as... ,in This indicates that the task will be transferred to the next level. The transmission queue of each neighbor, This indicates that the task will be pushed to the current satellite's computing queue and the computing processing for the current task phase will be executed.

[0041] S44: Reward function; Reward function for satellite network task allocation Different calculation methods are used under different conditions to optimize task performance while minimizing latency and resource consumption; each condition corresponds to a specific transmission or processing event and is associated with a corresponding reward or penalty.

[0042] Rewards upon successful mission landing at the satellite for: (twenty one) in, This indicates whether the task has been considered complete. To successfully transmit the reward, This represents the penalty coefficient for failure.

[0043] When data packets are lost during transmission, a reward is given. R for: (twenty two) in, This is the packet loss penalty coefficient. The delay penalty coefficient, The task start time. This is the moment for making a decision.

[0044] For a correctly completed task, a one-hop transfer or a one-stage calculation is rewarded. R for: (twenty three) in, To store reserved values, Indicates the last decision-making time. Penalty for exceeding storage limits.

[0045] S45: Reinforcement learning model training. The core framework of D3QN consists of two main components: an online network for selecting the optimal action. and the target network used to evaluate these actions Both have the same structure, consisting of multiple hidden layers used to retrieve data from states. Features are extracted and then split into two computational streams: one for estimating the state-value function. Another method for estimating the advantage function .final, The value is obtained by combining the two computational streams using the following formula: (twenty four) in, Indicates the number of possible actions. It represents any action in the action space.

[0046] Target value From the target network Calculation. For each empirical tuple Target value Defined as current reward Add discount factor With the target network In the next state and the optimal next action The product of the predicted values, The calculation formula is as follows: (25) in, Representative target network The parameters.

[0047] loss function Temporal Difference (TD) error is used to measure the difference between the online network's predicted value and the target value. Here, Represents online network The parameter is calculated using the following formula: (26) By minimizing this loss function D 3 QN It can effectively learn the optimal strategy.

[0048] The parameters of the online network are updated using gradient descent to minimize the loss function. The update formula is as follows: (27) in, It's the learning rate. This represents the gradient of the loss function with respect to the online network parameters. During the update process, the algorithm calculates the gradient of the loss function with respect to the online network parameters and adjusts the parameters accordingly.

[0049] S46: Relying on the ground-based digital twin system, the dynamic scenario of the low-orbit satellite network is reproduced, and model training and parameter optimization are completed. The trained strategy is deployed to the on-orbit satellite to realize real-time joint decision-making, and the strategy iteration is completed through satellite-ground interaction.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for multi-satellite collaborative computing and joint route optimization in low-Earth orbit satellite networks, characterized in that, The method includes the following steps: S1: Construct a low-Earth orbit satellite network topology model, specifically: discretize continuous time into time slots, and keep the satellite network topology relatively stable within each time slot, including satellite nodes, inter-satellite links, and the network topology state that changes over time. S2: The satellite updates its resource status representation through a temporal memory graph attention network. Specifically, in the temporal memory graph attention network, each satellite maintains a multi-hop resource cache vector on-board based on its local resource status feature vector. By exchanging caches with one-hop neighbors, the resource status information gradually spreads outward in the time dimension, thereby gradually forming a resource status view of the multi-hop neighborhood. In each time slot, the current local resource status representation, the resource status representation of one-hop neighbors, and the historical multi-hop neighborhood resource information represented by the cache are used as three types of information sources. A multi-hop attention mechanism is introduced to weight and aggregate the three types of information, and the residual term of the local resource status representation is superimposed on the aggregation result to obtain the updated satellite resource status representation, thereby realizing multi-hop perception of on-board resources. S3: Establish a multi-satellite collaborative computing and routing joint optimization model for low-Earth orbit satellite networks. Specifically, this involves comprehensively considering the satellite's storage resources, transmission queue status, computing queue status, and mission characteristics to establish an optimization model aimed at minimizing the total latency of Earth observation missions. S4: The optimization problem in step S3 is solved using a deep reinforcement learning method. Specifically, the deep reinforcement learning model is trained offline based on satellite network data provided by the terrestrial digital twin system until the algorithm converges, and the trained strategy is periodically sent to the satellite in orbit.

2. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 1, characterized in that, Step S1 specifically includes: S11: Usage Time Extension Chart To represent the dynamic network topology of low-Earth orbit satellite networks; where, , Represents the first in the system One satellite; Represents the set of all inter-satellite links, satellite , Links between them are connected by edges. express; Representing the time domain, in each time slot Internally, the link status and satellite network topology are updated based on satellite orbit parameters. Remain unchanged; S12: For One of the satellites in the constellation at that time Its resource status characteristics are represented by a vector. Indicates; among which, express Time Satellite The remaining storage space; Indicates the satellite at time t The length of the computation queue represents the computational resource load. express Time Satellite The transmission queue length represents the transmission resource load. S13: Regarding At this moment, satellite To satellite inter-satellite links Its state is determined by a tuple Indicates; among which, This indicates the link rate, which is detected in real time by onboard equipment. This indicates the current link distance, calculated using satellite orbit parameters. Indicates the link from the satellite to The current transmission queue length is obtained from the link buffer statistics.

3. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 2, characterized in that, Step S2 specifically includes: S21: Transform the three-dimensional resource state feature vector Mapped to In the feature space, we obtain the local embedding vector; let... Let be the characteristic transformation matrix, where Given a preset positive integer feature dimension, the local embedding vector is defined as follows: （1） Where, vector For satellite At any moment of Dimensional resource embedding representation; In addition, in each satellite Maintain one Dimensional hop resource cache vector It is used to store the state information of multi-hop neighborhood resources obtained through hop-by-hop diffusion from neighbors; during system initialization, the cache vector is set to zero, i.e. ; S22: In each time slot The satellite performs a multi-hop cache update process; the satellite Set the current cache vector Send to all its one-hop neighbors Simultaneously receive neighbor cache vectors Subsequently, the cache itself and its neighbors' caches are averaged and aggregated to obtain the intermediate cache vector. : （2） in, Indicates satellite The set of direct neighbors, For satellite The number of direct neighbors; Next, the local embedding vector at the current time will be... Write to the cache to obtain the cache for the next time slot: （3） in, This is a forgetting factor used to strike a trade-off between "emphasizing the current local resource state" and "retaining historical multi-hop information"; as time slots progress, Will contain The system provides statistical information on the resource status of multiple satellites within a central hop neighborhood at multiple historical moments. This information is propagated spatially through hop-by-hop exchange of neighbor caches and multi-hop fusion is achieved through time series analysis. S23: In time slot During the resource sensing phase, satellite With local embedding vectors One-hop neighbor embedding and multi-hop cache vector Based on this, and to facilitate a unified representation of resource information from different sources, the information source categories are numbered as follows: Construct three types of information sources: This represents the current local resource information of the satellite, and the formula is as follows: （4） This represents the satellite's one-hop neighbor resource information at the current moment, and the formula is as follows: （5） , indicating that it is a cache vector Representation of historical multi-hop neighborhood resource information: （6） Perform linear transformations on the three information sources respectively; specifically, for each hop number... Set the feature transformation matrix ,definition: （7） in, In order to target the Intermediate feature representation extracted from skip information sources; matrix These are learnable parameters that are automatically adjusted during training via gradient descent. They are used to convert resource information with different hop counts into feature representations suitable for attention mechanisms. S24: To adaptively characterize three types of hop count information for satellites The importance of current resource awareness is highlighted, leading to the introduction of an attention mechanism; firstly, a query vector is generated from the local embedding vector: （8） in, For learnable query transformation matrix, Indicates satellite The query characteristics at the current moment; For each jump number Construct a concatenated vector of the query vector and the corresponding information source features. : （9） in, This represents the vector concatenation operation; let... For the learnable attention parameter vector, the activation function is... LeakyReLU calculates attention score : （10） Where LeakyReLU is a linear rectified function with leakage; Then, the information sources of the three hop counts are normalized using the softmax function to obtain the multi-hop attention weights. : （11） in, ,and This indicates the importance allocation for 0-hop, 1-hop, and multi-hop information sources; Based on multi-hop attention weights, information sources with three hop counts are weighted and aggregated to obtain the aggregation result of single-layer message passing: （12） To preserve the distinctiveness of local resource states, a residual structure is introduced on top of the aggregation result; let the residual transformation identity matrix be... Then the satellite At any moment The final resource state is characterized as The formula is as follows: （13） in, This is the ReLU activation function.

4. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 3, characterized in that, Step S3 specifically includes: S31: Constructing an observation mission model; when an observation mission is generated on an Earth observation satellite. The task will be broken down into a chain of subtasks with numerical dependencies. Each subtask state is represented by a six-tuple. Indicates; among which, This represents the initial data size of the task; This indicates the computing power requirements at each stage of the task; This represents the amount of data processed after each stage of the task is completed. Indicates the current processing stage of the task; The landing satellite represents the data return journey; the on-board calculation results of the observation data will eventually be transmitted back to the ground station via the landing satellite. It is a binary variable; S32: Construct a transmission and processing delay model for the observation task; assume an observation task is performed on a satellite. Generate satellites within the coverage area of ​​the ground station. The target satellite is to be landed; the data transmission path is... The satellite performing the calculation is ,and The times when the missions arrive at each satellite transmission queue are as follows: The arrival time of each satellite computing queue is The total latency of a task during transmission and processing. for: （14） in, To delay the transmission time, For transmission delay, To calculate the delay, To delay queuing; S33: In satellite observation missions, the objective is to find data transmission paths within the constraints of satellite storage capacity. and computational satellites that complete each stage of the mission To minimize the overall task latency, the optimization objective function is constructed as follows: （15） in, Indicates satellite At any moment Storage resource usage, This indicates the satellite's maximum storage capacity.

5. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 4, characterized in that, In step S32, the propagation delay The distance is determined by the relay link, as shown in the following formula: （16） in, represent Satellites that land at all times Distance to the ground station It represents the speed of light.

6. The method for multi-satellite collaborative computing and joint route optimization in low-Earth orbit satellite networks according to claim 4, characterized in that, In step S32, transmission delay The amount of data and the link speed before and after each stage of the task are determined by the amount of data before and after the calculation. Let's say the task is in the first stage. The data size after each stage is The task is in The transmission path after computing satellites is The formula is as follows: （17） in, This indicates the transmission path between the starting satellite and the first computing satellite. Indicates the transmission path The Middle to Inter-satellite link rate between satellites This represents the downlink speed from the landing satellite to the ground station. Representing a path Number of satellites included.

7. The method for multi-satellite collaborative computing and joint route optimization in low-Earth orbit satellite networks according to claim 4, characterized in that, In step S32, the delay is calculated. The computational power required for the mission and the computational power of the satellite are determined by the following formula: （18） in, For the first task Each calculation stage, For the first The computing power of a computing satellite.

8. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 4, characterized in that, In step S32, queuing delay This includes the queuing delay of tasks in the satellite's computation and transmission queues, which depends on the transmission queue length of each link and the computation queue length of each computation satellite, as shown in the following formula: （19） Part One This represents the sum of the queuing delays of each relay link. Indicates reaching the th The timing of each relay satellite indicates the backlog in the transmission queue leading to the next hop; Part Two The waiting time is calculated for each satellite. Indicates at time At that time, the first The backlog of computational queues for each satellite; the final part This represents the queuing delay of the downlink for the landing satellite. Indicates the landing of satellites At any moment The length of the downlink queue.

9. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 1, characterized in that, Step S4 specifically includes: S41: Employing deep reinforcement learning methods D 3 QN This approach addresses the joint optimization of computation and routing in low-Earth orbit satellite networks, enabling distributed decision-making in dynamic environments. The agent takes the local satellite resource state representation, mission state, and inter-satellite link state perceived by the time memory graph attention network as input, and outputs joint computation and routing decisions, thereby achieving on-orbit distributed execution. S42: State space; when the mission arrives, mission information and the local resource information currently sensed by the satellite will be provided as input. DRL Agent, among which, DRL Represents deep reinforcement learning; state space ,in, Indicates resource status. Indicates task status; resource status Provides satellite resource information for the vicinity at the current time; mission status. This provides information about the computational requirements and transmission destination for the task; S43: Action space; Action space is represented as... ,in This indicates that the task will be transferred to the next level. The transmission queue of each neighbor, This indicates that the task will be pushed to the current satellite's computing queue and the computing processing for the current task phase will be executed; S44: Reward function; Reward function for satellite network task allocation Different calculation methods are used under different conditions to optimize task performance while minimizing latency and resource consumption; each condition corresponds to a specific transmission or processing event and is associated with a corresponding reward or penalty. S45: Training the reinforcement learning model; S46: Relying on the ground-based digital twin system, the dynamic scenario of the low-orbit satellite network is reproduced, and model training and parameter optimization are completed. The trained strategy is deployed to the on-orbit satellite to realize real-time joint decision-making, and the strategy iteration is completed through satellite-ground interaction.

10. The method for multi-satellite collaborative computing and route joint optimization in low-Earth orbit satellite networks according to claim 9, characterized in that, In step S44, when the mission reaches the landing satellite, a reward is given. for: （20） in, This indicates whether the task has been considered complete. To successfully transmit the reward, This is the penalty coefficient for failure; When data packets are lost during transmission, a reward is given. R for: （21） in, This is the packet loss penalty coefficient. The delay penalty coefficient, The task start time. For the current decision-making moment; For a correctly completed task, a one-hop transfer or a one-stage calculation is rewarded. R for: （22） in, To store reserved values, Indicates the last decision-making time. Penalty for exceeding storage limits.

Images