A variable scale satellite constellation orbit planning method based on graph proximal optimization algorithm

Abstract

The application discloses a variable scale satellite group orbit planning method based on a graph proximal optimization algorithm, and realizes real-time and efficient control under the conditions of dynamic change of node quantity and limited communication link. Firstly, the orbit and communication relationship of the satellite group at a time point are abstracted into a dynamic graph. Subsequently, a reinforcement learning structure based on a graph neural network and a recurrent neural network is constructed. A proximal policy optimization algorithm is adopted for policy and value function asynchronous iteration on the satellite, and all calculation and parameter updating are completed on the satellite, without a central control satellite. After each satellite completes local algorithm updating, the satellite exchanges parameter difference with neighbor satellites meeting a reliability condition, and is weighted according to a satellite link quality and a physical distance hybrid weight. The method can be widely applied to application scenarios, such as earth imaging, global communication, navigation and the like, which need constellation level cooperation, and provides a complete technical system for safe, efficient and autonomous operation of a large scale satellite group.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of space orbit planning, and application of multi-agent reinforcement learning technology in orbit control and intelligent decision of a star cluster. BACKGROUND

[0002] With the maturity of batch manufacturing and commercial launch services, a constellation or formation composed of dozens to hundreds of satellites has become an important technical route for obtaining high temporal and spatial resolution remote sensing data and providing global continuous communication. Compared with single-satellite systems, such multi-satellite systems exhibit significant complexity in orbit scheduling and cooperative control. In addition, the rapid evolution of the number of satellites and the formation shape across the orbital plane makes it difficult for traditional orbit planning based on fixed-scale variables or centralized solvers to meet the engineering needs of real-time re-planning as the configuration changes.

[0003] The introduction of distributed and self-learning paradigms in multi-satellite orbit planning breaks through the bottleneck of traditional centralized optimization in large-scale applications. Deep reinforcement learning is considered a powerful means to improve the autonomy of satellite clusters because it can achieve end-to-end policy search under unknown disturbances, multi-source nonlinear constraints, and incomplete observation conditions. However, existing reinforcement learning research still follows the "centralized training-distributed execution" architecture: all trajectory data need to be aggregated to the ground or the main star server for unified gradient calculation, and then the updated network parameters are distributed to each node. This mode not only fails to meet real-time requirements in situations where inter-satellite links are limited and backhaul windows are short, but also implicitly requires the satellite scale to remain consistent during the training period and deployment, making it difficult to handle flexible scenarios where nodes are added or deleted at any time. Some research attempts to embed a graph neural network into a policy model to utilize time-varying topology information, but its parameter updates usually rely on centralized aggregation or parameter server frameworks, still unable to break away from global synchronization constraints, resulting in compromised convergence when links are intermittent or some nodes fail. In contrast, the fully decentralized graph reinforcement learning system proposed in the application avoids dependence on a central node from the underlying communication mechanism to the policy iteration process, with each satellite independently completing gradient calculation and exchanging compressed neural network parameter information only with a limited neighborhood, efficiently achieving continuous learning and robust cooperation under conditions of dynamic expansion of node scale, unstable links, and frequent reconstruction of topology.

[0004] In summary, there is an urgent need for a new orbit planning method that can adapt to the environment of independent operation with node addition and deletion, while maintaining high-order dynamic accuracy and computational affordability. The scalable orbit planning system for star cluster proposed in the application not only significantly improves the accuracy and efficiency of multi-satellite cooperative maneuvering, but also maintains the stability of the formation topology and the continuity of the task in a fully decentralized communication-limited environment. The system naturally adapts to changes in the size of the constellation, providing a solid theoretical foundation and engineering feasibility for future large-scale, long-life, and high-complexity space missions. SUMMARY

[0005] To address the aforementioned issues, this invention provides a variable-scale satellite constellation orbit planning method based on a graph near-end optimization algorithm. This method can be widely applied to applications requiring constellation-level coordination, such as Earth imaging, global communication, and navigation, providing a complete technical system for the safe, efficient, and autonomous operation of large-scale satellite constellations.

[0006] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0007] A variable-scale satellite constellation orbit planning method based on graph near-end optimization algorithm includes the following steps:

[0008] S1. Establish a dynamic graphical model for constellation orbit planning: at time... First, let's look at the dynamic graph sequence. Characterizing the time-varying topological relationships of star clusters: where Give the current set of satellites in orbit, and its number. The number of tasks can be increased or decreased in real time. Represents the set of edges that satisfy distance or communication constraints; the position, velocity, and remaining fuel of each satellite are encapsulated in the node feature matrix. The relative distance, relative velocity, and link quality between the two satellites are encapsulated into the edge feature tensor. The orbital motion and communication constraints are uniformly encoded into a well-structured data format that can be automatically updated over time. Subsequent artificial intelligence algorithms only need to process this dynamic network graph to perform global collaborative planning.

[0009] Each satellite carries complete state information in the image, which can be used for decision-making. Each satellite introduces node feature vectors. ,in These are position, velocity, and acceleration, which describe the satellite's motion state. It represents remaining fuel, reflecting mobile resources; This encoding represents the task status, allowing the strategy to distinguish between satellite roles and fault conditions. It compresses heterogeneous information into a fixed length, facilitating direct processing by the neural network, and ensures that changes in the number of nodes do not require modification of the network dimension through weight sharing. In the network model, an edge feature vector... subscript Indicates the two satellites connected by the edge, with time superscript. Indicates the current moment; It is the Euclidean distance between the two satellites; the second term The difference in the velocity vectors of the two stars, i.e., their relative velocities, reflects potential convergence / separation trends; the third term... is the communication availability coefficient, 1 means the link is fully available at this moment, 0 means the link is interrupted. Reinforcement learning algorithm will automatically learn in the training process that the satellite pairs with closer distance, smaller relative velocity and more stable link should be given higher interaction weight in task assignment or formation control.

[0010] To explicitly adjust the adjacency weight in the message passing process, the weighted adjacency matrix is constructed as

[0011]

[0012] where controls the decay amplitude of distance. This formula maps the physical distance to a continuous coefficient between 0-1, so that the information of long distance is naturally weakened when graph convolution, which preserves the necessary global coupling and suppresses the noise propagation; at the same time, the exponential form ensures the gradient smoothness, which is convenient for end-to-end training. Overall, these four formulas cooperatively define a dynamic graph model of variable scale, real-time update and complete decentralization for the satellite constellation, which provides a unified and scalable data basis and information flow mechanism for the subsequent execution neural network-judgment neural network reinforcement learning strategy based on GNN-RNN.

[0013] S2, a completely decentralized multi-agent reinforcement learning framework and execution neural network-judgment neural network network structure design:

[0014] Under the decentralized intelligent control framework, each satellite can make independent decisions only by relying on its own detection data and limited neighborhood communication. Therefore, at any time the first satellite constructs a local observation vector

[0015] ,

[0016] where is the node's own feature, is the neighbor set that meets the communication / distance threshold, is the corresponding edge feature, which ensures that any decision only depends on the "self + neighbor" information; thus the local strategy

[0017]

[0018] In this model, the symbol represents the maneuvering instruction that the satellite needs to execute at the current time , i.e. the acceleration. The symbol is the internal memory vector stored by the recurrent network at the last time, which is used to retain the historical information of orbit evolution to support the current decision, where the satellite number is omitted by the subscript, and the indicates that it comes from the last time. The symbol represent the trainable parameters shared by the whole neural network (including graph neural network and recurrent neural network); since all satellites use the same set of They can independently and in parallel perform inference and update on their respective onboard computers without relying on any central control node, thus achieving true decentralization coordination.

[0019] The input data at the current time is fed into the neural network, and the first layer of the neural network (Actor) - Critic performs Layer message passing graph neural network; for the Layer hidden vector There are

[0020]

[0021] Where the superscript indicates the time corresponding to this calculation; the subscript represents the satellite being updated, and represents all neighbor satellites that meet the communication and distance threshold; is the original feature vector of the satellite (including position, velocity, fuel, etc.), which is specifically processed for the input vector of the current node, specifically processed for the input vector of the neighbor node, both of which are shared in the Layer of the whole graph, making the number of model parameters independent of the number of satellites; is the attention weight obtained by soft-max normalization, and the higher the value, the closer the neighbor to the target satellite , the better the link or the more relevant state; is a nonlinear activation function used to enhance the network's expression ability. After Layer iteration, each satellite integrates the spatial dependence information of the neighborhood in one inference.

[0022] After Layer graph neural network, the node satellite obtains the spatial feature vector at time ; this vector and the memory vector saved in the gate recurrent unit at the last time are fed into a single-layer GRU, which integrates the current spatial information and historical orbit trajectory through the gating operation, and outputs a new memory vector

[0023]

[0024] Since the GRU automatically performs selective forgetting and updating of information, The dimension is fixed all the time, so that both the short-time dynamics and the stability of long-time training are preserved.

[0025] In the actor, the memory vector of the last time step is fed into two fully connected channels to generate the mean and covariance of the action distribution. Then the original thrust vector is sampled from a normal distribution and compressed by a differentiable saturation function to the engine limit , i.e.

[0026] This way, the gradient can be backpropagated, and the output thrust command is always within the physical limit.

[0027] In the critic, the GNN-RNN encoder is used to extract the spatial-temporal information, and then a linear regression layer with only one neuron is added to the output memory vector to get the scalar state value , where the subscript denotes the satellite number, and the superscript denotes the current time step.

[0028]

[0029] where is the trainable weight of the critic, and is the expected value of the future reward accumulated by the discount factor from the current time step . The critic outputs the scalar state value to measure the goodness of an action and reduce the gradient variance, thus accelerating the convergence.

[0030] Both the policy and the value are updated by the proximal optimization loss with truncated importance weights; for each time step, the advantage

[0031]

[0032] where is the discount rate, and is the target network with the old parameters.

[0033] To update the parameters of the actor, a loss function called CLIP is used to limit the policy gradient during the optimization process, thus maintaining the stability of the update. Specifically, the calculation formula of the CLIP loss is:

[0034]

[0035] is the CLIP loss, which is a loss function used to limit the magnitude of policy updates.

[0036] is the importance weighted ratio, which measures the difference between the current policy and the old policy . is the action (direction, magnitude, etc.) taken by the satellite at time . is the advantage function, which represents the improvement in value of taking an action compared to the baseline value of the current policy. It is calculated as the difference between the current policy and the existing policy. is a clipping operation that limits the importance ratio to be within the range . is a clipping threshold, usually set to a small constant (e.g., 0.1), used to prevent large policy updates and maintain stability during training.

[0037] In addition, the critic network is updated by minimizing the mean squared error loss , which is given by:

[0038]

[0039] where is the state value prediction of the critic network for the state of the satellite at time , representing the estimated future return of the satellite in that state. is the actual discounted return, obtained through environmental feedback (reward), representing the discounted cumulative return from the current time to the future.

[0040] Each satellite uses a replay buffer containing only its own and neighbor information to compute gradients in parallel, asynchronously synchronize weights, preserving decentralized execution, and maintaining learning consistency through parameter broadcasting when bandwidth allows. The entire design linearly concatenates three types of information: "space interaction-time dependence-policy optimization", forming a GNN+RNN execution neural network-critic neural network algorithm without central nodes, distributed training and execution, and robust to the number of satellites and topology.

[0041] S3. High-precision iterative simulation of satellite orbit state based on RK8 method:

[0042] To ensure high accuracy of satellite orbit state evolution in the reinforcement learning process, the orbit dynamics of each satellite is modeled by a first-order ordinary differential equation set. The equation set is as follows:

[0043]

[0044] wherein, is the state vector of the satellite, including three-dimensional position and velocity , both of which are three-dimensional vectors; is the thrust instruction output by the actor, representing the direction and size of the thrust applied by the satellite in three-dimensional space. The function integrates all factors affecting the satellite, including the main gravity of the earth, the J2 non-uniform term, atmospheric resistance, solar radiation pressure and other orbital disturbances. The relationship between the state and the acceleration is directly given to the numerical integrator for solution. To ensure the accuracy and efficiency of numerical calculation, the present application adopts the Runge-Kutta 8th order method (RK8) for iterative simulation of the orbit state. In each environment step , the 13-stage 8th order method of RK8 is used for updating, and the slopes of the 13 stages are first calculated, and the expression is as follows:

[0045]

[0046] wherein, is the current adaptive step size, is the coefficient for updating, and the specific value is determined by the RK8 algorithm; is a constant coefficient in the RK8 method, used to adjust the weight of each slope; is the acceleration obtained by the actor. Next, the state at the current time is calculated according to the slopes:

[0047]

[0048] wherein, and are coefficients that meet the 8th order algebraic consistency, ensuring the accuracy of numerical integration.

[0049] The orbit state calculated by the RK8 method is used as the next observation in the reinforcement learning environment, and is stored in the local replay buffer together with the corresponding reward. Since this method can effectively control the error while maintaining high-order accuracy, it ensures that the policy gradient can more accurately reflect the true satellite orbit dynamics in the subsequent reinforcement learning process.

[0050] S4, decentralized fusion of neural network network parameters and strategy generalization mechanism:

[0051] In the decentralized training phase, each satellite maintains a set of evaluation neural network parameter vectors only related to itself , and performs a pure local random gradient descent update according to the gradient obtained by local sampling , to obtain an intermediate result

[0052]

[0053] Wherein is the local learning rate, which ensures that each node can improve the value estimation independently even if it is temporarily disconnected; then, in the neighborhood of the satellite where the communication link reliability exceeds the threshold and the relative distance is not greater than , the satellite exchanges respective with all and completes the decentralized fusion in the form of weighted average:

[0054]

[0055] Wherein the weight is determined by the link reliability and the physical proximity, balances the influence of both, is the distance attenuation scale; the formula ensures that the weight is non-negative and the sum is 1, thereby embedding the classical consensus average into the spatial network with limited communication and dynamic topology, and making the neighbors with good links and short distances contribute more to the value estimation. Since all calculations only rely on local neighborhood information, the entire fusion process can be executed asynchronously on each node without a global clock or centralized parameter server, and naturally supports topology changes: when a new satellite accesses, it can quickly obtain the initialization

[0056]

[0057] Wherein is the last synchronization time of the neighbor; when a satellite fails to exit, its parameters are automatically excluded from the normalization denominator of its neighbors, without affecting the continuity of the value estimation of the remaining nodes. The decentralized fusion mechanism significantly reduces the sensitivity of the evaluation neural network to observation bias of a single node, improves the generalization ability of the value function in different formation sizes and task scenarios, while ensuring that the policy gradient still comes from completely local information flow, in line with the design goal of distributed training-distributed execution.

[0058] The beneficial effects of the present application are:

[0059] 1. Complete decentralization, significantly improved reliability and real-time performance: Adopting a fully distributed architecture, each satellite independently completes decision-making and parameter updating, eliminating the dependence on central nodes and the risk of single-point failure; Local computing on the satellite avoids long-link communication delay, and still achieves millisecond-level response in link-limited scenarios, greatly improving the system's anti-interference ability and task continuity.

[0060] 2. Naturally adapt to dynamic changes in scale, with strong engineering flexibility: Through sparse index dynamic graph model and shared parameter design, support satellite number expansion and contraction (new / retired satellites only need to update the index); New satellites can be quickly initialized through neighborhood parameter fusion, and failed satellites can automatically leave the system without interrupting the task or retraining the model, significantly reducing the deployment and maintenance cost of large-scale constellation.

[0061] 3. Precision and efficiency, adapt to complex space scenarios: GNN-RNN fusion extracts inter-satellite spatial interaction and orbital history features, combined with PPO algorithm to ensure policy decision accuracy; RK8 high-order numerical integration ensures high consistency between orbit simulation and real environment; Local parameter interaction mechanism optimizes bandwidth overhead, and still can achieve high-precision cooperation (such as position error ≤50m) and efficient task execution in complex scenarios such as multi-source disturbance, heterogeneous nodes, and intermittent links. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is a flowchart of the present application. DETAILED DESCRIPTION

[0063] To make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings. In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the scope of the present application, so the present application is not limited to the specific implementation disclosed below.

[0064] Reference Figure 1 , a variable-scale satellite group orbit planning method based on a graph near-optimal algorithm, comprising the following steps:

[0065] Step 1: Establish a dynamic graph model for satellite group orbit planning

[0066] The present embodiment first abstracts the variable-scale satellite group in time into a set of dynamic graphs that automatically update over time to uniformly describe orbit motion and communication topology. The core definitions are as follows:

[0067]

[0068] where subscript denotes the current time; where subscript denotes the current set of on-orbit satellites, whose number is dynamically adjusted in real-time according to mission progress; denotes the set of edges satisfying distance or communication constraints; the position, velocity, and remaining fuel of each satellite are encapsulated into the node feature matrix , while the relative distance, relative velocity, and link quality between two satellites are encapsulated into the edge feature tensor .

[0069] Node feature encoding: To let each satellite carry the complete local information needed for decision-making, the node vector of the th satellite is defined as

[0070]

[0071] where are position, velocity, and acceleration, respectively, describing the satellite motion state; is the remaining fuel, reflecting the maneuvering resource; denotes the mission state, allowing the strategy to distinguish satellite roles and failure conditions. All components are linearly normalized to the interval [-1, 1], ensuring that different physical quantities are dimensionally unified and facilitating gradient propagation.

[0072] Edge feature encoding: For any pair of satellites satisfying , the edge vector is constructed as

[0073]

[0074] where denotes the two satellites connected by the edge, and the time superscript denotes the current time; is the Euclidean distance between the two satellites; the second term describes the difference in velocity vectors between the two satellites, i.e., the relative velocity, which can reflect the potential convergence / divergence trend; the third term is the communication availability coefficient, where 1 indicates that the link is fully available at this moment, and 0 indicates that the link is interrupted. This three-dimensional encoding unifies geometry, dynamics, and communication quality, facilitating the subsequent graph neural network to automatically learn that "close distance, small relative velocity, and good link quality" should be given higher interaction weights without manual parameter tuning.

[0075] Adjacency weight adjustment: To explicitly weaken distant noise in the message passing phase, the weighted adjacency matrix

[0076]

[0077] where is the distance decay scale hyper-parameter. The exponential mapping compresses the physical distance into the continuous interval (0, 1] to ensure that the information aggregation from distant nodes decays exponentially during graph convolution; meanwhile, the first-order derivative of the function is continuous, which is beneficial for end-to-end gradient optimization. The matrix is stored in sparse format and only non-zero edges are calculated.

[0078] Dynamic graph update mechanism: Each control cycle is executed in three steps: ① All satellites output new by high-order numerical integrators (see step three); ② Update and corresponding according to the latest distance and link evaluation results; ③ If a satellite is launched or leaves the network, only the corresponding index in needs to be inserted or deleted, and the rows and columns of need to be adjusted, without the need to rebuild the entire graph. This design ensures that the model still maintains level of insertion and deletion overhead when the number of nodes changes dramatically.

[0079] Through the above four groups of formulas and update steps, the dynamic graph model constructed by the embodiment can accurately reflect the spatio-temporal coupling relationship of the satellite constellation within milliseconds of refresh cycle, and provide size-expandable and semantically consistent input tensors for the subsequent GNN-RNN execution-judgment network.

[0080] Step two: fully decentralized multi-agent reinforcement learning framework and execution-judgment network structure design

[0081] First, within each time , the th satellite constructs a local observation vector according to its own sensor readings and neighborhood information collected through available links.

[0082]

[0083] where is the node feature, including inertial position , velocity , actual acceleration in the previous step , remaining fuel , and task status code ; the set is composed of neighbor satellites that satisfy the distance threshold and link availability ; the edge feature represents the Euclidean distance, the velocity difference module length, and the link availability factor, respectively. This design ensures that any decision only relies on local observable data, meeting the decentralized execution constraint.

[0084] Subsequently, send to the first layer Graph neural network (GNN) with heavy message passing. For the Layer hidden vector (Subscript denotes satellite node, layer number Depth of graph neural network) performs

[0085]

[0086] Where Matrix Only acts on the node feature of the current node, Only acts on the node feature of the neighboring node, both of which are shared by all nodes in the graph; Take LeakyReLU; attention weight

[0087]

[0088] Output by a trainable scoring function And in the neighborhood Node satellite direction normalization, guarantee So that the neighboring stars with close distance, good link or state correlation have greater influence when information is aggregated. After completing Layer transmission, get spatial embedding .

[0089] To introduce historical dynamic dependence, the Memory Input single-layer gated recurrent unit (GRU):

[0090]

[0091] Memory vector Dimension fixed, independent of sequence length; GRU automatically retains or forgets information through reset gate and update gate, capturing short-term dynamic trends while avoiding gradient vanishing.

[0092] The actor network (Actor) uses two fully connected mappings , Convert To action distribution parameters; then sample the original thrust vector From the Gaussian distribution, and limit it to the engine upper limit through differentiable saturation conversion :

[0093]

[0094] Where Actor global shared parameters; bold Indicates the final acceleration command.

[0095] The Critic shares the GNN-GRU encoder with the Actor but outputs state values with a linear layer at the end.

[0096]

[0097] where is an evaluation weight independent of . This value function is defined as the expected cumulative discounted return from time onward when the current policy is kept unchanged, which serves as a low-variance baseline.

[0098] Policy optimization: uses the truncated importance-weighted proximal optimization loss. For each experience trajectory of length , the advantage

[0099]

[0100] Again, let

[0101]

[0102] Update the Actor, where is the importance-weighted ratio, which measures the difference between the current policy and the old policy . is the action (direction, magnitude, etc. of the thrust) taken by the satellite at time . is the advantage function, which represents the improvement of taking a certain action over the baseline value of the current policy. It is calculated as the difference between the current policy and the existing policy. is a clipping operation that limits the importance ratio to the range . is a clipping threshold, usually set to a small constant (e.g. 0.1), to prevent large policy updates and maintain the stability of the training process.

[0103] Critic synchronous minimization

[0104]

[0105] is the actual discounted return. Each satellite only relies on its own + neighbor data to calculate the gradient and asynchronously average parameters without the need for a central server. The algorithm maintains consistent convergence rates and final performance without the need for reparameterization, demonstrating the natural robustness of the framework to changes in node number and topology.

[0106] Step three: high-precision iterative simulation of satellite orbit state based on RK8 method

[0107] This implementation describes the orbital motion and external thrust of each satellite as a unified system of first-order ordinary differential equations.

[0108]

[0109] in, It is the satellite's state vector, which contains its three-dimensional position. and speed Both are three-dimensional vectors; It is the thrust command output by the execution network (Actor), representing the direction and magnitude of the thrust applied by the satellite in three-dimensional space. Function This study integrates all influencing factors affecting the satellite, including Earth's principal gravity, J2 non-uniformity, atmospheric drag, and orbital perturbations such as solar radiation pressure. The state-acceleration relationship is directly solved by a numerical integrator. To ensure the accuracy and efficiency of numerical calculations, this invention employs the Runge-Kutta 8th-order method (RK8) for iterative simulation of the orbital state. At each environmental step... In the process, the 13-stage, 8th-order method of RK8 is used for updating. First, the slopes of the 13 stages are calculated. Its expression is:

[0110]

[0111] in, It is the current adaptive step size. These are the coefficients used for updating; their specific values ​​are determined by the RK8 algorithm. These are constant coefficients in the RK8 method, used to adjust the weight of each slope; The acceleration is obtained by executing the network. Next, the current state is calculated based on these slopes:

[0112]

[0113] in, and These are coefficients that satisfy the consistency of 8th-order algebras, ensuring the accuracy of numerical integration.

[0114] An outer control cycle Composed of several different sizes Cascaded composition: The algorithm accumulates in an internal loop. When the accumulated time first exceeds Back to exactly using linear interpolation The state, and the state As the next observation of the intelligent agent by the environment.

[0115] Step 4: Decentralized fusion of critic neural network parameters and strategy generalization mechanism

[0116] In the decentralized training phase, each satellite maintains a set of critic neural network parameter vectors related only to itself, and performs a pure local stochastic gradient descent update according to the gradient obtained by local sampling to obtain an intermediate result

[0117]

[0118] where is the local learning rate, ensuring that each node can independently improve the value estimate even if it is temporarily disconnected; then, within the neighborhood that satisfies the communication link reliability threshold and the relative distance is not greater than , the satellite exchanges its with all and completes decentralized fusion in the form of weighted average:

[0119]

[0120] where the weight is determined by the link reliability and physical proximity, balancing the influence of both, is the distance decay scale; this formula ensures that the weight is non-negative and the sum is 1, embedding the classical consensus average into a spatial network with limited communication and dynamic topology, and making the neighbors with "good links and close distances" contribute more to the value estimate. Since all calculations only rely on local neighborhood information, the entire fusion process can be executed asynchronously on each node without the need for a global clock or centralized parameter server, and naturally supports topology changes: when a new satellite joins, it can quickly obtain initialization

[0121]

[0122] where is the last synchronization time of the neighbor; when a satellite fails and exits, its parameters are automatically excluded from the normalization denominator of its neighbors, without affecting the continuity of the value estimate of the remaining nodes. This decentralized fusion mechanism significantly reduces the sensitivity of the critic neural network to observation bias of a single node, improves the generalization ability of the value function in different formation sizes and task scenarios, while ensuring that the policy gradient still comes from completely local information flow, consistent with the design goal of distributed training-distributed execution.

[0123] Step four: Decentralized fusion of critic network parameters and strategy generalization mechanism

[0124] In this step, each satellite maintains a local critic parameter vector and implements fully decentralized value function homogenization through a two-stage process of "local update → neighborhood fusion". First, the satellite performs a pure local stochastic gradient descent based on the gradient sampled in the current period, with learning rate :

[0125]

[0126] The satellite can still independently correct the value estimate in time according to the latest trajectory, avoiding error accumulation. Subsequently, it enters the parameter fusion stage: the satellite exchanges intermediate parameters only with neighbors that satisfy the "link reliability " and "distance " conditions, forming a neighborhood set

[0127]

[0128] To consider both link quality and physical proximity, an unnormalized kernel function

[0129]

[0130] where controls the weights of the two factors, and the exponential term causes the influence of distant nodes to decay exponentially with . Normalizing yields a consistency weight

[0131]

[0132] Thus, the weighted average fusion is completed locally on the satellite

[0133]

[0134] The above formula ensures that the weights are non-negative and sum to 1, and the parameter update remains within the neighborhood convex hull, suppressing abnormal node noise without relying on a global clock; each satellite can trigger fusion asynchronously after detecting that the neighborhood information is complete, exhibiting natural topological flexibility. When a new satellite is connected, it can quickly obtain the initial parameters using the same rule as long as it establishes a link that meets the threshold

[0135]

[0136] where ​For the first batch of communicable neighbors, if a satellite fails to be offline, its parameters are automatically excluded in the neighbor normalization denominator, and the remaining nodes continue to iterate seamlessly. With Both are derived from the dynamic graph edge feature tensor That is, the distributed consistency of the value function on the dynamic, weighted and asynchronous spatial network is realized.

[0137] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A variable size satellite constellation orbit planning method based on graph-based proximal optimization algorithm, characterized in that, Comprising the following steps: S1, establish a dynamic graph model of constellation orbit planning: abstract the satellite constellation as a dynamic graph structure evolving continuously over time, map each satellite as a node in the graph, and determine whether to establish an edge between two satellites based on the relative distance and communication link quality in space; S2, design a completely decentralized multi-agent reinforcement learning framework: each satellite is an independent agent that only relies on its own observations and neighborhood communication results to make decisions without the need for ground or central node coordination; A combination structure of graph neural network and recurrent neural network is adopted to extract spatial interaction patterns and historical dependencies of orbit dynamics; The execution neural network branch maps the hidden state to continuous thrust instructions, including direction and size, and executes them; The evaluation neural network branch evaluates the value of the same hidden state to provide a low-variance benchmark for policy gradient; S3, high-precision iterative simulation of satellite orbit state based on RK8 method: use the eighth-order Runge-Kutta algorithm as the core numerical integrator. During simulation, consider the comprehensive acceleration input dynamics model of controlled thrust, earth non-spherical gravity, second-order flattening perturbation, and atmospheric resistance; calculate the instantaneous state derivative; Then evaluate multiple slope samples in one step, strictly control the high-order integral error of position and velocity at the next time by high-order combination prediction, and realize high-precision orbit state prediction; S4, realize decentralized fusion of evaluation neural network parameters and strategy generalization mechanism: after completing local gradient descent, each satellite sends the updated value network parameters or gradient fragments to the satellites in its neighborhood, and synchronously receives the data from the opposite side; The node calculates the weight based on link reliability and relative distance, and performs weighted averaging on multiple parameters to generate a new local evaluation neural network.

2. The method of claim 1, wherein, In S1, once the edge is established, real-time distance, relative velocity, and link availability indicators are encapsulated as edge features; As the satellites move along the orbit, the attributes of nodes and edges and the entire topology are automatically refreshed, ensuring that the model continuously reflects the real spatial relationship.

3. The method of claim 1, wherein, In S1, the storage method of the graph uses sparse indexing, and the addition or deletion of nodes only requires inserting or deleting index entries without rebuilding the entire graph, so it can naturally adapt to the batch launch or retirement of satellites during the task.

4. The method of claim 1, wherein, In S1, in order to balance global scale and local precision, the model supports hierarchical adjacency, and weakly coupled relationships at long distances are weakened to be negligible.

5. The method of claim 1, wherein, In S1, the dynamic graph interface is decoupled from the reinforcement learning environment, and any external orbit simulator can inject state into the model through standardized feature tensors, allowing algorithm developers to quickly integrate new dynamics or constraint models without modifying the core framework.

6. The method of claim 1, wherein, In S2, all parameters are updated iteratively using the proximal policy optimization algorithm, and the loss is calculated locally, with gradients shared within a limited neighborhood, maintaining training stability without introducing global dependencies.

7. The method of claim 1, wherein, In S2, the framework naturally supports dynamic changes in the number of satellites and topology: new nodes can reason with shared weights, and offline old nodes do not disrupt the training of other nodes.

8. The method of claim 1, wherein, In S2, to reduce the burden, the network layer uses parameter sharing, and batch reasoning can be performed in parallel on the same hardware; gradient synchronization only transmits weight differences, reducing communication traffic.

9. The method of claim 1, wherein, In S4, the local consensus mechanism keeps the value function consistent globally while avoiding the bandwidth pressure caused by large-scale broadcasting when the topology changes.

Images