Multi-star traversal access sequence planning method and device based on attention mechanism

Abstract

The present application relates to a multi-satellite traversal access sequence planning method and device based on an attention mechanism, wherein the method is based on the initial dynamic properties of the space target set to be visited. O ( n 2 ) time complexity to quickly calculate the access sequence and the duration of each transfer segment. A sequencer is constructed to generate the access sequence and the duration of each transfer segment; and an evaluator is constructed to estimate the total transfer cost of the traversal access. The actor-critic framework is used to build a reinforcement learning environment, in which the sequencer is the actor and the evaluator is the critic; the loss function of the sequencer and the evaluator is defined; the neural network is trained using an asynchronous parallel learning method, and the space target data used for training is randomly generated within the value domain, thereby enhancing the generalization level of the neural network. This method can be applied to the rapid planning of the traversal access sequence and rapid cost estimation of a single spacecraft to multiple space targets.

Description

Technical Field

[0001] The present application relates to the field of combinatorial optimization technology, and in particular to a multi-star traversal access sequence planning method and device based on an attention mechanism. Background Art

[0002] The multi-satellite traversal visit sequence planning problem is a typical time-varying traveling salesman problem. The visit refers to a space rendezvous or flyby, and the traveling salesman is an orbital transfer vehicle (OTV), which needs to visit all the satellites within a given mission time. n Each space target is a space debris, satellite or asteroid. The spatial position and velocity of OTV and each target change continuously with time, so the cost of OTV's transfer flight between targets is a function of time. The inherent combination of TSP and the unique dynamic environment of space targets result in a mixture of discrete sequence variables and continuous time variables, resulting in a huge solution space and many local optimal solutions.

[0003] There are two traditional solutions. The first is to construct it as a nonlinear programming problem, with the absolute time when each target is visited as a variable. The sequence is obtained by comparing the early and late times and optimized by the gradient method. Due to the large number of local optima and sensitivity to initial values, it is difficult to effectively obtain the optimal solution. The second is to construct it as a mixed integer optimization problem, using heuristic algorithms such as ant colony algorithm or branch and bound to optimize the access sequence, and using the gradient method to accurately optimize the transfer duration of each segment. It can converge effectively, but requires a large number of iterations and the solution efficiency is not high.

[0004] In recent years, many scholars in the field of space mission planning have studied the use of neural networks to replace traditional solutions to quickly obtain near-optimal solutions, and have achieved remarkable results in classification, regression, prediction, and game theory. In 2015, Oriol Vinyals proposed a pointer network based on long short-term memory networks and attention mechanisms, which can solve classic combinatorial optimization problems such as convex hulls, Delaunay triangulation, and TSP, but is not suitable for multi-satellite traversal visits. Summary of the invention

[0005] Based on this, it is necessary to provide a multi-star traversal access sequence planning method and device based on attention mechanism for the above technical problems. After offline training, the neural network can be used for online fast task analysis or global optimization of multi-OTV traversal access tasks for large-scale targets.

[0006] A multi-star traversal access sequence planning method based on an attention mechanism, the method comprising:

[0007] The dynamic characteristic attributes of a given space target at the initial moment of a mission include: the semi-major axis, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee and true anomaly of the target orbit at the start epoch of the mission.

[0008] An intelligent agent model based on the attention mechanism is constructed; the intelligent agent model includes a sorter and an evaluator; the sorter is used to generate an access sequence and the transfer duration of each segment; the evaluator is used to estimate the total transfer cost of multi-star traversal access.

[0009] The actor-critic framework is used to build a reinforcement learning environment and define the loss function.

[0010] According to the target set to be planned, the reinforcement learning environment and the loss function, an asynchronous multi-threaded learning method is used to train the intelligent agent model to obtain a trained intelligent agent model.

[0011] The trained intelligent agent model is applied to the rapid planning of the traversal visit sequence and transfer cost estimation of a single-orbit transfer aircraft to multiple space targets, and the traversal visit sequence planning results and transfer cost estimation results of the single-orbit transfer aircraft to multiple mobile targets are obtained.

[0012] A multi-star traversal access sequence planning device based on an attention mechanism, the device comprising:

[0013] The dynamic characteristic attribute determination module is used to determine the dynamic characteristic attributes of a given space target at the initial moment of the mission; the dynamic characteristic attributes include: the semi-major axis, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee and true anomaly of the target orbit at the start epoch of the mission.

[0014] An attention-based agent model building module is used to build an attention-based agent model; the agent model includes a sorter and an evaluator; the sorter is used to generate an access sequence and the transfer duration of each segment; the evaluator is used to estimate the total transfer cost of multi-star traversal access.

[0015] The intelligent agent model training module based on the attention mechanism is used to construct a reinforcement learning environment using an actor-critic framework and define a loss function; according to the target set to be planned, the reinforcement learning environment and the loss function, the intelligent agent model is trained using an asynchronous multi-threaded learning method to obtain a trained intelligent agent model.

[0016] The traversal access module for multiple space targets is used to apply the trained intelligent agent model to the rapid planning of the traversal access sequence and transfer cost estimation of a single-orbit transfer aircraft to multiple space targets, and obtain the traversal access sequence planning results and transfer cost estimation results of a single-orbit transfer aircraft to multiple mobile targets.

[0017] The above-mentioned multi-satellite traversal access sequence planning method and device based on the attention mechanism, the method is based on the initial dynamic properties of the space target set to be visited, The time complexity is fast to calculate the access sequence and the transfer duration of each segment. A sequencer is constructed to generate the access sequence and the transfer duration of each segment; and an evaluator is constructed to estimate the total transfer cost of the traversal access. The actor-critic framework is used to build a reinforcement learning environment, in which the sequencer is used as the actor and the evaluator is used as the critic, and the loss functions of the actor and the critic are defined. The neural network is trained using an asynchronous parallel learning method, and the space target data used for training is randomly generated within the value domain, thereby enhancing the generalization level of the neural network. This method can be applied to the rapid planning of the traversal access sequence and rapid cost estimation of a single spacecraft to multiple space targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a multi-star traversal access sequence planning method based on an attention mechanism in one embodiment;

[0019] Figure 2 A schematic diagram of hierarchical division of solutions to a multi-satellite traversal access task planning problem in another embodiment;

[0020] Figure 3 is a computational flow chart of an attention operation of an attention mechanism in another embodiment;

[0021] Figure 4 A schematic diagram of the structure of a sequencer in another embodiment;

[0022] Figure 5 A schematic diagram of a reinforcement learning environment in another embodiment;

[0023] Figure 6 A flowchart of a neural network training algorithm according to another embodiment;

[0024] Figure 7 A flowchart of application of a trained neural network in another embodiment in large-scale target selection or global optimization of multi-OTV to multi-satellite traversal rendezvous target allocation;

[0025] Figure 8 This is a structural diagram of a multi-star traversal access sequence planning device based on an attention mechanism in one embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] In one embodiment, Figure 1 As shown, a multi-star traversal access sequence planning method based on an attention mechanism is provided, and the method includes the following steps:

[0028] Step 100: The dynamic characteristic attributes of the given space target at the initial moment of the mission; the dynamic characteristic attributes include: the semi-major axis, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee and true anomaly of the target orbit at the start epoch of the mission.

[0029] Specifically, the space target is a satellite, an asteroid or near-Earth debris.

[0030] The multi-satellite traversal access sequence planning problem is constructed as a mixed integer optimization problem. The order in which the orbital transfer vehicle (OTV) visits the targets and the transfer time between the targets are used as design variables. The transfer cost is minimized under the transfer time limit and the OTV maneuverability limit. The transfer cost is usually the speed increment or propellant consumption. The solution level of the multi-satellite traversal access mission planning problem is divided into the following: Figure 2 shown.

[0031] The design variables are the target numbers in order of access:

[0032] (1)

[0033] And the duration of each transition between targets:

[0034] (2)

[0035] in, Indicates the sequence number of the target visited by the orbital transfer vehicle. is the unique number of each target, For the The duration of the segment transfer, is the mission initial epoch, For the The arrival epoch of the segment transfer.

[0036] The constraints are task time constraints:

[0037] (3)

[0038] Orbital transfer vehicle maneuverability constraints:

[0039] (4)

[0040] Access decision constraints:

[0041] (5)

[0042] in, and are the lower and upper bounds of the transfer duration of each segment, is the upper bound of the mission epoch, is the upper limit of the speed increment for a single pulse maneuver, is the upper limit of the total speed increment that can be produced by the OTV thrusters, both are given constants. The maneuverability constraint of the OTV can also be expressed by the total fuel consumption. and OTV is in the Segment transfer to Epoch The position and velocity vectors, and Respectively The target visited in the segment transfer arrives at the OTV epoch The position and velocity vectors. and are the allowed terminal position and velocity deviations respectively, representing the state matching conditions for rendezvous / flyover, and are both given constants.

[0043] If the orbit transfer vehicle has no mass changes during the flight except for maneuvers, the objective function is defined to minimize the total velocity increment in the sequential transfer:

[0044] (6)

[0045] in, OTV is the j The total velocity increment applied by the segment transfer.

[0046] The multi-satellite traversal visit sequence planning problem is formulated as a mixed integer optimization problem, with the order in which spacecraft visit targets and the transfer time between targets as design variables to minimize the transfer cost.

[0047] Determine the dynamic characteristic attributes of each space target input into the neural network in the mission scenario , Indicates the number i The properties of the target, given the range of values ​​for these properties.

[0048] Specifically, for the two-body Kepler dynamics scenario, taking multi-asteroid detection as an example, denoted as Example 1,

[0049] Let its target feature attribute be , respectively, are the semi-major axis, eccentricity, orbit inclination, right ascension of ascending node, argument of perigee and mission start epoch of the target orbit The true anomaly angle.

[0050] Step 102: Construct an intelligent agent model based on the attention mechanism; the intelligent agent model includes a sorter and an evaluator; the sorter is used to generate an access sequence and the transfer duration of each segment; the evaluator is used to estimate the total transfer cost of multi-star traversal access.

[0051] Specifically, the sorter uses a sorting embedding layer to process the dynamic feature attributes of the current target, obtains the corresponding embedded features, encodes the embedded features, and decodes them using a sorting decoder based on the attention mechanism to determine the next target to be visited, and uses the gradient method local search method to determine the transfer duration of the next target.

[0052] The evaluator maps the characteristic attributes of all targets to the characteristic attributes of the high-dimensional space through the evaluator embedding layer, encodes the characteristic attributes of the high-dimensional space, sums the encoding results to obtain a comprehensive encoding state vector, and decodes the comprehensive encoding state to obtain the evaluation value scalar of the total transfer cost.

[0053] Step 104: Use the actor-critic framework to build a reinforcement learning environment and define the loss function.

[0054] Specifically, an Actor-Critic framework is used to construct a reinforcement learning environment, in which the ranker is used as the Actor and the evaluator is used as the Critic, and the loss functions of the Actor and the Critic are defined.

[0055] Step 106: According to the target set to be planned, the reinforcement learning environment and the loss function, the intelligent agent model is trained using an asynchronous multi-threaded learning method to obtain a trained intelligent agent model.

[0056] Specifically, a target set to be planned is randomly generated according to a preset value domain of the dynamic characteristic attribute.

[0057] Before the target feature attributes are input into the neural network, all components need to be normalized within their respective value ranges.

[0058] Table 1 The value range of target feature attributes in the example

[0059]

[0060] In order to avoid the dynamic characteristics of some targets in training being too similar, which would hinder feature recognition and differentiation during training, each time a new target is generated, all attributes are normalized within their respective value ranges, and the normalized Euclidean distance between the new target and all existing targets is calculated. If there is any If the value is less than a given threshold, the target is regenerated until the condition is met. In addition, the transfer accessibility between any two targets needs to be judged, that is, whether there is a transfer scheme that meets the OTV maneuverability constraint within the mission time to ensure that each newly generated target has a reachable target.

[0061] Step 108: Apply the trained intelligent agent model to the rapid planning of the traversal access sequence and transfer cost estimation of the single-orbit transfer aircraft to multiple space targets, and obtain the traversal access sequence planning results and transfer cost estimation results of the single-orbit transfer aircraft to multiple mobile targets.

[0062] Specifically, the trained neural network is applied to the rapid planning and cost estimation of the traversal visit sequence of a single orbital transfer vehicle (OTV) to multiple space targets. The time complexity sorter realizes the fast planning of multi-star traversal access sequence, through The time complexity estimator implements a fast approximation of the total transfer cost of multi-star traversal visits.

[0063] In the above-mentioned multi-satellite traversal access sequence planning method based on the attention mechanism, the method is based on the initial dynamic properties of the space target set to be visited. The time complexity is fast to calculate the access sequence and the transfer duration of each segment. A sequencer is constructed to generate the access sequence and the transfer duration of each segment; and an evaluator is constructed to estimate the total transfer cost of the traversal access. The actor-critic framework is used to build a reinforcement learning environment, in which the sequencer is used as the actor and the evaluator is used as the critic to define the loss function of the actor and the critic. The neural network is trained using an asynchronous parallel learning method, and the space target data used for training is randomly generated within the value domain, thereby enhancing the generalization level of the neural network. This method can be applied to the rapid planning of the traversal access sequence and rapid cost estimation of a single spacecraft to multiple space targets.

[0064] In one embodiment, the dynamic characteristic attributes in step 100 also include: the average rate of change of the right ascension of the ascending node, the argument of perigee and the true anomaly with time of the target orbit under perturbation.

[0065] Specifically, in addition, consider The perturbed near-Earth scene, taking near-Earth debris removal as an example, is recorded as Example 2, and its target feature attribute is , where the last three newly added attributes are The average rate of change of the right ascension of the ascending node, the argument of perigee and the true anomaly with time under perturbation:

[0066] (7)

[0067] in, is the average radius of the Earth, is the gravitational constant.

[0068] In one embodiment, step 102 includes: constructing a sorter; the sorter includes: a sorting embedding layer, a sorting encoder, and a sorting decoder; the sorting embedding layer is constructed using MLP and residual connection, the sorting encoder is constructed using MHSA or Bi-LSTM, and the sorting decoder is a target selection probability decoder constructed using an attention mechanism; constructing an evaluator; the evaluator includes: an evaluator embedding layer, an evaluator encoder, and an evaluator decoder; the evaluator embedding layer and the evaluator encoder have the same structure as the sorting embedding layer and the sorting encoder, respectively, but have different trainable parameter values; the evaluator decoder is constructed using MLP.

[0069] Specifically, the sorter works in a loop mode. In each cycle, given the current target, the next target to be visited is selected and the transfer time is solved: first, the trajectory state of the current target and the target to be visited is extrapolated to the current moment as the input data of the neural network; second, the input data is encoded based on the multi-head self-attention (MHSA) network or the bidirectional long-short term memory (Bi-LSTM) network to extract the comprehensive characteristics of the target; then, based on the attention mechanism, an attention probability weight is assigned to each target to be visited, and the target to be visited next is determined by probability sampling; finally, a gradient method is used to determine the transfer time from the current target to the next target. After completing a cycle, the selected target is set as the current target and deleted from the target set to be selected. The sorter loops in this way until the access sequence and the transfer time of each segment are determined.

[0070] The evaluator encodes the input data based on MHSA or Bi-LSTM, and then maps the encoded data into an estimate of the total transfer cost through a multilayer perceptron (MLP).

[0071] Residual connections are used to avoid network degradation and gradient disappearance by replacing the input matrix X And the output matrix Y When using residual connections, the dimensions in the attention mechanism are usually , , , and Set to the same , which does not cause a significant loss of feature capability, then input and output The dimensions are naturally equal.

[0072] The calculation process of an attention operation of the attention mechanism is as follows Figure 3 As shown, it calculates the input vector For all input vectors The attention vector , , is the total number of input vectors. First, by using the linear transformation matrix , and The input vector Map to query space, keyword space and value space to obtain query vectors respectively , keyword vector Sum value vector :

[0073] (8)

[0074] in, , and are trainable parameters of the neural network and are Shared, the dimensions are , and ; , , and are the dimensions of input, query space, keyword space, and value space respectively; the query vector and keyword vector Represent the search information and index information respectively. Then, calculate and The matching degree between For another input Attention score ( Can be equal to ):

[0075] (9)

[0076] Among them, softmax is a normalized exponential activation function, which is used to convert the sum of each element of the input vector into 1:

[0077] (10)

[0078] A is the mining input For another input The attention function of the matching weights:

[0079] (11)

[0080] in, Five types of attention functions are represented, namely content-based, additive, general, dot-product and scaled dot-product attention functions, which are all prior art. and The dimensions of the query space and keyword space can be different. The matching matrix in W 1. W 2 and matching vector and The matching matrix in are the trainable parameters of the neural network, and their dimensions are , , and ; is the matching dimension of the attention mechanism. tanh is the hyperbolic tangent activation function, which operates on each element in the input tensor and outputs a tensor of the same dimension:

[0081] (12)

[0082] Attention score As the weight coefficient, the value vector Weighted, we get right The attention output vector :

[0083] (13)

[0084] Get all output vectors The operation is called self-attention operation, denoted by symbol S, which extracts the implicit correlation features between all input vectors. For S, the main operation in the forward calculation process and gradient back propagation process is matrix multiplication, which can be accelerated by parallel computing of GPU (Graphics Processing Units). For example, its effectiveness and computational efficiency have been verified in existing studies.

[0085] (14)

[0086] In order to improve the feature extraction capability, multiple , and The self-attention neural network is called multi-head self-attention (MHSA). Among them, a self-attention neural network is called a head. is the head number, is the total number of heads. Output Y is the sum of all header outputs:

[0087] (15)

[0088] Among them, the dimensions of the value spaces of different heads should be the same.

[0089] In one embodiment, the structure of the sequencer is as follows Figure 4 As shown, in the sorter: the target set to be planned is input into the sorting embedding layer, the feature dimension is increased through MLP, and then the residual connection is used to obtain the embedded feature:

[0090] (16)

[0091] in, For the The embedded features corresponding to the dynamic feature attributes of each target, is the number of targets in the target set to be planned, For the The dynamic characteristics of the target, MLP A multi-layer perceptron.

[0092] The embedded features are input into the sorting encoder, and MHSA or Bi-LSTM is used for preliminary encoding. The preliminary encoding results are processed by MLP after residual connection, and the encoded data obtained after residual connection is:

[0093] (17)

[0094] in, For the The encoded data of the target, For the The preliminary encoding result of each target is the result of residual connection. is a multi-head attention module, It is a Bi-LSTM module.

[0095] The encoded data is input into the sorting decoder and decoded using the attention mechanism, and the probability of the target being selected in a transition is obtained as:

[0096] (18)

[0097] (19)

[0098] (20)

[0099] (twenty one)

[0100] (twenty two)

[0101] in, For the The probability of being selected in the segment transition, For the j Segment transfer The probability of selecting a target is is the temperature coefficient, For the j In the segment transfer i The attention weight of each target, is the attention weight of each optional target after the selected target is screened out, and is a linear change matrix, which is a trainable parameter of the sorting decoder; To extract the Segment transfer current target and target The attention function of the matching weights between To mark the Whether a target has been selected into the sequence.

[0102] According to The probability of the segment being selected completes the selection of the corresponding segment transfer target, and obtains the number of the target to be accessed by the segment transfer. ; Update the current target to the target , update the current time to ; According to the current state data of the current target and all candidates at the current moment, select the next target to be visited; According to the next target to be visited, use the gradient method to locally search the transfer duration and calculate the transfer cost.

[0103] In one embodiment, according to the next target to be visited, a gradient method is used to locally search for the transfer duration and the transfer cost is calculated, including: according to the next target to be visited, a gradient method is used to locally search for the transfer duration; according to the orbital state, departure epoch and transfer duration of the current target and the next target to be visited, an approximate value of the optimal transfer speed increment is determined as:

[0104] (twenty three)

[0105] in, For the Segment transfer speed increment, For the Segment transfer duration, For the The access target of the segment transfer, It is the optimal transfer speed increment estimation method.

[0106] Specifically, the sequencer works in a cyclic manner. In one cyclic step, the attribute values ​​of the current target and the candidate target at the current epoch are input into the sequencer, and the sequencer determines the next target to be visited and calculates the transfer time of the segment. The following is a detailed description of one cyclic step.

[0107] The current target and the orbital states of the candidate targets that have not yet been visited are extrapolated to At this moment, get the target dynamic characteristic data at the current moment , which does not include targets that have been visited. To mark the Whether the target has been selected into the sequence, if not yet selected, , if it has been selected into the sequence, .

[0108] The dynamic characteristics of each target are analyzed through an MLP The embedding is performed separately, which is called the embedding layer MLP. MLP is a prior art, and its details are given in the Appendix. The input of the embedding layer MLP is the dynamic characteristics of a certain target , the output is the embedding feature corresponding to the target .enter and output The function of the embedding layer MLP is to increase the dimension of features, usually embedding the dimension of features Much larger than the dynamic characteristics The dimension of . Using residual connection, the input vector Add to output vector , and perform layer normalization.

[0109] The candidate target comprehensive encoder is constructed by MHSA or Bi-LSTM, which is called the sorting encoder. MHSA and Bi-LSTM are prior arts, and their details are given in the appendix. The input of the sorting encoder is the embedded features corresponding to each target. , the output is the preliminary encoding data corresponding to each target . Let the internal linear mapping dimensions of MHSA or Bi-LSTM be d , then each preliminary coded data The dimensions are d Using residual connection, the input vector Add to output vector , and perform layer normalization. Each preliminary encoded data is encoded through an encoding MLP Mapping to encoded data , the dimension of MLP is also d . Using residual connection, the input vector Add to output vector , and perform layer normalization.

[0110] The target selection probability decoder is constructed through the attention mechanism, called the ranking decoder. The encoded data of the current target Map to the query space and obtain the query vector corresponding to the target . Through the linear transformation matrix The encoded data of the target to be selected Map them to the keyword space respectively to obtain the keyword vectors corresponding to each candidate target . Among them, the linear transformation matrix and are trainable parameters of the neural network.

[0111] (twenty four)

[0112] Calculating the query vector and keyword vector Attention score:

[0113] (25)

[0114] in, Is to extract Segment transfer current target and target The attention function of the matching weights between . Make the first Segment transfer The probability of selecting a target Approximately 0. Softmax is a normalized exponential activation function that converts the sum of each element of the input vector into 1:

[0115] (26)

[0116] Hyperparameters is the temperature coefficient, when The larger its value, the j The probability of different candidate targets being selected in segment transfer The smaller the difference between , the more exploratory the algorithm is. Usually, let It gradually decays during training.

[0117] according to Complete Select the target of the segment transfer and obtain the The number of the target to be accessed by the segment transfer . Update the current target to the target , update the current time to .

[0118] Let all trainable parameters of the sorter be .

[0119] The above neural network selects the next target to be visited based on the current state data of the current target and all candidates at the current moment. After determining the next target to be visited, it is necessary to further determine the optimal transfer time between the two targets. And calculate the transfer costs.

[0120] The transfer time is searched locally by the gradient method. First, the single-segment transfer speed increment is given. The fast approximation method is a prior art and will not be elaborated here. The orbital state, departure epoch and transfer duration of the current target and the next target are input, and the approximate value of the optimal transfer speed increment is output as shown in formula (23).

[0121] In the j Access target of segment transfer After determination, the numerical optimization algorithm based on gradient descent is used to locally optimize the Segment transfer duration and Segment transfer duration ;in, The initial value of is the calculation result in the previous cycle. The initial value of is randomly generated within the value range; after local optimization, Save, As the initial value for the next local optimization. , only locally optimize the first transfer duration .

[0122] The pseudo code of the overall loop process of the sorter's neural network and local optimization of transfer time is as follows:

[0123] Step 1: Initialization ;

[0124] Step 2: The dynamic state of each candidate target X Extrapolate to ,have to ;

[0125] Step 3: In the interval Randomly generated ;

[0126] Step 4: and Input sorter, get ;

[0127] Step 5: Optimize using gradient descent make ;

[0128] Step 6:

[0129] Step 7: Extrapolate to ,have to ;

[0130] Step 8: and Input sorter, get ;

[0131] Step 9: In the interval Randomly generated ;

[0132] Step 10: Optimize using gradient descent and make

[0133] ;

[0134] Step 11: , , and Save to ;

[0135] Step 12:

[0136] Step 13: , and Save to ;

[0137] Step 14: ;

[0138] Step 15: ;

[0139] In one embodiment, in the evaluator: inputting the characteristic attributes of all targets into the evaluator embedding layer to obtain the characteristic attributes of the high-dimensional space;

[0140] (27)

[0141] in, For the The dynamic characteristics of the target, For the The embedding vector corresponding to the dynamic characteristic attributes of each target, MLP is a multi-layer perceptron, The number of targets in the target set to be planned.

[0142] The high-dimensional spatial feature attributes are encoded using the evaluator encoder, and the encoding results are summed to obtain the comprehensive encoding state vector:

[0143] (28)

[0144] (29)

[0145] (30)

[0146] in, For the The encoded data of the target, For the The preliminary encoding result of each target is the result of residual connection. is a multi-head attention module, It is a Bi-LSTM module.

[0147] The comprehensive encoded state vector is input into the evaluator decoder and decoded using the MLP module to obtain the evaluation value scalar of the total transfer cost.

[0148] Specifically, the evaluator has an embedding layer and an encoder with the same structure as the sorter but with different trainable parameter values, which are called the evaluator embedding layer and the evaluator encoder respectively. Mapping to high-dimensional space . The estimator encoder converts Mapping to encoded data ,Will The sum is obtained to obtain the comprehensive coding state vector as shown in formula (28).

[0149] The estimator decoder is an MLP that synthesizes the encoded state Mapped to a scalar value of the total transfer cost .

[0150] Let all trainable parameters of the estimator be .

[0151] The total transfer cost estimation operation of the evaluator is independent of the ranking operation of the ranker, but the two can build a reinforcement learning environment based on the actor-critic framework for unsupervised training.

[0152] In one embodiment, step 106 includes: constructing an actor-critic architecture reinforcement learning environment; wherein the actor is a sorter and the critic is an evaluator; and defining the loss function of the sorter as an advantage function policy gradient loss function:

[0153] (31)

[0154] in, is the actor’s loss function, is the label probability, which is a one-hot vector. is 1 and the rest of the elements are 0; is the first sample of the sequencer The number of the target to be accessed by the segment transfer, For the The probability of being selected in the segment transition, For the Segment transfer The probability of selecting a target is is the advantage function, which quantifies the update direction and magnitude of the sampling probability. A When the value of is negative, update Thus minimizing will make the index number The goal is The probability of being selected in the segment transfer approaches 1; when A When the value of is positive, update This will bring the probability closer to 0; is the estimated value of the transfer cost output by the evaluator, The total transfer cost corresponding to the sequence output by the sorter, are all trainable parameters of the sorter.

[0155] The loss function of the estimator is defined as the mean square error loss function:

[0156] (32)

[0157] in, is the critic’s loss function, are all trainable parameters of the estimator.

[0158] According to the actor's loss function and the critic's loss function, the loss function of the agent model is obtained.

[0159] Specifically, an Actor-Critic framework is used to construct a reinforcement learning environment and define a loss function.

[0160] Let the sequencer be Actor, and input the randomly generated Output access sequence , Transfer duration and transfer costs Let the evaluator be Critic, which quantitatively evaluates the performance of the Actor. After sufficient training, the evaluation value can be used as an approximation of the total transfer cost of the optimal traversal visit The Actor and the Critic form an interactive relationship during training, and the transfer cost corresponding to the sequence obtained by the Actor is It is the label of the critic’s supervised learning, which improves the accuracy of the critic’s evaluation; the performance value of the critic’s evaluation It is the baseline for measuring the direction and span of the update of the Actor's trainable parameters. The schematic diagram of the reinforcement learning environment is as follows Figure 5 shown.

[0161] The loss function of Actor is defined as shown in formula (31).

[0162] The definition of the critic’s loss function is shown in Equation (32). and The MSE between the trainable parameters of the network optimization means making the total transfer cost of the Critic evaluation Actual optimal cost J near.

[0163] In one embodiment, step 106 includes: defining a global agent and local agents with the same number of parallel processes according to the agent model; all local agents have the same structure as the global agent; in a training round, the global agent copies the trainable parameters of the sorter and the evaluator to the local agents; inputting the target set to be planned into the sorter and the evaluator of the local agent to obtain the selection probability , access sequence , Transfer duration of each segment , total transfer cost J and the estimated total transfer cost ; The probability of selection , access sequence , Transfer duration of each segment , total transfer cost J and the estimated total transfer cost The experience data is packaged into an experience data and stored in the experience pool. If the number of experience data stored in the experience pool is greater than the preset upper limit, the earliest stored experience is deleted. If the number of experience data stored in the experience pool is greater than the preset lower limit, experience data is randomly sampled from the experience pool to train the sorter and evaluator, and the sorter loss function and the evaluator loss function are calculated to obtain the total loss of the local agent. The total loss of the local agent is back-propagated within the local agent, and the obtained gradient is passed back to the global agent. The trainable parameters of the global agent are updated using the adaptive parameter optimizer, and one round of training is completed, and the next training round is entered. When the upper limit of the number of training rounds is reached or the transfer cost is reached, J When the improvement rate of is less than a given value within 100 rounds, the training is stopped and the trained agent model is obtained.

[0164] Specifically, an asynchronous parallel learning method is used to train the neural network. The value range of the target feature attribute is defined. In order to enhance the generalization level of the neural network, the number of spatial targets and attribute values ​​in the training data are randomly generated within the value range. The training algorithm flow of the neural network is as follows: Figure 6 As shown. The agent model contains a sorter and an evaluator. Define a global agent and local agents with the same number of parallel processes, where all local agents have the same structure as the global agent.

[0165] In a training round, a local agent performs a complete multi-star traversal visit sequence planning. First, randomly generate the target attribute value to be visited , forming a randomly initialized environment. The global agent will have trainable parameters and Copy to the local agent. Input the sorter and evaluator of the local agent respectively to get the selection probability , access sequence , Transfer duration of each segment , total transfer cost and the estimated total transfer cost , pack the result data into an experience data and store it in the experience pool. If the number of experience data stored in the experience pool is greater than the given upper limit, delete the earliest stored experience. If the number of experience data stored in the experience pool is greater than the given lower limit, randomly sample experience data from the experience pool to train the neural network: Calculate the loss function and ; Perform gradient backpropagation within the local agent and propagate the resulting gradient back to the global agent; Use adaptive parameter optimizers such as AdaDelta or Adam to update the trainable parameters of the global agent, which is not limited to a specific parameter optimizer.

[0166] When the upper limit of the number of training rounds or the transfer cost is reached J When the improvement rate is less than a given value within 100 rounds, stop training. The pseudo code of the neural network training algorithm is:

[0167] Step 1: Initialize the global agent ;

[0168] Step 2: Initialize the global agent parameter optimizer ;

[0169] Step 3: Initialize multiple local agents ;

[0170] Step 4: Start a process for each local agent;

[0171] Step 5:

[0172] Step 6: ;

[0173] Step 7: Randomly generate dynamic properties with access target set ;

[0174] Step 8: Get through the sorter , , and ;

[0175] Step 9: Get through the evaluator ;

[0176] Step 10: , , , and Package and store in experience pool;

[0177] Step 11: if the experience pool is too full: delete the earliest experience data;

[0178] Step 12: If the experience pool stores enough experience data:

[0179] Step 13: Randomly sample a batch of data from the experience pool;

[0180] Step 14: Calculate each piece of data separately and ;

[0181] Step 15: Gradient back propagation and ;

[0182] Step 16: ;

[0183] Step 17: Update via parameter optimizer and ;

[0184] Step 18: ;

[0185] Step 19: ;

[0186] Step 20: Close the process;

[0187] The program is run on a personal computer with an NVIDIA GeForce RTX 4080 GPU with 48T floating-point operations (FLOPS) and an Intel i9-12900H CPU with 14 cores, 20 threads, 5.0GHz frequency, and 64GB memory. The neural network code is written based on Pytorch 1.11.0. The FLOPS inside the sorter and evaluator are allocated to the GPU based on the Compute Unified Device Architecture (CUDA) 10.2, and other logical operations are performed by the CPU.

[0188] The hyperparameter configuration of the neural network in the embodiment is shown in Table 2. When randomly generating samples, it may happen that the transfer between two targets cannot meet the time constraint and the maneuver constraint at the same time. At this time, the sample is regenerated until there is a transferable relationship with more than half of the other targets.

[0189] Table 2 Hyperparameter configuration of neural network

[0190]

[0191] The trained sorter and evaluator neural networks were compared with the ant colony algorithm for 10,000 independent tests. In each test, the spatial target data to be accessed was randomly generated. The relative error between the neural network output solution index and the ant colony algorithm solution index was calculated. , and divide it into , , , , and Six levels. For the sorter, , for the evaluator, Table 3 gives the statistical results of 10,000 independent tests.

[0192] Table 3 Statistics of neural network test results

[0193]

[0194] In one embodiment, step 108 includes: for a single orbit transfer vehicle, using a sequencer in a trained agent model to generate a near-optimal visit sequence for a set of multiple space targets, using an evaluator in the trained agent model to estimate a total transfer cost for the entire sequence J .

[0195] Specifically, for any given set of targets, the trained sorter can be used to directly generate a near-optimal access sequence { c j}, the trained estimator can be used to directly estimate the total transfer cost for the entire sequence J Both the sorter and the evaluator can be used for fast task analysis or global optimization of collaborative traversal visits of multiple OTVs to a large number of candidate targets, thereby searching for the optimal allocation scheme for assigning candidate targets to multiple OTVs or the optimal selection scheme for selecting targets to be visited from a large candidate set. The application process of the trained neural network in large-scale target selection or global optimization of multi-OTV traversal rendezvous target allocation is as follows: Figure 7 Embedding the trained ranker or evaluator into the global optimization of target assignment or target selection can reduce the overall complexity of the problem and reduce the search space.

[0196] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0197] In one embodiment, Figure 8 As shown, a multi-star traversal access sequence planning device based on an attention mechanism is provided, comprising: a dynamic feature attribute determination module, an agent model construction module based on an attention mechanism, an agent model training module based on an attention mechanism, and a traversal access module for multiple space targets, wherein:

[0198] The dynamic characteristic attribute determination module is used to determine the dynamic characteristic attributes of a given space target at the initial moment of the mission; the dynamic characteristic attributes include: the semi-major axis, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee and true anomaly of the target orbit at the start epoch of the mission.

[0199] The attention-based agent model building module is used to build an attention-based agent model; the agent model includes a sorter and an evaluator; the sorter is used to generate access sequences and the transfer duration of each segment; the evaluator is used to estimate the total transfer cost of multi-star traversal access.

[0200] The intelligent agent model training module based on the attention mechanism is used to construct a reinforcement learning environment using the actor-critic framework and define a loss function; according to the target set to be planned, the reinforcement learning environment and the loss function, the intelligent agent model is trained using an asynchronous multi-threaded learning method to obtain a trained intelligent agent model.

[0201] The traversal access module for multiple space targets is used to apply the trained intelligent agent model to the rapid planning of the traversal access sequence and transfer cost estimation of a single-orbit transfer aircraft to multiple space targets, and obtain the traversal access sequence planning results and transfer cost estimation results of a single-orbit transfer aircraft to multiple mobile targets.

[0202] In one of the embodiments, the dynamic characteristic attributes in the dynamic characteristic attribute determination module also include: the average change rate of the ascending node right ascension, perigee argument and true anomaly angle of the target orbit under perturbation over time.

[0203] In one of the embodiments, the intelligent agent model building module based on the attention mechanism is also used to build a sorter; the sorter includes: a sorting embedding layer, a sorting encoder, and a sorting decoder; the sorting embedding layer is constructed using MLP and residual connection, the sorting encoder is constructed using MHSA or Bi-LSTM, and the sorting decoder is a target selection probability decoder constructed using the attention mechanism; an evaluator is constructed; the evaluator includes: an evaluator embedding layer, an evaluator encoder, and an evaluator decoder; the evaluator embedding layer and the evaluator encoder have the same structure as the sorting embedding layer and the sorting encoder, respectively, but different trainable parameter values; the evaluator decoder is constructed using MLP.

[0204] In one embodiment, the intelligent agent model building module based on the attention mechanism is also used to: input the target set to be planned into the sorting embedding layer, perform feature dimension upgrading through MLP, and then use residual connection to obtain the embedded features as shown in formula (16); input the embedded features into the sorting encoder, use MHSA or Bi-LSTM for preliminary encoding, use residual connection to process the preliminary encoding result through MLP, and obtain the encoded data after residual connection as shown in formula (17); input the encoded data into the sorting decoder, use the attention mechanism for decoding, and obtain the probability of the target being selected in a transition as shown in formulas (18) to (22); according to the first The probability of the segment being selected completes the selection of the corresponding segment transfer target, and obtains the number of the target to be accessed by the segment transfer. ; Update the current target to the target , update the current time to ; According to the current state data of the current target and all candidates at the current moment, select the next target to be visited; According to the next target to be visited, use the gradient method to locally search the transfer duration and calculate the transfer cost.

[0205] In one of the embodiments, the attention-based agent model building module is also used to locally search for the transfer duration using a gradient method according to the next target to be visited; and to determine the approximate value of the optimal transfer speed increment according to the orbital state, departure epoch, and transfer duration of the current target and the next target to be visited, as shown in formula (23).

[0206] In one of the embodiments, the attention mechanism-based agent model building module is also used in the evaluator: input the characteristic attributes of all targets into the evaluator embedding layer to obtain the high-dimensional space characteristic attributes as shown in formula (27); encode the high-dimensional space characteristic attributes using the evaluator encoder, and add the encoding results to obtain the comprehensive encoding state vector as shown in formula (28); input the comprehensive encoding state vector into the evaluator decoder, and decode it using the MLP module to obtain the evaluation value scalar of the total transmission cost.

[0207] In one embodiment, the attention-based agent model training module is also used to construct an actor-critic architecture reinforcement learning environment, wherein the actor is a sorter and the critic is an evaluator;

[0208] The actor's loss function is defined as the advantage function policy gradient loss function as shown in formula (31); the critic's loss function is defined as the mean square error loss function as shown in formula (32); based on the actor's loss function and the critic's loss function, the loss function of the intelligent agent model is obtained.

[0209] In one embodiment, the agent model training module based on the attention mechanism is also used to define a global agent and local agents with the same number of parallel processes according to the agent model; wherein all local agents have the same structure as the global agent; in a training round, the global agent copies the trainable parameters of the sorter and the evaluator to the local agent; the target set to be planned is input into the sorter and the evaluator of the local agent to obtain the selection probability, the access sequence, the transfer duration of each segment, the total transfer cost and the estimated total transfer cost; the selection probability, the access sequence, the transfer duration of each segment, the total transfer cost and the estimated total transfer cost are packaged as an experience data and stored in the experience pool, and if the experience pool stores If the number of experience data is greater than the preset upper limit, the earliest stored experience is deleted; if the number of experience data stored in the experience pool is greater than the preset lower limit, experience data is randomly sampled from the experience pool to train the sorter and evaluator, and the sorter loss function and the evaluator loss function are calculated to obtain the total loss of the local agent; the total loss of the local agent is back-propagated by gradient within the local agent, and the obtained gradient is passed back to the global agent; the trainable parameters of the global agent are updated using the adaptive parameter optimizer, one round of training is completed, and the next training round is entered; when the upper limit of the number of training rounds is reached or the improvement rate of the transfer cost is less than a given value within 100 rounds, the training is stopped to obtain a trained agent model.

[0210] In one embodiment, the traversal access module for multiple space targets is further used to generate a near-optimal access sequence for a single orbit transfer vehicle to a set of multiple space targets using a sequencer in a trained agent model, and to estimate a total transfer cost of the entire sequence using an evaluator in the trained agent model. J .

[0211] For the specific limitations of the multi-star traversal access sequence planning device based on the attention mechanism, please refer to the limitations of the multi-star traversal access sequence planning method based on the attention mechanism above, which will not be repeated here. Each module in the above-mentioned multi-star traversal access sequence planning device based on the attention mechanism can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0212] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A multi-star traversal access sequence planning method based on attention mechanism, characterized in that: The method comprises: The dynamic characteristic attributes of the given space target at the initial time of the mission; the dynamic characteristic attributes include: the semi-major axis, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee and true anomaly of the target orbit at the start epoch of the mission; Constructing an intelligent agent model based on the attention mechanism; the intelligent agent model includes a sorter and an evaluator; the sorter is used to generate an access sequence and the transfer duration of each segment; the evaluator is used to estimate the total transfer cost of multi-star traversal access; Adopt the actor-critic framework to build a reinforcement learning environment and define the loss function; According to the target set to be planned, the reinforcement learning environment and the loss function, an asynchronous multi-threaded learning method is used to train the intelligent agent model to obtain a trained intelligent agent model; The trained agent model is applied to the rapid planning of the traversal access sequence and transfer cost estimation of a single-orbit transfer aircraft to multiple space targets, and the traversal access sequence planning results and transfer cost estimation results of a single-orbit transfer aircraft to multiple mobile targets are obtained; Among them, the construction of an intelligent agent model based on the attention mechanism includes: Construct the sorter; the sorter includes: a sorting embedding layer, a sorting encoder, and a sorting decoder; the sorting embedding layer is constructed by MLP and residual connection, the sorting encoder is constructed by MHSA or Bi-LSTM, and the sorting decoder is a target selection probability decoder constructed by attention mechanism; the sorter uses the sorting embedding layer to process the dynamic characteristic attributes of the current target, obtains the corresponding embedding features, encodes the embedded features, and decodes them by using a sorting decoder based on the attention mechanism to determine the next target to be visited, and uses the gradient method local search method to determine the transfer time of the next target; Construct the evaluator; the evaluator includes: an evaluator embedding layer, an evaluator encoder and an evaluator decoder; the evaluator embedding layer and the evaluator encoder have the same structure as the sorting embedding layer and the sorting encoder respectively, but have different trainable parameter values; the evaluator decoder is constructed using MLP; the evaluator maps the characteristic attributes of all targets to high-dimensional space characteristic attributes through the evaluator embedding layer, encodes the high-dimensional space characteristic attributes, sums the encoding results to obtain a comprehensive encoding state vector, and decodes the comprehensive encoding state to obtain an evaluation value scalar of the total transfer cost.

2. The method according to claim 1, characterized in that The dynamic characteristic attributes also include: the average change rate of the ascending node right ascension, perigee argument and true anomaly angle of the target orbit over time under perturbation.

3. The method according to claim 1, characterized in that In the sequencer: The target set to be planned is input into the sorting embedding layer, and the feature dimension is increased by MLP. Then, the residual connection is used to obtain the embedded feature: ； in, For the i The embedded features corresponding to the dynamic feature attributes of each target, i =1,2,…, n , n is the number of targets in the target set to be planned, For the i The dynamic characteristics of the target, MLP is a multi-layer perceptron; The embedded features are input into the sorting encoder, and MHSA or Bi-LSTM is used for preliminary encoding. The preliminary encoding result is processed by MLP after residual connection, and the encoded data obtained after residual connection is: ； in, For the i The encoded data of each target, For the i The preliminary encoding result of each target is the result of residual connection. is a multi-head attention module, It is a Bi-LSTM module; The encoded data is input into the sorting decoder and decoded using the attention mechanism to obtain the probability of the target being selected in a transition: ； ； ； ； ； in, For the j The probability of being selected in the segment transition, For the j Segment transfer i The probability of selecting a target is T s is the temperature coefficient, For the j In the segment transfer i The attention weight of each target, is the attention weight of each optional target after the selected target is screened out, W q and W k is a linear change matrix, which is a trainable parameter of the sorting decoder; A is the extraction of the j Segment transfer current target and target { x i }, f i To mark the i Whether a target has been selected into the sequence; According to j The probability of being selected in the segment completes the selection of the corresponding segment transfer target, and obtains the number of the target to be visited by the segment transfer c j+1 ; Update the current target to the target c j+1 , update the current time to t j ; Select the next target to be visited based on the current state data of the current target and all targets to be selected; According to the next target to be visited, the gradient method is used to locally search the transfer duration and calculate the transfer cost.

4. The method according to claim 3, characterized in that According to the next target to be visited, the gradient method is used to locally search the transfer duration and calculate the transfer cost, including: According to the next target to be visited, the gradient method is used to locally search the transfer duration; According to the orbital status of the current target and the next target to be visited, the departure epoch and the transfer duration, the approximate value of the optimal transfer speed increment is determined as: ； in, For the j Segment transfer speed increment, For the j Segment transfer duration, c j+1 For the j The access target of the segment transfer, It is the optimal transfer speed increment estimation method.

5. The method according to claim 1, characterized in that In the evaluator: Inputting the characteristic attributes of all targets into the evaluator embedding layer to obtain high-dimensional spatial characteristic attributes; ； in, For the i The dynamic characteristics of the target, For the i The embedding vector corresponding to the dynamic characteristic attributes of each target, MLP is a multi-layer perceptron, i =1,2,…, n , n The number of targets to be planned; The high-dimensional spatial feature attributes are encoded using the evaluator encoder, and the encoding results are summed to obtain a comprehensive encoding state vector: ； ； in, For the i The encoded data of each target, For the i The preliminary encoding result of each target is the result of residual connection. is a multi-head attention module, It is a Bi-LSTM module; The integrated coded state vector is input into the evaluator decoder and decoded by using an MLP module to obtain an evaluation value scalar of the total transfer cost.

6. The method according to claim 1, characterized in that The actor-critic framework is used to build a reinforcement learning environment and define the loss function, including: Constructing an actor-critic architecture reinforcement learning environment, wherein the actor is the ranker and the critic is the evaluator; The loss function of the sorter is defined as the advantage function policy gradient loss function: (1) in, L s is the actor’s loss function, is the label probability, c j is the first sample of the sequencer j The number of the target to be accessed by the segment transfer, For the j The probability of being selected in the segment transition, For the j Segment transfer i The probability of selecting a target is A = J - Q e is the advantage function, Q e is the estimated value of the transfer cost output by the evaluator, J The total transfer cost corresponding to the sequence output by the sorter, θ s are all trainable parameters of the sorter; The loss function of the estimator is defined as the mean square error loss function: ； in, L e is the critic’s loss function, θ e are all trainable parameters of the estimator; According to the loss function of the actor and the loss function of the critic, a loss function of the agent model is obtained.

7. The method according to claim 1, characterized in that According to the target set to be planned, the reinforcement learning environment and the loss function, the agent model is trained by using an asynchronous multi-threaded learning method to obtain a trained agent model, including: According to the agent model, a global agent and local agents having the same number as the parallel processes are defined; wherein all the local agents have the same structure as the global agent; In a training round, the global agent copies the trainable parameters of the sorter and the evaluator to the local agents; Input the target set to be planned into the sorter and evaluator of the local agent to obtain the selection probability, access sequence, transfer duration of each segment, total transfer cost and estimated total transfer cost; The selection probability, access sequence, transfer duration of each segment, total transfer cost and estimated total transfer cost are packaged into one piece of experience data and stored in the experience pool. If the number of experience data stored in the experience pool is greater than the preset upper limit, the earliest stored experience is deleted; if the number of experience data stored in the experience pool is greater than the preset lower limit, experience data is randomly sampled from the experience pool to train the sorter and evaluator, calculate the sorter loss function and the evaluator loss function, and obtain the total loss of the local agent; Perform gradient backpropagation on the total loss of the local agent within the local agent, and propagate the resulting gradient back to the global agent; use the adaptive parameter optimizer to update the trainable parameters of the global agent, complete one round of training, and enter the next training round; When the upper limit of the number of training rounds is reached or the improvement rate of the transfer cost is less than a given value within 100 rounds, the training is stopped and a trained agent model is obtained.

8. The method according to claim 1, characterized in that The trained agent model is applied to the rapid planning of the traversal access sequence and transfer cost estimation of a single-orbit transfer aircraft to multiple space targets, and the traversal access sequence planning results and transfer cost estimation results of a single-orbit transfer aircraft to multiple mobile targets are obtained, including: For a single-orbit transfer vehicle to a set of multiple space targets, the sequencer in the trained agent model is used to generate a near-optimal visit sequence, and the evaluator in the trained agent model is used to estimate the total transfer cost of the entire sequence.

9. A multi-star traversal access sequence planning device based on attention mechanism, characterized in that: The device comprises: A dynamic characteristic attribute determination module is used to determine the dynamic characteristic attributes of a given space target at the initial time of the mission; the dynamic characteristic attributes include: the semi-major axis, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee and true anomaly of the target orbit at the start epoch of the mission; An attention-based agent model building module is used to build an attention-based agent model; the agent model includes a sequencer and an evaluator; the sequencer is used to generate an access sequence and the duration of each transfer segment; the evaluator is used to estimate the total transfer cost of multi-star traversal access; An agent model training module based on an attention mechanism is used to construct a reinforcement learning environment using an actor-critic framework and define a loss function; according to the target set to be planned, the reinforcement learning environment and the loss function, an asynchronous multi-threaded learning method is used to train the agent model to obtain a trained agent model; The traversal access module for multiple space targets is used to apply the trained intelligent agent model to the rapid planning of the traversal access sequence and transfer cost estimation of a single-orbit transfer aircraft to multiple space targets, and obtain the traversal access sequence planning results and transfer cost estimation results of a single-orbit transfer aircraft to multiple mobile targets; Among them, the intelligent agent model construction module based on the attention mechanism is also used to construct the sorter; the sorter includes: a sorting embedding layer, a sorting encoder, and a sorting decoder; the sorting embedding layer is constructed using MLP and residual connection, the sorting encoder is constructed using MHSA or Bi-LSTM, and the sorting decoder is a target selection probability decoder constructed using the attention mechanism; the sorter uses the sorting embedding layer to process the dynamic feature attributes of the current target, obtains the corresponding embedded features, encodes the embedded features, and uses the sorting decoder based on the attention mechanism to decode, determines the next target to be visited, and uses The gradient method local search method determines the transfer time of the next target; constructs the evaluator; the evaluator includes: an evaluator embedding layer, an evaluator encoder and an evaluator decoder; the evaluator embedding layer and the evaluator encoder have the same structure as the sorting embedding layer and the sorting encoder respectively, but have different trainable parameter values; the evaluator decoder is constructed using MLP; the evaluator maps the characteristic attributes of all targets to high-dimensional space characteristic attributes through the evaluator embedding layer, encodes the high-dimensional space characteristic attributes, sums the encoding results to obtain a comprehensive encoding state vector, and decodes the comprehensive encoding state to obtain an evaluation value scalar of the total transfer cost.

Images