A communication path planning method for large-scale low-orbit constellation with sudden interference

Abstract

The present invention discloses a method for planning a low-orbit large-scale constellation communication path for sudden interference. The method includes the following steps: splitting a file to be transmitted to obtain sub-files to be transmitted, and constructing an intelligent agent based on the sub-files to be transmitted; constructing an intelligent agent's policy network, evaluation network, target policy network, and target evaluation network based on the intelligent agent group, and generating a preliminary low-orbit large-scale constellation communication path based on the intelligent agent's policy network, evaluation network, target policy network, and target evaluation network; obtaining sudden interference information, and dynamically adjusting the preliminary low-orbit large-scale constellation communication path based on the sudden interference information to obtain a low-orbit large-scale constellation communication path for sudden interference. The present invention improves the efficiency of large file transmission, reduces the possibility of comprehensive information interception, and can respond to sudden interference by dynamically adjusting or reselecting the communication path.

Description

Technical Field

[0001] The present invention relates to the field of communication path planning, and in particular to a method for planning communication paths for low-orbit large-scale constellations facing sudden interference. Background Art

[0002] The number of low-orbit satellites has begun to increase, but existing methods are still insufficient for using low-orbit satellites to transmit large files over very long distances. Currently, existing methods have proposed an optimization method for low-orbit constellation communication path planning based on the Dijkstra algorithm. This method transforms the problem of finding the optimal communication path into the problem of finding the shortest weighted path. Specifically, the low-orbit satellite network is abstracted into a weighted graph, and then the Dijkstra algorithm is used to plan the shortest path for each mission. This method has some effectiveness in communication path planning. However, this method has the following disadvantages:

[0003] (1) This method places each task on the same communication path for transmission. When the task contains large files, the transmission efficiency is slow. At the same time, since all the information of a task is on the same communication path, there is a risk of complete information interception, which has low security.

[0004] (2) This method is based on the shortest path weighted method and only considers distance, load and task priority. When faced with sudden interference, it cannot dynamically adjust the communication path.

[0005] Therefore, there is an urgent need in this field for a communication path planning method that can transmit large files on low-orbit satellites in the face of sudden interference and can solve the problems of transmission efficiency and security. Summary of the Invention

[0006] In response to the above-mentioned deficiencies in the prior art, the present invention provides a method for planning low-orbit large-scale constellation communication paths resistant to sudden interference.

[0007] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0008] A method for planning a communication path for a large-scale low-orbit constellation in the presence of sudden interference, comprising the following steps:

[0009] S1. Split the file to be transferred to obtain sub-files to be transferred, and construct an agent based on the sub-files to be transferred;

[0010] S2. Build the agent's strategy network, evaluation network, target strategy network, and target evaluation network based on the agent group, and generate a preliminary low-orbit large-scale constellation communication path based on the agent's strategy network, evaluation network, target strategy network, and target evaluation network;

[0011] S3. Obtain burst interference information, and dynamically adjust the preliminary low-orbit large-scale constellation communication path based on the burst interference information to obtain a low-orbit large-scale constellation communication path that is resistant to burst interference.

[0012] Furthermore, in step S1, the basic information carried by the agent includes position coordinate information, speed information and environment information.

[0013] Furthermore, in step S2, a preliminary low-orbit large-scale constellation communication path is generated based on the agent's policy network, evaluation network, target policy network, and target evaluation network, including the following steps:

[0014] A1. Randomly initialize the network parameters and build an experience pool;

[0015] A2. Obtain the state information of the agent to form the observation information of the agent at the current time step;

[0016] A3. Based on the agent's observation information at the current time step, the agent's policy network is used to generate the agent's execution action at the current time step. The agent's reward at the current time step and the agent's observation information at the next time step are obtained based on the agent's execution action at the current time step.

[0017] A4. Store the agent's observation information, executed actions, rewards at the current time step, and observation information at the next time step into the experience pool;

[0018] A5. Repeat steps A2-A4 until the experience pool is fully stored, and randomly sample the experience pool to obtain a sample set;

[0019] A6. Based on the sample set, use the agent's evaluation network to calculate the agent's reward estimate, and use the agent's target policy network and target evaluation network to calculate the agent's reward target value;

[0020] A7. Based on the agent's observation information, executed actions, reward estimates, and reward targets, update the agent's policy network parameters, evaluation network parameters, target policy network parameters, and target evaluation network parameters until the set update threshold conditions are met to obtain the agent's final policy network.

[0021] A8. Use the agent’s final policy network to generate preliminary low-orbit large-scale constellation communication paths.

[0022] Furthermore, in step A7, the loss function of the evaluation network of the agent is calculated based on the estimated reward value and the target reward value of the agent to update the evaluation network parameters of the agent, which is expressed as:

[0023]

[0024] Where: L(θ t+1，m，critic ) is the loss function of the evaluation network of the mth agent at the next time step t+1, θ t+1,m,critic is the network parameter of the evaluation network of the mth agent at the next time step t+1, E[·] is the expectation operator, is the reward estimate of the evaluation network of the mth agent at the current time step t, y is the target value generated by the target evaluation network, r t，m is the reward of the mth agent at the current time step t, γ is the discount factor, The reward target value of the target evaluation network of the mth agent at the next time step t+1.

[0025] Furthermore, in step A7, based on the agent's observation information, executed actions, and reward estimates, the agent's policy network parameters are updated, including the following steps:

[0026] B1. Calculate the agent’s policy gradient based on the agent’s observation information, execution actions, and reward estimates, expressed as:

[0027]

[0028] in: is the strategy μ of the mth agent with respect to the policy network parameters θ t，m，Actor The policy gradient, E[·] is the expectation operator, o, a~D are the observation information o and the execution action a sampled from the experience pool D, is the strategy μ(a t，m |o t，m ) About the policy network parameters θ t，m，Actor The policy gradient of is the reward estimate of the mth agent About executing action a t，m The policy gradient of , J(μ) is the objective function of the agent, and the expectation of the long-term cumulative reward of the agent;

[0029] B2. Update the policy network parameters of the agent according to the agent's policy gradient, expressed as:

[0030]

[0031] Where: θ t+1，m，Actor is the network parameter of the policy network of the mth agent at the next time step t+1, θ t，m，Actor is the network parameter of the policy network of the mth agent at the current time step t, and β is the learning rate.

[0032] Furthermore, in step A7, based on the agent's observation information, execution action, reward estimation value, and reward target value, the agent's target policy network parameters and target evaluation network parameters are updated, which are expressed as:

[0033] θ Target ←τθ+(1-τ)θ Target

[0034] Where: θ Target is the target network parameter of the agent, θ is the network parameter of the agent, and τ is the soft update parameter.

[0035] Furthermore, in step S3, the preliminary LEO large-scale constellation communication path is dynamically adjusted based on the burst interference information to obtain a LEO large-scale constellation communication path resistant to burst interference, including the following steps:

[0036] C1. Calculate the agent's expected reward on the preliminary low-orbit large-scale constellation communication path based on the burst interference information and the preliminary low-orbit large-scale constellation communication path;

[0037] C2. Calculate the overall reward of the agent on the initial low-orbit large-scale constellation communication path based on the agent's expected reward on the initial low-orbit large-scale constellation communication path;

[0038] C3. Sort the overall rewards of the intelligent agents in the preliminary low-orbit large-scale constellation communication path to obtain a dynamically adjusted priority sequence of the communication path. Use the dynamically adjusted priority sequence of the communication path to dynamically adjust the preliminary low-orbit large-scale constellation communication path to obtain a low-orbit large-scale constellation communication path that is resistant to sudden interference.

[0039] Furthermore, in step C1, based on the burst interference information and the preliminary low-orbit large-scale constellation communication path, the reward expectation of the agent in the preliminary low-orbit large-scale constellation communication path is calculated, which is expressed as:

[0040] E m (R m )=ΣP m R m

[0041] Where: E m (R m ) is the reward expectation of the mth agent under the initial low-orbit large-scale constellation communication path, P m is the probability that the mth agent is under sudden interference information, R m is the transmission reward of the mth agent under sudden interference information.

[0042] Furthermore, in step C2, based on the agent's expected reward in the preliminary low-orbit large-scale constellation communication path, the agent's overall reward in the preliminary low-orbit large-scale constellation communication path is calculated, which is expressed as:

[0043]

[0044] Where: Q is the overall reward of the mth agent under the preliminary low-orbit large-scale constellation communication path, M is the total number of agents, and μm is the weight factor of the mth agent relative to the overall reward.

[0045] The present invention has the following beneficial effects:

[0046] (1) The present invention splits the file to be transmitted into sub-files to be transmitted and constructs an intelligent body based on the sub-files to be transmitted. This avoids the problem of slow transmission efficiency caused by placing the files to be transmitted on the same communication path for transmission when the files to be transmitted are large. At the same time, it avoids the risk of information being completely intercepted due to all information of the files to be transmitted being on the same communication path, thereby improving the security of information transmission.

[0047] (2) The present invention constructs a strategy network, an evaluation network, a target strategy network, and a target evaluation network of an intelligent agent based on an intelligent agent group, and generates a preliminary low-orbit large-scale constellation communication path according to the strategy network, evaluation network, target strategy network, and target evaluation network of the intelligent agent; then obtains sudden interference information, and dynamically adjusts the preliminary low-orbit large-scale constellation communication path based on the sudden interference information to obtain a low-orbit large-scale constellation communication path resistant to sudden interference, thereby avoiding the problem that the shortest path weighted method only considers distance, load, and task priority and cannot dynamically adjust the communication path in the face of sudden interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The figure is a flowchart of a method for planning communication paths for large-scale low-orbit constellations facing sudden interference.

[0049] Figure 2 A schematic diagram of the transmission path planning after the files to be transmitted are split;

[0050] Figure 3 A schematic diagram showing the generation of preliminary low-orbit large-scale constellation communication paths using four networks. DETAILED DESCRIPTION

[0051] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0052] like Figure 1 As shown, a method for planning a communication path for a large-scale low-orbit constellation facing sudden interference includes steps S1-S3, which are specifically as follows:

[0053] S1. Split the file to be transferred to obtain sub-files to be transferred, and construct an intelligent agent based on the sub-files to be transferred.

[0054] In an optional embodiment of the present invention, a large file to be transmitted from satellite A to satellite B in a large-scale low-orbit satellite is evenly divided into M small sub-files to be transmitted. The present invention then constructs each sub-file to be transmitted into an intelligent agent. The operation of each intelligent agent is decided on each satellite of the current transmission node. Figure 2 As shown in Figure 2, after the splitting is completed, the system starts the parallel transmission mechanism, so that these sub-files to be transmitted can be transmitted simultaneously on different communication paths. The basic information carried by the agent includes position coordinate information, speed information and environmental information.

[0055] S2. Construct the agent's policy network, evaluation network, target policy network, and target evaluation network based on the agent group, and generate a preliminary low-orbit large-scale constellation communication path based on the agent's policy network, evaluation network, target policy network, and target evaluation network.

[0056] In an optional embodiment of the present invention, the present invention constructs a policy network, an evaluation network, a target policy network and a target evaluation network of an agent based on an agent group. The four networks correspond to the Actor, Critic, Target and the Target in MADDPG (Multi-Agent Deep Deterministic Policy Gradient). _ Actor and Target _ Critice.

[0057] like Figure 3 As shown in the figure, the present invention is based on the agent's strategy network (Actor), evaluation network (Critic), target strategy network (Target _ Actor and target evaluation network (Target _Critice) to generate preliminary LEO large-scale constellation communication paths, including the following steps:

[0058] A1. Randomly initialize the network parameters and build an experience pool.

[0059] Specifically, the present invention adjusts the policy network parameters θ of the agent Actor , evaluation network θ critic , target policy network θ Target_actor and target evaluation network θ T arg er_critic Random initialization is performed separately so that the network has a certain degree of randomness and diversity at the beginning of training, which is helpful for the subsequent learning and optimization process. The experience pool D is constructed and set to empty.

[0060] A2. Obtain the state information of the agent to form the observation information of the agent at the current time step.

[0061] Specifically, in the present invention, each agent obtains the position coordinate information and speed information of itself and other agents, and then these information together constitute the observation information of the agent, which is expressed as:

[0062] (o t，1 ,…,o t，m )

[0063] Among them: t,1 is the observation information of the first agent at the current time step t, o t,m is the observation information of the mth agent at the current time step t, t is the current time step, and m is the agent number.

[0064] A3. Based on the observation information of the agent at the current time step, the agent's policy network is used to generate the agent's execution action at the current time step, and the agent's reward at the current time step and the agent's observation information at the next time step are obtained based on the agent's execution action at the current time step.

[0065] Specifically, based on the observation information of the agent at the current time step, the policy network of each agent outputs actions respectively, that is, selects the next satellite node. Each agent will enter a new state, and then obtain the reward of the agent at the current time step and the observation information of the agent at the next time step.

[0066] A4. Store the agent's observation information, executed actions, rewards at the current time step, and observation information at the next time step into the experience pool.

[0067] A5. Repeat steps A2-A4 until the experience pool is fully stored, and randomly sample the experience pool to obtain a sample set.

[0068] A6. Based on the sample set, the agent's evaluation network is used to calculate the agent's reward estimate, and the agent's target policy network and target evaluation network are used to calculate the agent's reward target value.

[0069] The evaluation network of the agent is based on the observation information and action information of all agents in the sample set k at the current time step and the parameter θ t,m,critic Calculate the estimated reward for each agent Among them, μ represents a determined strategy, which means that the Q value is calculated based on a certain determined strategy. At the same time, the target evaluation network of the agent receives the next state (o t+1,1 ,…,o t+1,m ), and use the next action (a t+1，1 ,…,a t+1，m ), calculate the reward target value of each agent

[0070] A7. Based on the agent's observation information, execution actions, reward estimates, and reward target values, the agent's policy network parameters, evaluation network parameters, target policy network parameters, and target evaluation network parameters are updated until the set update threshold conditions are met to obtain the agent's final policy network.

[0071] The present invention calculates the loss function of the agent's evaluation network based on the agent's reward estimate and reward target value to update the agent's evaluation network parameters, which is expressed as:

[0072]

[0073] Where: L(θ t+1，m，critic ) is the loss function of the evaluation network of the mth agent at the next time step t+1, θ t+1，m，critic is the network parameter of the evaluation network of the mth agent at the next time step t+1, E[·] is the expectation operator, is the reward estimate of the evaluation network of the mth agent at the current time step t, y is the target value generated by the target evaluation network, r t，m is the reward of the mth agent at the current time step t, γ is the discount factor, The reward target value of the target evaluation network of the mth agent at the next time step t+1.

[0074] The present invention updates the policy network parameters of the agent based on the agent's observation information, execution actions and reward estimates, including the following steps:

[0075] B1. Calculate the agent’s policy gradient based on the agent’s observation information, execution actions, and reward estimates, expressed as:

[0076]

[0077] in: is the strategy μ of the mth agent with respect to the policy network parameters θ t，m，Actor The policy gradient, E[·] is the expectation operator, o, a~D are the observation information o and the execution action a sampled from the experience pool D, is the strategy μ(a t，m |o t，m ) About the policy network parameters θ t，m，Actor The policy gradient of is the reward estimate of the mth agent About executing action a t，m The policy gradient of , J(μ) is the objective function of the agent, and the expectation of the long-term cumulative reward of the agent.

[0078] B2. Update the policy network parameters of the agent according to the policy gradient of the agent, which is expressed as:

[0079]

[0080] Where: θ t+1，m，Actor is the network parameter of the policy network of the mth agent at the next time step t+1, θ t，m，Actor is the network parameter of the policy network of the mth agent at the current time step t, and β is the learning rate.

[0081] The present invention updates the target strategy network parameters and target evaluation network parameters of the agent based on the agent's observation information, execution action, reward estimation value and reward target value, which can be expressed as:

[0082] θ Target ←τθ+(1-τ)θ Target

[0083] Where: θ Target is the target network parameter of the agent, θ is the network parameter of the agent, and τ is the soft update parameter.

[0084] Specifically, in the present invention, τ is a relatively small soft update parameter. In this way, the parameters of the agent's target policy network and target evaluation network slowly approach those of the current evaluation network and policy network, respectively, making their updates relatively stable and avoiding excessive fluctuations in the training process. The present invention then optimizes the network parameters of the four networks until the set update threshold conditions are met to obtain the agent's final policy network.

[0085] A8. Use the agent’s final policy network to generate preliminary low-orbit large-scale constellation communication paths.

[0086] S3. Obtain burst interference information, and dynamically adjust the preliminary low-orbit large-scale constellation communication path based on the burst interference information to obtain a low-orbit large-scale constellation communication path that is resistant to burst interference.

[0087] In an optional embodiment of the present invention, the present invention obtains burst interference information by: an intelligent agent generates information about burst interference based on a preliminary low-orbit large-scale constellation communication path and environmental information to obtain the burst interference information. The present invention dynamically adjusts the preliminary low-orbit large-scale constellation communication path based on the burst interference information to obtain a low-orbit large-scale constellation communication path that is resistant to burst interference, including the following steps:

[0088] C1. Based on the burst interference information and the preliminary low-orbit large-scale constellation communication path, calculate the expected reward of the agent on the preliminary low-orbit large-scale constellation communication path, expressed as:

[0089] E m (R m )=ΣP m R m

[0090] Where: E m (R m ) is the reward expectation of the mth agent under the initial low-orbit large-scale constellation communication path, P m is the probability that the mth agent is under sudden interference information, R m is the transmission reward of the mth agent under sudden interference information.

[0091] C2. Based on the agent's expected reward in the initial low-orbit large-scale constellation communication path, calculate the agent's overall reward in the initial low-orbit large-scale constellation communication path, expressed as:

[0092]

[0093] Where: Q is the overall reward of the mth agent in the initial low-orbit large-scale constellation communication path, M is the total number of agents, μ mis the weight factor of the mth agent relative to the overall reward.

[0094] C3. Sort the overall rewards of the intelligent agents in the preliminary low-orbit large-scale constellation communication path to obtain a dynamically adjusted priority sequence of the communication path. Use the dynamically adjusted priority sequence of the communication path to dynamically adjust the preliminary low-orbit large-scale constellation communication path to obtain a low-orbit large-scale constellation communication path that is resistant to sudden interference.

[0095] Specifically, the present invention compares the Q values ​​of different paths in the dynamically adjusted priority sequence of communication paths to select the communication path corresponding to the maximum Q value, thereby obtaining a low-orbit large-scale constellation communication path resistant to sudden interference. This enables the intelligent agent to dynamically respond to various sudden interferences, continuously adjust and optimize the communication path, and realize the efficient and stable operation of the low-orbit satellite large file transmission system.

[0096] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0099] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0100] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A method for planning low-orbit large-scale constellation communication paths for sudden interference, characterized in that: The following steps are involved: S1. Split the file to be transferred to obtain sub-files to be transferred, and construct an agent based on the sub-files to be transferred; S2. Based on the agent group, construct the agent's policy network, evaluation network, target policy network, and target evaluation network, and generate a preliminary low-orbit large-scale constellation communication path based on the agent's policy network, evaluation network, target policy network, and target evaluation network, including the following steps: A1. Randomly initialize the network parameters and build an experience pool; A2. Obtain the state information of the agent to form the observation information of the agent at the current time step; A3. Based on the agent's observation information at the current time step, the agent's policy network is used to generate the agent's execution action at the current time step. The agent's reward at the current time step and the agent's observation information at the next time step are obtained based on the agent's execution action at the current time step. A4. Store the agent's observation information, executed actions, rewards at the current time step, and observation information at the next time step into the experience pool; A5. Repeat steps A2-A4 until the experience pool is fully stored, and randomly sample the experience pool to obtain a sample set; A6. Based on the sample set, use the agent's evaluation network to calculate the agent's reward estimate, and use the agent's target policy network and target evaluation network to calculate the agent's reward target value; A7. Based on the agent's observation information, executed actions, reward estimates, and reward targets, update the agent's policy network parameters, evaluation network parameters, target policy network parameters, and target evaluation network parameters until the set update threshold conditions are met to obtain the agent's final policy network. A8. Generate preliminary low-orbit large-scale constellation communication paths using the agent's final strategy network; S3. Obtain burst interference information, and dynamically adjust the preliminary low-orbit large-scale constellation communication path based on the burst interference information to obtain a low-orbit large-scale constellation communication path that is resistant to burst interference.

2. The method for planning a low-orbit large-scale constellation communication path for sudden interference according to claim 1, wherein: In step S1, the basic information carried by the agent includes position coordinate information, speed information and environment information.

3. The method for LEO large-scale constellation communication path planning for sudden interference according to claim 1, characterized in that: In step A7, the loss function of the agent's evaluation network is calculated based on the agent's estimated reward value and the reward target value to update the agent's evaluation network parameters, which is expressed as: Where: L(θ t+1,m,critic ) is the loss function of the evaluation network of the mth agent at the next time step t+1, θ t+1，m，critic is the network parameter of the evaluation network of the mth agent at the next time step t+1, E[·] is the expectation operator, is the reward estimate of the evaluation network of the mth agent at the current time step t, y is the target value generated by the target evaluation network, r t，m is the reward of the mth agent at the current time step t, γ is the discount factor, The reward target value of the target evaluation network of the mth agent at the next time step t+1.

4. The method for planning a low-orbit large-scale constellation communication path for sudden interference according to claim 1, wherein: In step A7, the agent's policy network parameters are updated based on the agent's observation information, execution actions, and reward estimates, including the following steps: B1. Calculate the agent’s policy gradient based on the agent’s observation information, execution actions, and reward estimates, expressed as: in: is the strategy μ of the mth agent with respect to the policy network parameters θ t，m，Actor The policy gradient, E[·] is the expectation operator, o, a~D are the observation information o and the execution action a sampled from the experience pool D, is the strategy μ(a t，m |o t，m ) About the policy network parameters θ t，m，Actor The policy gradient of is the reward estimate of the mth agent About executing action a t，m The policy gradient of , J(μ) is the objective function of the agent, and the expectation of the long-term cumulative reward of the agent; B2. Update the policy network parameters of the agent according to the agent's policy gradient, expressed as: Where: θ t+1，m，Actor is the network parameter of the policy network of the mth agent at the next time step t+1, θ t，m，Actor is the network parameter of the policy network of the mth agent at the current time step t, and β is the learning rate.

5. The method for planning a low-orbit large-scale constellation communication path for sudden interference according to claim 1, wherein: In step A7, based on the agent's observation information, execution action, reward estimation value, and reward target value, the agent's target policy network parameters and target evaluation network parameters are updated, expressed as: i Target ←τθ+(1-τ)θ Target Where: θ Tatget is the target network parameter of the agent, θ is the network parameter of the agent, and τ is the soft update parameter.

6. The method for planning a low-orbit large-scale constellation communication path for sudden interference according to claim 1, wherein: In step S3, the preliminary LEO large-scale constellation communication path is dynamically adjusted based on the burst interference information to obtain a LEO large-scale constellation communication path resistant to burst interference, including the following steps: C1. Calculate the agent's expected reward on the preliminary low-orbit large-scale constellation communication path based on the burst interference information and the preliminary low-orbit large-scale constellation communication path; C2. Calculate the overall reward of the agent on the initial low-orbit large-scale constellation communication path based on the agent's expected reward on the initial low-orbit large-scale constellation communication path; C3. Sort the overall rewards of the intelligent agents in the preliminary low-orbit large-scale constellation communication path to obtain a dynamically adjusted priority sequence of the communication path. Use the dynamically adjusted priority sequence of the communication path to dynamically adjust the preliminary low-orbit large-scale constellation communication path to obtain a low-orbit large-scale constellation communication path that is resistant to sudden interference.

7. The method for planning a low-orbit large-scale constellation communication path for sudden interference according to claim 6, wherein: In step C1, based on the burst interference information and the preliminary low-orbit large-scale constellation communication path, the reward expectation of the agent on the preliminary low-orbit large-scale constellation communication path is calculated, which is expressed as: E m (R m )=ΣP m R m Where: E m (R m ) is the reward expectation of the mth agent under the initial low-orbit large-scale constellation communication path, P m is the probability that the mth agent is under sudden interference information, R m is the transmission reward of the mth agent under sudden interference information.

8. The method for planning a low-orbit large-scale constellation communication path for sudden interference according to claim 6, wherein: In step C2, based on the agent's expected reward on the preliminary low-orbit large-scale constellation communication path, the agent's overall reward on the preliminary low-orbit large-scale constellation communication path is calculated, which is expressed as: Where: Q is the overall reward of the mth agent in the initial low-orbit large-scale constellation communication path, M is the total number of agents, μ m is the weight factor of the mth agent relative to the overall reward, E m (R m ) is the reward expectation of the mth agent under the preliminary low-orbit large-scale constellation communication path.

Images