A method, system, and apparatus for low earth orbit satellite network maintenance based on a theory of mind

Abstract

The present application relates to a low-orbit satellite network maintenance method, system and device based on mental theory. The method comprises: training a protection agent and a disturbance agent based on a game model respectively; during the training process, the disturbance agent dynamically adjusts the disturbance strategy according to the confidence of different disturbance types, and infers the belief of the protection agent; the protection agent calculates the path selection weight of the inter-satellite link in real time according to the disturbance result, and infers the belief of the disturbance agent through mental theory to dynamically adjust the protection strategy; at the same time, the protection agent and the disturbance agent reach Nash equilibrium through interactive learning, and then the protection agent calculates the comprehensive index of each data transmission path; based on the calculation of the comprehensive index, a low-disturbance data transmission path is selected for data transmission. The present application can improve the security defense capability and robustness of the satellite network in complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of satellite network security technology, and in particular to a method, system and device for maintaining low-Earth orbit satellite networks based on cognitive theory. Background Technology

[0002] With the continuous development of communication technology, low-Earth orbit (LEO) satellite networks, as the core infrastructure of 6G integrated space-ground communication, are playing an increasingly important role in various communication applications due to their wide global coverage and low-latency transmission. These networks can break geographical limitations, providing stable and efficient communication services to users worldwide, and becoming a key component supporting future communication systems.

[0003] However, low-Earth orbit (LEO) satellite networks present severe challenges to security due to their dynamic topology, multi-source heterogeneous disturbances, severely limited onboard resources, and strong space environment interference. Specifically, the dynamically changing topology makes real-time response difficult for static rule-based intrusion detection mechanisms; centralized security architectures suffer from lengthy decision-making processes and the potential for single-node anomalies to trigger overall systemic risks. Furthermore, the limited onboard resources and interference from the complex space environment further complicate the implementation of security measures.

[0004] Especially in multi-agent interaction scenarios, traditional deep reinforcement learning methods often suffer from bottlenecks such as model failure and slow convergence speed due to the lack of modeling ability for the evolution of perturbed agents' policies, and cannot meet the security protection needs of low-Earth orbit satellite networks in complex scenarios. Summary of the Invention

[0005] Therefore, it is necessary to provide a low-Earth orbit satellite network maintenance method, system, and equipment based on mental theory to address the above-mentioned technical problems and enhance the security defense capabilities and robustness of satellite networks in complex scenarios.

[0006] A method for maintaining a low-Earth orbit satellite network based on the theory of mind, the method comprising: In a low-Earth orbit satellite network environment, a game model is constructed between protective agents and disruptive agents; Based on the game model, the protective agent and the perturbation agent are trained respectively using the deep deterministic policy gradient algorithm; During training, the perturbation agent dynamically adjusts the perturbation strategy based on the confidence level of different perturbation types and infers the belief of the protection agent; the protection agent calculates the path selection weight of the inter-satellite link in real time based on the perturbation results, and infers the belief of the perturbation agent through mental theory, and dynamically adjusts the protection strategy. Meanwhile, the protective agent and the disturbance agent reach Nash equilibrium through interactive learning. Then, the protective agent calculates the comprehensive index of each data transmission path. Based on the comprehensive index, a low-disturbance data transmission path is selected for data transmission.

[0007] On the other hand, a low-Earth orbit satellite network maintenance device based on mental theory is also provided, comprising: The game model building module is used to build a game model between a protective agent and a disruptive agent in a low-Earth orbit satellite network environment. The agent training module is used to train the protective agent and the perturbation agent respectively based on the game model and through the deep deterministic policy gradient algorithm. The strategy update module is used during training to dynamically adjust the perturbation strategy based on the confidence level of different perturbation types and infer the belief of the protection agent; the protection agent calculates the path selection weight of the inter-satellite link in real time based on the perturbation results and infers the belief of the perturbation agent through mental theory, and dynamically adjusts the protection strategy. The interactive learning module is used to achieve Nash equilibrium between the protective agent and the perturbation agent through interactive learning. Then, the protective agent calculates a comprehensive index for each data transmission path. Based on the comprehensive index, a low-perturbation data transmission path is selected for data transmission.

[0008] On another front, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the mind-based low-Earth orbit satellite network maintenance method described above.

[0009] Compared with existing technologies, the low-Earth orbit satellite network maintenance method, system, and equipment based on mental theory provided by this invention have the following beneficial effects: This invention is applied to low-Earth orbit (LEO) satellite networks. By constructing a game theory model between a protective agent and a disruptive agent, it enables the protector to perform hierarchical reasoning about the disruptor's strategic intent. A deep deterministic policy gradient algorithm is used to train both the protective and disruptive agents separately. Introducing mental theories into the protective agent allows it to effectively infer the disruptive agent's policy logic, thereby effectively responding to intrusion and disruptive behaviors in LEO satellite networks. Furthermore, the disruptive agent dynamically adjusts its disruptive strategy based on the confidence level of different disruptive types, ensuring that the disruptive scenarios conform to the actual network environment. The protective agent calculates the path selection weights of inter-satellite links in real time and dynamically matches protective resources, enabling it to update its belief in link disruption risks based on real-time network conditions, thus improving protection accuracy. While training separately, interactive learning is used to achieve Nash equilibrium, preventing policy failures due to environmental changes and ensuring long-term operational stability. By selecting low-disturbance data transmission paths based on comprehensive indicators, the probability of data transmission interruptions and packet loss is significantly reduced, ensuring service continuity, enhancing resistance to unknown disturbances, and improving system generalization.

[0010] Experimental results show that the method proposed in this invention outperforms traditional routing strategies in key performance indicators such as throughput, latency, and packet loss rate, providing an effective solution for the security defense of low-Earth orbit satellite networks. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention, and those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the low-Earth orbit satellite network maintenance method based on mental theory provided in Example 1. Figure 2 This is a structural block diagram of the low-Earth orbit satellite network maintenance device based on mental theory provided in Example 2; Figure 3 This is an internal structural diagram of the computer device provided in Example 3.

[0013] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] It is understood that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0016] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] Example 1 This embodiment discloses a low-Earth orbit satellite network maintenance method based on the theory of mind. By constructing a multi-level game model between defenders and disruptors, it achieves strategy optimization in the continuous action space under the framework of a deep deterministic strategy gradient algorithm. By combining dynamic strategy adjustment, interactive learning, and multi-path scheduling, it dynamically optimizes the satellite network routing strategy through real-time analysis of the attack and defense game results, realizing proactive intrusion mitigation. This method can effectively mitigate threats such as distributed denial-of-service attacks (DDoS attacks) and link interference, and significantly improve the robustness and security of the satellite network in dynamic attack scenarios.

[0018] like Figure 1 The diagram shows the low-Earth orbit satellite network maintenance method based on mental theory disclosed in this embodiment, which includes the following steps: Step 201: In a low-Earth orbit satellite network environment, construct a game model between the protective agent and the disruptive agent.

[0019] Step 202: Based on the game theory model, train the protective agent and the perturbation agent respectively using the deep deterministic policy gradient algorithm.

[0020] Step 203: During training, the perturbation agent dynamically adjusts the perturbation strategy based on the confidence level of different perturbation types and infers the belief of the protection agent; the protection agent calculates the path selection weight of the inter-satellite link in real time based on the perturbation results, and infers the belief of the perturbation agent through mental theory, and dynamically adjusts the protection strategy.

[0021] Step 204: The protective agent and the disturbance agent reach Nash equilibrium through interactive learning. Then, the protective agent calculates the comprehensive index of each data transmission path. Based on the comprehensive index, the agent selects the low-disturbance data transmission path for data transmission.

[0022] In the specific implementation of step 201, a multi-path transmission network is constructed based on the low-orbit satellite network environment. The multi-path transmission network includes satellite nodes, gateway station nodes, inter-satellite links, and satellite-to-ground links. Data transmission paths are formed by any combination of satellite nodes, gateway station nodes, inter-satellite links, and satellite-to-ground links.

[0023] To effectively characterize the communication demands between source and destination nodes in a low-Earth orbit satellite network, global population distribution data is first mapped to the geographic coordinates of corresponding gateway stations. This allows for the derivation of traffic intensity between gateway stations, and the generation of various types of traffic demands based on this. It can be understood that data packets include, but are not limited to, latency-sensitive, bandwidth-sensitive, and general demand types. Due to the differences in data packet types, it is necessary to combine the traffic intensity of the gateway stations to generate various types of traffic demands specifically.

[0024] To meet the demands of efficient multipath data transmission in low-Earth orbit satellite network environments, the MPQUIC protocol is introduced for packet transmission. MPQUIC possesses excellent multipath capabilities, dynamically adapting to the frequently changing topology and high latency characteristics of satellite networks, ensuring highly reliable packet transmission. Considering the service requirements of different service types in satellite networks, packets are categorized into three types: latency-sensitive, bandwidth-sensitive, and general-needs. Appropriate packet sizes are set for each category, and different indicators are assigned different weights in the reward function design. For example, for latency-sensitive services, the system applies a higher penalty when latency exceeds a target threshold, thereby guiding the scheduling strategy to prioritize optimizing the latency performance of this type of traffic.

[0025] The packet header structure mainly includes: connection_ID, flow_id, packet_number, source_ip, destination_ip, and multiple_services_flag. Among them, connection_ID is a unique identifier for MPQUIC connections, used to distinguish different end-to-end connections; flow_id is used to distinguish different data flows; packet_number ensures the ordered transmission of packets; source_ip and destination_ip identify the sender and receiver of the packet, respectively; and multiple_services_flag identifies the service type to which the packet belongs. Each packet generated in the system will be parsed and its features extracted by the SDN controller, serving as the input basis for a deep reinforcement learning model.

[0026] Furthermore, due to the high-speed operation of low-Earth orbit (LEO) satellites, the connectivity of inter-satellite links (ISL) and ground-to-satellite links (GSL) changes rapidly over time. To address this, a discrete-time step-based dynamic topology modeling method is employed. At fixed time steps, the satellite's spatial position is dynamically calculated based on its current orbital parameters, and the network topology is updated in real-time accordingly, reconstructing the inter-satellite and ground-to-satellite links. Additionally, state parameters such as link bandwidth, latency, and packet loss rate are affected by random factors such as weather changes and network attacks. Introducing environmental noise, such as fluctuations caused by weather changes and random factors like network attacks, into the network topology update process effectively simulates external uncertainties and improves the ability to characterize the dynamic LEO satellite network environment. It is worth noting that after each network topology and link state update, the latest link state information is incorporated into the environmental state space, driving the generation of agent policies and directly affecting the dynamic solution process of Nash equilibrium.

[0027] After clarifying the traffic requirements and data packet transmission, the game theory model was constructed. The protector and the disruptor were modeled as a protective agent and a disruptive agent with a DDPG structure. The protective agent is responsible for selecting the optimal data transmission path to avoid the disturbance, while the disruptive agent is responsible for selecting the disturbance link to maximize the destructive effect.

[0028] Within the framework of Decentralized Partially Observable Multi-Agent Markov Decision Process (Dec-POMDP), defensive and perturbative agents formulate decision-making strategies based on their respective local observations and subjective belief states, aiming to maximize their expected rewards while minimizing the opponent's gains. A key characteristic is that the utility gain of the defensive agent is exactly equal to the utility loss suffered by the perturbative agent. Based on this, a game theory model is constructed, expressed as: ; In the formula, Representing the state space; This represents the action space of the protective intelligent agent (such as routing paths and traffic scheduling selection); This represents the action space of the perturbation agent (such as perturbation path selection and perturbation intensity control). Indicates the state transition probability; The reward function represents the protective agent; The reward function represents the perturbation of the agent; This represents the discount factor, used to measure the importance of long-term rewards.

[0029] It is worth noting that in this embodiment, in the state space The system incorporates observational information and beliefs about the perturbed agent. State transition probabilities are also included. This represents the probability distribution of transitions to the next state after both the protective agent and the perturbed agent choose an action in a given state. The reward function for the protective agent is also included. Reward function of the perturbed agent satisfy .

[0030] In the specific implementation of step 202, based on the game model, the protective agent and the perturbation agent continuously learn their corresponding policies through the Deep Deterministic Policy Gradient Algorithm (DDPG), aiming to approximate the Nash equilibrium of a two-player zero-sum random game. DDPG is a reinforcement learning algorithm for continuous action spaces, comprising an Actor (policy network) and a Critic (Q-value network).

[0031] Taking the protective agent as an example, that is, the action value function of the protector. Define the current state Protective actions and the actions of the disturber The expected cumulative reward is expressed as: ; During DDPG training, the objective is to minimize the perturbator's gain, and the expression for calculating its objective Q-value is: ; The protective agent actor network updates its parameters through policy gradients: ; In the formula, The reward function represents the protective agent; The state value function represents the next state; Indicates time step The target scalar used to train the Q-network at any given moment; Indicates time step Observed immediate reward; Indicates parameters The target Q-network; Indicates the next state; Represents a target (delayed) deterministic policy network for a protective agent; Represents a target (delayed) deterministic policy network for perturbing agents; The gradient representing the performance target of the protection strategy parameters; This means that we first take the partial derivative of the Q function with respect to the action of the protective agent, and then evaluate it at the point where the action of the protective agent is equal to the action given by the current protection policy. It is mainly used to measure the impact of small changes in action on value in a given state. Representation strategy Regarding its parameters The gradient.

[0032] The structure of the perturbation agent is similar to that of the protection agent, both using the DDPG structure and the training process is also similar. However, the goal is to minimize the protector's gain, and the corresponding gradient directions are opposite, which will not be elaborated here.

[0033] In the specific implementation of step 203, in intrusion detection, confidence level typically represents the degree of certainty in judging a certain disturbance. In order to simulate the disturbance effect of the disturber, the confidence level obtained from intrusion detection is used as one of the input features of the disturbance agent, and the disturbance agent will dynamically adjust the disturbance strategy according to the confidence level of different disturbance types.

[0034] It is understandable that before processing each data packet, the perturbation state is updated based on the current simulation time. When a perturbation scenario is triggered, the defensive agent simulates the opponent's thought process by introducing theories of mind; that is, the defensive agent simulates the perturbation agent's strategy and adjusts its own strategy accordingly. Its state vector can be represented as: .in It represents 12-dimensional network environment variables, including information such as network topology, traffic load, and link status; Includes information on the current perturbation path and intensity of the perturbing agent; This represents the protective agent's prediction of the perturbing agent's strategy.

[0035] Based on the aforementioned state vector, a Bayesian belief update mechanism is introduced to achieve dynamic optimization of the protective agent's policy inference against the perturbed agent. The protective agent is defined as the agent for each link... Given a set of prior path selection weights And at each simulation time step, based on the latest link observation data For example, network metrics such as link packet loss rate and latency anomalies are used to update the perpetrator's policy beliefs in real time. This involves combining observed data with prior information and calculating new path selection weights using Bayes' theorem, thus more accurately reflecting the actions the perpetrator might take. These new path selection weights are directly input into the protector's policy network, guiding the protector agent to avoid high-risk links during path selection and traffic allocation.

[0036] The protective agent calculates the path selection weights of the inter-satellite links in real time based on the disturbance results. The calculation expression is as follows: ; ; At the same time, a risk penalty is added to the path selection weight, expressed as: ; In the formula, Indicates a given The posterior perturbation probability; This represents a given disturbance to the link, observed. The possibility; This represents the prior probability of a disturbance along the path; This indicates that there is no disturbance to the link; observation is needed. The possibility; This represents the prior probability that the path remains undisturbed. This represents the prior probability of a disturbance on the path at the next moment; This indicates that at time t, given... The posterior perturbation probability; Indicates risk penalty; Indicates link delay; This represents the risk sensitivity coefficient. It's worth noting that the perturbation result is the data obtained after the perturbation agent implements the perturbation strategy.

[0037] Furthermore, the belief levels of protective and disruptive agents are hierarchically classified. This hierarchical classification includes level zero, level one, and level two.

[0038] At level zero, the perturbing agent randomly selects links to perturb, and the perturbing agent is singular and of low intensity; the protective agent does not predict the opponent's behavior, but only responds to the environment.

[0039] Therefore, both the protective agent and the disturbance agent adopt a level-zero policy, expressed as: ; At level one, the protective agent predicts the behavior of the perturbed agent at level zero, updates its beliefs using Bayesian methods, and dynamically adjusts the probability of the link being perturbed, thereby optimizing its own strategy to maximize expected returns.

[0040] Therefore, the protective agent infers that the perturbing agent adopts a level-zero policy and uses a level-one protection policy for protection, as expressed as: ; At level two, the perturbation agent implements the first-level mental theory to simulate the protector's routing choices, prioritizing the perturbation of its most frequently used paths; the protection agent introduces modeling of the perturbor's intentions into the policy network, and dynamically adjusts path allocation and traffic weights in combination with Bayesian beliefs to avoid potentially high-risk links.

[0041] Therefore, the perturbing agent infers that the protecting agent adopts a first-level protection strategy and uses a first-level perturbation strategy to perform the perturbation, as expressed as: ; The protective agent infers that the disturbing agent employs a first-level protection strategy and a second-level protection strategy, as follows: ; In the formula, Represents a zero-level policy; Represents the given state of the agent; Indicates a mapping action; Indicates a protective intelligent agent; Indicates a perturbation agent; This represents the primary protection strategy of the protective intelligent agent; This represents a protective strategy that maximizes expected benefits. Indicates actions on the perturbed agent Obey its zero-level policy The probability distribution takes the expected value; This represents the immediate reward function for the protective agent. Indicates the discount factor; Indicating in strategy combination The value function under the given conditions acts on the subsequent state; This indicates the new state to which the environment transitions after an action is performed; This represents the first-level perturbation policy of the perturbing agent; This represents a perturbation strategy that maximizes the expected return. Indicates actions taken by the protective intelligent agent. Comply with its Level 1 protection strategy The probability distribution takes the expected value; A function representing the immediate reward of the perturbed agent; This indicates that the protective intelligent agent follows the Level 1 protection strategy. The perturbation agent follows the perturbation strategy. The cumulative future discounted returns that the protective agent can expect to receive during action; This represents the secondary protection strategy of the protective intelligent agent; Indicates actions on the perturbed agent Obey its first-level perturbation strategy The probability distribution takes the expected value; This indicates that the protective agent follows the protection strategy. The perturbation agent follows the first-level perturbation strategy. When taking action, the protective agent can expect to receive a cumulative future discount return.

[0042] It is worth noting that, considering the limitations of low-Earth orbit satellite network environment on model computational complexity, this embodiment only considers two levels at its highest level. In each game, the level of the protective agent is always set to be one level higher than that of the perturbing agent.

[0043] Furthermore, to avoid model convergence difficulties due to environmental complexity in the early stages of training, after a perturbation scenario is triggered, both the protective agent and the perturbed agent are initially set to the lowest level, 0. By guiding the model to learn from simple samples and gradually transitioning to more complex tasks, the model can steadily adapt to increasingly challenging environments. Therefore, in this embodiment, the current level is gradually increased based on the training rounds, time steps, or agent performance, progressing from the basic level 0 to the maximum level 2. At each level, the protective agent utilizes a Bayesian belief update mechanism to dynamically adjust its predictions of the perturbed agent's behavior based on the observed network state, and inputs these beliefs into the protective agent.

[0044] Assuming the entire disturbance lasts for a duration of The current simulation time is The formula for selecting the level of the current protective agent is expressed as: ; In the formula, This represents the highest level of the protective intelligent agent; Indicates the start time of the disturbance; Indicates the duration of the disturbance.

[0045] In the specific implementation of step 204, as the training iterates, the protective agent and the perturbation agent continuously update their own strategies based on the other's optimal response, each seeking to maximize their own expected cumulative reward, expressed as: ; Based on maximizing their expected cumulative reward, the strategies of the protective agent and the perturbed agent tend towards a Nash equilibrium, expressed as: ; In the formula, This indicates that the protective agent is in a given state. The value function under; Indicates the protection strategy; Indicates the perturbation strategy; Indicates cumulative discount rewards; express Constantly disturb the agent's actions; express Protect the actions of intelligent agents at all times; This represents the initial state of the value function; express The given state of the agent at any given moment; This represents the stable strategy combination achieved by the protective agent and the disruptive agent in the game; This represents a protective strategy that maximizes expected benefits. This indicates the use of a minimal perturbation strategy.

[0046] Once equilibrium is reached, the protective agent selects multiple paths based on the Nash equilibrium. Considering comprehensive indicators such as latency, bandwidth usage, and disturbance probability, it selects the low-disturbance data transmission path for data transmission.

[0047] The data transmission path is scored based on a comprehensive index, and the calculation expression is as follows: ; In the formula, , , Indicates an adjustable weight parameter; Indicates path bandwidth; Indicates the path delay; Indicates a given The posterior perturbation probability.

[0048] Then, based on the score ranking, the low-disturbance, high-performance data transmission path is selected first for data transmission, thereby achieving efficient and secure multi-path dynamic routing for the LEO satellite network.

[0049] It should be understood that, although this embodiment Figure 1 The steps are shown sequentially as indicated by the arrows, but they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are performed; they can be executed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0050] Example 2 Based on the low-Earth orbit satellite network maintenance method based on mental theory in Embodiment 1, this embodiment discloses a low-Earth orbit satellite network maintenance device based on mental theory, such as... Figure 2 As shown, the low-Earth orbit satellite network maintenance device based on the theory of mind includes: a game model construction module 401, an agent training module 402, a policy update module 403, and an interactive learning module 404, wherein: The game model construction module 401 is used to construct a game model between a protective agent and a disturbing agent in a low-Earth orbit satellite network environment.

[0051] The agent training module 402 is used to train the protective agent and the perturbation agent respectively based on the game model and through the deep deterministic policy gradient algorithm.

[0052] The strategy update module 403 is used during training. The perturbation agent dynamically adjusts the perturbation strategy according to the confidence of different perturbation types and infers the belief of the protection agent. The protection agent calculates the path selection weight of the inter-satellite link in real time based on the perturbation results and infers the belief of the perturbation agent through mental theory, and dynamically adjusts the protection strategy.

[0053] The interactive learning module 404 is used to achieve Nash equilibrium between the protective agent and the disturbing agent through interactive learning. Then, the protective agent calculates the comprehensive index of each data transmission path. Based on the comprehensive index, the agent selects the low-disturbance data transmission path for data transmission.

[0054] In this embodiment, the specific working process and working principle of the game model construction module 401, agent training module 402, policy update module 403, and interactive learning module 404 are the same as those in Embodiment 1, and therefore will not be described again in this embodiment. Each unit module can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each of the above unit modules.

[0055] Example 3 like Figure 3 The diagram illustrates a terminal device disclosed in this embodiment, comprising a transmitter, a receiver, a memory, and a processor. The transmitter transmits instructions and data, the receiver receives instructions and data, the memory stores computer-executed instructions, and the processor executes the computer-executed instructions stored in the memory to implement the method described in Embodiment 1 above.

[0056] It is important to note that the aforementioned memory can be either standalone or integrated with the processor. When the memory is set up independently, the terminal device also includes a bus for connecting the memory and the processor.

[0057] Example 4 This embodiment discloses a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method in Embodiment 1 above.

[0058] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0059] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for maintaining low-Earth orbit satellite networks based on the theory of mind, characterized in that, The method includes: In a low-Earth orbit satellite network environment, a game model is constructed between protective agents and disruptive agents; Based on the game model, the protective agent and the perturbation agent are trained respectively using the deep deterministic policy gradient algorithm; During training, the perturbation agent dynamically adjusts the perturbation strategy based on the confidence level of different perturbation types and infers the belief of the protection agent; the protection agent calculates the path selection weight of the inter-satellite link in real time based on the perturbation results, and infers the belief of the perturbation agent through mental theory, and dynamically adjusts the protection strategy. The protective agent and the disturbance agent reach Nash equilibrium through interactive learning. Then, the protective agent calculates a comprehensive index for each data transmission path. Based on the comprehensive index, a low-disturbance data transmission path is selected for data transmission.

2. The method for maintaining low-Earth orbit satellite networks based on mental theory according to claim 1, characterized in that, The game model expression is as follows: ； In the formula, Representing the state space; This represents the action space of the protective intelligent agent; This represents the action space of the perturbation agent; Indicates the state transition probability; The reward function represents the protective agent; The reward function represents the perturbation of the agent; This represents the discount factor.

3. The method for maintaining a low-Earth orbit satellite network based on mental theory according to claim 1 or 2, characterized in that, The protective intelligent agent calculates the path selection weight of the inter-satellite link in real time, and the calculation expression is: ； ； At the same time, a risk penalty is added to the path selection weight, expressed as: ； In the formula, Indicates a given The posterior perturbation probability; This represents a given disturbance to the link, observed. The possibility; This represents the prior probability of a disturbance along the path; This indicates that there is no disturbance to the link; observation is needed. The possibility; This represents the prior probability that the path remains undisturbed. This represents the prior probability of a disturbance on the path at the next moment; This indicates that at time t, given... The posterior perturbation probability; Indicates risk penalty; Indicates link delay; Indicates the risk sensitivity coefficient; This represents link observation data; Indicates a link.

4. The method for maintaining low-Earth orbit satellite networks based on mental theory according to claim 3, characterized in that, Also includes: The belief levels of the protective agent and the disturbance agent are hierarchically classified.

5. The low-Earth orbit satellite network maintenance method based on mental theory according to claim 4, characterized in that, When dividing into levels, include level zero, level one, and level two; At level zero, both the protective agent and the disturbance agent employ a level zero strategy, as follows: ； At level one, the protective agent infers that the disturbing agent adopts a level zero strategy and uses a level one protection strategy for protection, as expressed as: ； At level two, the perturbation agent infers that the protection agent adopts a level one protection strategy and uses a level one perturbation strategy to perform perturbation, as follows: ； The protective agent infers that the disturbing agent adopts a first-level protection strategy and a second-level protection strategy for protection, as expressed as: ； In the formula, Represents a level zero policy; Represents the given state of the agent; Indicates a mapping action; Indicates a protective intelligent agent; Indicates a perturbation agent; This represents the primary protection strategy of the protective intelligent agent; This represents a protective strategy that maximizes expected benefits. Indicates actions on the perturbed agent Obey its zero-level policy The probability distribution takes the expected value; This represents the immediate reward function for the protective agent. Indicates the discount factor; Indicating in strategy combination The value function under the given conditions acts on the subsequent state; This indicates the new state to which the environment transitions after an action is performed; This represents the first-level perturbation policy of the perturbing agent; This represents a perturbation strategy that maximizes expected return. Indicates actions taken by the protective intelligent agent. Comply with its Level 1 protection strategy The probability distribution takes the expected value; A function representing the immediate reward of the perturbed agent; This indicates that the protective intelligent agent follows the Level 1 protection strategy. The perturbation agent follows the perturbation strategy. The cumulative future discounted returns that the protective agent can expect to receive during action; This represents the secondary protection strategy of the protective intelligent agent; Indicates actions on the perturbed agent Obey its first-level perturbation strategy The probability distribution takes the expected value; This indicates that the protective agent follows the protection strategy. The perturbation agent follows the first-level perturbation strategy. When taking action, the protective agent can expect to receive a cumulative future discount return.

6. The method for maintaining low-Earth orbit satellite networks based on mental theory according to claim 4, characterized in that, The formula for selecting the hierarchy of the protective intelligent agent is: ； In the formula, This represents the highest level of the protective intelligent agent; Indicates the start time of the disturbance; Indicates the duration of the disturbance.

7. The method for maintaining a low-Earth orbit satellite network based on mental theory according to claim 1 or 2, characterized in that, The protective agent and the perturbed agent learn to reach Nash equilibrium through interactive learning, including: The protective agent and the disturbance agent continuously update their own strategies based on the other's optimal response, each seeking to maximize their expected cumulative reward, as expressed as: ； Based on maximizing their expected cumulative reward, the strategies of the protective agent and the perturbation agent tend to reach a Nash equilibrium, expressed as: ； In the formula, This indicates that the protective agent is in a given state. The value function under; Indicates the protection strategy; Indicates the perturbation strategy; Indicates cumulative discount rewards; express Constantly disturb the agent's actions; express Protect the actions of intelligent agents at all times; This represents the initial state of the value function; express The given state of the agent at any given moment; This represents the stable strategy combination achieved by the protective agent and the disruptive agent in the game; This represents a protective strategy that maximizes expected benefits. This indicates the use of a minimal perturbation strategy.

8. The method for maintaining low-Earth orbit satellite networks based on mental theory according to claim 7, characterized in that, Based on the aforementioned comprehensive indicators, a low-disturbance data transmission path is selected for data transmission, including: The data transmission path is scored based on the aforementioned comprehensive indicators, and the calculation expression is as follows: ； In the formula, , , Indicates an adjustable weight parameter; Indicates path bandwidth; Indicates the path delay; Indicates a given The posterior perturbation probability; Then, based on the score ranking, a low-disturbance data transmission path is selected for data transmission.

9. A low-Earth orbit satellite network maintenance device based on the theory of mind, characterized in that, The device includes: The game model building module is used to build a game model between a protective agent and a disruptive agent in a low-Earth orbit satellite network environment. The agent training module is used to train the protective agent and the perturbation agent respectively based on the game model and through the deep deterministic policy gradient algorithm. The strategy update module is used during training to dynamically adjust the perturbation strategy based on the confidence level of different perturbation types and infer the belief of the protection agent; the protection agent calculates the path selection weight of the inter-satellite link in real time based on the perturbation results and infers the belief of the perturbation agent through mental theory, and dynamically adjusts the protection strategy. The interactive learning module is used to achieve Nash equilibrium between the protective agent and the perturbation agent through interactive learning. Then, the protective agent calculates a comprehensive index for each data transmission path. Based on the comprehensive index, a low-perturbation data transmission path is selected for data transmission.

10. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the low-Earth orbit satellite network maintenance method based on the theory of mind as described in any one of claims 1 to 8.

Images