Double-satellite cooperative confrontation behavior planning method and system

Abstract

The invention discloses a double-satellite cooperative confrontation behavior planning method and system, and belongs to the field of aircraft orbit control, and the method comprises the steps: selecting an optimal action to carry out orbit dynamics simulation deduction on a game confrontation process of a current friend-my satellite according to state information of satellites of two parties in a current round in combination with a Monte Carlo search method; the action corresponding to the child node with the highest upper limit confidence interval value is selected as the optimal simulation action adopted by our party in the current round; taking the optimal simulation action as the input of a winning rate calculation neural network, and outputting the optimal maneuvering action of our satellite in the current round; performing iterative search according to the optimal maneuvering action of the our satellite in the current round to obtain an action sequence of the our satellite so as to control the satellite to perform orbital maneuvering; according to the method, the on-orbit confrontation capability of the satellite is improved, the speed and stability of double-satellite collaborative confrontation behavior planning are improved, and the defect that an opposite target is easy to escape in single-satellite confrontation of the satellite in the prior art is overcome.

Description

Technical Field

[0001] The present invention relates to the field of spacecraft orbit control, and in particular to a dual-satellite collaborative confrontation behavior planning method and system. Background Art

[0002] In recent years, multi-agent cooperative games have attracted more and more attention from scholars at home and abroad. For cooperative game problems, the most popular typical problem is the cooperative confrontation game problem. At present, the main research objects of multi-agent cooperative confrontation games are drones, robots, cars, or there is no specific research object. There are few studies on the cooperative game of multiple spacecraft. The most classic one is the NashQ algorithm, which was first proposed by Hu et al. Its core is to enable the agent to learn the optimal joint strategy through the random game Lemke-Howson algorithm and use differential countermeasures and other methods to solve it. In 2017, the OpenAI team proposed a multi-agent reinforcement learning algorithm based on the Actor-Critic architecture, which uses the "centralized training, distributed execution" method to update the agent strategy. There is no communication mechanism for the agent in this algorithm. The cooperative behavior is guided only by the global critic of each agent, and the joint strategy is finally obtained through multiple iterations.

[0003] Since the pulse maneuver problem is a difficult problem in the aerospace field that is different from continuous linear control, and the simple multi-agent reinforcement learning method is difficult to use on satellites due to problems such as large computational volume. In the special significance of the geosynchronous orbit in space, the observation of specific target satellites through coordinated confrontation is an effective means of obtaining target information, but when a single satellite is used for observation, the target satellite will also adopt corresponding strategies for prevention, making it difficult to ensure the effectiveness of the observation mission. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a dual-satellite collaborative confrontation behavior planning method and system, which improves the satellite's on-orbit confrontation capability, while improving the speed and stability of dual-satellite collaborative confrontation behavior planning, and solves the shortcoming that the opponent's target is easy to escape in the existing situation of single-satellite confrontation.

[0005] The present invention is achieved through the following technical solutions:

[0006] A dual-satellite coordinated confrontation behavior planning method comprises the following steps:

[0007] Step 1: According to the state information of the satellites of both sides in the current round and combined with the Monte Carlo search method, an optimal action is selected to simulate the orbital dynamics of the current game confrontation process between the enemy and our satellites, thereby expanding the search process of the Monte Carlo tree and selecting the action corresponding to the child node with the highest upper confidence interval value as the optimal simulation action taken by our side in the current round;

[0008] Step 2: According to the state information of enemy and our satellites in each round of the game confrontation process, as well as the action with the highest winning rate and the corresponding sub-node in the process of each node search, a training data set is constructed, and the deep neural network of the constructed fully supervised learning method is trained to obtain a winning rate calculation neural network. The optimal simulated action taken by our side in the current round is used as input, and the winning rate calculation neural network outputs the optimal maneuvering action of our satellite in the current round;

[0009] Step 3: Perform an iterative search based on the optimal maneuvering action of our satellite in the current round until our satellite meets the mission requirements, obtain the action sequence of our satellite, and control the satellite to perform orbital maneuvers based on the action sequence of our satellite.

[0010] Preferably, the method for determining the status information of the satellites of both parties in the current round in step 1 is as follows:

[0011] Obtain the initial state information of the other party's satellite, establish a dual-satellite coordinated confrontation model, and set the initial state information of our two satellites according to the dual-satellite coordinated confrontation model;

[0012] According to the initial states of our two satellites and the other satellite, the relative motion equation between us and the enemy is established, the orbital states of our two satellites and the other satellite are deduced, and the real-time state information of our two satellites and the other satellite is obtained;

[0013] Orbital deduction is performed based on the satellite status information of both sides to obtain the satellite status information of both sides in the next round, and the relative distance between the enemy and our satellites is calculated through the status information.

[0014] Preferably, the enemy-to-friend relative motion equation of the two satellites is expressed as follows:

[0015]

[0016] Where n is the average orbital angular velocity of the reference satellite and Δt is the step size of a calculation time.

[0017] Preferably, the method for determining the optimal simulated action taken by our side in the current round in step 1 is as follows:

[0018] S4.1. Conduct orbital dynamics simulation and deduction of the current game confrontation process between the enemy and our satellites, thereby expanding the search process of the Monte Carlo tree until our satellite wins or the enemy satellite achieves successful avoidance, and obtain the probability of successful victory of the confrontation row under the sub-node action with the highest upper confidence interval value, as well as the entire simulated action sequence of our satellite in the current round;

[0019] S4.2, converting the game confrontation result between the two parties into a score and updating all the child nodes passed through during the game confrontation process;

[0020] S4.3, repeat steps S4.1-S4.2 to iteratively search from the root node until the statistical winning rate of different action nodes that can be selected in the current round is π i Continuously converge to the true value, and select the optimal action node as the optimal simulated action taken by our side in the current round based on the statistical winning rate of the simulated action sequence S.

[0021] Preferably, the calculation method of the upper confidence interval UCT value in step 1 is as follows:

[0022]

[0023] Among them, w i is the number of wins of our side at node i, l i is the number of failures of our side at node i, n i is the number of visits to node i, N is the total number of simulations for all nodes in this search, c is the exploration constant, and c1 and c2 are impact factors.

[0024] Preferably, in step 2, a state space S of dual-star collaborative confrontation is constructed according to the data training set, the state space S is used as the input of the neural network, and the winning rate corresponding to each action in the state space is used as the output of the neural network. The winning rate calculation neural network is obtained through forward propagation, error back propagation and iterative training combined with stochastic gradient descent method.

[0025] Preferably, the upper confidence interval UCT is rewritten according to the winning rate data output by the winning rate calculation neural network, and the rewritten confidence interval UCT * Calculate the optimal simulated action taken by our side in the previous round to obtain the optimal maneuvering action of our satellite in the current round.

[0026] Preferably, the expression of the rewritten confidence interval is as follows:

[0027] UCT * =Q i +U i +c1w i +c2l i

[0028] in, Subscript i represents the current UCT * The node where the value is calculated, b represents any action node in the action space under the current root node, and w i , l i and n are respectively the winning number, failure number and total number of visits of a node, p i is the output value of the winning rate calculation network, c is the exploration constant, and c1 and c2 are the influencing factors.

[0029] Preferably, in step 3, the relative distance between any one of our two satellites and the enemy satellite satisfies d r <d danger , the iterative search stops.

[0030] A system for a dual-satellite collaborative confrontation behavior planning method, comprising:

[0031] The game deduction module is used to select an optimal action to simulate the orbital dynamics of the current game confrontation process between the enemy and our satellites based on the state information of the satellites of both sides in the current round and in combination with the Monte Carlo search method, thereby expanding the search process of the Monte Carlo tree and selecting the action corresponding to the child node with the highest upper confidence interval value as the optimal simulation action taken by our side in the current round;

[0032] The neural network module is used to construct a training data set based on the state information of enemy and friendly satellites in each round of the game confrontation process, as well as the action with the highest winning rate and the corresponding sub-nodes in the search process of each node, train the deep neural network of the constructed fully supervised learning method to obtain a winning rate calculation neural network, and use the optimal simulated action taken by our side in the current round as input. The winning rate calculation neural network outputs the optimal maneuvering action of our satellite in the current round;

[0033] The planning module is used to iteratively search for the optimal maneuvering action of our satellite in the current round until our satellite meets the mission requirements, obtain the action sequence of our satellite, and control the satellite to perform orbital maneuvers according to the action sequence of our satellite.

[0034] Compared with the prior art, the present invention has the following beneficial technical effects:

[0035] The invention provides a method for planning the behavior of a dual-satellite coordinated confrontation. According to the state information of the satellites of both parties in the current round, a Monte Carlo tree search method is used for planning, and the action corresponding to the sub-node with the highest upper confidence interval value is selected as the optimal simulation action taken by our party in the current round; then, for the coordination problem of our satellite dual-satellite, a delayed "drive-away-entanglement" dual-satellite trajectory design method based on opposite semi-major axes and eccentricity differences is used, and a reinforcement learning framework is used to approximate the action space with the highest winning rate of our satellite. The winning rate parameters of the neural network are calculated according to the trained winning rate to optimize the upper confidence interval, and the optimal maneuvering action of our satellite is determined according to the optimized upper confidence interval, and then the action sequence of our satellite is obtained, and the satellite is controlled to perform orbital maneuvers according to the action sequence of our satellite. The method improves the satellite's on-orbit confrontation capability, and at the same time improves the speed and stability of the dual-satellite coordinated confrontation behavior planning, and solves the disadvantage that the opponent's target is easy to escape in the existing situation in the single-satellite confrontation of the satellite. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flow chart of a dual-satellite collaborative confrontation behavior planning method.

[0037] Figure 2 The present invention is a flow chart of a dual-satellite collaborative confrontation behavior planning method. DETAILED DESCRIPTION

[0038] The present invention will be further described in detail below in conjunction with the accompanying drawings, which are intended to explain the present invention rather than to limit it.

[0039] See also Figure 1 and 2 , a dual-satellite coordinated confrontation behavior planning method, comprising the following steps:

[0040] Step 1: Obtain the initial state information of the opponent's satellite, establish a dual-satellite collaborative confrontation model, and set the initial state information of our two satellites according to the dual-satellite collaborative confrontation model.

[0041] According to the initial state P of the other satellite 0(d) , and the semi-major axis difference a, eccentricity difference × in the specific coordinated confrontation scenario to establish a binary star coordinated confrontation model, and set the initial state P of our binary star according to the binary star coordinated confrontation model 0(w1) , P 0(w2) . Let a1 and a2 be opposite in positive and negative, e1 and e2 be opposite in positive and negative, a represents the difference in semi-major axis between the two satellites in the initial state, and e represents the difference in eccentricity between the enemy and us in the initial state.

[0042] According to the initial state information of the opponent's and our satellites, the relative coordination state information M1[a1,e1] and M2[a2,e2] between our two satellites and the opponent's satellite in the initial state are calculated respectively. The relative coordination state information means that our two satellites move toward the target at the opposite position of the opponent's satellite and approach the target according to the predetermined orbit.

[0043] Step 2: Establish the enemy-to-enemy relative motion equation based on the initial states of our two satellites and the opponent's satellite, perform orbit deduction on the enemy-to-enemy relative motion equation, and obtain the real-time state information of our two satellites and the opponent's satellite.

[0044] Assuming that the other satellite is periodically flying around a virtual satellite at the initial orbital position, and taking the virtual satellite as the reference satellite and its local orbital coordinate system as the reference coordinate system, the relative motion equation between the two satellites is established.

[0045] The x-axis of the local orbital coordinate system used points to the direction of the reference satellite, the y-axis points to the extension of the line connecting the center of the earth and the satellite, and the z-axis satisfies the right-hand rule. Since the reference satellite is a geostationary orbit satellite, its orbital eccentricity e = 0, and the distance between the two satellites and the reference point is much smaller than the orbital radius of the reference satellite, the analytical solution of the CW equation is used as the enemy-friend relative motion equation of the two satellites, which can be expressed as:

[0046]

[0047] Where n is the average orbital angular velocity of the reference satellite and Δt is the step size of a calculation time.

[0048] The input of the enemy-to-us relative motion equation is the state information of our binary satellite and the opponent's satellite at time t, and the output is the state information at time t+Δt.

[0049] From the initial moment, through orbit deduction, the state information of our binary star at time t is as follows:

[0050]

[0051]

[0052] Among them, x t(w1) ,y t(w1) ,z t(w1) is the relative position of our satellite at time t, is the relative speed of our satellite at time t.

[0053] The status information of the other satellite at time t is as follows:

[0054]

[0055] The status information of our two satellites at time t+Δt is as follows:

[0056]

[0057]

[0058] The status information of the other satellite at time t+Δt is as follows:

[0059]

[0060] Set the movement rounds of both sides T = nΔt.

[0061] Step 3: Determine the relative distance d between our two satellites and the other satellite based on their current status information r , specifically defined as:

[0062] d r1=|x t(w1) -x t(d) |+|y t(21) -y t(d) |+|z t(w1) -z t(d) |

[0063] d r2 =|x t(w2) -x t(d) |+|y t(w2) -y t(d) |+|z t(w2) -z t(d) |

[0064] The relative distance d between any of our satellites and the other satellite r Satisfy r ≤d move , then execute step 3, otherwise loop step 2, d move Indicates the calculated starting distance of our satellite.

[0065] Step 4: According to the status information of the satellites of both sides in the current round, select an optimal action to simulate the orbital dynamics of the current game confrontation process between the enemy and our satellites, thereby expanding the search process of the Monte Carlo tree and selecting the action corresponding to the sub-node with the highest UCT value in the upper confidence interval as the optimal simulation action taken by our side in the current round.

[0066] The formula for the UCT value is:

[0067]

[0068] where w i is the number of wins of our side at node i, l i is the number of failures of our side at node i, n i is the number of visits to node i, N is the total number of simulations for all nodes in this search, c is the exploration constant, and c1 and c2 are impact factors.

[0069] According to the game confrontation behavior of our satellite against the enemy satellite, it is assumed that when our satellite conducts coordinated confrontation, within T rounds after the movement of both parties, the relative distance between any of our two satellites and the enemy satellite satisfies d r <d danger , it means that our satellite successfully carried out the dual-satellite coordinated confrontation behavior. r Indicates the relative distance between our satellite and the enemy satellite, d danger Indicates the maximum capture distance of an enemy satellite.

[0070] In order to solve the game confrontation problem between enemy and friendly satellites in high orbit, a method combining Monte Carlo tree search (MCTS) and deep neural network is adopted: our satellite obtains initial state information M at a distance from the enemy satellite, determines the initial orbit to achieve successful confrontation behavior, and r ≤d move MCTS search process is performed at the same time.

[0071] During the confrontation, our satellite W1 first designs its orbit and maneuvers according to the state information M1. r ≤d move When , the MCTS method is used to select, expand, simulate, and backtrack data according to the number of rounds, and the optimal action under the UCT is obtained, and the orbit design of the round is carried out accordingly. Within T rounds after our satellite W1 uses the MCTS method to maneuver, the satellite maintains the dogfight process with the enemy satellite.

[0072] Our satellite W2 performs delayed maneuvers after satellite W1 uses the MCTS method to adjust its orbit and maneuver T1 rounds. At this time, the enemy satellite has responded to its approaching trajectory according to our satellite W1. Satellite W2 can make synchronous adjustments according to the adjusted trajectory of the enemy satellite, so that the two satellites can achieve the effects of expulsion and entanglement respectively during the confrontation process.

[0073] The method for our two satellites to conduct a search in a single round is:

[0074] S4.1. Starting from the optimal action taken in the current round, the orbital dynamics simulation and deduction are carried out for the current game confrontation process between the enemy and our satellites, so as to expand the search process of the Monte Carlo tree. This process continues until our satellite wins or the enemy satellite achieves successful avoidance, and the probability of successful victory under the sub-node action with the highest upper confidence interval value is obtained; by simulating and deducing the actions corresponding to all UCT values ​​in the action space of the party in the current round, all the search tree branches that a single satellite has walked through in the entire simulated game confrontation process using the MCTS method are obtained, that is, all the simulated action sequences S[S0,S1,…,S k ].

[0075] S4.2. Convert the result of the game confrontation between the two parties into a score and feed it back to all the child nodes that the game confrontation process passes through, and update the number of wins w of all child nodes i 、Number of failures l i and the number of visits n i .

[0076] Specifically, after each simulation, the number of visits to each node in the "action sequence" is n i+1. If our side wins, the number of wins for each node in this "action sequence" is w i +1; if we fail, the number of failures of each node in this "action sequence" is l i +1.

[0077] S4.3, repeat steps S4.1-S4.2 to iteratively search from the root node until the statistical winning rate of different action nodes that can be selected in the current round is π i It continues to converge to the true value, selects the optimal action node as the optimal simulated action taken by our side in the current round according to the statistical winning rate of the simulated action sequence S, and transfers the collaborative confrontation process to the next round state.

[0078] Step 5: Construct a training data set based on the status information of enemy and friendly satellites in each round, as well as the actions with the highest winning rate and the corresponding sub-nodes in the search process of each node. Train the deep neural network of the constructed fully supervised learning method to obtain the winning rate calculation neural network. Take the optimal simulated action taken by our side in the current round as input, and the winning rate calculation neural network outputs the optimal maneuvering action of our satellite in the current round.

[0079] For a large number of initial working conditions with different orbital semi-major axis differences and different eccentricity differences, MCTS is used to make action decisions, and the state and probability information recorded by the decision tree in each round are integrated to construct a training data set of "relative coordination state information, state information of enemy and friendly satellites, actions with the highest winning rate and corresponding nodes". A fully supervised learning method is used to design a deep neural network and determine the number of neural network layers and the number of neurons in each layer.

[0080] In the planning of dual-satellite coordinated confrontation behavior, according to the distance d between the two satellites r , delayed movement round T1 of our satellite W2, the minimum distance d between our satellite and the other satellite when using the MCTS method r , the distance between us and the reference point d c , the angle α between the line connecting the other party and the reference point and the positive direction of the x-axis of the reference orbit coordinate system r , the angle α between the line connecting our side and the reference point and the positive direction of the x-axis of the reference orbit coordinate system c The above variables together constitute the state space S = [d r ,T1,d rmin ,d c ,α r ,α c ] T With S as the input of the neural network, and the winning rate corresponding to each action in the state space in the training data set as the output of the neural network, the winning rate calculation neural network is obtained through forward propagation, error back propagation, and stochastic gradient descent iterative training.

[0081] According to the winning rate data output by the neural network, the upper confidence interval is rewritten according to the winning rate data, and the optimal simulated action taken by our side in the previous round is calculated according to the rewritten confidence interval to obtain the optimal maneuvering action of our satellite in the current round, guiding MCTS to select and expand. The expression of the rewritten confidence interval is as follows:

[0082] Change the traditional UCT value calculation formula to:

[0083] UCT*=Q i +U i +c1w i +c2l i

[0084] in, Subscript i represents the current UCT * The node where the value is calculated, and b represents any action node in the action space under the current root node. i , l i and n are the winning number, failure number and total number of visits of a node respectively. i is the output value of the winning rate calculation network, c is the exploration constant, and c1 and c2 are the influencing factors.

[0085] When MCTS performs a "selection", the current node state information S is first input into the winning rate calculation network, and the network outputs the winning rate p of each child node. i , and then calculate the UCT of each child node * Value, select UCT * The node with the largest value is used for subsequent expansion simulation, allocating more search times to the high winning rate area. At the same time, the winning rate data obtained by MCTS simulation combined with the winning rate neural network is used to optimize the winning rate calculation network parameters according to the winning rate data, and the optimal maneuvering action taken by our side in the current round is obtained. It is input into step 2 for orbit deduction of the next round, and the state space information of our satellite and the enemy satellite under the motion action is obtained.

[0086] Step 6: Perform an iterative search based on the optimal maneuvering action of our satellite in the current round until our satellite meets the mission requirements, obtain the action sequence of our satellite, and control the satellite to perform orbital maneuvers based on the action sequence of our satellite.

[0087] According to the state space information of our satellite and the enemy satellite under the optimal action in step 5, calculate the current state space of both satellites and judge whether our satellite meets the mission requirements. If it meets the mission requirements, that is, within T rounds after the movement of both sides, the relative distance between any of our two satellites and the enemy satellite meets d r <ddanger , then stop the iterative search, otherwise repeat the above steps 2 to 5 until our satellite meets the mission requirements or reaches the maximum number of iterations, and obtain the action sequence of our satellite. According to the action sequence of our satellite, control the satellite to perform orbital maneuvers and complete the dual-satellite coordinated confrontation behavior planning.

[0088] The present invention provides a dual-satellite collaborative confrontation behavior planning method. First, based on reasonable assumptions, the relative motion equation between our satellite and the opponent's satellite is established, and the real-time motion parameters of our satellite and the opponent's satellite are obtained. Secondly, according to the confrontation behavior between our satellite and the opponent's satellite, the conditions for our satellite to achieve a guaranteed confrontation are determined, and the Monte Carlo tree (MCTS) search method is used for planning for both enemy and our satellites. Then, in response to the coordination problem of our dual satellites, a delayed "drive away-entanglement" dual-satellite trajectory design method based on opposite semi-major axes and eccentricity differences is adopted. Finally, a reinforcement learning framework is established to approximate the action space with the highest winning rate of our satellite, and the upper confidence interval (UCT) of the winning rate neural network is optimized. * ), and finally formed a dual-satellite coordinated confrontation behavior planning method based on Monte Carlo search tree and deep neural network; this method improves the satellite's on-orbit confrontation capability, while improving the speed and stability of dual-satellite coordinated confrontation behavior planning, solving the shortcoming that the opponent's target is easy to escape in the existing situation of single-satellite confrontation.

[0089] The present invention also provides a dual-satellite collaborative confrontation behavior planning system, comprising:

[0090] The game deduction module is used to select an optimal action to simulate the orbital dynamics of the current game confrontation process between the enemy and our satellites based on the state information of the satellites of both sides in the current round and in combination with the Monte Carlo search method, thereby expanding the search process of the Monte Carlo tree and selecting the action corresponding to the child node with the highest upper confidence interval value as the optimal simulation action taken by our side in the current round;

[0091] The neural network module is used to construct a training data set based on the state information of enemy and friendly satellites in each round of the game confrontation process, as well as the action with the highest winning rate and the corresponding sub-nodes in the search process of each node, train the deep neural network of the constructed fully supervised learning method to obtain a winning rate calculation neural network, and use the optimal simulated action taken by our side in the current round as input. The winning rate calculation neural network outputs the optimal maneuvering action of our satellite in the current round;

[0092] The planning module is used to iteratively search for the optimal maneuvering action of our satellite in the current round until our satellite meets the mission requirements, obtain the action sequence of our satellite, and control the satellite to perform orbital maneuvers according to the action sequence of our satellite.

[0093] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A dual-satellite coordinated confrontation behavior planning method, characterized in that: The following steps are involved: Step 1: According to the state information of the satellites of both sides in the current round and combined with the Monte Carlo search method, an optimal action is selected to simulate the orbital dynamics of the current game confrontation process between the enemy and our satellites, thereby expanding the search process of the Monte Carlo tree and selecting the action corresponding to the child node with the highest upper confidence interval value as the optimal simulation action taken by our side in the current round; Step 2: According to the state information of enemy and our satellites in each round of the game confrontation process, as well as the action with the highest winning rate and the corresponding sub-node in the process of each node search, a training data set is constructed, and the deep neural network of the constructed fully supervised learning method is trained to obtain a winning rate calculation neural network. The optimal simulated action taken by our side in the current round is used as input, and the winning rate calculation neural network outputs the optimal maneuvering action of our satellite in the current round; Step 3: Perform an iterative search based on the optimal maneuvering action of our satellite in the current round until our satellite meets the mission requirements, obtain the action sequence of our satellite, and control the satellite to perform orbital maneuvers based on the action sequence of our satellite.

2. A dual-satellite coordinated confrontation behavior planning method according to claim 1, characterized in that: The method for determining the status information of both satellites in the current round in step 1 is as follows: Obtain the initial state information of the other party's satellite, establish a dual-satellite coordinated confrontation model, and set the initial state information of our two satellites according to the dual-satellite coordinated confrontation model; According to the initial states of our two satellites and the other satellite, the relative motion equation between us and the enemy is established, the orbital states of our two satellites and the other satellite are deduced, and the real-time state information of our two satellites and the other satellite is obtained; Orbital deduction is performed based on the satellite status information of both sides to obtain the satellite status information of both sides in the next round, and the relative distance between the enemy and our satellites is calculated through the status information.

3. A dual-satellite coordinated confrontation behavior planning method according to claim 2, characterized in that: The expression of the relative motion equation of the enemy and the friendly satellites of both parties is as follows: Where n is the average orbital angular velocity of the reference satellite and Δt is the step size of a calculation time.

4. A dual-satellite coordinated confrontation behavior planning method according to claim 1, characterized in that: The method for determining the optimal simulated action taken by our side in the current round in step 1 is as follows: S4.

1. Conduct orbital dynamics simulation and deduction of the current game confrontation process between the enemy and our satellites, thereby expanding the search process of the Monte Carlo tree until our satellite wins or the enemy satellite achieves successful avoidance, and obtain the probability of successful victory of the confrontation row under the sub-node action with the highest upper confidence interval value, as well as the entire simulated action sequence of our satellite in the current round; S4.2, converting the game confrontation result between the two parties into a score and updating all the child nodes passed through during the game confrontation process; S4.3, repeat steps S4.1-S4.2 to iteratively search from the root node until the statistical winning rate of different action nodes that can be selected in the current round is π i Continuously converge to the true value, and select the optimal action node as the optimal simulated action taken by our side in the current round based on the statistical winning rate of the simulated action sequence S.

5. A dual-satellite coordinated confrontation behavior planning method according to claim 1, characterized in that: The calculation method of the upper confidence interval UCT value in step 1 is as follows: Among them, w i is the number of wins of our side at node i, l i is the number of failures of our side at node i, n i is the number of visits to node i, N is the total number of simulations for all nodes in this search, c is the exploration constant, and c1 and c2 are impact factors.

6. A dual-satellite coordinated confrontation behavior planning method according to claim 1, characterized in that: In step 2, the state space S of the dual-star collaborative confrontation is constructed according to the data training set. The state space S is used as the input of the neural network, and the winning rate corresponding to each action in the state space is used as the output of the neural network. The winning rate calculation neural network is obtained through forward propagation, error back propagation and iterative training combined with the stochastic gradient descent method.

7. A dual-satellite coordinated confrontation behavior planning method according to claim 6, characterized in that: The upper confidence interval UCT is rewritten based on the winning rate data output by the winning rate calculation neural network. * Calculate the optimal simulated action taken by our side in the previous round to obtain the optimal maneuvering action of our satellite in the current round.

8. A dual-satellite coordinated confrontation behavior planning method according to claim 7, characterized in that: The expression of the rewritten confidence interval is as follows: UCT * =Q i +U i +c1w i +c2l i in, Subscript i represents the current UCT * The node where the value is calculated, b represents any action node in the action space under the current root node, and w i , l i and n are respectively the winning number, failure number and total number of visits of a node, p i is the output value of the winning rate calculation network, c is the exploration constant, and c1 and c2 are the influencing factors.

9. A dual-satellite coordinated confrontation behavior planning method according to claim 7, characterized in that: In step 3, when the relative distance between any of our two satellites and the enemy satellite satisfies d r <d danger , the iterative search stops.

10. A system for executing the dual-satellite coordinated confrontation behavior planning method according to any one of claims 1 to 9, characterized in that: include: The game deduction module is used to select an optimal action to simulate the orbital dynamics of the current game confrontation process between the enemy and our satellites based on the state information of the satellites of both sides in the current round and in combination with the Monte Carlo search method, thereby expanding the search process of the Monte Carlo tree and selecting the action corresponding to the child node with the highest upper confidence interval value as the optimal simulation action taken by our side in the current round; The neural network module is used to construct a training data set based on the state information of enemy and friendly satellites in each round of the game confrontation process, as well as the action with the highest winning rate and the corresponding sub-nodes in the search process of each node, train the deep neural network of the constructed fully supervised learning method to obtain a winning rate calculation neural network, and use the optimal simulated action taken by our side in the current round as input. The winning rate calculation neural network outputs the optimal maneuvering action of our satellite in the current round; The planning module is used to iteratively search for the optimal maneuvering action of our satellite in the current round until our satellite meets the mission requirements, obtain the action sequence of our satellite, and control the satellite to perform orbital maneuvers according to the action sequence of our satellite.

Images