A Dynamic Obstacle Avoidance and Intelligent Game Theory Method for Spacecraft Swarms

Abstract

This invention discloses a dynamic obstacle avoidance and intelligent game-playing method for spacecraft swarms, comprising: S1: establishing a coordinate system with a virtual spacecraft operating in a circular orbit as the reference spacecraft; S2: performing relative motion dynamics modeling on the spacecraft swarm and the target in the coordinate system to obtain a relative motion dynamics model describing the virtual spacecraft and the spacecraft swarm; S3: acquiring the coordinate data of each sub-spacecraft in the spacecraft swarm in the coordinate system; S4: determining the distance between each sub-spacecraft and the obstacle based on the motion dynamics model and the coordinate data; S5: determining whether there is a distance greater than a preset threshold; if so, proceed to S6; otherwise, proceed to S7; S6: driving the spacecraft swarm to achieve an encirclement state based on the gravitational relationship between each sub-spacecraft and the target and the repulsive relationship between each sub-spacecraft and the obstacle determined by the artificial potential field method; S7: using an intelligent encirclement strategy to encircle the target obstacle.

Description

Technical Field

[0001] This invention relates to the field of spacecraft obstacle avoidance technology, specifically to a dynamic obstacle avoidance and intelligent game theory method for spacecraft swarms. Background Technology

[0002] Spacecraft miniaturization and swarming are important directions for the future development of aerospace technology. However, with the surge in the number of spacecraft and the continuous increase in space debris, orbital space is becoming increasingly congested, posing severe challenges to the safe and stable operation of spacecraft in orbit and the completion of space missions. Ensuring the safe and efficient completion of space missions while avoiding collisions has become a major technical challenge. Spacecraft swarming game requires that the spacecraft can formulate intelligent pursuit strategies that leverage its own advantages based on the number and characteristics of the target. During the game, how to achieve collision avoidance between the spacecraft and space obstacles, targets, and other spacecraft becomes a crucial factor determining the outcome of the swarming game.

[0003] Artificial potential field methods are a traditional and effective approach for spacecraft obstacle avoidance and path planning. Their advantages lie in their real-time performance, dynamic obstacle avoidance, and ability to effectively plan spacecraft paths. However, in spacecraft swarm competition, the complexity and dynamic nature of the process make it difficult to accurately capture the target when approaching it. Summary of the Invention

[0004] The purpose of this invention is to provide a dynamic obstacle avoidance and intelligent game theory method for spacecraft swarms. This method can fully consider situations where information about non-cooperative targets is incomplete and the movement patterns of dynamic obstacles are difficult to grasp. It designs dynamic obstacle avoidance algorithms and intelligent encirclement strategies to achieve cooperative obstacle avoidance among friendly spacecraft and complete the tracking and encirclement of targets.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] This invention provides a method for dynamic obstacle avoidance and intelligent game theory in spacecraft swarms, the method comprising:

[0007] S1: Consider a virtual spacecraft operating in a circular orbit as a reference spacecraft and establish a coordinate system;

[0008] S2: In the coordinate system described above, perform relative motion dynamics modeling on the spacecraft cluster and the target to obtain a relative motion dynamics model describing the virtual spacecraft and the spacecraft cluster;

[0009] S3: Obtain the coordinate data of each sub-spacecraft in the spacecraft cluster in the coordinate system;

[0010] S4: Determine the distance between each sub-spacecraft and the obstacle based on the aforementioned kinematics model and coordinate data;

[0011] S5: Determine if there is a distance greater than the preset threshold. If yes, proceed to S6; otherwise, proceed to S7.

[0012] S6: Based on the gravitational relationship between each sub-spacecraft and the target and the repulsive relationship between each sub-spacecraft and the obstacle determined by the artificial potential field method, drive the spacecraft cluster to achieve the encirclement state;

[0013] S7: Use intelligent encirclement strategies to encircle and capture target obstacles.

[0014] Alternatively, in S2, the relative motion dynamics model describing the virtual spacecraft and spacecraft cluster is:

[0015]

[0016] in, It is x i The first derivative, It is x i The second derivative, It is y i The second derivative, It is z i The second derivative, This indicates that the spacecraft cluster itself applies active control acceleration in the x, y, and z directions, respectively. i Let y represent the coordinates of the i-th sub-spacecraft in the x-direction, and y represent the coordinates of i Let z represent the coordinates of the i-th sub-spacecraft in the y-direction. i Let w represent the coordinates of the i-th sub-spacecraft in the z-direction, and w be the orbital angular velocity of the virtual spacecraft.

[0017] Alternatively, in S4, the positions of each sub-spacecraft and obstacle are discretized and represented as follows:

[0018] x(k+1)=G(T)x(k)+H(T)a(k)

[0019] Where x(k+1) is the position of the subspacecraft at time k+1, x(k) is the position of the subspacecraft at time k, and G(T) is the impulse response matrix and G(T)=Φ(T)=e AT =L -1 [(SI-A) -1 ], Φ(T) is the state transition matrix, L -1 Let S be the inverse Laplace operator, S be the complex frequency, and H(T) be the state transition matrix of the system. t is time, T is sampling time, and A is the system matrix. B is the input matrix and I is the identity matrix, w is the orbital angular velocity of the virtual spacecraft, and a(k) is the control acceleration of the sub-spacecraft at time k.

[0020] Alternatively, in S6, the gravitational relationship between the sub-spacecraft and the target is as follows:

[0021]

[0022] Among them, F a (i) represents the gravitational force between the i-th subspacecraft and the target, m represents the gravitational gain coefficient, and x i Let x represent the coordinates of the i-th sub-spacecraft in the x-direction. d Represents the target's coordinates in the x-direction, y-direction... i Let y represent the coordinates of the i-th sub-spacecraft in the y-direction. d Represents the target's coordinates in the y-direction, z-direction... i Let z represent the coordinates of the i-th sub-spacecraft in the z-direction. d This represents the target's coordinates in the z-direction.

[0023] Alternatively, in step S6, the repulsive force relationship between the sub-spacecraft and the obstacle is as follows:

[0024]

[0025] Among them, F b (i) represents the repulsive force between the i-th sub-spacecraft and the obstacle, n represents the repulsive force gain coefficient, and d i d represents the distance between the i-th sub-spacecraft and the obstacle. min This indicates the minimum safe distance between the sub-spacecraft and the obstacle. This indicates calculating the gradient.

[0026] Alternatively, in S6, the encirclement state includes:

[0027] The distance between the spacecraft cluster and the target is less than a preset threshold; and at any given time, the spatial relationship between the spacecraft cluster and the target satisfies the following: the projection onto any coordinate axis plane must have at least two satellites in non-adjacent quadrants of the coordinate system with the target as the origin.

[0028] Alternatively, the intelligent containment strategy includes:

[0029] Based on the characteristics and constraints of the encirclement mission, design the reward function for each sub-spacecraft;

[0030] The reward functions of each sub-spacecraft are combined to obtain the return function;

[0031] Maximize the reward function and use the maximized reward function as the network parameters of the deep reinforcement learning algorithm to train the sub-spacecraft;

[0032] When the training reward value reaches the convergence condition, save and export the training results at this time, and use them as a test for the intelligent trapping strategy.

[0033] Alternatively, the reward function is:

[0034] Q t+1 (s t ,a t )=Q t (s t ,a t )+α[r t+1 +γmaxQ t+1 (s t+1 ,a')-Q t (s t ,a t )]

[0035] Among them, Q t+1 (s t ,a t Q is the spacecraft's reward function at time t+1. t (s t ,a t Let r be the reward function at time t, α be the learning rate, and r be the learning rate. t+1 It is the reward at step t+1, γ is the discount factor, and s t It is the state at time t, a t It is the action at time t.

[0036] Alternatively, the function for maximizing the return is:

[0037]

[0038] Where, π i (x) is the maximum reward function for the i-th spacecraft, a i Let a be the acceleration of the i-th spacecraft. n Q is the acceleration of the nth spacecraft. * (s,a) is the optimal value function.

[0039] The present invention has the following beneficial effects:

[0040] This invention divides the spacecraft swarm game problem into two stages and designs control methods based on the advantages of artificial potential field method and deep reinforcement learning, thereby realizing the solution of dynamic obstacle avoidance and intelligent game strategy of spacecraft swarm, and finally achieving efficient and accurate approach and encirclement of the target. Attached Figure Description

[0041] Figure 1 This is a flowchart of the spacecraft swarm dynamic obstacle avoidance and intelligent game theory method of the present invention;

[0042] Figure 2 Constraint graph for target encirclement;

[0043] Figure 3 Divide the area around the target into a map. Detailed Implementation

[0044] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0045] This invention provides a method for dynamic obstacle avoidance and intelligent game theory in spacecraft swarms, with reference to... Figure 1 As shown, the spacecraft swarm dynamic obstacle avoidance and intelligent game theory method includes:

[0046] S1: Consider a virtual spacecraft operating in a circular orbit as a reference spacecraft and establish a coordinate system;

[0047] S2: In the coordinate system described above, perform relative motion dynamics modeling on the spacecraft cluster and the target to obtain a relative motion dynamics model describing the virtual spacecraft and the spacecraft cluster;

[0048] Since both the spacecraft cluster and the target move near a virtual master spacecraft orbiting in a circular orbit during the pursuit, and the distance between the spacecraft is much smaller than the orbital radius of the master spacecraft, the CW equations can be used to describe the relative motion dynamics model of the virtual spacecraft and the spacecraft cluster:

[0049]

[0050] in, It is x i The first derivative, It is x i The second derivative, It is y i The second derivative, It is z i The second derivative, This indicates that the spacecraft cluster itself applies active control acceleration in the x, y, and z directions, respectively. i Let y represent the coordinates of the i-th sub-spacecraft in the x-direction, and y represent the coordinates of i Let z represent the coordinates of the i-th sub-spacecraft in the y-direction. i Let w represent the coordinates of the i-th sub-spacecraft in the z-direction, and w be the orbital angular velocity of the virtual spacecraft.

[0051] S3: Obtain the coordinate data of each sub-spacecraft in the spacecraft cluster in the coordinate system;

[0052] S4: Determine the distance between each sub-spacecraft and the obstacle based on the aforementioned kinematics model and coordinate data;

[0053] To facilitate the solution of the relative motion state of the spacecraft, the CW equations are first written in state-space form, and then discretized:

[0054]

[0055] in, It is x i The first derivative, x i Let A represent the coordinates of the i-th sub-spacecraft in the x-direction, and let A be the system matrix. B is the input matrix and a represents acceleration, and w represents the orbital angular velocity of the virtual spacecraft.

[0056] Based on this, the positions of each sub-spacecraft and obstacle are represented by discretization as follows:

[0057] x(k+1)=G(T)x(k)+H(T)a(k)

[0058] Where x(k+1) is the position of the subspacecraft at time k+1, x(k) is the position of the subspacecraft at time k, and G(T) is the impulse response matrix and G(T)=Φ(T)=e AT =L -1 [(SI-A) -1 ], Φ(T) is the state transition matrix, L -1 Let S be the inverse Laplace operator, S be the complex frequency, and H(T) be the state transition matrix of the system. t is time, T is sampling time, and A is the system matrix. B is the input matrix and I is the identity matrix, w is the orbital angular velocity of the virtual spacecraft, and a(k) is the control acceleration of the sub-spacecraft at time k.

[0059] S5: Determine if there is a distance greater than the preset threshold. If yes, proceed to S6; otherwise, proceed to S7.

[0060] S6: Based on the gravitational relationship between each sub-spacecraft and the target and the repulsive relationship between each sub-spacecraft and the obstacle determined by the artificial potential field method, drive the spacecraft cluster to achieve the encirclement state;

[0061] In this invention, space is first defined as a gravitational region and a repulsive region based on the artificial potential field method. Under the guidance of gravity, the spacecraft cluster can move from a distance toward the target, while the repulsive force can ensure that the spacecraft cluster avoids dynamic obstacles and also prevents collisions within the cluster.

[0062] The gravitational relationship between the sub-spacecraft and the target in this invention is as follows:

[0063]

[0064] Among them, F a (i) represents the gravitational force between the i-th subspacecraft and the target, m represents the gravitational gain coefficient, and x i Let x represent the coordinates of the i-th sub-spacecraft in the x-direction. d Represents the target's coordinates in the x-direction, y-direction... i Let y represent the coordinates of the i-th sub-spacecraft in the y-direction. d Represents the target's coordinates in the y-direction, z-direction... i Let z represent the coordinates of the i-th sub-spacecraft in the z-direction. d This represents the target's coordinates in the z-direction.

[0065] The repulsive force relationship between the subspacecraft and the obstacle is as follows:

[0066]

[0067] Among them, F b (i) represents the repulsive force between the i-th sub-spacecraft and the obstacle, n represents the repulsive force gain coefficient, and d i d represents the distance between the i-th sub-spacecraft and the obstacle. min This indicates the minimum safe distance between the sub-spacecraft and the obstacle. This indicates calculating the gradient.

[0068] In addition, the encirclement and suppression measures include:

[0069] The distance between the spacecraft cluster and the target is less than a preset threshold; and at any given time, the spatial relationship between the spacecraft cluster and the target satisfies the following: the projection onto any coordinate axis plane must have at least two satellites in non-adjacent quadrants of the coordinate system with the target as the origin.

[0070] refer to Figure 2 As shown, taking the xy plane as an example, at least two satellites must be located in the first and third quadrants or the second and fourth quadrants respectively to meet the encirclement and capture conditions.

[0071] S7: Use intelligent encirclement strategies to encircle and capture target obstacles.

[0072] Alternatively, the intelligent containment strategy includes:

[0073] Based on the characteristics and constraints of the encirclement mission, design the reward function for each sub-spacecraft;

[0074] The reward functions of each sub-spacecraft are combined to obtain the return function;

[0075] The reward function is:

[0076] Q t+1 (s t ,a t )=Q t (s t ,a t )+α[r t+1 +γmaxQ t+1 (s t+1 ,a')-Q t (s t ,a t )]

[0077] Among them, Q t+1 (s t ,a t Q is the spacecraft's reward function at time t+1. t (s t ,a t Let r be the reward function at time t, α be the learning rate, and r be the learning rate. t+1 It is the reward at step t+1, γ is the discount factor, and s t It is the state at time t, a t It is the action at time t.

[0078] Maximize the reward function and use the maximized reward function as the network parameters of the deep reinforcement learning algorithm to train the sub-spacecraft;

[0079] The function for maximizing returns is:

[0080]

[0081] Where, π i (x) is the maximum reward function for the i-th spacecraft, a i Let a be the acceleration of the i-th spacecraft. n Q is the acceleration of the nth spacecraft. * (s,a) is the optimal value function.

[0082] When the training reward value reaches the convergence condition, save and export the training results at this time, and use them as a test for the intelligent trapping strategy.

[0083] In addition, the characteristics of encirclement and capture missions include: unknown target strategy information;

[0084] The constraints of the encirclement mission include obstacle avoidance constraints, time constraints, fuel consumption constraints, and the random setting of the initial state within a certain range.

[0085] This invention decomposes the cluster game process into two stages: dynamic approximation and encirclement, such as... Figure 3 As shown, based on the different characteristics of the two stages, artificial potential field method and deep reinforcement learning algorithm are selected respectively to achieve successful pursuit of the target by the friendly spacecraft swarm. Constraints such as fuel consumption, dynamic obstacle avoidance, and avoidance of collisions between spacecraft are considered during the game process. The dynamic obstacle avoidance and intelligent game strategy of the swarm are obtained through the idea of ​​hierarchical design.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A spacecraft cluster dynamic obstacle avoidance and intelligent game method, characterized in that, The spacecraft cluster dynamic obstacle avoidance and intelligent game method comprises: S1: considering a virtual spacecraft running on a circular orbit as a reference spacecraft, a coordinate system is established; S2: in the coordinate system, relative motion dynamics modeling is performed on the spacecraft cluster and the target, to obtain a relative motion dynamics model describing the virtual spacecraft and the spacecraft cluster; S3: coordinate data of each sub-spacecraft in the spacecraft cluster in the coordinate system is obtained; S4: according to the relative motion dynamics model and the coordinate data, the distance between each sub-spacecraft and the obstacle is determined; S5: it is judged whether there is a condition that the distance is greater than a preset threshold, if yes, S6 is entered, otherwise, S7 is entered; S6: according to the gravitational relationship between each sub-spacecraft and the target and the repulsive relationship between each sub-spacecraft and the obstacle determined based on the artificial potential field method, the spacecraft cluster is driven to reach a hunting state; S7: an intelligent hunting strategy is used to hunt the target obstacle; The hunting state comprises: The distance between the spacecraft cluster and the target is less than a preset threshold, and at any time, the spatial position relationship between the spacecraft cluster and the target satisfies: in any coordinate axis plane, at least 2 satellites are in non-adjacent quadrants of the coordinate system with the target as the origin; The intelligent hunting strategy comprises: According to the characteristics and constraint conditions of the hunting task, a reward function of each sub-spacecraft is designed; The reward functions of each sub-spacecraft are combined to obtain a return function; The return function is maximized, and the maximized return function is taken as a network parameter of a deep reinforcement learning algorithm to train the sub-spacecraft; When the return value of the training reaches a convergence condition, the training result at this time is saved and exported as an intelligent hunting strategy for testing; The return function is: in, yes t The spacecraft's reward function at time +1 yes t The reward function at time step, It's the learning rate. It is the first Step rewards It is a discount factor. yes t The state at any given moment, yes t Actions at any given moment.

2. The spacecraft cluster dynamic obstacle avoidance and intelligent game playing method according to claim 1, characterized in that, In S2, the relative motion dynamics model describing the virtual spacecraft and the spacecraft cluster is: wherein is a first derivative of is a second derivative of is a second derivative of is a second derivative of denotes the spacecraft cluster's own imposed principal control acceleration in the x direction, y direction and z direction, respectively, denotes the coordinates of the i thsubspacecraft in the direction, denotes the coordinates of the i thsubspacecraft in the direction, denotes the coordinates of the i thsubspacecraft in the direction, is the virtual spacecraft's orbital angular velocity.

3. The spacecraft cluster dynamic obstacle avoidance and intelligent game playing method according to claim 1, characterized in that, In S4, the position of each sub-spacecraft and the obstacle is expressed as: wherein is the position of the sub-spacecraft at time is the position of the sub-spacecraft at time is the impulse response matrix and , is the state transition matrix, L -1 is the inverse Laplace operator, S is the complex frequency, is the state transition matrix of the system and , t is the time, T is the sampling time, is the system matrix and , is the input matrix and , is the identity matrix, is the orbital angular velocity of the virtual spacecraft, is the control acceleration of the sub-spacecraft at time 4. The spacecraft cluster dynamic obstacle avoidance and intelligent game playing method according to claim 1, characterized in that, In S6, the gravitational relationship between the sub-spacecraft and the target is: in, Indicates the first i The gravitational pull between the spacecraft and the target Represents the gravitational gain coefficient. Indicates the first i The spacecraft in Directional coordinates Indicates the target is Directional coordinates Indicates the first i The spacecraft in Directional coordinates Indicates the target is Directional coordinates Indicates the first i The spacecraft in Directional coordinates Indicates the target is Coordinates of direction.

5. The spacecraft cluster dynamic obstacle avoidance and intelligent game playing method according to claim 1, characterized in that, In S6, the repulsive relationship between the sub-spacecraft and the obstacle is: wherein, represents a repulsive force between the i represents a repulsive force gain coefficient, represents a repulsive force between the i represents a minimum safe distance between the represents a gradient.​​ 6. The spacecraft cluster dynamic obstacle avoidance and intelligent game playing method according to claim 1, wherein, The maximized return function is: where, is the maximum payoff function for the th spacecraft, is the maximum payoff function for the th spacecraft, is the acceleration of the th spacecraft, is the optimal value function.

Images