A spacecraft pursuit-evasion mission combination optimization control method based on SAC and LGVF

Abstract

The application provides a spacecraft pursuit task combination optimization control method based on SAC and LGVF, first, a hierarchical control method is used to divide the tasks in the pursuit and escape scene into different stages, and a hierarchical simplified model of the spacecraft pursuit task is established; secondly, an improved deep reinforcement learning flexible actor critic (SAC) algorithm is proposed to establish an autonomous motion planning control architecture, which provides the ability to handle dynamic uncertain states for the pursuit spacecraft; finally, the Lyapunov guide vector field method (LGVF) is introduced into the improved SAC algorithm framework to form a combination control method, which compresses the solution space size to optimize the solution process in the huge solution space. The method can enable the pursuit spacecraft to autonomously complete the spacecraft pursuit task in the scene where the external information is partially observable and unpredictable, and can provide real-time autonomous control capability and improve the task success rate.

Description

Technical Field

[0001] This invention relates to the field of spacecraft autonomous control technology, and in particular to a combined optimized control method for spacecraft pursuit missions based on SAC and LGVF. Background Technology

[0002] The problem of spacecraft air combat is a hot research topic in the field of air warfare today. With the improvement of performance, the role of spacecraft is not only battlefield reconnaissance, but also pursuit missions, completing the transformation from reconnaissance platform to combat platform.

[0003] Research on spacecraft pursuit technology began abroad as early as the 1970s, with the core objective of controlling spacecraft to track targets while avoiding threats and ensuring their own safety. Traditional methods for solving combinatorial optimization problems include exact algorithms, approximate algorithms, and heuristic algorithms, many of which have been proven effective. reliable Stable. Traditional methods rarely utilize common features between problems to find a universal solution. They require building a new solver to solve different instances of a similar problem. Therefore, this type of method cannot be applied to dynamic combinatorial optimization problems where the scenario changes in real time. Summary of the Invention

[0004] The purpose of this invention is to provide a combined optimization control method for spacecraft pursuit missions based on SAC and LGVF, which solves two major problems in spacecraft pursuit missions in dynamic and unknown environments: first, the unpredictability of external information, such as the escape method of the target, the location of the threat, and the range of fire; second, the partial observability of external information, as only some states in the environment can be obtained through a series of sensors on the spacecraft.

[0005] The technical solution adopted in this invention is as follows:

[0006] A combined optimization control method for spacecraft pursuit missions based on SAC and LGVF includes the following steps:

[0007] Step 1: Establish a spacecraft pursuit mission scenario model. The specific process is as follows:

[0008] The spacecraft pursuit mission in a dynamic unknown environment involved in this invention is described as a process in which the spacecraft needs to fly over threats in the scenario and pursue dynamically escaping spacecraft. An optimization function model is established for this problem as shown in formula (1):

[0009] mint c =G[f(P),f(E),f(T)] i (1)

[0010] objective function t cThis refers to the pursuit of spacecraft P with the goal of capturing the escaped spacecraft E in the shortest possible time, G[f(P),f(E),f(T)]. i [)] refers to the fusion of overall scene measurement information, f(P), f(E), f(T) i These refer to the status information of the pursuing spacecraft, the escaping spacecraft, and each threat T. i Represents the i-th threat;

[0011] The dynamic differential model of the pursuing and escaping spacecraft in this scenario is shown in Equation (2):

[0012]

[0013] Where, x i ,y i This is the spacecraft's current location information. They are x i ,y i The differential components, that is, the components of velocity in the two vector directions; v i a represents the speed of the spacecraft. i Indicates the acceleration of a spacecraft. It is the speed v i The differential, i.e. the rate of change of velocity, is equal to the acceleration a. i ;ψ i ω represents the heading angle of the spacecraft. i Represents the angular velocity of a spacecraft. It is the heading angle ψ i The differential component, namely the rate of change of heading angle, is equal to the angular velocity ω. i The angular velocity value of the spacecraft being pursued depends on the output of the reinforcement learning algorithm model;

[0014] The status and initial state of the spacecraft and the threat are shown in formula (3):

[0015]

[0016] Where, x i0 ,y i0 Δx represents the initial position of the spacecraft or threat. i ,Δy i x represents the change in the location of the spacecraft or threat. i (t0),y i (t0) represents the spacecraft or threat originating from the initial position x. i0 ,y i0 After Δx i ,Δy i The new position generated by the displacement change; v i0 The initial velocity, v, represents the speed at which the spacecraft or threat originates. i(t0) represents the velocity of the spacecraft or threat at any time t0; ψ i0 The initial angle representing the heading angle of the spacecraft or threat, Δω i ψ represents the change in the heading angle of a spacecraft or threat. i (t0) represents the heading angle of the spacecraft or threat, determined by the initial angle ψ. i0 After Δω i The new heading angle generated by the change in angle; R i0 R represents the fire range of a spacecraft or threat initialization. i (t0) represents the effective range of fire from the spacecraft or threat at any time t0, compared to the initial effective range R. i0 The threats remain unchanged; they are randomly distributed throughout the scene to simulate scene complexity.

[0017] The condition for the success of this scenario mission is that the distance between the pursuing spacecraft P and the escaping spacecraft E is less than the pursuing spacecraft's pursuit range, as shown in formula (4):

[0018] d PE ≤R P (4)

[0019] Where, d PE R is the distance between the pursuing spacecraft P and the escaped spacecraft E. P This refers to the range of pursuit capabilities for the spacecraft P.

[0020] The scenario is set to fail when the weapon ranges of the pursuing spacecraft P and the threat T overlap, i.e., the distance between the pursuing spacecraft and the threat is less than the safe distance, as shown in formula (5):

[0021]

[0022] in, This refers to the distance between the pursuing spacecraft and each threat; l refers to the safe distance between the pursuing spacecraft and the threat, defined as the sum of the effective ranges of the pursuing spacecraft and the threat. Wherein, according to formula (3), Representing the i-th threat T i The scope of the threat;

[0023] Step 2: Based on the spacecraft pursuit mission scenario model established in Step 1, design the state space model, action space model, and state transition model for the pursuing spacecraft and the escaping spacecraft, respectively.

[0024] Step 3: Based on the spacecraft pursuit mission scenario model established in Step 1, a hierarchical simplified model of the spacecraft pursuit mission is established through the hierarchical control method, so that the spacecraft pursuit mission is simplified into multi-level sub-tasks.

[0025] Step 4: Based on the established hierarchical simplified model of spacecraft pursuit mission, an improved deep reinforcement learning flexible actor critic algorithm is proposed to establish an autonomous motion planning and control architecture, which provides the ability to handle dynamic uncertain states for spacecraft pursuit and meets the current dynamic pursuit scenario that requires real-time optimization.

[0026] Step 5: Based on the established hierarchical simplified model of the spacecraft pursuit mission, under the autonomous motion planning and control architecture of the improved SAC algorithm, a combined method is formed by introducing the Lyapunov guided vector field method to optimize the learning process of the SAC algorithm and form a combined optimization method.

[0027] Step 6: Apply the combined optimization method described in Step 5 to the hierarchical simplified model of the spacecraft pursuit mission established in Step 4, and train the autonomous motion planning model of the pursuit spacecraft.

[0028] Step 7: Load the autonomous motion planning model of the spacecraft trained in Step 6 into an online pursuit mission simulation scenario where the information part is observable but unpredictable, and test it. Improve the combined optimization method by using the test results of the spacecraft performing the pursuit mission.

[0029] Step 2 specifically includes the following steps:

[0030] Step 2.1: Design of the state-space model for the pursuing spacecraft, the escaping spacecraft, and the threat, specifically:

[0031] By setting up the tracking spacecraft, the escape spacecraft, and the threat-bearing airborne GPS equipment and gyroscope, the position and velocity information of the spacecraft can be obtained; the airborne fire control radar payload of the tracking spacecraft can obtain the position and velocity information of the target, as shown in formula (6):

[0032] f(i) = [x i ,y i ,v i ,ψ i i = P, E, T (6)

[0033] This method uses relative information relationships to establish a state-space model, State, which compresses the measurement space, reduces the input processing pressure on the neural network, and improves the algorithm performance, allowing the algorithm to focus on learning the solution. The state-space model expression is designed as shown in formula (7):

[0034]

[0035] Where, d PE This refers to the distance between the pursuing spacecraft and the escaping spacecraft. α refers to the distance between the pursuing spacecraft and various threats.PE It is the angle between the velocity directions of the pursuing spacecraft P and the escaping spacecraft E and the target's line-of-sight (LOS). It is to hunt down spacecraft P and various threats T. i The angle between the velocity direction and the target line of sight (LOS), where LOS refers to the direction of the vector pointing from the position of the pursuing spacecraft to the target;

[0036] Step 2.2: Design of the motion space model for the pursuit and escape spacecraft, specifically:

[0037] The control inputs for the spacecraft being pursued are angular velocity and acceleration. It is assumed that the spacecraft's dynamic equations set the spacecraft to uniform motion, and the action space is shown in equation (8):

[0038] Action=[ω] (8)

[0039] The maximum angular velocity of the spacecraft is set to be less than 25.5 rad / sec, i.e., ω∈[-25.5,25.5], and the counterclockwise direction in the top view is taken as the positive direction;

[0040] Step 2.3: Design of the state transition model for the pursuing and escaping spacecraft, specifically:

[0041] The spacecraft motion state transition equation is shown in equation (9):

[0042]

[0043] Where 'i' refers to the spacecraft being pursued or the spacecraft escaping; and 's' refers to the spacecraft in its current state. t Take action A t Interact with the scene to obtain the change in state transition, current state s t Add the change to obtain the next state s t+1 .

[0044] Step 3, based on the spacecraft pursuit mission scenario model established in Step 1, uses a hierarchical control method to create a hierarchical simplified model for the spacecraft pursuit mission, simplifying the mission into multiple levels of sub-tasks. Specifically:

[0045] First, the first phase of the mission refers to the situation where the pursuing spacecraft does not receive any threat signals from the environmental measurement information. At this time, the pursuing spacecraft should continuously move towards the escaping spacecraft under the drive of the designed autonomous motion planning model.

[0046] Secondly, the second phase of the mission refers to the moment when the pursuing spacecraft receives threat information from environmental measurements, i.e., when the threat appears in the pursuing spacecraft's line-of-sight (LOS) of the target. At this point, the pursuing spacecraft's current mission should be to perform an evasive maneuver around the threat. The pursuing spacecraft should first fly into the flight trajectory set by the LGVF method, then perform an evasive maneuver along the trajectory. Finally, when the angle between the pursuing target, the pursuing spacecraft, and the threat is an obtuse angle, i.e., the pursuing spacecraft has successfully evaded the threat, the pursuing spacecraft exits the current second phase mission and enters the first phase mission, continuing to move toward the escape spacecraft.

[0047] Step 4 specifically includes:

[0048] An autonomous motion planning and control architecture is established through the SAC algorithm of deep reinforcement learning. Its end-to-end algorithmic nature enables the spacecraft being pursued to learn the common features of the problem through training. At the same time, the offline trained model can be directly used for online testing applications, providing the ability to handle dynamic uncertain states and meeting the current dynamic pursuit scenarios that require real-time optimization. The proposed entropy coefficient, which is adaptive according to the different sub-tasks of the spacecraft being pursued, improves the learning efficiency of the SAC algorithm and the tracking accuracy of the spacecraft.

[0049] For general deep reinforcement learning algorithms, the learning objective is to learn a policy that maximizes the expected value of the cumulative reward obtained from the interaction between the spacecraft and the environment, as shown in formula (10):

[0050]

[0051] Wherein, R(s) t ,a t ) represents the state s t Take action a t The reward value obtained, strategy π * The goal is to maximize the expected total reward value; while the SAC algorithm belongs to the maximum entropy policy reinforcement learning algorithm, that is, in addition to the above basic goal, it also requires that the action entropy of each output of the policy be maximized, as shown in formula (11):

[0052]

[0053] Where α is the entropy coefficient, H(π(·|s t The entropy term is required to maximize the entropy, which is mainly to randomize the strategy, that is, to balance the probability of each action output as much as possible. Chasing spacecraft requires exploring all possible optimal paths.

[0054] Step 5, in detail, is as follows:

[0055] In complex scenarios where threats randomly appear in the LOS view, the spacecraft needs to make real-time evasion maneuver decisions. In this case, a trajectory controller designed based on the Lyapunov guidance vector field algorithm is needed to handle threat T. i Establish a vector field model and design an evasion trajectory for the pursuing spacecraft to guide it to stably fly around the threat;

[0056] The design of the return function is shown in formula (12):

[0057] r = r a +r b +r c (12)

[0058] Where r represents the total return value, r a The sparse reward function represents the reward given when the pursuing spacecraft successfully captures the target and the reward given when the pursuing spacecraft is intercepted. Its form is shown in formula (13):

[0059]

[0060] r b It is one of the guided reward components of the design. In the whole process, the angle between the velocity direction of the pursuing spacecraft and the escape spacecraft and the target line of sight (LOS) is calculated in each iteration step. After controlling its dimension through the weight μ1, it is added to the total reward value. Its form is shown in formula (14):

[0061] r b =-μ1α PE (14)

[0062] r c The second aspect of the design-guided reward system is that the process of pursuing the spacecraft to the escape spacecraft is divided into two stages, as shown in formula (15):

[0063]

[0064] Stage 1 refers to the phase where the pursuing spacecraft does not perceive any threat, i.e., when the pursuing spacecraft is in the first stage of its mission. During this phase, driven by its designed autonomous motion planning model, the pursuing spacecraft continuously moves towards the escape spacecraft. c It does not play a guiding role; Stage 2 refers to the situation where a threat appears in the path of the vector pointing from the pursuing spacecraft to the escaped spacecraft, i.e., when the pursuing spacecraft is in the second stage of the mission. At this time, a vector field model is established for the threat T according to the Lyapunov guidance vector field algorithm, and the vector field radius is designed to be d. PT +w P , where w PThis refers to the width of the spacecraft being pursued; the threat level is expressed as an exponential function. At this point, r... c The distance between the spacecraft being pursued and the threat is guided; otherwise, the resulting negative reward value is added to the total reward value as a penalty after the reward value dimension is controlled by weight μ2.

[0065] Step 6 applies the above-mentioned combined optimization method to the established spacecraft pursuit mission scenario to train the autonomous motion planning model of the pursuing spacecraft. The specific process is as follows:

[0066] Step 6.1: Load the spacecraft pursuit task into the deep reinforcement learning algorithm, specifically:

[0067] In the established spacecraft pursuit mission scenario, the spacecraft to be pursued will be in its current state s t Take action a t Transition to the next state s t+1 The state transition process can be viewed as a Markov process involving the introduction of actions and rewards in reinforcement learning; according to the five-tuple of the Markov process model... In the description, S is the state space in the process, i.e., the State mentioned above, and A is the set of actions, i.e., the Action mentioned above. It is a time series set, where p is the state transition probability function and γ is the reward function for state stepping.

[0068] At each decision time t, the state transition probability matrix p obtained from the finite action space is... t As shown in formula (16):

[0069] p t =[p(s′|s,a1)...p(s′|s,a1) N (16)

[0070] Where s represents the current state of the spacecraft being tracked, and the actions to be performed are {a1,...,a...}. N After that, it enters a new state s′; a i Let be the i-th action in the action space, and N be the total number of actions in the action space. p(s′|s,a) i ) represents the action performed by the spacecraft in pursuit. i The probability of reaching a new state s′; the reward matrix R generated by the interaction between the spacecraft and the environment. i (s,a) and the total return function R(s,a) it constitutes are shown in Equations (17) and (18), respectively:

[0071] R i (s,a)=γ(s i ,a i (17)

[0072] R(s,a)=∑ t γ(s t ,a t (18)

[0073] The total reward function, combined with the maximum entropy strategy in claim 4 above, yields the optimization objective of the SAC algorithm as shown in formula (11) above.

[0074] Step 6.2: Train the autonomous motion planning model for the pursuing spacecraft, specifically:

[0075] By adjusting the network parameters, the autonomous motion planning model is improved to guide the spacecraft's movements under different states; by adjusting the learning rate, the model training time is shortened and the model training efficiency is maximized; by adjusting the weights of each reward in the reward function, the optimization process of the spacecraft in the vast action space is guided; by adjusting the entropy coefficient, the algorithm's learning efficiency and spacecraft tracking accuracy are improved; through the above adjustments, the total reward value of the algorithm tends to stabilize and converge to a constant as the number of training iterations increases.

[0076] Step 7 specifically includes the following steps:

[0077] First, adjust the SAC algorithm network model parameters and autonomous motion planning framework structure based on the feedback results to adapt to the spacecraft pursuit mission scenario and improve the autonomous motion planning framework described in step 4.

[0078] Secondly, the LGVF model parameters described in step 5 are adjusted based on the feedback results, so that the evasion trajectory of the spacecraft designed by the vector field is more stable and better suited to the spacecraft pursuit mission scenario.

[0079] Finally, based on the feedback results, the parameters of the autonomous motion planning model described in step 6.2 are adjusted to improve the model training efficiency and the tracking accuracy of the spacecraft.

[0080] The aforementioned feedback optimization methods for the algorithm are used to improve the combined optimization method.

[0081] The beneficial effects of this invention are:

[0082] Through the above technical solutions, this invention provides a combined optimization control method for spacecraft pursuit missions based on SAC and LGVF, belonging to the field of spacecraft autonomous control technology. This invention provides a combined optimization method for pursuit-escape game problems involving multiple interceptors. First, based on the established spacecraft pursuit mission scenario, the spacecraft pursuit mission is divided into multi-level sub-tasks using a hierarchical control method. An improved SAC algorithm is proposed to establish an autonomous motion planning and control architecture for the spacecraft. The Lyapunov guided vector field method is introduced to design the trajectory controller for the pursuing spacecraft. This invention combines the optimization capabilities of traditional model algorithms, the powerful perception capabilities of deep learning, and the trial-and-error learning capabilities of reinforcement learning to form a new combined optimization algorithm, achieving new progress in the field of spacecraft autonomous control technology. It can be applied to scenarios where spacecraft need to autonomously avoid multiple interceptors during tracking and other missions. This method has relatively low requirements for the spacecraft's own computing resources, providing both real-time autonomous control capabilities and improving mission success rates. Attached Figure Description

[0083] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] Figure 1 This is a flowchart of the present invention;

[0085] Figure 2 This is a two-dimensional planar geometric model diagram of the present invention;

[0086] Figure 3 This is a convergence graph of the model training for the LGVF-SAC combinatorial optimization algorithm proposed in this invention.

[0087] Figure 4 This is a model training convergence graph of Original SAC, a comparative algorithm of the combinatorial optimization algorithm proposed in this invention. Detailed Implementation

[0088] like Figures 1-4 As shown in this embodiment, a combined optimization control method for spacecraft pursuit missions based on SAC and LGVF is implemented through the following steps:

[0089] Step 1: Establish a scenario model for the pursuit-escape game problem. The specific process is as follows:

[0090] The spacecraft pursuit mission in a dynamic unknown environment involved in this invention is described as a process in which the spacecraft needs to fly over threats in the scenario and pursue dynamically escaping spacecraft. An optimization function model is established for this problem as shown in formula (1):

[0091] mint c =G[f(P),f(E),f(T)] i (1)

[0092] objective function t c This refers to the pursuit spacecraft P (Pursuit) aiming to capture the escape spacecraft E (Escape) in the shortest possible time, where G[f(P),f(E),f(T)] i [)] refers to the fusion of overall scene measurement information, f(P), f(E), f(T) i These refer to the status information of the pursuing spacecraft, the escaping spacecraft, and the threat T (Threat), respectively.

[0093] The dynamic differential model of the pursuing and escaping spacecraft in this scenario is shown in Equation (2):

[0094]

[0095] Where, x i ,y i This is the spacecraft's current location information. They are x i ,y i The differential components, that is, the components of velocity in the two vector directions; v i a represents the speed of the spacecraft. i Indicates the acceleration of a spacecraft. It is the speed v i The differential, i.e. the rate of change of velocity, is equal to the acceleration a. i ;ψ i ω represents the heading angle of the spacecraft. i Represents the angular velocity of a spacecraft. It is the heading angle ψ i The differential component, namely the rate of change of heading angle, is equal to the angular velocity ω. i The angular velocity value of the spacecraft being pursued depends on the output of the reinforcement learning algorithm model;

[0096] The status and initial state of the spacecraft and the threat are shown in formula (3):

[0097]

[0098] Where, x i0 ,y i0 Δx represents the initial position of the spacecraft or threat. i,Δy i x represents the change in the location of the spacecraft or threat. i (t0),y i (t0) represents the spacecraft or threat originating from the initial position x. i0 ,y i0 After Δx i ,Δy i The new position generated by the displacement change; v i0 The initial velocity, v, represents the speed at which the spacecraft or threat originates. i (t0) represents the velocity of the spacecraft or threat at any time t0; ψ i0 The initial angle representing the heading angle of the spacecraft or threat, Δω i ψ represents the change in the heading angle of a spacecraft or threat. i (t0) represents the heading angle of the spacecraft or threat, determined by the initial angle ψ. i0 After Δω i The new heading angle generated by the change in angle; R i0 R represents the fire range of a spacecraft or threat initialization. i (t0) represents the effective range of fire from the spacecraft or threat at any time t0, compared to the initial effective range R. i0 The threats remain unchanged; they are randomly distributed throughout the scene to simulate scene complexity.

[0099] The condition for the success of this scenario mission is that the distance between the pursuing spacecraft P and the escaping spacecraft E is less than the pursuing spacecraft's pursuit range, as shown in formula (4):

[0100] d PE ≤R P (4)

[0101] Where, d PE R is the distance between the pursuing spacecraft P and the escaped spacecraft E. P This refers to the range of pursuit capabilities for the spacecraft P.

[0102] The scenario is set to fail when the weapon ranges of the pursuing spacecraft P and the threat T overlap, i.e., the distance between the pursuing spacecraft and the threat is less than the safe distance, as shown in formula (5):

[0103]

[0104] in, This refers to the distance between the pursuing spacecraft and each threat; l refers to the safe distance between the pursuing spacecraft and the threat, defined as the sum of the effective ranges of the pursuing spacecraft and the threat. Wherein, according to formula (3), Representing the i-th threat T iThe scope of the threat;

[0105] Step 2. Based on the scenario model established in Step 1, design state-space models for the pursuing spacecraft, the escaping spacecraft, and the threat; action-space models for the pursuing and escaping spacecraft; and state transition models for the pursuing and escaping spacecraft. Step 2 specifically includes the following steps:

[0106] Step 2.1: Design of the state-space model for the pursuing spacecraft, the escaping spacecraft, and the threat, specifically:

[0107] By setting up the tracking spacecraft, the escape spacecraft, and the threat-bearing airborne GPS equipment and gyroscope, the position and velocity information of the spacecraft can be obtained; the airborne fire control radar payload of the tracking spacecraft can obtain the position and velocity information of the target, as shown in formula (6):

[0108] f(i) = [x i ,y i ,v i ,ψ i i = P, E, T (6)

[0109] This method uses relative information relationships to establish a state-space model, State, which compresses the measurement space, reduces the input processing pressure on the neural network, and improves the algorithm performance, allowing the algorithm to focus on learning the solution. The state-space model expression is designed as shown in formula (7):

[0110]

[0111] Where, d PE This refers to the distance between the pursuing spacecraft and the escaping spacecraft. α refers to the distance between the pursuing spacecraft and various threats. PE It is the angle between the velocity directions of the pursuing spacecraft P and the escaping spacecraft E and the target's line-of-sight (LOS). It is to hunt down spacecraft P and various threats T. i The angle between the velocity direction and the target line of sight (LOS), where LOS refers to the vector direction from the position of the pursuing spacecraft to the target.

[0112] Step 2.2: Design of the motion space model for the pursuit and escape spacecraft, specifically:

[0113] The control inputs for tracking the spacecraft are designed as angular velocity and acceleration. In order to make the algorithm focus on learning the solution, this invention assumes that the spacecraft's dynamic equations set the spacecraft to uniform motion, and the action space is as shown in formula (8):

[0114] Action=[ω] (8)

[0115] In this method, the angular velocity of common spacecraft is used to design the action space. The maximum angular velocity of the spacecraft is set to be less than 25.5 rad / sec, i.e., ω∈[-25.5,25.5], and the counterclockwise direction in the top view is taken as the positive direction.

[0116] Step 2.3: Design of the state transition model for the pursuing and escaping spacecraft, specifically:

[0117] The spacecraft motion state transition equation is shown in equation (9):

[0118]

[0119] Here, 'i' refers to the spacecraft being pursued or the spacecraft escaping. The spacecraft is currently in state 's'. t Take action A t Interact with the scene to obtain the change in state transition, current state s t Add the change to obtain the next state s t+1 .

[0120] Step 3: Based on the established spacecraft pursuit mission scenario, a hierarchical simplified model is built for the spacecraft pursuit mission using a hierarchical control method. This simplifies the spacecraft pursuit mission into multiple levels of sub-tasks, specifically:

[0121] First, the first phase of the mission refers to the situation where the pursuing spacecraft does not receive any threat signals from its environmental measurements. At this point, the pursuing spacecraft should continuously move toward the escaping spacecraft under the drive of its designed autonomous motion planning model.

[0122] Secondly, the second phase of the mission refers to the moment when the pursuing spacecraft receives threat information from environmental measurements, meaning the threat appears within the pursuing spacecraft's line-of-sight (LOS) of the target. At this point, the pursuing spacecraft's current mission should be to perform an evasive maneuver around the threat. The pursuing spacecraft should first fly into the flight path set by the LGVF method, then perform an evasive maneuver along the trajectory. Finally, when the angle between the pursuing target, the pursuing spacecraft, and the threat is obtuse, indicating that the pursuing spacecraft has successfully evaded the threat, the pursuing spacecraft exits the current second phase of the mission and enters the first phase, continuing its movement towards the escape spacecraft.

[0123] That is, based on the measurement information of the spacecraft on the current environment, the present invention establishes a hierarchical simplified model for the spacecraft pursuit mission through a hierarchical control method, thereby simplifying the spacecraft pursuit mission into the two-stage mission mentioned above.

[0124] Step 4. Based on the established hierarchical simplified model of spacecraft pursuit missions, an improved deep reinforcement learning flexible actor critic (SAC) algorithm is proposed to establish an autonomous motion planning and control architecture. This provides the ability to handle dynamic uncertainties in spacecraft pursuit, meeting the current needs of real-time optimization in dynamic pursuit scenarios. Specifically:

[0125] An autonomous motion planning and control architecture is established using the SAC algorithm based on deep reinforcement learning. Its end-to-end algorithmic nature enables the spacecraft being tracked to learn common features of the problem through training. Furthermore, offline-trained models can be directly used for online testing applications, providing the ability to handle dynamic and uncertain states and meeting the needs of current dynamic tracking scenarios requiring real-time optimization. The proposed adaptive entropy coefficient based on the different sub-tasks of the tracked spacecraft improves the learning efficiency of the SAC algorithm and the tracking accuracy of the spacecraft.

[0126] For general deep reinforcement learning algorithms, the learning objective is to learn a policy model that maximizes the expected value of the cumulative reward obtained from the interaction between the spacecraft and the environment, as shown in formula (10):

[0127]

[0128] Wherein, R(s) t ,a t ) represents the state s t Take action a t The reward value obtained, strategy π * The goal is to maximize the expected total reward. The SAC algorithm is a maximum entropy policy reinforcement learning algorithm, meaning that in addition to the basic objective mentioned above, it also requires maximizing the entropy of each output of the policy, as shown in formula (11):

[0129]

[0130] Where α is the entropy coefficient, H(π(·|s t The entropy term () represents the maximum entropy, primarily aimed at randomizing the strategy. This means maximizing the probability of each output action, ensuring a balanced output. This implies that the spacecraft being pursued needs to explore all possible optimal paths. In this case, the spacecraft learns not just one, but as many ways as possible to complete the mission, allowing the learned strategy to be applicable to more complex tasks. Secondly, it provides stronger robustness; the spacecraft can explore various optimal paths from different actions, making it easier to adjust when faced with interference.

[0131] Leveraging the characteristics of the entropy term, this invention adaptively adjusts the entropy coefficient based on the different stages of the mission for the pursuing spacecraft. First, when the pursuing spacecraft does not receive threat signals from environmental measurements, it should continuously fly towards the target. At this stage, a maximum entropy strategy should be used to encourage the spacecraft to actively explore the optimal trajectory. The maximum entropy coefficient, automatically adjusted using the SAC algorithm, is retained during this phase. Second, when the pursuing spacecraft receives threat information from environmental measurements, it should fly along a pre-set orbital trajectory. During this stage, the entropy coefficient is multiplied by a constant to attenuate it, limiting the spacecraft's exploratory behavior and ensuring stable flight along the pre-set trajectory.

[0132] Step 5: Based on the established hierarchical simplified model of the spacecraft pursuit mission, under the improved SAC autonomous motion planning and control architecture, a combined method is formed by introducing the Lyapunov guided vector field method to optimize the learning process of the SAC algorithm, thus forming the combined optimization method. The specific process is as follows:

[0133] In complex scenarios where threats randomly appear in the LOS (Land of Safety) view, the spacecraft being tracked needs to make real-time evasion maneuver decisions. In this case, a trajectory controller designed based on the Lyapunov guidance vector field algorithm is used to manage threat T. i A vector field model was established, and an evasion trajectory for the pursuing spacecraft was designed to guide the pursuing spacecraft to stably fly around the threat.

[0134] The design of the return function is shown in formula (12):

[0135] r = r a +r b +r c (12)

[0136] Where r represents the total return value, r a The sparse reward function represents the reward given when the pursuing spacecraft successfully captures the target and the reward (negative value) given when the pursuing spacecraft is intercepted. Its form is shown in formula (13):

[0137]

[0138] r b It is one of the guided reward components of the design. In the whole process, the angle between the velocity direction of the pursuing spacecraft and the escape spacecraft and the target line of sight (LOS) is calculated in each iteration step. After controlling its dimension through the weight μ1, it is added to the total reward value. Its form is shown in formula (14):

[0139] r b =-μ1α PE (14)

[0140] rc The second aspect of the design-guided reward system is that the process of pursuing the spacecraft to the escape spacecraft is divided into two stages, as shown in formula (15):

[0141]

[0142] Stage 1 refers to the phase where the pursuing spacecraft does not perceive any threat, i.e., when the pursuing spacecraft is in the first stage of its mission. During this phase, driven by its designed autonomous motion planning model, the pursuing spacecraft continuously moves towards the escape spacecraft. c It does not play a guiding role; Stage 2 refers to the situation where a threat appears in the path of the vector pointing from the pursuing spacecraft to the escaped spacecraft, i.e., when the pursuing spacecraft is in the second stage of the mission. At this time, a vector field model is established for the threat T according to the Lyapunov guidance vector field algorithm, and the vector field radius is designed to be d. PT +w P , where w P This refers to the width of the spacecraft being pursued; the threat level is expressed as an exponential function. At this point, r... c The distance between the spacecraft being pursued and the threat is guided; otherwise, the resulting negative reward value is added to the total reward value as a penalty after the reward value dimension is controlled by weight μ2.

[0143] Step 6 applies the above combined optimization method to the established spacecraft pursuit mission scenario to train the autonomous motion planning model for the pursuing spacecraft. The specific process is as follows:

[0144] Step 6.1: Load the spacecraft pursuit task into the deep reinforcement learning algorithm, specifically:

[0145] In the established spacecraft pursuit mission scenario, the spacecraft to be pursued will be in its current state s t Take action a t Transition to the next state s t+1 The state transition process can be viewed as a Markov process involving the introduction of actions and rewards in reinforcement learning. This is based on the five-tuple structure of the Markov process model. In the description, S is the state space in the process, i.e., the State mentioned above, and A is the set of actions, i.e., the Action mentioned above. It is a time series set, where p is the state transition probability function and γ is the reward function for each state step.

[0146] At each decision time t, the state transition probability matrix p obtained from the finite action space is... t As shown in formula (16):

[0147] p t =[p(s′|s,a1)...p(s′|s,a1) N(16)

[0148] Where s represents the current state of the spacecraft being tracked, and the actions to be performed are {a1,...,a...}. N After that, it enters a new state s′; a i Let be the i-th action in the action space, and N be the total number of actions in the action space. p(s′|s,a) i ) represents the action performed by the spacecraft in pursuit. i The probability of reaching the new state s′. The reward matrix R generated by the interaction between the pursuit spacecraft and the environment. i (s,a) and the total return function R(s,a) it constitutes are shown in Equations (17) and (18), respectively:

[0149] R i (s,a)=γ(s i ,a i (17)

[0150] R(s,a)=Σ t γ(s t ,a t (18)

[0151] The total reward function, combined with the maximum entropy strategy in claim 4 above, yields the optimization objective of the SAC algorithm as shown in formula (11) above.

[0152] Step 6.2: Train the autonomous motion planning model for the pursuing spacecraft, specifically:

[0153] By adjusting the network parameters, the autonomous motion planning model is improved to guide the spacecraft's movements under different states. Adjusting the learning rate shortens the model training time and maximizes training efficiency. Adjusting the weights of each reward in the reward function guides the spacecraft's optimization process within a vast action space. Adjusting the entropy coefficient enhances the algorithm's learning efficiency and spacecraft tracking accuracy. Through these adjustments, the algorithm's total reward value tends to stabilize and converge to a constant as the number of training iterations increases.

[0154] Step 7: Load the autonomous motion planning model for the spacecraft trained in Step 6 into an online pursuit mission scenario where the information portion is observable but unpredictable. The combined optimization method is refined based on feedback from testing the spacecraft's performance in the pursuit mission. First, adjust the SAC algorithm network model parameters and the autonomous motion planning framework structure based on the feedback results to adapt to the spacecraft pursuit mission scenario and improve the autonomous motion planning framework. Second, adjust the LGVF model parameters based on the feedback results to make the spacecraft's threat avoidance trajectory, designed using vector fields, more stable and better suited to the spacecraft pursuit mission scenario. Finally, adjust the algorithm parameters based on the feedback results and the model training scheme described in Step 6.2, and execute the model training process again. Through these feedback optimization methods, the combined optimization method is improved.

[0155] In recent years, with the development of artificial intelligence technology, deep learning has broken through the barriers of traditional methods in many fields and achieved remarkable breakthroughs. As an important branch of deep learning, deep reinforcement learning is mainly used for sequential decision-making, that is, making action choices based on the current environmental state and continuously adjusting the model based on the feedback of the actions to achieve the set goal. Combinatorial optimization problems involve the optimal selection of decision variables in a discrete decision space, which has naturally similar characteristics to the "action selection" of reinforcement learning. Moreover, the "offline training, online decision-making" characteristics of deep reinforcement learning enable the online real-time solution of combinatorial optimization problems. Therefore, using deep reinforcement learning methods to solve combinatorial optimization problems is a good choice.

[0156] Considering the complexity of external information in practical applications, such as its unpredictability and partial observability, scenarios are difficult to solve directly by establishing mathematical models. Furthermore, the optimality of solutions directly output by non-model-based learning methods is hard to guarantee. To ensure the flexibility and reliability of the solution method, a better approach is to solve the problem using a combinatorial optimization method constructed through joint learning. First, a hierarchical control method is used to divide the task in the pursuit scenario into different stages, establishing a hierarchical simplified model. Second, an improved deep reinforcement learning flexible actor critic (SAC) algorithm is proposed to establish an autonomous motion planning and control architecture, providing the ability to handle dynamic uncertainties in spacecraft pursuit and meeting the current need for real-time optimization in dynamic pursuit scenarios. Finally, within the improved SAC algorithm framework, the Lyapunov guided vector field method (LGVF) is introduced to form a combinatorial method, compressing the solution space size to optimize the solution process in a large solution space.

[0157] After multiple tests and verifications in simulation environments, the SAC-LGVF combined optimization algorithm proposed in this invention can achieve model convergence after 90,000 training steps in each initialization scenario. In contrast, the original SAC algorithm not only struggles to converge in scenarios with the same layout, but also suffers from overfitting too early during training, leading to a decrease in convergence. As shown in Table 1, when the two trained models were tested in the same simulation environment, the spacecraft controlled by the original SAC algorithm had a lower success rate in executing missions, while the spacecraft controlled by the SAC-LGVF combined optimization algorithm had a significantly higher success rate in executing missions.

[0158] Table 1

[0159] LGVF-SAC 87％ 90,000 steps Original SAC 26％ Difficult to converge, overfitting

[0160] This invention simplifies the complexity of spacecraft pursuit scenarios by performing layered simplification. By combining the improved SAC algorithm with LGVF, it enhances the algorithm's learning and mission execution capabilities. This not only provides spacecraft with autonomy and dynamic pursuit capabilities but also ensures the control accuracy and mission success rate of spacecraft in autonomous mission execution.

Claims

1. A combined optimization control method for spacecraft pursuit missions based on SAC and LGVF, characterized in that, Specifically, it includes the following steps: Step 1: Establish a spacecraft pursuit mission scenario model. The specific process is as follows: This invention relates to spacecraft pursuit missions in dynamic, unknown environments, describing the process of pursuing a spacecraft by traversing threats within the scenario and tracking dynamically escaping spacecraft. An optimization function model is established for this problem, as shown in the formula. As shown: ； objective function It refers to the pursuit of spacecraft Capture the escaped spacecraft in the shortest possible time With the goal, This refers to the fusion of overall measurement information for the scene. , , These refer to the pursuit of spacecraft, the escape of spacecraft, and various threats. Status information, Representing the One threat; A dynamic differential model of the pursuing and escaping spacecraft is established for this scenario, as shown in the formula. As shown: ； in, This is the spacecraft's current location information. They are The differential components, that is, the components of velocity in the two vector directions; Indicates the speed of the spacecraft. Indicates the acceleration of a spacecraft. It is speed The differential, i.e. the rate of change of velocity, is equal to the acceleration. ; Indicates the heading angle of the spacecraft. Represents the angular velocity of a spacecraft. It is the heading angle The differential component, namely the rate of change of heading angle, is equal to the angular velocity. The angular velocity value of the spacecraft being pursued depends on the output of the reinforcement learning algorithm model; The state and initial state of the spacecraft and the threat are given by the formula. As shown: ； in, Represents the initial location of the spacecraft or threat. The amount representing the change in the location of the spacecraft or threat. Representing a spacecraft or threat from its initial position go through The new position generated by the displacement change; The initial velocity representing the spacecraft or threat. Represent any The speed of spacecraft or threats at any given moment; The initial angle representing the heading angle of the spacecraft or threat. The heading angle representing a spacecraft or threat is determined by the initial angle. go through The new heading angle generated by the change in angle; Represents the fire range of a spacecraft or threat initialization. Representing a spacecraft or threat in any The range of firepower at any given moment, compared to the initial range of firepower. The threats remain unchanged; they are randomly distributed throughout the scene to simulate scene complexity. The condition for success in this scenario is to pursue the spacecraft. With escape spacecraft The distance is less than the pursuit range of the pursuing spacecraft, as shown in the formula. As shown: ； in, It is to hunt spacecraft With escape spacecraft distance, It refers to the pursuit of spacecraft The scope of the pursuit; The scenario is set to fail when the mission is in pursuit of a spacecraft. With threats The weapon ranges overlap, meaning the distance between the pursuing spacecraft and the threat is less than the safe distance, as shown in the formula. As shown: ； in, This refers to the distance between the spacecraft being pursued and the various threats. The safe distance between the pursuing spacecraft and the threat is defined as the sum of the effective ranges of the pursuing spacecraft and the threat. , where, according to the formula The above, Representing the A threat The scope of the threat; Step 2: Based on the spacecraft pursuit mission scenario model established in Step 1, design the state space model, action space model, and state transition model for the pursuing spacecraft and the escaping spacecraft, respectively. Step 3: Based on the spacecraft pursuit mission scenario model established in Step 1, a hierarchical simplified model of the spacecraft pursuit mission is established through the hierarchical control method, so that the spacecraft pursuit mission is simplified into multi-level sub-tasks. Step 4: Based on the established hierarchical simplified model of spacecraft pursuit mission, an improved deep reinforcement learning flexible actor critic algorithm is proposed to establish an autonomous motion planning and control architecture, which provides the ability to handle dynamic uncertain states for spacecraft pursuit and meets the current dynamic pursuit scenario that requires real-time optimization. Step 5: Based on the established hierarchical simplified model of the spacecraft pursuit mission, under the autonomous motion planning and control architecture of the improved SAC algorithm, a combined method is formed by introducing the Lyapunov guided vector field method to optimize the learning process of the SAC algorithm and form a combined optimization method. Step 6: Apply the combined optimization method described in Step 5 to the hierarchical simplified model of the spacecraft pursuit mission established in Step 4, and train the autonomous motion planning model of the pursuit spacecraft. Step 7: Load the autonomous motion planning model of the spacecraft trained in Step 6 into an online pursuit mission simulation scenario where the information part is observable but unpredictable, and test it. Improve the combined optimization method by using the test results of the spacecraft performing the pursuit mission.

2. The spacecraft pursuit mission combined optimization control method based on SAC and LGVF, characterized in that, Step 2 specifically includes the following steps: Step 2.1: Design of the state-space model for the pursuing spacecraft, the escaping spacecraft, and the threat, specifically: By setting up tracking spacecraft, escaping spacecraft, and threat-carrying onboard GPS equipment and gyroscopes, one can obtain their own position and velocity information; the tracking spacecraft's onboard fire control radar payload can obtain the target's position and velocity information, as shown in the formula. As shown: ； This method uses relative information relationships to establish a state-space model. This reduces the measurement space size, alleviates the input processing pressure on the neural network to improve algorithm performance, and allows the algorithm to focus on learning the solution. The state-space model expression is designed as shown in the formula. As shown: ； in, This refers to the distance between the pursuing spacecraft and the escaping spacecraft. This refers to the distance between the pursuing spacecraft and various threats. It is to hunt spacecraft and escape spacecraft The angle between the velocity direction and the target's line-of-sight (LOS). It is to hunt spacecraft and various threats The angle between the velocity direction and the target line of sight (LOS), where LOS refers to the vector direction from the position of the pursuing spacecraft to the target; Step 2.2: Design of the motion space model for the pursuit and escape spacecraft, specifically: The control inputs for the pursuit spacecraft are designed as angular velocity and acceleration. It is assumed that the spacecraft's dynamic equations set it to uniform motion, and the action space... As in the formula As shown: ； Among them, the maximum angular velocity of the spacecraft is set to be less than ,Right now Take the counterclockwise direction of the top view as the positive direction; Step 2.3: Design of the state transition model for the pursuing and escaping spacecraft, specifically: The spacecraft motion state transition equation is as follows: As shown: ； in, This refers to the pursuit and escape of a spacecraft; the current state of the spacecraft. Take action Interact with the scene to obtain changes in state transitions, and the current state. Add the change to obtain the next state. .

3. The spacecraft pursuit mission combined optimization control method based on SAC and LGVF as described in claim 1, characterized in that, Step 3, based on the spacecraft pursuit mission scenario model established in Step 1, uses a hierarchical control method to create a hierarchical simplified model for the spacecraft pursuit mission, simplifying the mission into multiple levels of sub-tasks. Specifically: First, the first phase of the mission refers to the situation where the pursuing spacecraft does not receive any threat signals from the environmental measurement information. At this time, the pursuing spacecraft should continuously move towards the escaping spacecraft under the drive of the designed autonomous motion planning model. Secondly, the second phase of the mission refers to the moment when the pursuing spacecraft receives threat information from environmental measurements, i.e., when the threat appears in the pursuing spacecraft's line-of-sight (LOS) of the target. At this point, the pursuing spacecraft's current mission should be to perform an evasive maneuver around the threat. The pursuing spacecraft should first fly into the flight trajectory set by the LGVF method, then perform an evasive maneuver along the trajectory. Finally, when the angle between the pursuing target, the pursuing spacecraft, and the threat is an obtuse angle, i.e., the pursuing spacecraft has successfully evaded the threat, the pursuing spacecraft exits the current second phase mission and enters the first phase mission, continuing to move toward the escape spacecraft.

4. The spacecraft pursuit mission combined optimization control method based on SAC and LGVF as described in claim 1, characterized in that, Step 4 specifically includes: An autonomous motion planning and control architecture is established through the SAC algorithm of deep reinforcement learning. Its end-to-end algorithmic nature enables the spacecraft being pursued to learn the common features of the problem through training. At the same time, the offline trained model can be directly used for online testing applications, providing the ability to handle dynamic uncertain states and meeting the current dynamic pursuit scenarios that require real-time optimization. The proposed entropy coefficient, which is adaptive according to the different sub-tasks of the spacecraft being pursued, improves the learning efficiency of the SAC algorithm and the tracking accuracy of the spacecraft. For typical deep reinforcement learning algorithms, the learning objective is to learn a policy that maximizes the expected cumulative reward obtained from the interaction between the spacecraft and its environment, as shown in the formula. As shown: ； in, Represents the state Take action The reward value obtained, strategy The goal is to maximize the expected total reward; while the SAC algorithm belongs to the maximum entropy policy reinforcement learning algorithm, which, in addition to the above basic goal, also requires that the action entropy of each output of the policy be maximized, as shown in the formula. As shown: ； in, The entropy coefficient, The purpose of maximizing entropy is to randomize the strategy, that is, to balance the probability of each action output as much as possible. Chasing a spacecraft requires exploring all possible optimal paths.

5. The spacecraft pursuit mission combined optimization control method based on SAC and LGVF as described in claim 1, characterized in that, Step 5, the specific process is as follows: In complex scenarios where threats randomly appear in the LOS (Land of Safety) view, the spacecraft needs to make real-time evasion maneuver decisions. In this case, a trajectory controller designed based on the Lyapunov guidance vector field algorithm is used to detect the threat. Establish a vector field model and design an evasion trajectory for the pursuing spacecraft to guide it to stably fly around the threat; The design of the return function is as shown in the formula. As shown: ； in, Represents the total return value. This represents the sparse reward function, specifically the reward given when the pursuing spacecraft successfully captures the target and the reward given when the pursuing spacecraft is intercepted. Its form is as follows: As shown: ； This is one of the guided reward components of the design. Throughout the process, the angle between the velocity directions of the pursuing and escaping spacecraft and the target line-of-sight (LOS) is calculated at each iteration step, and weighted accordingly. After controlling its dimensions, it is added to the total return value, in the form of the formula. As shown: ； Secondly, the guided reward system designed to facilitate the pursuit of the escaped spacecraft is divided into two phases, as shown in the formula. As shown: ； in, This refers to the situation where the pursuing spacecraft does not perceive any threat, i.e., when the pursuing spacecraft is in the first phase of its mission. At this time, driven by its designed autonomous motion planning model, the pursuing spacecraft continuously moves towards the escaping spacecraft. It does not play a guiding role; This refers to a situation where a threat appears in the path of the vector pointing from the pursuing spacecraft to the escaped spacecraft, i.e., when the pursuing spacecraft is in the second phase of its mission. In this case, the threat is assessed using the Lyapunov guidance vector field algorithm. Establish a vector field model, with the radius of the vector field designed as follows: ,in This refers to the width of the spacecraft being pursued; the threat level is expressed as an exponential function. Guide the pursuit of spacecraft control and distance from the threat; otherwise, the resulting negative reward is weighted. After controlling the dimensions of the reward value, it is added to the total reward value as a penalty.

6. The spacecraft pursuit mission combined optimization control method based on SAC and LGVF as described in claim 1, characterized in that, Step 6 applies the above-mentioned combined optimization method to the established spacecraft pursuit mission scenario to train the autonomous motion planning model of the pursuing spacecraft. The specific process is as follows: Step 6.1: Load the spacecraft pursuit task into the deep reinforcement learning algorithm, specifically: In the established spacecraft pursuit mission scenario, the spacecraft to be pursued will be in its current state. Take action Transition to the next state The state transition process can be viewed as a Markov process involving the introduction of actions and rewards in reinforcement learning. According to the quintuple of the Markov process model describe, It is the state space of the process, i.e., the above. , It is a set of actions, namely the above. , It is a time series collection. Let be the state transition probability function. The reward function during state stepping; At every decision moment The state transition probability matrix obtained from a finite action space As in the formula As shown: ； in, To track the spacecraft's current status, execute actions. Then enter a new state ; For the first in the action space One action, The total number of actions in the action space. Represents the actions taken to pursue spacecraft. Reaching a new state The probability; the reward matrix generated by the interaction between the spacecraft and the environment. and the total return function it constitutes As shown in the formulas ,formula As shown: ； ； Combining the total reward function with the maximum entropy strategy in claim 4 above yields the optimization objective of the SAC algorithm as shown in the formula above. As shown; Step 6.2: Train the autonomous motion planning model for the pursuing spacecraft, specifically: By adjusting the network parameters, the autonomous motion planning model is improved to guide the actions of spacecraft in different states; by adjusting the learning rate, the model training time is shortened and the model training efficiency is maximized; by adjusting the weights of each reward in the reward function, the optimization process of spacecraft in the vast action space is guided; by adjusting the entropy coefficient, the algorithm learning efficiency and spacecraft tracking accuracy are improved. Through the above adjustments, the total reward value of the algorithm tends to stabilize and converge to a constant as the number of training iterations increases.

7. The spacecraft pursuit mission combined optimization control method based on SAC and LGVF according to claim 6, characterized in that, Step 7 specifically includes the following steps: First, adjust the SAC algorithm network model parameters and autonomous motion planning framework structure based on the feedback results to adapt to the spacecraft pursuit mission scenario and improve the autonomous motion planning framework described in step 4. Secondly, the LGVF model parameters described in step 5 are adjusted based on the feedback results, so that the evasion trajectory of the spacecraft designed by the vector field is more stable and better suited to the spacecraft pursuit mission scenario. Finally, the parameters of the autonomous motion planning model described in step 6.2 are adjusted based on the feedback results to improve the model training efficiency and the tracking accuracy of the spacecraft. The aforementioned feedback optimization methods for the algorithm are used to improve the combined optimization method.

Images