Integrated guidance method for multi-aircraft system based on falling angle constraint interception and collision avoidance between aircrafts, electronic equipment and medium

Abstract

The invention discloses an integrated guidance method for a multi-aircraft system based on falling angle constraint interception and collision avoidance between aircrafts, electronic equipment and a medium. The method comprises the following steps: establishing a falling angle constraint guidance law based on an expected interception angle; building a collision avoidance guidance law based on graph neural network reinforcement learning; coupling cooperation is carried out on the falling angle constraint guidance law and the collision avoidance guidance law, and integrated guidance for the aircraft is achieved. Through the total guidance law formed by the fall angle constraint guidance law and the collision avoidance guidance law designed by reinforcement learning, collision avoidance among aircrafts of a multi-aircraft system and cooperative interception of a single high maneuvering target based on fall angle constraint can be realized.

Description

Technical Field

[0001] The present invention relates to the field of multi-agent collaborative guidance, and more specifically, to a multi-aircraft system integrated guidance method based on angle-of-fall constraint interception and collision avoidance between aircraft, electronic equipment, and medium. Background Art

[0002] With advances in science and technology, multi-vehicle coordinated interception technology has become an important means of improving target interception accuracy and efficiency. This is especially true when dealing with highly maneuverable targets, as multi-vehicle systems offer a higher probability of successful interception than single-vehicle systems. By leveraging the coordinated action of multiple aircraft, multi-vehicle systems can significantly improve interception rates in situations involving high-speed maneuvers and evasive maneuvers. However, ensuring the safety and effectiveness of multi-vehicle coordinated interception in highly dynamic environments remains a critical issue.

[0003] Traditional multi-vehicle systems typically employ independent guidance and simple trajectory planning strategies in collaborative interception missions. Specifically, each vehicle relies on a pre-set guidance law, lacking effective multi-vehicle collaborative collision avoidance and dynamic collaborative guidance mechanisms. When multiple vehicles attempt to intercept the same target simultaneously, overlapping paths can lead to collisions, resulting in mission failure. Traditional multi-vehicle systems typically employ fixed guidance laws or pre-planned paths, lacking intelligent coordination mechanisms and unable to adjust guidance strategies based on the target's maneuverability and dynamic changes in the environment. This lack of flexibility renders the system ineffective against highly maneuverable targets, making it prone to misses and interference between multiple vehicles. Furthermore, when a large number of vehicles are involved in an interception, the traditional system's independent control strategies can lead to a significant overlap of vehicle paths, significantly increasing the collision avoidance requirements and control complexity, making it difficult for traditional methods to effectively address this issue.

[0004] In recent years, reinforcement learning (RL), as an important tool for intelligent decision-making, has demonstrated powerful policy learning capabilities in high-dimensional control spaces and complex game interactions. In particular, within the framework of multi-agent reinforcement learning (MARL), agents can achieve a transition from local optimality to global coordination through distributed training, thereby enabling efficient intelligent coordination and dynamic adaptation in complex tasks. The integration of traditional guidance methods with reinforcement learning strategies not only retains the advantages of traditional methods in engineering implementation and physical controllability, but also enhances the system's environmental adaptability, autonomous learning, and task generalization capabilities through intelligent strategies. At the guidance law level, reinforcement learning can be embedded in the guidance framework through bias correction and adaptive weight adjustment, forming a stable and robust intelligent enhanced guidance system. At the decision-making level, it enables real-time perception and refined response to complex environments. This fusion paradigm effectively promotes the transition from model-driven to policy-driven guidance and control, improving the applicability and robustness of multi-agent systems. It also provides a theoretical basis and technical path for the intelligent upgrade of future unmanned collaborative systems, with significant engineering application value.

[0005] Therefore, it is necessary to develop an integrated guidance method, electronic equipment and medium for a multi-aircraft system based on angle-constrained interception and collision avoidance between aircraft.

[0006] The information disclosed in the background technology section of the present invention is only intended to deepen the understanding of the general background technology of the present invention, and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to those skilled in the art. Summary of the Invention

[0007] The present invention proposes an integrated guidance method, electronic equipment and medium for interception and collision avoidance between aircraft in a multi-aircraft system based on angle constraint. The method can use the backstepping method to realize the interception guidance law based on angle constraint, and the multi-agent reinforcement learning based on graph neural network to realize the collision avoidance guidance law between aircraft in the multi-aircraft system. It realizes the interception of highly maneuverable targets by the multi-aircraft system based on the expected angle while maintaining the safety and stability of the multi-aircraft system. It is practical in solving the actual application of collaborative interception and guidance of multi-aircraft systems.

[0008] In a first aspect, an embodiment of the present disclosure provides a multi-aircraft system integrated guidance method for interception based on angle constraint and collision avoidance between aircraft, comprising:

[0009] Establish an angle-constrained guidance law based on the desired intercept angle;

[0010] Establish a collision avoidance guidance law based on graph neural network reinforcement learning;

[0011] The landing angle constraint guidance law is coupled and coordinated with the collision avoidance guidance law to achieve integrated guidance for the aircraft.

[0012] Preferably, the landing angle constraint guidance law is:

[0013]

[0014] Among them, M i is the aircraft, T is the target,

[0015] Preferably, establishing a collision avoidance guidance law based on graph neural network reinforcement learning includes:

[0016] The reinforcement learning architecture for the collision avoidance guidance law is constructed. Each aircraft includes an Actor network that inputs its own observations and outputs its current action, a Critic network that inputs the states and actions of all agents and estimates Q values, and a corresponding target network for stabilizing training.

[0017] Multiple aircraft share information to form a fully connected graph structure. Through the graph neural network, aircraft can extract higher-order collaborative features from the information of other aircraft.

[0018] Set up a distance-based multi-stage collision avoidance reward function to prevent collisions between aircraft.

[0019] Preferably, the Actor network takes the global node feature matrix as input, maps the local observations of each aircraft to a high-dimensional latent space through a linear encoder, and fuses the features of all neighboring nodes through a multi-layer GNN aggregation operation;

[0020] In each layer of GNN, the strategy of removing its own neighbors is adopted to calculate the sum of all node features except its own and subtract the current node feature to realize the message passing mechanism on the fully connected graph;

[0021] After multi-layer aggregation, the feature vectors of all nodes are averaged to obtain a global aggregate representation. Finally, the output layer generates an action vector, which is normalized by the Tanh activation function and scaled with the scaling factor.

[0022] Preferably, the critic network encodes the state and action separately and adds them together, extracts collaborative features through the graph aggregation process, and finally outputs the joint Q value as:

[0023]

[0024] Preferably, the reinforcement learning architecture includes:

[0025] Define aircraft Mi The status is The input of the reinforcement learning architecture is the difference between the state of each aircraft and the rest of the aircraft. Assume that there are N aircraft in the multi-aircraft system. For aircraft M i , each neighbor node is aircraft M j The status is:

[0026]

[0027] For aircraft M i The final observation input is:

[0028]

[0029] The output of the reinforcement learning architecture is the aircraft M i Collision avoidance overload instructions make At each step state update

[0030] Preferably, setting a distance-based multi-stage collision avoidance reward function includes:

[0031] The i-th aircraft M i With other aircraft M j The distance is Then its collision avoidance reward r i for:

[0032] when When the aircraft M i Stay away from other aircraft and receive a small positive reward. The instantaneous reward is r i =0.05; when When the linear interpolation mechanism is used to smooth the reward, the instantaneous reward changes from r i = -0.1 transition to r i =0.05;

[0033] Among them, the instantaneous reward is:

[0034]

[0035] when When , the inverse square decay term is introduced to suppress the approach behavior, and the instantaneous reward is:

[0036]

[0037] when When , it means that there has been a collision between the aircraft, this round of training ends, and a strong sparse penalty r is given to each aircraft. i c=-20; when the aircraft successfully intercepts the attacking aircraft and no collision occurs between aircraft during the interception process, each aircraft is given a strong sparse reward r i u =20.

[0038] Preferably, coupling the landing angle constraint guidance law with the collision avoidance guidance law comprises:

[0039] When the aircraft M in the multi-aircraft system i With aircraft M j Distance Less than the collision avoidance response distance d exp When , the system automatically activates the collision avoidance guidance law between aircraft;

[0040] When the operating time of the multi-aircraft system reaches more than 70% of the set total interception time, the collision avoidance guidance law automatically becomes invalid and the landing angle constraint guidance law takes over.

[0041] In a second aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0042] a memory storing executable instructions;

[0043] A processor runs the executable instructions in the memory to implement the multi-aircraft system's integrated guidance method for interception based on landing angle constraints and collision avoidance between aircraft.

[0044] In a third aspect, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-aircraft system's integrated guidance method for interception based on angle constraints and collision avoidance between aircraft.

[0045] Its beneficial effects are:

[0046] (1) Aiming at the interception of a single high-maneuverable target by a multi-aircraft system, a guidance law based on the backstepping method is designed for the multi-aircraft system with an angle constraint, which can achieve saturation interception of a single high-maneuverable target by the multi-aircraft system at different angles;

[0047] (2) Aiming at the collision avoidance problem during the interception process of a multi-aircraft system, a multi-agent reinforcement learning algorithm based on a graph neural network is used to design a collision avoidance guidance law, which can achieve safety constraints during the attack process of a multi-aircraft system;

[0048] (3) Based on the above two guidance laws, a reasonable coupling cooperation mode is designed to realize the automatic adaptation of collision avoidance and guidance based on landing angle constraints of multiple aircraft during the interception process, ensuring that each aircraft can efficiently hit the target at a predetermined angle while maintaining a safe distance, thereby improving the intelligent coordination capability of the multi-aircraft system.

[0049] The methods and apparatus of the present invention have other features and advantages that will be apparent from or will be described in detail in the accompanying drawings and subsequent detailed descriptions incorporated herein, which together serve to explain the specific principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0051] Figure 1 A flowchart showing the steps of a multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft according to one embodiment of the present invention.

[0052] Figure 2 A schematic diagram of an integrated guidance framework for interception based on angle constraints and collision avoidance between aircraft in a multi-aircraft system according to an embodiment of the present invention is shown.

[0053] Figure 3 A schematic diagram of a kinematic scenario of cooperative interception of multiple aircraft in a two-dimensional environment according to an embodiment of the present invention is shown.

[0054] Figure 4 A schematic diagram of a CTDE-MADDPG architecture based on a graph neural network according to an embodiment of the present invention is shown.

[0055] Figure 5 A schematic diagram of a feature extraction network according to an embodiment of the present invention is shown.

[0056] Figure 6 A schematic diagram showing the relative distance between aircraft and the distance constraint between aircraft according to an embodiment of the present invention is shown.

[0057] Figure 7 A schematic diagram showing reinforcement learning training results according to an embodiment of the present invention is shown.

[0058] Figure 8 A schematic diagram showing the effect of integrated guided interception based on angle-of-fall constraint interception and collision avoidance between aircraft according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0060] Figure 1 A flowchart showing the steps of a multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft according to one embodiment of the present invention.

[0061] like Figure 1 As shown, the multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft includes:

[0062] Step 101, establishing an angle-of-fall constraint guidance law based on a desired intercept angle;

[0063] Step 102: Establishing a collision avoidance guidance law based on graph neural network reinforcement learning;

[0064] Step 103: Coupling the landing angle constraint guidance law with the collision avoidance guidance law to achieve integrated guidance for the aircraft.

[0065] In one example, the angle-constrained guidance law is:

[0066]

[0067] Among them, M i is the aircraft, T is the target,

[0068] In one example, establishing a collision avoidance guidance law based on graph neural network reinforcement learning includes:

[0069] Build a reinforcement learning architecture for the collision avoidance guidance law. Each aircraft includes an Actor network that inputs its own observations and outputs its current action, a Critic network that inputs the states and actions of all agents and estimates Q values, and a corresponding Target network for stabilizing training.

[0070] Multiple aircraft share information to form a fully connected graph structure. Through the graph neural network, aircraft can extract higher-order collaborative features from the information of other aircraft.

[0071] Set up a distance-based multi-stage collision avoidance reward function to prevent collisions between aircraft.

[0072] In one example, the Actor network takes the global node feature matrix as input, maps the local observations of each aircraft to a high-dimensional latent space through a linear encoder, and fuses the features of all neighboring nodes through a multi-layer GNN aggregation operation;

[0073] In each layer of GNN, the strategy of removing its own neighbors is adopted to calculate the sum of all node features except its own and subtract the current node feature to realize the message passing mechanism on the fully connected graph;

[0074] After multi-layer aggregation, the feature vectors of all nodes are averaged to obtain a global aggregate representation. Finally, the output layer generates an action vector, which is normalized by the Tanh activation function and scaled with the scaling factor.

[0075] In one example, the critic network encodes the state and action separately and adds them together, extracts collaborative features through the graph aggregation process, and finally outputs the joint Q value as:

[0076]

[0077] In one example, a reinforcement learning architecture includes:

[0078] Define aircraft M i The status is The input of the reinforcement learning architecture is the difference between the state of each aircraft and the rest of the aircraft. Assume that there are N aircraft in the multi-aircraft system. For aircraft M i , each neighbor node is aircraft M j The status is:

[0079]

[0080] For aircraft M i The final observation input is:

[0081]

[0082] The output of the reinforcement learning architecture is the aircraft M i Collision avoidance overload instructions make At each step state update

[0083] In one example, setting up a distance-based multi-stage collision avoidance reward function includes:

[0084] The i-th aircraft M i With other aircraft M j The distance is Then its collision avoidance reward r i for:

[0085] when When the aircraft M i Stay away from other aircraft and receive a small positive reward. The instantaneous reward is r i =0.05; when When the linear interpolation mechanism is used to smooth the reward, the instantaneous reward changes from r i = -0.1 transition to r i =0.05;

[0086] Among them, the instantaneous reward is:

[0087]

[0088] when When , the inverse square decay term is introduced to suppress the approach behavior, and the instantaneous reward is:

[0089]

[0090] when When , it means that there has been a collision between the aircraft, this round of training ends, and a strong sparse penalty r is given to each aircraft. i c =-20; when the aircraft successfully intercepts the attacking aircraft and no collision occurs between aircraft during the interception process, each aircraft is given a strong sparse reward r i u =20.

[0091] In one example, coupling the angle constraint guidance law with the collision avoidance guidance law includes:

[0092] When the aircraft M in the multi-aircraft system i With aircraft M j Distance Less than the collision avoidance response distance d exp When , the system automatically activates the collision avoidance guidance law between aircraft;

[0093] When the operating time of the multi-aircraft system reaches more than 70% of the set total interception time, the collision avoidance guidance law automatically becomes invalid and the landing angle constraint guidance law takes over.

[0094] Specifically, the present invention combines the angle-constrained guidance law based on backstepping and the multi-agent reinforcement learning collision avoidance guidance law based on graph neural networks, and realizes safety constraints and multi-angle saturation interception when a multi-aircraft system intercepts a single highly maneuverable target through a coupled collaborative mechanism, thereby effectively improving the safety and interception effectiveness of the multi-aircraft system.

[0095] Figure 2 A schematic diagram of an integrated guidance framework for interception based on angle constraints and collision avoidance between aircraft in a multi-aircraft system according to an embodiment of the present invention is shown.

[0096] like Figure 2As shown in this paper, first, a guidance law based on impact angle constraints is designed for a multi-vehicle system to intercept a single highly maneuverable target. Second, a multi-agent reinforcement learning-based collision avoidance guidance law based on graph neural networks is proposed to address the safety constraints between the multi-vehicle system. By coupling the impact angle constraint guidance law and the collision avoidance guidance law in a time- and distance-cooperative manner to form an overall guidance law, it is possible to achieve collision avoidance between multi-vehicle systems and the coordinated interception of a single highly maneuverable target based on impact angle constraints.

[0097] Figure 3 A schematic diagram of a kinematic scenario of cooperative interception of multiple aircraft in a two-dimensional environment according to an embodiment of the present invention is shown.

[0098] like Figure 3 The figure shows the kinematic scenario of multi-aircraft cooperative interception in a two-dimensional environment. Assuming that the influence of gravity is ignored during the modeling process, the aircraft M i The nonlinear two-dimensional kinematic equation between the target T is:

[0099]

[0100] in, For aircraft M i The relative distance to the target T; For aircraft M i The sight angle pointing to the target T; Aircraft M i The speed and speed inclination; v T ,θ T are the velocity and velocity inclination of the target T, respectively. The velocity inclination change rate of the aircraft and the target is determined by the ratio of their normal acceleration to velocity, which is expressed as follows:

[0101]

[0102] Secondly, it is defined that the multi-aircraft system can achieve safe coordination and achieve the required interception angle at the time of interception under the following three conditions:

[0103] 1. Information exchange between each aircraft in a multi-aircraft system. For an aircraft, target information can be obtained directly or indirectly through the seeker;

[0104] 2. At the end of guidance, each aircraft in the multi-aircraft system needs to achieve the desired interception angle, that is, for each aircraft:

[0105]

[0106] in, For aircraft M i The sight angle with the target T; For aircraft M iThe expected intercept angle of target T;

[0107] 3. During the guidance process, the safety collision avoidance mechanism between multiple aircraft systems is effective, that is, for the multi-aircraft system:

[0108]

[0109] Among them, d min is the minimum distance between aircraft. If the distance between two aircraft exceeds this value, the multi-aircraft system collision avoidance fails. For aircraft M i With aircraft M j The relative distance between them.

[0110] Finally, the present invention proposes a coupling cooperation mechanism based on the angle constraint interception guidance law and the collision avoidance guidance law between aircraft in a multi-aircraft system. Specifically, when the aircraft M in the multi-aircraft system i With aircraft M j Distance Less than the collision avoidance response distance d exp When the multi-aircraft system runs for more than 70% of the set total interception time, the collision avoidance guidance law automatically becomes invalid and the angle constraint interception guidance law takes over, ensuring that each aircraft hits the target at a predetermined angle in the final interception phase. This coupling and cooperation mechanism ensures the efficient and coordinated interception performance of the multi-aircraft system while ensuring safety. At the same time, the overall guidance law of the aircraft during the entire interception process satisfies:

[0111]

[0112] Among them, a max This is the maximum overload instruction that the aircraft can withstand.

[0113] Establish the guidance law with angle constraint based on the expected intercept angle. First, the backstepping method is used to design the guidance law with angle constraint based on the expected intercept angle. i The nonlinear two-dimensional kinematic equation between the aircraft M and the target T can be obtained i The rate of change of the sight angle with the target T The derivative of :

[0114]

[0115] Divide the above formula into two parts and calculate them separately, namely for The first item Substituting the above formula into the equation, we get:

[0116]

[0117] for The second item Substitute aircraft M i The nonlinear two-dimensional kinematic equation between the target T and the velocity angle change rate is:

[0118]

[0119] Combining the above two and Available The final expression is:

[0120]

[0121] Considering the aircraft guidance The nonlinear system formed by , we can get:

[0122]

[0123] Define error variables z1 and z2 such that Converges to 0, expressed as:

[0124]

[0125] Taking the derivative of the error variables z1 and z2 yields:

[0126]

[0127] in, This item is uncertain and is related to the state of the target.

[0128] Construct the Lyapunov function as:

[0129]

[0130] Taking its derivative we get:

[0131]

[0132] Since in this formula we need to make That is, we need to make both of the two terms non-positive. For the first term, we need to make c2 ≥ 1 during the design process. The focus of the design is the second term. To make the second term non-positive, the goal is to make:

[0133]

[0134] Substitute the above formula into The nonlinear system formed by , the angle constraint guidance law can be obtained as:

[0135]

[0136] Figure 4 A schematic diagram of a CTDE-MADDPG architecture based on a graph neural network according to an embodiment of the present invention is shown.

[0137] Figure 5 A schematic diagram of a feature extraction network according to an embodiment of the present invention is shown.

[0138] Establish a collision avoidance guidance law based on graph neural network reinforcement learning. In order to achieve efficient collaborative collision avoidance of multiple aircraft systems in complex dynamic environments, this paper adopts a hybrid framework that integrates graph neural networks and multi-agent deterministic policy gradients. Figure 4 The figure shows the collision avoidance guidance law reinforcement learning architecture adopted by the present invention. The architecture adopts the idea of ​​centralized training and distributed execution. In this process, each aircraft has an Actor network for inputting its own observations and outputting current actions, a Critic network for inputting the states and actions of all agents and for estimating Q values, and a corresponding target network for stable training. The multi-aircraft system has a stable information sharing capability, and the information between agents is fully interoperable, forming a fully connected graph structure. Each aircraft is both a node in the graph and can exchange information with all other nodes. Through the graph neural network, the aircraft can extract higher-order collaborative features from the information of other aircraft, rather than simply splicing all states. As shown in FIG Figure 5 The Actor and Critic networks shown in the figure both use feature extraction architectures. The Actor network takes a global node feature matrix as input, first mapping each agent's local observations to a high-dimensional latent space via a linear encoder. Then, a multi-layer GNN aggregation operation fuses the features of all neighboring nodes. Within each GNN layer, a strategy of removing its own neighbors is employed. This strategy calculates the sum of all node features except its own and subtracts the current node's features, implementing a message passing mechanism on a fully connected graph. Its expression is:

[0139]

[0140] After multiple layers of aggregation, the feature vectors of all nodes are averaged to obtain a global aggregate representation. The output layer finally generates an action vector, which is normalized by the Tanh activation function and scaled with a scaling factor. The Critic network uses a similar structure, encoding the state and action separately and adding them together. Then, the graph aggregation process is also used to extract collaborative features, and the final output is the joint Q value, which is expressed as:

[0141]

[0142] This structure can not only explicitly model the dependencies between multiple agents, but also automatically learn efficient collaboration strategies under a fully connected communication topology, enhancing the adaptability and generalization capabilities of the strategies to systems of different scales.

[0143] Define aircraft M i The status is The input of the reinforcement learning network is the difference between the state of each aircraft and the rest of the aircraft. Assuming that there are N aircraft in the multi-aircraft system, for each aircraft, the relative relationship between it and the rest of the aircraft is more important than the absolute state of the aircraft. i Each neighbor node that needs to be considered is the aircraft M j Status is For aircraft M i The final observation input is:

[0144]

[0145] The observation dimension is 4(N-1) dimensions.

[0146] The output of the reinforcement learning neural network is the aircraft M i Collision avoidance overload instructions In the design process At each step state update

[0147] Figure 6 A schematic diagram showing the relative distance between aircraft and the distance constraint between aircraft according to an embodiment of the present invention is shown.

[0148] In order to effectively guide the aircraft to maintain a safe distance and prevent collisions between aircraft during the interception process based on landing constraints, a distance-based multi-stage collision avoidance reward function is designed. Figure 6 Schematic diagram of the aircraft range, the reward function is based on the relative distance between aircraft Collision avoidance response distance d from the aircraft exp , maximum collaborative constraint distance d max and minimum safety distance d min Specifically, the relationship between the i-th aircraft M i With other aircraft M j The distance is Then its collision avoidance reward r i The definition is as follows: When the aircraft M i Stay away from other aircraft and receive a small positive reward. The instantaneous reward is r i =0.05; when When the linear interpolation mechanism is used to smooth the reward, the instantaneous reward changes from r i= -0.1 transition to r i = 0.05, the instantaneous reward is defined as:

[0149]

[0150] when In order to prevent the aircraft from getting too close, an inverse square decay term is introduced to suppress the approach behavior. The instantaneous reward is:

[0151]

[0152] when When , it means that there has been a collision between the aircraft, this round of training ends, and a strong sparse penalty r is given to each aircraft. i c =-20; when the aircraft successfully intercepts the attacking aircraft and no collision occurs between aircraft during the interception process, each aircraft is given a strong sparse reward r i u =20. This design, through a phased reward mechanism, encourages the multi-vehicle system to maintain safe spacing during the initial training phase, reducing collision risk. Furthermore, the reward function incorporates smooth interpolation and nonlinear attenuation, effectively mitigating the discrete nature of the reward signal and improving the stability and sensitivity of policy learning.

[0153] The key points and protection points of this method include:

[0154] 1. An integrated guidance framework for multi-vehicle systems based on angle-constrained interception and inter-vehicle collision avoidance: This framework integrates an angle-constrained guidance law based on backstepping and employs a multi-agent reinforcement learning collision avoidance guidance law based on a graph neural network. Through a coupled collaborative mechanism, it ensures both safety constraints and angle-constrained interception in dynamic missions, significantly improving the efficiency of coordinated interception and mission success rates for multi-vehicle systems.

[0155] 2. Multi-agent reinforcement learning collision avoidance guidance law based on graph neural network: To solve the collision avoidance problem in the interception process between multiple aircraft systems, a multi-agent reinforcement learning method based on graph neural network is adopted. By introducing a reasonable state space and action space, and designing a reasonable reward function, the safety of the multi-aircraft system during the interception process is improved.

[0156] The present invention also provides an electronic device, which includes: a memory storing executable instructions; a processor running the executable instructions in the memory to implement the above-mentioned multi-aircraft system based on angle constraint interception and integrated guidance method for collision avoidance between aircraft.

[0157] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft.

[0158] To facilitate understanding of the solutions and effects of the embodiments of the present invention, three specific application examples are given below. Those skilled in the art should understand that these examples are only for facilitating understanding of the present invention, and any specific details thereof are not intended to limit the present invention in any way.

[0159] Example 1

[0160] To verify the effectiveness and safety of a multi-vehicle system's coordinated interception of a single, highly maneuverable target using an integrated guidance method based on angle-constrained interception and collision avoidance between vehicles, a series of simulation experiments were designed to validate the guidance law. The experiments consisted of two main parts: first, setting appropriate hyperparameters for reinforcement learning training; and second, numerical simulations verifying the integrated guidance method based on angle-constrained interception and collision avoidance between vehicles.

[0161] This paper uses the PyTorch library to implement the reinforcement learning portion of the training for collision avoidance and angle-constrained guidance for a multi-aircraft system. The network parameters are optimized using the Adam optimizer. Table 1 shows the parameter settings used. Based on the fast convergence of DDPG, the proposed algorithm was trained for 100 rounds, with the exploration parameter ∈ decaying exponentially from an initial value of 1.0 to a minimum value of 0.05. During each round, random parameters were introduced to initialize the positions of the aircraft and the attacking aircraft.

[0162] Table 1 Guidance law design parameter settings

[0163]

[0164]

[0165] Figure 7 A schematic diagram showing reinforcement learning training results according to an embodiment of the present invention is shown.

[0166] like Figure 7 As shown in the figure, (a) is the training LOSS curve, and (b) is the training reward curve, which shows the average LOSS and reward change trends of each round in the process of collaborative training of two aircraft agents based on the GNN-MADDPG method. Figure 7As can be seen, during the training process, the Actor Loss curve remains consistently negative and gradually stabilizes, indicating that the Actor network is able to continuously improve its performance in policy learning and that the policy is gradually converging. The Critic Loss fluctuates significantly in the early stages of training but rapidly decreases and approaches zero, indicating that the Critic network's Q-value predictions are becoming increasingly accurate. Overall, both the Actor and Critic remain stable in the later stages of training, indicating that both the policy and value function have converged and that the training effect is good. At the same time, the overall reward level shows a clear upward trend, reflecting the gradual optimization of the aircraft policy and the continued improvement of the training effect. This reward trend demonstrates that the proposed GNN-MADDPG algorithm can effectively guide multiple aircraft to learn high-quality collaborative collision avoidance strategies in complex environments.

[0167] Figure 8 A schematic diagram showing the effect of integrated guided interception based on angle-of-fall constraint interception and collision avoidance between aircraft according to an embodiment of the present invention is shown.

[0168] like Figure 8 Figure 1 shows the effectiveness of integrated guidance and interception based on angle-of-fall constraint interception and collision avoidance between aircraft. Figure (a) compares the interception trajectories of two aircraft with and without the reinforcement learning strategy. Aircraft M1 and M2 are represented by two different orange colors, with the solid and dashed lines distinguishing whether the reinforcement learning strategy was introduced. The blue dashed line represents the target's flight trajectory, with the target overload always at 0.5g. The dots represent the starting positions of each aircraft, and the five-pointed star represents the interception endpoint. As can be seen from the figure, while the two aircraft ultimately successfully intercept the target without the reinforcement learning strategy, there is significant overlap and proximity between their trajectories during flight, indicating limited collaborative collision avoidance capabilities and a potential collision risk. In contrast, after the reinforcement learning-based collision avoidance guidance law was introduced, the aircraft demonstrated enhanced collaborative avoidance capabilities while maintaining interception accuracy. Specifically, the two aircraft maintained a more reasonable relative distance during flight, with more separated trajectories, effectively avoiding mid-course cross-interference. Figure (b) shows the time-varying curve of the relative distance between aircraft M1 and M2 with and without the reinforcement learning control strategy, thereby comparing the differences in the collaborative collision avoidance performance of the two strategies. The blue dotted line in the figure represents the change in the distance between aircraft when the reinforcement learning strategy is not introduced, and the orange solid line represents the result after adopting the multi-agent reinforcement learning strategy based on the graph neural network; at the same time, the minimum safety distance and the maximum collaborative constraint distance are marked in the figure as reference standards for judging the effectiveness of collaborative collision avoidance. It can be observed from the figure that without the introduction of reinforcement learning, the distance between aircraft changes dramatically in the initial stage, and the minimum distance drops to below d around t = 5s. minThere is a serious risk of collision. However, after introducing the reinforcement learning strategy, the distance between the aircraft remained above 320 meters throughout the mission. To obtain higher rewards, the multi-aircraft system even maintained the distance above 700 meters throughout the mission. This result further verifies that the designed multi-agent reinforcement learning method based on graph neural networks can significantly improve the coordinated collision avoidance capabilities between aircraft, effectively constrain relative motion states, and enhance the overall safety and coordination of the system. Figure (c) shows the temporal evolution of the total g-load of the two aircraft during the mission, used to evaluate the dynamic response performance of the aircraft guidance system under the multi-agent reinforcement learning guidance based on graph neural networks. The orange and red curves in the figure correspond to the total g-load variation trajectories of aircraft M1 and M2, respectively. As can be seen from the figure, the overall acceleration changes of the two aircraft are within the physically controllable range and meet the set input g-load constraints. During the mission, the total g-load of the two aircraft did not exhibit sudden overshoots or severe oscillations, indicating that under the constraints of the proposed strategy, the agents can effectively achieve rapid response while maintaining the dynamic stability of the control system. Figure (d) shows the time-varying trajectory of the two vehicles' angle-of-missile constraint errors during the coordinated intercept mission, used to evaluate the effectiveness of controlling the vehicle heading and desired intercept angle during the terminal guidance phase. The desired intercept angle errors for both vehicles exhibited a monotonic convergence trend, with both errors being less than 1 degree at the end of the intercept.

[0169] Example 2

[0170] The present disclosure provides an electronic device, which includes: a memory storing executable instructions; and a processor running the executable instructions in the memory to implement the above-mentioned multi-aircraft system based on angle constraint interception and integrated guidance method for collision avoidance between aircraft.

[0171] An electronic device according to an embodiment of the present disclosure includes a memory and a processor.

[0172] The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc.

[0173] The processor may be a central processing unit (CPU) or other form of processing unit having data processing capability and / or instruction execution capability, and may control other components in the electronic device to perform desired functions. In one embodiment of the present disclosure, the processor is used to execute the computer-readable instructions stored in the memory.

[0174] Those skilled in the art should understand that in order to solve the technical problem of how to obtain a good user experience, this embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the scope of protection of this disclosure.

[0175] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.

[0176] Example 3

[0177] An embodiment of the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the integrated guidance method for multi-aircraft system based on angle constraint interception and collision avoidance between aircraft.

[0178] According to an embodiment of the present disclosure, a computer-readable storage medium stores non-transitory computer-readable instructions, which, when executed by a processor, execute all or part of the steps of the aforementioned methods of the embodiments of the present disclosure.

[0179] The above-mentioned computer-readable storage media include, but are not limited to, optical storage media (e.g., CD-ROMs and DVDs), magneto-optical storage media (e.g., MOs), magnetic storage media (e.g., magnetic tapes or mobile hard disks), media with built-in rewritable non-volatile memory (e.g., memory cards), and media with built-in ROM (e.g., ROM cartridges).

[0180] Those skilled in the art should understand that the above description of the embodiments of the present invention is only for the purpose of illustrative purposes only to illustrate the beneficial effects of the embodiments of the present invention, and is not intended to limit the embodiments of the present invention to any given examples.

[0181] While various embodiments of the present invention have been described above, the above description is intended to be illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft, characterized in that: include: Establish an angle-constrained guidance law based on the desired intercept angle; Establish a collision avoidance guidance law based on graph neural network reinforcement learning; The landing angle constraint guidance law is coupled and coordinated with the collision avoidance guidance law to achieve integrated guidance for the aircraft.

2. The multi-aircraft system according to claim 1, wherein: The landing angle constraint guidance law is: Among them, M i is the aircraft, T is the target, is the relative distance between the aircraft and the target, is the aircraft velocity inclination angle, θ T is the target velocity inclination angle, is the line of sight angle between the aircraft and the target, is the expected interception angle of the aircraft to the target, t go is the remaining interception time, is the first state variable of the constructed nonlinear system, is the first state variable of the constructed nonlinear system, is the nonlinear system disturbance, a T is the target overload, c1, c2, c3 are the guidance law parameters respectively.

3. The multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft according to claim 1, wherein: Building a collision avoidance guidance law based on graph neural network reinforcement learning includes: The reinforcement learning architecture for the collision avoidance guidance law is constructed. Each aircraft includes an Actor network that inputs its own observations and outputs its current action, a Critic network that inputs the states and actions of all agents and estimates Q values, and a corresponding target network for stabilizing training. Multiple aircraft share information to form a fully connected graph structure. Through the graph neural network, aircraft can extract higher-order collaborative features from the information of other aircraft. Set up a distance-based multi-stage collision avoidance reward function to prevent collisions between aircraft.

4. The multi-aircraft system according to claim 3, wherein: The Actor network takes the global node feature matrix as input, maps the local observations of each aircraft to a high-dimensional latent space through a linear encoder, and fuses the features of all neighboring nodes through a multi-layer GNN aggregation operation; In each layer of GNN, the strategy of removing its own neighbors is adopted to calculate the sum of all node features except its own and subtract the current node feature to realize the message passing mechanism on the fully connected graph; After multi-layer aggregation, the feature vectors of all nodes are averaged to obtain a global aggregate representation. Finally, the output layer generates an action vector, which is normalized by the Tanh activation function and scaled with the scaling factor.

5. The multi-aircraft system according to claim 3, wherein: The critic network encodes the state and action separately and adds them together, extracts collaborative features through the graph aggregation process, and finally outputs the joint Q value as:

6. The multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft according to claim 3, wherein: The reinforcement learning architecture includes: Define aircraft M i The status is The input of the reinforcement learning architecture is the difference between the state of each aircraft and the rest of the aircraft. Assume that there are N aircraft in the multi-aircraft system. For aircraft M i , each neighbor node is aircraft M j The status is: For aircraft M i The final observation input is: The output of the reinforcement learning architecture is the aircraft M i Collision avoidance overload instructions make At each step state update 7. The multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft according to claim 3, wherein: Setting up a distance-based multi-stage collision avoidance reward function involves: The i-th aircraft M i With other aircraft M j The distance is Then its collision avoidance reward r i for: when When the aircraft M i Stay away from other aircraft and receive a small positive reward. The instantaneous reward is r i =0.05; when When the linear interpolation mechanism is used to smooth the reward, the instantaneous reward changes from r i = -0.1 transition to r i =0.05; Among them, the instantaneous reward is: when When , the inverse square decay term is introduced to suppress the approach behavior, and the instantaneous reward is: when When , it means that there has been a collision between the aircraft, this round of training ends, and a strong sparse penalty r is given to each aircraft. i c =-20; when the aircraft successfully intercepts the attacking aircraft and no collision occurs between aircraft during the interception process, each aircraft is given a strong sparse reward r i u =20.

8. The multi-aircraft system integrated guidance method based on angle constraint interception and collision avoidance between aircraft according to claim 1, wherein: Coupling the angle constraint guidance law with the collision avoidance guidance law includes: When the aircraft M in the multi-aircraft system i With aircraft M j Distance d Mi,Mj Less than the collision avoidance response distance d exp When , the system automatically activates the collision avoidance guidance law between aircraft; When the operating time of the multi-aircraft system reaches more than 70% of the set total interception time, the collision avoidance guidance law automatically becomes invalid and the landing angle constraint guidance law takes over.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing executable instructions; A processor that runs the executable instructions in the memory to implement the integrated guidance method for interception based on landing angle constraints and collision avoidance between aircraft for a multi-aircraft system according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the integrated guidance method for interception based on angle constraint and collision avoidance between aircraft for a multi-aircraft system according to any one of claims 1 to 8.

Images