Communication degradation adaptive cooperative interference and trajectory planning integrated method

Abstract

The invention relates to a communication degradation self-adaptive cooperative interference and trajectory planning integration method. The method comprises the following steps: configuring an agent group and an environment; when the communication rate is not lower than 0.1 kbps, a cooperative interference model is constructed based on a multi-agent cooperative strategy trained by an agent replacement framework, and when the communication rate is lower than 0.1 kbps, a cooperative interference model is constructed based on a bipartite graph, and target interference suppression is carried out; a trajectory planning method based on a sparse A * algorithm is adopted to construct a collaborative avoidance model, and when the output of the collaborative interference model reaches a steady state, the current position, the predicted arrival position, all target positions and the radius of a suppressed threat area of each agent are input into the collaborative avoidance model; and generating a ground track through the lateral track key parameters, generating an evasion waypoint according to the ground track, and executing an evasion action by each agent to reach a target position. According to the method, the collaborative decision-making problem of performing interference resource allocation on a plurality of targets in a task area and avoiding a threat area under the conditions of communication degradation and real-time battle damage is solved.

Description

Technical Field

[0001] The present invention belongs to the field of swarm intelligence collaborative decision-making, and relates to an integrated method for collaborative jamming and trajectory planning with communication degradation adaptation. Background Art

[0002] At present, the wide application of multi-agent reinforcement learning in fields such as intelligent logistics, UAV swarm control, and game AI has made it an important method for realizing swarm intelligence and gradually become a research hotspot in the field of machine learning and even artificial intelligence. In recent years, the emergence of various multi-agent reinforcement learning algorithms has injected vigorous vitality into this research field. This part classifies multi-agent deep reinforcement learning methods into two types: communication and collaboration.

[0003] Different from single-agent, in a multi-agent scenario, agents need to communicate effectively in order to make the best decision overall. Considering the large number of agents, it is crucial how to effectively transmit information between agents and utilize it. The goal of an agent is to reach a consensus and cooperate through communication. Each agent needs to learn to communicate with other agents and process the information received during the communication process to achieve collaboration. Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL) are relatively representative communication-based algorithms. They follow the centralized training and decentralized execution framework and both use a centralized Q-network to transmit information between agents. The output of this network includes not only Q-values but also information for interaction between agents. RAIL and DIAL algorithms can only transmit discretized information, which limits the amount of information and real-time degree of communication between agents. To solve this problem, Sukhbaatar et al. proposed the CommNet algorithm, which constructs a communication channel with the ability to transmit continuous information between agents, ensuring that any agent in the environment can transmit information in real time. Considerable progress has also been made in domestic research on communication-based MDRL. The most famous of these is the Bidirectionally-Coordinated Nets (BiCNet) proposed by the Alibaba team, which aims to complete the micro-management tasks in the real-time strategy game StarCraft II, that is, to achieve the control of friendly units in a low-level, short-term combat environment. The multi-agent team at Peking University also proposed a communication model based on the attention mechanism (ATOC Architecture), enabling agents to have the ability to autonomously select communication objects.

[0004] Although the method based on explicit communication is a new development trend in multi-agent deep reinforcement learning, a large amount of work also focuses on collaborative methods that do not involve communication. The method based on collaboration is an important algorithm framework among them. Value Decomposition Networks (VDN) and its improved versions such as QMIX and QTRAN split the global reward of the environment according to the contribution made by each agent to the environment. After decomposition, each value function is only related to the historical state and action of the agent itself. Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is an algorithm based on the AC framework and follows the principle of centralized training and decentralized execution. Counterfactual Multi-Agent Policy Gradients (COMA) mainly solves the multi-agent credit assignment problem, that is, how to assign reward values to each agent in a cooperative environment where only the global reward can be obtained. There have also been significant progress in the research of multi-agent collaborative algorithms in China. The Deep Reinforcement Learning Laboratory of Tianjin University proposed the Weighted Double Deep Q-Network (WDDQN) algorithm. This method incorporates the double Q-network structure and the Lenient Reward theory into the classic DQN algorithm. The former mainly solves the inherent overestimation problem of deep reinforcement learning algorithms, and the latter focuses on improving the random policy update ability in cooperative multi-agent environments. In addition, due to the curse of dimensionality in MARL, which leads to poor scalability and low sampling efficiency, the Deep Reinforcement Learning Laboratory of Tianjin University proposed a unified agent permutation framework (Agent Permutation Invariant, API). This framework uses permutation invariance (PI) and permutation equivariance (PE) inductive biases to reduce the multi-agent state space.

[0005] The advantage of the communication-based method is that the explicit channel established between agents by the algorithm can enable agents to learn better collective policies. However, its main disadvantage is that due to the large number of parameters required for the establishment of the channel, the design architecture of the algorithm is generally more complex. Although the collaborative method does not require a complex communication modeling process, due to the incorporation of the rules of traditional multi-agent algorithms in the training process, it has both ease of implementation and high efficiency. However, its disadvantage is that the applicable environment is relatively single (it cannot handle fully adversarial environments and scenarios with limited communication). Summary of the Invention

[0006] The technical problem solved by the present invention is: aiming at the problem of how to allocate interference resources for multiple targets in a mission area and cooperate to avoid threat areas under communication degradation, a method for integrating cooperative interference and trajectory planning adaptable to communication degradation is proposed, which solves the problems in the prior art that it is difficult to adapt to changes in communication rate, difficult to adapt to zero communication conditions, and unable to cope with changes in the number of agents caused by battle damage during the process of allocating interference resources and avoiding threat areas, and improves the autonomous cooperative decision-making ability of the cluster.

[0007] The solution of the present invention to solve the technical problem is: in the first aspect, a method for integrating cooperative interference and trajectory planning adaptable to communication degradation is proposed, including the following steps:

[0008] A cluster is formed by multiple mobile agents equipped with jammers and passive radars to perform tasks. A mission area is set, and each agent has a set target position. The mission process is set as the agent cluster starts from the starting position and reaches its respective target positions in the mission area;

[0009] When the communication rate is not lower than 0.1 kbps, a cooperative interference model is constructed based on the multi-agent cooperative strategy trained by the agent replacement framework. During the process of each agent reaching the mission area, the cooperative interference model controls the agent to continuously use the carried passive radar to detect the targets in the mission area, share the detection information according to the communication conditions, receive the information shared by other agents and input it into the cooperative interference model. The cooperative interference model outputs the target sequence to be suppressed by each agent, and based on this, the actual threat area radius after each target is suppressed is calculated;

[0010] When the communication rate is lower than 0.1 kbps, a cooperative interference model is constructed based on the bipartite graph method. The agents and targets are defined as two types of nodes with different attributes to establish connection relationships. The current number of agents and the number of targets are input, and a globally optimal interference resource allocation scheme is obtained by using the cooperative interference model. Each agent predicts the interference behavior of other agents through this scheme to achieve cooperative interference in the zero communication state;

[0011] A trajectory planning method based on the sparse A* algorithm is used to construct a cooperative avoidance model. When all jammers are turned on to the maximum power, the output of the cooperative interference model reaches a steady state. At this time, the current positions, expected arrival positions, positions of all targets, and the actual threat area radius after suppression of each agent are input into the cooperative avoidance model. The ground trajectory is generated through the key parameters of the lateral trajectory and the avoidance waypoints are generated according to the ground trajectory. Each agent performs an avoidance action and reaches the set target position.

[0012] Further, constructing a cooperative interference model based on the multi-agent cooperative strategy trained by the agent replacement framework includes:

[0013] Obtain the local observation data of each agent, including its own position and the target positions it sees. Decode the communication information transmitted by other agents to obtain the positions of teammates and the target positions they see. Use the above information as the input information for each agent;

[0014] After passing through a value network for each agent's input information, obtain the local Q-values of each agent, and use a hybrid network to merge the single-agent local value functions to obtain the joint Q-value;

[0015] Set the weight w of the network parameters to be non-negative and the bias to be unrestricted to achieve optimization monotonicity. Take the argmax of the joint Q-value, which is equivalent to taking the argmax of each local Q-value, to ensure that when optimizing the joint Q-value, each local Q-value is also maximized;

[0016] Adopt the method of centralized training and distributed execution to train the entire network architecture. Each agent outputs the target sequence of its own interference suppression, and compresses and encodes its own position and the observed target position as communication information and transmits it to other agents to meet the situation of communication rate degradation and use it as the decision input information for the next step.

[0017] Furthermore, construct a cooperative interference model based on the bipartite graph method, including:

[0018] Use the bipartite graph model to model the entire interference stage as a bipartite graph. Regard agents and targets as two types of nodes with different attributes in the bipartite graph. There are edges only between nodes with different attributes. The practical meaning of connecting two nodes a and b is that agent a suppresses interference target b;

[0019] Obtain the current number of agents and the number of targets that have been observed, and construct a complete bipartite graph;

[0020] Each agent inputs the current number of agents and the number of targets that have been observed into the cooperative interference model based on the bipartite graph, and obtains a globally optimal cooperative interference strategy, including the predicted interference target sequences of other agents. Calculate the actual threat zone radius after each target is suppressed according to the prediction results and use it as the input of the cooperative avoidance model.

[0021] Furthermore, adopt a trajectory planning method based on the sparse A* algorithm to construct a cooperative avoidance model, including:

[0022] During the search process of the sparse A* algorithm, map the flight path to a two-dimensional ground trajectory and perform the following operations:

[0023] The track constraint conditions are incorporated into the sparse A* algorithm. Let the minimum step length of the agent be L, the maximum turning angle be θ, and the velocity direction of the agent be introduced. Then, the search interval for the next position from the current position is restricted within a sector area with an angle of 2θ.

[0024] The maximum track length L max represents the upper limit of the remaining energy or time for a specific task. During the search process, tracks with a length greater than L max are considered invalid tracks. For a given node x, only when D(x)+SL(x)≤L max is satisfied, x is added to the possible path nodes, where D(x) is the actual distance traveled from the starting position to x, and SL(x) is the straight-line distance from x to the target.

[0025] Define the track cost function of the sparse A* algorithm as: f(x) = g(x)+h(x), where g(x) is the actual cost from the starting point to the current node x, representing the actual distance the agent has flown to expand to the current node, and h(x) is the heuristic function, representing the estimated detection arc segment from the current node x to the target position. f(x) is the total cost of the nodes in the search space.

[0026] When the starting point, end point, target position, and threat area of the given agent are provided, the sparse A* algorithm obtains a trajectory that meets the threat avoidance requirements and task requirements through step-by-step search and solution.

[0027] Furthermore, the track constraint conditions include the minimum track length, maximum turning angle, longest track constraint, enemy threat area constraint, and flight task constraint that needs to reach the target position and reduce the detection arc segment.

[0028] Furthermore, after reaching the set target position, the following steps are also executed:

[0029] After the task ends, by counting the actions of each agent's interference, the number of times each target is interfered and suppressed and the threat area radius after suppression are obtained, and this result is used as the final effect of the cooperative interference model; count the arc segment lengths of the track detected by the enemy threat area before and after each agent executes the avoidance action, and use the average reduction of the detection arc segment length as the final effect of the cooperative avoidance model.

[0030] In the second aspect, a communication degradation adaptive integrated system for cooperative interference and trajectory planning is proposed, including an agent group configuration module, a cooperative interference module with communication, a cooperative interference module with zero communication, a cooperative avoidance module, and an effect evaluation module;

[0031] The intelligent agent group configuration module is used to form a cluster of multiple intelligent agents with mobility and equipped with jammers and passive radars to perform tasks, set the task area, and each intelligent agent has a set target position. The set task process is that the intelligent agent cluster starts from the starting position and reaches its respective target positions in the task area;

[0032] The collaborative jamming module with communication is used to construct a collaborative jamming model based on the multi-agent collaborative strategy trained by the intelligent agent replacement framework when the communication rate is not less than 0.1 kbps. During the process of each intelligent agent reaching the task area, the collaborative jamming model controls the intelligent agent to continuously use the carried passive radar to detect the targets in the task area, share the detection information according to the communication conditions, receive the information shared by other intelligent agents and input it into the collaborative jamming model. The collaborative jamming model outputs the target sequence for interference suppression of each intelligent agent, and calculates the actual threat area radius after each target is suppressed accordingly;

[0033] The collaborative jamming module without communication is used to construct a collaborative jamming model based on the bipartite graph method when the communication rate is lower than 0.1 kbps. Define the intelligent agent and the target as two different types of nodes to establish a connection relationship, input the current number of intelligent agents and the number of targets, and use the collaborative jamming model to obtain the globally optimal interference resource allocation scheme. Each intelligent agent predicts the interference behavior of other intelligent agents through this scheme to achieve collaborative jamming in the zero-communication state;

[0034] The collaborative avoidance module constructs a collaborative avoidance model by using the trajectory planning method based on the sparse A* algorithm. When the output of the collaborative jamming model reaches a steady state, input the current positions, expected arrival positions of each intelligent agent, the positions of all targets, and the actual threat area radius after suppression into the collaborative avoidance model, generate the ground trajectory through the key parameters of the lateral trajectory and generate the avoidance waypoints according to the ground trajectory;

[0035] The effect evaluation module is used to count the interference actions of each intelligent agent after the task is completed, obtain the number of times each target is interfered and suppressed and the threat area radius after suppression, and use this result as the final effect of the collaborative jamming model; count the arc lengths of the tracks of each intelligent agent detected by the enemy threat area before and after performing the avoidance actions, and use the mean reduction of the detection arc lengths as the final effect of the collaborative avoidance model.

[0036] In a third aspect, a computer-readable storage medium is proposed. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the integrated method for communication degradation adaptive collaborative jamming and trajectory planning are implemented.

[0037] Fourthly, an electronic device is proposed, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the integrated method for communication degradation adaptive cooperative jamming and trajectory planning are implemented.

[0038] Fifthly, a computer program product is proposed, including a computer program which, when executed by a processor, implements the steps of the integrated method for communication degradation adaptive cooperative jamming and trajectory planning.

[0039] The beneficial effects of the present invention compared with the prior art are as follows:

[0040] (1) The present invention uses the API (Agent Permutation Invariant) algorithm to build a cooperative jamming model, encodes and processes the characteristics of damaged aircraft through permutation invariance and permutation equivariance, realizes the adaptability to the quantity change problem caused by real-time battle damage in the real combat environment, and at the same time solves the problem of cooperative jamming resource allocation under normal, restricted, and extremely low communication conditions except zero communication through compression encoding and decoding processing of shared communication information;

[0041] (2) The present invention uses a bipartite graph model to model aircraft and ships as two nodes with different attributes in the graph, constructs a complete bipartite graph by inputting the current number of aircraft and the number of target ships, and each aircraft can predict the interference actions of other agents without communication sharing, solving the problem of cooperative jamming resource allocation in the zero communication state;

[0042] (3) The present invention uses a trajectory planning method based on the sparse A* method to build a cooperative avoidance model, plans the ground trajectory from the starting point to the target point that meets the relevant constraint conditions, minimizes the detected arc segment and detection probability of the aircraft cluster, and at the same time, by optimizing the algorithm input to be independent of communication sharing, can adapt to the zero communication condition and the quantity change problem caused by battle damage;

[0043] (4) The present invention takes the jamming model as the main control module of the cooperative jamming and trajectory planning task in the whole cooperative jamming and trajectory planning task. When the jamming stage reaches a certain steady state, the jamming model transmits the final jamming result to the avoidance model and calls it, and the avoidance model gives the avoidance strategy to realize the integration of cooperative jamming and trajectory planning. Description of the Drawings

[0044] Figure 1 It is a flowchart of an integrated method for communication degradation adaptive cooperative jamming and trajectory planning proposed by the present invention;

[0045] Figure 2This is the architecture diagram of the API algorithm model adopted by the interference model in the present invention. Detailed implementation manners

[0046] An integrated method for collaborative interference and trajectory planning with communication degradation adaptability proposed by the present invention, as Figure 1 shown, includes the following steps:

[0047] Form a cluster of multiple mobile agents equipped with jammers and passive radars to perform tasks, set the task area, each agent has a set target position, and set the task process as the agent cluster starts from the starting position and reaches its respective target positions in the task area;

[0048] When the communication rate is not lower than 0.1 kbps, construct a collaborative interference model based on the multi-agent collaborative strategy trained by the agent replacement framework. During the process of each agent reaching the task area, the collaborative interference model controls the agent to continuously use the carried passive radar to detect the targets in the task area, share the detection information according to the communication conditions, receive the information shared by other agents and input it into the collaborative interference model. The collaborative interference model outputs the target sequence for interference suppression of each agent, and calculates the actual threat area radius after each target is suppressed accordingly;

[0049] When the communication rate is lower than 0.1 kbps, construct a collaborative interference model based on the bipartite graph method, define the agents and targets as two types of nodes with different attributes to establish connection relationships, input the current number of agents and the number of targets, and use the collaborative interference model to obtain the globally optimal interference resource allocation scheme. Each agent predicts the interference behaviors of other agents through this scheme to achieve collaborative interference in the zero-communication state;

[0050] Adopt a trajectory planning method based on the sparse A* algorithm to construct a collaborative avoidance model. When all jammers are turned on to the maximum power, the output of the collaborative interference model reaches a steady state. At this time, input the current positions, expected arrival positions of each agent, the positions of all targets, and the actual threat area radius after suppression into the collaborative avoidance model, generate a ground trajectory through the key parameters of the lateral trajectory and generate avoidance waypoints according to the ground trajectory. Each agent performs an avoidance action and reaches the set target position;

[0051] After the task is completed, by counting the interference actions of each agent, obtain the number of times each target is interfered and suppressed and the threat area radius after suppression, and use this result as the final effect of the collaborative interference model; count the arc lengths of the trajectories of each agent detected by the enemy threat area before and after performing the avoidance action, and use the average reduction of the detected arc lengths as the final effect of the collaborative avoidance model.

[0052] Embodiment 1

[0053] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0054] As Figure 1 shown, the four steps included in the method of the embodiment will be described in detail:

[0055] Step 1: Configure the agent group and the environment.

[0056] In this embodiment, 48 aircraft agents are used as vehicles, each carrying a set of intelligent control payloads and passive radars. The aircrafts are in a formation of 1 master and 2 slaves. The master aircraft of each formation is equipped with a jammer, and each jammer can interfere with up to three targets simultaneously. The mission area is a square area of 200 kilometers by 200 kilometers, and there are 12 unknown targets in the area, including three different types of targets with different radar radius detection ranges.

[0057] Set the mission process. The agents take off from a place 1200 km away from the target area and fly towards the preset target position. When the agents enter the radar detection range of the enemy target, the jammer is turned on to interfere and suppress the enemy target. When all jammers are turned on to the maximum power, the interference reaches a steady state. The interference model calls the avoidance model, and the agents execute avoidance actions and fly towards the target position along the track passing through the avoidance point, and finally reach the target position.

[0058] Step 2: Establish a cooperative interference model under different communication conditions and perform target interference suppression.

[0059] (1). When the communication rate is not less than 0.1 kbps, a cooperative interference model is constructed based on the multi-agent cooperative strategy trained by the agent permutation framework. During the process of each agent reaching the mission area, the cooperative interference model controls the agent to continuously use the passive radar carried to detect the targets in the mission area, share the detection information according to the communication conditions, receive the information shared by other agents and transmit it into the cooperative interference model, and make a decision to obtain the interference resource allocation result.

[0060] In this embodiment, the communication rate is divided into four categories, namely normal (above 10 kbps), restricted (1 kbps - 10 kbps), extremely low (0.1 kbps - 1 kbps), and zero communication (below 0.1 kbps); for the first three categories of communication rate situations, as Figure 2 shown, the present invention adopts a multi-agent cooperative strategy based on the agent permutation framework (Agent Permutation Invariant, API) to train and establish a cooperative interference model, which can perform target interference suppression based on the detected target position information and the communication information shared by other agents.

[0061] Among them, the method for constructing a cooperative jamming model based on a multi-agent cooperative strategy trained by an agent replacement framework is specifically as follows:

[0062] ① Obtain the local observation data of each agent, including its own position and the target positions it sees. Decode the communication information transmitted by other agents to obtain the positions of teammates and the target positions they see. Use the above information as the input information for each agent;

[0063] ② After passing through a value network for the input information of each agent, obtain the local Q-value of each agent, and use a mixing network to merge the single-agent local value functions to obtain the joint Q-value;

[0064] ③ Set the weight w of the network parameters to be non-negative and the bias to be unrestricted to achieve optimization monotonicity. Taking the argmax of the joint Q-value is equivalent to taking the argmax of each local Q-value, so as to ensure that when optimizing the joint Q-value, each local Q-value can also be maximized, realizing the tacit cooperation between multi-agents;

[0065] ④ Adopt the method of centralized training and distributed execution to train the entire network architecture. Each agent outputs the target sequence of its own interference suppression, and compresses and encodes its own position and the observed target position as communication information and transmits it to other agents, so as to meet the situation of communication rate degradation and use it as the decision-making input information for the next step.

[0066] In summary, the present invention uses the API architecture to establish a cooperative jamming model, which can meet the requirement of transmitting communication information when the communication rate is not less than 0.1 kbps, and realizes the tacit cooperation between multi-agents. In addition, due to the problem of the change in the number of agents caused by agent battle damage in the real combat environment, during the training process, we introduce permutation invariance and permutation equivariance. Permutation invariance means that for the same input set, arbitrarily disturbing the input order does not change the output value of the function. Permutation equivariance means that rearranging the order of the input set will cause the corresponding function output to be rearranged in the same order. Therefore, when the number of agents changes, the feature vectors obtained by the feature encoding module for the features of the damaged aircraft are 0, and the features of the damaged aircraft will not affect the cooperative jamming model from generating the strategic actions of other agents, and thus a cooperative jamming model adapted to the change in the number of agents is trained.

[0067] (2) When the communication rate is lower than 0.1 kbps, a collaborative interference model is constructed based on the bipartite graph method. The agents and targets are defined as nodes with two different attributes to establish connection relationships. The current number of agents and targets is input, and the globally optimal interference resource allocation scheme is obtained using the collaborative interference model. Each agent predicts the interference behaviors of other agents through this scheme, realizing collaborative interference in a zero-communication state.

[0068] Among them, the method of constructing a collaborative interference model based on the bipartite graph method to achieve collaborative interference in a zero-communication state is as follows:

[0069] ① Use the bipartite graph model to model the entire interference stage as a bipartite graph. The agents and targets are regarded as nodes with two different attributes in the bipartite graph. There are edges only between nodes with different attributes. The practical meaning of connecting two nodes a and b is that agent a suppresses target b by interference.

[0070] ② Obtain the current number of agents and the number of targets that have been observed, and construct a complete bipartite graph based on this.

[0071] ③ Each agent inputs the current number of agents and the number of targets that have been observed into the collaborative interference model based on the bipartite graph, and a globally optimal collaborative interference strategy can be obtained, including the predicted interference target sequence of other agents. It can realize collaborative interference in a zero-communication state without sharing communication information, and the actual threat area radius after each target is suppressed can be calculated based on this result, which is used as the input of the collaborative avoidance model.

[0072] Step 3: Adopt a trajectory planning method based on the sparse A* algorithm to construct a collaborative avoidance model. When all aircraft observe all targets, at this time, the jammer is turned on to the maximum power, and the output of the collaborative interference model reaches a steady state. At this time, input the current positions, expected arrival positions of each agent, the positions of all targets, and the actual threat area radius after suppression into the collaborative avoidance model. Generate a ground trajectory through the key parameters of the lateral trajectory and generate avoidance waypoints based on the ground trajectory. Each agent executes an avoidance action and reaches the set target position.

[0073] Among them, based on the sparse A* algorithm, the positions of enemy ships and the distribution of threat areas have been obtained through passive radar information and the collaborative interference model. The task of trajectory planning can be described as: Considering the key parameters of the lateral trajectory including the minimum track length, maximum turning angle, and longest track constraint, and combining threat constraints and flight mission constraints, plan a trajectory that satisfies the relevant constraint conditions from the current position to the target position. Since the traditional A* algorithm usually represents the planning environment in the form of a grid and uses the node expansion method for expansion, we map the track to a two-dimensional ground trajectory for analysis during the algorithm search process. The specific method is as follows:

[0074] (1) Incorporate the trajectory constraint conditions into the sparse A* algorithm. Let the minimum step length of the aircraft be L and the maximum turning angle be θ, and introduce the velocity direction of the aircraft. Then, the search range for the next position from the current position is restricted within a sector area with an angle of 2θ. The trajectory constraint conditions include the minimum trajectory length, the maximum turning angle, the longest trajectory constraint, the enemy threat area constraint, and the flight mission constraint that needs to reach the target position and reduce the detection arc segment.

[0075] (2) The maximum trajectory length L max represents the upper limit of the remaining energy or time for a specific task. During the search process, trajectories with a length greater than L max are considered invalid trajectories and will not be considered. For a given node x, only when D(x)+SL(x)≤L max is satisfied, will x be added to the possible path nodes, where D(x) is the actual distance traveled from the starting position to x, and SL(x) is the straight-line distance from x to the target.

[0076] (3) Define the trajectory cost function of the sparse A* algorithm as: f(x) = g(x)+h(x), where x is the current node, g(x) is the actual cost from the starting point to node x, representing the actual distance the aircraft has flown to reach the current node, h(x) is the heuristic function, representing the estimated detection arc segment from the current node x to the target position, and f(x) is the total cost of the nodes in the search space, which is also the objective to be optimized in this solution.

[0077] (4) After the starting point, end point, target position, and threat area of the aircraft are given, the sparse A* algorithm obtains a trajectory that meets the threat avoidance requirements and task requirements through step-by-step search and solution.

[0078] In summary, the sparse A* algorithm combines the constraint conditions into the search algorithm, effectively reducing the search space, shortening the search time, and minimizing the detected arc segment and detection probability of the aircraft cluster. During the entire collaborative avoidance process, the agent can obtain the optimal avoidance waypoints by inputting its current position, the expected arrival position, the target position, and the radius of the suppressed threat area, without the need to transmit communication information, and can adapt to the zero communication condition and the quantity change problem caused by battle damage.

[0079] Step 4. After the task ends, by counting the interference actions of each agent, obtain the number of times each target is interfered and suppressed and the radius of the suppressed threat area, and use this result as the final effect of the collaborative interference model; count the arc segment lengths of the trajectories detected by the enemy threat area before and after each agent performs the avoidance action, and use the average reduction of the detection arc segment lengths as the final effect of the collaborative avoidance model.

[0080] In summary, for the integrated method of cooperative jamming and trajectory planning provided by the present invention, during the entire cooperative jamming and trajectory planning task, the cooperative jamming model is used as the main control module for the cooperative jamming and trajectory planning task. When a certain steady state is reached in the jamming phase, for example, when all targets are seen by the passive radars mounted on each agent, the jamming strategy is no longer updated. At this time, the cooperative jamming model calls the cooperative avoidance model, and the cooperative avoidance model gives the avoidance strategy to achieve the integration of cooperative jamming and trajectory planning.

[0081] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes, and decorations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention all belong to the protection scope of the technical solution of the present invention.

[0082] The content not detailedly described in the specification of the present invention belongs to the well-known technology of those skilled in the art.

Claims

1. An integrated method for cooperative jamming and trajectory planning with communication degradation adaptability, characterized in that The steps include: Form a cluster consisting of multiple mobile agents equipped with jammers and passive radars to perform tasks. Set the task area, and each agent has a preset target position. Set the task process as the agent cluster starts from the starting position and reaches their respective target positions in the task area; Under the condition that the communication rate is not lower than 0.1 kbps, build a cooperative jamming model based on the multi-agent cooperation strategy trained by the agent replacement framework. During the process of each agent reaching the task area, the cooperative jamming model controls the agent to continuously use the carried passive radar to detect the targets in the task area, share the detection information according to the communication conditions, receive the information shared by other agents and input it into the cooperative jamming model. The cooperative jamming model outputs the target sequence of interference suppression for each agent, and calculates the actual threat area radius after each target is suppressed accordingly; Under the condition that the communication rate is lower than 0.1 kbps, build a cooperative jamming model based on the bipartite graph method. Define the agents and targets as two types of nodes with different attributes to establish a connection relationship. Input the current number of agents and the number of targets, and use the cooperative jamming model to obtain the globally optimal interference resource allocation scheme. Each agent predicts the interference behavior of other agents through this scheme to achieve cooperative jamming in the zero-communication state; Build a cooperative avoidance model using the trajectory planning method based on the sparse A* algorithm. When all jammers are turned on to the maximum power, the output of the cooperative jamming model reaches a steady state. At this time, input the current positions, expected arrival positions of each agent, the positions of all targets, and the actual threat area radius after suppression into the cooperative avoidance model. Generate the ground trajectory through the key parameters of the lateral trajectory and generate the avoidance waypoints according to the ground trajectory. Each agent executes the avoidance action and reaches the set target position.

2. The integrated method for cooperative jamming and trajectory planning with communication degradation adaptability according to claim 1, wherein Build a cooperative jamming model based on the multi-agent cooperation strategy trained by the agent replacement framework, including: Obtain the local observation data of each agent, including its own position and the target positions it sees. Decode the communication information transmitted by other agents to obtain the positions of teammates and the target positions they see. Use the above information as the input information of each agent; After the input information of each agent passes through a value network, obtain the local Q value of each agent, and use a hybrid network to merge the single-agent local value function to obtain the joint Q value; Set the weight w of the network parameters to be non-negative and the bias bias is not restricted to achieve optimization monotonicity. Take the argmax of the joint Q value, which is equivalent to taking the argmax of each local Q value, to ensure that when optimizing the joint Q value, each local Q value is also maximized; Train the entire network architecture in a centralized training and distributed execution manner. Each agent outputs the target sequence of its own interference suppression, and compresses and encodes its own position and the observed target positions as communication information and transmits it to other agents to meet the situation of communication rate degradation and serve as the decision input information for the next step.

3. An integrated method for cooperative jamming and trajectory planning with communication degradation adaptability according to claim 1, characterized in that, Build a cooperative jamming model based on the bipartite graph method, including: The entire interference phase is modeled as a bipartite graph using a bipartite graph model. Agents and targets are regarded as nodes with two different attributes in the bipartite graph, and there are edges only between nodes with different attributes. The practical meaning of connecting two nodes a and b is that agent a suppresses target b by interference. Obtain the current number of agents and the number of targets that have been observed, and construct a complete bipartite graph. Each agent inputs the current number of agents and the number of targets that have been observed into the collaborative interference model based on the bipartite graph, and obtains a globally optimal collaborative interference strategy, including the predicted interference target sequences of other agents. According to the prediction results, calculate the actual threat area radius after each target is suppressed, which is used as the input of the collaborative avoidance model.

4. An integrated method for cooperative interference and trajectory planning with communication degradation adaptability according to claim 1, characterized in that Adopt a trajectory planning method based on the sparse A* algorithm to construct a collaborative avoidance model, including: During the search process of the sparse A* algorithm, map the flight path to a two-dimensional ground trajectory and perform the following operations: Incorporate the flight path constraint conditions into the sparse A* algorithm. Assume that the minimum step length of the agent is L, the maximum turning angle is θ, and the velocity direction of the agent is introduced. Then the search interval for the next position from the current position is restricted within a fan-shaped area with an angle of 2θ. Maximum track length L max Represents the upper limit of the remaining energy or time for a specific task. During the search process, tracks with a length greater than L max are considered invalid tracks. For a given node x, x is added to the possible path nodes only when D(x) + SL(x) ≤ L max is satisfied, where D(x) is the actual distance traveled from the starting position to x, and SL(x) is the straight-line distance from x to the target; Define the flight path cost function of the sparse A* algorithm as: f(x) = g(x) + h(x), where g(x) is the actual cost from the starting point to the current node x, representing the actual distance the agent has flown to reach the current node, h(x) is the heuristic function, representing the estimated detection arc segment from the current node x to the target position, and f(x) is the total cost of the node in the search space. When the starting point, ending point, target position, and threat area of a given agent are provided, the sparse A* algorithm obtains a trajectory that meets the threat avoidance requirements and task requirements through step-by-step search.

5. An integrated method for cooperative jamming and trajectory planning with communication degradation adaptability according to claim 4, characterized in that The flight path constraint conditions include the minimum flight path length, the maximum turning angle, the longest flight path constraint, the enemy threat area constraint, and the flight task constraint of needing to reach the target position and reduce the detection arc segment.

6. The integrated method for cooperative jamming and trajectory planning with communication degradation adaption according to claim 1, wherein After reaching the set target position, the following steps are also executed: After the task ends, by counting the interference actions of each agent, obtain the number of times each target is interfered and suppressed and the threat area radius after suppression, and use this result as the final effect of the collaborative interference model; count the arc segment lengths of the flight paths detected by the enemy threat area before and after each agent executes the avoidance action, and use the average reduction of the detection arc segment lengths as the final effect of the collaborative avoidance model.

7. An integrated system for cooperative jamming and trajectory planning with communication degradation adaptability, characterized in that, It includes an agent group configuration module, a collaborative interference module with communication, a collaborative interference module without communication, a collaborative avoidance module, and an effect evaluation module; The agent group configuration module is used to form a cluster of multiple agents with mobility and equipped with jammers and passive radars to execute tasks, set the task area, each agent has a set target position, and set the task process as the agent cluster starting from the starting position and reaching their respective target positions in the task area. The collaborative jamming module with communication is used to construct a collaborative jamming model based on the multi-agent collaborative strategy trained by the agent replacement framework when the communication rate is not lower than 0.1 kbps. During the process of each agent reaching the task area, the collaborative jamming model controls the agent to continuously use the passive radar carried to detect the targets in the task area, share the detection information according to the communication conditions, receive the information shared by other agents and input it into the collaborative jamming model. The collaborative jamming model outputs the target sequence for each agent's interference suppression, and calculates the actual threat area radius after each target is suppressed accordingly. The collaborative jamming module without communication is used to construct a collaborative jamming model based on the bipartite graph method when the communication rate is lower than 0.1 kbps. Define the agents and targets as two types of nodes with different attributes to establish a connection relationship, input the current number of agents and the number of targets, and use the collaborative jamming model to obtain the globally optimal interference resource allocation scheme. Each agent predicts the interference behavior of other agents through this scheme to achieve collaborative jamming in the zero-communication state. The collaborative avoidance module constructs a collaborative avoidance model by using the trajectory planning method based on the sparse A* algorithm. When the output of the collaborative jamming model reaches a steady state, input the current positions, expected arrival positions of each agent, the positions of all targets, and the actual threat area radius after suppression into the collaborative avoidance model, generate the ground trajectory through the key parameters of the lateral trajectory and generate the avoidance waypoints according to the ground trajectory. The effect evaluation module is used to count the interference actions of each agent after the task is completed, obtain the number of times each target is interfered and suppressed and the threat area radius after suppression, and use this result as the final effect of the collaborative jamming model; count the arc length of the track detected by the enemy threat area before and after each agent executes the avoidance action, and use the average reduction of the detected arc length as the final effect of the collaborative avoidance model.

8. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images