A dynamic generation method for a heterogeneous unmanned cluster

Abstract

The present application belongs to the technical field of unmanned cluster equipment combination method, and particularly relates to a dynamic generation method for a heterogeneous unmanned cluster. The method fuses D-DQN and MOEA to solve MOP in the construction of a kill net, and has certain advantages in efficiency and performance. For the dynamic generation problem caused by the possible destruction of red and blue nodes in a typical cluster confrontation scene, two solutions, namely a "scheme matrix traversal method" and a "hypothetical target method", are proposed. Through simulation, on the basis of obtaining good visual effects, the comparison experiment shows that the evaluation index weight will correspondingly affect the optimization result, and the construction efficiency of the heterogeneous unmanned cluster kill net is improved.

Description

Technical Field

[0001] This invention belongs to the technical field of unmanned swarm equipment combination methods, and specifically relates to a method for dynamically generating kill nets for heterogeneous unmanned swarms. Background Technology

[0002] The kill chain consists of six links: Find, Fix, Track, Target, Engage, and Assess. Each link is interconnected, forming a structured combat chain, known as F2T2EA.

[0003] The kill chain operates by having a sensor network of reconnaissance nodes transmit enemy information to the command and control network, which then destroys the enemy target through strike nodes. Because the objective of a single operation is often to destroy a key enemy node, the decision-centric kill chain has become central to future warfare, serving as the core of integrated combat capabilities and precision strikes.

[0004] In recent years, with the improvement of computing power, the types of kill chains have become increasingly diverse. In addition to the earliest independent kill chains, there have also emerged "node-radial" kill chains that use high-performance friendly nodes as the chain hubs, and "mesh" kill chains (i.e., kill nets) that merge multiple independent kill chains with shared nodes.

[0005] Currently, kill net construction is mainly based on rules or computation. Algorithms that use genetic algorithms to solve the kill net equipment combination problem often suffer from slow construction speed and low construction quality, and there is a lack of research on dynamic generation methods and the combat platform of "heterogeneous unmanned swarm". Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a method for dynamically generating kill nets for heterogeneous unmanned swarms.

[0007] The specific technical solution of the present invention is as follows:

[0008] A method for dynamically generating kill nets for heterogeneous unmanned swarms includes the following steps:

[0009] Step S1: Select kill net evaluation indicators to reflect the relationship between the equipment capabilities of the red and blue teams;

[0010] The kill network evaluation metrics include: redundancy metrics, risk metrics, sensitivity metrics, and network entropy metrics.

[0011] Step S2: MOEA selected based on D-DQN adaptive operator;

[0012] Using the NSGA-II (second-generation non-dominated sorting genetic algorithm) process as a framework, in the operator candidate set, the D-DQN network adaptively selects the appropriate genetic operator based on the parent information when the genetic algorithm needs to generate offspring, solving the multi-objective optimization problem of constructing a heterogeneous unmanned swarm kill network and obtaining multiple optimal solution schemes.

[0013] Step S3: Select the optimal kill net construction scheme based on the weight aggregation value;

[0014] The kill net evaluation index values ​​of multiple optimal solutions are weighted and averaged to obtain a weighted aggregate value. This aggregate value is then used to rank the multiple optimal solutions and select the optimal solution from them.

[0015] Step S4: Construct the communication network topology;

[0016] First, the depth-first search (DFS) algorithm is used to decompose the multi-connected graph into a series of independent connected graphs. Then, the Prim method is used to construct the network topology for each connected graph.

[0017] Step S5: Dynamically update the kill net;

[0018] Whether to dynamically update the kill net depends on whether the battlefield situation changes.

[0019] Step S6: Determine whether the changes in the battlefield situation are within the responsive range, that is, whether there is a feasible solution among the multiple optimal solutions. If so, return to step S4; otherwise, return to step S1.

[0020] Preferably, in step S1, the redundancy index is the average number of kill chains that all blue team nodes can form; the risk index is the proportion of kill chains lost after each red team node is destroyed to the original number; the sensitivity index is the number of other layer nodes that the command node in the kill network calls; and the network entropy index is the network entropy of the communication network.

[0021] Preferably, in step S2, the red team node and the kill network evaluation index are taken as states, the selection of genetic operators in the evolution process is taken as actions, the Q value learned by the neural network is taken as the basis for the selection of genetic operators, and a credit allocation strategy, a candidate set of operators and an adaptive operator selection strategy are defined.

[0022] Preferably, step S5 specifically includes:

[0023] The kill net is dynamically updated based on changes in the red and blue team nodes:

[0024] (1) When the number of red team nodes increases or the number of blue team nodes decreases, the original kill network remains unchanged;

[0025] (2) When the number of red nodes decreases, the multiple optimal solutions obtained in step S2 are traversed from the best to the worst performance until all the nodes contained therein are available.

[0026] (3) When a blue party node is added, if the added node belongs to the same category as an existing blue party node, a relay sequential attack strategy is adopted, that is, after destroying the previous node, the added blue party node of the same category is destroyed immediately; if the added blue party node does not belong to the same category as any existing blue party node, that is, it is a new target category, then the standby redundant kill chain is assigned to take effect and complete the attack on the added blue party node.

[0027] Preferably, in step S2,

[0028] The credit allocation strategy is to determine the reward strategy in reinforcement learning. The Chebyshev method is used as the aggregation function, where the objective function value is the maximum weighted distance between each individual's target and its target reference value, as shown in equation (8).

[0029]

[0030] In the formula, x represents the decision variable, w is the weight assigned to the objective function, and z * The reference point is M, the number of objective functions is w. c The weights assigned to the c-th objective function, f c (x) represents the function value of the c-th objective function;

[0031] The reward value for offspring is shown in equation (9):

[0032]

[0033] In the formula, y represents the decision variable of the parent generation that generates offspring x.

[0034] Preferably, in step S2,

[0035] The genetic operators selected from the candidate set of operators are the simulated binary crossover operator, crossover operator, first difference evolution operator, or second evolutionary difference operator, and polynomial mutation is performed on the generated offspring solutions;

[0036] a) Simulate the binary crossover operator, as shown in equation (10):

[0037]

[0038] Where d = 1, ..., D, x = (x1, ..., x2) d ,...,x D (This is a child's solution.) There are two parent generations, and:

[0039]

[0040] Where μ is a uniformly distributed random value sampled in [0,1], and η is a predefined parameter.

[0041] b) Crossover operator, as shown in equation (12):

[0042]

[0043] Where r1 and r2 are uniformly distributed random values ​​sampled in [0,1], gen is the current generation number, and maxgen is the maximum generation number.

[0044] c) The first difference evolution operator, where DE / rand / 1 is defined as in equation (13):

[0045]

[0046] It is the i-th parent, F and CR are preset constants, and r is a uniformly distributed random value sampled in [0,1].

[0047] d) The second difference evolution operator, DE / rand / 2, is defined as shown in equation (14):

[0048]

[0049] Where F and CR are predefined parameters, and r is a uniformly distributed random value sampled in [0,1].

[0050] Once a child solution is generated, a polynomial mutation is performed, as shown in equations (15) and (16):

[0051] x d =x d +(u d -l d )δ (15)

[0052]

[0053] Among them, u d and l d These are the upper and lower bounds of the decision variable, which can take values ​​of 1 and 0 respectively. μ is a uniformly distributed random value sampled in [0,1], and η is a predefined parameter.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] 1. This invention integrates D-DQN with MOEA to solve the MOP (Mean Opportunity Orientation) problem in kill net construction. This method has certain advantages in terms of efficiency and performance. MOEA, based on D-DQN adaptive operator selection, can, after training, select the most suitable genetic operator for the user-specified kill net evaluation metric weights and the characteristics of each parent solution. This generates offspring with greater average weight aggregation and fitness improvement, thus achieving a better and more user-suitable heterogeneous unmanned swarm kill net construction scheme with the same time cost.

[0056] 2. This invention proposes a method for dynamically updating the kill net, namely the "scheme matrix traversal method" and the "assumed target method". Using the dynamic generation method proposed in this invention, new kill net construction schemes can be generated in real time based on changes in the battlefield situation.

[0057] 3. To address the issue that the communication network of the obtained optimal solution is not necessarily a complete connected graph, this invention further proposes a method for constructing a minimum spanning tree from a multi-connected graph using Prim, thus solving the problem of communication network topology construction in kill networks. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly introduced below. The features and advantages of the present invention can be more clearly understood by referring to the accompanying drawings. The accompanying drawings are schematic and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of a method for dynamically generating kill nets for heterogeneous unmanned swarms proposed in this invention.

[0060] Figure 2 This is a flowchart of MOEA based on the selection of D-DQN adaptive operators.

[0061] Figure 3 This is a schematic diagram of constructing a minimum spanning tree from a multi-connected graph.

[0062] Figure 4 This is a schematic diagram of the "assumed target method".

[0063] Figure 5 This is a complete diagram of the kill chain.

[0064] Figure 6 It is the reward curve of D-DQN.

[0065] Figure 7 It is the spatial distribution of the multi-objective Pareto solution.

[0066] Figure 8 It is the Pareto solution set matrix for a multi-objective system.

[0067] Figure 9 It is the connected graph of the multi-layer network model of the optimal solution.

[0068] Figure 10 It is the result of minimum spanning tree programming.

[0069] Figure 11 These are simulation results from the "assumed target method".

[0070] Figure 12 It is the UI for displaying the kill chain.

[0071] Figure 13 It's a drop-down menu in the UI.

[0072] Figure 14 It is one of the damage chains displayed in the UI. Detailed Implementation

[0073] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0074] This invention proposes a method for dynamically generating kill nets for heterogeneous unmanned swarms, such as... Figure 1 As shown, it includes the following steps:

[0075] S1: Selection of evaluation indicators for kill net;

[0076] Kill net evaluation metrics are the standards for measuring the quality of a kill net. The selected kill net evaluation metrics should accurately reflect the performance, risk, and other characteristics of the kill net, and be suitable for the application of heterogeneous unmanned swarms. Good evaluation metric selection can lead to a more powerful and robust kill net. Considering the characteristics of heterogeneous unmanned swarms and the need for metrics to reflect the strength of the kill net from various aspects, this invention selects the following four evaluation metrics, which also serve as the objective function for subsequent multi-objective optimization.

[0077] (1) Redundancy index;

[0078] Redundancy primarily refers to the number of different kill methods a kill net can employ against a target. It calculates the average number F(K) of kill chains that the kill net can form against all blue team nodes. A higher average value indicates stronger redundancy, and vice versa. The specific formula is as follows:

[0079]

[0080]

[0081] P kc =diag[(P1)] m×n (P2) n×n (P3) n×n (P4) n×m (3)

[0082] Where K is the path number matrix of each capability layer of the kill network, including P1, P2, P3, and P4, which represent the path number matrices of the reconnaissance, communication, command and control, and strike layers, respectively, as defined in equation (2). kc The kill chain number matrix is ​​calculated using equation (3), Tr(P) kc ) is P kc The trace, m is the number of blue-side nodes, and the sequence number a represents a different network. represents the number of simple paths between node i and node j in network a. A path is called a simple path if all vertices are unique. n represents the number of red-side nodes, and diag represents all elements on the diagonal of the matrix.

[0083] (2) Risk indicators;

[0084] The risk of a kill network primarily manifests as the impact on the entire kill network after an attack on a red-side node. This risk is influenced by the connectivity of nodes within the kill network; the more intermediary nodes (i.e., critical bridges) there are, the higher the risk. According to traditional network node risk metrics, the risk can be represented by the number of kill chains lost after a red-side node is destroyed. The more kill chains lost, the higher the risk of the kill network, and vice versa.

[0085] This invention uses the ratio R(K) of the number of kill chains lost by each target relative to the original number as a more comprehensive indicator of the risk of the kill network. The calculation formula is as follows:

[0086]

[0087] in, Let be the kill chain matrix after the red team node i is destroyed.

[0088] (3) Sensitivity indicators;

[0089] Under the Blue Team's countermeasure strategy, closed-loop speed is a crucial indicator for evaluating the effectiveness of the kill network. It can be reflected by the number of other layer nodes S(K) that the command node can invoke, i.e., sensitivity. The more other nodes the command node can invoke, the higher its sensitivity; conversely, the fewer the nodes, the lower the sensitivity. The calculation formula is as follows:

[0090]

[0091] in, Defined as in equation (6), n C2 This represents the number of nodes (nodes with accusation capabilities) on the Red side.

[0092]

[0093] (4) Network entropy index;

[0094] Entropy was originally a concept introduced from thermodynamics to describe the phenomenon that in an isolated system, without energy exchange, the system spontaneously tends towards a disordered state. The concept of network entropy is analogous to measuring the "randomness" of a network; the larger the value, the higher the randomness of the network graph.

[0095]

[0096] Where I(K) represents the network entropy, h e It represents the proportion of nodes with degree e, and max_d is the maximum degree in the network.

[0097] In communication networks, lower network entropy is better. The primary purpose of a communication network is to transmit information, and lower network entropy indicates that information is better organized and compressed, thus allowing for more efficient transmission and storage.

[0098] S2: MOEA based on D-DQN adaptive operator selection;

[0099] By treating the red-side nodes and kill net evaluation metrics as states, and the operator selection during the evolutionary process as actions, D-DQN and MOEA (Multi-Objective Evolutionary Algorithms) can be organically integrated. Throughout the population evolution process, the Q-value learned by the neural network is used as the basis for selecting genetic operators, efficiently solving the MOP (Multi-Objective Optimization Problem) encountered in constructing a heterogeneous unmanned swarm kill net. After determining the states and actions of DQN, several issues still need to be defined and resolved, including credit allocation strategy, operator candidate set, and adaptive operator selection strategy.

[0100] (1) Credit allocation strategy;

[0101] The credit allocation strategy is essentially determining the reward policy in Reinforcement Learning (RL). Since killnet construction is a multi-objective learning platform (MOP), the key to successful credit allocation lies in aggregating multiple evaluation metrics (objective functions) into a single function. This invention employs the Chebyshev method as the aggregation function.

[0102] The Chebyshev method is a nonlinear aggregation method. Unlike the weighted summation of the weighted aggregation method, it requires calculating the minimum value achieved by each objective in the population up to the present as a reference point (as an ideal objective value, which may not exist in the real scenario) before calculating the objective function value. The objective function value is the maximum weighted distance between each objective of an individual and its objective reference value (the minimum value found by that objective up to the present), as shown in Equation (8).

[0103]

[0104] In the formula, x represents the decision variable, w is the weight assigned to the objective function, and z * The reference point is M, the number of objective functions is w. c The weights assigned to the c-th objective function, f c (x) represents the value of the c-th objective function. It's important to note that when using this aggregation function method, it's crucial to ensure that the objective functions are scaled uniformly; that is, they must be normalized first. This will ensure that the aggregation function values ​​obtained by the Chebyshev method are more reasonable.

[0105] With the aggregation method of the objective function, the reward value of the offspring is determined using equation (9).

[0106]

[0107] In the formula, y represents the decision variable of the parent generation that generates offspring x. It's easy to deduce that the reward value of each offspring should be between [0,1]. However, in the actual evolutionary process, the offspring reward value obtained using the operator op is not directly used as the reward value of op. Since few but large improvements are considered more important than frequent but small improvements, a sequence R should be defined during credit allocation to store recent operator choices and reward value tuples. Then, at each credit allocation, the maximum reward value for the corresponding op is found in R and used as the reward.

[0108] (2) Operator candidate set;

[0109] The operator refers to the genetic operator, that is, the method used in a genetic algorithm to generate offspring. This invention currently selects one of four different genetic operators, including the simulated binary crossover operator, the crossover operator, the first differential evolution operator, and the second differential evolution operator. Because for the kill net construction problem of heterogeneous unmanned swarms, each solution is described by a sequence of 0s or 1s (indicating whether to use each node), which is exactly a binary string, there is no need to consider encoding issues before performing the crossover operation.

[0110] a) Simulate the binary crossover operator;

[0111] Simulated binary crossover is the most commonly used continuous optimization crossover operator in genetic algorithms, and it is good at handling multimodal landscapes, as shown in equation (10):

[0112]

[0113] Where d = 1, ..., D, x = (x1, ..., x2) d ,...,x D (This is a child's solution.) There are two parent generations, and:

[0114]

[0115] Where μ is a uniformly distributed random value sampled in [0,1], and η is a predefined parameter.

[0116] b) Crossover operator;

[0117] The cross operator is good at handling nonlinear variable relationships, as shown in equation (12):

[0118]

[0119] Where r1 and r2 are uniformly distributed random values ​​sampled in [0,1], gen is the current generation number, and maxgen is the maximum generation number.

[0120] c) The first difference evolution operator;

[0121] Differential evolution operators are also effective for handling complex variable relationships, where DE / rand / 1 is defined as in equation (13):

[0122]

[0123] It is the i-th parent, F and CR are preset constants, and r is a uniformly distributed random value sampled in [0,1].

[0124] d) The second difference evolution operator;

[0125] DE / rand / 2 is defined as shown in equation (14):

[0126]

[0127] Where F and CR are predefined parameters, and r is a uniformly distributed random value sampled in [0,1].

[0128] Specifically, two parent generations should be selected for the two crossover operators, while DE / rand / 1 and DE / rand / 2 require three and five parent generations respectively to generate child solutions.

[0129] Polynomial variation;

[0130] Once a child solution is generated, a polynomial mutation is performed, which can further improve the performance of multi-objective optimization, as shown in equations (15) and (16):

[0131] x d =x d +(u d -l d )δ (15)

[0132]

[0133] Among them, u d and l d These are the upper and lower bounds of the decision variable, which can take values ​​of 1 and 0 respectively. μ is a uniformly distributed random value sampled in [0,1], and η is a predefined parameter.

[0134] (3) Adaptive operator selection strategy;

[0135] After the credit allocation (i.e., reward assignment) of the operators is completed, the next step is to use D-DQN to adaptively select genetic operators. By inputting the current state (i.e., the solution and its objective function value) into the already trained neural network, the Q-values ​​of each action (i.e., the operator) are obtained, and then the ε-greedy method is used to select the action.

[0136] The ε-greedy algorithm uses a probability to balance exploration and exploitation: the agent explores with probability ε, ignoring the Q-value of actions and selecting actions through random sampling; it exploits with probability 1-ε, referring to the action's Q-value and using the action with the highest Q-value. It's important to note that in the early stages of DQN training, a larger ε should be used to better explore the Q-values ​​of each action; in the later stages of training, a smaller ε should be chosen to obtain a higher training score (reward). Therefore, ε should be set as a decreasing function during training.

[0137] The overall processing flow is as follows:

[0138] After completing the above steps, the algorithm flow of NSGA-II (Non-dominated Sorting Genetic Algorithm m-II) can be used as a framework. In the operator candidate set, a D-DQN network adaptively selects appropriate genetic operators based on parental information when the genetic algorithm needs to generate offspring, thus solving the multi-objective optimization problem of constructing a heterogeneous unmanned swarm kill network. The training steps of MO EA based on D-DQN adaptive operator selection are as follows, and the flowchart is shown below. Figure 2As shown. The execution steps after training are roughly the same as the training process for one session. The learning and update process of D-DQN is omitted in the algorithm pseudocode because it is similar to the learning method of ordinary D-DQN.

[0139]

[0140]

[0141] S3: Communication network topology;

[0142] For a given connected graph, the Prim method is used to construct a minimum spanning tree to solve the problem of building a communication network after optimizing the kill network. Calculating the minimum spanning tree of the network can find the optimal solution with the minimum communication lines or total cost.

[0143] Since the communication network of the optimal node kill network is not necessarily a single connected graph, but rather multiple independent connected graphs, this invention first uses a depth-first search (DFS) algorithm to decompose the multi-connected graph into a series of independent connected graphs. Then, for each connected graph, the Prim method is used to construct the network topology, as shown in the schematic diagram below. Figure 3 .

[0144] This method eliminates loops in a connected graph while maintaining its connectivity, thus avoiding resource waste and redundant paths. This can improve system stability and reliability in network design, communication networks, and other fields. It also preserves edge weights, facilitating the construction of killnets with varying edge weights later on.

[0145] S4: Dynamic generation of kill nets in response to changes in the battlefield situation;

[0146] In real combat scenarios, the available nodes are not static. During intense combat, nodes on either the red or blue side may be destroyed, losing their ability to build a network. Simultaneously, through resupply, both sides will generate new nodes that need to be planned. The key to transforming a statically constructed kill network into a dynamic one lies in quickly addressing these issues of node reduction and addition.

[0147] (1) The number of red nodes increases and the number of blue nodes decreases;

[0148] The issue of increased red team nodes can be addressed by optimizing the capability market elements as much as possible before global planning. While the reduction in blue team nodes may result in some resource waste due to pre-planned equipment combinations, it doesn't require special consideration as there's no risk of "missing" any nodes.

[0149] (2) The number of red team nodes is reduced;

[0150] Currently, the problem of reducing the number of red team nodes can be solved by assigning different weights to evaluation indicators and selecting multiple optimal solutions when performing multi-objective optimization. The order of these optimal solutions represents the performance of different kill nets. When no nodes are reduced, the optimal solution is selected. When the nodes included in the optimal solution are destroyed, the selected multiple optimal solutions are traversed from the best to the worst performing until all the nodes included in the solution are usable. This is known as the "solution matrix traversal method".

[0151] (3) The number of blue team nodes has increased;

[0152] To address the issue of increasing blue team nodes, the "assumed objective method" can be used, such as... Figure 4 While the Blue Team targets to be destroyed vary, they can generally be categorized into several major groups through prior investigation and research. For each major group, the reconnaissance relationship between the Red Team's reconnaissance equipment and the target's strike relationship are similar. During global planning, major target groups not present in the current Blue Team operation can be added, allowing for multi-target optimization with redundant targets. When no new Blue Team nodes are added, the kill chains targeting redundant targets in the kill network remain in standby mode. When a new Blue Team node appears requiring attack, if it belongs to the same major group as an existing Blue Team node, the cluster will adopt a relay-sequential attack strategy, destroying the previous target and immediately moving on to destroy the new Blue Team node. If it does not belong to the same major group as any existing Blue Team node (i.e., it falls within a pre-assumed target category), the cluster will assign the corresponding standby redundant kill chain to activate and complete the attack on the new Blue Team node.

[0153] Assuming that a single red team strike node or reconnaissance node can only target one blue team target at a time, while a single red team communication node or command and control node can process information about multiple blue team targets at the same time, then the number of nodes of the same type that the kill network can target at the same time can be calculated by equation (17):

[0154] N i =min(Rec i ,Str i ), i = A, B, ... (17)

[0155] Rec i Str represents the number of reconnaissance edges targeting the i-th blue team target. i This represents the number of edges to be hit for the i-th blue target, where i = A, B, ... represents taking different blue targets.

[0156] Example 1

[0157] For a heterogeneous unmanned swarm kill network, the red team nodes are categorized into unmanned ships, unmanned submarines, unmanned vehicles, drones, and satellites, totaling 48. The blue team nodes to be attacked are divided into five categories based on reconnaissance and attack methods: surface, underwater, land, air, and space. Here, space targets mainly refer to communication and reconnaissance satellites. The node composition is shown in Table 1.

[0158] Table 1. Node Composition of Heterogeneous Unmanned Swarm Kill Network

[0159]

[0160]

[0161] The connections between nodes in the reconnaissance, communication, command and control, and strike layers are shown in the matrices in Tables 2, 3, 4, and 5.

[0162] Table 2. Node Reconnaissance Relationships in Heterogeneous Unmanned Clusters

[0163]

[0164] Table 3 Communication Relationships of Heterogeneous Unmanned Cluster Nodes

[0165]

[0166]

[0167] Table 4. Command and Control Relationships of Heterogeneous Unmanned Cluster Nodes

[0168]

[0169] Table 5 Node Attack Relationships in Heterogeneous Unmanned Clusters

[0170]

[0171] These tables show a relationship between row nodes and column nodes. For example, in Table 2, the column node "30" (r) corresponding to the first row node "1" (b1) is... 30 ) represents r 30 b1 can be detected, that is, b1 and r 30 There is a reconnaissance relationship between them. It is worth noting that, for ease of description, Table 5 discusses the relationship between columns and rows, not rows and columns. Furthermore, the relationships described in the table indicate a direct communication or accusation relationship between the two nodes. For example, r1 and r... 25 There are indirect communication relationships through r5, but these are not reflected in the table. In Table 4, if the row and corresponding column numbers are the same, it means that the node can accuse itself (i.e., it can independently complete the accusation process and autonomously generate strike plans through reconnaissance), and vice versa.

[0172] A complete kill chain requires several stages: "target → reconnaissance → (communication) → command and control → strike." The communication stage is not mandatory, and the command and control stage can be completed by a single node (provided its corresponding diagonal value in the command and control matrix is ​​1). Figure 5 As shown.

[0173] A. Algorithm training for constructing heterogeneous unmanned swarm kill networks.

[0174] The parameters used in the simulation are shown in Table 6.

[0175] Table 6 Simulation Parameter Settings

[0176]

[0177] The evaluation and target networks employ a 7-layer fully connected neural network with an input layer size of 16, an output layer size of 4, and five hidden layers with sizes of 128, 256, 128, 64, and 32 respectively. The neural network is trained using Adam. The reward sequence R is chosen to be the same size as the population size popnum; the ε in the greedy algorithm is set as ε... start ε decay and ε end Three parameters are used to ensure that ε gradually decreases during training; the size of the experience replay buffer Γ should be as large as possible within the limits of computer performance and memory, and is set to 512 in this simulation; the larger the discount rate γ, the more the D-DQN agent considers long-term rewards, and vice versa. The purpose of this invention is to quickly obtain better successor solutions, so this is a problem that considers short-term rewards more, and the value of γ is set to 0.8; the network update frequency UPDATE_EVERY refers to how many times the network parameters are updated before assigning them to the target network, and is set to 4 here.

[0178] Based on the MOEA algorithm of D-DQN, write a Python program to train the D-DQN agent by assigning weights of 2 / 8, 3 / 8, 1 / 8, and 2 / 8 to four kill net evaluation metrics.

[0179] In the D-DQN algorithm, the agent randomly samples from the experience replay pool during the learning process, using the sampled experiences as input. Therefore, unlike other supervised learning methods, the loss curve of DQN is not a valid reference metric and is not shown here.

[0180] After 5.8 hours of training, the reward curve is as follows: Figure 6 The dashed line represents the result after filtering and smoothing, while the solid line represents the result after smoothing. The reward is defined as shown in equation (18).

[0181]

[0182] It is the reward obtained by individual i in the j-th iteration when choosing operator op.

[0183] Observing the curves reveals that there are still some fluctuations after smoothing. This is because RL can only be trained by obtaining rewards through interaction with the environment, and its input is randomly sampled from the experience replay pool. The curves show that the algorithm has a substantial effect on solving the problem of kill net construction, generally indicating that the algorithm can improve the speed and quality of kill net construction by adaptively selecting the most suitable genetic operators for offspring through training.

[0184] B. Algorithm execution for heterogeneous unmanned swarm kill networks.

[0185] Using the trained model, the multi-objective Pareto solution was obtained through algorithm optimization, taking a total of 9.7 minutes. Since this embodiment selected four evaluation metrics (i.e., objective functions), the distribution of its solutions cannot be illustrated in a single graph; therefore, a total of [number missing] graphs are needed. The figure shows the distribution of solutions in the optimization results. Figure 7 As shown.

[0186] It's worth noting that approximately 80% of both the training time and the time spent on a single optimization run after training are consumed in calculating risk metrics (because the time complexity of calculating risk metrics is O(n log n)). 3 (As the number of red team nodes increases, the computation time cost increases exponentially), making the depth-first search algorithm for finding simple paths the optimal choice for this type of problem. If the risk metric is replaced with other computationally less computationally intensive metrics, the algorithm's training and optimization times will be significantly reduced.

[0187] Ten distinct Pareto solutions were generated. Similarly, points in the space tended to be closer to locations with high redundancy, high sensitivity, low risk, and low network entropy. The solution set matrix was obtained by sorting the solutions according to the weights of the evaluation metrics, as shown below. Figure 8 As shown. The first row of each solution represents nodes 1-24, and the second row represents nodes 25-48.

[0188] Based on the weights assigned to each evaluation index, the relationship between the aggregated weights of each scheme and the evaluation indexes calculated by equation (19) is shown in Table 7.

[0189]

[0190] Table 7 Comparison of Pareto solutions and weighted aggregate values

[0191]

[0192]

[0193] Since the multi-objective optimization problem is about minimizing the values ​​of each objective function, solutions with smaller aggregated weight values ​​are ranked higher. According to Table 7, when the objective weights are set to [2 / 8, 3 / 8, 1 / 8, 2 / 8], the first solution has the highest aggregated weight value and is the optimal solution.

[0194] Its corresponding multi-layer network model connectivity graph is as follows: Figure 9 As shown, 1-48 represent the red team nodes (our own nodes), and AE represents the blue team nodes (the enemy nodes). The optimal solution has a redundancy of 662.00 across four evaluation metrics, and a risk level of [missing information].

[0195] =0.13, sensitivity =0.05, network entropy =2.10, and includes planning results for the dynamic generation problem added for the blue side node.

[0196] C. Simulation of communication network topology construction.

[0197] Using the method for generating a minimum spanning tree from a multi-connected graph according to the present invention, the minimum spanning tree result is obtained by taking the communication network connectivity graph of the selected scheme as input. Figure 10 As shown.

[0198] This is a communication network topology construction result that avoids resource waste and redundant paths. Since this graduation project did not consider edge and node weights, this network topology construction result is not unique. If the battle situation changes and the kill net is dynamically generated, a new minimum spanning tree can be constructed by re-invoking the Prim minimum spanning tree method for multi-connected graphs.

[0199] D. Simulation of dynamic generation of kill net.

[0200] The optimal solution matrix traversal method is used. When a node is damaged, the matrix sorted by aggregation function value is traversed from best to worst until a suitable solution is found. Figure 11 For r7, r9, r 36 When damaged, the algorithm derives the optimal solution vector at that moment.

[0201] Table 8 Implementation of the "Solution Matrix Traversal Method"

[0202]

[0203] Enter the index of the damaged node in the command line, and the system will output the optimal solution among the available nodes. If too many nodes are damaged, and none of the obtained Parato solutions can meet the requirements, then it is necessary to perform multi-objective optimization again based on the existing nodes to select the optimal solution.

[0204] Figure 9In the code, "7 3 7 7 11" represents the ability to target an additional 7, 3, 7, 7, and 11 blue team nodes for the five types of sea, underwater, land, air, and space applications, respectively. Intuitively, this can be represented as... Figure 11 Let's represent this. If more than three new underwater nodes are added to the blue team, the kill network will be saturated with underwater targets, and it will be impossible to attack all blue team underwater targets simultaneously. A relay-sequential attack approach is required. The "hypothetical target method," which pre-classifies blue team targets and uses redundant kill chains to simultaneously attack similar blue team targets, solves the dynamic programming problem caused by the increase in blue team nodes.

[0205] E. The impact of operator selection on the results.

[0206] To highlight the advantages of MOEA based on D-DQN adaptive operator selection, this embodiment uses four operators (simulated binary crossover operator, crossover operator, and two differential evolution operators) from the operator candidate set to complete the genetic algorithm, and compares the optimization time and average aggregation weight value of the five algorithms. All algorithms were performed under the previously simulated MOEA parameters, and the comparison results are shown in Table 9.

[0207] Table 9 Comparison of different operator choices

[0208]

[0209]

[0210] The average weighted aggregate value in the table is defined as shown in equation (20), where popnum is the population size, i.e., the number of solutions; Let represent the weight aggregation value of the i-th solution. It can be seen that the MOEA using the D-DQN adaptive operator selection outperforms algorithms using only simulated binary crossover, crossover, and differential evolution operators in optimization time, and outperforms all other algorithms using only one operator in terms of average weight aggregation value. This demonstrates that the MOEA based on D-DQN adaptive operator selection, after training, can select the most suitable genetic operator based on the user-specified evaluation metric weights and the characteristics of each parent solution, generating offspring with greater average weight aggregation value and improved fitness. This results in a better, more user-suitable heterogeneous unmanned swarm kill net construction scheme at the same time cost.

[0211] F. User interface.

[0212] Integrating the aforementioned simulation results into the created user interface (UI) yields the following results: Figure 12The visualization clearly uses the multi-layer network model defined earlier to show the relationships between nodes in the current optimal solution. A drop-down menu allows users to select the desired heterogeneous unmanned swarm kill chain to display in the optimization results. Clicking the "Show the kill chain" button will show the task flow of the selected kill chain. Figure 13 , Figure 14 As shown.

[0213] Figure 14 This displays a kill chain that starts from blue node A, passes through red node A for reconnaissance, communication, command and control, and attack, and finally returns to blue node A. The dropdown menu's "Comm" and "C2" indicate whether this node handles the communication or command and control phase, respectively. Users can select any kill chain from the kill net's constructed results for dynamic display.

[0214] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0215] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0216] In this invention, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0217] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dynamically generating kill nets for heterogeneous unmanned swarms, characterized in that, Includes the following steps: Step S1: Select kill net evaluation indicators to reflect the relationship between the equipment capabilities of the red and blue teams; The kill network evaluation metrics include: redundancy metrics, risk metrics, sensitivity metrics, and network entropy metrics. Step S2: MOEA selected based on D-DQN adaptive operator; Using the NSGA-II process as a framework, in the operator candidate set, the D-DQN network adaptively selects the appropriate genetic operator based on the parent information when the genetic algorithm needs to generate offspring, solving the multi-objective optimization problem of constructing a heterogeneous unmanned swarm kill network and obtaining multiple optimal solutions. Step S3: Select the optimal kill net construction scheme based on the weight aggregation value; The kill net evaluation index values ​​of multiple optimal solutions are weighted and averaged to obtain a weighted aggregate value. This aggregate value is then used to rank the multiple optimal solutions and select the optimal solution from them. Step S4: Construct the communication network topology; First, the depth-first search (DFS) algorithm is used to decompose the multi-connected graph into a series of independent connected graphs. Then, the Prim method is used to construct the network topology for each connected graph. Step S5: Dynamically update the kill net; Whether to dynamically update the kill net depends on whether the battlefield situation changes; Step S6: Determine whether the changes in the battlefield situation are within the responsive range, that is, whether there is a feasible solution among the multiple optimal solutions. If so, return to step S4; otherwise, return to step S1.

2. The method for dynamically generating kill nets for heterogeneous unmanned swarms according to claim 1, characterized in that, In step S1, the redundancy index is the average number of kill chains that all blue team nodes can form; the risk index is the proportion of kill chains lost after each red team node is destroyed to the original number; the sensitivity index is the number of other layer nodes that the command node in the kill network calls; and the network entropy index is the network entropy of the communication network.

3. The method for dynamically generating kill nets for heterogeneous unmanned swarms according to claim 1, characterized in that, In step S2, the red team node and the kill network evaluation index are taken as states, the selection of genetic operators in the evolution process is taken as actions, the Q value learned by the neural network is taken as the basis for the selection of genetic operators, and the credit allocation strategy, the candidate set of operators and the adaptive operator selection strategy are defined.

4. The method for dynamically generating kill nets for heterogeneous unmanned swarms according to claim 1, characterized in that, Step S5 specifically includes: The kill net is dynamically updated based on changes in the red and blue team nodes: (1) When the number of red team nodes increases or the number of blue team nodes decreases, the original kill network remains unchanged; (2) When the number of red nodes decreases, the multiple optimal solutions obtained in step S2 are traversed from the best to the worst performance until all the nodes contained therein are available. (3) When a blue party node is added, if the added node belongs to the same category as an existing blue party node, a relay sequential attack strategy is adopted, that is, after destroying the previous node, the added blue party node of the same category is destroyed immediately; if the added blue party node does not belong to the same category as any existing blue party node, that is, it is a new target category, then the standby redundant kill chain is assigned to take effect and complete the attack on the added blue party node.

5. The method for dynamically generating kill nets for heterogeneous unmanned swarms according to claim 3, characterized in that, In step S2 The credit allocation strategy is to determine the reward strategy in reinforcement learning. The Chebyshev method is used as the aggregation function, where the objective function value is the maximum weighted distance between each individual's target and its target reference value, as shown in equation (8). In the formula, x represents the decision variable, w is the weight assigned to the objective function, and z * The reference point is M, the number of objective functions is w. c The weights assigned to the c-th objective function, f c (x) represents the function value of the c-th objective function; The reward value for offspring is shown in equation (9): In the formula, y represents the decision variable of the parent generation that generates offspring x.

6. The method for dynamically generating kill nets for heterogeneous unmanned swarms according to claim 3, characterized in that, In step S2 The genetic operators selected from the candidate set of operators are the simulated binary crossover operator, crossover operator, first difference evolution operator, or second evolutionary difference operator, and polynomial mutation is performed on the generated offspring solutions; a) Simulate the binary crossover operator, as shown in equation (10): Where d = 1, ..., D, x = (x1, ..., x2) d ,...,x D (This is a child's solution.) There are two parent generations, and: Where μ is a uniformly distributed random value sampled in [0,1], and η is a predefined parameter; b) Crossover operator, as shown in equation (12): Where r1 and r2 are uniformly distributed random values ​​sampled in [0,1], gen is the current generation number, and maxgen is the maximum generation number; c) The first difference evolution operator, where DE / rand / 1 is defined as in equation (13): It is the i-th parent generation, F and CR are preset constants, and r is a uniformly distributed random value sampled in [0,1]. d) The second difference evolution operator, DE / rand / 2, is defined as shown in equation (14): Where F and CR are predefined parameters, and r is a uniformly distributed random value sampled in [0,1]. Once a child solution is generated, a polynomial mutation is performed, as shown in equations (15) and (16): x d =x d +(u d -l d )δ (15) Among them, u d and l d These are the upper and lower bounds of the decision variable, which can take values ​​of 1 and 0 respectively. μ is a uniformly distributed random value sampled in [0,1], and η is a predefined parameter.

Images