A method for dynamic reconstruction of a kill web based on an endogenous intelligent hybrid architecture

The HIM-MAPPO algorithm, with its endogenous intelligent hybrid architecture, resolves the contradiction between global optimality and millisecond-level agility in kill net reconstruction, achieving efficient dynamic reconstruction under electromagnetic interference and node loss conditions, thereby improving operational resilience and resource utilization efficiency.

CN123069859APending Publication Date: 2026-11-10NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611160930.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-31
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing kill net reconstruction architectures cannot simultaneously meet the requirements of global optimality and millisecond-level agility. Centralized optimization methods have excessively long solution times, distributed mechanisms struggle to achieve global coordination in non-stationary battlefield situations, and multi-agent deep reinforcement learning methods face bottlenecks in heterogeneous action spaces and credit allocation.

Method used

The HIM-MAPPO algorithm, based on an endogenous intelligence hybrid architecture, is adopted to internalize global training intelligence into edge policy parameters. By constructing a heterogeneous decentralized partially observable Markov decision process and an endogenous intelligence hybrid architecture, a multi-objective composite reward function and a cognitive gating mechanism are designed to achieve centralized training and distributed execution.

Benefits of technology

It significantly improves the rebuilding efficiency and operational resilience of the kill network under severe electromagnetic interference and high-intensity node loss conditions, ensuring millisecond-level response and resource utilization efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123069859A_ABST
    Figure CN123069859A_ABST
Patent Text Reader

Abstract

The application discloses a kind of based on endogenous intelligent hybrid architecture's killing net dynamic reconstruction method, to solve the problem of "global optimal planning" and "millisecond level distributed response" difficult to take into account.The method constructs time-varying graph topology model, random link quality model and running loop efficiency evaluation model, and then dynamically reconstructs as decentralized partial observable Markov decision process, proposes tenderer-bidder role mapping and the mixed hierarchical action space formed by continuous bidding, discrete tendering, and based on the cognitive gating mechanism of participation desire degree, the decision space is pruned, and the reconstruction is guided by multi-objective composite reward, finally constructs the hybrid architecture coordinated by planning evaluation layer and response layer, uses heterogeneous multi-agent proximal policy optimization algorithm, to train strategy network in a centralized training, distributed execution manner.Simulation results show that the method significantly improves the reconstruction success rate, combat resilience and decision timeliness under strong electromagnetic interference and large-scale node loss.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the cross-technical fields of dynamic reconstruction of multi-domain collaborative combat systems, deep reinforcement learning of multi-agent systems, and intelligent command and control of unmanned combat swarms, and particularly relates to a method for dynamic reconstruction of kill networks based on an endogenous intelligent hybrid architecture. Background Technology

[0002] In recent years, modern warfare has been transforming from a traditional linear kill chain centered on platforms to a kill network characterized by multi-domain collaboration and dynamic resilience. Driven by the "mosaic warfare" concept, the kill network integrates heterogeneous sensors, decision-makers, and attack weapons into a resilient resource pool. Through dynamic linkages, it achieves system-level resilience, enabling adaptive reconfiguration of operational loops when nodes suffer kinetic attacks or electronic jamming failures, thereby maintaining closed-loop operational capabilities. However, the kill network paradigm places unprecedented dual demands on command and control systems: firstly, global optimality, requiring joint orchestration of heterogeneous sensing, decision-making, and attack resources under non-stationary interference and node attrition conditions; secondly, distributed millisecond-level agility, requiring the reconstruction of alternative loops within millisecond-second timescales of the target exposure window.

[0003] Existing kill net reconstruction architectures cannot simultaneously meet both of the above requirements. Centralized optimization methods aim for a globally optimal solution, using heuristics such as ant colony optimization, multi-objective evolutionary algorithms, or non-dominated sorting differential evolution to generate the optimal kill chain blueprint. However, their solution time is typically on the order of seconds to minutes, exceeding the millisecond-level reconstruction window by three to four orders of magnitude, creating an insurmountable time budget gap. Distributed mechanisms, represented by the classic Contract Net Protocol (CNP) and its improved version (ICNP), rely on local negotiation between nodes to achieve millisecond-level agile response and have strong robustness to single-point failures. However, limited by manually designed heuristic bidding rules, ICNP cannot internalize the non-stationary global battlefield situation into local decision-making, resulting in a serious "tactical shortsightedness" problem. It is prone to suboptimal convergence and resource contention, making it difficult to guarantee global coordination under large-scale reconstruction.

[0004] In recent years, Deep Reinforcement Learning (DRL) and its multi-agent extension (MADRL), especially the multi-agent proximal policy optimization (MAPPO) algorithm, heterogeneous-agent proximal policy optimization (HAPPO) algorithm, multi-agent deep deterministic policy gradient (MADDPG) algorithm, and soft actor-critic (SAC) algorithm based on the centralized training with decentralized execution (CTDE) paradigm, have provided new ideas for distributed intelligent decision-making. However, directly applying the above algorithms to killnet dynamic reconstruction still faces three bottlenecks: First, killnet reconstruction has a typical hybrid heterogeneous action space—the bidding party performs discrete choices, while the bidding party performs continuous bidding, causing the isomorphic action assumption on which the standard MAPPO / MADDPG is based to structurally fail; Second, the scalarized global reward cannot decouple the contributions of each party in the heterogeneous bidding-tendering coupled task, resulting in a serious credit allocation problem; Third, the distributed agent passively responds after node failure, lacking proactive cognition and scalability, making it difficult to maintain millisecond-level response in large-scale killnets.

[0005] Therefore, there is an urgent need for a kill network dynamic reconstruction method that can simultaneously take into account "global optimization" and "millisecond-level agility" and can internalize the global tactical wisdom formed by centralized training into the strategy parameters of distributed edge nodes. Summary of the Invention

[0006] This invention addresses the key bottlenecks in the aforementioned background technology by proposing a kill network dynamic reconstruction method based on an endogenous intelligent hybrid architecture. It aims to structurally decouple two budget-incompatible objectives: "global optimal planning" and "millisecond-level distributed response." Furthermore, it internalizes globally trained intelligence into edge policy parameters using the Heterogeneous Interaction Manifold Multi-Agent Proximal Policy Optimization (HIM-MAPPO) algorithm, thereby significantly improving the kill network reconstruction success rate and operational resilience under severe electromagnetic interference and high-intensity node attrition conditions. To achieve the above objectives, the steps employed in this invention are as follows:

[0007] Step 1: Construct a system model for the kill net dynamic reconstruction problem, including using time-varying graphs. The described kill network topology model, the stochastic link quality model considering large-scale path loss, Nakagami-m small-scale fading and electromagnetic interference, and the integrated path reliability and task timeliness Operating loop performance evaluation model

[0008] Step 2: Model the kill net dynamic reconstruction problem as a heterogeneous decentralized partially observable Markov Decision Process (Hetero-Dec-POMDP). Define the role mapping mechanism between bidders and tenderers in heterogeneous agents, the local observation space corresponding to each role, and the hybrid hierarchical action space composed of continuous bidding actions and discrete bidding actions. And cognitive gating mechanisms based on "participation desire";

[0009] Step 3: Design a multi-objective composite reward function, including closed-loop efficiency. The main item is performance recovery reward, and the remaining resources are Shannon entropy. Resource balancing reward with total closed-loop delay T total The time-sensitive nature of punishment and the effectiveness of rewards for fulfilling hard constraints (C) valid Linear weighted coupling of the four components;

[0010] Step 4: Construct the Endogenous Intelligence Hybrid Architecture (EIHA), which consists of a "planning-evaluation layer" and a "response layer." The planning-evaluation layer acts as a centralized commentator during offline training and is deprecated after deployment. The response layer consists of edge nodes equipped with endogenous intelligence modules. The HIM-MAPPO algorithm is proposed, which combines a "heavy commentator" network with joint information fusion and a heterogeneous dual-stream actor network. The bidding actor outputs continuous actions with a parameterized beta distribution, and the tendering actor outputs categorically distributed actions that are renormalized by dynamic intent masks. The policy network is trained using a "centralized training, distributed execution" approach, which solidifies global tactical intelligence into the policy parameters of each node in the response layer.

[0011] Furthermore, the specific method for constructing a system model for the kill net dynamic reconfiguration problem is as follows:

[0012] The kill web is formalized as a time-varying graph. Node set Includes all friendly units and enemy targets, each node v i Composed of holographic 6-tuple Ξ i Characterizing its role, position, speed, physical attributes, remaining channel capacity, and processing latency, it is defined as:

[0013]

[0014] Where ψ i ∈{S, C, D, A, T} represents the functional role type, where sensor is S, communication is C, decision-making is D, attack is A, and target is T; and For 3D position coordinates and velocity vector; k i Including the sensing radius R sen and strike radius R str Physical properties; τ represents the remaining capacity vector for four types of channels: sensing, communication, decision-making, and attack. i To handle latency within the node;

[0015] Link e ij Information transmission quality q info (e ij The instantaneous signal-to-interference-plus-noise ratio is defined as:

[0016]

[0017] Where P t,i Let G be the transmit power of node i. ij For link e ijThe transmit and receive antenna gain, d ij Let h be the distance between node i and node j, and L(·) be the path loss function; where h ij For link e ij The small-scale fading channel coefficients follow a Nakagami-m distribution. The path loss exponent α is differentiated according to role: for the detection link S→T, which is a two-way radar attenuation, α = 4; and for the communication and command link S→C→D→A, which is a one-way propagation, α = 2. For the set of interference sources, I k,j For the interference power, σ 2 γ is the power of Gaussian white noise. th To establish a demodulation threshold, the link's information carrying capacity is further described using Shannon capacity.

[0018] c(e ij ) = B ij ·log2(1+SINR ij (3)

[0019] Among them B ij SINR is used to allocate bandwidth. ij For link e ij The instantaneous signal-to-interference-plus-noise ratio;

[0020] The effectiveness of striking the link A→T is characterized by a probabilistic model of capability comparison:

[0021]

[0022] in and D def These represent the attacker's offensive capability and the target's defensive strength, respectively. For indicator functions;

[0023] The link quality degradation caused by electromagnetic interference is modeled as follows:

[0024]

[0025] in Let R be the spherical suppression region for the k-th interference source. jam ;

[0026] Node failures are categorized into hard failures and soft failures. Hard failures are determined using Bayesian collaborative sensing posterior probability, where D is the probability of failure. i For node v i Physical destruction events If there are K consecutive missing heartbeat events, then:

[0027]

[0028] in Let δ be the link persistence probability derived from the kinematic link expiration time, Δt be the time interval between two adjacent heartbeats, and let δ be the link persistence probability derived from the kinematic link expiration time. conf The confidence threshold for hard failure decision is when Time-based refactoring is triggered; soft failure is defined as a node being physically intact but having exhausted its internal resources.

[0029]

[0030] Where m is the channel type index. The resource exhaustion decision threshold for the m-th type of channel of a node;

[0031] Operating loop The overall performance is determined by path reliability. With task agility Definition of product:

[0032]

[0033]

[0034] in For the tactical value of the target, τ is the total end-to-end delay of the loop. v With τ e T represents the internal processing delay of node v and the transmission delay of link e, respectively. max The target exposure window is δ, which is the time delay sensitivity coefficient; E rel Embodying the "weakest link" characteristic, E time Timeouts are penalized with hyperbolic tangent decay, and the loop bottleneck throughput Θ and the system-level aggregate throughput Θ summed over all active running loops are defined using the minimum cut principle. agg :

[0035]

[0036] Furthermore, the specific method for modeling the kill net dynamic reconstruction problem as a Hetero-Dec-POMDP is as follows:

[0037] The dynamic reconstruction problem is modeled as a problem consisting of seven tuples. The defined Hetero-Dec-POMDP, where The set of combat nodes is defined by the following elements: global state space, joint observation space, joint action space, state transition probability function, reward function, and discount factor γ. At each reconstruction step, the ICNP mechanism dynamically assigns each agent to one of two roles based on the perceived failure event: a tenderer (i∈N) that detects the failure and executes discrete decisions. ten), or Bidder (j∈N) who generates continuous bids based on internal state. bid );

[0038] (1) Status and Role-Specific Observation Space:

[0039] global state s t The functional requirements, encompassing dynamic graph topology, electromagnetic interference field distribution, and failure loops, are defined as follows:

[0040]

[0041] Execution phases t It is invisible to decentralized agents, but provides a key informational basis for heavy critics to build a global value baseline during the training phase;

[0042] The local observation vector of bidder node j is constructed as follows:

[0043]

[0044] Where, d bid This represents the dimension of the bidder's local observation vector. Aggregate the remaining resource status c j(t) Relative link quality to the bidding party And local topological embedding; the local observation vector of the bidding party node i is extended based on the bidding party's observation and is defined as:

[0045]

[0046] Where, d ten This represents the dimension of the local observation vector of the bidding party, and the local observation of the bidding party's node. Includes target tactical value And the specific resource requirements of the tasks to be processed;

[0047] (2) Hybrid Layered Motion Space: Hybrid Layered Motion Space Structurally, the decision-making process is decoupled into the "capability projection" and "optimal selection" stages. In the bidder flow, strategy π... bid Map local observations to continuous action vectors u j,t ∈[0,1] 3 To ensure the boundedness of resource commitments and promote stable gradient flow, the bidding strategy is parameterized using a Beta distribution:

[0048]

[0049] Where α θ and β θThis represents the concentration parameter of the bidder network output. This vector potentially encodes three key dimensions: willingness to participate (α). j Resource allocation β j and tactical cost γ j .

[0050] (3) Cognitive gating mechanism: In order to connect these continuous signals with discrete protocol logic, a cognitive gating mechanism is introduced to generate dynamic intention mask M. t ∈{0,1} N ,in

[0051] Gating threshold δ th During training, a linear annealing strategy is used to transition from 0.1 to 0.5, first broadly exploring the candidate pool and then gradually focusing on high-confidence bidders. This mechanism filters out nodes that are unwilling or incapable at the source, pruning the originally NP-hard search space into a manageable subset, thereby ensuring millisecond-level response.

[0052] Furthermore, the specific method for designing a multi-objective composite reward function is as follows:

[0053] To guide decentralized agents toward the Pareto optimal frontier that balances task performance, load distribution, and responsiveness, a scalar global reward function r consisting of four weighted objectives is proposed. t (s, a):

[0054]

[0055] Where ξ, λ1, and λ2 are the weight coefficients of the efficiency recovery term, resource entropy term, and timeliness penalty term, respectively; main part The goal is to restore the closed-loop efficiency of the new operating loop after incentive reconstruction. This objective is determined by the resource entropy term. The balancing term maximizes the Shannon entropy of the residual resource distribution of the winning node to prevent specific channels from being exhausted and to promote long-term load balancing. Furthermore, the timeliness penalty term minimizes the total closed-loop delay T relative to the target exposure window. total The selection process prioritizes nodes that are topologically adjacent and have superior computational performance, with a fixed effectiveness reward. This ensures that the hard physical constraints are met.

[0056] Furthermore, the specific method for constructing the endogenous intelligent hybrid architecture EIHA and the HIM-MAPPO algorithm is as follows:

[0057] The EIHA (Endogenous Intelligence Hybrid Architecture) structurally decouples the command hierarchy into two coordination layers:

[0058] (1) Planning-Assessment Layer: Residing in the command center, it undertakes two types of lifecycle functions. Before engagement, it analyzes the initial overall situation. The optimal initial running loop is generated and the kill net topology is initialized using a global optimization method; during the offline training phase, it is transformed into a central commentator in the CTDE paradigm to guide distributed edge nodes to learn collaborative strategies aligned with the global model; this layer is discarded during the deployment and execution phase.

[0059] (2) Response layer: It consists of heterogeneous combat nodes equipped with lightweight "endogenous intelligence modules". The nodes interact in a closed loop through "event-triggered negotiation" and are divided into three stages: reconstruction bidding, intelligent bidding and winning decision. The entire closed-loop decision-making process is completed in milliseconds.

[0060] The HIM-MAPPO algorithm implements the offline centralized training process of EIHA and consists of the following three types of networks:

[0061] (1) Joint Information Fusion Heavy Commentator Network V ψ As an omniscient supervisor, the global state s t The joint observations are spliced ​​together to form a fused situation vector z. t Defined as:

[0062] z t =[s t o 1,t , ..., o N,t (17)

[0063] z t After processing by a multilayer perceptron with hidden layer dimensions of 1024-512-256, the output is a global value estimate V. ψ (z t The return estimated by commentators using the Generalized Advantage Estimation (GAE) method. To minimize the mean squared error loss:

[0064]

[0065] Where B represents the mini-batch size. This architectural design ensures that the critic can provide precise value guidance to the distributed executor by capturing the impact of local refactoring actions on the entire system. The critic's asymmetric structure backpropagates global topological dependencies to the decentralized Actor through global-local knowledge distillation, internalizing the inter-role collaboration relationships that are invisible to local nodes during execution into "endogenous intuition," thereby structurally solving the credit allocation problem under coupled rewards.

[0066] (2) Bidder Actor Network Employing a multilayer perceptron, with input dimension d b →512→256→256, Local observation of consumer bidders Output Beta distribution concentration parameters Then sample the three-dimensional continuous bidding action vector u j,t ∈[0,1] 3 Sample according to the above-described Beta distribution parameterization method;

[0067] (3) Tendering party Actor Network Employing a multilayer perceptron, the concatenated vector of consumer bidding observations and bidding actions is used. Among them B t The output is a concatenation of all Bidder bidding actions, masked by M. t The normalized Categoricallogits are then used to sample discrete selection actions a. i,t :

[0068]

[0069] Where ε is the numerical stability constant to prevent the denominator from being zero; the parameter set θ of the two Actor networks b With θ t Strictly non-overlapping and independently optimized to avoid gradient interference caused by sharing the underlying backbone; although parameters are decoupled, they share the same task-level reward r. t And the GAE advantage estimated by heavy critics. As a unified optimization signal;

[0070] Training employs a "centralized training, distributed execution" approach. Each loop iteration has K... tr Round, T per round tr Step, each update J ppo There are 1 training round; in each step, all Bidders first generate bidding actions and construct an intent mask M. t Then, the Tenderer samples discretely selected actions and interacted with the environment to obtain rewards r. t It is then stored in the experience buffer, and an update is triggered every few steps.

[0071] During the deployment phase, heavy commentators are completely discarded, and each edge node retains only the corresponding Bidder Actor or Tenderer Actor for real-time decision-making. The "global tactical intelligence" obtained during the training phase is solidified in the network weights in the form of policy parameters, realizing "offline centralized training, online distributed execution, and millisecond-level dynamic reconstruction during deployment based on the endogenous intelligence contained in the parameters". Attached Figure Description

[0072] Figure 1 This is a concept diagram of a real battlefield that this invention is aimed at;

[0073] Figure 2This is a simulation scene diagram of the method proposed in this invention;

[0074] Figure 3 This is a block diagram of the endogenous intelligent hybrid architecture system proposed in this invention;

[0075] Figure 4 This is a simulation software diagram of the kill net dynamic reconstruction.

[0076] Figure 5 This is a simulation comparison of the average screen reward of the present invention and existing methods during dynamic reconstruction of the kill net;

[0077] Figure 6 This is the initial kill net topology simulated in AFSIM;

[0078] Figure 7 This is a diagram of kill net reconstruction topology changes oriented towards node hard failure, simulated in AFSIM.

[0079] Figure 8 This is a diagram of kill net reconstruction topology changes oriented towards node soft failures, simulated in AFSIM.

[0080] Figure 9 This is a graph showing the global effectiveness changes of the kill net during the Monte Carlo simulation process. Detailed Implementation

[0081] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0082] This invention proposes a method for dynamic reconfiguration of a kill network based on an endogenous intelligent hybrid architecture. The following operating conditions are first set:

[0083] 1. As attached Figure 1 With appendix Figure 2 As shown, assuming the battlefield is a square theater with sides of 300km, a heterogeneous kill network of the Red Force and a target system of the Blue Force are deployed. The Red Force has 60 heterogeneous nodes: one airborne early warning and control platform as decision-making node D, with a detection radius of 400km; a swarm of 28 UAVs as sensor nodes S, with a detection radius of 150km; 12 fighter jets as strike nodes A, with a strike radius of 100km; 16 bombers as strike nodes A, with a strike radius of 200km; one ground radar as sensor node S, with a detection radius of 450km; and two destroyers as strike nodes A, with a strike radius of 150km. The Blue Force has 37 target entities, including a formation of 20 fighter jets, 8 bombers, 3 electronic warfare / radar nodes, 1 destroyer, 2 static command posts, and 3 area air defense systems, with the area air defense systems having a strike radius of 120km.

[0084] 2. Electromagnetic environment and failure model parameters: Hard failure probability P hard =5×10-4 Soft failure probability P soft =2×10 -2 Interference radius R jam =15km, interference power Link capacity threshold γ th =10dB corresponds to the minimum signal-to-interference-plus-noise ratio requirement for quadrature phase-shift keying demodulation in tactical data links; HIM-MAPPO algorithm hyperparameters: Actor learning rate α lr =1×10 -4 Critic learning rate 5×10 -4 Cognitive gating threshold δ th Linear annealing is performed between 0.1 and 0.5, with a discount factor γ = 0.99, GAE parameter λ = 0.95, proximal policy optimization (PPO) truncation parameter ε = 0.2, and PPO update round number J. ppo =10, entropy coefficient 0.02, reward weights ξ=+5.0, λ1=+2.0, λ2=-0.1, Hard constraint penalty = -10.0.

[0085] 3. The deep reinforcement learning algorithms used to train the HIM-MAPPO strategy were all implemented using Python 3.9 and PyTorch, and deployed on a high-performance workstation equipped with an AMD Ryzen 9 7945HS CPU and an NVIDIA RTX 4060 GPU. (See attached image.) Figure 4 As shown, the training process utilizes a dual-fidelity co-simulation ecosystem comprised of the Python decision core and the AFSIM (Advanced Framework for Simulation, Integration and Modeling) combat engine: a lightweight Python-based environment supports high-throughput statistical training, while the AFSIM platform supports high-fidelity mission-level verification. Each simulation step represents a complete kill network reconstruction decision moment.

[0086] Under the above operating conditions, the kill net dynamic reconstruction method based on an endogenous intelligent hybrid architecture proposed in this invention has been implemented on the Python+AFSIM co-simulation platform, and the simulation results prove the effectiveness of the method. The specific implementation steps of the method proposed in this invention are as follows:

[0087] Step 1: Construct a system model for the dynamic reconfiguration problem of the kill net.

[0088] The kill web is formalized as a time-varying graph. Node set Includes all friendly units and enemy targets, each node v iComposed of holographic 6-tuple Ξ i Characterizing its role, position, speed, physical attributes, remaining channel capacity, and processing latency, it is defined as:

[0089]

[0090] Where ψ i ∈{S, C, D, A, T} represents the functional role type, where sensor is S, communication is C, decision-making is D, attack is A, and target is T; and For 3D position coordinates and velocity vector; k i Including the sensing radius R sen and strike radius R str Physical properties; τ represents the remaining capacity vector for four types of channels: sensing, communication, decision-making, and attack. i To handle latency within the node;

[0091] Link e ij Information transmission quality q info (e ij The instantaneous signal-to-interference-plus-noise ratio is defined as:

[0092]

[0093] Where P t,i Let G be the transmit power of node i. ij For link e ij The transmit and receive antenna gain, d ij Let h be the distance between node i and node j, and L(·) be the path loss function; where h ij Let be the small-scale fading channel coefficient of link eij, which follows a Nakagami-m distribution. The path loss exponent α is distinguished according to role, where the detection link S→T is a two-way radar attenuation and takes α=4, and the communication command link S→C→D→A is a one-way propagation and takes α=2. For the set of interference sources, I k,j For the interference power, σ 2 γ is the power of Gaussian white noise. th To establish a demodulation threshold, the link's information carrying capacity is further described using Shannon capacity.

[0094] C(e ij ) = B ij ·log2(1+SINR ij ) (twenty two)

[0095] Among them B ij SINR is used to allocate bandwidth. ij For link e ij The instantaneous signal-to-interference-plus-noise ratio;

[0096] The effectiveness of striking the link A→T is characterized by a probabilistic model of capability comparison:

[0097]

[0098] Where Ca tk and D def These represent the attacker's offensive capability and the target's defensive strength, respectively. For indicator functions;

[0099] The link quality degradation caused by electromagnetic interference is modeled as follows:

[0100]

[0101] in Let R be the spherical suppression region for the k-th interference source. jam ;

[0102] Node failures are categorized into hard failures and soft failures. Hard failures are determined using Bayesian collaborative sensing posterior probability, where D is the probability of failure. i For node v i Physical destruction events If there are K consecutive missing heartbeat events, then:

[0103]

[0104] in Let δ be the link persistence probability derived from the kinematic link expiration time, Δt be the time interval between two adjacent heartbeats, and let δ be the link persistence probability derived from the kinematic link expiration time. conf The confidence threshold for hard failure decision is when Time-based refactoring; soft failure is defined as a node being physically intact but having exhausted its internal resources:

[0105]

[0106] Where m is the channel type index. The resource exhaustion decision threshold for the m-th type of channel of a node;

[0107] Operating loop The overall performance is determined by path reliability. With task agility Definition of product:

[0108]

[0109] in For the tactical value of the target, τ is the total end-to-end delay of the loop. v With τ eT represents the internal processing delay of node v and the transmission delay of link e, respectively. max The target exposure window is δ, which is the time delay sensitivity coefficient; E rel Embodying the "weakest link" characteristic, E time Timeouts are penalized with hyperbolic tangent decay, and the loop bottleneck throughput Θ and the system-level aggregate throughput Θ summed over all active running loops are defined using the minimum cut principle. agg :

[0110]

[0111] Step 2: Model the kill web dynamic reconstruction problem as Hetero-Dec-POMDP.

[0112] The dynamic reconstruction problem is modeled as a problem consisting of seven tuples. The defined Hetero-Dec-POMDP, where The set of combat nodes is defined by the following elements: global state space, joint observation space, joint action space, state transition probability function, reward function, and discount factor γ. At each reconstruction step, the ICNP mechanism dynamically assigns each agent to one of two roles based on the perceived failure event: a tenderer (i∈N) that detects the failure and executes discrete decisions. ten ), or Bidder (j∈N) who generates continuous bids based on internal state. bid );

[0113] (1) Status and Role-Specific Observation Space:

[0114] global state s t The functional requirements, encompassing dynamic graph topology, electromagnetic interference field distribution, and failure loops, are defined as follows:

[0115]

[0116] Execution phases t It is invisible to decentralized agents, but provides a key informational basis for heavy critics to build a global value baseline during the training phase;

[0117] The local observation vector of bidder node j is constructed as follows:

[0118]

[0119] Where, d bid This represents the dimension of the bidder's local observation vector. Aggregate the remaining resource status c j(t) Relative link quality to the bidding party And local topological embedding; the local observation vector of the bidding party node i is extended based on the bidding party's observation and is defined as:

[0120]

[0121] Where, d ten This represents the dimension of the local observation vector of the bidding party, and the local observation of the bidding party's node. Includes target tactical value And the specific resource requirements of the tasks to be processed;

[0122] (2) Hybrid Layered Motion Space: Hybrid Layered Motion Space Structurally, the decision-making process is decoupled into the "capability projection" and "optimal selection" stages. In the bidder flow, strategy π... bid Map local observations to continuous action vectors u j,t ∈[0,1] 3 To ensure the boundedness of resource commitments and promote stable gradient flow, the bidding strategy is parameterized using a Beta distribution:

[0123]

[0124] Where α θ and β θ This represents the concentration parameter of the bidder network output. This vector potentially encodes three key dimensions: willingness to participate (α). j Resource allocation β j and tactical cost γ j .

[0125] (3) Cognitive gating mechanism: In order to connect these continuous signals with discrete protocol logic, a cognitive gating mechanism is introduced to generate dynamic intention mask M. t ∈{0,1} N ,in

[0126] Gating threshold δ th During training, a linear annealing strategy is used to transition from 0.1 to 0.5, first broadly exploring the candidate pool and then gradually focusing on high-confidence bidders. This mechanism filters out unwilling / incapable nodes at the source, pruning the originally NP-hard search space into a manageable subset, thereby ensuring millisecond-level response.

[0127] Step 3: Design a multi-objective composite reward function.

[0128] To guide decentralized agents toward the Pareto optimal frontier that balances task performance, load distribution, and responsiveness, a scalar global reward function r consisting of four weighted objectives is proposed. t (s, a):

[0129]

[0130] Where ξ, λ1, and λ2 are the weight coefficients of the efficiency recovery term, resource entropy term, and timeliness penalty term, respectively; main part The goal is to restore the closed-loop efficiency of the new operating loop after incentive reconstruction. This objective is determined by the resource entropy term. The balancing term maximizes the Shannon entropy of the residual resource distribution of the winning node to prevent specific channels from being exhausted and to promote long-term load balancing. Furthermore, the timeliness penalty term minimizes the total closed-loop delay T relative to the target exposure window. total The selection process prioritizes nodes that are topologically adjacent and have superior computational performance, with a fixed effectiveness reward. This ensures that the hard physical constraints are met.

[0131] Step 4: Construct the endogenous intelligent hybrid architecture EIHA and HIM-MAPPO algorithm.

[0132] The EIHA (Endogenous Intelligence Hybrid Architecture) structurally decouples the command layer into two coordination layers:

[0133] (1) Planning-Assessment Layer: Residing in the command center, it undertakes two types of lifecycle functions. Before engagement, it analyzes the initial overall situation. The optimal initial running loop is generated and the kill net topology is initialized using a global optimization method; during the offline training phase, it is transformed into a central commentator in the CTDE paradigm to guide distributed edge nodes to learn collaborative strategies aligned with the global model; this layer is discarded during the deployment and execution phase.

[0134] (2) Response layer: It consists of heterogeneous combat nodes equipped with lightweight "endogenous intelligence modules". The nodes interact through "event-triggered negotiation" in a closed loop, which is divided into three stages: reconstruction bidding, intelligent bidding and winning decision. The entire closed-loop decision-making process is completed in milliseconds.

[0135] The HIM-MAPPO algorithm implements the offline centralized training process of EIHA and consists of the following three types of networks:

[0136] (1) Joint Information Fusion Heavy Commentator Network V ψ As an omniscient supervisor, the global state s t The joint observations are spliced ​​together to form a fused situation vector z. t Defined as:

[0137] z t =[s t o 1,t , ..., o N,t (36)

[0138] z tAfter processing by a multilayer perceptron with hidden layer dimensions of 1024-512-256, the output is a global value estimate V. ψ (z t The commentator estimated the returns using the Generalized Advantage Estimation (GAE) method. To minimize the mean squared error loss:

[0139]

[0140] Where B represents the mini-batch size. This architectural design ensures that the critic can provide precise value guidance to the distributed executor by capturing the impact of local refactoring actions on the entire system. The critic's asymmetric structure backpropagates global topological dependencies to the decentralized Actor through global-local knowledge distillation, internalizing the inter-role collaboration relationships that are invisible to local nodes during execution into "endogenous intuition," thereby structurally solving the credit allocation problem under coupled rewards.

[0141] (2) Bidder Actor Network A multilayer perceptron is used, with an input dimension of d. b →512→256→256, Local observation of consumer bidders Output Beta distribution concentration parameters Then sample the three-dimensional continuous bidding action vector u j,t ∈[0,1] 3 Sample according to the above-described Beta distribution parameterization method;

[0142] (3) Tendering Party Actor Network Employing a multilayer perceptron, the concatenated vector of consumer bidding observations and bidding actions is used. Among them B t The output is a concatenation of all Bidder bidding actions, masked by M. t The normalized categorical logits are then used to sample discrete selection actions a. i,t :

[0143]

[0144] Where ε is the numerical stability constant to prevent the denominator from being zero; the parameter set θ of the two Actor networks b With θ t Strictly non-overlapping and independently optimized to avoid gradient interference caused by sharing the underlying backbone; although parameters are decoupled, they share the same task-level reward r. t And the GAE advantage estimated by heavy critics. As a unified optimization signal;

[0145] Training employs a "centralized training, distributed execution" approach. Each loop iteration has K... tr Round, T per round tr Step, each update J ppo There are several training rounds; in each step, all Bidders first generate bidding actions and construct an intent mask M, then the Tenderer samples discretely selected actions and interacts with the environment to obtain a reward r. t It is then stored in the experience buffer, and an update is triggered every few steps.

[0146] During the deployment phase, heavy commentators are completely discarded, and each edge node retains only the corresponding Bidder Actor or Tenderer Actor for real-time decision-making. The "global tactical intelligence" obtained during the training phase is solidified in the network weights in the form of policy parameters, realizing "offline centralized training, online distributed execution, and millisecond-level dynamic reconstruction during deployment based on the endogenous intelligence contained in the parameters".

[0147] The following section focuses on describing the training implementation details and simulation verification results:

[0148] (A) Training Implementation Details: At the beginning of each training session, the planning-evaluation layer assesses the current global situation. Generate the initial set of running loops And based on the average simulation duration T of one scene tr = 150-300 steps drive scene evolution. Within each step t, the cognitive gating and implicit learning reconstruction process of the HIM-MAPPO algorithm is executed sequentially—for each bidder j, from π bid Sample u j,t And extract the intention component α j,t Based on this, an intent mask M is generated. t Only if α j,t >δ th And set node j to 1 when it is alive; input the concatenated bidding party. With mask M t Input πten, after masking and probability renormalization, sample the winning action a_t; execute the joint action a. joint,t =(u t a t ), to obtain reward r t With the next overall situation s t+1 ; transfer tuple (z t o t α joint,t M t r t V ψ (z t Store in the experience replay buffer D bufAfter the update interval is reached, use GAE in D. buf Above computational advantages For each PPO round, the reviewer mean squared error loss, Bidder PPO loss, and Tenderer mask PPO loss are calculated separately in mini-batches, and the parameters ψ and θ are updated step by step with an independent optimizer. b θ t The buffer is cleared after each episode. All ablation variants are trained for a total of 10,000 episodes, covering 5 independent random seeds.

[0149] (B) Large-scale campaign-level scenario comparison experiment: In a large-scale campaign-level scenario with N=60 platforms, 37 targets, and 4 interference sources, the training convergence results of the proposed HIM-MAPPO algorithm and four comparative baselines—MADDPG, SAC, Trust Region Policy Optimization (TRPO), and Heuristic Ant Colony Optimization (ACO)—are attached. Figure 5 As shown. (Attached) Figure 5 Subgraphs (a), (b), (c), and (d) respectively illustrate the evolution trends of cumulative stage reward, reconstruction power, resource entropy, and average loop repair delay. It can be seen that:

[0150] (i) The algorithm proposed in this invention converges at the fastest speed—it breaks through 80% reconstruction success rate around the 1300th training session and finally stabilizes at about 88.5%, achieving a significant advantage of 17-21pp compared to the suboptimal baseline TRPO.

[0151] (ii) In terms of resource entropy index, the algorithm proposed in this invention converges to 0.92, proving that the proposed composite reward effectively drives balanced resource utilization;

[0152] (iii) The average loop repair delay is reduced to 20.5s, which is about 5s faster than the suboptimal baseline TRPO. The two-class separation strategy methods exhibit obvious training oscillations due to the sample staleness problem in the replay buffer under non-stationary conditions, while the proposed method effectively addresses the non-stationarity caused by interference source migration and node loss by relying on the same strategy PPO framework and CTDE asymmetric structure.

[0153] (C) AFSIM High-Fidelity Verification: The trained HIM-MAPPO model is deployed to the high-fidelity physical environment established by the AFSIM combat engine. A typical initial kill net topology is shown in the attached figure. Figure 6 As shown, the scenario includes 60 nodes for the red team and 37 targets for the blue team, along with four types of links: sensing, communication, decision-making, and attack. Use case studies are conducted for two typical scenarios: hard failure and soft failure.

[0154] (i) Hard failure use cases: see attached Figure 7 As shown, when the initial loop After the attack node R402 is damaged by passive energy, the algorithm proposed in this invention immediately triggers the ICNP reconstruction mechanism, and selects bomber R114 as the alternative attack node within milliseconds. The reconstructed loop achieves a bottleneck SINR of 5.56dB and an end-to-end delay of 26.90s, and the alternative node retains 50% of the spare channel.

[0155] (ii) Soft failure use cases: see attached Figure 8 As shown, when the reconnaissance UAV R603 is blinded by high-intensity interference, causing simultaneous impact on four operational loops covering targets B901, B802, B501, and B803, the proposed algorithm autonomously identifies spectrum-safe alternatives under cognitive gating guidance, converging to the fighter cluster R308-R311, which is geometrically closer to the target, and all five affected loops are successfully reconstructed.

[0156] (D) Monte Carlo Statistical Assessment: as attached Figure 9 As shown, under a 300s engagement window and 100 independent Monte Carlo experiments, the algorithm proposed in this invention achieves a median validity of 2901.5 with a 95% confidence interval of [2795.5, 2924.5], which is significantly better than TRPO, SAC, and MADDPG. The interquartile range of the algorithm proposed in this invention is the smallest among the four methods, proving that it has the lowest decision variance under random perturbation.

[0157] Based on the simulation results above, it can be concluded that the kill network dynamic reconstruction method based on endogenous intelligent hybrid architecture proposed in this invention can significantly improve the dynamic reconstruction success rate, closed-loop combat resilience, resource utilization efficiency, and decision-making timeliness of the kill network under severe electromagnetic interference, large-scale node loss, and non-stationary battlefield conditions compared with existing heuristic and multi-agent deep reinforcement learning methods. It structurally solves the core contradiction of the incompatibility between the two types of budgets: "global optimal planning" and "millisecond-level distributed response".

[0158] The contents not described in detail in this application are existing technologies known to those skilled in the art.

Claims

1. A method for dynamic reconfiguration of a kill network based on an endogenous intelligent hybrid architecture, comprising the following steps: Step 1: Construct a system model for the kill net dynamic reconstruction problem, including using time-varying graphs. The described kill network topology model, the stochastic link quality model considering large-scale path loss, Nakagami-m small-scale fading and electromagnetic interference, and the integrated path reliability and task timeliness Operating loop performance evaluation model The specific method is as follows: Forming the kill web as a time-varying graph Node set Includes all friendly units and enemy targets, each node v i Composed of holographic 6-tuple Ξ i Characterizing its role, position, speed, physical attributes, remaining channel capacity, and processing latency, it is defined as: Where ψ i ∈{S, C, D, A, T} represents the functional role type, where sensor is S, communication is C, decision-making is D, attack is A, and target is T; and For 3D position coordinates and velocity vector; k i Including the sensing radius R sen and strike radius R str Physical properties; τ represents the remaining capacity vector for four types of channels: sensing, communication, decision-making, and attack. i To handle latency within the node; Link e ij Information transmission quality q info (e ij The instantaneous signal-to-interference-plus-noise ratio is defined as: Where P t,i Let G be the transmit power of node i. ij For link e ij The transmit and receive antenna gain, d ij Let h be the distance between node i and node j, and L(·) be the path loss function; where h ij For link e ij The small-scale fading channel coefficients follow a Nakagami-m distribution. The path loss exponent α is differentiated according to role: for the detection link S→T, which is a two-way radar attenuation, α = 4; and for the communication and command link S→C→D→A, which is a one-way propagation, α = 2. For the set of interference sources, I k,j For the interference power, σ 2 γ is the power of Gaussian white noise. th To establish a demodulation threshold, the link's information carrying capacity is further described using Shannon capacity. C(e ij )=B ij ·log2(1+SINR ij ) (3) Among them B ij SINR is used to allocate bandwidth. ij For link e ij The instantaneous signal-to-interference-plus-noise ratio; The effectiveness of striking the link A→T is characterized by a probabilistic model of capability comparison: Where C atk and D dEf These represent the attacker's offensive capability and the target's defensive strength, respectively. For indicator functions; The link quality degradation caused by electromagnetic interference is modeled as follows: in Let R be the spherical suppression region for the k-th interference source. jam ; Node failures are categorized into hard failures and soft failures. Hard failures are determined using Bayesian collaborative sensing posterior probability, where D is the probability of failure. i For node v i Physical destruction events If there are K consecutive missing heartbeat events, then: in Let δ be the link persistence probability derived from the kinematic link expiration time (LET), Δt be the time interval between two adjacent heartbeats, and let δ be the link persistence probability derived from the kinematic link expiration time (LET). conf The confidence threshold for hard failure decision is when Time triggers reconstruction; Soft failure is defined as a node that is physically intact but has exhausted its internal resources: Where m is the channel type index. The resource exhaustion decision threshold for the m-th type of channel of a node; Operating loop The overall performance is determined by path reliability. With task agility Definition of product: in For the tactical value of the target, τ is the total end-to-end delay of the loop. v With τ e T represents the internal processing delay of node v and the transmission delay of link e, respectively. max The target exposure window is δ, which is the time delay sensitivity coefficient; E rel Embodying the "weakest link" characteristic, E time Timeouts are penalized with hyperbolic tangent decay, and the loop bottleneck throughput Θ and the system-level aggregate throughput Θ summed over all active running loops are defined using the minimum cut principle. agg : Step 2: Model the kill net dynamic reconstruction problem as a heterogeneous decentralized partially observable Markov decision process, defining the role mapping mechanism between bidders and tenderers in heterogeneous agents, the local observation space corresponding to each role, and the hybrid hierarchical action space composed of continuous bidding actions and discrete bidding actions. And a cognitive gating mechanism based on "participation desire," the specific method of which is as follows: The dynamic reconstruction problem is modeled as a problem consisting of seven tuples. The defined heterogeneous decentralized partial observable Markov decision process, in which The set of combat nodes is defined by the following elements: global state space, joint observation space, joint action space, state transition probability function, reward function, and discount factor γ. At each reconstruction step, the improved contract network protocol dynamically assigns each agent to one of two roles based on the perceived failure event: Tenderer (i∈N) who detects the failure and executes discrete decisions. ten ), or Bidder (j∈N) who generates continuous bids based on internal state. bid ); (1) Status and Role-Specific Observation Space: global state s t The functional requirements, encompassing dynamic graph topology, electromagnetic interference field distribution, and failure loops, are defined as follows: Execution phases t It is invisible to decentralized agents, but provides a key informational basis for heavy critics to build a global value baseline during the training phase; The local observation vector of bidder node j is constructed as follows: Where, d bid This represents the dimension of the bidder's local observation vector. Aggregate the remaining resource status c j(t ), relative link quality with the bidding party And local topological embedding; the local observation vector of the bidding party node i is extended based on the bidding party's observation and is defined as: Where, d ten This represents the dimension of the local observation vector of the bidding party, and the local observation of the bidding party's node. Includes target tactical value And the specific resource requirements of the tasks to be processed; (2) Hybrid Layered Motion Space: Hybrid Layered Motion Space Structurally, the decision-making process is decoupled into "capability projection" and "optimal selection" stages; in the bidder flow, strategy π bid Map local observations to continuous action vectors u j,t ∈[0,1] 3 To ensure the boundedness of resource commitments and promote stable gradient flow, the bidding strategy is parameterized using a Beta distribution: Where α θ and β θ This represents the concentration parameter of the bidder network output; this vector potentially encodes three key dimensions: willingness to participate α. j Resource allocation β j and tactical cost γ j ; (3) Cognitive gating mechanism: In order to connect these continuous signals with discrete protocol logic, a cognitive gating mechanism is introduced to generate dynamic intention masks M∈{0,1}. N ,in Gating threshold δ th During training, the linear annealing strategy is used to transition from 0.1 to 0.5, first broadly exploring the candidate pool and then gradually focusing on high-confidence bidders. This mechanism filters out unwilling / incapable nodes at the source, pruning the originally NP-hard search space into a manageable subset, thereby ensuring millisecond-level response. Step 3: Design a multi-objective composite reward function, including closed-loop efficiency. The main item is performance recovery reward, and the remaining resources are Shannon entropy. Resource balancing reward with total closed-loop delay T total The punishment is time-sensitive, while the reward is effective in fulfilling hard constraints. The linear weighted coupling of the four components is specifically achieved through the following method: To guide decentralized agents toward the Pareto optimal frontier that balances task performance, load distribution, and responsiveness, a scalar global reward function r consisting of four weighted objectives is proposed. t (s, a): Where ξ, λ1, and λ2 are the weight coefficients of the efficiency recovery term, resource entropy term, and timeliness penalty term, respectively; main part The goal is to restore the closed-loop efficiency of the new operating loop after incentive reconstruction; this objective is determined by the resource entropy term. The balancing term maximizes the Shannon entropy of the residual resource distribution of the winning node to prevent specific channel exhaustion and promote long-term load balancing; furthermore, the timeliness penalty term minimizes the total closed-loop delay T relative to the target exposure window. total The selection process prioritizes nodes that are topologically adjacent and have superior computational performance, with a fixed effectiveness reward. This ensures that the hard physical constraints are met; Step 4: Construct the Endogenous Intelligence Hybrid Architecture (EIHA), which consists of a "planning-evaluation layer" and a "response layer." The planning-evaluation layer acts as a centralized commentator during offline training and is deprecated after deployment. The response layer consists of edge nodes equipped with endogenous intelligence modules. A heterogeneous interactive manifold multi-agent proximal policy optimization algorithm, HIM-MAPPO, is proposed. This algorithm combines a "heavy commentator" network with joint information fusion and a heterogeneous dual-stream Actor network. The bidding Actor outputs parameterized Beta-distributed continuous actions, while the tendering Actor outputs Categorical-distributed actions renormalized by dynamic intent masks. The policy network is trained using a "centralized training, distributed execution" approach, embedding global tactical intelligence into the policy parameters of each node in the response layer. The specific method is as follows: EIHA structurally decouples the command hierarchy into two coordination layers: (1) Planning-Evaluation Layer: Residing in the command center, it undertakes two types of lifecycle functions; before the battle, it analyzes the initial overall situation. The optimal initial running loop is generated and the kill net topology is initialized using a global optimization method. During the offline training phase, it is transformed into a centralized training of a central commentator in the distributed execution CTDE paradigm, guiding distributed edge nodes to learn collaborative strategies aligned with the global model. This layer is discarded during the deployment and execution phase. (2) Response layer: It consists of heterogeneous combat nodes equipped with lightweight "intrinsic intelligence modules". The nodes interact through "event-triggered negotiation" in a closed loop, which is divided into three stages: reconstructed bidding, intelligent bidding and winning decision. The entire closed-loop decision-making process is completed in milliseconds. The HIM-MAPPO algorithm implements the offline centralized training process of EIHA and consists of the following three types of networks: (1) Joint Information Fusion Heavy Commentator Network V ψ As an omniscient supervisor, the global state s t The joint observations are spliced ​​together to form a fused situation vector z. t Defined as: With t =[s t ,about 1,t ,...,about N,t ] (17) z t After processing by a multilayer perceptron with hidden layer dimensions of 1024-512-256, the output is a global value estimate V. ψ (z t The return estimated by commentators using the generalized dominance estimation method. To minimize the mean squared error loss: Where B represents the mini-batch size; this architectural design ensures that the critic can provide precise value guidance for the distributed executor by capturing the impact of local refactoring actions on the entire system; the critic's asymmetric structure propagates global topological dependencies back to the decentralized Actor through global-local knowledge distillation, internalizing the inter-role collaboration relationships that are invisible to local nodes during execution into "endogenous intuition", thereby structurally solving the credit allocation problem under coupled rewards; (2) Bidder Actor Network Employing a multilayer perceptron, with input dimension d b →512→256→256, Local observation of consumer bidders Output Beta distribution concentration parameters Then sample the three-dimensional continuous bidding action vector u j,t ∈[0,1] 3 Sample according to the above-described Beta distribution parameterization method; (3) Tendering party Actor Network Employing a multilayer perceptron, the concatenated vector of consumer bidding observations and bidding actions is used. Among them B t The concatenation of all Bidder bidding actions outputs the Categorical logits, which are then masked and normalized by M. This is used to sample the discrete selection action a. i,t : Where ε is the numerical stability constant to prevent the denominator from being zero; the parameter set θ of the two Actor networks b With θ t Strictly non-overlapping and independently optimized to avoid gradient interference caused by sharing the underlying backbone; although parameters are decoupled, they share the same task-level reward r. t And the estimates by heavy critics As a unified optimization signal; The training employs a "centralized training, distributed execution" approach; each episode uses K loops. tr Round, T per round tr Step, each update J ppo There are 1 training round; in each step, all Bidders first generate bidding actions and construct an intent mask M. t Then, the Tenderer samples discretely selected actions and interacts with the environment to obtain rewards r. t It is then stored in the experience buffer, and an update is triggered every few steps. During the deployment phase, heavy commentators are completely discarded, and each edge node retains only the corresponding Bidder Actor or Tenderer Actor for real-time decision-making. The "global tactical intelligence" obtained during the training phase is solidified in the network weights in the form of policy parameters, realizing offline centralized training and online distributed execution. During deployment, the inherent intelligence contained in the parameters completes millisecond-level dynamic reconstruction.