Fast convergence networked anti-jamming decision method and system for wireless ad hoc networks

Abstract

The application discloses a fast convergence networked anti-interference decision method and system of a wireless self-organizing network, and an implementation scheme thereof comprises the following steps: constructing a key networking state information set; setting multiple task scenarios, collecting interaction trajectories and dividing the interaction trajectories into a support set and a query set; constructing a meta-learning network model comprising a context inference network, a strategy network and a value network, taking a weighted sum of an expected cumulative return and a KL divergence penalty term as an optimization target, and obtaining a scene potential variable by decoupling a public strategy and scene-specific characteristics from the support set trajectory; inputting the potential variable and the key networking state information into the trained meta-learning network model in combination, and outputting a cross-layer joint anti-interference decision action comprising physical layer waveform selection, working frequency point selection and network layer routing selection. The application can effectively reduce state interaction and calculation overhead, improve the anti-interference capability of the wireless self-organizing network, and be used for combined strategy selection of physical layer waveforms and network layer routing protocols in the wireless self-organizing network.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, specifically relating to a networked anti-interference decision-making method and system, which can be used for the selection of a combination strategy of physical layer waveforms and network layer routing protocols in wireless ad hoc networks. Background Technology

[0002] Mobile ad hoc networks (MANETs) play a crucial role in complex electromagnetic warfare environments due to their decentralized and self-organizing characteristics. However, with the development of cognitive electronic warfare technology, jamming patterns are becoming increasingly diverse and their intensity is dynamically variable, posing a severe challenge to MANETs from malicious interference.

[0003] To address the challenges of network transmission under dynamic interference environments, researchers have proposed a routing adaptation method based on multi-metric fusion and rule matching. In the paper "Research on a Novel Routing Switching Strategy," the authors designed a routing switching mechanism based on a weighted evaluation function, comprehensively considering multiple state factors such as node remaining energy, link stability, and signal strength. This mechanism can select a better transmission path to a certain extent based on network conditions. However, such methods based on preset rules or weights have significant limitations: the setting of weight factors often relies on expert experience, making it difficult to adapt to unknown electromagnetic environment changes; and the rule base is usually static, failing to achieve flexible and real-time responses to the ever-changing interference strategies of intelligent jammers, resulting in decision-making lag.

[0004] To address the shortcomings of rule-based methods, intelligent routing and network adaptation techniques based on machine learning, especially reinforcement learning (RL), have become a research hotspot in recent years. Studies such as "Software-defined joint routing and waveform selection for cognitive Ad Hoc networks" explore using cognitive techniques to perceive spectrum and network state, and dynamically adjust routing and waveform parameters through learning algorithms. These methods attempt to learn optimal strategies through trial and error with the environment, exhibiting stronger environmental adaptability compared to traditional methods. However, existing network adaptation methods based on standard reinforcement learning, such as Q-Learning and DQN, face significant challenges in practical applications: they typically assume that the interference environment is stable or slowly changing. Once the interference pattern changes abruptly, such as from single-tone interference to frequency sweeping interference, the model often needs to undergo a lengthy retraining process to converge, causing a severe deterioration in network communication quality during this period.

[0005] Furthermore, in the research on complex network decision-making based on deep reinforcement learning (DRL), although the powerful feature extraction capabilities of deep neural networks have solved the problem of state space explosion, most existing studies are based on the ideal assumption of complete state information. In actual battlefield environments with strong interference, the control signaling interaction between nodes is hindered, resulting in nodes only being able to obtain local, incomplete observation information. Existing distributed cooperative anti-interference algorithms in the literature often ignore the decision bias caused by this information incompleteness and lack cross-task generalization ability. They are unable to extract general anti-interference features from historical interference scenarios, resulting in the inability to achieve rapid adaptation under small sample sizes when facing new interference scenarios. This severely restricts the survivability and transmission efficiency of mobile ad hoc networks in actual combat and reduces the anti-interference performance of the algorithms.

[0006] Patent document CN117998419.A discloses a "multi-domain communication anti-interference method, system, and medium based on width reinforcement learning." This method models the multi-domain communication anti-interference problem as a Markov decision process, constructs a width policy network and a width target network, calculates the environmental state and reward of the transmission action in the previous time slot, obtains a complete set of experiences, and stores them in an experience pool. Based on samples in the experience pool, it calculates the pseudo-inverse of the output matrix of the input layer of the width policy network and the output matrix of the width target network, respectively, and constructs a target matrix by combining the calculated rewards. The weights of the width policy network are updated using the pseudo-inverse and the target matrix, and the updated width policy network is used to select the transmission action. This method continuously optimizes the width policy network through constant interaction with the interference environment, enabling the network output to approach the optimal transmission strategy with lower computational complexity and faster convergence speed. However, this method does not perform feature decoupling modeling for the inherent differences in interference scenarios, making it difficult to achieve fast convergence and robust decision-making under conditions of few samples in dynamic interference environments. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by proposing a fast convergence networked anti-interference decision-making method and system for wireless ad hoc networks, so as to improve the convergence speed and anti-interference capability of networks in complex electromagnetic environments.

[0008] The technical approach of this invention is as follows: First, it rapidly perceives current scene characteristics through latent variable inference. Then, by introducing game theory and information theory methods in the state information preprocessing stage, it filters key network state information, eliminates redundant noise, and achieves dimensionality reduction of the high-dimensional state space and locking in effective perception information. Second, it constructs a meta-reinforcement learning PEARL architecture based on probabilistic context embedding, using an inference network to infer latent variables representing the current scene from historical interaction trajectories, achieving scene feature extraction under unknown interference environments. Third, it introduces the variational information bottleneck (VIB) mechanism with KL divergence constraints during training, forcing common features to sink to policy network parameters while retaining unique features in latent variables. This enables rapid policy transfer in small-sample scenarios through the common policy and unique features of the anti-interference policy. Finally, it combines latent variables and the current state through the policy network to jointly output waveforms, frequency points, and routing actions, achieving cross-protocol layer resource collaborative optimization.

[0009] Based on the above approach, the implementation scheme of the present invention includes:

[0010] 1. A fast convergence networked anti-interference decision-making method for wireless ad hoc networks, characterized by comprising:

[0011] (1) Construct a multi-dimensional observable state information set that includes interference factors, network node factors, channel environment factors and service factors, and evaluate each state information to obtain a key network state information set;

[0012] (2) Different interference modes and network configuration combinations are modeled as different task scenarios. Interaction trajectory data containing status, action and reward are collected in each task scenario and divided into support set and query set.

[0013] (3) Construct a meta-learning network model that includes a context inference network, a policy network and a value network, and take the weighted sum of the expected cumulative reward and the KL divergence penalty term as the optimization objective of the model. Decouple the common policy and the unique features from the historical interaction trajectory of the support set and infer the unique latent variables of the current scene.

[0014] (4) Train the meta-learning network model. When a node enters an unknown interference environment, it accumulates context data through a small amount of interaction. The key networking status information and the latent variables are jointly input into the trained meta-learning network model, and the output includes cross-layer joint anti-interference decision actions including physical layer waveform selection, working frequency selection and network layer routing selection.

[0015] Furthermore, the construction of a multidimensional observable state information set including interference factors, network node factors, channel environment factors, and service factors in (1) is based on the analytic hierarchy process (AHP), and its implementation includes:

[0016] Establish a set of interference factors that include interference type, interference frequency band, interference bandwidth, interference power, and interference-to-signal ratio;

[0017] Establish a set of network node factors that include the number of neighboring nodes, node speed, frame error rate, bit error rate, neighbor link change rate, MAC layer load, and geographical location information;

[0018] Establish a set of channel environment factors that include channel type and noise level;

[0019] Establish a set of business factors that include node packet sending rate;

[0020] All state variables under the above four dimensions are summarized and integrated to form a multidimensional observable state information set containing 15 observable state variables, which serves as the basic candidate set for subsequent key state information screening.

[0021] Furthermore, in step (1), the evaluation of each state information in the multidimensional observable state information set yields a key network state information set, which is implemented by:

[0022] 1a) The Shapley value in cooperative game theory is used to quantify the marginal contribution of each state variable in the observable state information set to the decision against interference, and the state information is sorted and filtered according to the contribution to obtain a subset of decision contribution.

[0023] 1b) Based on the mutual information (MI) analysis in information theory, observe the information gain of each state in the state information set with network performance indicators, and select a subset of state information that has a significant gain on network throughput or delivery rate according to the mutual information size.

[0024] 1c) Calculate the mutual information between features within the union of Shapley values ​​and mutual information filtering results, remove redundant information with high correlation, and filter out a set of key network status information including interference frequency band, interference type, interference-to-signal ratio, MAC layer load, and neighbor node change rate, so as to reduce the input dimension from 15 dimensions to 6 dimensions.

[0025] Furthermore, the meta-learning network model constructed in (3) that includes a context inference network, a policy network, and a value network is implemented as follows:

[0026] 3a) Construct a context inference network by cascading a feature extraction layer, an average pooling layer, and an output layer;

[0027] 3b) Establish the input vector, the common backbone network, and the branch output network, and connect them sequentially to form the policy network;

[0028] 3c) Establish a value network consisting of a state value network and a dual-action value network connected in parallel;

[0029] 3d) After each fully connected layer of the context inference network, policy network and value network, a nonlinear activation function is connected, and then the context inference network, policy network and value network are cascaded to form a meta-learning network model for cross-scene anti-interference decision-making.

[0030] Furthermore, in step (3), the meta-learning network model is used to optimize the weighted sum of the expected cumulative reward and the KL divergence penalty term. This decouples the common policy from the specific features in the historical interaction trajectories of the support set, and infers the specific latent variables of the current scenario. The implementation includes:

[0031] 3e) Define a single-step reward function based on normalized throughput, normalized delivery rate, and waveform switching loss. :

[0032] 3f) Based on the single-step reward function Define cumulative reward function :

[0033] 3G) from supporting sets Trajectory information obtained by mid-sampling , track information The scene obtained by inputting into the context inference network latent variables Based on the latent variable z, the KL divergence constraint term is defined as follows:

[0034] 3h) Define the optimization objective function based on the cumulative reward function and the KL divergence constraint term.

[0035] 3i) Based on the optimization objective function, decouple the common policy from the unique features, and infer the unique latent variables of the scenario:

[0036] 3i1) Based on the constraints of the optimization objective function, common features with consistent gradient directions are stably internalized into the policy network parameters. To form a public strategy foundation ,in For decision-making actions, This is the current critical status information;

[0037] 3i2) Due to the different gradients with unique characteristics of different tasks, they cannot be determined by the policy network parameters. The absorbed properties retain these unique characteristics in the latent variables. The information in the middle refers to the unique features of the current scene.

[0038] 2. A fast-converging networked anti-interference decision-making system for wireless ad hoc networks, characterized in that it comprises:

[0039] The key status information filtering module is used to output a set of key network status information.

[0040] The scenario building module is used to combine different interference modes and network configurations to model task scenarios.

[0041] The data acquisition module is used to collect state-action-reward trajectory data in various scenarios, divide it into support sets and query sets according to preset ratios, and store them.

[0042] The scene feature inference module is used to extract aggregated features from the historical trajectories of the support set, output the Gaussian distribution parameters of the scene-specific features, and then obtain the specific feature variables that characterize the features of the interference scene through reparameterized sampling.

[0043] The online adaptive decision-making module is used to output cross-layer joint anti-interference actions and issue them for execution.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] Firstly, this invention obtains a set of highly ranked state information by using Shapley values ​​from game theory and MI values ​​from information theory. Redundant features are eliminated, which can effectively extract efficient key networking state information that is strongly related to decision-making and performance, reduce the communication overhead of full state interaction, and at the same time ensure the effectiveness and accuracy of anti-interference decision-making.

[0046] Secondly, this invention achieves feature decoupling through the variational information bottleneck (VIB) mechanism, which solidifies the anti-interference logic that is universal across scenarios into the network parameters. In new scenarios, only a small number of samples need to be collected to infer the latent variables of that scenario without adjusting the network weights. This reduces the adaptation time from thousands of iterations to a few time slots, enabling rapid adaptation to small samples in dynamic interference scenarios. It also reduces the convergence time required for traditional algorithms to adapt to new scenarios and ensures the network's anti-interference capability in complex interference environments. Attached Figure Description

[0047] Figure 1 is a flowchart of the fast convergence networked anti-interference decision-making method for wireless self-organizing networks of the present invention;

[0048] Figure 2 is a ranking chart of decision contribution evaluation based on game theory in the method of the present invention;

[0049] Figure 3 is a ranking diagram of performance gain evaluation based on information theory in the method of this invention;

[0050] Figure 4 is a heatmap based on information redundancy analysis in the method of the present invention;

[0051] Figure 5 is a block diagram of the fast convergence networked anti-interference decision system of the wireless self-organizing network of the present invention;

[0052] Figure 6 is a comparison of delivery rate and throughput performance using the present invention and existing rule-driven methods to make decisions on waveform, frequency point and routing protocol respectively;

[0053] Figure 7 is a comparison of delivery rate, throughput, and reward value when using the present invention and the existing soft actor-commentator SAC decision method to make decisions on waveform, frequency point, and routing protocol. Detailed Implementation

[0054] The embodiments of the present invention will now be described in full and in detail with reference to the accompanying drawings.

[0055] Example 1: A Fast-Converging Networked Anti-Interference Decision-Making Method for Wireless Ad Hoc Networks

[0056] Reference Figure 1 The implementation steps of this example include:

[0057] Step 1: Filter the status information of key network segments.

[0058] This step aims to filter out a subset of key state information from the large amount of observable state information in mobile ad hoc networks. This subset has a high contribution to cross-layer anti-interference decision-making and a significant effect on network performance gains, thereby reducing computational complexity and improving decision-making accuracy. The implementation is as follows:

[0059] 1.1) Construct a set of candidate state information:

[0060] Based on the Analytic Hierarchy Process (AHP), a candidate state information set is established from four dimensions: interference factors, network node factors, channel environment factors, and service factors. It contains 15 observable state variables, namely: interference type, interference frequency band, interference bandwidth, interference power, interference-to-signal ratio, number of neighboring nodes, node speed, frame error rate, bit error rate, neighbor link change rate, MAC layer load, location information, channel type, noise level, and node packet transmission rate.

[0061] 1.2) Evaluating the contribution of game theory to decision-making:

[0062] The Shapley value, a mathematical method used in cooperative game theory for benefit allocation, is employed to evaluate the marginal contribution of each state variable to the prediction results of the anti-interference decision model.

[0063] 1.2.1) Set the candidate state information The first in The marginal contribution of each feature to the decision outcome is defined as follows: :

[0064] ,

[0065] in, For feature subset, Indicates the use of feature subsets The decision value calculated by the decision model when input. To use feature subsets and the Each feature is the decision value calculated by the decision model when input;

[0066] 1.2.2) The marginal contribution of each state information is estimated using the marginal contribution formula. The 15 candidate features are then sorted in descending order of contribution value to obtain the ranking of the contribution of each state information to the three types of decisions: waveform, frequency point, and routing, as shown in Figure 2. Among them, Figure 2 (a) is a ranking diagram of key state information for waveform decision-making; Figure 2 (b) is a sorting diagram of key state information for frequency point decision-making; Figure 2 (c) is a sorting diagram of key state information for routing protocol decision-making.

[0067] from Figure 2 It is evident that, for waveform decision-making, the network state information contributing significantly is interference bandwidth, interference frequency band, interference-to-input ratio (CIRR), frame error rate, bit error rate (BER), and interference type; for frequency point decision-making, the network state information contributing significantly is, in descending order, interference frequency band, interference type, CIRR, and interference bandwidth; and for routing decision-making, the network state information contributing significantly is, in descending order, frame error rate, MAC load, BER, CIRR, neighbor link change rate, and node packet transmission rate.

[0068] 1.2.3) Take the intersection of the state information that contributes highly to the three types of decisions to obtain the set of key network state information, which includes interference bandwidth, interference frequency band, interference-to-signal ratio, interference type, frame error rate, bit error rate, MAC load, node packet sending rate and neighbor node change rate.

[0069] 1.3) Evaluating network performance gains based on information theory:

[0070] This step uses mutual information from information theory to measure the correlation between state information and network performance, and to assess the importance of network state information:

[0071] 1.3.1) The first The status information and the first The mutual information of a network performance metric is defined as follows:

[0072] ,

[0073] in, For joint probability distribution, For the first The marginal probability distribution of each state information, and For the first Marginal probability distribution of network performance metrics A collection of network performance metrics;

[0074] 1.3.2) The mutual information gain of each state information was estimated using the mutual information formula. The mutual information gain values ​​of the 15 candidate features were sorted in descending order, and the results are shown in Figure 3. Figure 3 It is evident that, for network performance, the network state information with higher information gain includes MAC load, frame error rate, neighbor link change rate, interference-to-signal ratio, bit error rate, node packet transmission rate, and moving speed.

[0075] 1.4) Redundancy removal is performed on the set of candidate state information to obtain the set of key state information. :

[0076] 1.4.1) Take the union of the contribution set and the information gain set to obtain the candidate state information set, which includes interference bandwidth, interference frequency band, interference-to-information ratio, interference type, frame error rate, bit error rate, MAC load, node packet transmission rate and neighbor node change rate.

[0077] 1.4.2) Calculate the first mutual information value in the candidate state information set using the above mutual information formula. The status information and the first The mutual information of each state is plotted in a heatmap, as shown in Figure 4.

[0078] from Figure 4 It is evident that there is high redundancy between interference frequency bands and interference bandwidth, high redundancy between MAC layer load and node packet transmission rate, and high redundancy between frame error rate and bit error rate. Therefore, by eliminating redundant information such as interference bandwidth, node packet transmission rate, and bit error rate from the candidate state set, the final set of key network state information can be obtained. The interference frequency band, interference type, interference-to-input ratio, MAC layer load, neighbor node change rate, and frame error rate are all considered.

[0079] Step two involves modeling the task scenario, collecting sample data, and dividing the dataset.

[0080] To provide training and testing datasets for meta-learning algorithms, task scenario modeling is required. Simultaneously, sample data needs to be collected for each task scenario, and the dataset needs to be partitioned. The implementation is as follows:

[0081] 2.1) Perform task scenario modeling:

[0082] 2.1.1) Perform interference configuration, which includes three types: interference type configuration, interference power configuration, and interference frequency band configuration.

[0083] Interference type configuration, including: single-tone interference, frequency sweeping interference, jamming interference, and intelligent tracking interference.

[0084] Interference power configuration is based on the interference-to-signal ratio. The interval is carried out.

[0085] Interference frequency band configuration refers to the setting of interference frequency bands that cover a portion of the frequency points or the entire frequency band.

[0086] 2.1.2) Configure the network, which includes... Configure the number of nodes in the interval, to Configure the node movement speed within the interval, as well as the parameters for network topology density;

[0087] 2.1.3) Combining the different interference configurations and network configurations described above yields a set of meta-learning tasks. :

[0088] ,

[0089] Where M is the number of scene tasks, This is one type of task scenario.

[0090] 2.2) Collect sample data and divide the dataset:

[0091] 2.2.1) Run multiple stages through a container simulation platform, collecting quadruple trajectory data to form each scenario. The dataset contains L=100~200 time slots in each stage, and each time slot collects quadruple trajectories. :

[0092] ,

[0093] in, This is a set of key network status information at this moment. This is the action vector at this moment. For waveform selection, Frequency selection, For routing protocol selection, The reward is based on a combination of throughput and delivery rate. This is the set of key network states for the next moment;

[0094] 2.2.2) Each scenario The collected sample data were all divided into support sets. and query set Two parts, where the support set is used to infer the latent features z of the task, containing Trajectories; query sets are used for policy optimization and performance evaluation during meta-training, containing A trajectory.

[0095] Step 3: Construct a meta-learning network model.

[0096] 3.1) Establishing a context inference network This is used to extract the latent feature vector z of the task from the support set trajectory, thereby achieving an implicit representation of the current disturbance scene:

[0097] 3.1.1) The number of neurons selected is The fully connected network is used as the feature extraction layer It is used for parallel feature encoding of single-step interactive samples to obtain the corresponding intermediate feature representation. ,in For the trajectory of the quadruplet The dimension;

[0098] 3.1.2) An average function is selected as the average pooling layer to represent intermediate features. Aggregation processing is performed to obtain aggregated features that are independent of the sample order. :

[0099] ,

[0100] in, Indicates the number of context trajectories. For trajectory labels;

[0101] 3.1.3) The number of neurons selected is The fully connected network as a parameterized output layer This is used to output the mean and variance parameters of the latent variables, thus forming a Gaussian distribution representation of the latent variables, where... The dimensions of the mean and variance parameters;

[0102] 3.1.4) Concatenate the feature extraction layer, average pooling layer, and output layer to form a context inference network;

[0103] 3.2) Establish a policy network:

[0104] 3.2.1) Transfer the current status information Latent variables output by the context inference network Concatenate the vectors to obtain the input vector;

[0105] 3.2.2) The number of neurons selected is A fully connected network as a public backbone network This is used to extract fused features from the input vector;

[0106] 3.2.3) The number of neurons selected is Fully connected networks and The normalization function serves as the waveform decision branch output network, used to output the waveform selection probability;

[0107] 3.2.4) The number of neurons selected is Fully connected networks and The normalization function serves as the output network for the frequency point decision branch, used to output the frequency point selection probability;

[0108] 3.2.5) The number of neurons selected is Fully connected networks and The normalization function serves as the output network for the routing protocol decision branch, used to output the routing protocol selection probability.

[0109] 3.2.6) Connect the input vector, the common backbone network, and the three output networks in sequence to form the policy network;

[0110] 3.3) Establishing a value network:

[0111] 3.3.1) Transfer the current status information Latent variables output by the context inference network The concatenation is used as the input vector for the state-value network. , and current decision-making actions The concatenation is used as the input vector for the action value network;

[0112] 3.3.2) The number of neurons selected is Fully connected networks as state-value networks , used to output state value estimates;

[0113] 3.3.3) Select two parallel neurons with a number of... A fully connected network as an action value network and The smaller value of the two Q-network outputs is used as the action value estimate;

[0114] 3.3.4) Connect the state value network and the dual-action value network in parallel to form a value network;

[0115] 3.4) After each fully connected layer of the context inference network, policy network and value network, a nonlinear activation function is connected, and then the context inference network, policy network and value network are cascaded to form a meta-learning network model for cross-scenario anti-interference decision-making.

[0116] Step four: Define optimization goals and extract common strategies and unique features of the scenario.

[0117] To extract common policies and specific features of a scenario, a KL divergence regularization term needs to be introduced into the optimization objective based on the Variational Information Bottleneck (VIB) mechanism. This effectively constrains the information content of latent variables, thereby driving the model to sink common features to the policy network parameters and retain specific features in the latent variables. The implementation is as follows:

[0118] 4.1) Based on normalized throughput Normalized delivery rate and waveform switching loss Define a single-step reward function :

[0119] ,

[0120] in, To normalize the throughput weights, To normalize the delivery rate weights, This is a waveform switching penalty coefficient to suppress frequent switching. In this example, it is taken as, but not limited to, [variable values]. , , ;

[0121] 4.2) Based on the single-step reward function Define cumulative reward function :

[0122] ,

[0123] in, =0.9 is the discount factor, t is the current time step, and L is the total number of time slots. Quadruple trajectory data for each stage;

[0124] 4.3) From the support set Trajectory information obtained by mid-sampling , track information The scene obtained by inputting into the context inference network latent variables Based on the latent variable z, the KL divergence constraint term is defined as follows: ,in It follows a standard normal prior distribution;

[0125] 4.4) Based on the cumulative reward function and the KL divergence constraint, the optimization objective function is defined as follows:

[0126] ,

[0127] in, To infer network parameters from context, For policy network parameters, For value network parameters, The expected reward for generating trajectories for the policy network. The expected value of the posterior distribution of the latent variable. For the overall expectation of all task scenarios, These are the weighting coefficients for the KL divergence term;

[0128] 4.5) Extract common strategies between scenes:

[0129] 4.5.1) Based on the KL divergence term in the optimization objective... Constraints on policy parameters Update:

[0130] ,

[0131] in, For task tags, This represents the total number of scene tasks during the update process. The gradients of each task are summed by an average term. Accumulation or cancellation is performed; when multiple tasks are aligned in a gradient direction, the gradients in that direction are accumulated, and the parameters are... Large update magnitude; when multiple tasks conflict in a certain gradient direction, the gradients in that direction will be canceled out, parameters The update was minor;

[0132] 4.5.2) Extract network parameters for common features between scenes based on the above gradient accumulation process. Thus forming the foundation of public strategy ;

[0133] 4.6) Due to the different gradient directions of specific features in different scenarios, they cannot be determined by policy parameters. Absorption will preserve scene-specific residual features in the latent variables. In, that is, extract the first Specific features of each scenario .

[0134] Step 5: Train the meta-learning network model.

[0135] This step uses gradient descent to train the meta-learning network model. The training process includes multiple meta-training rounds, each containing multiple stages. The implementation of a single meta-training round is as follows:

[0136] 5.1) Initialize training parameters:

[0137] 5.1.1) Set the number of training rounds to 1000, each round to contain 10 stages, the discount factor to 0.9, the learning rate to 0.95, the initial exploration rate to 1, the exploration decay rate to 0.995, the minimum exploration rate to 0.001, and the experience replay buffer capacity to 10000.

[0138] 5.1.2) Initialize the network parameters of the context inference network, policy network, value network, and their corresponding target network in the meta-learning model;

[0139] 5.2) Using the current waveform, frequency point, and routing protocol strategy, the system interacts with the communication environment in exploration mode, collects 20 trajectories as support set data, and inputs the support set data into the context inference network to infer the potential variables characterizing the current task scenario.

[0140] 5.3) Under the condition that the latent variables remain unchanged, continue to interact with the communication environment, collect 5 trajectories as query set data, and store the query set data in the experience playback buffer;

[0141] 5.4) Randomly select sample data with a batch size of 32 from the experience replay buffer, calculate the total loss function according to the optimization objective, obtain the gradient values ​​of each network parameter by taking the derivative of the total loss function, and pass the gradient values ​​to the value network, policy network and context inference network in sequence through backpropagation, and iteratively update each network parameter using the gradient descent method.

[0142] 5.5) Repeat steps 5.2) to 5.4) until the preset number of training rounds is reached to obtain the meta-learning network model after training.

[0143] Step 6: Using the trained meta-learning network model, obtain the anti-interference waveform, frequency point, and routing protocol decision parameters.

[0144] 6.1) Communication nodes in unknown interference scenarios Within each time slot, the system uses the current waveform, frequency point, and routing protocol to perform limited interactions with the environment, collecting trajectory data for that scenario. The input is fed into the context inference network of the trained meta-learning network model to infer the feature vectors, i.e., latent variables, that characterize the current disturbance environment. ;

[0145] 6.2) Use the aforementioned feature vectors to make decisions regarding waveform, frequency point, and routing protocol:

[0146] 6.2.1) Combine the current key network status information with the feature vector. After being concatenated, the data is input into the policy network of the trained meta-learning network model, and its waveform decision branch outputs the working waveform parameters of the current scene.

[0147] 6.2.2) Combine the current key network status information with the feature vector After being concatenated, the data is input into the policy network of the trained meta-learning network model, and the working frequency parameters of the current scene are output through its frequency decision branch.

[0148] 6.2.3) Combine the current key network status information with the feature vector. After being concatenated, the data is input into the policy network of the trained meta-learning network model, which outputs the routing protocol parameters for the current scene through its routing decision branch.

[0149] The output waveform, operating frequency, and routing protocol parameters constitute the parameters for obtaining the anti-interference network performance.

[0150] Example 2: A fast-converging, networked, anti-interference decision-making system for wireless ad hoc networks

[0151] Reference Figure 5 This example system includes a key state information filtering module 1, a scene construction module 2, a data acquisition module 3, a scene feature inference module 4, and an online adaptive decision-making module 5. The scene feature inference module 4 includes: a feature extraction submodule 41, a context aggregation submodule 42, a scene feature parameter output submodule 43, and a scene feature sampling submodule 44. The online adaptive decision-making module 5 includes: a state information acquisition submodule 51, an online scene feature inference submodule 52, and a strategy decision-making submodule 53.

[0152] The working principle of the entire system is as follows:

[0153] The key state information filtering module 1 is used to collect multi-dimensional observable state information from the wireless ad hoc network, and to perform importance analysis on the state information based on the state evaluation mechanism of game theory and information theory, to filter out a set of key network state information that has a high contribution to cross-layer anti-interference decision-making and a significant benefit to network performance, and to output the set of key network state information to the data acquisition module 3.

[0154] The scenario construction module 2 is used to combine and configure the interference parameters and network operation parameters in the wireless ad hoc network to construct multiple different scenario task sets, and output the scenario task sets to the data acquisition module 3.

[0155] The data acquisition module 3 is used to collect interaction trajectory data in the scene task set, divide the interaction trajectory data and the status information set into a support set and a query set according to a set ratio, and output the support set and query set to the scene feature inference module 4.

[0156] The scene feature inference module 4 is used to extract scene feature vectors representing the characteristics of the current task scene from historical trajectory data in the support set. Specifically: the feature extraction submodule 41 is used to perform feature encoding on the interaction trajectory data in the support set, extract intermediate feature representations, and output the intermediate features to the context aggregation submodule 42; the context aggregation submodule 42 is used to aggregate the intermediate feature representations to obtain global context features, and output the global context features to the scene feature parameter output submodule 43; the scene feature parameter output submodule 43 outputs distribution parameters corresponding to the scene feature vector based on the global context features and transmits them to the scene feature sampling submodule 44; the scene feature sampling submodule 44 is used to generate scene feature vectors representing the characteristics of the current task scene based on the distribution parameters, and output the scene feature vectors to the online adaptive decision module 5.

[0157] The online adaptive decision-making module 5 is used to achieve rapid adaptation in unknown interference scenarios and output waveforms, frequency points, and routing protocol parameters. Specifically: the status information acquisition submodule 51 is used to collect key network status information at the current moment and output the status information to the policy decision-making submodule 53; the online scenario feature inference submodule 52 is used to collect a small amount of online interaction trajectory data after a communication node enters an unknown interference scenario, generate a scenario feature vector corresponding to the current scenario, and output the feature vector to the policy decision-making submodule 53; the policy decision-making submodule 53 is used to concatenate the status information and the feature vector, input them into the policy network, and obtain physical layer working waveform parameters, working frequency point parameters, and routing protocol parameters, thereby realizing cross-layer joint anti-interference decision-making.

[0158] It should be noted that the above functional modules can be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, as a program instruction product. A program instruction product includes one or a set of program instructions. When the program instructions are loaded and executed on a computer, the described process or function is generated, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The program instructions can be stored in a computer-readable and writable storage medium, or transferred from one computer's readable and writable storage medium to another.

[0159] In this embodiment, the direct coupling or communication connection between the modules can be achieved through indirect coupling or communication connection via interfaces, devices, or modules. The functional modules and sub-modules in this embodiment can dynamically reside within a single processing unit, or each module can exist physically independently, or two or more modules can dynamically reside within a single processing unit. When these dynamic components are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable and writable storage medium. This storage medium can be a memory, disk, or optical disc, etc.

[0160] The effects of this invention can be further illustrated by the following simulation experiments:

[0161] I. Simulation Conditions

[0162] In this embodiment, the simulation communication platform is a container platform, the meta-learning architecture is built using Python, and the communication platform and the deep network exchange data through files.

[0163] II. Simulation Content

[0164] Simulation 1: Under the above conditions, the working waveform, frequency point, and routing protocol under complex interference environments are determined using both the present invention and existing rule-driven methods. Network delivery rate, throughput, and reward value are statistically analyzed, and the results are shown in Figure 6. Wherein:

[0165] Figure 6 (a) A comparison chart of delivery rate changes using the present invention and existing rule-driven methods as the interference scenario changes;

[0166] Figure 6 (b) A comparison graph showing the throughput changes of the present invention and existing rule-driven methods as the interference scenario changes;

[0167] Simulation results show that the method of the present invention improves the delivery rate by about 7.6% and the throughput by about 80% compared with the rule-driven method, demonstrating that the present invention has better anti-interference ability.

[0168] Simulation 2 uses both the present invention and the existing soft actor-critic SAC decision-making method to make decisions on the operating waveform, frequency point, and routing protocol under complex interference environments. Network delivery rate, throughput, and reward value are statistically analyzed. The results are as follows: Figure 7 .in:

[0169] Figure 7 (a) A comparison of delivery rate changes over time using the present invention and the existing soft actor-critic SAC decision method;

[0170] Figure 7 (b) A comparison graph of throughput changes over time using the present invention and existing soft actor-critic SAC decision-making methods;

[0171] Figure 7 (c) A comparison graph of reward value changes over time using the present invention and the existing soft actor-critic SAC decision-making method;

[0172] from Figure 7 As can be seen, under unknown new interference scenarios, the present invention improves the delivery rate by about 8.3%, the throughput by about 117%, and the reward rises faster compared to the existing soft actor-critic SAC decision method, demonstrating that the present invention has a faster convergence speed.

[0173] The simulation results show that the method of the present invention has better anti-interference ability in complex interference scenarios, and can effectively improve the convergence speed of the algorithm by utilizing the unique features of the scenario.

[0174] The above descriptions are merely two specific examples of the present invention and do not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and detail without departing from the principles and structure of the present invention. For example, step one and the key state information filtering module can adopt a state filtering method based on joint evaluation of Shapley value and mutual information, or feature importance analysis, correlation analysis, statistical learning methods, or deep learning feature selection methods; step three and the scene feature inference module can use recurrent neural networks, multi-head attention networks, or other deep learning structures to achieve context inference, or improve feature representation capabilities by adjusting the dimensions of latent variables or network structure, and are not limited to the fully connected network architecture described in this embodiment; step four and the optimization target construction module can use the soft actor-commentator algorithm, the proximal policy optimization algorithm, the deep deterministic policy gradient algorithm, or other reinforcement learning optimization methods to achieve policy training, or adjust KL The divergence weight coefficient optimizes the feature decoupling effect, but is not limited to the specific algorithm form described in this embodiment; step seven and the cross-layer joint anti-interference decision module can be extended to power control, channel coding method, adaptive modulation method or other communication parameter optimization decisions based on the physical layer waveform and frequency point selection and network layer routing protocol decision, and are not limited to the specific decision dimensions described in this embodiment. However, these modifications and changes based on the ideas of this invention are still within the scope of protection of the claims of this invention.

Claims

1. A method for fast converging networked interference mitigation decision in a wireless ad hoc network, characterized in that, include: (1) Construct a multi-dimensional observable state information set that includes interference factors, network node factors, channel environment factors and service factors, and evaluate each state information to obtain a key network state information set; (2) Different interference modes and network configuration combinations are modeled as different task scenarios. Interaction trajectory data containing status, action and reward are collected in each task scenario and divided into support set and query set. (3) Construct a meta-learning network model that includes a context inference network, a policy network and a value network, and take the weighted sum of the expected cumulative reward and the KL divergence penalty term as the optimization objective of the model. Decouple the common policy and the unique features from the historical interaction trajectory of the support set and infer the unique latent variables of the current scene. (4) Train the meta-learning network model. When a node enters an unknown interference environment, it accumulates context data through a small amount of interaction. The key networking status information and the latent variables are jointly input into the trained policy network, and the output includes cross-layer joint anti-interference decision actions including physical layer waveform selection, working frequency selection and network layer routing selection.

2. The method of claim 1, wherein, The construction of a multidimensional observable state information set including interference factors, network node factors, channel environment factors, and service factors in (1) is based on the analytic hierarchy process (AHP), and its implementation includes: Establish a set of interference factors that include interference type, interference frequency band, interference bandwidth, interference power, and interference-to-signal ratio; Establish a set of network node factors that include the number of neighboring nodes, node speed, frame error rate, bit error rate, neighbor link change rate, MAC layer load, and geographical location information; Establish a set of channel environment factors that include channel type and noise level; Establish a set of business factors that include node packet sending rate; All state variables under the above four dimensions are summarized and integrated to form a multidimensional observable state information set containing 15 observable state variables, which serves as the basic candidate set for subsequent key state information screening.

3. The method according to claim 1, characterized in that, The evaluation of each state information in the multidimensional observable state information set in (1) to obtain the key network state information set includes the following implementation: 1a) The Shapley value in cooperative game theory is used to quantify the marginal contribution of each state variable in the observable state information set to the decision against interference, and the state information is sorted and filtered according to the contribution to obtain a subset of decision contribution. 1b) Based on the mutual information (MI) analysis in information theory, observe the information gain of each state in the state information set with network performance indicators, and select a subset of state information that has a significant gain on network throughput or delivery rate according to the mutual information size. 1c) Calculate the mutual information between features within the union of Shapley values ​​and mutual information filtering results, remove redundant information with high correlation, and filter out a set of key network status information including interference frequency band, interference type, interference-to-signal ratio, MAC layer load, and neighbor node change rate, so as to reduce the input dimension from 15 dimensions to 6 dimensions.

4. The method according to claim 1, characterized in that, In step (2), different interference modes and network configuration combinations are modeled as different task scenarios, the implementation of which includes: 2a) Set interference configuration parameters for five types of interference, namely, no interference, single-tone interference, frequency sweeping interference, jamming interference, and intelligent tracking interference, and configure the interference power in the range of JSR=-10dB~20dB, and set the interference frequency band according to the mode of partial frequency point coverage or full frequency band coverage. 2b) Set network configuration parameters, including the number of nodes in the range of 50 to 120, the node movement speed in the range of 5 to 20 m / s, and the network topology density parameter configuration; 2c) Based on all possible combinations of the above interference and network configurations, construct the task set. Each of the tasks Each configuration corresponds to a unique combination, consisting of a specific type of interference, a specific power of interference, a specific frequency band coverage mode of interference, a specific number of nodes, a specific node movement speed, and a specific network topology density, in order to comprehensively cover various interference scenarios and network operating states.

5. The method according to claim 1, characterized in that, In step (2), the interaction trajectory data collected in each task scenario, including state, action, and reward, is divided into a support set and a query set. Its implementation includes: 2d) For each task scenario Trajectory data is collected through multiple stages of operation using a container simulation platform. Each stage contains L=100~200 time slots, and each time slot records a 6-dimensional key state vector. 3D action vector Scalar reward and the next state , forming a quadruple trajectory ; 2e) Divide the trajectory data within the L time slots of each task into a support set at a ratio of 4:

1. and query set The support set is used to infer the potential features of the task, and the query set is used for strategy optimization and performance evaluation.

6. The method according to claim 1, characterized in that, The meta-learning network model constructed in (3) includes a context inference network, a policy network, and a value network, and its implementation includes: 3a) Establish a context inference network: The trajectory data supported by the system is input into the feature extraction layer, which consists of a multi-layer fully connected network. Parallel feature encoding is performed on the single-step interaction samples to obtain the corresponding intermediate feature representation. The intermediate feature representations are aggregated by the average pooling layer to obtain a global context feature representation that is independent of the sample order, so as to characterize the overall interaction characteristics of the current task scenario. A parameterized output layer is set after the context aggregation layer to output the mean and variance parameters of the latent variables based on the global context features, thereby forming a Gaussian distribution representation of the latent variables; A context inference network is constructed by cascading the feature extraction layer, the average pooling layer, and the output layer. 3b) Establishing a policy network: The key network state information at the current moment is concatenated with the latent variables inferred from the context network output, and then... This is the input vector to the policy network; A three-layer fully connected network was selected as the common backbone network to extract the fusion features of the input vector; The fused features are input into the branch output networks of three parallel two-layer fully connected networks to output the physical layer waveform. Selection probability, operating frequency selection probability, and network layer routing selection probability; The input vector, the common backbone network, and the branch output network are connected sequentially to form the policy network; 3c) Establish a value network: The key network state information at the current moment is concatenated with the latent variables output by the context inference network to form a joint vector, which is then input into a three-layer fully connected network to construct a state value network for outputting state value estimates. State information, decision vector, and latent variables are concatenated into a joint vector and input into two identical three-layer fully connected networks to construct a dual-action value network. The smaller value of the outputs of the two Q networks is taken as the action value estimate. A value network is formed by connecting the state value network and the dual-action value network in parallel. 3d) After each fully connected layer of the context inference network, policy network and value network, a nonlinear activation function is connected, and then the context inference network, policy network and value network are cascaded to form a meta-learning network model for cross-scene anti-interference decision-making.

7. The method according to claim 1, characterized in that, In step (3), the meta-learning network model uses the weighted sum of the expected cumulative reward and the KL divergence penalty term as the optimization objective to decouple the common policy and unique features from the historical interaction trajectories of the support set, and infer the unique latent variables of the current scenario. Its implementation includes: 3e) Based on normalized throughput Normalized delivery rate and waveform switching loss Define a single-step reward function : ， in, To normalize the throughput weights, To normalize the delivery rate weights, This is a waveform switching penalty coefficient to suppress frequent switching; 3f) Based on the single-step reward function Define cumulative reward function : ， in, Here, t is the discount factor, t is the current time step, and L is the total number of time slots. Quadruple trajectory data for each stage; 3G) from supporting sets Trajectory information obtained by mid-sampling , track information The scene obtained by inputting into the context inference network latent variables Based on the latent variable z, the KL divergence constraint term is defined as follows: ,in It follows a standard normal prior distribution; 3h) Based on the cumulative reward function and the KL divergence constraint, the optimization objective function is defined as follows: ， in, To infer network parameters from context, For policy network parameters, For value network parameters, The expected reward for generating trajectories for the policy network. The expected value of the posterior distribution of the latent variable. For the overall expectation of all task scenarios, These are the weighting coefficients for the KL divergence term; 3i) Based on the optimization objective function, decouple the common policy from the unique features, and infer the unique latent variables of the scenario: 3i1) Based on the constraints of the optimization objective function, common features with consistent gradient directions are stably internalized into the policy network parameters. To form a public strategy base , where a is the decision action and s is the current key status information; 3i2) Due to the different gradients with unique characteristics of different tasks, they cannot be determined by the policy network parameters. The absorbed attributes retain these unique features in the latent variable z, which is the unique feature information of the current scene.

8. The method according to claim 1, characterized in that, The step (4) involves inputting the key network status information and the latent variables into the policy network, and outputting a cross-layer joint anti-interference decision action that includes physical layer operating waveforms, operating frequency selection, and network layer routing selection. Its implementation includes: 4a) After the communication node enters an unknown interference environment, collect key network status information of the current scenario, and at the same time collect a small amount of environmental interaction data from 5 to 10 time slots to accumulate context trajectory data. The input is fed into the contextual reasoning network to obtain the new scene feature vector. ; 4b) Combine the current scenario's key network status information with... After being concatenated, the data is input into the policy network. The physical layer decision branch in the policy network outputs the working waveform and working frequency of the physical layer, and the network layer routing protocol is output through the network layer decision branch in the policy network.

9. A fast-converging, networked, anti-interference decision-making system for wireless ad hoc networks, characterized in that, include: The key status information filtering module is used to output a set of key network status information. The scenario building module is used to combine different interference modes and network configurations to model task scenarios. The data acquisition module is used to collect state-action-reward trajectory data in various scenarios, divide it into support sets and query sets according to preset ratios, and store them. The scene feature inference module is used to extract aggregated features from the historical trajectories of the support set, output the Gaussian distribution parameters of the scene-specific features, and then obtain the specific feature variables that characterize the features of the interference scene through reparameterized sampling. The online adaptive decision-making module is used to output cross-layer joint anti-interference actions and issue them for execution.

10. The system according to claim 9, characterized in that, The latent variable inference module includes: The feature extraction submodule is used to input the supported centralized trajectory data into a multi-layer fully connected network, perform parallel feature encoding on single-step interaction samples, and obtain the corresponding intermediate feature representations; The context aggregation submodule is used to perform average pooling on the intermediate feature representation output by the feature extraction submodule to obtain a global context feature representation that is independent of the trajectory order. The scene feature parameter output submodule is used to output the mean and variance parameters of the scene features based on the global context features, and to construct a Gaussian distribution representation of the interference scene. The scene feature sampling submodule is used to generate feature variables that characterize the current scene based on the Gaussian distribution parameters through a reparameterized sampling method.

11. The system according to claim 9, characterized in that, The online adaptive decision-making module includes: The status information acquisition submodule is used to collect key network status information at the current moment after a communication node enters an unknown interference environment. The online scene feature inference submodule is used to collect a small amount of environmental interaction trajectory data within a preset number of time slots, and input the trajectory data into the scene feature inference module to generate the proprietary feature parameters corresponding to the current scene. The strategy decision submodule is used to concatenate the key networking status information with the proprietary feature parameters, input them into the strategy network, and output cross-layer joint anti-interference decision results.

Images