Anti-interference method for unmanned platform ad hoc networks based on DDPG and federated pruning

Abstract

This invention discloses an anti-interference method for unmanned platform ad hoc networks based on DDPG and federated pruning, relating to the field of communication signal processing technology. It utilizes time-frequency graphs to construct a local anti-interference decision model for unmanned platforms based on DDPG, and sends the local parameters of this model to a central node. The central node uses a federated learning mechanism to globally aggregate the local parameters of multiple unmanned platform local anti-interference decision models, obtaining a globally optimized anti-interference decision model. A federated pruning mechanism is then used to perform structured pruning on the globally optimized anti-interference decision model, resulting in a sparse decision model. The pruned sparse decision model parameters are distributed to each unmanned platform to continue training the local anti-interference decision model. This process of alternating between federated learning and federated pruning mechanisms continues until the local anti-interference decision model converges, reducing the computational complexity and communication load of reinforcement learning models in the communication network.

Description

Technical Field

[0001] This invention relates to the field of communication signal processing technology, specifically to an anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning. Background Technology

[0002] With the development of intelligent unmanned systems, unmanned platforms have shown great application potential in scenarios such as disaster response, intelligent logistics, and marine monitoring. Unmanned platforms include drones, unmanned ground vehicles, and unmanned surface vessels, which have advantages such as autonomous movement, rapid deployment, and strong environmental adaptability. In actual missions, multiple heterogeneous unmanned platforms often need to work together to build temporary communication networks to achieve mission status sharing, collaborative planning, and control. However, due to the diverse deployment environments and complex communication links, such as complex terrain obstruction, frequent electromagnetic interference, and malicious human interference, the communication reliability of unmanned platform collaborative networks faces severe challenges.

[0003] In existing technologies, deep reinforcement learning is introduced into communication anti-interference decision evaluation, which can perceive environmental changes and implement adaptive frequency hopping anti-interference strategies. Among them, the Deep Deterministic Policy Gradient (DDPG) algorithm can realize intelligent decision-making in continuous action space, and it is suitable for tasks such as continuous spectrum selection.

[0004] For example, Chinese invention patent (CN116073856A) discloses a "frequency hopping intelligent anti-interference decision-making method based on deep deterministic strategy", which specifically discloses the following: Step 1, establish a dual-variable frequency hopping communication system model; Step 2, establish an anti-interference decision-making model for the dual-variable frequency hopping communication system; Step 3, transform the optimization problem based on reinforcement learning; Step 4, anti-interference decision-making for the dual-variable frequency hopping communication system based on HDP-DDPG; Step 5, train the HDP-DDPG network and output the anti-interference decision-making model.

[0005] However, while achieving high precision and anti-interference performance, deep deterministic policy gradient algorithms also introduce a large number of network parameters and high computational load, making it difficult to adapt to the actual needs of unmanned platform nodes with limited resources. In addition, federated learning technology can enable multiple nodes to train the model collaboratively, effectively improving the model's generalization performance, but conventional federated learning does not consider the redundancy of model parameters, making it difficult to solve the problem of limited computing and communication resources. Summary of the Invention

[0006] The purpose of this invention is to provide an anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning. By combining deep reinforcement learning and federated learning techniques and introducing a structured pruning strategy for the deep reinforcement learning model, the invention achieves high-performance, low-overhead operation of the communication anti-interference decision model, reduces the computational complexity and communication load of the deep reinforcement learning model in the communication network, and improves the efficiency of anti-interference communication, thereby solving the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning, comprising the following steps:

[0008] Step 1: Collect environmental data using the unmanned platform sub-nodes and preprocess it to obtain a time-frequency diagram;

[0009] Step 2: Construct a local anti-interference decision model for the unmanned platform based on DDPG using time-frequency graphs, and send the local parameters of the local anti-interference decision model to the central node;

[0010] Step 3: The central node uses a federated learning mechanism to globally aggregate the local parameters of multiple unmanned platform local anti-interference decision models to obtain a globally optimized anti-interference decision model; and uses a federated pruning mechanism to perform structured pruning on the aggregated globally optimized anti-interference decision model to obtain a sparse decision model.

[0011] Step 4: Distribute the pruned sparse decision model parameters to each unmanned platform, continue training the local anti-interference decision model, and repeat the alternation of federated learning mechanism and federated pruning mechanism until the local anti-interference decision model converges.

[0012] Furthermore, the local anti-interference decision-making model for unmanned platforms based on DDPG in step 2 includes four neural networks: the main decision network, the main evaluation network, the target decision network, and the target evaluation network.

[0013] The neural network consists of 9 layers of 2D complex-valued convolution operations, 1 layer of flattening operation, 2 layers of fully connected operations, 1 layer of batch normalization operation, 1 layer of random deactivation operation, and 1 layer of fully connected operations connected in sequence.

[0014] The optimization objective of the master decision network is to maximize the state-action value function given by the master evaluation network. ,in, This represents the environmental state at time t. This represents the decision action at time t.

[0015] Furthermore, in step 2, the local anti-interference decision-making model for the unmanned platform based on DDPG defines the state-action value function using the following formula. :

[0016] ;

[0017] in, The electromagnetic environment status for the current time slot. This refers to the spectrum of the channel selected before this learning session. The decision action for the current time slot involves selecting a new channel based on this current state. As a discount factor, it controls the influence of future rewards. The instantaneous reward for the k-th step in the future, starting from the current time slot t, is the feedback information received after the action is taken. If there is no channel interference, it is a positive value; otherwise, it is a negative value if there is channel interference.

[0018] In addition, the policy function The parameter updates are achieved through the following policy gradient formula:

[0019] ;

[0020] In the formula, and These are the parameters for the target decision network and the target value network, respectively.

[0021] According to the above technical solution, step 3 includes the following steps:

[0022] Step 3-1: Using the central node, the local parameters of the received local anti-interference decision model are globally aggregated using the federated averaging algorithm to obtain the globally optimized anti-interference decision model and its parameters.

[0023] Furthermore, the specific aggregation formula for globally aggregating the local parameters of the local anti-interference decision model is as follows:

[0024] ;

[0025] ;

[0026] in, and These are the main decision network parameters and the main evaluation network parameters obtained from training the i-th child node, respectively. and These are the aggregated global master decision network parameters and global master evaluation network parameters, respectively, where N represents the number of unmanned platforms participating in this round of federated training. It is the local sample weight of the i-th unmanned platform. This represents the sample weight of the j-th unmanned platform, which can be dynamically adjusted based on the number of local training samples of the unmanned platform sub-nodes, the effective communication rate, or the model convergence index. It is used to reflect the weight contribution of the unmanned platform sub-nodes in the global model.

[0027] Step 3-2: Use the central node to evaluate the importance of the parameters of the global optimization anti-interference decision model and obtain the importance index;

[0028] Step 3-3: Based on the importance index in Step 3-2, set the expected compression rate, select the structural unit with the lowest score according to the proportion for global unified pruning, retain the channel set before the score, and construct the corresponding sparse mask matrix G.

[0029] Steps 3-4: Based on the generated sparse mask matrix G, structurally prune the parameters of the global optimization anti-interference decision model and construct a sparse decision model.

[0030] According to the above technical solution, in step 3-1, if the electromagnetic environment is ideal, the node quality is similar, and the sample weight can be approximated as 1, the global aggregation of the local parameters of the local anti-interference decision model can be simplified to the following formula:

[0031] ;

[0032] Among them, table Show the parameters of the local anti-interference decision model of the k-th unmanned platform. This represents the parameters of the globally optimized anti-interference decision model, where K is the number of unmanned platforms participating in the federated learning mechanism;

[0033] In steps 3-4, the parameters of the globally optimized anti-interference decision model are structurally pruned using a defined sparse mask matrix to reduce model redundancy, lower communication and inference overhead, and improve model deployment efficiency.

[0034] ;

[0035] in, This represents the parameters of the globally optimized, anti-interference decision-making model after structural pruning of the central node. The symbol represents element-wise product, and G represents the sparse mask matrix. This represents the parameters of the global optimization anti-interference decision model.

[0036] According to the above technical solution, in step 1, the environmental data collected by the unmanned platform sub-node is preprocessed to obtain discrete complex IQ signals, and the discrete complex IQ signals are transformed by time-frequency transformation using short-time Fourier transform to obtain a time-frequency diagram.

[0037] According to the above technical solution, the discrete complex IQ signal Recorded as:

[0038] ;

[0039] in, and These represent the real and imaginary parts of the signal, respectively, and h represents the complex unit;

[0040] Discrete complex IQ signal The time-frequency diagram is obtained after short-time Fourier transform. The transformation process is as follows:

[0041] ;

[0042] Where n and m represent time steps, and w(m) is a Hamming window function of length M. is the complex exponential basis function of the discrete Fourier transform.

[0043] According to the above technical solution, in step 2, the steps for constructing the local anti-interference decision model for the unmanned platform based on DDPG are as follows:

[0044] Step 2-1: Train the local robust decision-making model using the DDPG optimization structure that includes the loss of the evaluation network and the decision network;

[0045] Step 2-2: For the constructed evaluation network, train the state-action value function in a supervised manner using MSE. ,in, This represents the environmental state at time t. The decision action at time t is represented, and the network loss function is obtained. For the constructed decision network, the policy gradient method is adopted, and the objective function is maximized through gradient descent to obtain the decision network loss function. ;

[0046] Step 2-3: Evaluate the network loss function constructed based on step 2-2. and decision network loss function Gradient backpropagation and parameter updates are performed on the main evaluation network and the main decision network respectively, and finally a local anti-interference decision model is obtained.

[0047] According to the above technical solution, in step 2-2, the evaluation network adopts the minimum mean square error loss function to minimize the difference between the current Q value and the Bellman target generated by the target network;

[0048] Wherein, at time t, the target Q value is The reward is The Q-value under the next state and future decisions is The value network parameters under the next state and future decisions are: The discount factor is Therefore, the current target Q value can be expressed as:

[0049] ;

[0050] Based on the aforementioned target Q value, the loss function for evaluating the network is obtained. To minimize the mathematical expectation between the current actual Q value and the target Q value, i.e.:

[0051] ;

[0052] in, This represents the Q-value of the evaluation network under the current state and action. This represents the evaluation network parameters under the current decision. Represents the mathematical expectation. The mathematical expectation operator, This represents the training samples sampled from the experience replay pool;

[0053] The decision network employs a policy gradient method to maximize the reward brought by the action in the current state, and its loss function is the negative Q value of the action output by the decision network under the current evaluation network.

[0054] Among them, let This indicates the output action of the decision network in the current state. This represents the decision network parameters in the current state. If we evaluate the Q-value of the network in the current state for the current decision network, then the overall expected Q-value of the current decision network is... The expression is as follows:

[0055] ;

[0056] So what is the loss function of the decision network? for:

[0057] ;

[0058] in, This represents the samples actually used in the experience replay pool.

[0059] According to the above technical solution, in steps 2-3, the main evaluation network evaluates the network loss function. For the main evaluation network parameters Optimization is performed; the main decision network minimizes the decision network loss function. For the parameters of the master decision network Optimize the network, and then perform a soft update on the target network.

[0060] According to the above technical solution, the target network includes a target evaluation network and a target decision network, and the target network is soft-updated as follows:

[0061] ;

[0062] ;

[0063] in, and Let represent the parameters of the objective evaluation network and the objective decision network, respectively. This is the soft update coefficient. and These represent the parameters of the evaluation network and the decision network, respectively.

[0064] Based on the above technical solution, the state-action value function is updated based on the Bellman equation. ;

[0065] This can be expressed by the following formula:

[0066] ;

[0067] in, Indicates the learning rate. Indicates the action at time t. Indicates the discount factor. This represents the reward at time t.

[0068] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention addresses the challenges of complex environments, frequent interference, and limited communication resources in unmanned platform ad hoc networks. By combining reinforcement learning, federated learning, and other technologies, and further introducing a model structured pruning strategy, the present invention achieves high-performance, low-overhead operation of the communication anti-interference decision model, reduces the computational complexity and communication load of the reinforcement learning model in the communication network, improves anti-interference communication efficiency, and is suitable for practical deployment of unmanned platform ad hoc networks. Attached Figure Description

[0069] Figure 1 This is a flowchart of the self-organizing network anti-interference method of the present invention;

[0070] Figure 2 This is a block diagram of the decision-making and evaluation model of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Example: Figure 1 As shown, this invention provides a technical solution for an anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning, comprising the following steps:

[0073] Step 1: Collect environmental data using the unmanned platform sub-nodes and preprocess it to obtain a time-frequency diagram;

[0074] Specifically: the environmental data collected by the unmanned platform sub-nodes is preprocessed to obtain discrete complex IQ signals, and short-time Fourier transform is used to perform time-frequency transformation on the discrete complex IQ signals to obtain a time-frequency diagram.

[0075] In this embodiment, preferably, the discrete complex IQ signal Recorded as:

[0076] ;

[0077] in, and These represent the real and imaginary parts of the signal, respectively, and h represents the complex unit;

[0078] Discrete complex IQ signal The time-frequency diagram is obtained after short-time Fourier transform. The transformation process is as follows:

[0079] ;

[0080] Where n and m represent time steps, and w(m) is a Hamming window function of length M. For the complex exponential basis functions of the Discrete Fourier Transform, in practical implementations, to meet the requirements of neural networks for input real-valued tensors, [the following is used]: The real and imaginary parts serve as the two channels of the actual input tensor.

[0081] Step 2: Construct a local anti-interference decision model for the unmanned platform based on DDPG using time-frequency graphs, and send the local parameters of the local anti-interference decision model to the central node;

[0082] Specifically: For the constructed local anti-interference decision model of the unmanned platform, the state space includes the current time slot channel state and the historical channel state, and the action space is the channel selection probability within a continuous range.

[0083] like Figure 2 As shown, Figure 2 In this context, STFT stands for Short Time Fourier Transform. The local anti-interference decision model contains four neural networks: the main decision network, the main evaluation network, the target decision network, and the target evaluation network. The neural network consists of nine layers of 2D complex-valued convolution operations, one layer of flattening operation, two layers of fully connected operations, one layer of batch normalization operation, one layer of random deactivation operation, and one layer of fully connected operations that are connected in sequence.

[0084] In this embodiment, as a preferred embodiment, the steps for constructing a local anti-interference decision model for an unmanned platform based on DDPG are as follows:

[0085] Step 2-1: Train the local robust decision-making model using the DDPG optimization structure that includes the loss of the evaluation network and the decision network;

[0086] Step 2-2: For the constructed evaluation network, train the state-action value function in a supervised manner using MSE. ,in, This represents the environmental state at time t. The decision action at time t is represented, and the network loss function is obtained. For the constructed decision network, the policy gradient method is adopted, and the objective function is maximized through gradient descent to obtain the decision network loss function. ;

[0087] Specifically: In step 2-2, the evaluation network uses the minimum mean square error loss function to minimize the difference between the current Q value and the Bellman objective generated by the target network;

[0088] Wherein, at time t, the target Q value is The reward is The Q-value under the next state and future decisions is The value network parameters under the next state and future decisions are: The discount factor is Therefore, the current target Q value can be expressed as:

[0089] ;

[0090] Based on the aforementioned target Q value, the loss function for evaluating the network is obtained. To minimize the mathematical expectation between the current actual Q value and the target Q value, i.e.:

[0091] ;

[0092] in, This represents the Q-value of the evaluation network under the current state and action. This represents the evaluation network parameters under the current decision. This represents the training samples sampled from the experience replay pool;

[0093] The decision network employs a policy gradient method to maximize the reward brought by the action in the current state, and its loss function is the negative Q value of the action output by the decision network under the current evaluation network.

[0094] Among them, let This indicates the output action of the decision network in the current state. This represents the decision network parameters in the current state. If we evaluate the Q-value of the network in the current state for the current decision network, then the overall expected Q-value of the current decision network is... The expression is as follows:

[0095] ;

[0096] So what is the loss function of the decision network? for:

[0097] ;

[0098] in, This represents the samples actually used in the experience replay pool.

[0099] Among them, the state-action value function is updated based on the Bellman equation. ;

[0100] This can be expressed by the following formula:

[0101] ;

[0102] in, Indicates the learning rate. Indicates the action at time t. Indicates the discount factor. This represents the reward at time t.

[0103] Step 2-3: Evaluate the network loss function constructed based on step 2-2. and decision network loss function Gradient backpropagation and parameter updates are performed on the main evaluation network and the main decision network respectively, and finally a local anti-interference decision model is obtained.

[0104] Specifically: The main evaluation network evaluates the network loss function. For the main evaluation network parameters Optimization is performed; the optimization process is as follows:

[0105] ;

[0106] in, This represents the parameter update coefficients of the main evaluation network;

[0107] The main decision network minimizes the decision network loss function. For the parameters of the master decision network Optimization is performed; the optimization process is as follows:

[0108] ;

[0109] in, Represents the parameter update coefficients of the main decision network;

[0110] After optimization, a soft update is performed on the target network.

[0111] Furthermore: the target network includes a target evaluation network and a target decision network, and the target network is soft-updated as follows:

[0112] ;

[0113] ;

[0114] in, and Let represent the parameters of the objective evaluation network and the objective decision network, respectively. This is the soft update coefficient. and These represent the parameters of the evaluation network and the decision network, respectively; smaller values ​​indicate slower updates. This soft update mechanism effectively prevents drastic fluctuations in the target value during training, thereby improving the stability of model convergence.

[0115] Step 3: The central node uses a federated learning mechanism to globally aggregate the local parameters of multiple unmanned platform local anti-interference decision models to obtain a globally optimized anti-interference decision model; and uses a federated pruning mechanism to perform structured pruning on the aggregated global best anti-interference decision model to obtain a sparse decision model.

[0116] Specifically, in step 3-1, the central node uses the federated averaging algorithm to globally aggregate the local parameters of the received local anti-interference decision model to obtain the globally optimized anti-interference decision model and the parameters of the globally optimized anti-interference decision model.

[0117] Each unmanned platform sub-node uploads its local anti-interference decision model parameters to the central node; the uploaded content includes: main decision network parameters. and main evaluation network parameters ;

[0118] The formula for globally aggregating the local parameters of the local anti-interference decision model is as follows:

[0119] ;

[0120] ;

[0121] in, and These are the main decision network parameters and the main evaluation network parameters obtained from training the i-th child node, respectively. and These are the aggregated global master decision network parameters and global master evaluation network parameters, respectively, where N represents the number of unmanned platforms participating in this round of federated training. It is the local sample weight of the i-th unmanned platform. This represents the sample weight of the j-th unmanned platform, which can be dynamically adjusted based on the number of local training samples of the child node, the effective communication rate, or the model convergence index, and is used to reflect the weight contribution of the child node in the global model.

[0122] Step 3-2: Use the central node to evaluate the importance of the parameters of the global optimization anti-interference decision model and obtain the importance index;

[0123] In this embodiment, preferably: the central node sets global network parameters. and The importance of each network structural unit (such as convolutional channels and fully connected nodes) is evaluated. The evaluation method can be based on standards such as L1 / L2 norm, gradient sensitivity, and BN scaling factor.

[0124] Taking the master decision-making network as an example, the importance index of its first-level channel... It can be defined as:

[0125] ;

[0126] in, This represents the parameter tensor of channel c at layer l. This represents the p-norm.

[0127] Step 3-3: Based on the importance index in Step 3-2, set the expected compression rate, select the structural unit with the lowest score according to the proportion for global unified pruning, retain the channel set before the score, and construct the corresponding sparse mask matrix G.

[0128] In this embodiment, preferably, the expected compression ratio is set based on the aforementioned importance indicators. Select the structural unit with the lowest score proportionally and perform global pruning, retaining the top-scoring units. channel set And construct the corresponding pruning mask. :

[0129] ;

[0130] Steps 3-4: Based on the generated sparse mask matrix G, structurally prune the parameters of the global optimization anti-interference decision model and construct a sparse decision model.

[0131] In steps 3-4, specifically, the parameters of the globally optimized anti-interference decision model are... Structural trimming is performed using a defined sparse mask matrix:

[0132] ;

[0133] in, This represents the parameters of the globally optimized, anti-interference decision-making model after structural pruning of the central node. The symbol represents element-wise product, and G represents the sparse mask matrix.

[0134] In this embodiment, preferably, based on the generated uniform mask... The global model parameters are structurally pruned and a unified sparse model structure is constructed:

[0135] ;

[0136] ;

[0137] The central node distributes the pruned unified model. and Each unmanned platform sub-node reinitializes its local DDPG network structure and continues the policy optimization process. During this process, the pruning structure is fixed, and only the parameters are retained during training to improve convergence speed and model deployment friendliness.

[0138] Step 4: Distribute the pruned sparse decision model parameters to each unmanned platform, continue training the local anti-interference decision model, and repeat the alternation of federated learning mechanism and federated pruning mechanism until the local anti-interference decision model converges.

[0139] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for anti-interference of unmanned platform self-organizing networks based on DDPG and federated pruning, characterized in that, Includes the following steps: Step 1: Collect environmental data using the unmanned platform sub-nodes and preprocess it to obtain a time-frequency diagram; Step 2: Construct a local anti-interference decision model for the unmanned platform based on DDPG using time-frequency graphs, and send the local parameters of the local anti-interference decision model to the central node; Step 3: The central node uses a federated learning mechanism to globally aggregate the local parameters of multiple unmanned platform local anti-interference decision models to obtain a globally optimized anti-interference decision model; and uses a federated pruning mechanism to perform structured pruning on the aggregated globally optimized anti-interference decision model to obtain a sparse decision model. Step 4: Distribute the pruned sparse decision model parameters to each unmanned platform, continue training the local anti-interference decision model, and repeat the alternation of federated learning mechanism and federated pruning mechanism until the local anti-interference decision model converges. Step 3 includes the following steps: Step 3-1: Using the central node, the local parameters of the received local anti-interference decision model are globally aggregated using the federated averaging algorithm to obtain the globally optimized anti-interference decision model and its parameters. Step 3-2: Use the central node to evaluate the importance of the parameters of the global optimization anti-interference decision model and obtain the importance index; Step 3-3: Based on the importance index in Step 3-2, set the expected compression rate, select the structural unit with the lowest score according to the proportion for global unified pruning, retain the channel set before the score, and construct the corresponding sparse mask matrix G. Steps 3-4: Based on the generated sparse mask matrix G, structurally prune the parameters of the global optimization anti-interference decision model to obtain the sparse decision model; In step 3-1, the formula for globally aggregating the local parameters of the local anti-interference decision model is as follows: ； ； in, and These are the main decision network parameters and the main evaluation network parameters obtained from training the i-th child node, respectively. and These are the aggregated global master decision network parameters and global master evaluation network parameters, respectively, where N represents the number of unmanned platforms participating in this round of federated training. It is the local sample weight of the i-th unmanned platform. This represents the sample weight of the j-th unmanned platform; In steps 3-4, the parameters of the global optimization anti-interference decision model are adjusted by... Structural trimming of the defined sparse mask matrix: ； in, This represents the parameters of the globally optimized, anti-interference decision-making model after structural pruning of the central node. The symbol represents element-wise product, and G represents sparse mask matrix; In step 2, the steps for constructing the local anti-interference decision model for the unmanned platform based on DDPG are as follows: Step 2-1: Train the local robust decision-making model using the DDPG optimization structure that includes the loss of the evaluation network and the decision network; Step 2-2: For the constructed evaluation network, train the state-action value function in a supervised manner using MSE. ,in, This represents the environmental state at time t. The decision action at time t is represented, and the network loss function is obtained. For the constructed decision network, the policy gradient method is adopted, and the objective function is maximized through gradient descent to obtain the decision network loss function. ; Step 2-3: Evaluate the network loss function constructed based on step 2-2. and decision network loss function Gradient backpropagation and parameter updates are performed on the main evaluation network and the main decision network respectively, and finally a local anti-interference decision model is obtained.

2. The anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning according to claim 1, characterized in that, In step 1, the environmental data collected by the unmanned platform sub-node is preprocessed to obtain discrete complex IQ signals. Short-time Fourier transform is used to perform time-frequency transformation on the discrete complex IQ signals to obtain a time-frequency diagram.

3. The anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning according to claim 2, characterized in that: The discrete complex IQ signal Recorded as: ； in, and These represent the real and imaginary parts of the signal, respectively, and h represents the complex unit; Discrete complex IQ signal The time-frequency diagram is obtained after short-time Fourier transform. The transformation process is as follows: ； Where n and m represent time steps, and w(m) is a Hamming window function of length M. is the complex exponential basis function of the discrete Fourier transform.

4. The anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning according to claim 1, characterized in that, In step 2-2, the evaluation network uses the minimum mean square error loss function to minimize the difference between the current Q value and the Bellman objective generated by the target network. Wherein, at time t, the target Q value is The reward is The Q-value under the next state and future decisions is The value network parameters under the next state and future decisions are: The discount factor is Therefore, the current target Q value can be expressed as: ； Based on the aforementioned target Q value, the loss function for evaluating the network is obtained. To minimize the mathematical expectation between the current actual Q value and the target Q value, i.e.: ； in, This represents the Q-value of the evaluation network under the current state and action. This represents the evaluation network parameters under the current decision. This represents the training samples sampled from the experience replay pool. Represents the mathematical expectation. Operator for mathematical expectation; The decision network employs a policy gradient method to maximize the reward brought by the action in the current state, and its loss function is the negative Q value of the action output by the decision network under the current evaluation network. Among them, let This indicates the output action of the decision network in the current state. This represents the decision network parameters in the current state. If we evaluate the Q-value of the network in the current state for the current decision network, then the overall expected Q-value of the current decision network is... The expression is as follows: ； So what is the loss function of the decision network? for: ； in, This represents the samples actually used in the experience replay pool.

5. The anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning according to claim 1, characterized in that: In steps 2-3, the main evaluation network optimizes its parameters using the evaluation network loss function; the main decision network optimizes its parameters by minimizing the decision network loss function; and after optimization, the target network is soft-updated.

6. The anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning according to claim 5, characterized in that: The target network includes a target evaluation network and a target decision network. The target network is then soft-updated as follows: ； ； in, and Let represent the parameters of the objective evaluation network and the objective decision network, respectively. This is the soft update coefficient. and These represent the parameters of the evaluation network and the decision network, respectively.

7. The anti-interference method for unmanned platform self-organizing networks based on DDPG and federated pruning according to claim 1, characterized in that: Update the state-action value function based on the Bellman equation ; This can be expressed by the following formula: ； in, Indicates the learning rate. Indicates the action at time t. Indicates the discount factor. This represents the reward at time t.

Images