Intelligent interference model reward poisoning defense, training method and system
By constructing a reward-poisoning defense network for an intelligent interference communication system and training the defense framework using state transitions and action switching, the problem of low efficiency in existing reward-poisoning defense methods is solved, achieving rapid and effective defense and improving the anti-interference capability of intelligent communication.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies lack fast and effective methods to defend against reward poisoning in communication warfare, especially in intelligent communication systems, where reward attacks affect the agent's optimization strategy, and existing defense methods require a large amount of interactive data.
A smart interference communication system is constructed. A reward poisoning defense network is built through a deep neural network. The defense framework is trained by state transition and action switching. A dual experience pool structure is designed to quickly learn and output clean rewards to resist reward poisoning attacks.
It achieves the same defensive effect as the original interference algorithm with fewer interactions, improves the anti-interference capability of intelligent communication, and can effectively defend against various reward poisoning attacks.
Smart Images
Figure CN116667966B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of communication countermeasures technology, specifically relating to an intelligent interference model reward poisoning defense, training method and system. Background Technology
[0002] With the development of artificial intelligence technology, deep learning and reinforcement learning are emerging technological fields. Combining the excellent feature extraction capabilities of deep learning with the advantages of reinforcement learning in sequential decision-making, deep reinforcement learning is widely used in applications such as autonomous driving, path planning, and adversarial communication. Reinforcement learning learns optimal policies through interaction with the environment, while deep learning extracts environmental features. Its good generalization ability allows reinforcement learning to be applied to complex task scenarios.
[0003] In recent years, the security issues of deep learning have attracted attention. For example, a well-trained neural network can accurately identify the type of panda in an image. However, adding a tiny, invisible perturbation to the image, changing the values of a few pixels, can cause the neural network to misidentify the panda. In applications such as traffic light recognition and autonomous driving, such attacks on images would pose a significant security risk. Furthermore, deep reinforcement learning not only inherits most of the attack methods common to deep learning but also includes perturbations unique to reinforcement learning, such as attacks on the environment, rewards, and policies. Since the reward is the target of reinforcement learning optimization, attacks on the reward will undoubtedly affect the agent's optimization policy, and may even prevent the agent from optimizing the policy altogether.
[0004] Existing defenses against reward attacks in reinforcement learning mostly require lengthy interactions and the collection of large amounts of experience samples. However, research on defenses against reward poisoning in the field of intelligent communication jamming is extremely limited. In real-world adversarial scenarios, extensive interactions in communication environments are impractical. Therefore, researching a fast and effective defense against reward poisoning is urgently needed. Summary of the Invention
[0005] To address the shortcomings of existing technologies and enhance the anti-interference capability of intelligent communication, this invention adopts the following technical solution:
[0006] A method for training an intelligent interference model with a reward-based poisoning defense mechanism includes the following steps:
[0007] Step S1: Construct an intelligent jamming communication system that includes a sender, a receiver, and a jammer;
[0008] Step S2: Obtain the communication parameters of the sender and receiver;
[0009] Step S3: Implement a reward poisoning attack on the reward obtained by the interfering party;
[0010] Step S4: Obtain samples that were successfully interfered with and those that failed to interfere with;
[0011] Step S5: Construct a reward poisoning defense network based on a deep neural network, obtain the poisoned communication state after being attacked by a reward poisoning attack, the interference actions under the poisoned reward optimization, and output the restored reward of the interfering party; based on the poisoned communication state after being attacked by a reward poisoning attack and the interference actions under the poisoned reward optimization, construct a defense framework to minimize the difference between the restored clean reward and the original reward; based on the output of the defense network and the success or failure of interference of the samples, construct the loss function of the defense network.
[0012] Step S6: Integrate the defense framework with the defense network, replace the poisoned reward with the restored clean reward, and train for intelligent interference.
[0013] Further, step S5 includes the following steps:
[0014] Step S51: Construct a reward poisoning defense network based on a deep neural network. The defense network has parameters. The defense network consists of seven fully connected layers;
[0015] Step S52: The defense network obtains the communication status of both communicating parties. and corresponding interference actions This indicates the poisoning state of the original intelligent interference model after it has been subjected to a reward poisoning attack. This indicates the strategy under the optimized poison reward system;
[0016] Step S53: In the early stage of training, The dataset stored in the experience pool as training data, where T takes values of 1 and 0 to represent state transition flags, is denoted as:
[0017]
[0018] in To perform the action Afterwards, the receiver's state changes. When T is 0, it means the receiver's state has changed, indicating that the sample was successfully interfered with and is placed in the experience pool D1. When T is 1, it means the receiver's state remains unchanged, indicating that the sample was not interfered with and is placed in the experience pool D2.
[0019] Step S54: The mathematical model for constructing the defense framework is as follows:
[0020]
[0021] in r represents the reward of the original interference model, and the goal of the defense framework is to minimize the difference between the restored reward and the original reward;
[0022] Step S55: The defense network is trained using the following formula:
[0023] loss_d=-y*log(T)-(1-y)*log(1-T)
[0024] Where y is the predicted output of the defense network, T is the state transition flag, and log is the logarithm symbol in mathematics;
[0025] Step S56: When the number of training rounds exceeds the activation threshold of 5, the defense framework is activated. and Input the defense network, output the clean reward of restoration.
[0026] Furthermore, in step S1, the communication and anti-interference methods of the transmitter and receiver in the intelligent interference communication system include the following modulation modes: binary phase shift keying (BPSK), quadrature phase shift keying (QPSK), and frequency shift keying (FSK). The transmission power is divided into two levels, and the communication frequency is within 100MHz to 2GHz. When interference occurs, the frequency switching step is 100MHz. After interference, the transmitter and receiver will first increase the transmission power, then switch the communication frequency, and finally change the modulation method.
[0027] Furthermore, in step S1, the interfering party employs a soft actor critic (SAC) algorithm based on deep reinforcement learning, which includes a policy network and a value network. The parameters of the policy network are... Value network parameters are The interfering agent acts as an intelligent agent in reinforcement learning. Its environment consists of the communication state between the sender and receiver, and its action is the interference waveform emitted by the interfering agent. The interfering agent acquires the communication mode of the sender and receiver at time t. The input 's' to the policy network and value network is the communication state at time t plus the interference action at time t. The output policy of the policy network is the interference parameters quantized at time t+1. The output of the policy network is expressed as:
[0028]
[0029] Where μ is the mean of the Gaussian distribution, σ is the covariance of the Gaussian distribution, and π represents the distribution of the strategy;
[0030] The policy network parameters are updated by minimizing the KL (Kullback-Leibler) divergence, which is defined as follows:
[0031]
[0032] Where N B Represents the training batch, log is the logarithm symbol in mathematics, Q iLet λ represent the output of the Q-network, and λ be the entropy coefficient. According to the definition of KL divergence, the update of the policy network is expressed by the following formula:
[0033]
[0034] Where β' represents the learning rate of the policy network;
[0035] The value network parameters are updated by minimizing the mean square Bellman error (MSBE), which is defined as follows:
[0036]
[0037] Q i' This represents the output of the target Q-network. The update of the value network is expressed by the following formula:
[0038]
[0039] Where β represents the learning rate of the Q-network, Represents the gradient symbol.
[0040] Furthermore, the communication parameters obtained in step S2 include the spectrum usage of the communication signal and the modulation mode of the communication signal.
[0041] Furthermore, the spectrum usage is obtained using the energy detection method, based on the detection statistic T. i (y):
[0042]
[0043] Where y i (k) represents the input signal, N represents the number of signal sample energy samples, and T represents the input signal. i (y) and the preset threshold value λ i The comparison yields a decision d regarding the spectrum status and the existence of the primary user. i .
[0044] Furthermore, the modulation waveform of the communication signal is obtained, and the automatic modulation signal recognition of the communication signal is adopted by deep learning based on the Long Short-Term Memory (LSTM) network. The network structure is constructed by the convolutional neural network and the LSTM network. The input data are the I and Q signals of the modulation signal, and 128 points are sampled for each signal.
[0045] Furthermore, in step S3, the poisoning method employed is a reward flip attack, which perturbs the reward. From the noise disturbance parameter e + and e - Characterization:
[0046]
[0047] Where r + and r - s represents the value of the binary reward. t s represents the state at time t. t+1 Let a represent the state at time t+1. t Let P represent the perturbation action at time t, P represent the transition probability sign, and r represent the reward of the original perturbation model. When the reward value exceeds two values, the reward confusion matrix C is used. M×M express:
[0048]
[0049] Each value c j,k Reward R j There is a certain probability of flipping to R k ,Right now:
[0050]
[0051] In this invention, the reward is a binary reward, r + This indicates that the bit error rate (SER) is greater than X, r - This indicates that the bit error rate is less than X, where X represents the bit error rate threshold, which is set to 0.1. The attack probability is expressed as:
[0052] P1(r(SER <X)=r - |r(SER>X)=r + )
[0053] P2(r(SER>X)=r + |r(SER <X)=r - ).
[0054] A method for defending against reward poisoning attacks using an intelligent interference model is provided, which uses a reward poisoning defense network obtained from the aforementioned intelligent interference model reward poisoning defense training method to defend against reward poisoning attacks in a communication system.
[0055] A smart interference model reward poisoning defense device includes a sender, a receiver, an interfering party, a sensing module, a reward poisoning module, and a defense module. The sensing module analyzes the communication parameters of the sender and receiver; the reward poisoning module performs a reward poisoning attack on the rewards obtained by the interfering party; and the defense module is a reward poisoning defense network obtained through the aforementioned smart interference model reward poisoning defense method.
[0056] The advantages and beneficial effects of this invention are as follows:
[0057] This invention discloses a method and system for defending against and training intelligent interference models using reward poisoning. The reward poisoning defense framework is a universal framework that can be embedded into any reinforcement learning framework with low coupling to existing algorithms. Compared to some existing defense frameworks that require massive amounts of empirical data, the reward poisoning defense framework designed in this invention can quickly achieve the same performance as existing interference algorithms with fewer interactions. Using state transitions as the training method for the defense framework not only defends against reward flip attacks but also effectively defends against various reward poisoning attacks, thereby improving the anti-interference capability of intelligent communication. Attached Figure Description
[0058] Figure 1 This is a schematic diagram of the system structure according to an embodiment of the present invention.
[0059] Figure 2 This is a flowchart illustrating the implementation of the defense framework in an embodiment of the present invention.
[0060] Figure 3 This is a schematic diagram of the structure of the defense network in an embodiment of the present invention. Detailed Implementation
[0061] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0062] like Figure 1 As shown, a dual-experience-pool intelligent interference model reward poisoning defense system includes an intelligent interference communication system, a sensing module, a reward poisoning module, and a defense module.
[0063] A smart communication jamming system comprises a transmitter, a receiver, and a jammer. The communication method between the transmitter and receiver includes three parameters: modulation mode, transmit power, and communication frequency. Modulation modes include Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), and Frequency Shift Keying (FSK). Transmit power is divided into two levels, and the communication frequency ranges from 100MHz to 2GHz. When jammed, the frequency switching step is 100MHz. Upon being jammed, the transmitter and receiver will first increase the transmit power, then switch the communication frequency, and finally change the modulation mode.
[0064] In the intelligent communication jamming system, the jammer's intelligent jamming method is a soft actor / critic algorithm based on deep reinforcement learning. It consists of one policy network (for actors) and four Q-networks (i.e., value networks, for critics). The parameters of the policy network are... The four Q-networks consist of two Q-networks (Q1 network and Q2 network) and two target Q-networks (target Q1 network and target Q2 network), with the following network parameters: Both the policy network and the Q network consist of fully connected layers. The jammer, acting as the agent in reinforcement learning, is exposed to the communication states of the sender and receiver, and its actions are the jamming waveforms it emits. The jammer obtains the communication patterns of the sender and receiver at time t through a spectrum sensing module. In the intelligent jamming algorithm, the input 's' to the policy network and value network is the communication state at time t plus the jamming action at time t. The output policy of the policy network is the quantized jamming parameters at time t+1. The output of the policy network can be expressed as:
[0065]
[0066] Where μ is the mean of the Gaussian distribution, σ is the covariance of the Gaussian distribution, and π represents the policy distribution. The policy network parameters are updated by minimizing the Kullback-Leibler (KL) divergence, which is defined as follows:
[0067]
[0068] Where N B Represents the training batch, log is the logarithm symbol in mathematics, Q i Let λ represent the output of the Q-network, and λ be the entropy coefficient. According to the definition of KL divergence, the update of the policy network is expressed by the following formula:
[0069]
[0070] Where β' represents the learning rate of the policy network.
[0071] The value network parameters are updated by minimizing the mean square Bellman error (MSBE), which is defined as follows:
[0072]
[0073] Q i' This represents the output of the target Q-network. The updates of the Q1 and Q2 networks are expressed by the following formulas:
[0074]
[0075] Where β represents the learning rate of the Q-network, L represents the gradient sign, and L() represents the mean square Bellman error of the corresponding Q-network.
[0076] The sensing module includes a spectrum sensing module and a modulation waveform recognition module. The spectrum sensing module detects the spectrum usage of the communication signal and estimates its approximate power, while the modulation waveform recognition module identifies the modulation mode of the communication signal. The spectrum sensing module uses energy detection to detect the spectrum status and primary users. Energy detection is simple to implement; its principle is to detect the signal power received from the licensed frequency band and compare it to a threshold value for decision-making. The detection statistic T... i (y):
[0077]
[0078] Where y i (k) represents the input signal, and N represents the number of signal sample energy samples. Let T... i (y) and the preset threshold value λ i The comparison yields a decision d regarding the spectrum status and the existence of the primary user. i The energy detection method can not only detect spectrum occupancy but also estimate approximate power, making it suitable for this invention. The modulation waveform recognition module employs a deep learning-based automatic modulation signal recognition method for communication signals, based on a Long Short-Term Memory (LSTM) network. The network structure is constructed using a Convolutional Neural Network (CNN) and LSTM, with input data consisting of the I and Q signals of the modulation signal, each sampled at 128 points. This approach improves the recognition rate while reducing the number of training parameters and shortening the training time.
[0079] The reward poisoning module uses a reward flip attack as its poisoning method. This perturbs the reward. From the noise disturbance parameter e + and e - Characterization:
[0080]
[0081] Where r + and r - s represents the value of the binary reward. t s represents the state at time t. t+1 Let a represent the state at time t+1. t Let represent the perturbation action at time t, P represent the transition probability sign, and r represent the reward of the original perturbation model. When the reward value exceeds two values, a reward confusion matrix C is used. M×M express:
[0082]
[0083] Each value c j,k Reward R j There is a certain probability of flipping to R k ,Right now:
[0084]
[0085] In this invention, the reward is a binary reward, r + This indicates that the bit error rate (SER) is greater than X, r - This indicates that the bit error rate is less than X, where X represents the bit error rate threshold, which is set to 0.1. The attack probability is expressed as:
[0086] P1(r(SER <X)=r _ |r(SER>X)=r + )
[0087] P2(r(SER>X)=r + |r(SER <X)=r _ )
[0088] The reward defense module includes a dual experience pool structure, storing samples that were successfully interfered with and those that failed to interfere with, respectively. Both experience pools D1 and D2 have a capacity of 1000. The experience pools are implemented using a unidirectional queue data structure. For data pairs, when the experience pool contains more than 1000 data pairs, the data is removed from the end of the queue.
[0089] like Figure 2 As shown, the Reward Poisoning Defense Framework includes the following steps:
[0090] (1) Construct a reward poisoning defense network based on a deep neural network. The defense network has parameters. like Figure 3 As shown, the defense network consists of seven fully connected layers, with the number of neurons in the seven hidden layers being 128, 512, 512, 1024, 1024, 512, and 256, respectively.
[0091] (2) The defense network input is the communication status of the two communicating parties as sensed by the spectrum sensing module. This represents the poisoned state of the original intelligent interference model after being subjected to a reward poisoning attack. The original intelligent interference model outputs interference actions based on this state. This indicates the strategy under the poison reward optimization. and As input to the defense network. For example... Figure 3 As shown, After passing through hidden layer 1 and hidden layer 2, After passing through hidden layers 3 and 4, the outputs of hidden layers 2 and 4 are merged and input into hidden layer 5, and then through hidden layers 6 and 7 to obtain a one-dimensional output.
[0092] (3) In the early stage of training, The dataset stored in the experience pool for training is denoted as follows: T takes values of 1 and 0, representing state transition flags. T is expressed as:
[0093]
[0094] in To perform the action Then, the receiver enters the state. When T is 0, it means that the receiver's state has changed, indicating that the sample was successfully interfered with and is placed in the experience pool D1; when T is 1, it means that the receiver's state remains unchanged, indicating that the sample was not interfered with and is placed in the experience pool D2.
[0095] (4) The mathematical model of the defense framework is:
[0096]
[0097] in r represents the reward of the original perturbation model. The goal of the defense framework is to minimize the difference between the restored reward and the original reward. Later in the training process, δ decreases, and the output of the defense framework gets closer and closer to the reward of the original perturbation model, until δ equals 0, at which point the output of the defense framework is exactly equal to the reward of the original perturbation model.
[0098] (5) The training formula for the defense network is:
[0099] loss_d=-y*log(T)-(1-y)*log(1-T)
[0100] Where y is the predicted output of the defense network, T is the state transition flag, and log is the logarithm symbol in mathematics.
[0101] (6) When the number of training rounds exceeds the activation threshold of 5, the defense framework will be activated. and Input the defense network, output the clean reward of restoration.
[0102] This invention constructs an intelligent interference countermeasure system and designs a defense method against reward poisoning attacks. It uses states, actions, and action switching to train and predict the real clean reward, and uses the restored clean reward to replace the poisoned reward, so that the performance of the original intelligent interference model is not affected by reward poisoning attacks.
[0103] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A training method for intelligent interference model reward-based poisoning defense, characterized in that... Includes the following steps: Step S1: Construct an intelligent jamming communication system that includes a sender, a receiver, and a jammer; Step S2: Obtain the communication parameters of the sender and receiver; Step S3: Implement a reward poisoning attack on the reward obtained by the interfering party; Step S4: Obtain samples that were successfully interfered with and those that failed to interfere with; Step S5: Construct a reward poisoning defense network based on a deep neural network, obtain the poisoned communication state after being attacked by a reward poisoning attack, the interference actions under the poisoned reward optimization, and output the restored reward of the interfering party; based on the poisoned communication state after being attacked by a reward poisoning attack and the interference actions under the poisoned reward optimization, construct a defense framework to minimize the difference between the restored clean reward and the original reward; based on the output of the defense network and the success or failure of interference of the samples, construct the loss function of the defense network. Specifically, the steps include the following: Step S51: Construct a reward poisoning defense network based on deep neural networks; Step S52: The defense network obtains the communication status of both communicating parties. and corresponding interference actions This indicates the poisoning state of the original intelligent interference model after it has been subjected to a reward poisoning attack. This indicates the strategy under the optimized poison reward system; Step S53: In the early stage of training, For the training dataset stored in the experience pool, where T represents the state transition flag, T is represented as: in To execute interference actions Afterwards, the receiver's state changes. When T is 0, it means the receiver's state has changed, indicating that the sample was successfully interfered with and is placed in the experience pool D1. When T is 1, it means the receiver's state remains unchanged, indicating that the sample was not interfered with and is placed in the experience pool D2. Step S54: The mathematical model for constructing the defense framework is as follows: in r represents the reward of the original interference model; Step S55: The defense network is trained using the following formula: loss_d=-y*log(T)-(1-y)*log(1-T) Where y is the predicted output of the defense network, T is the state transition flag, and log is the logarithm symbol in mathematics; Step S56: When the number of training rounds exceeds the activation threshold of 5, the defense framework is activated. and Input the defense network, output the clean reward of restoration. Step S6: Integrate the defense framework with the defense network, replace the poisoned reward with the restored clean reward, and train for intelligent interference.
2. The intelligent interference model reward poisoning defense training method according to claim 1, characterized in that: In step S1, the communication and anti-interference methods of the transmitter and receiver of the intelligent interference communication system include the modulation modes of binary phase shift keying, quadrature phase shift keying, and frequency shift keying. The transmission power is divided into two levels, and the communication frequency is within 100MHz to 2GHz. When interference occurs, the frequency switching step is 100MHz. After being interfered with, the transmitter and receiver will first increase the transmission power, then switch the communication frequency, and finally change the modulation method.
3. The intelligent interference model reward poisoning defense training method according to claim 1, characterized in that: In step S1, the interfering party employs a soft actor critic algorithm based on deep reinforcement learning, which includes a policy network and a value network. The parameters of the policy network are... Value network parameters are The interfering agent acts as an intelligent agent in reinforcement learning. Its environment consists of the communication state between the sender and receiver, and its action is the interference waveform emitted by the interfering agent. The interfering agent acquires the communication mode of the sender and receiver at time t. The input 's' to the policy network and value network is the communication state at time t plus the interference action at time t. The output policy of the policy network is the interference parameters quantized at time t+1. The output of the policy network is expressed as: Where μ is the mean of the Gaussian distribution, σ is the covariance of the Gaussian distribution, π represents the distribution of the strategy, and a t s represents the disturbance action at time t. t Indicates the state at time t; The policy network parameters are updated by minimizing the KL divergence, which is defined as follows: Where N B Represents the training batch, log is the logarithm symbol in mathematics, Q i Let λ represent the output of the Q-network, and λ be the entropy coefficient. According to the definition of KL divergence, the update of the policy network is expressed by the following formula: Where β' represents the learning rate of the policy network; The update of the value network parameters is defined by minimizing the mean square Bellman error as follows: Q i' The output of the target Q-network is represented by the following formula: Where β represents the learning rate of the Q-network, Represents the gradient symbol.
4. The intelligent interference model reward poisoning defense training method according to claim 1, characterized in that: The communication parameters obtained in step S2 include the spectrum usage of the communication signal and the modulation mode of the communication signal.
5. The intelligent interference model reward poisoning defense training method according to claim 4, characterized in that: The spectrum usage was obtained using the energy detection method, based on the detection statistic T. i (y): Where y i (k) represents the input signal, N represents the number of signal sample energy samples, and T represents the input signal. i (y) and the preset threshold value λ i The comparison yields a decision d regarding the spectrum status and the existence of the primary user. i .
6. The intelligent interference model reward poisoning defense training method according to claim 4, characterized in that: The modulation waveform of the communication signal is obtained, and the automatic modulation signal recognition of the communication signal is adopted by deep learning based on the Long Short-Term Memory (LSTM) network. The network structure is constructed by convolutional neural network and LSTM network. The input data are the I and Q signals of the modulation signal, and each signal is sampled.
7. The intelligent interference model reward poisoning defense training method according to claim 1, characterized in that: In step S3, the poisoning method used is a reward flip attack to perturb the reward. From the noise disturbance parameter e + and e - Characterization: Where r + and r - s represents the value of the binary reward. t s represents the state at time t. t+1 Let a represent the state at time t+1. t Let P represent the perturbation action at time t, P represent the transition probability sign, and r represent the reward of the original perturbation model. When the reward value exceeds two values, the reward confusion matrix C is used. M×M express: Each value c j,k Reward R j There is a certain probability of flipping to R k ,Right now: The reward is a binary reward, r + This indicates that the bit error rate (SER) is greater than X, r - This indicates that the bit error rate is less than X, where X represents the bit error rate threshold. The attack probability is expressed as: P1(r(SER<X)=r - |r(SER>X)=r + ) P2(r(SER>X)=r + |r(SER<X)=r - )。 8. A method for preventing poisoning through intelligent interference models, characterized in that... The reward poisoning defense network obtained by the intelligent interference model reward poisoning defense training method according to claims 1-7 is used to defend against reward poisoning attacks in a communication system.
9. A smart interference model reward poisoning defense device, comprising a sender, a receiver, an interfering party, and a sensing module, characterized in that: The device further includes a reward poisoning module and a defense module. The communication parameters of the sender and receiver are obtained by parsing through the perception module. The reward poisoning module performs a reward poisoning attack on the reward obtained by the interfering party. The defense module is a reward poisoning defense network obtained by the intelligent interference model reward poisoning defense method described in claim 8.