Anti-jamming frequency hopping pattern optimization method based on recursive prediction and uncertainty perception and medium

Abstract

This invention discloses an anti-interference frequency hopping pattern optimization method and medium based on recursive prediction and uncertainty awareness. The method includes the following steps: observing and backtracking the power spectrum of the communication channel; modeling the communication channel selection as a partially observable Markov decision process; collecting historical power spectra and constructing a power spectrum matrix dataset, and building a power spectrum recursive prediction model; recursively predicting future power spectra; introducing a Monte Carlo Dropout mechanism; determining the optimal length of the pattern to be generated by comparing inflection point detection with a dynamic threshold; constructing a communication channel decision model; determining each hop of the communication channel in the optimal length pattern, and generating a complete communication channel sequence. This invention solves the problems of fixed pattern length and lack of foresight in communication channel selection caused by the lack of prediction and evaluation capabilities for future interference states of the communication channel in traditional communication methods, as well as the practical application limitations caused by excessive synchronization overhead in time-slot agility methods based on deep reinforcement learning.

Description

Technical Field

[0001] This invention relates to the field of wireless communication anti-interference technology, specifically to an anti-interference frequency hopping pattern optimization method and medium based on recursive prediction and uncertainty perception. Background Technology

[0002] With the rapid development of artificial intelligence technology, time-slot agility methods based on deep reinforcement learning (DRL) have gradually emerged. These methods, through online learning and dynamic selection of communication channels by the agent, can achieve anti-jamming performance far exceeding traditional techniques in complex interference environments. However, DRL-based time-slot agility methods do not rely on predefined patterns but dynamically select the communication channel at each hop, posing a significant challenge to pattern synchronization between legitimate transmitters and receivers. To ensure communication reliability, the communication system must introduce additional signaling and control overhead to achieve pattern synchronization, which significantly increases implementation complexity and greatly limits the engineering application of this type of method. Furthermore, frequency-hopping methods based on block agility dynamically adjust the offset parameters of each block pattern through block window agility, balancing anti-jamming performance and synchronization overhead to some extent. However, the anti-jamming performance of these methods still declines to varying degrees under various interference scenarios, especially when facing intelligent interference attacks with learning and adaptive evolution capabilities. Because the observation capabilities of the legitimate communication agent are limited, and its decision-making and learning speed may be slower than that of the interfering agent, it is difficult to converge to an effective strategy in a timely manner, leading to a significant reduction in anti-jamming performance. Summary of the Invention

[0003] The purpose of this invention is to provide an anti-interference frequency hopping pattern optimization method and medium based on recursive prediction and uncertainty perception, so as to solve the problem that traditional communication methods lack the ability to predict and evaluate the future interference state of the communication channel in non-cooperative, time-varying and intelligent interference environments, resulting in fixed pattern length and lack of foresight in communication channel selection.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: A method for optimizing interference-resistant frequency hopping patterns based on recursive prediction and uncertainty awareness is provided, including the following steps: S1: Define a legitimate transmitter and a legitimate receiver to communicate in a frequency-hopping manner, and equip the legitimate transmitter with an agent. At the start of each pattern decision, the agent observes and backtracks the power spectrum of the communication channel for multiple consecutive hops and constructs the actual power spectrum waterfall. S2: Based on the intelligent agent and the actual power spectrum waterfall, the communication channel selection problem for each hop is modeled as a partially observable Markov decision process, and the optimization objective is to maximize the long-term cumulative reward expectation, which is used to jointly optimize the frequency hopping pattern length and the communication channel selection strategy. S3: Collect historical power spectra during the interaction between the agent and the environment and construct a power spectrum matrix dataset. Construct a power spectrum recursive prediction model based on a convolutional neural network and train it using the power spectrum matrix dataset. The power spectrum recursive prediction model takes the actual power spectrum waterfall as input and recursively predicts the power spectrum of future multi-hops as the predicted power spectrum. S4: In the prediction process of the power spectrum recursive prediction model, the Monte Carlo Dropout mechanism is introduced to perform multiple random forward propagations to quantify the prediction uncertainty of the power spectrum recursive prediction model for future multi-hop power spectrum prediction; based on the prediction uncertainty and the prediction error generated in the prediction process, the weighted loss sequence is calculated, and the optimal length of the current pattern to be generated is determined by inflection point detection and comparison with the dynamic threshold. S5: Construct a communication channel decision model based on deep reinforcement learning; use the observed power spectrum waterfall, which is formed by splicing the actual power spectrum waterfall and the predicted power spectrum, as the input of the communication channel decision model to select a communication channel for each hop in the pattern to determine the optimal length, and generate a complete communication channel sequence; the legitimate transmitter synchronizes the communication channel sequence as a pattern to the legitimate receiver.

[0005] To optimize the above technical solution, the specific limitations also include: The actual power spectrum waterfall For one The matrix is ​​defined as: ; in, The sample length of the power spectrum being traced back. The total number of available communication channels. For the first Power spectrum during the jump period ( ) indicates the first Power levels on each communication channel For decision-making moments, The pattern number is the serial number. For the first The length of each pattern.

[0006] Preferably, in step S1, the sample length of the backtracked power spectrum is greater than the maximum possible length of the pattern, denoted as... .

[0007] Furthermore, in step S2, the partially observable Markov decision process is represented by a seven-tuple: ; in, This represents the set of actual states of the environment. This represents the set of actions that an anti-interference agent can perform. This represents the set of observations that the agent can obtain. For the reward function, This is the state transition function. Let be the observation probability function. Discount factor; The agent aims to maximize the expected value of long-term cumulative rewards, specifically: ; in, The optimal strategy to maximize the expected value of long-term cumulative rewards. For long-term accumulated reward expectation, For the reward function, For the observation of intelligent agents, This represents the actual environmental conditions. For communication channels, ; The reward function is defined as follows: .

[0008] Furthermore, in step S3, the training of the power spectrum recursive prediction model adopts a planned sampling strategy, and the training iterations are based on probability. The teacher-forced model is used to calculate the one-step prediction loss and the probability. The recursive prediction loss is calculated using an autoregressive model. The sum of the one-step prediction loss and the recursive prediction loss is taken as the total loss. The parameters of the recursive prediction model are then optimized using the stochastic gradient descent algorithm.

[0009] Further, in step S4, the prediction uncertainty of the quantized power spectrum recursive prediction model for future multi-hop power spectrum prediction specifically includes: performing the same input... The recursive random forward propagation obtains at each step... The predicted power spectrum is calculated; the variance of the predicted power spectrum at each step is used to represent the prediction uncertainty at that step.

[0010] Preferably, in step S4, determining the optimal length of the current pattern to be generated based on prediction error and prediction uncertainty by comparing inflection point detection with a dynamic threshold specifically includes: calculating a weighted loss sequence by combining the root mean square error and prediction uncertainty of each prediction step; calculating the mean and standard deviation of the weighted loss sequence and updating the dynamic threshold; and comprehensively determining the optimal length of the pattern by performing thresholding on the weighted loss value and the dynamic threshold, and combining the results of inflection point detection on the first-order difference sequence of the weighted loss sequence.

[0011] Furthermore, in step S5, the communication channel decision model is constructed based on a deep Q-network, and its Q-function is optimized by minimizing a loss function: ; ; in, The loss function is the Q-function. To assess the environmental status Select Communication Channel The expected value of long-term cumulative rewards that can be obtained For the parameters of the Q network, For the target Q value, In the first The immediate reward obtained after performing the action. As a discount factor, For parameters The target network calculates the expected value of the long-term cumulative reward corresponding to the next state action.

[0012] Further, in step S5, the specific calculation formula for selecting a communication channel for each hop in the pattern to determine the optimal length is as follows: ; in, For the pattern Skip to the selected communication channel, The observed power spectrum waterfall is formed by splicing the actual power spectrum waterfall with the predicted power spectrum. To evaluate the Q-network based on the observed Select communication channel The expected value of long-term cumulative rewards, For the power spectrum recursive prediction model, the first Jump in communication channel The predicted power value.

[0013] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute the anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty awareness as described above.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception provided by this invention constructs a power spectrum recursive prediction model using a convolutional neural network within a partially observable Markov decision process framework. The model learns the temporal and frequency domain characteristics of historical power spectra and recursively generates a prediction sequence of future multi-hop channel power spectra. This provides an evaluation object for dynamic optimization of pattern length and extends the observations for subsequent communication channel decision models. This ensures that the final generated communication channel sequence is a sequence selection made within the same forward-looking cognitive framework. It solves the problem that, under incomplete observation conditions, agents cannot directly perceive future environmental states, making it difficult to generate multi-hop length patterns with overall synergy and optimal long-term benefits.

[0015] In the inference stage of the prediction model, this invention introduces a Monte Carlo Dropout mechanism to perform multiple random forward propagations, quantifying the prediction uncertainty of the power spectrum recursive prediction model for future multi-hop power spectrum predictions. Based on the prediction uncertainty and the prediction error generated during the prediction process, it comprehensively utilizes the inflection point detection method and the dynamic threshold method to adaptively determine the optimal length of the pattern under the current environment, transforming empirical or fixed planning step sizes into a data-driven dynamic optimization process. This solves the decision-making problem in time-domain planning in intelligent frequency hopping systems. Furthermore, based on dynamic thresholds and inflection point detection, it avoids the decline in decision quality caused by the accumulation of prediction errors with the step size, ensuring the overall performance of each synchronized and executed pattern within its validity period and improving the system's anti-interference capability.

[0016] This invention constructs a communication channel decision model based on deep reinforcement learning. By inputting an observed power spectrum waterfall composed of historical and predicted power spectra, the decision model considers the potential impact on the future state and benefits of several hops when selecting a communication channel, ensuring the synergy of the channel sequences within the pattern and the optimality of the overall long-term benefits. Simultaneously, it generates an overall synergistic and long-term optimal communication channel sequence for frequency hopping patterns of a certain length, solving the problems of excessive synchronization overhead in time-slot agility methods based on deep reinforcement learning and the degradation of anti-interference performance in block-agility frequency hopping methods. Attached Figure Description

[0017] Figure 1 : A flowchart illustrating the anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception of the present invention.

[0018] Figure 2 : Schematic diagram of wireless communication adversarial scenario based on pattern dynamic optimization.

[0019] Figure 3 : A schematic diagram of the anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception of the present invention.

[0020] Figure 4 : A schematic diagram of the time slot structure in the anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception of the present invention.

[0021] Figure 5 : Schematic diagram of power spectrum recursive prediction based on convolutional neural network.

[0022] Figure 6 The diagram shows the anti-interference performance of the optimized method of this invention when facing cross-comb interference attacks.

[0023] Figure 7 The anti-jamming performance diagram of the optimized method of the present invention when facing a combined jamming attack consisting of DRL-based intelligent jamming and frequency sweeping jamming. Detailed Implementation

[0024] The present invention will be further described in detail below through specific embodiments, but it should not be construed as limiting the scope of the subject matter of the present invention to the following embodiments. All technologies implemented based on the above content of the present invention fall within the scope of the present invention.

[0025] In one embodiment, this invention proposes an anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty awareness, the flowchart of which is shown below. Figure 1 As shown, the entire method includes the following steps: S1: Define a legitimate transmitter and a legitimate receiver to communicate in a frequency-hopping manner, and equip the legitimate transmitter with an agent. At the start of each pattern decision, the agent observes and backtracks the power spectrum of the communication channel for multiple consecutive hops, and constructs the actual power spectrum waterfall. S2: Based on the agent and the actual power spectrum waterfall, the communication channel selection problem of each hop is modeled as a partially observable Markov decision process, and the optimization objective is to maximize the long-term cumulative reward expectation, which is used to jointly optimize the frequency hopping pattern length and the communication channel selection strategy. S3: Collect historical power spectra during the interaction between the agent and the environment and construct a power spectrum matrix dataset. Build a power spectrum recursive prediction model based on a convolutional neural network and train it using the power spectrum matrix dataset. The power spectrum recursive prediction model takes the actual power spectrum waterfall as input and recursively predicts the power spectrum of future multi-hops as the predicted power spectrum. S4: In the prediction process of the power spectrum recursive prediction model, the Monte Carlo Dropout mechanism is introduced to perform multiple random forward propagations to quantify the prediction uncertainty of the power spectrum recursive prediction model for future multi-hop power spectrum prediction; based on the prediction uncertainty and the prediction error generated in the prediction process, the weighted loss sequence is calculated, and the optimal length of the current pattern to be generated is determined by inflection point detection and comparison with the dynamic threshold. S5: Construct a communication channel decision model based on deep reinforcement learning; use the observed power spectrum waterfall, which is formed by splicing the actual power spectrum waterfall and the predicted power spectrum, as the input of the communication channel decision model to select a communication channel for each hop in the pattern to determine the optimal length, and generate a complete communication channel sequence; the legitimate transmitter synchronizes the communication channel sequence as a pattern to the legitimate receiver.

[0026] In step S1, as Figure 2 As shown, the legitimate transmitter and legitimate receiver communicate via frequency hopping. An intelligent agent is installed at the legitimate transmitter to determine the pattern corresponding to future multi-hops and synchronize the pattern with the legitimate receiver. The bandwidth of the legitimate baseband signal is... bandwidth is The available frequency band is divided into bandwidth And non-overlapping communication channels , Indicates the first One communication channel; Sequence shared between legitimate transmitters and receivers Divided into One pattern: ; in, Indicates the first Each pattern represents a communication channel sequence. Indicates the first The first pattern Jump to the corresponding communication channel; use Indicates the maximum possible length of the pattern. Indicates the first The length of each pattern; In the Before the pattern begins, the legitimate transmitter traces back the sequence of events. Power spectrum waterfall construction during jump period Actual power spectrum waterfall For one The matrix is ​​defined as: ; in, The sample length of the power spectrum being traced back. The total number of available communication channels. For the first Power spectrum during the jump period ( ) indicates the first Power levels on each communication channel For decision-making moments, The pattern number is the serial number. For the first The length of the pattern; the sample length of the backtracking power spectrum is greater than the maximum possible length of the pattern, denoted as... ; The calculation formula is: ; in, Indicates the first Power levels on each communication channel For the first The power spectral density function estimated using the P-Welch algorithm during the jump period. The bandwidth of a single communication channel. This is the starting frequency of the frequency band.

[0027] like Figure 3 As shown, the agent uses the actual power spectrum waterfall. As input states, the pattern length and the communication channel for each hop are determined; such as Figure 4 As shown, in the transmission of the first Before the valid signal corresponding to each pattern is transmitted, the valid transmitter transmits the corresponding pattern via the control link. Synchronization is provided to legitimate receivers; in the... The first pattern Before the jump begins, the legitimate transmitter follows the pattern. Select communication channel In the The first pattern During the jump, the legitimate transmitter is in the communication channel Transmitting legal signals; in the first The first pattern During the hop, the legitimate receiver demodulates from the communication channel. The signal received on, and in the first The first pattern After the hop transmission is completed, a feedback signal is sent to the legitimate transmitter through the control link. If the transmission is successful, an ACK signal is sent back; otherwise, a NACK signal is sent back.

[0028] In step S2, the communication channel selection problem for each hop in the anti-interference frequency hopping pattern optimization is modeled as a partially observable Markov decision process, and represented by a 7-tuple: ; in, This represents the set of actual states of the environment. This represents the set of actions that an anti-interference agent can perform. This represents the set of observations that the agent can obtain. For the reward function, This is the state transition function. Let be the observation probability function. Discount factor; Before each hop begins, the agent performs observations... From the action set Select the communication channel to use during the current hop. In the pattern decision-making process, only the observations before the first hop are completely consistent with the actual environmental state, that is... Subsequent observations From state The middle Individual power spectrum and predicted power spectrum constitute: ; Because there is a difference between the predicted power spectrum and the actual power spectrum, the observation Compared with the actual environmental conditions There may be discrepancies; Define a reward function for immediate feedback on the agent's behavior: ; in, For the reward function in the environment state Select Communication Channel Instant rewards received after communication.

[0029] Use state transition function Defined in the environment state The legal transmitter performs the action. Afterwards, the environmental state shifted to The probability of observation; the probability function of observation Defined in the current environment state The lower agent observed The probability, the optimization objective of the power spectrum prediction model is to make ; The agent aims to maximize the expected value of long-term cumulative reward, specifically: ; in, The optimal strategy to maximize the expected value of long-term cumulative rewards. For long-term accumulated reward expectation, For the reward function, For the observation of intelligent agents, This represents the actual environmental conditions. For communication channels, This is the discount factor.

[0030] In step S3, historical power spectra are collected during the interaction between the agent and the environment, and a power spectrum matrix dataset is constructed. A power spectrum recursive prediction model is built based on a convolutional neural network and trained using the power spectrum matrix dataset. A schematic diagram of the power spectrum recursive prediction is shown below. Figure 5 As shown; In the Before each pattern begins, trace back the continuous sequence. Power spectrum during jump, construct power spectrum matrix : ; The power spectrum matrix is ​​stored in a container with a capacity of 1,000, using a first-in, first-out principle. cache ; A power spectrum recursive prediction model is constructed based on convolutional neural networks. The model's input is the power spectral waterfall. The output is the predicted value of the next-hop power spectrum. , These are the parameters of the convolutional neural network; The power spectrum recursive prediction model is trained using a planned sampling strategy, with probability... The teacher-forced model is used to calculate the one-step prediction loss and the probability. The recursive prediction loss is calculated using an autoregressive model, and the sum of the one-step prediction loss and the recursive prediction loss is taken as the total loss. The parameters of the recursive prediction model are then optimized using the stochastic gradient descent algorithm. For samples that use the teacher-forced model to calculate the one-step prediction loss ,structure Group sample label pairs:

[0031] in, Indicates the first Power spectrum waterfall before the jump begins. Indicates the first Power spectrum during jump, using cache Random sampling A set constructed from samples Using sets The sample labels constructed from the samples in the dataset are used to calculate the cumulative loss for one-step prediction: ; in, Here is the one-step prediction loss function computed under the teacher-forced model. For the first Predicted power spectrum during the jump period; For samples where the predicted loss is calculated using an autoregressive model ,structure Group sample label pairs: ; ; in, This represents a power spectrum waterfall constructed from the actual power spectrum and the predicted power spectrum. Representing the power spectrum during the next hop, using a cache Random sampling A set constructed from samples Using sets The sample labels constructed from the samples are used to calculate... The cumulative loss of step recursive prediction ; ; in, Here is the recursive prediction loss function calculated in the autoregressive model. The predicted power spectrum; Finally, the cumulative loss of the power spectrum recursive prediction is calculated. The stochastic gradient descent algorithm is used to optimize the parameters of the power spectrum recursive prediction model. ,in This represents the learning rate.

[0032] In step S4, during the prediction process of the power spectrum recursive prediction model, a Monte Carlo Dropout mechanism is introduced to perform multiple random forward propagations, quantifying the prediction uncertainty of the power spectrum recursive prediction model for future multi-hop power spectrum predictions, and performing the same input... The recursive random forward propagation obtains at each step... Group predicted power spectra; calculate the variance of the predicted power spectrum at each step to represent the prediction uncertainty at that step: In the Before each pattern begins, based on the dataset Each sample in Construct sample label pairs ; During the inference phase, for the same input conduct Second-rate Step-by-step recursive random forward propagation, obtaining at each step Group prediction power spectrum, the first step of prediction obtained The predicted power spectrum of the group is expressed as: ; Follow-up Step prediction obtained The predicted power spectrum of the group is expressed as: ; in, Indicates the first Dropout random mask during the next forward propagation; In the first In the second forward propagation, the model... Predicted power spectrum of the jump output; This represents the maximum possible length of the pattern, and also the maximum prediction step size allowed by the system. Indicates the first The power spectrum waterfall before the jump begins; In the first In the second forward propagation, the model... Predicted power spectrum of the jump output; The power spectrum waterfall is constructed; These are the parameters of the convolutional neural network; they are calculated at each step. Mean of the predicted power spectrum of the group: ; in, For the first Power spectrum jump The mean of the group prediction results; calculate The variance of the predicted power spectrum is used as a quantification of the prediction uncertainty: ; in, This is a quantified value for predicting uncertainty.

[0033] By combining the root mean square error and prediction uncertainty of each prediction step, a weighted loss sequence is calculated; the mean and standard deviation of the weighted loss sequence are calculated, and the dynamic threshold is updated; by using the threshold method to judge the weighted loss value and the dynamic threshold, and combining the inflection point detection results of the first-order difference sequence of the weighted loss sequence, the optimal length of the pattern is determined comprehensively. Calculate the loss of the mean of the predicted power spectrum at each step: ; in, For the first The root mean square error of the prediction step measures the accuracy of the prediction. Calculate the weighted loss at each step based on the uncertainty of the prediction model:

[0034] in, Indicates the first The weighted loss of each step This indicates the uncertainty of the prediction model; Calculate the statistics for the entire predicted sequence: ; ; ; in, The mean of the weighted loss sequence. The standard deviation of the weighted loss sequence. The average uncertainty of the entire prediction sequence; Update adaptive coefficients : ; Update dynamic threshold : ; in, The coefficient of the exponential moving average; Determine the maximum length of the pattern using the threshold method: ; in, These are candidate length values ​​obtained through the thresholding method; For weighted loss sequences Calculate the first-order difference: ; Determine the maximum length of a pattern using inflection point detection: ; in, These are candidate length values ​​obtained through the inflection point detection method; Based on the combined results of the threshold method and inflection point detection, a conservative selection of pattern length is made: ; In step S5, a communication channel decision model is constructed based on deep reinforcement learning, and a deep Q-network is used to approximate the value of each state-action pair: ; Among them, the Q function Representing state-action pairs The expected value of long-term cumulative rewards, This represents the parameters of the Q-network. As a reward signal, Discount factor; During the interaction between the intelligent agent and the environment, empirical sample data of state transitions are continuously collected. Stored in the experience replay pool in a first-in, first-out manner middle; In each iteration, from the experience replay pool Randomly sample a batch of empirical samples Calculate the target Q value for each empirical sample: ; in, For the target Q value, In the first The immediate reward obtained after performing the action. As a discount factor, The parameters of the target Q-network are represented by each The next iteration replicates the Q-network parameter update. For parameters The target network calculates the expected value of the long-term cumulative reward corresponding to the next state action; Based on empirical samples Calculate the Q-function to estimate the loss: ; in, The loss function is the Q-function. To assess the environmental status Select Communication Channel The expected value of long-term cumulative rewards that can be obtained For the parameters of the Q network, The target Q value; The loss is estimated by minimizing the Q-function, and the Q-network parameters are optimized online. : ; in, The learning rate used to optimize the Q network; For the The first jump in the pattern, the decision model is based on the observed environmental state. Combined with the first The power spectrum prediction results during the jump period are output as follows: The communication channel selected during the first hop in the pattern : ; For the The follow-up in the pattern Jump, the decision model is based on the agent's observations Combined with subsequent The power spectrum recursive prediction results during the jump period are output respectively. The subsequent pattern Communication channel selected during hop : ; ; The obtained communication channels are arranged in sequence to form a pattern. As the first A pattern.

[0035] The optimal pattern generated by a legitimate transmitter Synchronization is achieved once via the control link with the legitimate receiver. Subsequently, the legitimate transmitter and legitimate receiver... The hop communication strictly follows this pattern, without any synchronization signaling. After each hop, the receiver sends back an ACK / NACK to update the agent's experience.

[0036] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute the above-described anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty awareness.

[0037] To further understand the technical solution of the present invention, a detailed description is provided in conjunction with specific embodiments: The system simulation uses the PyTorch framework of Python, and the system model includes a group of legitimate users and multiple jammers.

[0038] Legitimate users work in MHz to In the MHz frequency band, the legal transmitter's transmit power is 30dBm, and the legal baseband signal bandwidth is 1MHz. The bandwidth is... The available frequency bands of MHz can be divided into There are 3 non-overlapping communication channels, each with a bandwidth of 1. The set of available communication channel center frequencies in MHz is as follows: .

[0039] Cross-comb jamming attacks can simultaneously target five equally spaced communication channels, with the following sets of attack channels: ; or: ; The jammer jumps every 10 hops in the assembly and The system switches between communication channels once, transmitting a jamming signal with a power of 50 dBm on each channel. A frequency-sweeping jamming attack continuously scans the entire frequency band at a rate of 1 GHz / s, forming a time-varying narrowband jamming signal with a power of 50 dBm and a linearly varying center frequency. A smart jamming attack selects three adjacent communication channels based on its observations and the decisions of the agent at the jammer, transmitting a jamming signal with a power of 50 dBm on each channel.

[0040] Figure 6 The diagram shows the anti-interference performance of the present invention in the face of cross-comb interference attacks. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty awareness proposed in this invention has better anti-interference performance than traditional frequency hopping methods. Compared with the slot agility method based on DRL, the additional synchronization overhead required is reduced by about 11 times. In addition, compared with the frequency hopping method based on block agility, it achieves an anti-interference performance gain of nearly 40% with the same required synchronization overhead.

[0041] Figure 7 This diagram illustrates the anti-jamming performance of the present invention against combined interference attacks consisting of DRL-based intelligent interference and frequency sweeping interference. The proposed method not only outperforms traditional frequency hopping methods in terms of anti-jamming performance, but also surpasses DRL-based time-slot agility methods and Minimax Q-learning-based time-slot agility methods. Compared with DRL-based time-slot agility methods, this method reduces the required additional synchronization overhead by approximately 8 times while maintaining high performance. Compared with block-agility-based frequency hopping methods, this method achieves an anti-jamming performance gain of nearly 30% with comparable synchronization overhead.

[0042] This invention proposes an anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty awareness. It employs a convolutional neural network model to recursively predict the power spectrum of future multi-hops and dynamically adjusts the pattern length based on the uncertainty estimate of the prediction model. Furthermore, a deep reinforcement learning algorithm determines the communication channel for each hop in the pattern based on the predicted power spectrum. This method minimizes pattern synchronization overhead while maintaining anti-interference performance. Compared to intelligent time-slot agile methods, this invention reduces the additional communication overhead required for pattern synchronization without significant attenuation of anti-interference performance. Compared to block-agile frequency hopping methods, this invention achieves better anti-interference performance with comparable additional communication overhead.

[0043] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0044] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent substitutions, and improvements made by those skilled in the art to the above embodiments without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. An anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception, characterized in that, Includes the following steps: S1: Define a legitimate transmitter and a legitimate receiver to communicate in a frequency-hopping manner, and equip the legitimate transmitter with an agent. At the start of each pattern decision, the agent observes and backtracks the power spectrum of the communication channel for multiple consecutive hops and constructs the actual power spectrum waterfall. S2: Based on the intelligent agent and the actual power spectrum waterfall, the communication channel selection problem for each hop is modeled as a partially observable Markov decision process, and the optimization objective is to maximize the long-term cumulative reward expectation, which is used to jointly optimize the frequency hopping pattern length and the communication channel selection strategy. S3: Collect historical power spectra during the interaction between the agent and the environment and construct a power spectrum matrix dataset. Construct a power spectrum recursive prediction model based on a convolutional neural network and train it using the power spectrum matrix dataset. The power spectrum recursive prediction model takes the actual power spectrum waterfall as input and recursively predicts the power spectrum of future multi-hops as the predicted power spectrum. S4: In the prediction process of the power spectrum recursive prediction model, the Monte Carlo Dropout mechanism is introduced to perform multiple random forward propagations to quantify the prediction uncertainty of the power spectrum recursive prediction model for future multi-hop power spectrum prediction; based on the prediction uncertainty and the prediction error generated in the prediction process, the weighted loss sequence is calculated, and the optimal length of the current pattern to be generated is determined by inflection point detection and comparison with the dynamic threshold. S5: Constructing a communication channel decision model based on deep reinforcement learning; The observed power spectrum waterfall, which is formed by splicing the actual power spectrum waterfall with the predicted power spectrum, is used as the input to the communication channel decision model. A communication channel is selected for each hop in the pattern of optimal length, generating a complete communication channel sequence. The legitimate transmitter synchronizes this communication channel sequence as a pattern to the legitimate receiver.

2. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: The actual power spectrum waterfall For one The matrix is ​​defined as: ； in, The sample length of the power spectrum being traced back. This represents the total number of available communication channels. For the first Power spectrum during the jump period ( ) indicates the first Power levels on each communication channel For decision-making moments, The pattern number. For the first The length of each pattern.

3. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 2, characterized in that: In step S1, the sample length of the backtracked power spectrum is greater than the maximum possible length of the pattern, denoted as: .

4. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: In step S2, the partially observable Markov decision process is represented by a seven-tuple: ； in, This represents the set of actual states of the environment. This represents the set of actions that an anti-interference agent can perform. This represents the set of observations that the agent can obtain. For the reward function, This is the state transition function. Let be the observation probability function. Discount factor; The agent aims to maximize the expected value of long-term cumulative rewards, specifically: ； in, The optimal strategy to maximize the expected value of long-term cumulative rewards. For long-term accumulated reward expectation, For the reward function, For the observation of intelligent agents, This represents the actual environmental conditions. For communication channels, ; The reward function is defined as follows: 。 5. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: In step S3, the power spectrum recursive prediction model is trained using a planned sampling strategy, with probabilistic sampling used in the training iterations. The teacher-forced model is used to calculate the one-step prediction loss and the probability. The recursive prediction loss is calculated using an autoregressive model. The sum of the one-step prediction loss and the recursive prediction loss is taken as the total loss. The parameters of the recursive prediction model are then optimized using the stochastic gradient descent algorithm.

6. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: In step S4, the prediction uncertainty of the quantized power spectrum recursive prediction model for future multi-hop power spectrum prediction specifically includes: performing the same input... The recursive random forward propagation obtains at each step... The predicted power spectrum is calculated; the variance of the predicted power spectrum at each step is used to represent the prediction uncertainty at that step.

7. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: In step S4, determining the optimal length of the current pattern to be generated based on prediction error and prediction uncertainty by comparing inflection point detection with a dynamic threshold specifically includes: calculating a weighted loss sequence by combining the root mean square error and prediction uncertainty of each prediction step; calculating the mean and standard deviation of the weighted loss sequence and updating the dynamic threshold; and comprehensively determining the optimal length of the pattern by performing thresholding on the weighted loss value and the dynamic threshold, and combining the results of inflection point detection on the first-order difference sequence of the weighted loss sequence.

8. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: In step S5, the communication channel decision model is constructed based on a deep Q-network, and its Q-function is optimized by minimizing the loss function: ； ； in, The loss function is the Q-function. To assess the environmental status Select Communication Channel The expected value of long-term cumulative rewards that can be obtained For the parameters of the Q network, For the target Q value, In the first The immediate reward obtained after performing the action. As a discount factor, For parameters The target network calculates the expected value of the long-term cumulative reward corresponding to the next state action.

9. The anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty perception according to claim 1, characterized in that: In step S5, the specific calculation formula for selecting a communication channel for each hop in the pattern to determine the optimal length is as follows: ； in, For the pattern Skip to the selected communication channel, The observed power spectrum waterfall is formed by splicing the actual power spectrum waterfall with the predicted power spectrum. To evaluate the Q-network based on the observed Select communication channel The expected value of long-term cumulative rewards, For the power spectrum recursive prediction model, the first Jump in communication channel The predicted power value.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program causes the computer to execute the anti-interference frequency hopping pattern optimization method based on recursive prediction and uncertainty awareness as described in any one of claims 1-9.

Images