A cognitive radar active anti-interference waveform generation method and system based on DQN and MAB

Abstract

The present invention discloses a method and system for generating active anti-interference waveforms for cognitive radars based on DQN and MAB, wherein the method comprises: obtaining a radar interference time-frequency diagram based on radar transmission signals and interference signals; simulating a radar countermeasure process and constructing a transmission waveform strategy library based on the radar interference time-frequency diagram; solving the waveform strategy library using a DQN algorithm, and optimizing the solution using a MAB algorithm to obtain an optimal transmission waveform; detecting the optimal transmission waveform using a CFAR detector to generate an optimal anti-interference waveform and parameters. The present invention integrates the global strategy search of DQN with the local parameter optimization of MAB, effectively solving the problem of dynamic anti-interference adaptation in multi-pulse timing interaction scenarios, and is suitable for airborne radar electronic countermeasure systems and real-time anti-interference requirements in complex electromagnetic environments.

Description

Technical Field

[0001] The present invention relates to the field of radar electronic countermeasure technology, and in particular to a method and system for generating active anti-interference waveforms for a cognitive radar based on DQN and MAB. Background Art

[0002] Radar plays a key role in the modern battlefield, enabling remote acquisition of battlefield situational awareness and seizing information control. However, active jamming, particularly mainlobe jamming, significantly restricts radar operation and effectiveness, thereby impacting precision strikes against enemy targets and the accuracy of military command. Current anti-jamming research focuses on designing radar's passive signal processing algorithms and active waveform transmission. However, these approaches primarily target fixed and static jammers, failing to fully consider the pseudo-random, highly dynamic, intelligent, and complex transmission behaviors that jammers may employ in competitive warfare. As jammers evolve toward intelligent and collaborative capabilities, existing single-static active and passive countermeasures are becoming increasingly inadequate.

[0003] Aiming for next-generation cognitive and collaborative electronic countermeasures, the design of radar anti-jamming measures emphasizes the need to fully consider the game-playing nature of jamming. Existing radar countermeasures suffer from insufficient dynamic adaptability to jamming, inefficient waveform parameter optimization, and weak composite jamming suppression capabilities. Overly idealistic modeling assumptions about jammer adversaries hinder generalization of policy learning algorithms. Furthermore, policy learning algorithms suffer from low sample efficiency and fail to fully utilize the radar's multi-dimensional transmit degrees of freedom. Summary of the Invention

[0004] To solve the technical problems in the above background, the present invention provides a method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB, the steps comprising:

[0005] Based on the radar transmission signal and the interference signal, obtain the radar interference time-frequency diagram;

[0006] Based on the radar interference time-frequency diagram, simulating the radar countermeasure process and building a transmission waveform strategy library;

[0007] The waveform strategy library is solved using the DQN algorithm, and the MAB algorithm is used to optimize the solution to obtain the optimal transmission waveform;

[0008] The optimal transmission waveform is detected by using a CFAR detector to generate an optimal anti-interference waveform and parameters.

[0009] Preferably, the method for obtaining the radar interference time-frequency map includes:

[0010] Establish the radar transmission signal and interference signal model of the frequency agile system, where the radar transmission signal is an agile LFM waveform:

[0011]

[0012] Where, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse;

[0013] The jamming signal is a deceptive and suppressive single jamming, including: distance deception jamming, speed deception jamming, narrowband pulse jamming, intermittent sampling direct forwarding jamming, intermittent sampling cyclic forwarding jamming, intermittent sampling repeated forwarding jamming, non-uniform intermittent sampling repeated forwarding jamming, modulated intermittent sampling repeated forwarding jamming, dense false target jamming, spectrum dispersion jamming, intelligent noise jamming and noise convolution jamming.

[0014] Preferably, after generating the radar interference time-frequency diagram, a dynamic feature extraction module based on zero-memory incremental learning is constructed to sense the spectrum characteristics of the interference signal in real time and generate an interference feature vector, the steps including:

[0015] The echo signal at the receiving end of the radar system is recorded as:

[0016] R(t)=S(t)+J(t)+N(t);

[0017] Where S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal;

[0018] STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is:

[0019] STFT(t,f)=∫R(τ)g * (τ-t)e -j2πfτ dτ;

[0020] Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

[0021] Preferably, the constructed dynamic feature extraction module adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples into a hyperdimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

[0022] Preferably, the constructed transmission waveform strategy library includes: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

[0023] Preferably, the step of solving the waveform strategy library using the DQN algorithm includes:

[0024] The state s required to construct the DQN model based on SINR and radar action t , defined as:

[0025] s t =[a t ,SINR t ];

[0026]

[0027] Where SINR is the signal to interference and noise ratio, P s is the echo power, P j is the interference power, P n is the noise power, a t For radar action;

[0028] Use SINR to construct reward function r t Control the action benefits and strategy tendencies of the radar agent:

[0029]

[0030] Where SINR is the signal to interference and noise ratio, t cheat is the time when the radar launches the false target, t observe The larger the SINR, the easier it is for the target signal to be detected by the receiver, and the better the performance of the radar system.

[0031] Finally, DQN is used to solve the optimal transmission waveform strategy under incomplete interference information.

[0032] Preferably, the DQN network training decision loss function is:

[0033]

[0034] Where r t is the reward function, γ is the discount factor, q(s t+1 ,a′;θ -) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter.

[0035] Preferably, the optimal transmission waveform parameters are adaptively optimized by the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p ; Among them, the regret value Regret(T) in the MAB algorithm is defined as:

[0036]

[0037] Where A is the action set, T is the number of rounds, μ(·) is the expectation of pulling the lever, and x(·) is the policy representation.

[0038] Preferably, the decision indicator of the CFAR detector is:

[0039]

[0040] Where n is the number of targets detected by the CFAR detector.

[0041] The present invention also provides a cognitive radar active anti-interference waveform generation system based on DQN and MAB, the system is used to implement the above method, including: an acquisition module, a construction module, a solution module and a detection module;

[0042] The acquisition module is used to obtain a radar interference time-frequency diagram based on the radar transmission signal and the interference signal;

[0043] The construction module is used to simulate the radar countermeasure process and build a transmission waveform strategy library based on the radar interference time-frequency diagram;

[0044] The solution module is used to solve the waveform strategy library using the DQN algorithm and optimize the solution result using the MAB algorithm to obtain the optimal transmission waveform;

[0045] The detection module is used to detect the optimal transmission waveform using a CFAR detector to generate an optimal anti-interference waveform and parameters.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention integrates the global strategy search of DQN and the local parameter optimization of MAB to effectively solve the problem of dynamic adaptation of anti-interference in multi-pulse timing interaction scenarios. It is suitable for airborne radar electronic countermeasure systems and real-time anti-interference requirements in complex electromagnetic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 Schematic diagram of a method flow in an embodiment of the present invention;

[0050] Figure 2 Schematic diagram of the method framework of an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of radar countermeasure process modeling according to an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the DQN network structure and training process according to an embodiment of the present invention;

[0053] Figure 5 A sample time-frequency diagram randomly selected from each interference signal in an embodiment of the present invention;

[0054] Figure 6 A time-frequency diagram of a sample randomly selected from each transmitted signal in the transmitted waveform decision library of an embodiment of the present invention;

[0055] Figure 7 A training loss curve diagram of the DQN training decision process according to an embodiment of the present invention;

[0056] Figure 8 Graph showing the number of detections and the detection probability during the MAB parameter optimization process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] In order to facilitate the description of the content of the method of the present invention, the following terms are explained.

[0060] Term 1: Deep Q-Network (DQN);

[0061] Term 2: Multi-armed Bandit (MAB);

[0062] Term 3: Linear Frequency Modulation (LFM);

[0063] Term 4: Markov Decision Process (MDP);

[0064] Term 5: Signal to Interference plus Noise Ratio (SINR);

[0065] Term 6: Constant False Alarm Rate (CFAR).

[0066] Example 1

[0067] like Figure 1 、 Figure 2 FIG. 1 is a flow chart of the method of this embodiment, and the steps include:

[0068] S1. Obtain a radar interference time-frequency diagram based on the radar transmission signal and the interference signal.

[0069] Establish a frequency-agile radar transmission signal and interference signal model with the ability to adjust bandwidth and time width; the radar transmission signal is an agile LFM waveform, and the interference signal is a deceptive and suppressive single interference.

[0070] The radar agile LFM waveform within a processing interval can be expressed as:

[0071]

[0072] Where, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse.

[0073] In a further embodiment, the radar jamming signal is a deceptive and suppressive single jamming, including distance deception jamming (DDJ), velocity deception jamming (VDJ), narrowband pulse jamming (NPJ), intermittent sampling direct jamming (ISDJ), intermittent sampling loop jamming (ISLJ), intermittent sampling repeater jamming (ISRJ), non-uniform intermittent sampling repeater jamming (NUISRJ), modulated intermittent sampling repeater jamming (MISRJ), dense false target jamming (DFTJ), smeared spectrum jamming (SMSPJ), smart noise jamming (Smart Noise Jamming), etc. Jamming (SNJ) and noise convolution jamming (NCJ) 12 kinds, in the embodiment, a sample time-frequency diagram randomly selected from each interference signal is as follows Figure 5 shown.

[0074] Radar instantaneous transmission power P T The maximum value of the peak transmission power P max And the average power maximum value P avg The constraint is calculated as follows:

[0075]

[0076] When the pulse width is short, the radar uses P max When the pulse width exceeds a critical point P T Affected by P avg The peak power is constrained by P max Decline; the radar transmit waveform strategy determines the T of each LFM waveform p , B and f c (n).

[0077] Radar jamming signals are deceptive and suppressive single jammers, including distance deception jammers, speed deception jammers, narrowband pulse jammers, intermittent sampling direct forwarding jammers, intermittent sampling cyclic forwarding jammers, intermittent sampling repeated forwarding jammers, non-uniform intermittent sampling repeated forwarding jammers, modulated intermittent sampling repeated forwarding jammers, dense false target jammers, spectrum dispersion jammers, intelligent noise jammers, and noise convolution jammers.

[0078] The radar interference time-frequency diagram is obtained based on the radar echo data. A dynamic feature extraction module based on zero-memory incremental learning is constructed to perceive the spectrum characteristics of the interference signal in real time and generate the interference feature vector. The specific steps are as follows:

[0079] The echo signal at the receiving end of the radar system is generally a combination of the target signal, the interference signal and the noise signal, which can be expressed as:

[0080] R(t)=S(t)+J(t)+N(t);

[0081] Where S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal;

[0082] STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is:

[0083] STFT(t,f)=∫R(τ)g * (τ-t)e -j2πfτ dτ;

[0084] Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

[0085] The dynamic feature extraction module based on zero-memory incremental learning adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples into a hyperdimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

[0086] S2. Based on the radar interference time-frequency diagram, simulate the radar countermeasure process and build a transmission waveform strategy library.

[0087] Cognitive intelligent radar has a closed-loop learning and self-evolution radar architecture, and has the capabilities of adaptive environmental perception, arbitrary transmission and reception design, intelligent processing and resource scheduling. Figure 3As shown in the figure, when the radar receives an interference signal, it first analyzes the signal characteristics of different interferences in the time-frequency domain in the radar interference time-frequency diagram, and perceives the interference characteristic parameters of the interference type; at the same time, it establishes an interference type resource library and an anti-interference transmission waveform resource library; then, the existing anti-interference transmission waveform in the radar anti-interference transmission waveform resource library is selected; finally, the effectiveness of the anti-interference transmission waveform is evaluated.

[0088] The anti-interference transmission waveform strategy resource library is designed based on the basic LFM waveform to perform complex modulation of multiple parameters such as frequency, phase, modulation rate polarity, and repetition rate. The constructed strong anti-competitive complex modulation waveforms include: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform, and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

[0089] Give full play to the advantages of active confrontation and increase the difficulty of interception analysis and injection interference of enemy jammers. In this embodiment, a sample time-frequency diagram randomly selected from each transmission signal in the transmission waveform decision library is as follows: Figure 6 shown.

[0090] After the transmission waveform strategy library is built, the action set, state set, and SINR-based reward function are set using the MDP concept. Specifically, the following are included:

[0091] The elements of MDP include radar agent action a t , state s t and the reward function r t In the MDP problem of cognitive radar active anti-interference, the action set A corresponds to the radar's anti-interference transmission waveform strategy library, the state set S corresponds to the radar's interference type library, and the single-step reward value R is abstracted as the SINR. By designing a corresponding algorithm structure, intelligent decision-making of the transmission waveform during the radar anti-interference process can be achieved.

[0092] Radar action a t Defined as:

[0093] a t =A[t];

[0094] Where A[t] is the action set composed of the anti-interference transmission waveform strategy library.

[0095] S3. Use the DQN algorithm to solve the waveform strategy library, and use the MAB algorithm to optimize the solution to obtain the optimal transmission waveform.

[0096] Based on the established frequency agile radar transmission signal and interference signal model, after completing the construction of the radar echo signal R(t), it is necessary to analyze the interference degree of the real echo signal to evaluate the recovery effect of the echo signal before and after the implementation of the anti-interference transmission waveform action. The state s required for constructing the DQN model based on SINR and radar action t , defined as:

[0097] s t =[a t ,SINR t ];

[0098]

[0099] Where SINR is the signal to interference and noise ratio, P s is the echo power, P j is the interference power, P n is the noise power, a t For radar action.

[0100] Reward function r in the generation process of emission waveform selection strategy t It is used to control the action benefits and strategy tendencies of the radar agent. In order to characterize the characteristic relationship between the jammer and the radar signal, SINR is used to transform the reward function r t , defined as:

[0101]

[0102] Where SINR is the signal to interference and noise ratio, t cheat is the time when the radar launches the false target, t observe The larger the SINR, the easier it is for the receiver to detect the target signal, and the better the radar system performance. Generally speaking, when the SINR is above 10dB, the radar system performance can be considered good.

[0103] DQN is used to solve the optimal transmission waveform strategy under incomplete interference information. The DQN network structure and training process diagram in the embodiment are shown in the following figure. Figure 4 The specific implementation steps are as follows:

[0104] (1) Initialization parameters;

[0105] (2) Randomly initialize the radar jamming signal state;

[0106] (3) Entering the anti-interference transmission waveform selection module, selecting the best anti-interference transmission waveform output by the estimation value network;

[0107] (4) Enter the radar to execute the anti-interference transmission waveform action module, obtain the single-step return value, and perform state transition through the target value network to obtain the next state;

[0108] (5) Enter the experience data storage module and store the data obtained from the training experiment into the memory bank;

[0109] (6) Determine the number of samples in the memory bank;

[0110] (7) Determine whether the current number of training times meets the minimum network training condition. If so, proceed to step 8 for network training; otherwise, proceed to step 9.

[0111] (8) First, select M sample data from the memory bank, obtain the estimated Q value through the estimated value network, obtain the target Q value through the target value network, solve the loss function, train the network, and update the network parameters;

[0112] (9) If it is greater than the number of iterations in this round, go to step 2 to conduct the next round of training experiments; otherwise, go to step 3 to continue this round of experiments.

[0113] Specifically, during the radar anti-interference process, the optimal anti-interference transmission waveform corresponding to the identified interference signal is intelligently determined from the anti-interference transmission waveform strategy resource library based on the anti-interference countermeasure principle.

[0114] The DQN network training decision loss function is:

[0115]

[0116] Where r t is the reward function, γ is the discount factor, q(s t+1 ,a;;θ - ) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter; use the chain rule to find the partial derivative of the loss function with respect to the network parameter, and update the weight according to the back propagation algorithm.

[0117] The optimal transmission waveform parameters are adaptively optimized through the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p , specifically including:

[0118] In the original MAB, each lever is independent. When the levers are not independent, they are parameterized to form a parameterized MAB. There is a clear correspondence between the radar's cognitive transmit waveform strategy selection and the parameterized levers. The radar has K preset active transmit waveform strategies, corresponding to the K levers. The radar's transmit strategy is represented and driven by a low-dimensional vector. The radar aims to maximize the cumulative number of detections, which corresponds to minimizing the regret value.

[0119] The regret value Regret(T) in the MAB algorithm is defined as:

[0120]

[0121] Where A is the action set, T is the number of rounds, μ(·) is the expectation of pulling the lever, and x(·) is the policy representation.

[0122] S4. Use a CFAR detector to detect the optimal transmission waveform and generate the optimal anti-interference waveform and parameters.

[0123] Under the assumption of ideal target detection, sending the pulse compression result to the CFAR detector yields a clear target peak detection result. However, under non-ideal interference conditions, noise interference can cause the CFAR detector to be unable to effectively extract the target peak, resulting in a false target judgment. Deceptive interference can create multiple false target peaks, causing the CFAR detector to detect multiple peak targets.

[0124] Set the decision index of the CFAR detector to:

[0125]

[0126] Where n is the number of targets detected by the CFAR detector.

[0127] Example 2

[0128] The effects of the present invention will be further illustrated by simulation in conjunction with this embodiment.

[0129] 1. Simulation conditions

[0130] In this embodiment, the above-mentioned 12 radar active interference signals and 8 anti-interference emission waveforms are used to conduct a decision-making performance experiment on the radar anti-interference method. According to the above-mentioned experimental steps and environment, the experimental parameters are set as follows: the greed factor is 0.75, the learning rate is 0.7, and the discount factor is 0.8. A 3-layer network is used, and the network parameters are randomly initialized. The algorithm of the present invention is applied to a specific interference type. In each confrontation selection process, one of the 8 anti-interference emission waveforms is selected and the parameters are optimized. 10 rounds of confrontation experiments are designed, and 500 confrontation decisions are made in each round. The entire experimental process is run on a server configured with an Intel i7-12700F CPU and an NVIDIA GeForce RTX 4080Ti GPU (16G), and the algorithm is implemented in the CUDA 13.1 environment using the Python 3.8 programming language.

[0131] 2. Simulation results

[0132] Table 1

[0133] Interference scenario Optimal anti-interference waveform Intermittent sampling repeated forwarding interference Intrapulse Costas step linear frequency modulation waveform Spectral dispersive interference Intrapulse random step linear frequency modulation waveform Intelligent noise jamming Interpulse Costas step linear frequency modulation waveform

[0134] Combined with Table 1 and Figure 7 and Figure 8 It can be seen that the method of the present invention integrates the global strategy search of DQN and the local parameter optimization of MAB. In 12 typical interference scenarios, the average decision time is less than 40ms, the detection probability is improved from 0.72 to 0.91, and the model training decision process converges quickly, realizing the coordinated anti-interference of radar waveform generation and parameter optimization. It is significantly superior to the existing technology in terms of dynamic interference suppression, resource efficiency, real-time performance, etc., and has broad military and civilian prospects.

[0135] Example 3

[0136] The present invention also provides a cognitive radar active anti-interference waveform generation system based on DQN and MAB, including: an acquisition module, a construction module, a solution module and a detection module; the acquisition module is used to obtain a radar interference time-frequency diagram based on the radar transmission signal and the interference signal; the construction module is used to simulate the radar countermeasure process and construct a transmission waveform strategy library based on the radar interference time-frequency diagram; the solution module is used to use the DQN algorithm to solve the waveform strategy library, and use the MAB algorithm to optimize the solution result to obtain the optimal transmission waveform; the detection module is used to use a CFAR detector to detect the optimal transmission waveform and generate the optimal anti-interference waveform and parameters.

[0137] The following will describe in detail how the present invention solves technical problems in real life in conjunction with this embodiment.

[0138] The acquisition module obtains the radar interference time-frequency diagram based on the radar transmission signal and the interference signal.

[0139] Establish a frequency-agile radar transmission signal and interference signal model with the ability to adjust bandwidth and time width; the radar transmission signal is an agile LFM waveform, and the interference signal is a deceptive and suppressive single interference.

[0140] The radar agile LFM waveform within a processing interval can be expressed as:

[0141]

[0142] Where, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse.

[0143] In a further embodiment, the radar jamming signal is a deceptive and suppressive single jamming, including distance deception jamming (DDJ), velocity deception jamming (VDJ), narrowband pulse jamming (NPJ), intermittent sampling direct jamming (ISDJ), intermittent sampling loop jamming (ISLJ), intermittent sampling repeater jamming (ISRJ), non-uniform intermittent sampling repeater jamming (NUISRJ), modulated intermittent sampling repeater jamming (MISRJ), dense false target jamming (DFTJ), smeared spectrum jamming (SMSPJ), smart noise jamming (Smart Noise Jamming), etc. Jamming (SNJ) and noise convolution jamming (NCJ) 12 kinds, in the embodiment, a sample time-frequency diagram randomly selected from each interference signal is as follows Figure 5shown.

[0144] Radar instantaneous transmission power P T The maximum value of the peak transmission power P max And the average power maximum value P avg The constraint is calculated as follows:

[0145]

[0146] When the pulse width is short, the radar uses P max When the pulse width exceeds a critical point P T Affected by P avg The peak power is constrained by P max Decline; the radar transmit waveform strategy determines the T of each LFM waveform p , B and f c (n).

[0147] Radar jamming signals are deceptive and suppressive single jammers, including distance deception jammers, speed deception jammers, narrowband pulse jammers, intermittent sampling direct forwarding jammers, intermittent sampling cyclic forwarding jammers, intermittent sampling repeated forwarding jammers, non-uniform intermittent sampling repeated forwarding jammers, modulated intermittent sampling repeated forwarding jammers, dense false target jammers, spectrum dispersion jammers, intelligent noise jammers, and noise convolution jammers.

[0148] The radar interference time-frequency diagram is obtained based on the radar echo data. A dynamic feature extraction module based on zero-memory incremental learning is constructed to perceive the spectrum characteristics of the interference signal in real time and generate the interference feature vector. The specific steps are as follows:

[0149] The echo signal at the receiving end of the radar system is generally a combination of the target signal, the interference signal and the noise signal, which can be expressed as:

[0150] R(t)=S(t)+J(t)+N(t);

[0151] Where S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal;

[0152] STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is:

[0153] STFT(t,f)=∫R(τ)g * (τ-t)e -j2πfτ dτ;

[0154] Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

[0155] The dynamic feature extraction module based on zero-memory incremental learning adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples into a hyperdimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

[0156] The construction module simulates the radar countermeasure process and builds a transmission waveform strategy library based on the radar interference time-frequency diagram.

[0157] Cognitive intelligent radar has a closed-loop learning and self-evolution radar architecture, and has the capabilities of adaptive environmental perception, arbitrary transmission and reception design, intelligent processing and resource scheduling. Figure 3 As shown in the figure, when the radar receives an interference signal, it first analyzes the signal characteristics of different interferences in the time-frequency domain in the radar interference time-frequency diagram, and perceives the interference characteristic parameters of the interference type; at the same time, it establishes an interference type resource library and an anti-interference transmission waveform resource library; then, the existing anti-interference transmission waveform in the radar anti-interference transmission waveform resource library is selected; finally, the effectiveness of the anti-interference transmission waveform is evaluated.

[0158] The anti-interference transmission waveform strategy resource library is designed based on the basic LFM waveform to perform complex modulation of multiple parameters such as frequency, phase, modulation rate polarity, and repetition rate. The constructed strong anti-competitive complex modulation waveforms include: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform, and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

[0159] Give full play to the advantages of active confrontation and increase the difficulty of interception analysis and injection interference of enemy jammers. In this embodiment, a sample time-frequency diagram randomly selected from each transmission signal in the transmission waveform decision library is as follows: Figure 6 shown.

[0160] After the transmission waveform strategy library is built, the action set, state set, and SINR-based reward function are set using the MDP concept. Specifically, the following are included:

[0161] The elements of MDP include radar agent action a t , state s t and the reward function r tIn the MDP problem of cognitive radar active anti-interference, the action set A corresponds to the radar's anti-interference transmission waveform strategy library, the state set S corresponds to the radar's interference type library, and the single-step reward value R is abstracted as the SINR. By designing a corresponding algorithm structure, intelligent decision-making of the transmission waveform during the radar anti-interference process can be achieved.

[0162] Radar action a t Defined as:

[0163] a t =A[t];

[0164] Where A[t] is the action set composed of the anti-interference transmission waveform strategy library.

[0165] The solution module uses the DQN algorithm to solve the waveform strategy library and uses the MAB algorithm to optimize the solution results to obtain the optimal transmission waveform.

[0166] Based on the established frequency agile radar transmission signal and interference signal model, after completing the construction of the radar echo signal R(t), it is necessary to analyze the interference degree of the real echo signal to evaluate the recovery effect of the echo signal before and after the implementation of the anti-interference transmission waveform action. The state s required for constructing the DQN model based on SINR and radar action t , defined as:

[0167] s t =[a t ,SINR t ];

[0168]

[0169] Where SINR is the signal to interference and noise ratio, P s is the echo power, P j is the interference power, P n is the noise power.

[0170] Reward function r in the generation process of emission waveform selection strategy t It is used to control the action benefits and strategy tendencies of the radar agent. In order to characterize the characteristic relationship between the jammer and the radar signal, SINR is used to transform the reward function r t , defined as:

[0171]

[0172] Where SINR is the signal to interference and noise ratio, t cheat is the time when the radar launches the false target, t observeThe larger the SINR, the easier it is for the receiver to detect the target signal, and the better the radar system performance. Generally speaking, when the SINR is above 10dB, the radar system performance can be considered good.

[0173] DQN is used to solve the optimal transmission waveform strategy under incomplete interference information. The DQN network structure and training process diagram in the embodiment are shown in the following figure. Figure 4 The specific implementation steps are as follows:

[0174] (1) Initialization parameters;

[0175] (2) Randomly initialize the radar jamming signal state;

[0176] (3) Entering the anti-interference transmission waveform selection module, selecting the best anti-interference transmission waveform output by the estimation value network;

[0177] (4) Enter the radar to execute the anti-interference transmission waveform action module, obtain the single-step return value, and perform state transition through the target value network to obtain the next state;

[0178] (5) Enter the experience data storage module and store the data obtained from the training experiment into the memory bank;

[0179] (6) Determine the number of samples in the memory bank;

[0180] (7) Determine whether the current number of training times meets the minimum network training condition. If so, proceed to step 8 for network training; otherwise, proceed to step 9.

[0181] (8) First, select M sample data from the memory bank, obtain the estimated Q value through the estimated value network, obtain the target Q value through the target value network, solve the loss function, train the network, and update the network parameters;

[0182] (9) If it is greater than the number of iterations in this round, go to step 2 to conduct the next round of training experiments; otherwise, go to step 3 to continue this round of experiments.

[0183] Specifically, during the radar anti-interference process, the optimal anti-interference transmission waveform corresponding to the identified interference signal is intelligently determined from the anti-interference transmission waveform strategy resource library based on the anti-interference countermeasure principle.

[0184] The DQN network training decision loss function is:

[0185]

[0186] Where r t is the reward function, γ is the discount factor, q(s t+1 ,a;;θ -) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter; use the chain rule to find the partial derivative of the loss function with respect to the network parameter, and update the weight according to the back propagation algorithm.

[0187] The optimal transmission waveform parameters are adaptively optimized through the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p , specifically including:

[0188] In the original MAB, each lever is independent. When the levers are not independent, they are parameterized to form a parameterized MAB. There is a clear correspondence between the radar's cognitive transmit waveform strategy selection and the parameterized levers. The radar has K preset active transmit waveform strategies, corresponding to the K levers. The radar's transmit strategy is represented and driven by a low-dimensional vector. The radar aims to maximize the cumulative number of detections, which corresponds to minimizing the regret value.

[0189] The regret value Regret(T) in the MAB algorithm is defined as:

[0190]

[0191] Where A is the action set, T is the number of rounds, μ(·) is the expectation of pulling the lever, and x(·) is the policy representation.

[0192] Finally, the detection module uses a CFAR detector to detect the optimal transmission waveform and generate the optimal anti-interference waveform and parameters.

[0193] Under the assumption of ideal target detection, sending the pulse compression result to the CFAR detector yields a clear target peak detection result. However, under non-ideal interference conditions, noise interference can cause the CFAR detector to be unable to effectively extract the target peak, resulting in a false target judgment. Deceptive interference can create multiple false target peaks, causing the CFAR detector to detect multiple peak targets.

[0194] Set the decision index of the CFAR detector to:

[0195]

[0196] Where n is the number of targets detected by the CFAR detector.

[0197] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB, characterized in that the steps include: Based on the radar transmission signal and the interference signal, obtain the radar interference time-frequency diagram; Based on the radar interference time-frequency diagram, simulating the radar countermeasure process and building a transmission waveform strategy library; The waveform strategy library is solved using the DQN algorithm, and the solution is optimized using the MAB algorithm to obtain the optimal transmission waveform. The steps include: The states required to construct the DQN model based on SINR and radar motion , defined as: ； ； Where, is the signal-to-interference-and-noise ratio, is the echo power, is the interference power, is the noise power, For radar action; Using SINR to construct reward function Control the action benefits and strategy tendencies of the radar agent: ； Where SINR is the signal-to-interference-and-noise ratio, is the time when the radar launches the false target, The larger the SINR, the easier it is for the target signal to be detected by the receiver, and the better the performance of the radar system. Finally, DQN is used to solve the optimal transmission waveform strategy under incomplete interference information; The optimal transmission waveform is detected by using a CFAR detector to generate the optimal anti-interference waveform and parameters; the optimal transmission waveform parameters are adaptively optimized through a strategy parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency ,bandwidth and pulse width ; Among them, the regret value in the MAB algorithm Defined as: ； Where, For action sets, For rounds, Expectations for the tie rod, For strategy representation, For radar action.

2. The method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB according to claim 1 is characterized in that: The method for obtaining the radar interference time-frequency map includes: Establish the radar transmission signal and interference signal model of the frequency agile system, where the radar transmission signal is an agile LFM waveform: ； Where, , is the instantaneous power of radar transmission, is a rectangular window function, , is the pulse width, is the pulse repetition period, is the signal bandwidth, is the number of pulses, For the The carrier frequency of the pulse; The jamming signal is a deceptive and suppressive single jamming, including: distance deception jamming, speed deception jamming, narrowband pulse jamming, intermittent sampling direct forwarding jamming, intermittent sampling cyclic forwarding jamming, intermittent sampling repeated forwarding jamming, non-uniform intermittent sampling repeated forwarding jamming, modulated intermittent sampling repeated forwarding jamming, dense false target jamming, spectrum dispersion jamming, intelligent noise jamming and noise convolution jamming.

3. The method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB according to claim 2 is characterized in that: After generating the radar interference time-frequency diagram, a dynamic feature extraction module based on zero-memory incremental learning is constructed to sense the spectrum characteristics of the interference signal in real time and generate an interference feature vector. The steps include: The echo signal at the receiving end of the radar system is recorded as: ； Where, is the target signal, is the interference signal, is the noise signal; STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is: ； Where, is the radar echo signal, is the window function, * is the conjugate operation, For time, is the frequency, and τ represents the time domain integration time variable.

4. The method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB according to claim 3 is characterized in that: The constructed dynamic feature extraction module adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples into a hyperdimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

5. The method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB according to claim 1 is characterized in that: The constructed transmission waveform strategy library includes: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

6. The method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB according to claim 1, characterized in that: The DQN network training decision loss function is: ； Where, is the reward function, is the discount factor, is the target value network output, is the estimated value network output, are the estimated network parameters, is the target value network parameter, The status of the radar.

7. The method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB according to claim 1, characterized in that: The decision index of the CFAR detector is: ； Where n is the number of targets detected by the CFAR detector.

8. A cognitive radar active anti-interference waveform generation system based on DQN and MAB, the system is used to implement the method according to any one of claims 1 to 7, characterized in that: include: Acquisition module, construction module, solution module and detection module; The acquisition module is used to obtain a radar interference time-frequency diagram based on the radar transmission signal and the interference signal; The construction module is used to simulate the radar countermeasure process and build a transmission waveform strategy library based on the radar interference time-frequency diagram; The solution module is used to solve the waveform strategy library using the DQN algorithm and optimize the solution result using the MAB algorithm to obtain the optimal transmission waveform; The detection module is used to detect the optimal transmission waveform using a CFAR detector to generate an optimal anti-interference waveform and parameters.

Images