Cognitive radar active anti-interference waveform generation method and system based on DQN and MAB

Abstract

The invention discloses a cognitive radar active anti-interference waveform generation method and system based on DQN and MAB, and the method comprises the steps: obtaining a radar interference time-frequency diagram based on a radar transmitting signal and an interference signal; based on the radar interference time-frequency diagram, simulating a radar confrontation process and constructing a transmitted waveform strategy library; solving the waveform strategy library by using a DQN algorithm, and optimizing a solving result by using an MAB algorithm to obtain an optimal transmitted waveform; and a CFAR detector is adopted to detect the optimal emission waveform, and the optimal anti-interference waveform and parameters are generated. According to the method, the global strategy search of the DQN and the local parameter optimization of the MAB are fused, the anti-interference dynamic adaptation problem in a multi-pulse time sequence interaction scene is effectively solved, and the method is suitable for the real-time anti-interference requirements of an airborne radar electronic countermeasure system and a complex electromagnetic environment.

Description

Technical Field

[0001] The present invention relates to the field of radar electronic countermeasure technology, and in particular to a method and system for generating active anti-interference waveforms of a cognitive radar based on DQN and MAB. Background Art

[0002] Radar plays a key role in remotely acquiring battlefield situation and seizing information control in modern battlefields. However, active interference, especially main lobe interference, greatly restricts the normal operation and effectiveness of radar, which in turn affects the precision strike on enemy targets and the accuracy of military command. Current anti-interference research focuses on the design of passive signal processing algorithms and active waveform transmission of radar, but it mainly targets fixed and static interference sources, and fails to fully consider the pseudo-random, highly dynamic, intelligent and composite emission behaviors that the jammer may adopt in the game confrontation. As jammers develop towards intelligence and collaboration, the existing single static active and passive countermeasures have gradually become inadequate.

[0003] For the next generation of cognitive and collaborative electronic countermeasures, it is emphasized that the game attributes of interference must be fully considered when designing radar anti-interference measures. Existing radar countermeasures have problems such as insufficient dynamic adaptability to interference, low efficiency in waveform parameter optimization, and weak composite interference suppression capabilities. The modeling assumptions of the jammer opponent are too ideal, making it difficult for the policy learning algorithm to generalize. In addition, the sample efficiency of the policy learning algorithm is low, and it fails to fully utilize the multi-dimensional emission degrees of freedom of the radar. Summary of the invention

[0004] In order to solve the technical problems in the above background, the present invention provides a method for generating active anti-interference waveform of cognitive radar based on DQN and MAB, the steps comprising:

[0005] Based on the radar transmission signal and the interference signal, obtain the radar interference time-frequency diagram;

[0006] Based on the radar interference time-frequency diagram, simulating the radar countermeasure process and building a transmission waveform strategy library;

[0007] The waveform strategy library is solved by using the DQN algorithm, and the solution is optimized by using the MAB algorithm to obtain the optimal transmission waveform;

[0008] The optimal transmission waveform is detected by using a CFAR detector to generate an optimal anti-interference waveform and parameters.

[0009] Preferably, the method for obtaining the radar interference time-frequency diagram includes:

[0010] The radar transmission signal and interference signal model of the frequency agile system is established, where the radar transmission signal is an agile LFM waveform:

[0011]

[0012] In the formula, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse;

[0013] The jamming signal is a deceptive and suppressive single jamming, including: distance deception jamming, speed deception jamming, narrowband pulse jamming, intermittent sampling direct forwarding jamming, intermittent sampling cyclic forwarding jamming, intermittent sampling repeated forwarding jamming, non-uniform intermittent sampling repeated forwarding jamming, modulated intermittent sampling repeated forwarding jamming, dense false target jamming, spectrum dispersion jamming, intelligent noise jamming and noise convolution jamming.

[0014] Preferably, after generating the radar interference time-frequency diagram, a dynamic feature extraction module based on zero-memory incremental learning is constructed to sense the spectrum characteristics of the interference signal in real time and generate an interference feature vector, the steps comprising:

[0015] The echo signal at the receiving end of the radar system is recorded as:

[0016] R(t)=S(t)+J(t)+N(t);

[0017] In the formula, S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal;

[0018] STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is:

[0019] STFT(t,f)=∫R(τ)g * (τ-t)e -j2πfτ dτ;

[0020] Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

[0021] Preferably, the constructed dynamic feature extraction module adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples to a hyperdimensional feature space to form a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

[0022] Preferably, the constructed transmission waveform strategy library includes: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile change signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

[0023] Preferably, the step of solving the waveform strategy library using the DQN algorithm includes:

[0024] The states s required to construct the DQN model based on SINR and radar action t , defined as:

[0025] s t =[a t ,SINR t ];

[0026]

[0027] Where SINR is the signal to interference noise ratio, P s is the echo power, P j is the interference power, P n is the noise power, a t For radar action;

[0028] Using SINR to construct reward function r t Control the action benefits and strategy tendencies of the radar agent:

[0029]

[0030] Where SINR is the signal to interference noise ratio, t cheat is the time when the radar launches the false target, t observe is the jammer observation window time; the larger the SINR, the easier it is for the target signal to be detected by the receiver, and the better the performance of the radar system;

[0031] Finally, DQN is used to solve the optimal transmission waveform strategy under incomplete interference information.

[0032] Preferably, the DQN network training decision loss function is:

[0033]

[0034] In the formula, r t is the reward function, γ is the discount factor, q(s t+1 ,a′;θ -) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter.

[0035] Preferably, the optimal transmission waveform parameters are adaptively optimized through the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p ; Among them, the regret value Regret(T) in the MAB algorithm is defined as:

[0036]

[0037] Where A is the action set, T is the round, μ(·) is the expectation of pulling the lever, and x(·) is the strategy representation.

[0038] Preferably, the decision index of the CFAR detector is:

[0039]

[0040] Where n is the number of targets detected by the CFAR detector.

[0041] The present invention also provides a cognitive radar active anti-interference waveform generation system based on DQN and MAB, the system is used to implement the above method, including: an acquisition module, a construction module, a solution module and a detection module;

[0042] The acquisition module is used to obtain a radar interference time-frequency diagram based on the radar transmission signal and the interference signal;

[0043] The construction module is used to simulate the radar countermeasure process and construct a transmission waveform strategy library based on the radar interference time-frequency diagram;

[0044] The solution module is used to solve the waveform strategy library using the DQN algorithm, and optimize the solution result using the MAB algorithm to obtain the optimal transmission waveform;

[0045] The detection module is used to detect the optimal transmission waveform using a CFAR detector to generate an optimal anti-interference waveform and parameters.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention integrates the global strategy search of DQN and the local parameter optimization of MAB, effectively solving the problem of dynamic adaptation of anti-interference in multi-pulse timing interaction scenarios, and is suitable for airborne radar electronic countermeasure systems and real-time anti-interference requirements in complex electromagnetic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a method framework of an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of radar countermeasure process modeling according to an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the DQN network structure and training process according to an embodiment of the present invention;

[0053] Figure 5 A sample time-frequency diagram randomly selected from each interference signal in an embodiment of the present invention;

[0054] Figure 6 A sample time-frequency diagram randomly selected from each transmission signal in the transmission waveform decision library of the embodiment of the present invention;

[0055] Figure 7 A training loss curve diagram of the DQN training decision process in an embodiment of the present invention;

[0056] Figure 8 It is a curve diagram of detection times and detection probability in the MAB parameter optimization process of an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] In order to facilitate the description of the content of the method of the present invention, the following terms are explained.

[0060] Term 1: Deep Q-Network (DQN);

[0061] Term 2: Multi-armed Bandit (MAB);

[0062] Term 3: Linear Frequency Modulation (LFM);

[0063] Term 4: Markov Decision Process (MDP);

[0064] Term 5: Signal to Interference plus Noise Ratio (SINR);

[0065] Term 6: Constant False Alarm Rate (CFAR).

[0066] Embodiment 1

[0067] like Figure 1 , Figure 2 FIG. 1 is a schematic diagram of the method flow of this embodiment, and the steps include:

[0068] S1. Based on the radar transmission signal and the interference signal, obtain the radar interference time-frequency diagram.

[0069] A frequency agile radar transmission signal and interference signal model is established, and the bandwidth and time width can be adjusted. The radar transmission signal is an agile LFM waveform, and the interference signal is a deceptive and suppressive single interference.

[0070] The agile LFM waveform of the radar within a processing interval can be expressed as:

[0071]

[0072] In the formula, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse.

[0073] In a further embodiment, the radar jamming signal is a deceptive and suppressive single jamming, including distance deception jamming (Distance Deception Jamming, DDJ), velocity deception jamming (Velocity Deception Jamming, VDJ), narrow-band pulse jamming (Narrow Pulse Jamming, NPJ), intermittent sampling direct jamming (Intermittent Sampling Direct Jamming, ISDJ), intermittent sampling loop jamming (Intermittent Sampling Loop Jamming, ISLJ), intermittent sampling repeated forwarding jamming (Intermittent Sampling Repeater Jamming, ISRJ), non-uniform intermittent sampling repeated forwarding jamming (NUISRJ), modulated intermittent sampling repeated forwarding jamming (Modulated Intermittent Sampling Repeater Jamming, MISRJ), dense false target jamming (DFTJ), smeared spectrum jamming (Smeared Spectrum Jamming, SMSPJ), smart noise jamming (SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming Jamming (SNJ) and noise convolution jamming (NCJ) 12 kinds, in the embodiment, a sample time-frequency diagram randomly selected from each interference signal is as follows Figure 5 shown.

[0074] Radar instantaneous transmission power P T The maximum peak transmission power P max And the average power maximum value P avg The constraint is calculated as follows:

[0075]

[0076] When the pulse width is short, the radar uses P max When the pulse width exceeds a critical point, P T Affected by P avg The peak power is constrained by P max The radar transmit waveform strategy determines the T of each LFM waveform. p , B and f c (n).

[0077] Radar jamming signals are deceptive and suppressive single jammers, including distance deception jammers, speed deception jammers, narrowband pulse jammers, intermittent sampling direct forwarding jammers, intermittent sampling cyclic forwarding jammers, intermittent sampling repeated forwarding jammers, non-uniform intermittent sampling repeated forwarding jammers, modulated intermittent sampling repeated forwarding jammers, dense false target jammers, spectrum dispersion jammers, intelligent noise jammers, and noise convolution jammers.

[0078] The radar interference time-frequency diagram is obtained based on the radar echo data, and the spectral characteristics of the interference signal are perceived in real time by building a dynamic feature extraction module based on zero-memory incremental learning to generate an interference feature vector. The specific steps are as follows:

[0079] The echo signal at the receiving end of the radar system is generally a combination of the target signal, the interference signal and the noise signal, which can be expressed as:

[0080] R(t)=S(t)+J(t)+N(t);

[0081] In the formula, S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal;

[0082] STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is:

[0083] STFT(t,f)=∫R(τ)g * (τ-t)e -j2πfτ dτ;

[0084] Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

[0085] The dynamic feature extraction module based on zero-memory incremental learning adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples to the hyper-dimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

[0086] S2. Based on the radar interference time-frequency diagram, simulate the radar countermeasure process and build a transmission waveform strategy library.

[0087] Cognitive intelligent radar has a closed-loop learning and self-evolving radar architecture, and has the capabilities of environmental adaptive perception, arbitrary transmission and reception design, intelligent processing and resource scheduling. Figure 3As shown in the figure, the radar receives the interference signal. First, the signal characteristics of different interferences in the time-frequency domain in the radar interference time-frequency diagram are analyzed, and the interference characteristic parameters of the interference type are perceived; at the same time, an interference type resource library and an anti-interference transmission waveform resource library are established; then, the existing anti-interference transmission waveforms in the radar anti-interference transmission waveform resource library are selected; finally, the effectiveness of the anti-interference transmission waveform is evaluated.

[0088] The anti-interference transmission waveform strategy resource library is designed based on the basic LFM waveform for complex modulation of multi-dimensional parameters such as frequency, phase, modulation frequency polarity, and repetition rate. The strong anti-competitive complex modulation waveforms constructed include: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

[0089] Give full play to the advantages of active confrontation and increase the difficulty of interception analysis and injection interference of enemy jammers. In this embodiment, a sample time-frequency diagram randomly selected from each transmission signal in the transmission waveform decision library is as follows: Figure 6 shown.

[0090] After the transmission waveform strategy library is built, the action set, state set and SINR-based reward function are set through the MDP idea, including:

[0091] The elements of the MDP include the radar agent action a t , state s t and the reward function r t In the MDP problem of cognitive radar active anti-interference, the action set A corresponds to the radar's anti-interference transmission waveform strategy library, the state set S corresponds to the radar's interference type library, and the single-step return value R is abstracted as SINR. By designing the corresponding algorithm structure, intelligent decision-making of the transmission waveform in the radar anti-interference process can be achieved;

[0092] Radar Actiona t Defined as:

[0093] a t =A[t];

[0094] Where A[t] is the action set composed of the anti-interference transmission waveform strategy library.

[0095] S3. Use the DQN algorithm to solve the waveform strategy library, and use the MAB algorithm to optimize the solution to obtain the optimal transmission waveform.

[0096] Based on the established frequency agile radar transmission signal and interference signal model, after completing the construction of the radar echo signal R(t), it is necessary to analyze the interference degree of the real echo signal to evaluate the recovery effect of the echo signal before and after the anti-interference transmission waveform action is implemented. The state s required to construct the DQN model based on SINR and radar action t , defined as:

[0097] s t =[a t ,SINR t ];

[0098]

[0099] Where SINR is the signal to interference noise ratio, P s is the echo power, P j is the interference power, P n is the noise power, a t For radar action.

[0100] Reward function r in the generation process of emission waveform selection strategy t It is used to control the action benefits and strategy tendencies of the radar agent. In order to characterize the characteristic relationship between the jammer and the radar signal, SINR is used to convert the reward function r t , defined as:

[0101]

[0102] Where SINR is the signal to interference noise ratio, t cheat is the time when the radar launches the false target, t observe is the jammer observation window time; the larger the SINR, the easier it is for the target signal to be detected by the receiver, and the better the performance of the radar system. Generally speaking, when the SINR is above 10dB, the performance of the radar system can be considered good.

[0103] DQN is used to solve the optimal transmission waveform strategy under incomplete interference information. The DQN network structure and training process diagram in the embodiment are shown in Figure 4 As shown, the specific implementation steps are as follows:

[0104] (1) Initialization parameters;

[0105] (2) Randomly initialize the radar jammer signal state;

[0106] (3) Entering the anti-interference transmission waveform selection module, selecting the best anti-interference transmission waveform output by the estimation value network;

[0107] (4) Enter the radar to execute the anti-interference transmission waveform action module, obtain the single-step return value, and perform state transition through the target value network to obtain the next state;

[0108] (5) Enter the experience data storage module and store the data obtained from the training experiment into the memory bank;

[0109] (6) Determine the number of samples in the memory bank;

[0110] (7) Determine whether the current number of training times meets the minimum network training condition. If so, proceed to step 8 for network training; otherwise, proceed to step 9;

[0111] (8) First, select M sample data from the memory bank, obtain the estimated Q value through the estimated value network, obtain the target Q value through the target value network, solve the loss function, train the network, and update the network parameters;

[0112] (9) If it is greater than the number of iterations in this round, proceed to step 2 for the next round of training experiments; otherwise, proceed to step 3 to continue this round of experiments.

[0113] Specifically, in the radar anti-interference process, according to the anti-interference countermeasure principle, an optimal anti-interference transmission waveform corresponding to the identified interference signal is intelligently determined from the anti-interference transmission waveform strategy resource library;

[0114] The DQN network training decision loss function is:

[0115]

[0116] In the formula, r t is the reward function, γ is the discount factor, q(s t+1 ,a;;θ - ) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter; use the chain rule to find the partial derivative of the loss function with respect to the network parameter, and update the weight according to the back propagation algorithm.

[0117] The optimal transmission waveform parameters are adaptively optimized through the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p , including:

[0118] Each lever in the original MAB is independent. When the levers are not independent, the parameterized MAB is obtained by parameterizing the levers. There is an obvious correspondence between the radar cognitive emission waveform strategy selection and the parameterized levers. The radar presets K kinds of active emission waveform strategies, corresponding to K levers; the radar emission strategy is characterized and driven by a low-dimensional vector. The radar hopes to maximize the cumulative number of detections, which corresponds to minimizing the regret value.

[0119] The regret value Regret(T) in the MAB algorithm is defined as:

[0120]

[0121] Where A is the action set, T is the round, μ(·) is the expectation of pulling the lever, and x(·) is the strategy representation.

[0122] S4. Use the CFAR detector to detect the optimal transmission waveform and generate the optimal anti-interference waveform and parameters.

[0123] Under the assumption of ideal target detection, a clear target peak detection result will be obtained after the pulse compression result is sent to the CFAR detector. However, under non-ideal interference conditions, noise interference will cause the CFAR detector to fail to effectively extract the target peak and make a judgment of no target, and deception interference will form multiple false target peaks, causing the CFAR detector to detect multiple peak targets.

[0124] The decision index of the CFAR detector is set as:

[0125]

[0126] Where n is the number of targets detected by the CFAR detector.

[0127] Embodiment 2

[0128] The effects of the present invention will be further illustrated by simulation in conjunction with this embodiment.

[0129] 1. Simulation conditions

[0130] In this embodiment, the above 12 radar active interference signals and 8 anti-interference emission waveforms are used to conduct a decision-making performance experiment of the radar anti-interference method. According to the above experimental steps and environment, the experimental parameters are set as follows: the greed factor is 0.75, the learning rate is 0.7, and the discount factor is 0.8. A 3-layer network is used, and the network parameters are randomly initialized. The algorithm of the present invention is applied to a specific interference type. In each confrontation selection process, one of the 8 anti-interference emission waveforms is selected and the parameters are optimized. 10 rounds of confrontation experiments are designed, and 500 confrontation decisions are made in each round. The entire experimental process is run on a server configured with an Intel i7-12700FCPU and an NVIDIA GeForce RTX 4080Ti GPU (16G), and the algorithm is implemented in the CUDA 13.1 environment using the Python3.8 programming language.

[0131] 2. Simulation results

[0132] Table 1

[0133] Interference scenario Optimal anti-interference waveform Intermittent sampling repeated forwarding interference Intra-pulse Costas step-linear frequency modulation waveform Spectral Dispersive Interference Intra-pulse random step linear frequency modulation waveform Intelligent noise jammer Pulse-to-pulse Costas step-linear frequency modulation waveform

[0134] Combined with Table 1 and Figure 7 and Figure 8 It can be seen that the method of the present invention integrates the global strategy search of DQN and the local parameter optimization of MAB. In 12 typical interference scenarios, the average decision time is less than 40ms, the detection probability is improved from 0.72 to 0.91, and the model training decision process converges quickly, realizing the coordinated anti-interference of radar waveform generation and parameter optimization. It is significantly superior to the existing technology in terms of dynamic interference suppression, resource efficiency, real-time performance, etc., and has broad military and civilian prospects.

[0135] Embodiment 3

[0136] The present invention also provides a cognitive radar active anti-interference waveform generation system based on DQN and MAB, including: an acquisition module, a construction module, a solution module and a detection module; the acquisition module is used to obtain a radar interference time-frequency diagram based on a radar transmission signal and an interference signal; the construction module is used to simulate a radar confrontation process and construct a transmission waveform strategy library based on the radar interference time-frequency diagram; the solution module is used to solve the waveform strategy library using a DQN algorithm, and optimize the solution result using a MAB algorithm to obtain an optimal transmission waveform; the detection module is used to detect the optimal transmission waveform using a CFAR detector to generate an optimal anti-interference waveform and parameters.

[0137] The following will explain in detail how the present invention solves technical problems in real life in conjunction with this embodiment.

[0138] The acquisition module obtains the radar interference time-frequency diagram based on the radar transmission signal and the interference signal.

[0139] A frequency agile radar transmission signal and interference signal model is established, and the bandwidth and time width can be adjusted. The radar transmission signal is an agile LFM waveform, and the interference signal is a deceptive and suppressive single interference.

[0140] The agile LFM waveform of the radar within a processing interval can be expressed as:

[0141]

[0142] In the formula, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse.

[0143] In a further embodiment, the radar jamming signal is a deceptive and suppressive single jamming, including distance deception jamming (Distance Deception Jamming, DDJ), velocity deception jamming (Velocity Deception Jamming, VDJ), narrow-band pulse jamming (Narrow Pulse Jamming, NPJ), intermittent sampling direct jamming (Intermittent Sampling Direct Jamming, ISDJ), intermittent sampling loop jamming (Intermittent Sampling Loop Jamming, ISLJ), intermittent sampling repeated forwarding jamming (Intermittent Sampling Repeater Jamming, ISRJ), non-uniform intermittent sampling repeated forwarding jamming (NUISRJ), modulated intermittent sampling repeated forwarding jamming (Modulated Intermittent Sampling Repeater Jamming, MISRJ), dense false target jamming (DFTJ), smeared spectrum jamming (Smeared Spectrum Jamming, SMSPJ), smart noise jamming (SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming, SmartNoise Jamming Jamming (SNJ) and noise convolution jamming (NCJ) 12 kinds, in the embodiment, a sample time-frequency diagram randomly selected from each interference signal is as follows Figure 5shown.

[0144] Radar instantaneous transmission power P T The maximum peak transmission power P max And the average power maximum value P avg The constraint is calculated as follows:

[0145]

[0146] When the pulse width is short, the radar uses P max When the pulse width exceeds a critical point, P T Affected by P avg The peak power is constrained by P max The radar transmit waveform strategy determines the T of each LFM waveform. p , B and f c (n).

[0147] Radar jamming signals are deceptive and suppressive single jammers, including distance deception jammers, speed deception jammers, narrowband pulse jammers, intermittent sampling direct forwarding jammers, intermittent sampling cyclic forwarding jammers, intermittent sampling repeated forwarding jammers, non-uniform intermittent sampling repeated forwarding jammers, modulated intermittent sampling repeated forwarding jammers, dense false target jammers, spectrum dispersion jammers, intelligent noise jammers, and noise convolution jammers.

[0148] The radar interference time-frequency diagram is obtained based on the radar echo data, and the spectral characteristics of the interference signal are perceived in real time by building a dynamic feature extraction module based on zero-memory incremental learning to generate an interference feature vector. The specific steps are as follows:

[0149] The echo signal at the receiving end of the radar system is generally a combination of the target signal, the interference signal and the noise signal, which can be expressed as:

[0150] R(t)=S(t)+J(t)+N(t);

[0151] In the formula, S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal;

[0152] STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is:

[0153] STFT(t,f)=∫R(τ)g * (τ-t)e -j2πfτ dτ;

[0154] Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

[0155] The dynamic feature extraction module based on zero-memory incremental learning adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples to the hyper-dimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

[0156] The construction module simulates the radar countermeasure process and builds a transmission waveform strategy library based on the radar interference time-frequency diagram.

[0157] Cognitive intelligent radar has a closed-loop learning and self-evolving radar architecture, and has the capabilities of environmental adaptive perception, arbitrary transmission and reception design, intelligent processing and resource scheduling. Figure 3 As shown in the figure, the radar receives the interference signal. First, the signal characteristics of different interferences in the time-frequency domain in the radar interference time-frequency diagram are analyzed, and the interference characteristic parameters of the interference type are perceived; at the same time, an interference type resource library and an anti-interference transmission waveform resource library are established; then, the existing anti-interference transmission waveforms in the radar anti-interference transmission waveform resource library are selected; finally, the effectiveness of the anti-interference transmission waveform is evaluated.

[0158] The anti-interference transmission waveform strategy resource library is designed based on the basic LFM waveform for complex modulation of multi-dimensional parameters such as frequency, phase, modulation frequency polarity, and repetition rate. The strong anti-competitive complex modulation waveforms constructed include: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

[0159] Give full play to the advantages of active confrontation and increase the difficulty of interception analysis and injection interference of enemy jammers. In this embodiment, a sample time-frequency diagram randomly selected from each transmission signal in the transmission waveform decision library is as follows: Figure 6 shown.

[0160] After the transmission waveform strategy library is built, the action set, state set and SINR-based reward function are set through the MDP idea, including:

[0161] The elements of the MDP include the radar agent action a t , state s t and the reward function r tIn the MDP problem of cognitive radar active anti-interference, the action set A corresponds to the radar's anti-interference transmission waveform strategy library, the state set S corresponds to the radar's interference type library, and the single-step return value R is abstracted as SINR. By designing the corresponding algorithm structure, intelligent decision-making of the transmission waveform in the radar anti-interference process can be achieved;

[0162] Radar Actiona t Defined as:

[0163] a t =A[t];

[0164] Where A[t] is the action set composed of the anti-interference transmission waveform strategy library.

[0165] The solution module uses the DQN algorithm to solve the waveform strategy library, and uses the MAB algorithm to optimize the solution results to obtain the optimal transmission waveform.

[0166] Based on the established frequency agile radar transmission signal and interference signal model, after completing the construction of the radar echo signal R(t), it is necessary to analyze the interference degree of the real echo signal to evaluate the recovery effect of the echo signal before and after the anti-interference transmission waveform action is implemented. The state s required to construct the DQN model based on SINR and radar action t , defined as:

[0167] s t =[a t ,SINR t ];

[0168]

[0169] Where SINR is the signal to interference noise ratio, P s is the echo power, P j is the interference power, P n is the noise power.

[0170] Reward function r in the generation process of emission waveform selection strategy t It is used to control the action benefits and strategy tendencies of the radar agent. In order to characterize the characteristic relationship between the jammer and the radar signal, SINR is used to convert the reward function r t , defined as:

[0171]

[0172] Where SINR is the signal to interference noise ratio, t cheat is the time when the radar launches the false target, t observeis the jammer observation window time; the larger the SINR, the easier it is for the target signal to be detected by the receiver, and the better the performance of the radar system. Generally speaking, when the SINR is above 10dB, the performance of the radar system can be considered good.

[0173] DQN is used to solve the optimal transmission waveform strategy under incomplete interference information. The DQN network structure and training process diagram in the embodiment are shown in Figure 4 As shown, the specific implementation steps are as follows:

[0174] (1) Initialization parameters;

[0175] (2) Randomly initialize the radar jammer signal state;

[0176] (3) Entering the anti-interference transmission waveform selection module, selecting the best anti-interference transmission waveform output by the estimation value network;

[0177] (4) Enter the radar to execute the anti-interference transmission waveform action module, obtain the single-step return value, and perform state transition through the target value network to obtain the next state;

[0178] (5) Enter the experience data storage module and store the data obtained from the training experiment into the memory bank;

[0179] (6) Determine the number of samples in the memory bank;

[0180] (7) Determine whether the current number of training times meets the minimum network training condition. If so, proceed to step 8 for network training; otherwise, proceed to step 9;

[0181] (8) First, select M sample data from the memory bank, obtain the estimated Q value through the estimated value network, obtain the target Q value through the target value network, solve the loss function, train the network, and update the network parameters;

[0182] (9) If it is greater than the number of iterations in this round, proceed to step 2 for the next round of training experiments; otherwise, proceed to step 3 to continue this round of experiments.

[0183] Specifically, in the radar anti-interference process, according to the anti-interference countermeasure principle, an optimal anti-interference transmission waveform corresponding to the identified interference signal is intelligently determined from the anti-interference transmission waveform strategy resource library;

[0184] The DQN network training decision loss function is:

[0185]

[0186] In the formula, r t is the reward function, γ is the discount factor, q(s t+1 ,a;;θ -) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter; use the chain rule to find the partial derivative of the loss function with respect to the network parameter, and update the weight according to the back propagation algorithm.

[0187] The optimal transmission waveform parameters are adaptively optimized through the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p , including:

[0188] Each lever in the original MAB is independent. When the levers are not independent, the parameterized MAB is obtained by parameterizing the levers. There is an obvious correspondence between the radar cognitive emission waveform strategy selection and the parameterized levers. The radar presets K kinds of active emission waveform strategies, corresponding to K levers; the radar emission strategy is characterized and driven by a low-dimensional vector. The radar hopes to maximize the cumulative number of detections, which corresponds to minimizing the regret value.

[0189] The regret value Regret(T) in the MAB algorithm is defined as:

[0190]

[0191] Where A is the action set, T is the round, μ(·) is the expectation of pulling the lever, and x(·) is the strategy representation.

[0192] Finally, the detection module uses a CFAR detector to detect the optimal transmission waveform and generate the optimal anti-interference waveform and parameters.

[0193] Under the assumption of ideal target detection, a clear target peak detection result will be obtained after the pulse compression result is sent to the CFAR detector. However, under non-ideal interference conditions, noise interference will cause the CFAR detector to fail to effectively extract the target peak and make a judgment of no target, and deception interference will form multiple false target peaks, causing the CFAR detector to detect multiple peak targets.

[0194] The decision index of the CFAR detector is set as:

[0195]

[0196] Where n is the number of targets detected by the CFAR detector.

[0197] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for generating active anti-interference waveforms for cognitive radar based on DQN and MAB, characterized in that the steps include: Based on the radar transmission signal and the interference signal, obtain the radar interference time-frequency diagram; Based on the radar interference time-frequency diagram, simulating the radar countermeasure process and building a transmission waveform strategy library; The waveform strategy library is solved by using the DQN algorithm, and the solution is optimized by using the MAB algorithm to obtain the optimal transmission waveform; The optimal transmission waveform is detected by using a CFAR detector to generate an optimal anti-interference waveform and parameters.

2. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 1 is characterized in that: The method for obtaining the radar interference time-frequency diagram includes: The radar transmission signal and interference signal model of the frequency agile system is established, where the radar transmission signal is an agile LFM waveform: In the formula, P T is the instantaneous power of radar transmission, rect(·) is the rectangular window function, T p is the pulse width, T r is the pulse repetition period, B is the signal bandwidth, N is the number of pulses, f c (n) is the carrier frequency of the nth pulse; The jamming signal is a deceptive and suppressive single jamming, including: distance deception jamming, speed deception jamming, narrowband pulse jamming, intermittent sampling direct forwarding jamming, intermittent sampling cyclic forwarding jamming, intermittent sampling repeated forwarding jamming, non-uniform intermittent sampling repeated forwarding jamming, modulated intermittent sampling repeated forwarding jamming, dense false target jamming, spectrum dispersion jamming, intelligent noise jamming and noise convolution jamming.

3. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 2 is characterized in that: After generating the radar interference time-frequency diagram, a dynamic feature extraction module based on zero-memory incremental learning is constructed to perceive the spectrum characteristics of the interference signal in real time and generate an interference feature vector, the steps comprising: The echo signal at the receiving end of the radar system is recorded as: R(t)=S(t)+J(t)+N(t); In the formula, S(t) is the target signal, J(t) is the interference signal, and N(t) is the noise signal; STFT is used to perform time-frequency analysis on the radar echo signal to generate a time-frequency diagram. The expression of STFT is: STFT(t,f)=∫R(τ)g * (t-t)e -j2πfτ dτ; Where R(·) is the radar echo signal, g(·) is the window function, * is the conjugate operation, t is the time, f is the frequency, and τ represents the time domain integration time variable.

4. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 3 is characterized in that: The constructed dynamic feature extraction module adopts a 5-layer CNN structure to map the input radar interference time-frequency image samples to a hyperdimensional feature space, forming a set of 512-dimensional prototype vectors stored in the prototype space; in the prototype space, each interference category corresponds to a specific prototype vector.

5. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 1 is characterized in that: The constructed transmission waveform strategy library includes: linear frequency modulation signal waveform, intra-pulse random step linear frequency modulation signal waveform, intra-pulse frequency modulation positive and negative polarity agile change signal waveform, chaotic signal waveform, intra-pulse Costas step linear frequency modulation signal waveform, inter-pulse Costas step linear frequency modulation signal waveform, intra-pulse Costas step discrete frequency coding-inter-pulse Costas step signal waveform and intra-pulse Costas step linear frequency modulation-inter-pulse Costas step signal waveform.

6. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 1 is characterized in that: The steps of solving the waveform strategy library using the DQN algorithm include: The states s required to construct the DQN model based on SINR and radar action t , defined as: S t =[a t ,SINR t ]; Where SINR is the signal to interference noise ratio, P s is the echo power, P j is the interference power, P n is the noise power, a t For radar action; Using SINR to construct reward function r t Control the action benefits and strategy tendencies of the radar agent: Where SINR is the signal to interference noise ratio, t cheat is the time when the radar launches the false target, t observe is the jammer observation window time; the larger the SINR, the easier it is for the target signal to be detected by the receiver, and the better the performance of the radar system; Finally, DQN is used to solve the optimal transmission waveform strategy under incomplete interference information.

7. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 6 is characterized in that: The DQN network training decision loss function is: In the formula, r t is the reward function, γ is the discount factor, q(s t+1 ,a';θ - ) is the target value network output, q(s t ,a;θ) is the estimated network output, θ is the estimated network parameter, θ - is the target value network parameter.

8. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 7 is characterized in that: The optimal transmission waveform parameters are adaptively optimized through the strategic parameterized MAB algorithm, and the optimal parameter combination is dynamically selected, including the center frequency f c , bandwidth B and pulse width T p ; Among them, the regret value Regret(T) in the MAB algorithm is defined as: Where A is the action set, T is the round, μ(·) is the expectation of pulling the lever, and x(·) is the strategy representation.

9. The method for generating active anti-interference waveform of cognitive radar based on DQN and MAB according to claim 1 is characterized in that: The decision index of the CFAR detector is: Where n is the number of targets detected by the CFAR detector.

10. A cognitive radar active anti-interference waveform generation system based on DQN and MAB, the system is used to implement the method according to any one of claims 1 to 9, characterized in that: include: Acquisition module, construction module, solution module and detection module; The acquisition module is used to obtain a radar interference time-frequency diagram based on the radar transmission signal and the interference signal; The construction module is used to simulate the radar countermeasure process and construct a transmission waveform strategy library based on the radar interference time-frequency diagram; The solution module is used to solve the waveform strategy library using the DQN algorithm, and optimize the solution result using the MAB algorithm to obtain the optimal transmission waveform; The detection module is used to detect the optimal transmission waveform using a CFAR detector to generate an optimal anti-interference waveform and parameters.

Images