Radar anti-dense decoy jamming method and system based on deep reinforcement learning

Abstract

The application relates to the field of electronic communication technology and discloses a radar anti-dense false target jamming method and system based on deep reinforcement learning, wherein the steps of the radar anti-dense false target jamming method based on deep reinforcement learning are as follows: a dense false target jamming simulation model is constructed; a jamming state is defined; a reward function of anti-jamming reinforcement learning is designed to evaluate the anti-jamming effect reward after a certain action is taken; after feature extraction and processing of complex value echo signals received by the radar based on a twin network, the features are fused to obtain fused features; the fused features are input into a policy network to complete adaptive learning and optimization of the anti-jamming action and generate an optimal anti-jamming action; and the emission waveform of the radar is adjusted according to the optimal anti-jamming action; the application completes autonomous learning and artificial intelligence optimization of the anti-jamming action through deep reinforcement learning, effectively suppresses dense false target jamming, and improves radar target recognition and detection precision.

Description

Technical Field

[0001] This invention relates to the field of information and communication engineering technology, and in particular to a radar anti-dense decoy jamming method and system based on deep reinforcement learning. Background Technology

[0002] In the electromagnetic environment of modern battlefields, radars face increasingly complex electronic interference. Radar interference can be categorized into three main types: noise interference, deception interference, and composite interference. Among these, dense decoy interference, as an advanced form of deception interference, demonstrates significant tactical value in modern combat environments. With its high interference efficiency, strong concealment, and flexible implementation, it has become one of the main threats to various types of radars, including early warning, guidance, and fire control radars. This type of interference can sample, store, and forward radar echoes, and through modulation of parameters such as amplitude, time delay, and Doppler, generate a large number of decoy signals with amplitude, range, and velocity characteristics highly similar to real targets. These signals can enter the receiving channel from the radar main lobe, causing a sharp increase in the false alarm rate, the overwhelming of real targets, and the collapse of point-track aggregation and track correlation, severely impacting radar target detection, tracking, and identification performance.

[0003] Currently, traditional anti-jamming methods mostly rely on fixed thresholds, fixed waveforms, or manually preset rules, such as constant false alarm rate detection, conventional waveform agility, point filtering, and simple track correlation. These methods have significant shortcomings when facing dense false target interference: on the one hand, false targets and real targets have similar characteristics, and it is difficult to effectively distinguish them by conventional processing in the signal domain or data domain alone; on the other hand, the interference pattern and intensity can change dynamically, and traditional methods lack online decision-making and autonomous optimization capabilities, resulting in low intelligence, poor adaptability, and insufficient robustness, making it difficult to maintain detection performance continuously in a dynamic game electromagnetic environment.

[0004] Traditional radar anti-jamming methods and architectures lack an AI-optimized operating system and an intelligent, autonomous iterative optimization mechanism. In military electronic scenarios, dense false targets are highly similar to real targets, and the jamming patterns and intensity exhibit dynamic game-theoretic characteristics. Traditional anti-jamming methods (such as waveform agility and CFAR detection) rely on manually preset rules and lack online autonomous decision-making and adaptive optimization capabilities. This can easily lead to problems such as excessively high false alarm rates, missed detection of real targets, and track tracking failures, making them unable to effectively cope with the threat of dense false targets in complex battlefield environments. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and to provide a radar anti-dense false target interference method and system based on deep reinforcement learning. Through deep reinforcement learning, the anti-interference action is autonomously learned and intelligently optimized. The artificial intelligence optimization algorithm is used to realize dynamic anti-interference decision-making, output the optimal control action and adaptively adjust the radar transmission waveform, effectively suppress dense false target interference, and improve the radar target recognition accuracy and tracking stability.

[0006] The technical solution of this invention is as follows: On one hand, this invention discloses a radar anti-dense false target interference method based on deep reinforcement learning, which includes the following steps: S1. Construct a simulation model of dense false target interference to simulate the interaction between the radar system and dense false target interference. S2, Define the interference state; S3. Design a reward function for robust reinforcement learning. The evaluation of the anti-interference effect after taking a certain action is rewarded; S4. Based on the twin network, the complex-valued echo signal received by the radar is processed by feature extraction and then fused to obtain the fused features. ; S5, Integrating Features The input is fed into the policy network to complete the adaptive learning and optimization of anti-interference actions, and generate the optimal anti-interference actions; S6. Adjust the radar's transmission waveform according to the optimal anti-jamming action.

[0007] As can be seen from the above scheme, the present invention is based on a reinforcement learning framework. After processing the complex-valued echo signal received by the radar, the optimal anti-jamming action is output by a policy network. The present invention can cope with unknown and time-varying interference strategies through adaptive and data-driven approaches.

[0008] In S2, under dense false target forwarding interference, the interference state expression at time t is: ; in, This indicates the lower frequency of the signal transmitted by the jammer. Indicates signal bandwidth; Indicates the duration of the jamming reconnaissance window. This indicates the duration of the jamming window.

[0009] S1.1 The radar transmit signal is generated using linear frequency modulation, as shown in the following formula: ; in, It is the transmitted linear frequency modulated signal, where B is the bandwidth of the frequency modulation, T is the pulse width, t is the time variable, and j is the imaginary unit; S1.2, Simulate the echo signal of the target / dummy target, and introduce time delay and Doppler frequency shift. The formula for the echo signal is as follows: ; ; ; in, is the echo signal at time t; A is the echo amplitude including the attenuation factor; This indicates a Doppler frequency shift; The echo delay time is related to the target distance, therefore This indicates the time delay of the echo signal. This represents the radial velocity of the target relative to the radar. Indicates the radar signal wavelength, and , This indicates the radar carrier frequency, where c is the speed of light constant. When the target approaches the radar, >0, when far away <0; R represents the round-trip time of the signal, and R represents the distance between the target and the radar. S1.3, Construction A false target signal, and then The signals from individual false targets are linearly superimposed to form a composite, dense false target interference signal: ; ; in, This is the k-th false signal at time t. For its specific time delay, This is its specific Doppler frequency shift. Let be the echo amplitude of the kth echo containing the attenuation factor; S1.4 Draw the corresponding image.

[0010] S4 includes the following steps: S4.1 Acquire the echo signal received by the radar, wherein the echo signal is a complex-valued echo signal containing both real target echoes and dense false target interference signals. : ; in, The imaginary unit satisfies ; S4.2, the complex-valued echo signal real part sequence and imaginary part sequence Each Siamese network with the same structure is input into a separate network for feature extraction to obtain the real part features. Features of the imaginary part The definition of a twin network is as follows: ; ; in, Represents a convolutional network. Indicates a fully connected network. Then it is Activation function; S4.3, through an embedding layer with the same structure and ,Will and Mapping to the embedding space of the same dimension yields... and ,right and By splicing the data, the fused features are obtained. : ; ; in, It is a fused feature resulting from the concatenation of two features, where and These are the features mapped from the real and imaginary features, respectively.

[0011] S5 includes the following steps: S5.1, Based on fusion features Anti-interference actions are output to the network through strategy selection. The strategy selection network structure adopts a fully connected network, and outputs anti-interference actions. Q value: ; in Choose network parameters for the policy, where Q is the state-action value function. It is a policy network; S5.2, Based on fusion features Evaluate the network using a strategy Counter-interference actions The effectiveness is evaluated and the evaluation value is output, including the strategy evaluation network. A fully connected network is used, and the following formula is satisfied: ; in, For the strategy scoring network parameters, To assess the value of the action; S5.3 Maximize long-term cumulative rewards by updating network parameter outputs through Q-learning. : ; in, It is the target Q value. It is the instantaneous reward at time t, determined by the policy evaluation network. Based on fusion characteristics and actions roll out, This is a discount factor used to control the importance of future rewards. To maximize the value of all possible actions in the action space, ensure that the current action is the optimal action in the current state, and output the Q value to the target network; S5.4. The mean squared error loss function is used to measure the difference between the Q-value predicted by the network under the current strategy and the target Q-value. The formula is as follows: ; Among them, the loss function middle It is the current network input. This represents the expected sample size sampled from the experience replay buffer D, where the sample contains features of the current state. Execution of actions Instant rewards Features of the state at the next moment , It is the target Q value. It is the Q-value output by the network under the current policy selection; S5.5. Update the network parameters using gradient descent with the Adam optimizer to minimize the loss function: ; in, These are the trainable parameters in the network. For learning rate, This indicates that the loss function is applied to the parameters. gradient, This indicates an assignment operation, used for parameter updates.

[0012] S3 includes the following steps: S3.1 Calculate the target signal-to-interference-plus-noise ratio at the radar receiver. : ; in, This represents the radar cross-section of the target at this moment. To increase the channel gain of the radar jammer reaching the target. It is the power of the radar to receive ambient noise. Let n be the carrier frequency of the radar's nth pulse. The frequency of the jammer, It is the radar transmission power. It is the power of the target jammer, when , It is 1 if it is true, otherwise it is 0. S3.2, Reward Function Satisfy the following formula: ; in, It is the target detection probability. It is the false alarm rate. To obtain the signal-to-interference-plus-noise ratio of the target for radar, These represent the weighting coefficients for each indicator.

[0013] On the other hand, the present invention also discloses an anti-interference system, which includes a radar signal transmission module for simulating radar transmission waveforms and outputting radar transmission power. The carrier frequency of the nth pulse The system includes: a target and jamming signal generation module for simulating real target signals and dense fake target jamming signals; and a signal propagation module for simulating the signal propagation process in space and calculating the channel gain from radar to the target jammer. The signal receiving module is used to simulate radar receiving signals, receiving both real echo signals and dense false target interference signals, and outputting the target signal power and interference power of the receiver. and radar received ambient noise power . Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a detailed step diagram of step 1 in this invention; Figure 3 This is a flowchart of interference-resistant reinforcement learning; Figure 4 This is a schematic diagram illustrating the changes in radar transmitted signals; Figure 5 This is a schematic diagram illustrating the changes in the actual target received by the radar; Figure 6 This is a schematic diagram of the echo signal after pulse compression. Figure 7 It is the Doppler spectrum of the echo signal; Figure 8 It is an echo signal diagram of a single dummy target; Figure 9It is a composite signal diagram of multiple fake targets; Figure 10 It is a range-Doppler two-dimensional plot calculated from the echo of the real target; Figure 11 It is a range-Doppler two-dimensional map calculated from echoes of spurious targets. Detailed Implementation

[0015] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0016] like Figures 1 to 11 As shown, this invention is a radar anti-dense decoy jamming method based on deep reinforcement learning, which includes the following steps: S1. Construct a simulation model of dense false target interference to simulate the interaction between the radar system and dense false target interference. S2, Define the interference state; S3. Design a reward function for robust reinforcement learning. The evaluation of the anti-interference effect after taking a certain action is rewarded; S4. Based on the twin network, the complex-valued echo signal received by the radar is processed by feature extraction and then fused to obtain the fused features. ; S5, Integrating Features The input is fed into the policy network to complete the adaptive learning and optimization of anti-interference actions, and generate the optimal anti-interference actions; S6. Adjust the radar's transmission waveform according to the optimal anti-jamming action.

[0017] The jamming state refers to the radar state under dense decoy jamming, including current jamming environment information and radar system performance information. In radar countermeasures, dense decoy jamming is an electronic warfare tactic that disrupts the enemy's radar detection and tracking capabilities by generating a large number of false target signals. Its purpose is to make the radar unable to distinguish between real and false targets, thereby reducing the radar's detection performance.

[0018] In S2, under dense false target forwarding interference, the interference state expression at time t is: ; in, This indicates the lower frequency of the signal transmitted by the jammer. Indicates signal bandwidth; Indicates the duration of the jamming reconnaissance window. This indicates the duration of the jamming window. Assuming that the radar detection waveform is included in time t1, the jammer will copy the radar waveform extensively and continuously transmit it in time t2, causing the radar receiver to be in a state of resource oversaturation and unable to distinguish the feature information of the real target.

[0019] S1 includes the following steps: S1.1 Generating the radar transmission signal. The radar transmission signal is usually a modulated pulse signal, and the common modulation method is linear frequency modulation (LFM). The frequency of this signal changes linearly with time and is often used for radar transmission to obtain good range resolution. The formula is as follows: ; in, It is the transmitted linear frequency modulated signal, where B is the bandwidth of the frequency modulation, T is the pulse width, t is the time variable, and j is the imaginary unit; S1.2 Simulate the echo signal of the target / dummy target, and introduce time delay and Doppler frequency shift. That is, the transmitted signal is the result of the target reflection, propagation attenuation, Doppler frequency shift and other effects. In other words, the signal reflected back from the target has time delay and Doppler frequency shift, so it is introduced into the simulation model. The formula for the echo signal is as follows: ; ; ; in, It is the echo signal at time t; It is the echo amplitude (including the attenuation factor). This indicates a Doppler frequency shift, caused by the relative motion between the target and the radar; The echo delay time is related to the target distance, therefore This indicates the time delay of the echo signal. This represents the radial velocity of the target relative to the radar. Indicates the radar signal wavelength, and , This represents the radar carrier frequency, where c is the speed of light constant (approximately 3 × 10⁸ m / s). When the target approaches the radar, >0, when far away <0; R represents the round-trip time of the signal, and R represents the distance between the target and the radar. Since the radar transmits a signal to the target and reflects it back, the total distance is 2R. S1.3, Construction A false target signal, and then The signals from individual false targets are linearly superimposed to form a composite, dense false target interference signal: ; ; in, This is the k-th false signal at time t. For its specific time delay, This is its specific Doppler frequency shift. Let be the echo amplitude of the kth echo containing the attenuation factor; S1.4 Draw the corresponding image.

[0020] The basic idea of ​​decoy jamming is to simulate the characteristics of radar echo signals and inject false signals into the radar receiving channel, causing the radar to misjudge the target's position, number, or speed.

[0021] S4 includes the following steps: S4.1 Acquire the echo signal received by the radar, wherein the echo signal is a complex-valued echo signal containing both real target echoes and dense false target interference signals. : ; in, The imaginary unit satisfies ; S4.2, the complex-valued echo signal real part sequence and imaginary part sequence Each Siamese network with the same structure is input into a separate network for feature extraction to obtain the real part features. Features of the imaginary part The definition of a twin network is as follows: ; ; in, Represents a convolutional network. Indicates a fully connected network. Then it is Activation function; S4.3, through an embedding layer with the same structure and ,Will and Mapping to the embedding space of the same dimension yields... and ,right and By splicing the data, the fused features are obtained. : ; ; in, It is a fused feature resulting from the concatenation of two features, where and These are the features mapped from the real and imaginary features, respectively.

[0022] S5 includes the following steps: S5.1, Based on fusion features Anti-interference actions are output to the network through strategy selection. The strategy selection network structure adopts a fully connected network, and outputs anti-interference actions. Q value: ; in Choose network parameters for the policy, where Q is the state-action value function. It is a policy network; S5.2, Based on fusion features Evaluate the network using a strategy Counter-interference actions The effectiveness is evaluated and the evaluation value is output, including the strategy evaluation network. A fully connected network is used, and the following formula is satisfied: ; in, For the strategy scoring network parameters, To assess the value of the action; S5.3 Maximize long-term cumulative rewards by updating network parameter outputs through Q-learning. : ; in, It is the target Q value. It is the instantaneous reward at time t, determined by the policy evaluation network. Based on fusion characteristics and actions roll out, This is a discount factor used to control the importance of future rewards. To maximize the value of all possible actions in the action space, ensuring that the current action is the optimal action in the current state, and to obtain the Q-value of the target network output, the parameters... These are typically parameters of the target network, do not participate in gradient updates, and are only used when calculating the target Q-value to avoid training oscillations; S5.4. The mean squared error loss function is used to measure the difference between the Q-value predicted by the network under the current strategy and the target Q-value. The formula is as follows: ; Among them, the loss function middle It is the current network input. This represents the expected sample size sampled from the experience replay buffer D, where the sample contains features of the current state. Execution of actions Instant rewards Features of the state at the next moment , It is the target Q value. It is the Q-value of the network output by the current policy selection network. The main goal is to minimize this loss so that the current network gradually approaches the target network, thereby learning the optimal policy. S5.5. Update the network parameters using gradient descent with the Adam optimizer to minimize the loss function: ; in, These are the trainable parameters in the network. For learning rate, This indicates that the loss function is applied to the parameters. gradient, This indicates an assignment operation, used for parameter updates.

[0023] In this embodiment, and Since they have the same structure, in order to maintain feature compatibility, the policy evaluation network evaluates anti-interference actions. In state The effectiveness of the output signal is determined by the reward signal or value estimate.

[0024] S3 includes the following steps: S3.1 Calculate the target signal-to-interference-plus-noise ratio at the radar receiver. : ; in, This represents the radar cross-section of the target at this moment. To increase the channel gain of the radar jammer reaching the target. It is the power of the radar to receive ambient noise. Let n be the carrier frequency of the radar's nth pulse. The frequency of the jammer, It is the radar transmission power. It is the power of the target jammer, when , It is 1 if it is true, otherwise it is 0. S3.2, Reward Function Satisfy the following formula: ; in, It is the target detection probability. It is the false alarm rate. To obtain the signal-to-interference-plus-noise ratio of the target for radar, These represent the weighting coefficients for each indicator.

[0025] In S3, the reward function is used to address dense false target interference. The following aspects can be considered: Target detection probability Improvement: Rewards correct detection of targets, and penalizes missed detections.

[0026] False alarm rate Reduced: When the reward for a false alarm rate decreases, the penalty for a false alarm increases.

[0027] Anti-jamming performance of radar systems: The signal-to-interference-plus-noise ratio (SIR) of the target obtained by the radar represents the ratio of the target echo signal to the interference and noise, that is, the ratio of the target signal power received at the receiving end to the interference plus noise power.

[0028] After S5 is completed, S4 and S5 are repeated to continuously extract signal features, select strategies, evaluate and update parameters until the loss function converges and the optimal anti-interference strategy is obtained.

[0029] Figure 4 This demonstrates the changes in the real and imaginary parts of a linear frequency modulated (LFM) signal transmitted by a radar over time. Because the frequency changes linearly with time, the signal phase increases quadratically, exhibiting an oscillation with "increasing frequency" (both the real and imaginary parts are sinusoidal fluctuations, and the frequency increases with time). Figure 5 This demonstrates the changes in the real and imaginary parts of the actual target echo received by the radar over time. The echo signal is the result of the transmitted signal being reflected by the target, and includes: time delay, Doppler frequency shift, and attenuation. It is used to reconstruct the target's "range + velocity" modulation of the radar signal and serves as the raw input for subsequent processing. Figure 6 This demonstrates the output after pulse compression (matched filtering) of the echo signal. Pulse compression involves convolving (or matching filtering) the echo with the "conjugate inversion" of the transmitted signal, utilizing the "wideband long pulse" characteristic of LFM to compress it into a narrow pulse (similar to "compressing a long signal into a spike"). Figure 7 This displays the Doppler spectrum of the echo signal (frequency domain amplitude after performing an FFT on the echo). The Doppler shift reflects the radial velocity of the target relative to the radar. Performing an FFT on the echo signal transforms the "phase change in the time domain" into the "peak amplitude in the frequency domain," with the peak position corresponding to the Doppler shift. ; Figure 8 Display the echo signal of a single dummy target (real and imaginary parts changing over time). Figure 9 Multiple composite signal images of fake targets are shown; Figure 10 and Figure 11This is a range-Doppler two-dimensional plot calculated by the radar for both real target echoes and echoes containing false targets. The horizontal axis represents the Doppler frequency shift (Hz), the vertical axis represents the range (m), and the color brightness represents the signal amplitude.

[0030] On the other hand, the present invention discloses an anti-interference system, which includes a radar signal transmission module for simulating radar transmission waveforms and outputting radar transmission power. The carrier frequency of the nth pulse The system includes: a target and jamming signal generation module for simulating real target signals and dense fake target jamming signals; and a signal propagation module for simulating the signal propagation process in space and calculating the channel gain from radar to the target jammer. The signal receiving module is used to simulate radar receiving signals, receiving both real echo signals and dense false target interference signals, and outputting the target signal power and interference power of the receiver. and radar received ambient noise power .

[0031] This invention can be applied to military electronics, defense technology, and other fields, and solves the following technical problems: 1. The problem of insufficient adaptability of traditional radar anti-jamming methods: In military electronic scenarios, dense false targets are highly similar to real targets, and the jamming patterns and jamming intensity have dynamic game characteristics. Traditional anti-jamming methods (such as waveform agility and CFAR detection) rely on manually preset rules and lack online autonomous decision-making and adaptive optimization capabilities. They are prone to problems such as excessively high false alarm rates, missed detection of real targets, and failure of track tracking, and cannot effectively deal with the threat of dense false targets in complex battlefield environments.

[0032] 2. Integration and Adaptation of Reinforcement Learning with Radar Anti-jamming Scenarios: Existing reinforcement learning anti-jamming solutions suffer from bottlenecks such as slow algorithm training convergence, poor generalization ability, and insufficient real-time performance, making it difficult to adapt to the engineering requirements of real-time radar detection and tracking. At the same time, how to reasonably define the state space, action space, and reward function in anti-jamming scenarios to achieve efficient interaction between reinforcement learning agents and radar systems and dense fake target jamming environments has become a key challenge restricting its practical application in the military electronics field.

[0033] Finally, it should be emphasized that the above description is not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A radar anti-dense decoy jamming method based on deep reinforcement learning, characterized in that, The method includes the following steps: S1. Construct a simulation model of dense false target interference to simulate the interaction between the radar system and dense false target interference. S2, Define the interference state; S3. Design a reward function for robust reinforcement learning. The evaluation of the anti-interference effect after taking a certain action is rewarded; S4. Based on the twin network, the complex-valued echo signal received by the radar is processed by feature extraction and then fused to obtain the fused features. ; S5, Integrating Features The input is fed into the policy network to complete the adaptive learning and optimization of anti-interference actions, and generate the optimal anti-interference actions; S6. Adjust the radar's transmission waveform according to the optimal anti-jamming action.

2. The radar anti-dense decoy jamming method based on deep reinforcement learning according to claim 1, characterized in that, In S2, under dense false target forwarding interference, the interference state expression at time t is: ； in, This indicates the lower frequency of the signal transmitted by the jammer. Indicates signal bandwidth; Indicates the duration of the jamming reconnaissance window. This indicates the duration of the jamming window.

3. The radar anti-dense decoy jamming method based on deep reinforcement learning according to claim 1, characterized in that, S1 includes the following steps: S1.1 The radar transmit signal is generated using linear frequency modulation, as shown in the following formula: ； in, It is the transmitted linear frequency modulated signal, where B is the bandwidth of the frequency modulation, T is the pulse width, t is the time variable, and j is the imaginary unit; S1.2, Simulate the echo signal of the target / dummy target, and introduce time delay and Doppler frequency shift. The formula for the echo signal is as follows: ； ； ； in, is the echo signal at time t; A is the echo amplitude including the attenuation factor; This indicates a Doppler frequency shift; The echo delay time is related to the target distance, therefore This indicates the time delay of the echo signal. This represents the radial velocity of the target relative to the radar. Indicates the radar signal wavelength, and , This indicates the radar carrier frequency, where c is the speed of light constant. When the target approaches the radar, >0, when far away <0; R represents the round-trip time of the signal, and R represents the distance between the target and the radar. S1.3, Construction A false target signal, and then The signals from individual false targets are linearly superimposed to form a composite, dense false target interference signal: ； ； in, This is the k-th false signal at time t. For its specific time delay, This is its specific Doppler frequency shift. Let be the echo amplitude of the kth echo containing the attenuation factor; S1.4 Draw the corresponding image.

4. The radar anti-dense decoy jamming method based on deep reinforcement learning according to claim 1, characterized in that, S4 includes the following steps: S4.1 Acquire the echo signal received by the radar, wherein the echo signal is a complex-valued echo signal containing both real target echoes and dense false target interference signals. : ； in, The imaginary unit satisfies ; S4.2, the complex-valued echo signal real part sequence and imaginary part sequence Each Siamese network with the same structure is input into a separate network for feature extraction to obtain the real part features. Features of the imaginary part The definition of a twin network is as follows: ； ； in, Represents a convolutional network. Indicates a fully connected network. Then it is Activation function; S4.3, through an embedding layer with the same structure and ,Will and Mapping to the embedding space of the same dimension yields... and ,right and By splicing the data, the fused features are obtained. : ； ； in, It is a fused feature resulting from the concatenation of two features, where and These are the features mapped from the real and imaginary features, respectively.

5. The radar anti-dense decoy jamming method based on deep reinforcement learning according to claim 1, characterized in that, S5 includes the following steps: S5.1, Based on fusion features The network outputs anti-interference actions through strategy selection. The strategy selection network structure adopts a fully connected network, and outputs anti-interference actions. Q value: ； in Choose network parameters for the policy, where Q is the state-action value function. It is a policy network; S5.2, Based on fusion features Evaluate the network using a strategy Counter-interference actions The effectiveness is evaluated and the evaluation value is output, including the strategy evaluation network. A fully connected network is used, and the following formula is satisfied: ； in, For the strategy scoring network parameters, To assess the value of the action; S5.3 Maximize long-term cumulative rewards by updating network parameter outputs through Q-learning. : ； in, It is the target Q value. It is the instantaneous reward at time t, determined by the policy evaluation network. Based on fusion characteristics and actions roll out, This is a discount factor used to control the importance of future rewards. To maximize the value of all possible actions in the action space, ensure that the current action is the optimal action in the current state, and output the Q value to the target network; S5.

4. The mean squared error loss function is used to measure the difference between the Q-value predicted by the network under the current strategy and the target Q-value. The formula is as follows: ； Among them, the loss function middle It is the current network input. This represents the expected sample size sampled from the experience replay buffer D, where the sample contains features of the current state. Execution of actions Instant rewards Features of the state at the next moment , It is the target Q value. It is the Q-value output by the network under the current policy selection; S5.

5. Update the network parameters using gradient descent with the Adam optimizer to minimize the loss function: ； in, These are the trainable parameters in the network. For learning rate, This indicates that the loss function is applied to the parameters. gradient, This indicates an assignment operation, used for parameter updates.

6. The radar anti-dense decoy jamming method based on deep reinforcement learning according to claim 1, characterized in that, S3 includes the following steps: S3.1 Calculate the target signal-to-interference-plus-noise ratio at the radar receiver. : ； in, This is the radar cross-section value of the target at this time. To increase the channel gain of the radar jammer reaching the target. It is the power of the radar to receive ambient noise. Let n be the carrier frequency of the radar's nth pulse. The frequency of the jammer, It is the radar transmission power. It is the power of the target jammer, when , It is 1 if it is true, otherwise it is 0. S3.2, Reward Function Satisfy the following formula: ； in, It is the target detection probability. It is the false alarm rate. To obtain the signal-to-interference-plus-noise ratio of the target for radar, These represent the weighting coefficients for each indicator.

7. A system for applying the deep reinforcement learning-based radar anti-dense decoy jamming method according to any one of claims 1-6, characterized in that, The system includes a radar signal transmission module, which simulates radar transmission waveforms and outputs radar transmission power. The carrier frequency of the nth pulse The system includes: a target and jamming signal generation module for simulating real target signals and dense fake target jamming signals; and a signal propagation module for simulating the signal propagation process in space and calculating the channel gain from radar to the target jammer. The signal receiving module is used to simulate radar receiving signals, receiving both real echo signals and dense false target interference signals, and outputting the target signal power and interference power of the receiver. and radar received ambient noise power .

Images