A method and system for optimizing a jamming and deception composite waveform based on reinforcement learning, and a storage medium

Abstract

This invention discloses a method, system, and storage medium for optimizing composite jamming waveforms involving suppression and deception based on reinforcement learning. The invention models the online optimization problem of jamming waveforms as a sequential decision-making process. First, a reconfigurable composite jamming waveform is constructed by recombining suppression and deception jamming components in the time domain. Then, a sequential decision-making model is established, with the state space containing waveform structure parameters and radar detection features, and the action space containing segment arrangement and power adjustment. A reward function that fuses jamming effectiveness and transmit power constraints is designed. Finally, a deep reinforcement learning network is used to train the decision-making module offline to output the optimized composite jamming waveform. This invention achieves time-domain fusion and refined power allocation of suppression and deception jamming, significantly reducing the interference-to-signal ratio and peak-to-average power ratio, and improving the power utilization efficiency and low-interception performance of the jamming waveform.

Description

Technical Field

[0001] This invention belongs to the fields of electromagnetic spectrum game theory, electronic interference and artificial intelligence, and specifically relates to a method, system and storage medium for optimizing composite interference waveforms of suppression and deception based on reinforcement learning. Background Technology

[0002] The generation of interference waveforms has always been a core research topic in intelligent electromagnetic spectrum games. With the rapid development of radar technology, especially the emergence of cognitive radar with intelligent sensing and adaptive anti-jamming capabilities, jamming missions have placed higher demands on the adaptability, complexity, and stealth of waveforms.

[0003] Jamming waveform generation is a crucial step in jamming operations, and its quality directly determines the success or failure of jamming a radar system. Essentially, it modulates intercepted radar signals to disrupt radar detection performance. The main methods include: suppressing interference through noise modulation or directly transmitting high-power noise to cover or drown out target echoes; or using Digital Radio Frequency Storage (DRFM) technology to sample and forward intercepted signals, generating false target signals highly similar to the target echo, inducing the radar to track the false target. However, traditional radar jamming waveform generation techniques have significant limitations when dealing with modern cognitive radar. First, traditional methods heavily rely on pre-set fixed mathematical formulas or DRFM modulation rules, resulting in waveforms with strong regularity and singular characteristics. Modern radars, using advanced signal processing algorithms, can easily identify and filter out these statistically consistent jamming patterns. Second, due to the temporal nature of radar signals, jamming exhibits a certain lag, causing false targets to lag behind real targets, thus weakening the effectiveness of the jamming to some extent. Furthermore, balancing jamming power remains a significant obstacle for traditional methods: insufficient jamming power makes it difficult to exceed the CFAR detection threshold; excessive jamming power, while improving jamming effectiveness, consumes more resources and may reveal the jammer's location. To address these issues, many researchers have recently explored intelligent jamming technology, aiming to endow jammers with autonomous learning capabilities.

[0004] Existing Solution 1: Xue Hai et al. proposed an intelligent interference signal generation algorithm based on an improved convolutional autoencoder. This algorithm extracts signal features by introducing a complex convolutional network and a residual module, and generates interference signals highly similar to the original waveform by modifying hidden layer features. This method generates interference in a data-driven manner, resulting in some improvement in interference effectiveness. However, the quality of the generated interference signal is highly dependent on the accuracy of the hidden layer feature extraction.

[0005] Existing Solution Two: Jiang Yilin et al. proposed a GAN-based interference waveform generation method. This method uses a segment of the radar signal header as input and a trained generator to predict and generate the complete interference waveform. This method reduces the reliance on complete radar signal interception, improves the lag in interference effects to some extent, and significantly increases the difficulty for radar to identify interference. However, when dealing with high-dimensional continuous waveform parameter spaces, the generator may struggle to accurately fit the global distribution of the optimal interference effect.

[0006] Existing Solution 3: Mario Coutino et al. proposed a reinforcement learning-based radar waveform optimization method. This method uses radar waveform parameters (such as pulse duration, spacing, and modulation type) as the optimization object. A near-end policy optimization algorithm trains the decision module to gradually learn the optimal waveform design strategy through interaction with the environment, improving the radar's adaptability and real-time performance in dynamic environments. However, this solution primarily optimizes the performance of the radar detection end, and its reward mechanism does not consider performance indicators specific to jamming tasks (such as false target distribution).

[0007] The aforementioned existing solutions can improve the intelligence and effectiveness of jamming to a certain extent, and their jamming effect is significantly better than traditional jamming methods, but they still have some limitations. These methods mostly focus on the optimization of single-pattern jamming, and fail to achieve coordinated control of suppression and deception components from the perspective of waveform structure and energy distribution; moreover, the generated waveforms often lack multi-objective joint constraints on jamming effect and resource efficiency, making it difficult to balance actual jamming effect and concealment. Summary of the Invention

[0008] The purpose of this invention is to provide a method, system, and storage medium for optimizing composite interference waveforms of suppression and deception based on reinforcement learning, which solves the problems of traditional interference waveforms relying on fixed formulas, resulting in single interference forms, easy detection and identification, and interference signal lag weakening the interference effect.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] A reinforcement learning-based method for optimizing composite interference waveforms involving suppression and deception includes the following steps:

[0011] Step 1: Construct the basic reconfigurable unit of the interference waveform, divide the preset suppression interference basic component and the deception interference basic component into multiple independent and controllable segments in the time domain, and generate the composite interference waveform through segment selection and amplitude control.

[0012] Step 2: Establish a sequential decision-making process model. The online optimization problem of interference waveforms is modeled as a sequential decision-making process, and the deep Q-network algorithm is used to solve the sequential decision-making process. The state space, action space, and reward function of the sequential decision-making process are defined as follows: The state space includes the structural parameters of the current composite interference waveform segment, the detection result characteristics at the radar end, and the overall power information of the current composite interference waveform; the action space includes the permutation and combination operations and amplitude adjustment operations of the segment; the reward function is a comprehensive evaluation function based on the interference effectiveness index and the transmit power constraint index.

[0013] Step 3: Construct and train a deep reinforcement learning network. Use a deep reinforcement learning algorithm to build a decision network. Use the sequence decision process model as the interaction environment. Train offline to enable the decision module to learn the optimal policy mapping from state to action.

[0014] Step 4: Generate optimized interference waveforms. Using the trained decision network, output the corresponding action according to the current adversarial state, and the decision module generates optimized composite interference waveforms based on the action.

[0015] Further, in step 1, the basic component of the suppression interference is a smart noise convolution interference signal, which is generated by convolving band-limited Gaussian white noise with the linear frequency modulated signal transmitted by the radar; the basic component of the deception interference is a smart noise forwarding interference signal, which is generated by multiplying band-limited Gaussian white noise by a gain that follows a normal distribution in the time domain to obtain modulated noise, convolving the modulated noise with the linear frequency modulated signal transmitted by the radar, and generating a false target peak that precedes the real target by performing a cyclic shift operation on the modulated smart noise convolution interference signal.

[0016] Furthermore, in step 1, the clever noise convolution interference signal for:

[0017]

[0018] in, The radar signal received by the jammer; It is band-limited Gaussian white noise; This is a convolution operation.

[0019] Clever noise relay interference signal for:

[0020]

[0021] in, The shift interval; The number of shifts; The circular index for the shift operation; For time; Indicates will Move to the right in a circular motion point; The fundamental component of the smart noise interference signal is expressed as follows:

[0022]

[0023] in, The modulation noise is composed of band-limited Gaussian white noise. Multiply by a gain that follows a normal distribution in the time domain. To obtain, that is ;

[0024] The generated composite interference waveform is:

[0025]

[0026] in, The number of segments is the length of the segments. clever noise convolution interference signal and clever noise relay interference signal Divide the time domain evenly into Each is an independent and controllable segment; For fragment indexing; For the first The start time of each segment; for After arrangement optimization, the first The location is selected using a clever noise convolution interference segment; for After arrangement optimization, the first The location was selected to cleverly relay interference with the basic segment; and These are the two interference signals, numbered 1 and 2 respectively. The amplitude control factor of the segment; the composite interference waveform is generated by superimposing the above two amplitude-weighted interference segment sequences in time sequence.

[0027] Furthermore, in step 2, a Markov Decision Process (MDP) model is established, modeling the interference waveform optimization problem as a Markov Decision Process, and the Deep Q-Network algorithm (DQN) is used to solve the Markov Decision Process MDP; the state space of the Deep Q-Network algorithm includes the current... Segment arrangement and power information, characteristics of CFAR detection results after echo signal pulse compression, current The interference-to-signal ratio (JSR) and peak-to-average power ratio (PAPR) are used as indices. The action space quantifies the continuous signal segment power allocation problem into discrete amplitude adjustments, including parameter sets for the segments. Perform permutation and combination operations, as well as discretization gain or attenuation adjustment operations on the amplitude of specific segments; the reward function is:

[0028]

[0029] in, For the first The instant reward value for each step; For the first The environmental conditions of the step; This serves as the basic reward for successful interference, reflecting the interference waveform's simultaneous fulfillment of the conditions for successful interference, which include both suppression and deception. The reward is based on the power optimization of the current interference signal JSR; the lower the JSR, the higher the reward. The waveform quality reward is based on PAPR; the lower the PAPR, the higher the reward. This is the basic penalty for failed interference. To suppress some of the reward, this reflects the degree to which the echo signal at the real target is suppressed by the CFAR threshold; To deceive part of the reward, it reflects the number of valid false targets generated in front of the target by the current waveform. and If the interference fails, the network is propelled toward the target direction of successful interference. The sum of the above-mentioned reward items is the final reward output.

[0030] Furthermore, in step 3, the deep reinforcement learning algorithm is a deep Q-network algorithm, and the decision network includes the current network. and target network The dual-network structure, in which, This refers to the current environmental state. For the current action; These are the parameters for the current network, used to update in real time after each interaction step; The target network parameters are periodically copied and synchronized from the current network to calculate the target Q-value to stabilize the training process.

[0031] Furthermore, in step 3, offline training employs dynamic... Strategy: Based on probability Conduct random exploration, with probability Select the action corresponding to the maximum Q value output by the current network and utilize it. The quadruples generated by the interaction between the decision module and the environment grow dynamically during the training process. Stored in the experience replay pool, according to the aforementioned rewards. The absolute value of the sample is used to calculate the priority of the sample. During training, samples are drawn from the experience replay pool according to the priority. Every fixed step, the parameters of the current network are copied to the target network to update the parameters of the target network.

[0032] Furthermore, in step 4, the decision module determines the parameters of the independent controllable segment based on the parameter set. Adjustments were made to generate an optimized composite interference waveform. ,

[0033]

[0034] in, , , , These are the optimized waveform segment parameters.

[0035] Furthermore, it also includes the step of establishing a radar jamming scenario model: before step 1, establishing the radar end-to-end interference-to-signal ratio. Interference-to-signal ratio with jammer A deterministic proportional relationship is established between the two parameters, used to calculate the jamming effect at the radar end based on the jammer's local parameters in an interactive environment. This deterministic proportional relationship is applicable to self-defense jamming scenarios and is derived based on the power equation, under the condition that the distance between the radar and the target is equal to the distance between the jammer and the radar. The deterministic proportional relationship K is... .

[0036] A computer system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a reinforcement learning-based method for optimizing composite interference waveforms of suppression and deception.

[0037] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a reinforcement learning-based method for optimizing composite interference waveforms involving suppression and deception.

[0038] The beneficial effects of this invention are as follows:

[0039] 1. This study proposes a reinforcement learning-based method for optimizing interference waveforms. This method models the complex interference waveform optimization problem as a Markov decision process, and the optimized waveform can serve as a high-quality template for adaptive interference tasks. By modeling and simulating the interference environment, the interference effect is evaluated using CFAR detection results from radar-received echo signals. A multi-dimensional reward evaluation system integrating interference effect, interference-to-signal ratio (ISR), and peak-to-average power ratio (PAPR) is constructed to drive waveform optimization. Compared with waveforms obtained by traditional methods, the optimized waveform significantly reduces ISR and PAPR while maintaining interference effectiveness. Furthermore, compared to some other optimization methods, the Markov decision framework and interactive training method employed by reinforcement learning have certain advantages in exploring the complex parameter space of interference waveforms.

[0040] 2. This study proposes a segmented fusion and power allocation mechanism that integrates suppressive and deceptive jamming in the time domain. This mechanism divides the jamming signal into multiple independently controllable segments, enabling fine-grained control over the timing and power proportion of each segment. Through interactive training between the decision module and the simulation environment, this method learns the optimal power allocation strategy: precisely increasing the power of efficient segments while reducing the power of inefficient segments, ultimately obtaining a composite jamming waveform that combines noise suppression and advanced decoy characteristics. Such composite waveforms are difficult to design using traditional methods, avoiding the limitations of single jamming forms and thus improving overall power utilization efficiency. Attached Figure Description

[0041] Figure 1 System overall flowchart;

[0042] Figure 2 Radar jamming scenario model diagram;

[0043] Figure 3 Optimized time-domain diagram of interference signal;

[0044] Figure 4 (a) Graph showing the results of the optimized matched filtering of the interference signal;

[0045] Figure 4 (b) Graph of CA-CFAR detection results for the optimized interference signal;

[0046] Figure 5 The image shows the CA-CFAR detection results of the traditional interference signal under the same JSR conditions as the optimized signal. Detailed Implementation

[0047] The present invention will now be further described with reference to the accompanying drawings.

[0048] The method described in this invention can autonomously optimize in a closed-loop interactive manner under complex electromagnetic countermeasures environments, generating waveform templates that combine high interference effectiveness with low resource consumption. This effectively solves the problems of traditional methods relying on fixed templates, single interference patterns, blind power allocation, and the lack of multi-objective joint optimization capabilities in existing intelligent methods.

[0049] To verify the effectiveness of the present invention, in accordance with Figure 1 The process shown establishes a simulation environment and constructs a DQN network. Then, the suppression and deception interference waveforms to be optimized are obtained using traditional methods. After optimization, the waveforms are compared with those of traditional methods.

[0050] Step 1: Establish a radar jamming scenario model, such as Figure 2 As shown. The radar transmits a linear frequency modulated (LFM) signal for target detection, and the center frequency of the simulated radar signal used is... pulse width ,bandwidth Sampling rate After intercepting the signal, the jammer generates an interference signal. This interference signal, combined with the real target echo, forms a composite echo that returns to the radar receiver. The radar employs CA-CFAR detection technology to reduce the false alarm probability. Reference unit In a self-defense jamming scenario, the distance between the radar and the target is equal to the distance between the jammer and the radar. From the power equation, it can be derived that... ,in For radar-to-interference ratio, Let JSR be the signal-to-interference ratio at the jammer end, and K be the ratio between the two. This relationship establishes the basis for the mapping between the local parameters of the jammer and the jamming effect at the radar end. Therefore, the JSR of the radar end can be estimated based on the jammer's JSR.

[0051] Step 2: Constructing the basic jamming components. The suppression component employs smart noise convolution jamming, convolving the modulated noise with the radar signal. The modulated noise is band-limited Gaussian white noise, with its frequency band aligned with the radar signal bandwidth. The deception component employs smart noise forwarding jamming, generating false target peaks that lead the real target through cyclic shifting operations, based on the convolution jamming.

[0052] Clever noise convolution interference signal for:

[0053]

[0054] in, The radar signal received by the jammer; It is band-limited Gaussian white noise; This is a convolution operation.

[0055] Clever noise relay interference signal for:

[0056]

[0057] in, The shift interval; The number of shifts; The circular index for the shift operation; For time; Indicates will Move to the right in a circular motion point; The fundamental component of the smart noise interference signal is expressed as follows:

[0058]

[0059] in, The modulation noise is composed of band-limited Gaussian white noise. Multiply by a gain that follows a normal distribution in the time domain. To obtain, that is ;

[0060] By controlling the shift interval Number of shifts It allows for the design of the location and number of false targets, achieving deception and interference with advanced effects.

[0061] The above two lengths are The basic interference signals are all divided into two groups in the time domain. An independent and controllable segment, here The expression for the reconstruction of the interference waveform is:

[0062]

[0063] in, The number of segments is the length of the segments. clever noise convolution interference signal and clever noise relay interference signal Divide the time domain evenly into Each is an independent and controllable segment; For fragment indexing; For the first The start time of each segment; for After arrangement optimization, the first The location is selected using a clever noise convolution interference segment; for After arrangement optimization, the first The location was selected to cleverly relay interference with the basic segment; and These are the two interference signals, numbered 1 and 2 respectively. The amplitude control factor of the segment; the composite interference waveform is generated by superimposing the above two amplitude-weighted interference segment sequences in time sequence.

[0064] Step 3: Construct Markov Decision Process (MDP) elements. Establish a Markov Decision Process (MDP) model, modeling the interference waveform optimization problem as a Markov Decision Process, and use the Deep Q-Network algorithm (DQN) to solve the Markov Decision Process (MDP).

[0065] The state space of the deep Q-network algorithm contains the current state. Segment arrangement and power information, characteristics of CFAR detection results after echo signal pulse compression, current The interferometric ratio (JSR) and peak-to-average ratio (PAPR) are indices.

[0066] The action space of the deep Q-network algorithm: The problem of power allocation for continuous signal segments is quantized into discrete amplitude adjustments, including adjustments to the parameter set of the segments. Perform permutation and combination operations, as well as discretize the amplitude of specific segments and adjust the gain or attenuation.

[0067] Reward Function: Constructing a Comprehensive Evaluation System :

[0068]

[0069] in, For the first The instant reward value for each step; For the first The environmental conditions of the step; This serves as the basic reward for successful interference, reflecting the interference waveform's simultaneous fulfillment of the conditions for successful interference, which include both suppression and deception. The reward is based on the power optimization of the current interference signal JSR; the lower the JSR, the higher the reward. The waveform quality reward is based on PAPR; the lower the PAPR, the higher the reward. This is the basic penalty for failed interference. To suppress some of the reward, this reflects the degree to which the echo signal at the real target is suppressed by the CFAR threshold; To deceive part of the reward, it reflects the number of valid false targets generated in front of the target by the current waveform; and If the interference fails, the network is propelled toward the target direction of successful interference. The sum of the above-mentioned reward items is the final reward output.

[0070] A positive reward is given when the optimized waveform successfully suppresses real targets or generates advanced false targets in radar CFAR detection. To constrain transmit power, a negative reward is applied when the waveform JSR or PAPR is too high, forcing the decision module to find low-power, low-peak-power strategies.

[0071] Step 4: Construct the DQN network. Establish a network containing the current network. and target network The dual-network structure consists of two identical fully connected networks, where the number of neurons in the input layer corresponds to the dimension of the state space, and the number of neurons in the output layer corresponds to the set of selectable actions. This refers to the current environmental state. For the current action; These are the parameters for the current network, used to update in real time after each interaction step; The target network parameters are periodically copied and synchronized from the current network to calculate the target Q-value to stabilize the training process.

[0072] Step 5: Set the loss function. Minimize the mean squared error between the target Q-value and the predicted Q-value using the RMSProp optimizer:

[0073]

[0074] in, , where Q is the target Q value; This is a discount factor used to measure the weight of future rewards; For training batches; To perform the action The next state after that; The parameters of the current network are used as candidate actions for the next state; at fixed steps, the parameters of the current network are... Copy to the target network to update the target network parameters. .

[0075] Step Six: Train the DQN network. Use dynamic... Strategy, based on probability Conduct random exploration, with probability Select the action corresponding to the maximum Q value output by the current network and utilize it. The simulation grows dynamically during training, thus balancing the exploration of new strategies with the utilization of known optimal strategies. The simulation trained for 800 rounds, with the target interference effect or reaching the maximum number of steps as the termination condition for each round. The value initially increases gradually from 0 to 0.9, and remains constant at 0.9 after 600 rounds. In each interaction, the decision-making module selects an action based on the current state. The environment generates a new interference waveform based on the action and simulates the radar's pulse compression and CA-CFAR detection process to calculate the interference effect, JSR, and PAPR. A reward is then given, and the system moves to the next state. An experience replay mechanism is introduced, which uses the four-tuples generated by the interaction... The data is stored in the experience replay pool, and the reward is incorporated into the experience priority calculation. Samples are extracted for training according to the priority of experience replay. The network parameters are updated periodically at fixed steps. The decision-making module will gradually learn to generate efficient, low-power, and covert composite interference waveforms through continuous interaction.

[0076] Step 7: Generate optimized interference waveforms. Using the trained decision network, output corresponding actions based on the current adversarial state, and the decision module then adjusts the response based on these actions. The parameter set was adjusted to finally generate an optimized composite interference waveform that satisfies both suppression and deception interference effects. :

[0077]

[0078] in, , , , These are the optimized waveform segment parameters.

[0079] Example:

[0080] The optimized interference waveform of this invention was verified, and the results are as follows: Figure 3 and Figure 4 As shown, suppression of real targets and generation of advanced decoys can be achieved at a JSR of 7.96 dB and a PAPR of 10.03 dB. The jamming effect of traditional jamming methods at JSR=7.96 dB is as follows... Figure 5 As shown, the unoptimized combined interference generates false targets ahead of the target, but the peak pulse compression energy at the real target is still higher than the detection threshold, allowing for the detection of the real target. To achieve a similar interference effect as the optimized version, a JSR of at least 11.5 dB is required. By comparing the interference effects, it can be seen that the method proposed in this invention can achieve a certain level of low power consumption and interference concealment while ensuring the interference effect.

[0081] In particular, in some preferred embodiments of the present invention, a computer device is also provided, including a memory and a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the reinforcement learning-based suppression and deception composite interference waveform optimization method described in any of the above embodiments.

[0082] In some other preferred embodiments of the present invention, a computer-readable storage medium is also provided, on which a computer program / instruction is stored, wherein when the computer program is executed by a processor, the steps of the reinforcement learning-based suppression and deception composite interference waveform optimization method described in any of the above embodiments are implemented.

[0083] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above embodiments of the reinforcement learning-based suppression and deception composite interference waveform optimization method, which will not be repeated here.

[0084] Computer-readable storage media encompass a variety of types, including persistent and non-persistent, portable and fixed. These media store information using different technologies, and the content can be machine instructions, data structures, program modules, or other types of data. Some typical examples of computer storage media include: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), various types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory and other storage technologies, optical storage media such as CD-ROM and digital video disc (DVD), magnetic storage devices such as magnetic tape and disks, and other non-transferable media used to store information accessible to computing devices. It is important to note that the computer-readable media described herein do not include temporary storage media, such as modulated data signals and carrier waves.

[0085] Those skilled in the art will further recognize that the operation of the module can be achieved using existing technical protocols or programs, without relying on new computer programs themselves. The units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0086] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing composite interference waveforms involving suppression and deception based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Construct the basic reconfigurable unit of the interference waveform, divide the preset suppression interference basic component and the deception interference basic component into multiple independent and controllable segments in the time domain, and generate the composite interference waveform through segment selection and amplitude control. Step 2: Establish a sequential decision-making process model. The online optimization problem of interference waveforms is modeled as a sequential decision-making process, and the deep Q-network algorithm is used to solve the sequential decision-making process. The state space, action space, and reward function of the sequential decision-making process are defined as follows: The state space includes the structural parameters of the current composite interference waveform segment, the detection result characteristics at the radar end, and the overall power information of the current composite interference waveform; the action space includes the permutation and combination operations and amplitude adjustment operations of the segment; the reward function is a comprehensive evaluation function based on the interference effectiveness index and the transmit power constraint index. Step 3: Construct and train a deep reinforcement learning network. Use a deep reinforcement learning algorithm to build a decision network. Use the sequence decision process model as the interaction environment. Train offline to enable the decision module to learn the optimal policy mapping from state to action. Step 4: Generate optimized interference waveforms. Using the trained decision network, output the corresponding action according to the current adversarial state, and the decision module generates optimized composite interference waveforms based on the action.

2. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 1, characterized in that, In step 1, the basic component of the suppression interference is a smart noise convolution interference signal, which is generated by convolving band-limited Gaussian white noise with the linear frequency modulated signal transmitted by the radar; the basic component of the deception interference is a smart noise forwarding interference signal, which is generated by multiplying band-limited Gaussian white noise by a gain that follows a normal distribution in the time domain to obtain modulated noise, convolving the modulated noise with the linear frequency modulated signal transmitted by the radar, and generating a false target peak that precedes the real target by performing a cyclic shift operation on the modulated smart noise convolution interference signal.

3. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 2, characterized in that, The clever noise convolution interference signal in step 1 for: in, The radar signal received by the jammer; It is band-limited Gaussian white noise; This is a convolution operation. Clever noise relay interference signal for: in, The shift interval; The number of shifts; The circular index for the shift operation; For time; Indicates will Move to the right in a circular motion point; This is the fundamental component of the agile noise interference signal; The generated composite interference waveform for: in, The number of segments is the length of the segments. clever noise convolution interference signal and clever noise relay interference signal Divide the time domain evenly into Each is an independent and controllable segment; For fragment indexing; For the first The start time of each segment; for After arrangement optimization, the first The location is selected using a clever noise convolution interference segment; for After arrangement optimization, the first The location was selected to cleverly relay interference with the basic segment; and These are the two interference signals, numbered 1 and 2 respectively. The amplitude control factor of the segment; the composite interference waveform is generated by superimposing the above two amplitude-weighted interference segment sequences in time sequence.

4. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 3, characterized in that, In step 2, a Markov Decision Process (MDP) model is established, modeling the interference waveform optimization problem as a Markov decision process, and the Deep Q-Network (DQN) algorithm is used to solve the MDP. The state space of the DQN algorithm includes the current state... Segment arrangement and power information, characteristics of CFAR detection results after echo signal pulse compression, current The interference-to-signal ratio (JSR) and peak-to-average power ratio (PAPR) are used as indices. The action space quantifies the continuous signal segment power allocation problem into discrete amplitude adjustments, including parameter sets for the segments. Perform permutation and combination operations, as well as discretization gain or attenuation adjustment operations on the amplitude of specific segments; the reward function is: in, For the first The instant reward value for each step; For the first The environmental conditions of the step; This serves as the basic reward for successful interference, reflecting the interference waveform's simultaneous fulfillment of the conditions for successful interference, which include both suppression and deception. The reward is based on the power optimization of the current interference signal JSR; the lower the JSR, the higher the reward. The waveform quality reward is based on PAPR; the lower the PAPR, the higher the reward. This is the basic penalty for failed interference. To suppress some of the reward, this reflects the degree to which the echo signal at the real target is suppressed by the CFAR threshold; To deceive part of the reward, it reflects the number of valid false targets generated in front of the target by the current waveform; and If the interference fails, the network is propelled toward the target direction of successful interference. The sum of the above-mentioned reward items is the final reward output.

5. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 4, characterized in that, In step 3, the deep reinforcement learning algorithm is a deep Q-network algorithm, and the decision network includes the current network. and target network The dual-network structure, in which, This refers to the current environmental state. For the current action; These are the parameters for the current network, used to update in real time after each interaction step; The target network parameters are periodically copied and synchronized from the current network to calculate the target Q-value to stabilize the training process.

6. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 5, characterized in that, In step 3, offline training adopts dynamic methods. Strategy: Based on probability Conduct random exploration, with probability Select the action corresponding to the maximum Q value output by the current network and utilize it. The quadruples generated by the interaction between the decision module and the environment grow dynamically during the training process. Stored in the experience replay pool, according to the aforementioned rewards. The absolute value of the sample is used to calculate the priority of the sample. During training, samples are drawn from the experience replay pool according to the priority. Every fixed step, the parameters of the current network are copied to the target network to update the parameters of the target network.

7. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 6, characterized in that, In step 4, the decision module determines the parameters of the independent controllable segment based on the parameter set. Adjustments were made to generate an optimized composite interference waveform. , in, , , , These are the optimized waveform segment parameters.

8. The method for optimizing composite interference waveforms of suppression and deception based on reinforcement learning according to claim 1, characterized in that, It also includes the step of establishing a radar jamming scenario model: before step 1, the radar end-to-end interference-to-signal ratio is established. Interference-to-signal ratio with jammer The deterministic proportional relationship between them is used to calculate the jamming effect at the radar end based on the local parameters of the jammer in an interactive environment; the deterministic proportional relationship is applicable to self-defense jamming scenarios and is derived based on the power equation under the condition that the distance between the radar and the target is equal to the distance between the jammer and the radar. The deterministic proportional relationship K is .

9. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

Images