A radar radiation characteristic management and control and optimization scheduling method based on reinforcement learning

Abstract

The application provides a radar radiation characteristic management and control and optimization scheduling method based on reinforcement learning, relates to the technical field of radar anti-interference, and comprises the following steps: S1, constructing a waveform library and a power control strategy, and configuring a radar transmitting signal by using the waveform library and the power control strategy; S2, judging whether the radar transmitting signal is intercepted or not, and if yes, generating an interference signal, otherwise, not generating the interference signal; S3, superimposing and synthesizing the result of S2 and a target echo to obtain a radar receiving signal, and processing the radar receiving signal to obtain target state information and a detection performance evaluation index; and S4, taking the waveform library and the power control strategy as an executable action set, taking the interception probability as a core optimization target, and taking the result of S3 as a decision feedback basis, designing and training a reinforcement learning decision network. The application can construct a dynamic attack and defense environment to enhance the anti-interception performance of the radar transmitting waveform in a complex electromagnetic environment.

Description

Technical Field

[0001] This invention relates to the field of radar anti-jamming technology, and in particular to a radar radiation characteristic control and optimization scheduling method based on reinforcement learning. Background Technology

[0002] Modern radar systems not only need to possess core performance characteristics such as multi-target detection and continuous stable tracking, but also must be adaptable to complex electromagnetic environments to meet the engineering application requirements of low intercept and anti-jamming. To achieve long-range target detection, traditional radars typically employ high-power electromagnetic signals. These radiated signals are easily detected and intercepted by electronic support systems, radar warning receivers, and other equipment. Once the signal is intercepted, the adversary can not only quickly analyze the signal characteristics and issue early warnings, but also guide active jamming equipment to conduct targeted interference. Therefore, radar systems need to dynamically adjust the transmitted signal parameters based on target characteristics and the external environment to reduce the probability of signal interception, achieving a dynamic balance between detection performance and low intercept performance.

[0003] Existing radar systems primarily enhance their anti-interception capabilities through two main technologies: at the signal level, employing techniques such as complex modulation waveforms, carrier frequency agility, and repetition rate agility to reduce the regularity of signal characteristics and improve signal processing gain; at the system level, relying on power management and "wide transmit, narrow receive" technologies to reduce the probability of radar signal interception by controlling radiated power and airspace coverage. However, these technologies often exist as independent modules, lacking dynamic collaborative optimization of multi-dimensional parameters such as waveform, power, and operating mode. Furthermore, they lack adaptive perception and intelligent decision-making capabilities in response to changes in the electromagnetic environment, making it difficult to quickly perceive environmental changes and make optimal control decisions in complex electronic warfare environments. This fails to fundamentally resolve the inherent contradiction between detection performance and low interception performance, and also makes it difficult to simultaneously meet anti-jamming requirements.

[0004] In recent years, artificial intelligence technologies such as reinforcement learning (RL) have provided new technical pathways for the development of intelligent anti-jamming technology for radar. Through continuous interaction with the environment, reinforcement learning algorithms can autonomously learn and approximate optimal parameter control strategies, effectively improving the problems of fixed radiation parameter configurations and poor adaptability to dynamic environments in traditional radar low-interception control methods. This lays the theoretical foundation for building a new generation of intelligent anti-jamming systems. However, current applications of reinforcement learning in radar low-interception control mainly focus on the optimization and adjustment of single parameters such as waveform or power. A collaborative scheduling model for multi-dimensional radiation characteristic parameters has not yet been established, nor has a multi-objective optimization decision-making system been simultaneously incorporated, including radar detection performance constraints and signal interception probability constraints. To address these shortcomings in existing technologies, there is an urgent need for an intelligent control method that can achieve collaborative optimization of multi-dimensional radar radiation characteristic parameters and adapt to dynamic and complex electromagnetic environments. Summary of the Invention

[0005] The main objective of this invention is to propose a radar radiation characteristic control and optimization scheduling method based on reinforcement learning, which can construct a dynamic attack and defense environment to enhance the anti-interception performance of radar transmission waveforms in complex electromagnetic environments.

[0006] This invention is achieved through the following technical solution:

[0007] A radar radiation characteristic control and optimization scheduling method based on reinforcement learning includes the following steps:

[0008] Step S1: Construct a waveform library and power control strategy suitable for various scenarios, and use the waveform library and power control strategy to configure a radar transmission signal that meets the target detection range requirements and achieves the radar peak power.

[0009] Step S2: Detect radar transmission signals using the electromagnetic environment, and determine whether the detected radar transmission signals have been intercepted by the energy threshold detection method. If so, generate an interference signal; otherwise, do not generate an interference signal. The optimal energy threshold is obtained based on the Neyman-Pearson detection criterion and the overlikelihood ratio test.

[0010] Step S3: Superimpose the result of step S2 with the target echo corresponding to the radar transmitted signal to obtain the radar received signal. Process the radar received signal to obtain target status information and detection performance evaluation indicators.

[0011] Step S4: Using the constructed waveform library and power control strategy as the set of executable actions, with the interception probability as the core optimization objective, and the results of step S3 as the decision feedback basis, design and train a reinforcement learning decision network based on a deep Q-network.

[0012] Furthermore, in step S1, the waveform library includes basic signals and agile signals. The basic signals include simple pulses, linear frequency modulated signals, and phase-coded signals. The agile signals include carrier frequency agile signals and pulse repetition interval agile signals. The power control strategy includes high-power transmission, low-power transmission, and radar silence and switching to photoelectric detection.

[0013] Furthermore, in step S2, energy is extracted from the detected radar transmission signal. When the extracted energy is greater than the optimal energy threshold, it is determined that the radar transmission signal has been intercepted. Then, either forwarding deception jamming or suppression jamming is generated according to a preset probability. Otherwise, it is determined that the radar transmission signal has not been intercepted, and no jamming signal is generated.

[0014] Furthermore, in step S2, the energy threshold is expressed as... ,in, For complementary error functions, This represents the probability of a false alarm. The null hypothesis means that the measured values ​​contain only noise and interference, and do not contain target echo signals. The measured values ​​are N-dimensional sampled data vectors, and each sampled data point is an independent and identically distributed Gaussian random variable with zero mean. Let be the standard deviation of the Gaussian random variable.

[0015] Furthermore, in step S2, the forwarding spoofing interference is generated using digital radio frequency storage technology, represented as... ,in, For intercepted radar transmission signals, Delay for forwarding spoofing interference, The amplitude of the forwarding deception interference; the suppression interference is noise amplitude modulation interference, represented as... ,in, DC bias, It is the amplitude modulation function, set to have a mean of 0 and a variance of 0. stochastic processes, The center frequency of the modulation function. Evenly distributed in Above, and with Independent and identically distributed.

[0016] Furthermore, step S3, which processes the radar received signal, includes the following steps:

[0017] Step S31: Perform pulse compression and coherent accumulation on the radar received signal to generate a range-Doppler two-dimensional map;

[0018] Step S32: Extract the target's real-time radial distance and velocity from the range-Doppler image;

[0019] Step S33: Calculate the detection performance evaluation index based on the extracted target information. The evaluation index includes target detection probability, tracking accuracy, range resolution and velocity resolution.

[0020] Furthermore, in step S4, the reinforcement learning decision network includes a state awareness module and a decision module. The state awareness module is used to collect core data from the entire radar operation process in real time and construct a state vector to be input into the decision module. The decision-making module includes an estimation network and a target value network with completely identical structures. The update process of the estimation network is as follows: first, input the current state vector. The current state is predicted through an estimation network. Corresponding different radar radiation parameter adjustment actions of Value, then through Greedy principle choice And proceed to the next state. At the same time, obtain the current reward value. The next state is calculated using the target value network. The largest The network parameters are updated by subtracting the estimated value from the calculated value. After completing the set number of iterations, a soft update method is used. Update the parameters of the target value network ,in, For estimating the parameters of the value network, This represents the soft-spacing update coefficient and the radar radiation parameter adjustment action. It is a combination of waveform action and power action.

[0021] Furthermore, in step S4, the state vector This includes the real-time distance and relative speed between the radar and the target; the waveform type and power level of the current radar signal; the real-time radial distance and speed of the target; detection performance evaluation indicators; whether the radar signal has been intercepted; and the type of interference.

[0022] Furthermore, the reward function is expressed as follows: ,in, This represents the cumulative duration during which the radar's transmitted signal is intercepted in a single training round. This refers to the total duration of a single training round. The farthest distance at which a radar signal can be successfully intercepted. This is the maximum detection range of the radar. This represents the complexity coefficient of the current transmitted waveform; The duration of continuous transmission of the current waveform; The parameter switching penalty term is calculated from the number of waveform switching times, and the weighting coefficient satisfies... .

[0023] Furthermore, the aforementioned The Greed Principle The optimal action corresponding to the maximum Q-value output of the probability selection estimation network is chosen based on the probability. The probability of randomly selecting an action to explore the environment. The effect gradually decreases linearly with the increase of training steps.

[0024] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0025] This invention first constructs a waveform library and power control strategy adaptable to various scenarios. Using the waveform library and power control strategy, a radar transmission signal is configured to meet the target detection range requirements and achieve peak radar power. After transmitting the radar transmission signal, the electromagnetic environment is used to detect the signal. An energy threshold detection method is used to determine whether the detected radar transmission signal has been intercepted. If so, an interference signal is generated; otherwise, no interference signal is generated. If an interference signal is generated, it is superimposed with the target echo to obtain the radar received signal. If no interference signal is generated, the target echo is used as the radar received signal. The radar received signal is processed to obtain target state information and detection performance evaluation indicators. Finally, a reinforcement learning intelligent decision-making network based on a deep Q-network is designed and trained. This reinforcement learning intelligent decision-making network uses the constructed waveform library and power control strategy as the set of executable actions, optimizes the target with the interception probability as the core, and uses the result of step S3 as the decision feedback basis. This invention utilizes reinforcement learning to optimize radar signal transmission strategies and enables radar and electronic reconnaissance systems to engage in iterative adversarial battles through reinforcement learning, thereby constructing a dynamic offensive and defensive environment. This allows for adaptive judgment of the opponent's receiver status during the adversarial process and real-time selection of the best radar radiation parameter adjustment strategy under the current environment, thereby enhancing the anti-interception performance of radar transmission waveforms in complex electromagnetic environments. Attached Figure Description

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Figure 1 This is a flowchart of the present invention.

[0028] Figure 2 This is a detailed flowchart of the present invention.

[0029] Figure 3 This is a flowchart illustrating the workflow of the reinforcement learning decision network of this invention.

[0030] Figure 4 This is a curve showing the change in reward value during the training process of the reinforcement learning decision network of this invention.

[0031] Figure 5 This is a time-domain diagram of the radar transmission signal of this invention.

[0032] Figure 6 This is a frequency domain diagram of the radar transmission signal of this invention.

[0033] Figure 7 This is the range-Doppler two-dimensional spectrum obtained after processing the radar received signal according to the present invention.

[0034] Figure 8 This is a time-domain diagram generated after the radar transmission signal is intercepted in this invention.

[0035] Figure 9This is a frequency domain diagram generated after the radar transmission signal is intercepted by the present invention.

[0036] Figure 10 This is a diagram showing the radar flight trajectory and waveform switching during a single test of this invention.

[0037] Figure 11 This is the photoelectric confidence curve during a single test of the present invention. Detailed Implementation

[0038] The present invention will be further described below through specific embodiments.

[0039] like Figure 1 and Figure 2 As shown, the radar radiation characteristic control and optimization scheduling method based on reinforcement learning includes the following steps:

[0040] Step S1: Construct a waveform library and power control strategy suitable for various scenarios, and use the waveform library and power control strategy to configure a radar transmission signal that meets the target detection range requirements and achieves the radar peak power.

[0041] All signal parameters in the waveform library are flexibly configurable. Selectable transmit signals are divided into two categories: basic signals and agile signals. Basic signals include simple pulses, linear frequency modulated signals, and phase-coded signals. Agile signals include carrier frequency agile signals and pulse repetition interval agile signals. Power control strategies include high-power transmission, low-power transmission, and radar silence with switching to photoelectric detection.

[0042] Step S2: Detect radar transmission signals using the electromagnetic environment, and determine whether the detected radar transmission signals have been intercepted by the energy threshold detection method. If so, generate an interference signal; otherwise, do not generate an interference signal. The optimal energy threshold is obtained based on the Neyman-Pearson detection criterion and the overlikelihood ratio test.

[0043] Radar signals are transmitted by a radar transmitter. The interceptor detects these signals based on electromagnetic signals; the specific detection process is based on existing technology. Energy is extracted from the detected radar signals. If the extracted energy exceeds the optimal energy threshold T, the radar signal is determined to have been intercepted. Then, either repeater-type deception jamming or suppression jamming is generated according to a preset probability. Otherwise, the radar signal is determined not to have been intercepted, and no jamming signal is generated.

[0044] The energy threshold is the core parameter for interception determination. Based on the Neyman-Pearson detection criterion, the optimal energy threshold is solved through the Likelihood Ratio Test (LRT). The derivation process is as follows:

[0045] Suppose the sample (measured value) to be tested contains Each sampled data , represented as Establish a binary hypothesis testing model, where the null hypothesis is... Assuming the measured values ​​contain only noise and interference, a non-zero assumption is made. The measured value is the sum of noise, interference, and the target echo signal; the following two can be given. : and .

[0046] Based on the above assumptions, the core performance indicator for signal detection is defined as: detection probability. False alarm probability Probability of missed detection .

[0047] This derivation is based on a mean of 0 and a variance of . Gaussian white noise environment, assuming Let be an N-dimensional sampled data vector, whose sampled points are independent and identically distributed Gaussian random variables with zero mean.

[0048] When assuming At the time of its establishment, Follows an N-dimensional normal distribution ,in The identity matrix is ​​of order N, and the covariance matrix is ​​a scalar. The product of the identity matrix and the identity matrix.

[0049] When assuming At the time of its establishment, Follows an N-dimensional normal distribution ,in This is the sampling vector of the target radar signal.

[0050] The optimal implementation of the Neyman-Pearson detection criterion is the likelihood ratio test, which constructs the likelihood ratio function as the ratio of the joint probability density functions of the sampled data under the two hypotheses: ,in Null hypothesis The joint probability density function of the sampled data when it is established. Non-zero hypothesis The joint probability density function of the sampled data when the likelihood ratio test is established. The decision rule for the likelihood ratio test is... Substituting the joint probability density function of the N-dimensional Gaussian distribution into the likelihood ratio function and taking the logarithm yields the log-likelihood ratio. The optimal test statistic that matches the energy threshold detection method is obtained by simplification. It directly represents the energy level of the detected signal.

[0051] The decision rule based on the energy threshold detection method, when the test statistic... At that time, it was determined that the radar signal had been intercepted. At that time, it was determined that the radar signal had not been intercepted.

[0052] when At the time of its establishment, Since t follows a normal distribution, the expression for the false alarm probability can be obtained as follows: ,make It can be simplified to ,in The complementary error function is defined as follows: The optimal energy threshold can be solved. That is, the optimal energy threshold can be achieved through Data samples, assuming known noise variance and expected false alarm probability The calculation yields the optimal energy threshold. When a specific false alarm probability threshold is set according to the actual application scenario, the system can deduce the optimal energy threshold through the above statistical signal processing model.

[0053] Relay-based deception jamming is generated using digital radio frequency storage technology; the intercepted radar transmission signal is obtained by the interceptor receiver. The delay of forwarding-style deception interference is The magnitude of forwarding-style deception interference is Then the full-pulse sampling delay forwarding interference can be expressed as: .

[0054] Suppression jamming is a type of noise amplitude modulation jamming. Its core principle is to overwhelm or disrupt enemy signals by emitting high-intensity jamming signals, thereby blocking their information transmission. Specifically, it is represented as... ,in, DC bias, It is the amplitude modulation function, set to have a mean of 0 and a variance of 0. stochastic processes, The center frequency of the modulation function. Evenly distributed in Above, and with Independent and identically distributed.

[0055] During training, target echoes are generated based on radar transmit power and target radial distance. Interference commands from the interceptor are acquired, and the interference type is determined according to these commands. Simultaneously, interference signals are generated based on user-input interference parameters. Finally, the target echoes and interference signals are superimposed and synthesized to form a radar received signal suitable for system reinforcement learning.

[0056] Step S3: Superimpose the result of step S2 with the target echo corresponding to the radar transmitted signal to obtain the radar received signal. Process the radar received signal to obtain target status information and detection performance evaluation indicators.

[0057] Processing radar received signals includes the following steps:

[0058] Step S31, Preprocessing: Perform pulse compression and coherent accumulation on the radar received signal to generate a range-Doppler two-dimensional map;

[0059] Step S32, Target Information Extraction: Extract the real-time radial distance and velocity of the target from the range-Doppler image;

[0060] Step S33, Quantitative evaluation of detection performance: Calculate the detection performance evaluation index based on the extracted target information. The evaluation index includes target detection probability, tracking accuracy, range resolution and velocity resolution.

[0061] Step S34, Feedback Output: The extracted target state information and detection performance evaluation indicators are simultaneously output to the reinforcement learning decision network as the state input and reward calculation basis of the decision network.

[0062] Step S4: Using the constructed waveform library and power control strategy as the set of executable actions, with the interception probability as the core optimization objective, and the results of step S3 as the decision feedback basis, design and train a reinforcement learning decision network based on a deep Q-network.

[0063] The reinforcement learning decision network aims to optimize low-interception performance. Based on real-time radar operating status, electromagnetic environment information, and detection performance feedback, it outputs the optimal control strategy for radar radiation parameters, achieving multi-dimensional collaborative optimization of radar transmit waveform, radiated power, and operating mode. The system comprises a state awareness module and a decision module. The state awareness module collects core data from the entire radar operation process in real time and constructs the state vector input to the decision module. The core data in the entire radar operation process includes data from radar signal transmission, target echo acquisition, and electromagnetic signal reception and processing. State vector. It covers all core information about radar operation across three dimensions, specifically including the following three dimensions: relative situation between radar and target: real-time distance. Relative velocity Radar status: waveform type and power level of the current radar transmitted signal; Performance and environmental feedback information: target status information (real-time radial distance and speed of the target) and detection performance indicators, whether the radar transmitted signal has been intercepted, and type of interference.

[0064] State vector The data is fed into the decision-making module in real time. The decision-making module consists of an estimation network and a target value network with identical structures, both of which are fully connected neural networks. The estimation network is used for real-time decision-making, while the target value network is used for the stable training process.

[0065] Estimated value network output Used to evaluate the actions to be performed in the current state. Corresponding expected cumulative return in the future. Target value network output. The objective function is used to construct the optimization objective of the Q function. Based on the Bellman optimality equation, the expression for the optimization objective is: ,in This represents the current reward value. This indicates the currently selected discrete action.

[0066] Action space decision output is discrete action Defined as a combination of waveform actions and power actions. The waveform action set includes simple pulse, linear frequency modulation, pseudo-code phase modulation, and frequency agility; the power action set includes high-power transmission, low-power transmission, radar silence, and photoelectric detection.

[0067] The reward function is the optimization objective of the decision model, with the core objective being to improve the radar's low-intercept performance while also considering detection performance constraints and parameter switching stability. It is expressed as follows: ,in, This represents the cumulative duration during which the radar's transmitted signal is intercepted in a single training round. This refers to the total duration of a single training round. The farthest distance at which a radar signal can be successfully intercepted. This is the maximum detection range of the radar. This is the complexity coefficient of the current transmitted waveform, which corresponds one-to-one with the preset waveform library: the complexity coefficient of agile waveforms is higher than that of basic waveforms, and the complexity coefficient of simple pulses is the lowest. The duration of continuous transmission of the current waveform; The parameter switching penalty term is calculated from the number of waveform switching times, and the weighting coefficient satisfies... .

[0068] like Figure 3 As shown, the update process of the estimation network is as follows: First, input the current state vector. The current state is predicted through an estimation network. Corresponding different radar radiation parameter adjustment actions of Value, then through Greedy principle choice And proceed to the next state. At the same time, obtain the current reward value. The next state is calculated using the target value network. The largest The network parameters are updated by subtracting the estimated value from the calculated value. , represented as ,in, The Greed Principle The optimal action corresponding to the maximum Q-value output of the probability selection estimation network is chosen based on the probability. The probability of randomly selecting an action to explore the environment. The initial value is set to 1, and it gradually decreases linearly to 0.1 and remains thereafter as the number of training steps increases, thus achieving a balance between environmental exploration and strategy utilization.

[0069] After completing the set number of iterations, a soft update method is used. Update the parameters of the target value network This avoids strong coupling between the estimation network and the target network, improving the convergence and stability of the training process. For estimating the parameters of the value network, This represents the soft-interval update coefficient. Since the target value exhibits a certain degree of stability over a period of time, this can reduce the estimated value to some extent. Network and target value The coupling between networks improves the convergence and stability of the network.

[0070] During training, the simulation environment extrapolates a time step, updates the states of all entities, and generates the state vector for the next time step. The interference is generated by the receiver in two random time steps within each scene. In the next time step after the interference is generated, the network needs to select a suitable waveform to counter the interference, and the reward is given based on the countermeasure result. The system makes decisions based on an ε-greedy strategy. The radar loads the new waveform and power parameters determined by the decision and transmits it. At the same time, the simulation system calculates and displays the actual distance between the projectile and the target in real time based on the motion model.

[0071] The radar's actions will trigger feedback from the other side and the environment, thus completing an interaction. By conducting multiple experiments following the above steps, multiple sets of radar transmission waveform configuration strategies composed of different waveforms, power, and interception receivers with different operating parameters under various parameter scenarios are obtained. The reward value R is given based on its actual simulation performance. Figure 4 The graph shows the change in reward value during training, with the horizontal axis representing the number of training rounds and the vertical axis representing the cumulative reward value per round. In the early exploration phase, the reward value per round is lower, gradually increasing with each training round. The vertical axis represents the reward per round; the reward per round is smaller in the early exploration phase, and after 500 rounds, the reward fluctuations are smaller, indicating that the network has reached a good fit. After training, during testing, the current state vector is input into the trained model to output the optimal radar radiation parameter control action.

[0072] The following is a complete and reproducible description of this invention: For an X-band short-range detection radar, the core parameters are as follows: the initial signal is set to a linear frequency modulated signal with a pulse width of 10µs, a bandwidth of 10MHz, a PRT of 20µs, a sampling rate of 100MHz, a target velocity of 300m / s, an initial distance of 100km between the radar and the target, a carrier frequency agility range of 10MHz-20MHz for the frequency hopping signal, a QPSK phase-coded signal, and a receiver single-trip loss factor of 0.5dB / km. Selectable transmit power and operating mode strategies include: high-power transmission (peak power 100W), low-power transmission (peak power 10W), radar silence, and photoelectric detection. The receiver sequentially performs pulse compression and coherent accumulation processing on the received mixed signal to generate a range-Doppler two-dimensional map (RD map), extracts the target radial distance and velocity information, and outputs it to the reinforcement learning decision module.

[0073] The process is divided into two phases: model training and online deployment, forming a closed-loop control system throughout. In the model training phase, the simulation environment and reinforcement learning decision network parameters are initialized. Each simulation round lasts 100 time steps, updating the target situation and electromagnetic environment status at each step. An ε-greedy strategy is used to select radiation parameter actions, completing signal transmission, environmental interaction, reward calculation, and network parameter updates. The total training rounds are 2000. When the cumulative reward fluctuation is ≤5% for 100 consecutive rounds, the network is considered converged, and training is complete. In the online deployment phase, the trained model is loaded, and the current environmental interference is assessed by real-time acquisition of echo signals to select the optimal combination of radiation parameters and control the radar transmission module, completing real-time closed-loop control.

[0074] pass Figures 5 to 11 The testing process and performance verification of the present invention are described, and the reinforcement learning decision network after training convergence is loaded throughout the test.

[0075] In the initial stage of testing, the radar transmits a linear frequency modulated reference signal by default. The time-domain waveform of this signal is shown in the figure below. Figure 5 As shown, the frequency domain spectrum is as follows: Figure 6 As shown. During the test, the radar completed an environmental perception and decision-making process every 10 seconds and generated an RD map to assess the risk of signal interception and the degree of interference. Figure 7 The RD diagram generated by radar echo processing when the radial distance between the radar and the target is 1000m.

[0076] The intercept receiver determines the interception of radar signals based on energy threshold detection. If the detected signal energy is higher than the preset detection threshold, the signal is determined to have been successfully intercepted, and suppression jamming or deception jamming is generated according to a preset probability. If the signal energy is lower than the threshold, no jamming is generated. Figure 8 , Figure 9The time-domain waveform and frequency-domain spectrum of the intercepted signal are displayed respectively during the test, which clearly shows the time-domain and frequency-domain characteristics of the radar transmitted signal and verifies the determination of the signal interception status.

[0077] Figure 10 The curve showing the change in radial distance between the radar and the target over time during the test intuitively reflects the radar's continuous and stable tracking capability under the adaptive scheduling strategy of this invention. During the test, the radar's waveform, power, and operating mode adaptive scheduling logic are as follows:

[0078] (1) Waveform selection

[0079] Figure 10 In the diagram, a gray circular marker indicates that the radar maintains LFM signal transmission, and an asterisk (*) indicates that the radar has identified interception risk and interference effects and decides to switch the transmission waveform. As the radial distance between the radar and the target decreases, the waveform adaptive switching sequence is: LFM → pseudo-code phase modulation → LFM → frequency hopping → LFM → frequency hopping → LFM.

[0080] (2) Power adaptive control

[0081] In the initial stage of the test, the radar used high-power transmission. After detecting that the signal was intercepted and there was interference, it automatically switched to low-power transmission.

[0082] (3) Adaptive switching of working modes

[0083] Figure 10 During the 240s-290s interval, a hollow circle mark indicates that the radar is off and switched to photoelectric detection mode; Figure 11 The graph shows the change curve of the confidence level of photoelectric detection during the decision-making process. The black curve represents the actual change curve of the confidence level during the test, and the gray curve represents the ideal change curve of the confidence level. When the confidence level of photoelectric detection exceeds the 75% threshold, the radar automatically shuts down and switches to the external radiation source mode to complete the continuous tracking and detection of the target. After 290 seconds, when the confidence level of photoelectric detection falls below the threshold, the radar automatically restarts and returns to the radar detection mode.

[0084] The above test process fully realizes the adaptive scheduling of radiation parameters throughout the entire process of a single target detection, verifying the core capabilities of the present invention in multi-dimensional parameter collaborative optimization and dynamic environment adaptive adaptation.

[0085] In this invention, the terms "first," "second," and "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. The use of terms such as "upper," "lower," "left," "right," "front," and "rear" to indicate orientation or positional relationships is based on the orientation or positional relationships shown in the accompanying drawings and is only for the convenience of describing the invention, not to indicate or imply that the device referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation on the scope of protection of this invention. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0086] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0087] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A radar radiation characteristic control and optimization scheduling method based on reinforcement learning, characterized in that: Includes the following steps: Step S1: Construct a waveform library and power control strategy suitable for various scenarios, and use the waveform library and power control strategy to configure a radar transmission signal that meets the target detection range requirements and achieves the radar peak power. Step S2: Detect radar transmission signals using the electromagnetic environment, and determine whether the detected radar transmission signals have been intercepted by the energy threshold detection method. If so, generate an interference signal; otherwise, do not generate an interference signal. The optimal energy threshold is obtained based on the Neyman-Pearson detection criterion and the overlikelihood ratio test. Step S3: Superimpose the result of step S2 with the target echo corresponding to the radar transmitted signal to obtain the radar received signal. Process the radar received signal to obtain target status information and detection performance evaluation indicators. Step S4: Using the constructed waveform library and power control strategy as the set of executable actions, with the interception probability as the core optimization objective, and the results of step S3 as the decision feedback basis, design and train a reinforcement learning decision network based on a deep Q-network.

2. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 1, characterized in that: In step S1, the waveform library includes basic signals and agile signals. The basic signals include simple pulses, linear frequency modulation signals, and phase-coded signals. The agile signals include carrier frequency agile signals and pulse repetition interval agile signals. The power control strategy includes high-power transmission, low-power transmission, and radar silence and switching to photoelectric detection.

3. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 2, characterized in that: In step S2, energy is extracted from the detected radar transmission signal. When the extracted energy is greater than the optimal energy threshold, it is determined that the radar transmission signal has been intercepted. Then, either forwarding deception jamming or suppression jamming is generated according to a preset probability. Otherwise, it is determined that the radar transmission signal has not been intercepted and no jamming signal is generated.

4. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 3, characterized in that: In step S2, the energy threshold is expressed as ,in, For complementary error functions, This represents the probability of a false alarm. The null hypothesis means that the measured values ​​contain only noise and interference, and do not contain target echo signals. The measured values ​​are N-dimensional sampled data vectors, and each sampled data point is an independent and identically distributed Gaussian random variable with zero mean. Let be the standard deviation of the Gaussian random variable.

5. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 4, characterized in that: In step S2, the forwarding spoofing interference is generated using digital radio frequency storage technology, and is represented as follows: ,in, For intercepted radar transmission signals, Delay for forwarding spoofing interference, The amplitude of the forwarding deception interference; the suppression interference is noise amplitude modulation interference, represented as... ,in, DC bias, It is the amplitude modulation function, set to have a mean of 0 and a variance of 0. stochastic processes, The center frequency of the modulation function, Evenly distributed in Above, and with Independent and identically distributed.

6. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 5, characterized in that: Step S3, which processes the radar received signal, includes the following steps: Step S31: Perform pulse compression and coherent accumulation on the radar received signal to generate a range-Doppler two-dimensional map; Step S32: Extract the target's real-time radial distance and velocity from the range-Doppler image; Step S33: Calculate the detection performance evaluation index based on the extracted target information. The evaluation index includes target detection probability, tracking accuracy, range resolution and velocity resolution.

7. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 6, characterized in that: In step S4, the reinforcement learning decision network includes a state awareness module and a decision module. The state awareness module is used to collect core data from the entire radar operation process in real time and construct a state vector to be input to the decision module. The decision-making module includes an estimation network and a target value network with completely identical structures. The update process of the estimation network is as follows: first, input the current state vector. The current state is predicted through an estimation network. Corresponding different radar radiation parameter adjustment actions of Value, then through Greedy principle choice And proceed to the next state. At the same time, obtain the current reward value. The next state is calculated using the target value network. The largest The network parameters are updated by subtracting the estimated value from the calculated value. After completing the set number of iterations, a soft update method is used. Update the parameters of the target value network ,in, For estimating the parameters of the value network, This represents the soft-spacing update coefficient and the radar radiation parameter adjustment action. It is a combination of waveform action and power action.

8. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 7, characterized in that: In step S4, the state vector This includes the real-time distance and relative speed between the radar and the target; the waveform type and power level of the current radar signal; the real-time radial distance and speed of the target; detection performance evaluation indicators; whether the radar signal has been intercepted; and the type of interference.

9. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 8, characterized in that: The reward function is expressed as: ,in, This represents the cumulative duration during which the radar's transmitted signal is intercepted in a single training round. This refers to the total duration of a single training round. The farthest distance at which a radar signal can be successfully intercepted. This is the maximum detection range of the radar. This represents the complexity coefficient of the current transmitted waveform; The duration of continuous transmission of the current waveform; The parameter switching penalty term is calculated from the number of waveform switching times, and the weighting coefficient satisfies... .

10. The radar radiation characteristic control and optimization scheduling method based on reinforcement learning according to claim 9, characterized in that: The The Greed Principle The optimal action corresponding to the maximum Q-value output of the probability selection estimation network is chosen based on the probability. The probability of randomly selecting an action to explore the environment. The effect gradually decreases linearly with the increase of training steps.

Images