A radar jamming decision-making method, apparatus, system, and storage medium

Abstract

This invention discloses a radar jamming decision-making method, apparatus, system, and storage medium, comprising: Step S1, obtaining the potential relationship between radar operating mode switching and jamming signal release by the jammer based on the radar-jamming adversarial environment; Step S2, inputting the threat level corresponding to the current radar operating mode as an environmental state parameter into a competitive dual-depth Q-network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction, and parameter update operations to obtain the radar jamming decision result. The technical solution of this invention solves the problems of poor policy convergence and insufficient experience storage in traditional reinforcement learning algorithms.

Description

Technical Field

[0001] This invention belongs to the field of electronic countermeasures technology, and particularly relates to a radar jamming decision-making method, device, system, and storage medium. Background Technology

[0002] In modern radar electronic warfare environments, enemy cognitive radar can dynamically and adaptively adjust its operating mode based on battlefield situational awareness results, thereby achieving scanning, tracking, and locking effects on our penetrating units, posing a significant threat to our strategic objectives. Based on prior knowledge, the operational phases of cognitive radar can be roughly divided into five parts: coarse search, fine search, surveillance, tracking, and locking, with the threat level increasing sequentially. The purpose of our penetrating units in jamming cognitive radar is to guide it to the mode with the lowest threat level. To achieve this, it is necessary to construct an intelligent jamming decision structure targeting cognitive radar.

[0003] In traditional fields, there are many types of active jamming signals targeting radar, but they can be broadly categorized into three types: suppressive jamming, deceptive jamming, and comprehensive jamming. Introducing artificial intelligence technology into the radar jamming decision-making process, using deep reinforcement learning methods, and setting appropriate environmental characteristic parameters and reward functions, can significantly improve decision-making efficiency.

[0004] A search of existing technical literature revealed that Liang Meng et al., in their paper "A Radar Autonomous Jamming Decision-Making Method Based on Advantageous Action-Commentary" published in *Aerospace Electronic Countermeasures* (2023, 39(05):17-21.), proposed an autonomous radar jamming decision-making method based on advantageous action-commentary. This algorithm has the ability to quickly learn and make decisions on jamming strategies in unknown environments. However, the action selection strategy ε-greedy used is fixed throughout the process, which causes policy fluctuations in the later stages of strategy generation, affecting the jamming decision-making effect. In view of this, this invention proposes an adaptive greedy adjustment algorithm, which changes the fixed greedy value ε to an ε-greedy curve that varies with the number of steps, thus conforming to the general thinking of human learning. Zhao Jiachen et al., in their paper "A Radar Intelligent Decision Generation Algorithm Based on Deep Reinforcement Learning" published in *Modern Radar* (2022, 44(12):25-33.), proposed a decision network training algorithm based on dual deep Q network (DDQN), which effectively improves the intelligence level of radar and is used to overcome the overestimation of Q value caused by the coupling of the target network and the evaluation network in the deep Q network (DQN) algorithm. However, the experience playback mechanism used still has shortcomings. In view of this, the present invention proposes a dual experience replay mechanism based on an expert experience replay pool and a priority experience extraction mechanism. This mechanism divides the experience replay pool into a normal experience replay pool and an expert experience replay pool, and uses a priority experience sampling method in the normal experience replay pool to maximize the acquisition of high-value experience. Based on the priority experience sampling mechanism, this mechanism realizes dual deep-priority experience replay, enabling further utilization of high-value interaction experience, thereby improving the policy accuracy of the model.

[0005] In summary, traditional reinforcement learning algorithms still suffer from poor policy convergence and insufficient experience storage. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a radar jamming decision-making method, device, system, and storage medium, which solves the problems of poor policy convergence effect and insufficient experience storage in traditional reinforcement learning algorithms.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A radar jamming decision-making method, comprising:

[0009] Step S1: Based on the radar-jamming machine countermeasures environment, obtain the potential relationship between radar operating mode switching and jamming machine releasing jamming signals;

[0010] Step S2: Input the threat level corresponding to the current radar operating mode as an environmental state parameter into a competitive dual-depth Q-network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction and parameter update operations to obtain the radar jamming decision result.

[0011] As a preferred option, the radar includes five operating modes: coarse search mode, fine search mode, surveillance mode, tracking mode, and lock-on mode, with the corresponding threat levels increasing sequentially. The operational scenario is set as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode; after confirming the target, the radar switches to surveillance mode; after continuously monitoring for a preset period of time, the radar switches to tracking mode; after continuously tracking for a preset period of time, the radar switches to lock-on mode.

[0012] As a preferred option, the jammer's action library includes six different actions: amplitude modulation noise jamming, frequency modulation noise jamming, spectrum dispersion jamming, slice combination jamming, intermittent sampling and forwarding jamming, and no jamming measures.

[0013] The present invention also provides a radar jamming decision-making device, comprising:

[0014] A module is built to determine the potential relationship between radar operating mode switching and jamming signal release by the jammer in a radar-jamming confrontation environment.

[0015] The decision module is used to input the threat level corresponding to the current radar operating mode as an environmental state parameter into a competitive dual-depth Q-network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction and parameter update operations, so as to obtain the radar jamming decision result.

[0016] As a preferred option, the radar includes five operating modes: coarse search mode, fine search mode, surveillance mode, tracking mode, and lock-on mode, with the corresponding threat levels increasing sequentially. The operational scenario is set as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode; after confirming the target, the radar switches to surveillance mode; after continuously monitoring for a preset period of time, the radar switches to tracking mode; after continuously tracking for a preset period of time, the radar switches to lock-on mode.

[0017] As a preferred option, the jammer's action library includes six different actions: amplitude modulation noise jamming, frequency modulation noise jamming, spectrum dispersion jamming, slice combination jamming, intermittent sampling and forwarding jamming, and no jamming measures.

[0018] The present invention also provides a radar jamming decision system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a radar jamming decision method when executed by the processor.

[0019] The present invention also provides a storage medium storing a computer program that executes a radar jamming decision method when running.

[0020] This invention combines reinforcement learning and deep learning networks, proposing a Dueling Double Deep Q-Network based on Adaptive Greedy Adjustment and Priority Dual Experience Replay (AGA-PDER-D3QN). Based on the D3QN model, it combines an adaptive greedy adjustment algorithm with a dual experience replay mechanism based on an expert experience replay pool and a priority experience extraction mechanism. This enables the jamming agent to engage in countermeasures against the radar in a combat environment, continuously reducing the radar's threat level and ultimately completing the predetermined penetration mission. This invention avoids the shortcomings of manual decision-making in terms of effectiveness, optimizes the experience storage and training result retrieval methods in traditional reinforcement learning algorithms, and improves the scalability of the decision-making system in terms of jamming actions.

[0021] Compared to existing jamming strategy formulation methods, this invention addresses the problems of time-consuming manual decision-making, inability to predict radar situational changes, and the need for pre-setting jamming countermeasures. It designs a radar intelligent jamming decision-making method based on AGA-PDER-D3QN deep reinforcement learning. A coarse search mode is used as the initial radar operating mode, and the changes in Q-network parameters during the training process are analyzed. The deep reinforcement learning-based radar intelligent jamming decision-making method employed in this invention not only has faster decision-making speed and accuracy but also reduces the data storage pressure on the radar combat system to some extent. The jamming algorithm implementation scheme after decision-making best matches the current radar operating mode change trend. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1This is a flowchart of a radar jamming decision-making method according to an embodiment of the present invention;

[0024] Figure 2 This describes the change in reward value per round in this invention;

[0025] Figure 3 This illustrates the variation in the number of steps required per round in this invention.

[0026] Figure 4 This describes the changes in the final outcome of each round in this invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Example 1:

[0030] This invention provides a radar jamming decision-making method, comprising:

[0031] Step S1: Based on the radar-jamming machine countermeasures environment, obtain the potential relationship between radar operating mode switching and jamming machine releasing jamming signals;

[0032] Step S2: Input the threat level corresponding to the current radar operating mode as an environmental state parameter into a competitive dual-depth Q-network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction and parameter update operations to obtain the radar jamming decision result.

[0033] As one embodiment of the present invention, the radar includes five operating modes: coarse search mode, fine search mode, surveillance mode, tracking mode, and lock-on mode, with the corresponding threat levels increasing sequentially. The operational scenario is set as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode; after confirming the target, the radar switches to surveillance mode; after continuously monitoring for a preset period of time, the radar switches to tracking mode; after continuously tracking for a preset period of time, the radar switches to lock-on mode.

[0034] As one embodiment of the present invention, the jammer's action library includes six different actions, namely: noise amplitude modulation jamming, noise frequency modulation jamming, spectrum dispersion jamming, slice combination jamming, intermittent sampling forwarding jamming, and no jamming measures.

[0035] Example 2:

[0036] like Figure 1 As shown, an embodiment of the present invention provides a radar jamming decision-making method, including:

[0037] Step 1: Constructing an interference and countermeasure environment

[0038] (1) In this invention, the radar is configured with five operating modes: coarse search mode m0, fine search mode m1, surveillance mode m2, tracking mode m3, and lock-on mode m4, with the corresponding threat levels increasing sequentially. The operational scenario is set as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode; after confirming the target, the radar switches to surveillance mode; after continuous surveillance for a period of time, the radar switches to tracking mode; after continuous tracking for a period of time, the radar switches to lock-on mode.

[0039] (2) In this invention, the jammer's action library includes six different actions, specifically: amplitude modulation noise jamming a1, frequency modulation noise jamming a2, spectrum dispersion jamming a3, slice combination jamming a4, intermittent sampling and forwarding jamming a5, and no jamming measures a0. Among them, amplitude modulation noise jamming can use its frequency band blocking characteristics to suppress the radar in the coarse search mode, keeping the radar in the coarse search mode; frequency modulation noise jamming can use its frequency band aiming characteristics to further improve the suppression capability of the radar without changing the jamming power, thereby preventing the radar from switching from the fine search mode to the subsequent monitoring, tracking, and locking modes; when the radar is in the monitoring, tracking, and locking modes, the jammer can use deceptive jamming signals, such as spectrum dispersion jamming, slice combination jamming, and intermittent sampling and forwarding jamming, to construct false targets with high similarity and induce the radar's observed target to the false target, thereby ensuring the safety of the real target.

[0040] (3) Without taking any interference measures, its radar mode transition probability is set as follows:

[0041] When the jammer enters the radar detection area, the radar operates in the current mode m. i The mode transition probability is:

[0042]

[0043] Where t is the current time, ω c This is the radar detection factor; the larger the value, the stronger the radar's detection sensitivity.

[0044] (4) The mode transfer probability of the radar will also change under different jamming measures:

[0045] When the jammer takes suppressive jamming action, i.e., a j =a1 or a j When =a2, the radar is in the current mode m i The mode transition probability is:

[0046]

[0047] Where p(m) i ,a j (m) represents the current radar mode. i Next, perform action a j The decrease in the probability of post-mode transition.

[0048] When the jammer takes a deceptive jamming action, i.e., a j =a3、a j =a4 or a j When =a5, the radar is in the current mode m i The mode transition probability is:

[0049]

[0050] Where p(m) i ,a j (m) represents the current radar mode. i Next, perform action a j The probability that the radar will detect a false target.

[0051] (5) Except for coarse search mode m0 and lock-on mode m4, if the radar fails to detect the target in the current mode for N=10 consecutive times, the radar mode will revert to the previous mode.

[0052] (6) In this invention, the state set is set as follows:

[0053] S = [m t ,v Jx ,v Jy ,l Jx ,l Jy ]

[0054] Where, m t For the current radar operating mode, v Jx and v Jy These are the transverse and longitudinal components of the jammer's velocity, respectively. Jx and l Jy These are the horizontal and vertical coordinates of the jammer's location, respectively.

[0055] (7) In this invention, the action set is set as follows:

[0056] A = [a0, a1, a2, a3, a4, a5]

[0057] (8) In this invention, the single-step reward function is set as follows:

[0058] First, the changes in radar operating modes should be considered:

[0059]

[0060] That is, when the radar operating mode is increased, the agent should be punished accordingly; when the radar operating mode is decreased, the agent should be rewarded accordingly; when the radar operating mode remains unchanged, no reward or punishment should be given to the agent.

[0061] Secondly, the penetration distance of the jammer should be taken into account:

[0062]

[0063] The greater the distance the jammer can penetrate, the greater the reward given to the agent.

[0064] In addition, the global reward after the final breakthrough should be considered, namely:

[0065]

[0066] Therefore, the single-step reward function can be expressed in the following form:

[0067]

[0068] Where x is the total number of penetration steps required at the end of a single training round.

[0069] The state set, action set, and single-step reward function are the core pre-structures of reinforcement learning, providing the necessary preconditions for training the perturbation strategy.

[0070] Step 2: Constructing a radar jamming decision-making mechanism

[0071] (1) In this invention, a Dueling Double Deep Q-Network (D3QN) is used as the basic decision-making model. Its Q-value network adopts a fully connected Deep Neural Network (DNN) structure, and additionally adds a state concern layer and an action value layer to the hidden layers. This enables the decision-making model to distinguish the subjects affected by the state, avoiding fluctuations in training performance caused by environmental changes. Furthermore, this model transfers the right to estimate the action at the next time step from the Q-target network to the Q-estimation network, while the Q-value calculation at the next time step is still performed by the Q-target network. This decouples action selection and Q-value calculation, effectively reducing the probability of overestimation in the model.

[0072] (2) In this invention, the action selection mechanism of the decision-making model should be more in line with general learning thinking, namely, exploration and trial in the early stage to obtain as many diverse experience feedbacks as possible; exploration and learning in the middle stage to guide the model to implement higher-value actions; and learning and consolidation in the later stage to generate feasible interference strategies as soon as possible. Therefore, this invention designs an ε-greedy action selection mechanism based on an adaptive greedy adjustment algorithm.

[0073] The adaptive greedy adjustment algorithm consists of two parts: a baseline curve and greedy adjustment coefficients. First, a monotonically decreasing curve l(t) with a variation interval of [0,1] is selected:

[0074]

[0075] Then, the greedy adjustment coefficient is calculated using past historical experience.

[0076]

[0077] Among them, T μ The period of change of the greedy adjustment coefficient is given, and the initial value of the greedy adjustment coefficient is μ0 = 0.

[0078] The above formula can be interpreted as follows: when the average historical reward is less than or equal to 0, it indicates that the model has not yet learned effective experience. In this case, the greedy value needs to be increased to make the model more inclined to continue exploring the environment. When the average historical reward is greater than 0, it indicates that the model has learned effective experience. In this case, the greedy value needs to be decreased to make the model more inclined to select the optimal action. Therefore, the greedy value at the current moment can be defined as:

[0079] ε t =l(t)+μ i

[0080] This mechanism dynamically adjusts the greed value based on historical experience and the benchmark greed adjustment curve, enabling the agent to adaptively adjust its action selection tendency according to the learning effect, effectively improving the flexibility of the model's action selection.

[0081] (3) In this invention, to further improve the efficiency of the decision-making model in utilizing experience, a dual experience replay mechanism based on an expert experience replay pool and a priority experience extraction mechanism is designed. This mechanism divides the experience replay pool into a general experience replay pool and an expert experience replay pool, and uses a priority experience sampling method in the general experience replay pool to obtain as much high-value experience as possible. At the same time, a random sampling mechanism is used to sample the expert experience replay pool, and the samples obtained from the general experience pool are mixed and then fed into the model for learning.

[0082] (4) By combining the D3QN model mentioned in (1) with the mechanisms in (2) and (3), a competitive dual deep Q-network based on adaptive greedy adjustment and priority dual experience replay is obtained. This model is then used to learn the interference strategy.

[0083] (5) Obtain the current radar operating mode. Based on the radar operating mode, jammer speed, and its relative position, obtain the current state parameter s. t The Q-values ​​of all actions are then fed into a Q-estimation network for evaluation. After evaluating all action Q-values, the current action 'a' is selected based on an adaptive greedy adjustment algorithm and an ε-greedy action selection mechanism. t ∈A, where A is the action set.

[0084] The ε-greedy action selection mechanism, which incorporates the adaptive greedy adjustment algorithm, can be summarized as follows:

[0085]

[0086] Where ε t It can be obtained from the greedy value calculation method in (2).

[0087] (6) Set the current action a t Applying this to the environment, we obtain the state s at the next moment. t+1 , and according to s t and s t+1 Calculate the current reward r t . This experience string [s t ,a t ,r t ,s t+1 The experience strings are stored in the ordinary experience replay pool in chronological order, and the priority value corresponding to each experience string is calculated. The calculation process is as follows:

[0088] p i =|δi |+β

[0089] Where, δ i Let be the timing error of the i-th experience string, and β be a very small non-zero constant to ensure that the extraction priority of all experiences is greater than 0. Furthermore, the timing error of the i-th experience string can be expressed as:

[0090] δ i =r i +γ·Q(s i ′,a i ′;θ - )-Q(s i ,a i ;θ)

[0091] Here, γ is the decay factor, which is used to control the degree of influence of past experience on the value of the current moment.

[0092] Therefore, the probability of extracting each experience string is:

[0093]

[0094] Where α∈[0,1] represents the priority level of sampling. The higher α is, the more the model tends to use priority sampling to extract empirical samples. When α=0, the sampling method changes from priority sampling to random sampling.

[0095] Finally, the priority values ​​are stored in a summation tree structure.

[0096] (7) Based on the status value and reward value in the current experience string, determine whether it meets the conditions for entering the expert experience replay pool. Each status value corresponds to an admission threshold, and the initial threshold is 0. When the reward value of the experience is greater than the threshold, it is entered into the expert experience replay pool, and the admission threshold is adjusted to the reward value of the experience.

[0097] (8) Repeat steps (5)-(7) until the data in the normal experience replay pool meets the minimum sampling requirements.

[0098] (9) The ordinary experience replay pool is sampled using the priority experience sampling mechanism, and the expert experience replay pool is sampled using the random sampling mechanism. The sampled data from the two experience pools are mixed and then given to the Q-value network for learning.

[0099] Based on the next time-instance state s in the sample t+1 And estimate the action a′ to calculate the Q target network output value:

[0100]

[0101] Based on the current state s in the sample tand current action a t Calculate the Q-estimated network output value:

[0102] Q evaluate =Q(s) t ,a t ;θ)

[0103] Using the above calculation results and the current reward r t Parameters such as the attenuation factor γ are used to calculate the model loss function, where the loss function is in the form of a mean squared error function:

[0104]

[0105] The gradient value of this function is used to perform reverse gradient updates on the Q-estimation network.

[0106] (10) Set the number of network parameter migration intervals, that is, after a certain number of Q-estimated network weight updates, the internal weight parameters of the Q-estimated network are completely copied to the Q target network in a hard update manner.

[0107] (11) Repeat (5)-(10) until the Q-value network converges, that is, the decision results of the Q target network and the Q estimation network are almost consistent.

[0108] After the model is trained, the Q-target network provides the optimal action suggestion based on the current state. After the jammer executes the optimal action, the environment changes. By repeating the above operation, a continuous jamming strategy can be obtained.

[0109] like Figure 2 The figure shows the change in reward value per round in this invention. It can be seen that as the model continuously learns from the interference interaction experience, its average round reward value remains largely positive after approximately 300 rounds. This indicates that the interference strategy learned by the agent after 300 rounds has achieved a certain degree of effectiveness.

[0110] like Figure 3 The diagram shows the variation in the number of steps required per round in this invention. It can be seen that as the model continuously learns from interference interaction experience, the number of action steps required per round decreases to about 4 after approximately 300 rounds, indicating that the agent has learned the fastest interference implementation path to reach the non-threatening radar operating mode.

[0111] like Figure 4 The diagram shows the changes in round results in this invention. It can be seen that the agent continuously engages in trial and error and makes corrections before reaching 300 rounds. After reaching 300 rounds, subsequent rounds successfully reduce the radar's operating mode to the lowest coarse search mode, minimizing the threat level.

[0112] This demonstrates that the combination of interference algorithms provided after interference decision-making has an excellent overall interference effect on radar.

[0113] Example 3:

[0114] This invention provides a radar jamming decision-making device, comprising:

[0115] A module is built to determine the potential relationship between radar operating mode switching and jamming signal release by the jammer in a radar-jamming confrontation environment.

[0116] The decision module is used to input the threat level corresponding to the current radar operating mode as an environmental state parameter into a competitive dual-depth Q-network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction and parameter update operations, so as to obtain the radar jamming decision result.

[0117] As one embodiment of the present invention, the radar includes five operating modes: coarse search mode, fine search mode, surveillance mode, tracking mode, and lock-on mode, with the corresponding threat levels increasing sequentially. The operational scenario is set as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode; after confirming the target, the radar switches to surveillance mode; after continuously monitoring for a preset period of time, the radar switches to tracking mode; after continuously tracking for a preset period of time, the radar switches to lock-on mode.

[0118] As one embodiment of the present invention, the jammer's action library includes six different actions, namely: noise amplitude modulation jamming, noise frequency modulation jamming, spectrum dispersion jamming, slice combination jamming, intermittent sampling forwarding jamming, and no jamming measures.

[0119] Example 4:

[0120] This invention also provides a radar jamming decision system, including: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a radar jamming decision method when executed by the processor.

[0121] Example 5:

[0122] This invention also provides a storage medium storing a computer program that executes a radar jamming decision-making method during runtime.

[0123] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A radar jamming decision-making method, characterized in that, include: Step S1: Based on the radar-jamming machine countermeasures environment, obtain the potential relationship between radar operating mode switching and jamming machine releasing jamming signals; Step S2: Input the threat level corresponding to the current radar operating mode as an environmental state parameter into the competitive dual deep Q network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction and parameter update operations to obtain the radar jamming decision result. The radar has five operating modes: coarse search mode, fine search mode, surveillance mode, tracking mode, and lock-on mode, with the corresponding threat levels increasing sequentially. The operational scenario is as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode. After the target is confirmed, the radar switches to surveillance mode; after continuous surveillance for a preset period of time, the radar switches to tracking mode; after continuous tracking for a preset period of time, the radar switches to lock mode. The jammer's action library contains six different actions: amplitude modulation noise jamming, frequency modulation noise jamming, spectrum dispersion jamming, slice combination jamming, intermittent sampling and forwarding jamming, and no jamming measures. In step 2, within the competitive dual-depth Q-network, an adaptive greedy adjustment algorithm is designed. Action selection mechanism; The adaptive greedy adjustment algorithm consists of two parts: a baseline curve and a greedy adjustment coefficient. First, a range of variation is selected as... monotonically decreasing curve : Calculating the greedy adjustment coefficient using past historical experience. in, Let $\frac{ ... ; The greedy value at the current moment can be defined as: The greed value will be dynamically adjusted based on historical experience and the benchmark greed adjustment curve, enabling the agent to adaptively adjust its action selection tendency according to the learning effect. In step S2, in the competitive dual-depth Q-network, a dual experience replay mechanism based on expert experience replay pool and priority experience extraction mechanism is designed. The experience replay pool is divided into ordinary experience replay pool and expert experience replay pool. Priority experience sampling method is used in ordinary experience replay pool. At the same time, random sampling mechanism is used to sample the expert experience replay pool. The samples are mixed with the samples obtained from the ordinary experience pool and then fed to the model for learning. Step S2 specifically includes: Step 1: Obtain the current state parameters based on the radar operating mode, jammer speed, and its relative position. The results are fed into a Q-estimation network for evaluation; after evaluating the Q-values ​​of all actions, an adaptive greedy adjustment algorithm is used. Action selection mechanism selects the current action. ,in For action sets; After incorporating the adaptive greedy adjustment algorithm The action selection mechanism can be summarized as follows: in, It is a greedy value; Step 2: Set the current action Applying it to the environment, to obtain the state at the next moment. and according to and Calculate the current reward ; this experience string They are stored in the ordinary experience replay pool in chronological order, and the priority value corresponding to the experience string is calculated as follows: in, For the first Timing error of an empirical series Let be a very small constant that is not equal to 0; the th The timing error of an empirical string can be expressed as: in, This is a decay factor used to control the degree to which past experience influences the value of the current moment; The probability of drawing each experience string is: in, To prioritize the selection based on importance, The higher the value, the more the model tends to use a priority sampling method to draw empirical samples. At that time, the sampling method changed from priority sampling to random sampling; Finally, the priority value is stored in the summation tree structure; Step 3: Based on the status value and reward value in the current experience string, determine whether it meets the conditions for entering the expert experience replay pool; each status value corresponds to an admission threshold, and the initial threshold is 0. When the reward value of the experience is greater than the threshold, it is entered into the expert experience replay pool, and the admission threshold is adjusted to the reward value of the experience. Step 4: Repeat steps 1-3 until the data in the ordinary experience replay pool meets the minimum sampling requirements; Step 5: Use the priority experience sampling mechanism to sample the ordinary experience replay pool, and use the random sampling mechanism to sample the expert experience replay pool. Mix the sampled data from the two experience pools and then feed them into the Q-value network for learning. Based on the next time-instance state in the sample and estimated action Calculate the Q-target network output value: Based on the current state in the sample and current action Calculate the Q-estimated network output value: Utilize current rewards Attenuation factor Parameters, calculate the model loss function, where the loss function is in the form of a mean squared error function: The gradient value of this function is used to perform back-gravity updates on the Q-estimation network; Step 6: Set the number of network parameter migration intervals, that is, after a certain number of Q-estimated network weight updates, completely copy the internal weight parameters of the Q-estimated network to the Q-target network in a hard update manner; Step 7: Repeat steps 1-6 until the Q-value network converges, meaning the decision results of the Q target network and the Q estimation network are almost identical. After the model is trained, the Q-target network provides the best action suggestion based on the current state. After the jammer executes the optimal action, the environment changes. By repeating the above operation, a continuous jamming strategy can be obtained.

2. A radar jamming decision-making device for implementing the radar jamming decision-making method of claim 1, characterized in that, include: A module is built to determine the potential relationship between radar operating mode switching and jamming signal release by the jammer in a radar-jamming confrontation environment. The decision module is used to input the threat level corresponding to the current radar operating mode as an environmental state parameter into a competitive dual-depth Q-network decision structure based on adaptive greedy adjustment and priority dual experience replay for value evaluation, experience input, experience extraction and parameter update operations, so as to obtain the radar jamming decision result. The radar has five operating modes: coarse search mode, fine search mode, surveillance mode, tracking mode, and lock-on mode, with the corresponding threat levels increasing sequentially. The operational scenario is as follows: the radar initially operates in coarse search mode; after detecting a suspected target, the radar switches to fine search mode. After the target is confirmed, the radar switches to surveillance mode; after continuous surveillance for a preset period of time, the radar switches to tracking mode; after continuous tracking for a preset period of time, the radar switches to lock mode. The jammer's action library contains six different actions: amplitude modulation noise jamming, frequency modulation noise jamming, spectrum dispersion jamming, slice combination jamming, intermittent sampling and forwarding jamming, and no jamming measures.

3. A radar jamming decision-making system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the radar jamming decision method as described in claim 1 when executed by the processor.

4. A storage medium, characterized in that, The storage medium stores a computer program, which executes the radar jamming decision-making method as described in claim 1 when it runs.

Images