A radar jamming decision-making method based on asynchronous multi-threading

Abstract

The present invention belongs to the field of radar cognitive jamming technology, and particularly relates to a radar jamming decision method based on asynchronous multithreading, by constructing a jammer model that interacts with a multifunctional radar environment, wherein the jammer model comprises: a number of jamming threads that select jamming patterns acting on the radar at a specified time from a jamming library through a policy network and use a value network to analyze the effectiveness of the jamming patterns during the radar task state transition process, and a global network for summarizing and distributing policy network and value network parameters in each jamming thread to update global network parameters; and using the jammer model to interfere with the target multifunctional radar cognition to achieve real-time confrontation defense. The present invention can greatly improve time efficiency, meet real-time requirements, and facilitate application in actual scenarios while ensuring decision-making performance.

Description

Technical Field

[0001] The invention belongs to the technical field of radar cognitive jamming, and in particular relates to a radar jamming decision method based on asynchronous multi-threading. Background Art

[0002] In radar countermeasure defense, special electronic equipment and devices are used to detect and interfere with the opponent's radar, which mainly includes radar reconnaissance and radar interference. The purpose is to obtain the opponent's radar technical information so as to take corresponding measures to hinder the normal operation of the radar and reduce the radar's working efficiency. In radar jamming technology, cognitive theory is introduced into radar jamming decision-making. Under the assumption that the number of working modes is known, a cognitive radar countermeasure process design based on Q-learning algorithm is proposed; and on the basis of unknown radar working modes, an intelligent radar countermeasure process design based on Q-learning is proposed; in addition, by analyzing the working state of multi-function radar and the corresponding interference style, a radar state transition diagram is constructed, and the influence of various parameters on the interference decision performance, as well as the decision-making process and the influence of transition probability on the decision path under the addition of new states are simulated and analyzed. However, the efficiency of this strategy gradually decreases with the increase of the number of radar tasks of MFR. The confrontation between the jammer and the jammed party in the existing radar countermeasure defense is fleeting. The existing technology still cannot meet the high real-time requirements of radar countermeasure decision-making. Summary of the invention

[0003] To this end, the present invention provides a radar interference decision-making method based on asynchronous multi-threading, which can greatly improve time efficiency and meet real-time requirements while ensuring decision-making performance, and is convenient for application in actual scenarios.

[0004] According to the design scheme provided by the present invention, a radar interference decision method based on asynchronous multi-threading mode includes the following contents:

[0005] Construct a jammer model that interacts with a multifunctional radar environment, the jammer model comprising: a plurality of jamming threads and a global network, wherein the jamming thread uses a policy network to select a jamming pattern that acts on the radar at a specified time from a jamming library, and uses a value network to analyze the effectiveness of the jamming pattern during the radar task state transition process, and the global network summarizes and distributes the policy network and value network parameters in each jamming thread to update the global network parameters;

[0006] Real-time countermeasure defense is achieved by interfering with the target's multi-function radar cognition through the jammer model.

[0007] As a radar jamming decision method based on asynchronous multi-threading in the present invention, further, in the jamming thread, the strategy network π(a t |s t ;θ) by calculating the radar mission state s at the specified time t tInterference style action taken a t The probability distribution of and selects interference styles from the interference library based on the probability distribution, where θ represents the policy network parameters.

[0008] As a radar interference decision method based on asynchronous multi-threading mode of the present invention, further, using the value network V(s t θ v ) Get the radar task status s at the specified time t t Perform a disruptive style action a t The advantage function is constructed based on the value function and the interference pattern return value. The advantage function is introduced to evaluate the state-action pair (radar task state s t , interference style action a t ), where the interference style reward value is obtained by the instant reward and attenuation factor of the radar state before and after the radar task state changes when the interference style action is taken at the next moment, θ v Represents the value network parameters.

[0009] As a radar interference decision method based on asynchronous multi-threading mode of the present invention, further, the advantage function is expressed as: Among them, n represents the maximum number of iterations, r t Indicates interference style action a t The immediate reward before and after the radar task status changes, γ i Represents the decay factor at iteration round i.

[0010] As a radar interference decision method based on asynchronous multi-threading mode of the present invention, further, interference pattern action a t The instant reward r before and after the radar task status changes t It is expressed as: Among them, T n →T end Indicates radar threat level T n Transition to minimum T end The state of T n →T n Indicates that the radar threat level remains unchanged, T n →T n-i Indicates that the radar threat level increases by i levels, T n →T n+i It means that the radar threat level is reduced by i levels.

[0011] As a radar interference decision method based on asynchronous multi-threading in the present invention, further, in each interference thread, the iterative update mechanism of network parameters is utilized and the iteration round is ended by judging whether the radar threat level corresponding to the radar task state at the moment of adopting the interference style is in the lowest threat level state or whether the number of iterations meets the maximum number of iterations, and the network parameters of the interference thread in the last iteration process are uploaded to the global network, so that the network parameters uploaded by each interference thread are summarized by the global network and distributed to each interference thread network.

[0012] Beneficial effects of the present invention:

[0013] The present invention uses a global network and multiple interference threads to achieve cognitive interference on the target's multi-function radar, meeting the high real-time requirements of radar confrontation decision-making, and can greatly improve time efficiency while ensuring decision-making performance. It has multi-threaded processing capabilities and fast calculation speed, and has good application prospects. It is further verified by experimental data that as the number of radar tasks that can be executed by the target increases, the higher the decision-making accuracy of this solution, the more obvious the advantage. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the process of the radar cognitive interference decision method based on A3C in the embodiment;

[0015] Figure 2 This is a schematic diagram of the jammer model architecture in the embodiment;

[0016] Figure 3 It is a diagram showing a moving average reward curve in the embodiment. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.

[0018] The embodiment of the present invention provides a radar interference decision method based on asynchronous multi-threading. Figure 1 As shown, it contains the following contents:

[0019] S101, constructing a jammer model that interacts with a multifunctional radar environment, wherein the jammer model includes: a plurality of jamming threads and a global network, wherein the jamming thread uses a policy network to select a jamming pattern that acts on the radar at a specified time from a jamming library, and uses a value network to perform validity analysis on the jamming pattern during the radar task state transition process, and the global network summarizes and distributes policy network and value network parameters in each jamming thread to update global network parameters;

[0020] S102. Real-time countermeasure defense is achieved by interfering with the target's multi-function radar cognition through the jammer model.

[0021] Based on the asynchronous advantage actor-critic algorithm (A3C), multiple agents are executed in parallel and asynchronously using the multi-threading function of the CPU. Figure 2 As shown in the figure, the global network is equivalent to a central brain, which summarizes and distributes the network parameters of each interference thread; the interference thread includes the jammer model, the environment model (the target multi-function radar) and the interaction mechanism. The jammer model interacts with the environment through three ways: interference style selection, radar task state description and interference style effectiveness evaluation. Based on the A3C algorithm, the target radar cognitive interference decision in radar countermeasure defense can greatly improve the time efficiency and accuracy while ensuring the decision-making performance, which is convenient for application in actual scenarios.

[0022] As a radar jamming decision method based on asynchronous multi-threading in an embodiment of the present invention, further, in the jamming thread, the strategy network π(a t |s t ;θ) by calculating the radar mission state s at the specified time t t Interference style action taken a t The probability distribution of the interference pattern is selected from the interference library based on the probability distribution, where θ represents the policy network parameter. Further, using the value network V(s t θ v ) Get the radar task status s at the specified time t t Perform a disruptive style action a t The advantage function is constructed based on the value function and the interference pattern return value. The advantage function is introduced to evaluate the state-action pair (radar task state s t , interference style action a t ), where the interference style reward value is obtained by the instant reward and attenuation factor of the radar state before and after the radar task state changes when the interference style action is taken at the next moment, θ v represents the value network parameter. Further, the advantage function can be expressed as: Among them, n represents the maximum number of iterations, r t Indicates interference style action a t The immediate reward before and after the radar task status changes, γ i Represents the decay factor at iteration round i.

[0023] For the enemy radar in radar confrontation, its radar tasks have a certain priority ranking. The higher the priority of the radar task, the higher the threat level to us (the threat level is mainly our own assessment of the radar status at that moment). If the priority of the radar task (that is, the threat level) decreases after the jammer implements the selected jamming pattern, the selected jamming pattern is effective; if the threat level of the radar task does not change or increases, the selected jamming pattern is invalid. Therefore, the jamming pattern action a t The instant reward r before and after the radar task status changes t It can be expressed as: Among them, T n →T end Indicates radar threat level T n Transition to minimum T end The state of T n →T n Indicates that the radar threat level remains unchanged, T n →T n-i Indicates that the radar threat level increases by i levels, T n →T n+i It means that the radar threat level is reduced by i levels.

[0024] As a radar interference decision method based on asynchronous multi-threading in an embodiment of the present invention, further, in each interference thread, an iterative update mechanism of network parameters is utilized and the iteration round is ended by judging whether the radar threat level corresponding to the radar task state at the moment of adopting the interference style is in the lowest threat level state or whether the number of iterations meets the maximum number of iterations, and the network parameters of the interference thread in the last iteration process are uploaded to the global network, so that the global network is used to summarize the network parameters uploaded by each interference thread and distribute them to each interference thread network.

[0025] In the jammer model, the radar task state of the multi-function radar at a certain moment is detected as s. t , use the strategy network to select a jamming style from the jamming library to act on the radar, analyze the effectiveness of the jamming style according to the change of the radar task, obtain the immediate reward r, and detect the radar task state at the next moment; repeat the above steps, when the radar task threat level reaches the lowest state or the number of iterations is greater than the maximum number of iterations of this thread, the jammer model updates the strategy network parameters θ' and the value network parameters θ v ', and finally the network parameters θ′θ in this interference thread v 'Passed to the global network. Its implementation algorithm can be designed as follows:

[0026]

[0027]

[0028] In order to verify the effectiveness of this solution, the following is a further explanation of this solution combined with experimental data:

[0029] The threat level of the radar task conversion relationship table is set to 15. Through the above algorithm and according to the corresponding conversion relationship table, 10 independent experiments are carried out, and each experiment simulates 5000 rounds. And the model training time and moving average reward of the algorithm in the embodiment of this case and the cognitive interference decision series method based on DQN (DQN, Double DQN and Prioritized Replay DQN, where DoubleDQN and Prioritized Replay DQN are two improved algorithms of DQN, which are applied to the cognitive interference decision of multi-function radar confrontation as the comparison object of the algorithm in the embodiment of this case) are compared and analyzed.

[0030] The model training time is shown in the following table, which is the average time per round for different comparison methods:

[0031]

[0032] It can be seen from the above table that compared with the cognitive interference decision-making method based on DQN, the average time per round of the other three cognitive interference decision-making methods is reduced to a certain extent. In the embodiment of this case, the cognitive interference decision-making algorithm based on A3C has the most obvious advantage, reducing the average time per round by about 60 times.

[0033] Moving average reward pair Figure 3 As shown in the figure, when the number of radar tasks increases, the moving average reward of the cognitive interference decision algorithm based on A3C in this case has certain advantages over the DQN series algorithms in the entire stage, which indicates that the cognitive interference decision method based on A3C has better performance when the radar tasks are complex.

[0034] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0035] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A radar jamming decision-making method based on asynchronous multi-threading, characterized in that: Contains the following: A jammer model interacting with a multifunctional radar environment is constructed. The jammer model includes: several jamming threads and a global network. The jamming thread uses a policy network to select a jamming pattern acting on the radar at a specified time from a jamming library, and uses a value network to analyze the effectiveness of the jamming pattern during the radar task state transition. The global network summarizes and distributes the policy network and value network parameters in each jamming thread to update the global network parameters. In the jamming thread, the policy network π(a t |s t ;θ) by calculating the radar mission state s at the specified time t t Interference style action taken a t The probability distribution of the interference pattern is selected from the interference library based on the probability distribution, θ represents the policy network parameters; using the value network V(s t θ v ) Get the radar task status s at the specified time t t Perform a disruptive style action a t The advantage function is constructed based on the value function and the interference pattern reward value, and the advantage function is used to evaluate the advantage of the state-action pair. The state-action pair is represented as (radar task state s t , interference style action a t ), the interference style reward value is obtained by the instant reward and attenuation factor of the radar state at the current moment and the radar task state before and after the interference style action is taken at the next moment, θ v represents the value network parameter; the advantage function is expressed as: n represents the maximum number of iterations, r t Indicates interference style action a t The immediate reward before and after the change of the radar task status, γ i represents the attenuation factor under iteration round i; Real-time countermeasure defense is achieved by interfering with the target's multi-function radar cognition through the jammer model.

2. The radar interference decision-making method based on asynchronous multi-threading according to claim 1 is characterized in that: Interference style actiona t The instant reward r before and after the radar task status changes t It is expressed as: Among them, T n →T end Indicates radar threat level T n Transition to minimum T end The state of T n →T n Indicates that the radar threat level remains unchanged, T n →T n-i Indicates that the radar threat level increases by i levels, T n →T n+i It means that the radar threat level is reduced by i levels.

3. The radar interference decision-making method based on asynchronous multi-threading according to claim 1 is characterized in that: In each interference thread, the iterative update mechanism of network parameters is used to end the iteration round by judging whether the radar threat level corresponding to the radar task state at the time of adopting the interference style is at the lowest threat level or whether the number of iterations meets the maximum number of iterations. The network parameters of the interference thread in the last iteration are uploaded to the global network, so that the global network can be used to summarize the network parameters uploaded by each interference thread and distribute them to each interference thread network.

Images