A multi-radar node cooperative anti-jamming strategy pushing method

Abstract

The application discloses a kind of multi-radar node cooperative anti-interference strategy pushing method, utilize the branch type Dueling Double DQN strategy generation network trained according to the observation state of intelligent agent, for each action branch of each radar respectively select corresponding branch action, generate the branch action matrix corresponding to radar;Using control mapper, branch action matrix is converted into engineering control parameter, after legality verification, it is issued to each radar to execute complete radar signal processing link, to realize anti-interference strategy pushing;When training strategy generation network, task type reward function consisting of local detection, system level detection, tracking continuity, false alarm penalty and action switching penalty is used, which meets the radar task target, while considering local detection, system level detection, real target sorting, track confirmation, false alarm rate and action switching cost, encourage strategy to achieve balance between signal quality, target continuous tracking and system stability.

Description

Technical Field

[0001] This invention belongs to the field of radar anti-jamming technology, specifically relating to a method for pushing out multi-radar node collaborative anti-jamming strategies. Background Technology

[0002] With the continuous development of modern electronic countermeasures technology, radar systems face complex jamming scenarios, including narrowband targeting jamming, broadband jamming, slice-based forwarding jamming, dense false target jamming, and multiple jamming sources and systems operating concurrently. Suppression jamming increases the noise floor and reduces the target peak significance; deceptive jamming creates false peaks by forwarding, copying, or delaying the modulation of radar signals, suppressing the detection ranking of real targets and affecting target discovery, confirmation, and continuous tracking. In multi-radar cooperative detection scenarios, different radars, targets, and jammers have different spatial geometric relationships, carrier frequency conditions, and jamming levels. The locally optimal action of a single radar does not necessarily lead to the system-wide optimal effect. If multiple nodes lack global situational awareness fusion and differentiated division of labor mechanisms, problems such as repetitive conservatism, lack of coordination, or conflicting actions may occur. Therefore, the generation of anti-jamming strategies in complex electromagnetic environments needs to simultaneously consider the global cooperative information of multiple radars, the local observation differences of a single radar, and engineering feasibility constraints.

[0003] Traditional radar anti-jamming methods typically rely on expert rules or fixed policy tables. These methods are structurally clear and simple to implement, showing some effectiveness in single or stable jamming scenarios. However, when facing dynamic, complex jamming, they struggle to adjust in real-time based on target motion, jamming source location, radar's current operating mode, historical control actions, and detection / tracking feedback. This leads to problems such as policy response lag, inappropriate combinations of anti-jamming measures, increased false alarms, or decreased track continuity. Reinforcement learning provides a new technical path for the adaptive generation of radar anti-jamming strategies. However, if only jamming type, signal-to-noise ratio, or simplified situational variables are treated as abstract states, and anti-jamming methods as a few discrete actions, the policy learning results are difficult to demonstrate their effectiveness in the real signal processing chain. Furthermore, if all control variables of the three radars are directly flattened into a single joint action number, the action space grows exponentially with the number of radars and control dimensions, resulting in a large Q-value output, low exploration efficiency, poor sample utilization, and difficulty in training convergence.

[0004] Most existing solutions are designed specifically for a single anti-jamming technology. While these methods can achieve localized effects against specific jamming mechanisms, they neglect the coupling relationships between various anti-jamming actions. In reality, the radar's operating mode, waveform type, scan period, transmit power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe concealment all collectively affect transmitted waveform generation, jamming coupling strength, receive gain, detection threshold, false alarm rate, and track stability. Optimizing each control variable separately can easily lead to improvements in one metric but a decrease in overall mission performance. Traditional anti-jamming strategies typically optimize individual radars, focusing on their signal quality, detection probability, or jamming suppression effectiveness. However, in a distributed multi-radar system, the spatial geometry, carrier frequency conditions, observation quality, and jamming severity differ between different radars and targets / jammers. A locally optimal strategy for a single radar does not necessarily lead to a system-wide optimal effect. Without global situational awareness fusion and differentiated division of labor among radars, multiple radars may adopt repetitive, conservative, or uncoordinated control actions, failing to fully leverage the system-wide collaborative detection advantage achieved by successful detection from any single radar. While reinforcement learning offers adaptive advantages for generating radar anti-jamming strategies, directly decomposing all control variables of three radars into a single joint action number results in an action space that grows exponentially with the number of radars and the control dimension. Directly combining several actions into a flat action space leads to a massive Q-value output, low sample utilization efficiency, difficulty in training convergence, and insufficient generalization ability. Therefore, general reinforcement learning methods, without action decomposition and structured modeling, are difficult to effectively apply to the generation of multi-radar collaborative anti-jamming strategies. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a method for pushing out multi-radar node collaborative anti-interference strategies. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a method for pushing out multi-radar node collaborative anti-interference strategies, the method comprising: Acquire I / Q echoes, platform status, interference type and tracking results from multiple radars in complex interference environments, and form the agent's observation status through unified coding; The trained branched Dueling Double DQN strategy is used to generate a network that selects the corresponding branch action for each action branch of each radar based on the agent's observation state, thereby generating the branch action matrix for each radar. The branch action matrix is ​​converted into engineering control parameters using a control mapper. After legality verification, the parameters are sent to each radar to execute the complete radar signal processing link, so as to realize the anti-interference strategy push. The branched Dueling Double DQN policy generation network adopts a hierarchical structure consisting of an I / Q encoder, a global state encoder, a shared fusion layer, and a radar-specific decision head. The results of each radar performing a complete radar signal processing link are used as task feedback, and a task-oriented reward function is employed to train the branched Dueling Double DQN policy generation network based on this feedback. During training, an online network and a target network are set up. The online network selects a greedy action in the next state, and the target network evaluates the value of the greedy action to achieve the training of the branched Dueling Double DQN policy generation network.

[0006] In one embodiment of the present invention, the agent observes the state, including: Each radar's corresponding I / Q snapshot, platform status, previous action characteristics, action switching flag, jammer type code, and global track summary.

[0007] In one embodiment of the present invention, the engineering control parameters include: Operating mode, waveform type, scan period, transmit power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe masking.

[0008] In one embodiment of the present invention, a complete radar signal processing link includes: The transmitter generates a transmission pulse based on engineering control parameters; The target model is based on the transmitted pulse, and the target echo is calculated according to the position, velocity, radar cross section (RCS), and radar equations. The receiver superimposes the target echo, thermal noise, and jammer time-domain interference in the time domain, and performs matched filtering, range sampling, and Doppler FFT to form a range-Doppler power map. Based on the range-Doppler power map, anti-interference strategies are pushed out through CFAR detection and track management.

[0009] In one embodiment of the present invention, the control mapper is provided with a legality constraint that forces the sidelobe masking to be turned off when sidelobe masking is enabled but sidelobe cancellation is not enabled.

[0010] In one embodiment of the present invention, in the branched Dueling Double DQN policy generation network, The I / Q encoder uses a three-layer one-dimensional convolutional structure to encode the I / Q snapshots corresponding to each radar, thereby obtaining the I / Q features of each radar. Each radar shares the parameters of the I / Q encoder. The global state encoder uses a fully connected network to encode the platform state, the action features of the previous moment, the action switching flag, the jammer type encoding, and the global track summary to obtain the state features. The shared fusion layer concatenates the I / Q features and state features of each radar and maps them into shared global features; based on the shared global features, a dedicated decision representation is obtained. The radar-specific decision head outputs the Q value of each action branch based on the specific decision representation, and forms a joint branch action matrix based on the action corresponding to the maximum Q value in each action branch.

[0011] In one embodiment of the present invention, obtaining a specific decision representation based on shared global features includes: Each radar will share global features and stitch them together with its own I / Q features to obtain stitched features; The radar-specific fusion layer obtains a unique decision representation based on the splicing characteristics.

[0012] In one embodiment of the present invention, the task-oriented reward function is centered on the task result and consists of local detection reward, system-level detection reward, tracking continuity reward, false alarm penalty and action switching penalty.

[0013] In a second aspect, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of the multi-radar node collaborative anti-interference strategy push method provided in the first aspect of the present invention.

[0014] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the multi-radar node collaborative anti-interference strategy push method provided in the first aspect of the present invention.

[0015] The beneficial effects of this invention are: In the solution provided by this invention, each policy action is actually applied to the radar signal processing link, and the actual detection and tracking results are fed back to the branched Dueling Double DQN policy generation network, which is closer to the engineering verification requirements in complex electromagnetic countermeasures scenarios. This invention performs a structured decomposition of the multi-radar joint action space, breaking down the control actions of each radar into low-dimensional discrete branches, and achieving global perception and local execution through I / Q encoding, global fusion, and radar-specific decision heads. Compared with the solution of directly deploying three radar joint actions, this invention retains the combined control capability while avoiding the explosion of output dimensions. Furthermore, the task-oriented reward function designed in this invention is more in line with the radar mission objectives. Compared with the solution that only uses SINR or single-step detection probability, this invention simultaneously considers local detection, system-level detection, real target ranking, track confirmation, false alarm rate, and action switching cost, which can encourage the policy to achieve a balance between signal quality, continuous target tracking, and system stability. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the steps of a multi-radar node collaborative anti-interference strategy push method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the scenario, problem setting, and closed-loop modeling of a multi-radar node collaborative anti-interference strategy push method provided in an embodiment of the present invention. Figure 3 This is a flowchart of the branched Dueling Double DQN strategy generation network in a multi-radar node collaborative anti-jamming strategy push method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0018] This invention provides a method for pushing out multi-radar node collaborative anti-interference strategies, an electronic device, and a storage medium.

[0019] Below, we will first introduce a method for pushing out multi-radar node collaborative anti-interference strategies provided by an embodiment of the present invention.

[0020] like Figure 1 As shown in the figure, the multi-radar node collaborative anti-interference strategy push method provided by the embodiment of the present invention may include the following steps: S1 acquires I / Q echoes, platform status, interference type and tracking results of multiple radars in complex interference environment, and forms the intelligent agent observation status through unified coding; S2, using the trained branched Dueling Double DQN strategy to generate a network, based on the agent's observation state, selects the corresponding branch action for each action branch of each radar, generating the branch action matrix for each radar. S3 uses a control mapper to convert the branch action matrix into engineering control parameters. After legality verification, the parameters are sent to each radar to execute the complete radar signal processing link in order to realize the anti-interference strategy push. The branched Dueling Double DQN policy generation network adopts a hierarchical structure consisting of an I / Q encoder, a global state encoder, a shared fusion layer, and a radar-specific decision head. The results of each radar performing a complete radar signal processing link are used as task feedback, and a task-oriented reward function is employed to train the branched Dueling Double DQN policy generation network based on this feedback. During training, an online network and a target network are set up. The online network selects a greedy action in the next state, and the target network evaluates the value of the greedy action to achieve the training of the branched Dueling Double DQN policy generation network.

[0021] For example, in the simulation environment, this embodiment of the invention explicitly models three radars, two jammers, and one maneuvering target. The jamming targets cover two major categories: noise and deception. At the strategy level, the system decomposes the control actions of each radar into nine action branches. Through shared IQ coding, global fusion, and independent radar-specific decision heads for each radar, it achieves coordinated control with global perception and local execution. Compared with the traditional approach of discretizing the entire control system into a single, ultra-large action number, this structure retains the ability to express combined actions while significantly reducing the learning difficulty of the output space.

[0022] Optionally, the number of radars can be expanded from three to two, four or more; the number of jammers can be expanded from two to multiple; and the number of targets can be expanded from a single target to multiple targets. Under the premise of maintaining the joint observation of multiple radars, branch action output and closed-loop feedback mechanism, the purpose of the embodiments of the present invention can be achieved.

[0023] Specifically, in this embodiment of the invention, a closed-loop simulation environment is used as the reinforcement learning interaction object, a branched Dueling Double DQN is used as the policy generation network, and a control mapper is used as the interface between the policy and the radar platform to complete the collaborative anti-jamming policy push of three radars under the combined jamming conditions of two jammers.

[0024] The simulation scenario explicitly models three radars, two jammers, and one maneuvering target; the spatial positions of the three radars can be... , , The locations of the two jammers are as follows: , The target velocity range is 10-20 m / s, and the altitude range is 2000-5000 m. The RCS adopts the Swerling Type I stochastic fluctuation model. Each round is 30 steps long with a time step of 0.05 s. In each step, the control mapper converts the branch actions of the three radars output by the policy generation network into engineering control parameters and applies them to the corresponding radar platforms; the scene manager advances the target motion and jammer pointing state; for radars that have reached the scan period, the complete signal processing link is executed; for radars that have not reached the scan period, the processing results of the previous moment are reused to simulate the observation refresh delay caused by the longer scan period.

[0025] A complete radar signal processing chain can include: The transmitter generates a transmission pulse based on engineering control parameters; The target model is based on the transmitted pulse, and the target echo is calculated according to the position, velocity, radar cross section (RCS), and radar equations. The receiver superimposes the target echo, thermal noise, and jammer time-domain interference in the time domain, and performs matched filtering, range sampling, and Doppler FFT to form a range-Doppler power map. Based on the range-Doppler power map, anti-interference strategies are pushed out through CFAR detection and track management.

[0026] The transmitter generates a transmission pulse based on engineering control parameters, which may include: The transmitter generates a transmission pulse based on the operating mode, waveform type, frequency, transmission power, and CPI pulse count.

[0027] The simulation environment retains the time-domain snapshots containing interference before matched filtering, cropped or padded with zeros to form fixed-length I / Q segments as policy input. When the range-Doppler power maps of the three radars are of the same size, the system employs power-level fusion, which involves adding the power maps of each radar to form a fused power map. This fusion method is simple, stable, and interpretable, and is built around the mission objective of whether the cooperative system can detect and continuously track targets.

[0028] The embodiments of this invention construct a closed-loop simulation environment for multi-radar cooperative anti-jamming tasks. This environment explicitly models the adversarial relationship between three radars, two jammers, and one maneuvering target, so that the anti-jamming actions output by reinforcement learning can directly act on the complete link, including radar transmission, target echo, jamming superposition, matched filtering, range-Doppler processing, detection, and tracking, thereby making the policy learning results correspond to the actual radar anti-jamming effect.

[0029] Engineering control parameters may include: Operating mode, waveform type, scan period, transmit power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe masking.

[0030] No. n Radar at all times t Engineering control parameters It consists of nine discrete action branches: ; in, Indicates the working mode. Indicates the waveform type. Indicates the scan cycle. Indicates the transmission power. Indicates frequency agility. Indicates frequency diversity. Indicates narrow pulse rejection. This indicates that the sidelobe cancels out. This indicates sidelobe cloning. The value range of each branch can be... The nine action branches can form 1152 original branch action combinations, which, after legality constraint processing, correspond to 864 non-repeating legal engineering control combinations. In the policy network, each radar actually outputs 20 branch Q values ​​instead of directly outputting 864 or 1152 joint action Q values, which greatly reduces the output scale.

[0031] This invention employs a branched action space modeling method, decomposing the control actions of each radar into nine discrete action branches: operating mode, waveform type, scan period, transmit power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe concealment. Action validity constraints ensure the executable nature of the strategy output. This design avoids the dimensional explosion problem caused by directly expanding the joint action space of the three radars.

[0032] Optionally, the action branch can be equipped with control variables such as bandwidth, PRF, pulse number, CFAR threshold, beam pointing, beamwidth, and polarization mode; or some discrete actions can be converted into continuous parameters, generated by a continuous control algorithm or a hybrid action reinforcement learning algorithm.

[0033] The embodiments of the present invention do not simply treat anti-interference measures as abstract action labels, but embed them into the simulation link. Various engineering control parameters affect the final detection and tracking results through methods such as transmit waveform, carrier frequency, power, receive gain, range-Doppler image post-processing, and detection point filtering.

[0034] Regarding sidelobe cancellation and sidelobe cloning modeling, let the angular deviation of the interference source relative to the direction of the receiving main lobe be . The main lobe half-width is The lower limit of sidelobe gain is The fundamental directional gain of the radar receiving this interference source can be abstracted as: ; in, This indicates the gain of the radar receiver.

[0035] When sidelobe cancellation is enabled and the interference source is located in the sidelobe direction, the system calculates the cancellation depth based on the angular excess of the interference source relative to the main lobe edge, the number of simultaneous sidelobe interference sources, and the mismatch loss. And multiply the interference receiving gain by the residual factor. This mechanism means that the sidelobe cancellation effect is not a fixed constant, but varies with the interference angle and multiple interference sources: ; in, This represents the equivalent directional gain of the interference signal at the radar receiver after sidelobe cancellation processing.

[0036] When sidelobe concealment is enabled and the interference source is located in the sidelobe region, the system further multiplies the receiving gain of the interference source by a fixed concealment factor. .

[0037] The control mapper is configured with validity constraints. When sidelobe masking is enabled but sidelobe cancellation is not enabled, sidelobe masking will be forcibly disabled to avoid generating control combinations that do not conform to engineering constraints. ; in, This represents the equivalent gain of the interference signal at the radar receiver after sidelobe cancellation and sidelobe camouflage processing.

[0038] Narrow pulse rejection is used to handle isolated, sharp, and locally significant anomalous peaks in the range-Doppler image. The system calculates the peak significance of candidate peaks relative to the local background median on the range-Doppler power map. Local sharpness of candidate peaks Minimum distance between candidate peak and neighboring significant peak The 3dB peak width of the candidate peak, expressed as the number of distance units. Constructing anomaly scoring : .

[0039] in, This represents the threshold for determining peak significance. express The threshold for determining local sharpness. This represents the threshold value corresponding to the neighborhood isolation. express The corresponding threshold value.

[0040] When a peak value simultaneously meets the criteria of significance, sharpness, isolation, and narrow peak width, and the anomaly score is not lower than the threshold, the system replaces the anomalous units near the peak with local background statistics. After CFAR detection, the detection points are further filtered to remove narrow and isolated anomalous detection points. For dense false target environments, when the number of detection points exceeds the threshold, the system enables the local dominance preservation rule, retaining only relatively significant detection points within the distance-Doppler neighborhood.

[0041] Frequency agility is used to reduce the frequency locking capability of narrowband targeting interference and deceptive relay interference. The candidate carrier frequency set is as follows: When frequency agility is enabled, each pulse within the same CPI randomly selects a carrier frequency from the candidate carrier frequency set; the transmitted pulse is superimposed with a corresponding frequency offset phase ramp on top of the reference pulse: .

[0042] in, Indicates the first p The actual carrier frequency selected for each pulse Indicates time t No. p One actual transmitted pulse signal, Indicates time t A reference pulse without applied frequency offset. Indicates the radar's carrier frequency.

[0043] For deceptive jamming, the difference between the frequency tracked by the jammer and the current carrier frequency of the radar is used. Calculate the frequency mismatch factor. If If the interference coupling is large, then... Regular attenuation is implemented, and a minimum coupling threshold is set. For pulse-by-pulse frequency hopping within CPI, reacquisition steps are introduced to introduce a time delay in the jammer's adaptation to the new frequency. ;in, This represents the frequency matching efficiency factor between the jamming signal and the radar's current carrier frequency. This indicates that the jammer has tracked the absolute value of the frequency difference between the carrier frequency and the radar's current carrier frequency. Indicates the bandwidth scaling factor. Indicates the radar signal bandwidth. This represents the minimum lower bound of frequency matching efficiency.

[0044] In terms of waveform agility modeling, the transmitter supports both LFM and BPSK waveforms. The LFM waveform is used to provide stable pulse compression capability, and its complex envelope... It can be represented as: .

[0045] The BPSK waveform is constructed using Barker-13 phase coding, forming a matched filter template different from LFM. Forwarding jamming typically replicates or delays the acquired radar reference pulse. When the radar switches between LFM and BPSK, the forwarding template previously acquired by the jammer becomes inconsistent with the current radar's matched filter template. This inconsistency is addressed through waveform mismatch factors. To describe this effect, the configuration value is 0.12, i.e., if but ;if but .in, This indicates the type of transmission waveform currently being used by the radar. This indicates the type of radar waveform previously captured by the jammer and used for copying or forwarding. This represents the waveform matching efficiency factor. Indicates the waveform mismatch attenuation coefficient. .

[0046] Frequency diversity is used to improve the system's anti-jamming robustness by utilizing the carrier frequency differences among multiple radars. When frequency diversity is off, each radar uses the intermediate candidate carrier frequency by default; when frequency diversity is on, the intermediate candidate carrier frequency is used by default. n Each radar selects candidate carrier frequencies according to its own serial number, thus forming joint diversity of spatial nodes and frequency nodes, that is, if but ;if but .in, Indicates the first n Radar at all times t Frequency diversity switching state Indicates the first n The carrier frequency of the radar This indicates the middle or default carrier frequency in the candidate carrier frequency set. Represents the candidate carrier set, n Indicates the radar node number.

[0047] When the range-Doppler images generated by multiple radars have the same size, the system performs power-level fusion. Assume... For the first n The range-Doppler power map of the radar is then fused into a power map. It can be represented as: ; in, N This indicates the total number of radars.

[0048] The operating mode and scan cycle jointly determine the radar data update rhythm. Search mode is tied to a 32-pulse CPI, emphasizing rapid detection; tracking mode is tied to a 64-pulse CPI, emphasizing stable accumulation and continuous confirmation. The scan cycle, with values ​​of 1, 2, and 4, determines whether the current step refreshes the entire signal processing chain. Single-pulse signal-to-noise ratio (SNR) Accumulated signal-to-noise ratio with CPI It can be as follows: ; ; in, Indicates time t The transmission power, Indicates antenna gain. Indicates wavelength. Indicates the target's radar cross-section. Indicates radar distance, Represents the Boltzmann constant. Indicates standard noise temperature. Indicates the receiver sampling frequency. Indicates the receiver noise figure. Indicates the loss factor. This indicates the number of pulses accumulated within one CPI.

[0049] Single jamming types include four categories: narrowband targeting, wideband jamming, slice relay, and dense decoys. The first two categories belong to noise suppression, while the latter two belong to deception. Effective composite combinations are constructed according to combinations of the same type and type, as well as combinations of noise and deception types, resulting in a total of 12 effective jamming combinations.

[0050] Optionally, for embodiments of the present invention, the interference type can be extended to include frequency sweeping interference, comb spectrum interference, slice forwarding interference, intermittent sampling forwarding interference, range gate dragging, velocity gate dragging, etc. Correspondingly, a new time-domain or frequency-domain generation model can be added to the interference generation module, and the interference type encoding can be extended in the observation.

[0051] This invention integrates various anti-interference measures into a unified reinforcement learning closed-loop decision-making process, including frequency agility, frequency diversity, waveform agility, transmit power control, scan period control, narrow pulse elimination, sidelobe cancellation, and sidelobe concealment. This enables various anti-interference methods to no longer work in isolation, but to adaptively combine them under a unified strategy network based on the current interference situation.

[0052] The state observed by the agent can include: Each radar's corresponding I / Q snapshot, platform status, previous action characteristics, action switching flag, jammer type code, and global track summary.

[0053] Let the firstn Radar at all times t I / Q Snapshots are The platform status is The action characteristics at the previous moment were The action switching flag is The jammer type code is The global track summary is Then at time t The agent's observed state can be represented as: .

[0054] In this embodiment of the invention, the I / Q snapshots, platform status, previous action characteristics, action switching flags, jammer type codes, and global track summaries of the three radars are used as inputs to the branched Dueling Double DQN strategy generation network, enabling the network to make decisions by utilizing the jamming characteristics at the signal level and combining the multi-radar cooperative situation and historical control information.

[0055] Scenario, problem setting, and closed-loop modeling flowchart, such as Figure 2 As shown, from Figure 2 As can be seen, in the signal-level closed-loop modeling process of multi-radar collaborative anti-jamming, the simulation scenario is first initialized, and parameters such as target position, velocity, altitude, and radar cross-section are randomly generated at the beginning of each round. In each decision step, the control mapper converts the branch actions output by the strategy generation network into the corresponding engineering control parameters for each radar, and the scenario manager advances the target motion state and jammer pointing state according to the time step. Then, it is determined whether each radar has reached its set scan refresh time: if the scan refresh time has been reached, CPI transmit pulse generation, target echo parameter calculation, target echo superposition with thermal noise and jamming signals, matched filtering, range sampling, and Doppler FFT processing are executed sequentially to form a range-Doppler power map, and the I / Q snapshot before matched filtering is retained as the strategy input for the next decision step; if the scan refresh time has not been reached, the signal processing result of the radar at the previous moment is reused. Afterwards, two-dimensional CA-CFAR detection is performed on the range-Doppler power map of each radar, the detection results and sorting positions of the real targets are recorded, the track manager of each radar is updated, and a global track summary is formed. Finally, the reward for the current moment is calculated based on the results of detection, tracking, false alarms, and action switching, and the agent's observation state for the next moment is constructed, thus forming a closed loop between policy actions, signal processing, task feedback, and policy updates.

[0056] The branched Dueling Double DQN policy generation network adopts a hierarchical structure consisting of an I / Q encoder, a global state encoder, a shared fusion layer, and a radar-specific decision head. The key components are the branched Dueling head, Double DQN target, Y-step reward, and priority experience replay. The flowchart of the branched Dueling Double DQN policy generation network is as follows: Figure 3 As shown, the strategy generation network extracts radar received signal features through I / Q encoders, extracts platform, action, interference and track information through global state encoders, and then forms shared situational features through a shared fusion layer. Each radar's dedicated decision head outputs the Q value of different action branches, realizing coordinated control of global perception and local execution.

[0057] In a branched Dueling Double DQN policy generation network: The I / Q encoder uses a three-layer one-dimensional convolutional structure to encode the I / Q snapshots corresponding to each radar, thereby obtaining the I / Q features of each radar. Each radar shares the parameters of the I / Q encoder. The global state encoder uses a fully connected network to encode the platform state, the action features of the previous moment, the action switching flag, the jammer type encoding, and the global track summary to obtain the state features. The shared fusion layer concatenates the I / Q features and state features of each radar and maps them into shared global features; based on the shared global features, a dedicated decision representation is obtained. The radar-specific decision head outputs the Q value of each action branch based on the specific decision representation, and forms a joint branch action matrix based on the action corresponding to the maximum Q value in each action branch.

[0058] Dedicated decision representations derived from shared global features can include: Each radar will share global features and stitch them together with its own I / Q features to obtain stitched features; The radar-specific fusion layer obtains a unique decision representation based on the splicing characteristics.

[0059] Specifically, in the three-layer one-dimensional convolutional structure, the number of convolutional channels can be 32, 64, and 128 respectively, and the kernel size can be 7, 5, and 5 respectively, with a stride of 2 for all. After adaptive average pooling, it is mapped to 192-dimensional I / Q features through a fully connected layer. The three radars share the same I / Q encoder parameters to improve sample utilization and maintain the consistency of input representations for each radar. The global state encoder can use a two-layer fully connected network to map the 69-dimensional structured state to 128-dimensional state features. The I / Q features of the three radars are concatenated with the state features to form a 704-dimensional input, which is then mapped to 384-dimensional shared global features through a shared fusion layer. Each radar then concatenates the shared global features with its own I / Q features, and obtains a 384-dimensional exclusive decision representation through a radar-specific fusion layer. For each action branch of each radar, the network sets a Dueling branch head to model the state-value function. V and action advantage function A Let the first... n Radar No. k Each action branch in the latent vector On Q The value is: ; in, Represents the state value function. Represents the action advantage function. Indicates the first k The action currently to be evaluated in each action branch. Representation pattern The following set of actions, This represents the action index during the summation process, used to traverse the candidate action set.

[0060] This structure enables the network to evaluate the branch value of operating mode, waveform, scan period, power, and various anti-interference switches separately, avoiding the decomposition of the joint operation of the three radars into a single huge output layer, while retaining the differentiated division of labor capabilities of the three radars.

[0061] It is understandable that similar substitutions of the activation function and optimizer in the strategy generation network are equivalent substitutions of this invention.

[0062] In this embodiment of the invention, the results of each radar performing a complete radar signal processing link are used as task feedback. A task-oriented reward function is used to train a branched Dueling Double DQN strategy to generate a network based on the task feedback. The task-oriented reward function is centered on the task result and consists of local detection reward, system-level detection reward, tracking continuity reward, false alarm penalty, and action switching penalty.

[0063] Task-based reward function The expression is as follows: ; in, Represents the local detection weights. This indicates a reward for localized detection. Indicates the system-level detection weight. This indicates a system-level detection reward. Indicates the tracking weight. Indicates tracking reward, Indicates the false alarm weight. Indicates a penalty for false alarms. Indicates the weight of action switching. This indicates a penalty for switching actions.

[0064] The local detection bonus is obtained from the combined detection quality scores of all three radars. For the first... n The radar's local score is determined by whether the actual target is detected. Whether it enters the trackable candidate and real target ranking rewards Joint decision: ; in, Indicates the first n Radar time t Local score.

[0065] System-level detection reward indicates whether at least one radar detected a real target in the current step; tracking reward consists of the number of confirmed tracks, the number of best track hits, the number of consecutive confirmation steps, and confirmation flags; false alarm penalty is calculated based on the average false alarm rate; action switching penalty is calculated by the control mapper based on the control difference between two consecutive steps. The weights of each item in the task-oriented reward function can be set as follows: , , , , The specific calculation methods for false alarm penalties and action switching penalties are as follows: ; in, This represents the average false alarm rate. This represents the average switching cost.

[0066] The branched Dueling Double DQN policy generation network provided in this embodiment sets up an online network and a target network during training. To suppress the Q-value overestimation problem of traditional DQN, a target construction method of Double DQN is adopted: the online network selects a greedy action in the next state, and the target network evaluates the value of this greedy action. The specific calculation formula is as follows: ; in, For a moment The corresponding Double DQN target value, From time The cumulative discount reward for the first Y steps is calculated. Let Y be the discount factor after Y environmental steps. This is a termination flag; the state transition reaches the termination state within Y steps. ,otherwise , The action value function output by the target network. For a moment t +Y represents the agent's observed state. For online networks in observation status The action corresponding to the maximum Q value selected below. The action value function output by the online network. These are candidate actions.

[0067] Default The Y-step reward allows delayed gains to propagate back to preceding state-action pairs more quickly, thereby improving the policy's comprehensive perception of subsequent tracking continuity and action switching costs. The experience replay pool employs a priority experience replay mechanism, allocating sampling priorities based on the absolute value of the TD error and using importance sampling weights to correct for non-uniform sampling bias. Overall loss... L The weighted Huber loss mean for three radars and nine action branches: ; in, M Indicates the number of samples in a mini-batch. Indicates the number of radars. Indicates the number of action branches. Indicates the first j The weights of each sample, This represents the Huber loss function. This represents the set of trainable parameters for the policy generation network. Indicates the first j In the nth sample n Radar No. k The Q-value corresponding to each action branch Indicates the first j The observed state in each sample Indicates the first j In the nth sample n Radar No. k Each action branch is the actual action selected or recorded. Indicates the first j In the nth sample nRadar No. k The Double DQN objective value corresponding to the action branch is determined by the first action branch. j The cumulative reward of each sample in the Y-step and the Q-value estimate of the target network for subsequent states together constitute the total reward.

[0068] Training adopted - Greedy exploration strategy, with random exploration probability gradually decreasing from 1.0 to 0.05; AdamW optimizer used, with a learning rate of 1e-4; discount factor. Batch size is 256; replay pool capacity is 80,000; minimum training buffer sample size is 4,000; target network hard update interval is 1,500 steps; gradient pruning threshold is 5.0.

[0069] In the training phase of this invention, the online network, target network, and priority experience replay pool are first initialized, and the initial parameters of the online network are synchronized to the target network. At the beginning of each training round, an effective composite jamming combination is randomly selected, and the states of the radar, jammer, and target are initialized. The online network outputs the Q-values ​​of each radar and each action branch based on the current agent observation state, and selects branch actions according to a gradually decaying strategy. The control mapper converts the selected branch actions into engineering control parameters and applies them to the corresponding radar. The simulation environment sequentially executes scene state advancement, radar signal generation, jamming superposition, matched filtering, range-Doppler processing, target detection, and track update to obtain local target detection results, system-level detection results, track state, false alarm rate, and action switching cost. The task-oriented reward function calculates the immediate reward at the current moment based on the above task feedback.

[0070] The current observation state, branch action matrix, immediate reward, next-time observation state, and termination flag are used to form a state transition sample. The system continuously accumulates the reward for Y environmental steps, forming a Y-step cumulative reward, and stores the corresponding state transition sample in the priority experience replay pool. Thus, the immediate reward output by the task-oriented reward function is directly used in the calculation of the Double DQN objective value through the Y-step cumulative reward.

[0071] Once the number of samples in the experience replay pool reaches a preset threshold, a small batch of samples is extracted according to the priority corresponding to the TD error. The online network calculates the Q-value of the actual action executed in the current state and selects the greedy action for each action branch in the next state; the target network evaluates the value of the greedy action selected by the online network to obtain the Y-step Double DQN target value. The weighted Huber loss is calculated based on the error between the current Q-value and the target value, and the online network parameters are updated through backpropagation and the AdamW optimizer. Gradient pruning is used during the update process to avoid excessive gradients, and the experience replay priority of each sample is updated according to the average absolute TD error of each sample across all radars and action branches.

[0072] After a preset number of training update steps, the online network parameters are synchronized to the target network. As the number of training steps increases, the probability of random exploration gradually decreases from 1.0 to 0.05, causing the strategy to gradually shift from random exploration to utilizing learned high-value actions. This process is repeated until a preset number of training rounds are completed or the strategy performance meets the set requirements.

[0073] After training, the branching Dueling Double DQN policy generation network trained during the online inference phase no longer explores randomly. Instead, it selects the branch action with the largest Q value for each radar and each action branch, resulting in a 3×9 branch action matrix for the three radars. The control mapper converts this matrix into engineering control parameters such as operating mode, waveform, scan period, power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe concealment, and pushes them to the radar platform after validating their validity.

[0074] This invention addresses several technical challenges, including the difficulty of adapting traditional fixed strategy tables to dynamic composite interference, the inability of independent optimization of a single radar to leverage distributed synergy, the disconnect between the abstract reinforcement learning environment and the real radar signal processing link, the excessively large action space of the three-radar joint control leading to difficulties in reinforcement learning training, and the difficulty of directly mapping strategy outputs to engineering-executable control parameters. A closed-loop simulation environment consisting of three radars, two jammers, and one maneuvering target is constructed. In each decision step, the reinforcement learning strategy outputs the branch actions of the three radars. The control mapper converts these into control variables such as operating mode, waveform type, scan period, transmit power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe concealment. These control variables further influence CPI waveform generation, target echo calculation, interference coupling, receiver gain, matched filtering, range-Doppler image processing, CFAR detection, and track feedback, forming a closed-loop, trainable, and verifiable anti-jamming strategy delivery process. Using I / Q snapshots, platform status, previous action characteristics, action switching flags, jammer type codes, and global track summaries from three radars as joint observation inputs, a branched Dueling Double DQN network with shared I / Q coding, global state fusion, and a radar-specific decision head is employed to collaboratively decide on the multidimensional control variables of the three radars. A task-oriented reward function, comprised of local detection, system-level detection, tracking continuity, false alarm penalties, and action switching penalties, guides the policy generation network to achieve a comprehensive optimum in terms of detection capability, tracking stability, false alarm control, and action smoothness.

[0075] The multi-radar node collaborative anti-jamming strategy push method provided in this embodiment of the invention has the following advantages: This invention does not merely optimize anti-jamming strategies at the abstract multi-agent decision-making level, but rather embeds reinforcement learning strategies into a complete radar closed-loop simulation. Each policy action actually affects the transmission, propagation, jamming coupling, reception, matched filtering, range-Doppler processing, detection, and tracking links, and ultimately feeds back the actual detection and tracking results to the agents. Therefore, it more closely matches the engineering verification needs of complex electromagnetic countermeasures scenarios.

[0076] This anti-jamming strategy push method structurally decomposes the multi-radar joint action space, breaking down the control actions of each radar into nine low-dimensional discrete branches. It achieves global perception and local execution through shared I / Q coding, global fusion, and radar-specific heads. Compared to directly deploying three-radar joint actions, this method retains combined control capabilities while avoiding an explosion of output dimensions.

[0077] The anti-interference strategy push method retains the physical mechanism at the level of anti-interference measures. Sidelobe cancellation is modeled by angle-related residual gain, sidelobe cloning is modeled by fixed cloning factor, frequency agility is modeled by pulse-by-pulse carrier frequency selection and jammer re-acquisition delay, waveform agility is modeled by LFM / BPSK template mismatch, and narrow pulse elimination is modeled by abnormal peak structure analysis.

[0078] The mission-oriented reward function is more in line with radar mission objectives. Compared with schemes that only use SINR or single-step detection probability, the embodiments of the present invention simultaneously consider local detection, system-level detection, real target ranking, track confirmation, false alarm rate, and action switching cost, which can encourage the strategy to achieve a balance between signal quality, continuous target tracking, and system stability.

[0079] Secondly, embodiments of the present invention also provide an electronic device, such as... Figure 4 As shown, it includes a processor 001, a communication interface 002, a memory 003, and a communication bus 004, wherein the processor 001, the communication interface 002, and the memory 003 communicate with each other through the communication bus 004. The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of the method for pushing out any multi-radar node collaborative anti-interference strategy provided in the first aspect of the present invention.

[0080] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0081] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0082] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0083] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0084] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0085] Thirdly, corresponding to the multi-radar node collaborative anti-interference strategy push method provided in the first aspect, this embodiment of the invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of any multi-radar node collaborative anti-interference strategy push method provided in the first aspect of this invention.

[0086] For the electronic device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant details can be found in the description of the method embodiments.

[0087] It should be noted that the electronic device and storage medium in the embodiments of the present invention are respectively electronic devices and storage media that apply the above-mentioned multi-radar node collaborative anti-interference strategy push method. Therefore, all embodiments of the above-mentioned multi-radar node collaborative anti-interference strategy push method are applicable to the electronic device and storage medium, and can achieve the same or similar beneficial effects.

[0088] It should be noted that, in the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0089] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for pushing out multi-radar node collaborative anti-interference strategies, characterized in that, include: Acquire I / Q echoes, platform status, interference type and tracking results from multiple radars in complex interference environments, and form the agent's observation status through unified coding; The trained branched Dueling Double DQN strategy is used to generate a network that selects the corresponding branch action for each action branch of each radar based on the agent's observation state, thereby generating the branch action matrix for each radar. The branch action matrix is ​​converted into engineering control parameters using a control mapper. After legality verification, the parameters are sent to each radar to execute the complete radar signal processing link, so as to realize the anti-interference strategy push. The branched Dueling Double DQN policy generation network adopts a hierarchical structure consisting of an I / Q encoder, a global state encoder, a shared fusion layer, and a radar-specific decision head. The results of each radar performing a complete radar signal processing link are used as task feedback, and a task-oriented reward function is employed to train the branched Dueling Double DQN policy generation network based on this feedback. During training, an online network and a target network are set up. The online network selects a greedy action in the next state, and the target network evaluates the value of the greedy action to achieve the training of the branched Dueling Double DQN policy generation network.

2. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 1, characterized in that, The agent's observed state includes: Each radar's corresponding I / Q snapshot, platform status, previous action characteristics, action switching flag, jammer type code, and global track summary.

3. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 1, characterized in that, The engineering control parameters include: Operating mode, waveform type, scan period, transmit power, frequency agility, frequency diversity, narrow pulse rejection, sidelobe cancellation, and sidelobe masking.

4. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 1, characterized in that, The complete radar signal processing link includes: The transmitter generates a transmission pulse based on engineering control parameters; The target model is based on the transmitted pulse, and the target echo is calculated according to the position, velocity, radar cross section (RCS), and radar equations. The receiver superimposes the target echo, thermal noise, and jammer time-domain interference in the time domain, and performs matched filtering, range sampling, and Doppler FFT to form a range-Doppler power map. Based on the range-Doppler power map, anti-interference strategies are pushed out through CFAR detection and track management.

5. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 3, characterized in that, The control mapper is equipped with a legality constraint. When sidelobe masking is enabled but sidelobe cancellation is not enabled, sidelobe masking will be forcibly disabled.

6. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 2, characterized in that, In the branched Dueling Double DQN policy generation network, The I / Q encoder uses a three-layer one-dimensional convolutional structure to encode the I / Q snapshots corresponding to each radar, thereby obtaining the I / Q features of each radar. Each radar shares the parameters of the I / Q encoder. The global state encoder uses a fully connected network to encode the platform state, the action features of the previous moment, the action switching flag, the jammer type encoding, and the global track summary to obtain the state features. The shared fusion layer concatenates the I / Q features and state features of each radar and maps them into shared global features; based on the shared global features, a dedicated decision representation is obtained. The radar-specific decision head outputs the Q value of each action branch based on the specific decision representation, and forms a joint branch action matrix based on the action corresponding to the maximum Q value in each action branch.

7. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 6, characterized in that, Based on shared global features, we obtain specific decision representations, including: Each radar will share global features and stitch them together with its own I / Q features to obtain stitched features; The radar-specific fusion layer obtains a unique decision representation based on the splicing characteristics.

8. The method for pushing out a multi-radar node collaborative anti-interference strategy according to claim 1, characterized in that, The task-based reward function is centered on the task result and consists of local detection rewards, system-level detection rewards, tracking continuity rewards, false alarm penalties, and action switching penalties.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of the multi-radar node collaborative anti-interference strategy push method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-radar node collaborative anti-interference strategy push method according to any one of claims 1-8.

Images