An electromagnetic spectrum game intelligent training device and method

By constructing an intelligent training device for electromagnetic spectrum games, the problem of insufficient data support in electromagnetic spectrum games was solved, and the intelligent model was effectively applied in real-time games with complex and unknown objectives, thus realizing intelligent growth.

CN118316547BActive Publication Date: 2024-11-22THE QUARTERMASTER RES INST OF THE GENERAL LOGISTICS DEPT OF THE CPLA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410595618.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-14
Publication Date
2024-11-22
Estimated Expiration
2044-05-14

AI Technical Summary

Technical Problem

The current electromagnetic spectrum game training process lacks high-quality, large-sample, multimodal data and adversarial target support, making it difficult to realize the effective application of intelligent algorithms in real-time games with complex and unknown targets.

Method used

An intelligent training device for electromagnetic spectrum game is constructed, comprising an infrastructure layer and a hardware resource layer. Through a spectrum adversarial game control subsystem, a data and resource management subsystem, a programmable target simulation subsystem, a software-defined interference side simulation subsystem, a radio frequency channel simulation subsystem, and a complex environment simulation subsystem, signal generation, channel simulation, and environment simulation are achieved, enabling intelligent training and evaluation.

Benefits of technology

This study realizes the effective application of intelligent models in electromagnetic spectrum games. Through real-time interaction and algorithm control between the target and interference sides, it achieves the goal of intelligent growth and solves the problem of insufficient data support for intelligent algorithm research and model training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118316547B_ABST
    Figure CN118316547B_ABST
Patent Text Reader

Abstract

The present invention discloses an electromagnetic spectrum game intelligent training device and method, the device comprising: an infrastructure layer and a hardware resource layer; the infrastructure layer is used to control the electromagnetic spectrum game intelligent training process and provide data resources and computing resources; the hardware resource layer is used to realize signal generation, channel simulation and environment simulation in the electromagnetic spectrum game process; the infrastructure layer includes a spectrum confrontation game control subsystem and a data and resource management subsystem; the spectrum confrontation game control subsystem is used to evaluate the spectrum confrontation game training process and obtain confrontation effect evaluation information and training effect evaluation information. The present invention solves the problems existing in the current electromagnetic spectrum game training process, such as real-time game facing complex unknown targets, lack of high-quality, large sample, multi-modal data and confrontation target support, and promotes the effective application of intelligent models in electromagnetic spectrum game intelligent training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of electromagnetic space security, and in particular to an electromagnetic spectrum game intelligent training device and method. Background Art

[0002] At present, in the existing spectrum game process, the interference party often relies on expert experience and prior knowledge to conduct unilateral offline model training under limited data conditions. The resulting recognition and confrontation algorithm models are difficult to be directly used in different scenarios, different tasks, and different goals. It is even more difficult to achieve self-evolution and autonomous growth of the model under the closed-loop game confrontation conditions of "intelligent" versus "intelligent", and intelligent versus anti-intelligent. The current electromagnetic spectrum game training process has the following problems:

[0003] On the one hand, in the face of real-time games with complex unknown targets, intelligent algorithm research and model training lack high-quality, large sample, multimodal data and adversarial target support. Current artificial intelligence research needs to rely on a large amount of sample data for learning, but there is a lack of multimodal and large sample data support in the field of actual spectrum games. The actual spectrum data is difficult to obtain, the sample size is small, and the diversity is insufficient. It is difficult to use the unilateral offline model training directly, and it is even more difficult to achieve the self-evolution of the model. In addition, the electromagnetic space is changing rapidly, and the large amount of collected data has problems such as one-sided incompleteness and little high-quality data. In the actual spectrum game process, data collection is difficult and reuse is difficult; in the actual field test process, due to personnel, venue, and funding restrictions, data collection is only for the current specific scene, which is difficult to fully reflect the actual situation, and the data volume is small, the cost is high, and the accumulation cycle is long; in the field of spectrum games, spectrum simulation is mainly based on functional feature-level simulation. Although the data generation speed is fast and the scale is large, the data fidelity is low and it is quite different from the actual signal-level data.

[0004] On the other hand, spectrum perception, cognition and interference algorithms are diverse, but current research on electromagnetic spectrum game intelligence lacks support for adversarial targets and environments. The functions and performance indicators of intelligent algorithms are not well known to users or even designers. How to build an effective electromagnetic spectrum game intelligent training device and method based on intelligent models is an urgent problem to be solved. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an electromagnetic spectrum game intelligent training device and method to solve the problems existing in the current electromagnetic spectrum game training process, such as real-time games facing complex unknown targets, lack of high-quality, large samples, multimodal data and support for adversarial targets in intelligent algorithm research and model training, and to promote the effective application of intelligent models in electromagnetic spectrum game intelligent training.

[0006] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses an electromagnetic spectrum game intelligent training device, including: an infrastructure layer and a hardware resource layer;

[0007] The infrastructure layer is connected to the hardware resource layer and is used to control the electromagnetic spectrum game intelligent training process and provide data resources and computing resources;

[0008] The hardware resource layer is used to realize signal generation, channel simulation and environment simulation in the electromagnetic spectrum game process;

[0009] The infrastructure layer includes a spectrum confrontation game control subsystem and a data and resource management subsystem;

[0010] The spectrum confrontation game control subsystem is connected to the data and resource management subsystem, and is used to generate and display the simulation scene of the spectrum confrontation game, control the spectrum confrontation game training process; evaluate the spectrum confrontation game training process, and obtain confrontation effect evaluation information and training effect evaluation information;

[0011] The data and resource management subsystem is used to collect and store data generated by each subsystem during the intelligent training of electromagnetic spectrum game, and to perform resource management on the hardware resource layer; and to store offline data sets used to train interference decision-making evaluation models.

[0012] The hardware resource layer includes a programmable target simulation subsystem, a software-defined interference side simulation subsystem, a radio frequency channel simulation subsystem, and a complex environment simulation subsystem;

[0013] The programmable target simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem to generate a target signal; receive the channel target signal and the channel interference signal sent by the radio frequency channel simulation subsystem, process the channel target signal and the channel interference signal to obtain a result target signal; and send the result target signal to the spectrum confrontation game control subsystem;

[0014] The software-defined interference side simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to receive the channel target signal sent by the radio frequency channel simulation subsystem, perform signal feature extraction processing on the channel target signal to obtain signal feature information; perform interference decision processing on the signal feature information to obtain a corresponding interference signal, and send the interference signal to the radio frequency channel simulation subsystem;

[0015] The radio frequency channel simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to simulate the radio frequency channel, attenuate the input signal and superimpose the environmental signal processing;

[0016] The complex environment simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to generate an environmental signal based on the radiation signal of the background target information, and send the environmental signal to the radio frequency channel simulation subsystem.

[0017] The software-defined interference side simulation subsystem includes a signal reconnaissance module, an interference decision module, and an interference signal generation module;

[0018] The signal reconnaissance module is used to extract signal characteristics of the received signal to obtain signal characteristic information; the signal characteristic information includes carrier frequency, bandwidth, symbol rate, amplitude, and modulation type;

[0019] The interference decision module is implemented using a trained interference decision evaluation model;

[0020] The interference signal generating module is used to receive the interference signal parameters obtained by the interference decision module; generate a corresponding interference signal according to the interference signal parameters; the interference signal parameters include the interference signal center frequency, bandwidth, duty cycle, and modulation mode.

[0021] The interference decision evaluation model includes an input module, an evaluation module, and a decision network; the evaluation module includes a behavior evaluation network and a target evaluation network;

[0022] The input module receives data using the input terminals and outputs the data using the output terminals;

[0023] The input end of the decision network is connected to the output end of the input module;

[0024] The input end of the behavior evaluation network is connected to the output end of the input module;

[0025] The output end of the behavior evaluation network is connected to the input end of the target evaluation network;

[0026] The input end of the decision network is connected to the output end of the target evaluation network;

[0027] The input end of the behavior evaluation network is connected to the output end of the decision network;

[0028] The decision network comprises: a first input layer, a first hidden layer, a first upsampling layer, a first convolutional layer, a second hidden layer, a third hidden layer and a first output layer;

[0029] The behavior evaluation network comprises: a second input layer, a fourth hidden layer, a second upsampling layer, a fifth hidden layer, a second convolutional layer, a first residual attention layer, and a second output layer;

[0030] The target evaluation network comprises: a third input layer, a sixth hidden layer, a seventh hidden layer, an eighth hidden layer and a third output layer;

[0031] The first to eighth hidden layers are all implemented using a fully connected network;

[0032] The behavior evaluation network is used to process the input signal feature information and interference signal parameters to obtain a first estimated value; and update its parameters using the parameter update value;

[0033] The target evaluation network processes the signal characteristic information and interference signal parameters to obtain an interference effect value; and updates its parameters using parameter update values;

[0034] The decision network is used to process the signal characteristic information and the interference effect value to obtain interference signal parameters; and update the parameters using parameter update values.

[0035] The training process of the interference decision evaluation model includes:

[0036] S1, randomly initialize the parameters of the evaluation module and decision network of the interference decision evaluation model; initialize the training number value to 1; obtain an offline data set; the offline data set includes offline data; each offline data includes signal feature information, interference signal parameters and interference effect value; the interference effect value is obtained through adversarial effect evaluation information;

[0037] S2, randomly sampling the offline data set to obtain a sampled data set; inputting the signal feature information in the sampled data set into the radar interference decision evaluation model;

[0038] S3, using the interference decision evaluation model to process the input data to obtain interference signal parameters and interference effect values ​​corresponding to the signal feature information;

[0039] S4, using a first parameter updating model to update parameters of the behavior evaluation network and the target evaluation network;

[0040] S5, updating the parameters of the decision network using a second parameter updating model;

[0041] S6, increase the number of training times by 1, and determine whether the number of training times is greater than the set training threshold; if not, determine whether the sum of all interference effect values ​​obtained is greater than the sum of all interference effect values ​​in the sampling data set; if it is greater than the sum of all interference effect values ​​in the sampling data set, complete the training process of the interference decision evaluation model; if it is not greater than the sum of all interference effect values ​​in the sampling data set, execute S2; if it is greater than the training threshold, complete the training process of the interference decision evaluation model.

[0042] The calculation expression of the first parameter updating model is:

[0043]

[0044] θ←0+v;

[0045] In the formula, when the behavior evaluation network is updated, x (i) is the interference signal parameter of the i-th data in the sampled data set. When the target evaluation network is updated, x (i) is the interference effect value of the i-th data in the sample data set, is the loss function, v is the parameter update value, θ is the parameter of the behavior evaluation network or the target evaluation network, η is the initial parameter learning rate, α is the first adjustment parameter, 0≤α≤1, It means to find the partial derivative of variable θ, f(x (i) ; θ) represents the first estimated value calculated by the behavior evaluation network for the i-th data in the sampled data set, or the interference effect value calculated by the target evaluation network for the i-th data in the sampled data set, f(·) is the calculation function corresponding to the behavior evaluation network or the target evaluation network; exp represents the power operation of the constant e; η and α are preset values;

[0046] The second parameter updates the model, and its calculation expression is:

[0047]

[0048] Where, ψ represents the parameters of the decision network, l3 represents the update step size of the decision network, represents the gradient of the ψ parameter in L3(ψ), L3(ψ) represents the loss function of the decision network, represents the parameter update value of the decision network, Indicates the use of The parameter ψ of the decision network is updated.

[0049] In a second aspect of the embodiment of the present invention, an electromagnetic spectrum game intelligent training method is disclosed, which is applied to the electromagnetic spectrum game intelligent training device, comprising:

[0050] Acquire scenario planning information and scenario assumption information; the scenario planning information includes the initial position of the target device, the initial position of the interference device, the movement information of the target device, and the movement information of the interference device;

[0051] The scenario information includes the geographical environment information and background target information of the electromagnetic spectrum game intelligent training; the background target information includes the movement trajectory and radiation signal of the background target;

[0052] Using the spectrum confrontation game control subsystem, the scenario planning information is loaded into the scenario assumption information to obtain simulation scenario information;

[0053] Using a complex environment simulation subsystem, based on the radiation signal of the background target information, an environment signal is generated, and the environment signal is sent to the radio frequency channel simulation subsystem;

[0054] Generate a target signal by using a programmable target simulation subsystem, send the target signal to a radio frequency channel simulation subsystem, receive a channel target signal and a channel interference signal sent by the radio frequency channel simulation subsystem, process the channel target signal and the channel interference signal to obtain a result target signal; send the result target signal to the spectrum confrontation game control subsystem;

[0055] Utilizing the software-defined interference side simulation subsystem, receiving a channel target signal sent by the radio frequency channel simulation subsystem, performing signal feature extraction processing on the channel target signal to obtain signal feature information; performing interference decision processing on the signal feature information to obtain a corresponding interference signal, and sending the interference signal to the radio frequency channel simulation subsystem;

[0056] Using the radio frequency channel simulation subsystem, the target signal is attenuated and superimposed with the environmental signal to obtain a channel target signal; the interference signal is attenuated and superimposed with the environmental signal to obtain a channel interference signal; the channel target signal is sent to the programmable target simulation subsystem and the software-defined interference side simulation subsystem; the channel interference signal is sent to the programmable target simulation subsystem;

[0057] Utilizing the spectrum confrontation game control subsystem, evaluating and processing the result target signal to obtain confrontation effect evaluation information, and sending the confrontation effect evaluation information to the offline database and the software-defined interference side simulation subsystem; performing training evaluation processing on the confrontation effect evaluation information to obtain training effect evaluation information, and sending the training effect evaluation information to the offline database;

[0058] The data and resource management subsystem is used to collect and store data generated by each subsystem during the intelligent training of electromagnetic spectrum game.

[0059] The step of evaluating the result target signal to obtain the confrontation effect evaluation information includes:

[0060] The difference value between the result target signal and the corresponding target signal is calculated; and the confrontation effect evaluation information is determined according to the inverse of the difference value.

[0061] The training evaluation process is performed on the confrontation effect evaluation information to obtain the training effect evaluation information, including:

[0062] Using the confrontation effect evaluation information under a number of scenario planning information and scenario assumption information, a confrontation effect evaluation information set is constructed;

[0063] Using the confrontation effect evaluation information set, a confrontation effect evaluation matrix is ​​constructed; the elements x in the i-th row and the j-th row of the confrontation effect evaluation matrix ij , is the confrontation effect evaluation information under the i-th scenario planning information and the j-th scenario assumption information;

[0064] Performing a first standard calculation process on the confrontation effect evaluation matrix to obtain a first standard matrix;

[0065] The expression of the first standard calculation process is:

[0066]

[0067] Where m represents the number of scenario planning information, x ij represents the element of the ith row and jth column of the confrontation effect evaluation matrix, z ij represents the element of the i-th row and j-th column of the first standard matrix;

[0068] Performing column optimization processing on the first standard matrix to obtain an optimal solution vector;

[0069] The column optimal processing is to take out the largest number in each column to form the optimal solution vector z + ; The expression of the optimal solution vector is:

[0070]

[0071] Where n is the number of scenario information;

[0072] Performing column worst processing on the first standard matrix to obtain a worst solution vector;

[0073] The worst column processing is to take out the smallest number in each column to form the worst solution vector z - ; The expression of the worst solution vector is:

[0074]

[0075] The worst solution vector and the best solution vector are scored and calculated to obtain training effect evaluation information; the score calculation expression is:

[0076]

[0077] In the formula, s is the training effect evaluation information, ω j Assume the importance weight of the information for the jth scenario; ω j , obtained by presetting, or by calculating the variance value of each column of the first standard matrix.

[0078] The beneficial effects of the present invention are:

[0079] Based on the intelligent model, the present invention constructs an effective electromagnetic spectrum game intelligent training device and method. The present invention generates signal-level data by constructing target-side and interference-side environmental simulators, connects the target side and the interference side for real-time interaction, and uses an algorithm to control the interference-side signal parameters to form different degrees of interference effects on the electromagnetic target. The target side gives timely feedback on the interference effect according to its own interference state, thereby realizing an algorithm training closed loop and achieving the purpose of intelligent growth.

[0080] The present invention solves the problems existing in the current electromagnetic spectrum game training process, namely, real-time game for complex unknown targets, lack of high-quality, large sample, multimodal data and support for adversarial targets in intelligent algorithm research and model training, and promotes the effective application of intelligent models in intelligent training of electromagnetic spectrum games. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a composition framework diagram of the device of the present invention;

[0082] Figure 2 Schematic diagram of information interaction of the device of the present invention. DETAILED DESCRIPTION

[0083] In order to better understand the content of the present invention, an embodiment is given here.

[0084] Figure 1 It is a composition framework diagram of the device of the present invention; Figure 2 Schematic diagram of information interaction of the device of the present invention.

[0085] In order to promote the effectiveness of intelligent models in this field, it is necessary to first conduct scientific tests on intelligent algorithms and find out their performance limits, so as to clearly define the technical level and capabilities of intelligent algorithms. It is necessary to build a real-time dynamic, software-definable advanced system electromagnetic target simulation platform to provide an agile and unknown closed-loop confrontation research environment to support the training and test verification of algorithm models.

[0086] In a first aspect of an embodiment of the present invention, an electromagnetic spectrum game intelligent training device is disclosed, comprising: an infrastructure layer and a hardware resource layer;

[0087] The infrastructure layer is connected to the hardware resource layer and is used to control the electromagnetic spectrum game intelligent training process and provide data resources and computing resources;

[0088] The hardware resource layer is used to realize signal generation, channel simulation and environment simulation in the electromagnetic spectrum game process.

[0089] The infrastructure layer includes a spectrum confrontation game control subsystem and a data and resource management subsystem;

[0090] The spectrum confrontation game control subsystem is connected to the data and resource management subsystem, and is used to generate and display the simulation scene of the spectrum confrontation game, control the spectrum confrontation game training process; evaluate the spectrum confrontation game training process, and obtain confrontation effect evaluation information and training effect evaluation information;

[0091] The data and resource management subsystem is used to collect and store data generated by each subsystem during the intelligent training of electromagnetic spectrum game, and to manage the hardware resource layer; store offline data sets used to train the interference decision-making evaluation model;

[0092] Based on the infrastructure layer and the hardware resource layer, this device can realize various system services, data resources, system applications and standard specifications; system services include target / interference / environmental signal generation, target reception, target signal reconnaissance, target information processing, scene information processing, channel parameter calculation, interference decision-making and display control, etc.; data resources mainly include target data, channel data, environmental data, interference side data; system applications include third-party algorithm training, human-computer game, machine-machine game, challenge competition, algorithm push and business support, etc.; standard specifications include design specifications, integration specifications, data specifications, algorithm specifications and user specifications, etc.

[0093] The resource management refers to collecting and monitoring the working status values ​​of each subsystem of the hardware resource layer; when the working status value is abnormal, an early warning message is generated;

[0094] The hardware resource layer includes a programmable target simulation subsystem, a software-defined interference side simulation subsystem, a radio frequency channel simulation subsystem, and a complex environment simulation subsystem;

[0095] The programmable target simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem to generate a target signal; receive the channel target signal and the channel interference signal sent by the radio frequency channel simulation subsystem, process the channel target signal and the channel interference signal to obtain a result target signal; and send the result target signal to the spectrum confrontation game control subsystem;

[0096] The software-defined interference side simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to use the software-defined interference side simulation subsystem to receive the channel target signal sent by the radio frequency channel simulation subsystem, perform signal feature extraction processing on the channel target signal to obtain signal feature information; perform interference decision processing on the signal feature information to obtain a corresponding interference signal, and send the interference signal to the radio frequency channel simulation subsystem;

[0097] The radio frequency channel simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to simulate the radio frequency channel, attenuate the input signal and superimpose the environmental signal processing;

[0098] The complex environment simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to generate an environmental signal based on the radiation signal of the background target information, and send the environmental signal to the radio frequency channel simulation subsystem.

[0099] The software-defined interference side simulation subsystem includes a signal reconnaissance module, an interference decision module, and an interference signal generation module;

[0100] The signal reconnaissance module is used to extract signal characteristics of the received signal to obtain signal characteristic information; the signal characteristic information includes carrier frequency, bandwidth, symbol rate, amplitude, and modulation type;

[0101] The signal reconnaissance module is implemented by a digital radio frequency receiver; the carrier frequency extraction and processing is implemented by the first-order difference method or the FFT method; the bandwidth extraction and processing is implemented by measuring the 3dB bandwidth; the amplitude is implemented by measuring the signal power and then taking the square root; the code element rate is extracted by the delayed multiplication method; the modulation type is identified by a communication signal modulation identification method based on statistical moments.

[0102] The interference decision module is implemented using a trained interference decision evaluation model;

[0103] The interference signal generating module is used to receive the interference signal parameters obtained by the interference decision module; generate a corresponding interference signal according to the interference signal parameters; the interference signal parameters include the interference signal center frequency, bandwidth, duty cycle, and modulation mode;

[0104] The interference decision evaluation model includes an input module, an evaluation module, and a decision network; the evaluation module includes a behavior evaluation network and a target evaluation network;

[0105] The input module receives data using the input port and outputs the data using the output ports;

[0106] The input end of the decision network is connected to the output end of the input module;

[0107] The input end of the behavior evaluation network is connected to the output end of the input module;

[0108] The output end of the behavior evaluation network is connected to the input end of the target evaluation network;

[0109] The input end of the decision network is connected to the output end of the target evaluation network;

[0110] The input end of the behavior evaluation network is connected to the output end of the decision network;

[0111] The decision network comprises: a first input layer, a first hidden layer, a first upsampling layer, a first convolutional layer, a second hidden layer, a third hidden layer and a first output layer;

[0112] The behavior evaluation network comprises: a second input layer, a fourth hidden layer, a second upsampling layer, a fifth hidden layer, a second convolutional layer, a first residual attention layer, and a second output layer;

[0113] The target evaluation network includes: a third input layer, a sixth hidden layer, a seventh hidden layer, an eighth hidden layer and a third output layer.

[0114] The network structures of the target evaluation network and the behavior evaluation network are exactly the same.

[0115] The hidden layers are all implemented using a fully connected network.

[0116] ReLU is used as the activation function between each hidden layer to realize parameter transfer.

[0117] The residual attention network is composed of multiple layers of attention modules stacked together, and pre-activated residual units, ResNet and Inception are used as basic units of the residual attention network.

[0118] The behavior evaluation network is used to process the input signal feature information and interference signal parameters to obtain a first estimated value; and update its parameters using the parameter update value;

[0119] The target evaluation network processes the signal characteristic information and interference signal parameters to obtain an interference effect value; and updates its parameters using parameter update values;

[0120] The decision network is used to process the signal characteristic information, interference signal parameters, and interference effect values ​​to obtain interference signal parameters; and update the parameters using parameter update values;

[0121] The processing method of the behavior evaluation network, the target evaluation network, and the decision network may be a neural network forward propagation algorithm;

[0122] The training process of the interference decision evaluation model includes:

[0123] S1, randomly initialize the parameters of the evaluation module and decision network of the interference decision evaluation model; initialize the training number value to 1; obtain an offline data set; the offline data set includes offline data; each offline data includes signal feature information, interference signal parameters and interference effect value; the interference effect value is obtained through adversarial effect evaluation information;

[0124] S2, randomly sampling the offline data set to obtain a sampled data set; inputting the signal feature information in the sampled data set into the radar interference decision-making evaluation model;

[0125] S3, using the interference decision evaluation model to process the input data to obtain interference signal parameters and interference effect values ​​corresponding to the signal feature information;

[0126] S4, using a first parameter updating model to update parameters of the behavior evaluation network and the target evaluation network;

[0127] S5, updating the parameters of the decision network using a second parameter updating model;

[0128] S6, increase the number of training times by 1, and determine whether the number of training times is greater than the set training threshold; if not, determine whether the sum of all interference effect values ​​obtained is greater than the sum of all interference effect values ​​in the sampling data set; if it is greater than the sum of all interference effect values ​​in the sampling data set, complete the training process of the interference decision evaluation model; if it is not greater than the sum of all interference effect values ​​in the sampling data set, execute S2; if it is greater than the training threshold, complete the training process of the interference decision evaluation model.

[0129] The calculation expression of the first parameter updating model is:

[0130]

[0131] θ←θ+v;

[0132] In the formula, when the behavior evaluation network is updated, x (i) is the interference signal parameter of the i-th data in the sampled data set. When the target evaluation network is updated, x (i) is the interference effect value of the i-th data in the sample data set, is the loss function, v is the parameter update value, θ is the parameter of the behavior evaluation network or the target evaluation network, η is the initial parameter learning rate, α is the first adjustment parameter, 0≤α≤1, It means to find the partial derivative of variable θ, f(x (i) ; θ) represents the first estimated value calculated by the behavior evaluation network for the i-th data in the sampled data set, or the interference effect value calculated by the target evaluation network for the i-th data in the sampled data set, f(·) is the calculation function corresponding to the behavior evaluation network or the target evaluation network; exp represents the power operation of the constant e; η and α are preset values.

[0133] The second parameter updates the model, and its calculation expression is:

[0134]

[0135] Where, ψ represents the parameters of the decision network, l3 represents the update step size of the decision network, represents the gradient of the ψ parameter in L3(ψ), L3(ψ) represents the loss function of the decision network, represents the parameter update value of the decision network, Indicates the use of Updating the parameter ψ of the decision network;

[0136] The loss function of the decision network is obtained by calculating the difference between the interference effect value calculated by the target evaluation network on the data in the sampled data set and the interference effect value of the data in the sampled data set.

[0137] In a second aspect of an embodiment of the present invention, an electromagnetic spectrum game intelligent training method is disclosed, comprising:

[0138] Obtaining scenario planning information and scenario assumption information; the scenario planning information includes the initial position of the target device, the initial position of the interference device, the motion information of the target device, and the motion information of the interference device; the motion information includes the speed and direction;

[0139] The scenario information includes the geographical environment information and background target information of the electromagnetic spectrum game intelligent training; the background target information includes the movement trajectory and radiation signal of the background target;

[0140] Using the spectrum confrontation game control subsystem, the scenario planning information is loaded into the scenario assumption information to obtain simulation scenario information;

[0141] In the simulation scene information, the target device position and the interference device position move from the target device initial position and the interference device initial position according to the target device motion information and the interference device motion information, respectively, and are updated according to time, and the updated position information is included in the simulation scene information; the spectrum confrontation game control subsystem displays the simulation scene information in real time;

[0142] Using a complex environment simulation subsystem, based on the radiation signal of the background target information, an environment signal is generated, and the environment signal is sent to the radio frequency channel simulation subsystem;

[0143] Using the radio frequency channel simulation subsystem, the target signal is attenuated and superimposed with the environmental signal to obtain a channel target signal; the interference signal is attenuated and superimposed with the environmental signal to obtain a channel interference signal; the channel target signal is sent to the programmable target simulation subsystem and the software-defined interference side simulation subsystem; the channel interference signal is sent to the programmable target simulation subsystem;

[0144] Generate a target signal by using a programmable target simulation subsystem, send the target signal to a radio frequency channel simulation subsystem, receive a channel target signal and a channel interference signal sent by the radio frequency channel simulation subsystem, process the channel target signal and the channel interference signal to obtain a result target signal; send the result target signal to the spectrum confrontation game control subsystem;

[0145] Utilizing the software-defined interference side simulation subsystem, receiving a channel target signal sent by the radio frequency channel simulation subsystem, performing signal feature extraction processing on the channel target signal to obtain signal feature information; performing interference decision processing on the signal feature information to obtain a corresponding interference signal, and sending the interference signal to the radio frequency channel simulation subsystem;

[0146] Utilizing the spectrum confrontation game control subsystem, evaluating and processing the result target signal to obtain confrontation effect evaluation information, and sending the confrontation effect evaluation information to the offline database and the software-defined interference side simulation subsystem; performing training evaluation processing on the confrontation effect evaluation information to obtain training effect evaluation information, and sending the training effect evaluation information to the offline database;

[0147] The software-defined interference side simulation subsystem utilizes the spectrum confrontation game evaluation information to perform interference decision optimization processing.

[0148] The data and resource management subsystem is used to collect and store data generated by each subsystem during the intelligent training of electromagnetic spectrum game.

[0149] Before each training session, the interference decision evaluation model is trained;

[0150] After each training session, the offline database is updated using adversarial effect evaluation information, interference signals, and target signals.

[0151] The step of evaluating the result target signal to obtain the confrontation effect evaluation information includes:

[0152] The difference value between the result target signal and the corresponding target signal is calculated; and the confrontation effect evaluation information is determined according to the inverse of the difference value.

[0153] Specifically, the difference value between the result target signal and the corresponding target signal is obtained by calculating the Euclidean distance between the result target signal and the corresponding target signal.

[0154] The countermeasure effect evaluation information is obtained by multiplying the reciprocal of the difference value by a preset constant value.

[0155] The training evaluation process is performed on the confrontation effect evaluation information to obtain the training effect evaluation information, including:

[0156] Using the confrontation effect evaluation information under a number of scenario planning information and scenario assumption information, a confrontation effect evaluation information set is constructed;

[0157] Using the confrontation effect evaluation information set, a confrontation effect evaluation matrix is ​​constructed; the elements x in the i-th row and the j-th row of the confrontation effect evaluation matrixij , is the confrontation effect evaluation information under the i-th scenario planning information and the j-th scenario assumption information;

[0158] Performing a first standard calculation process on the confrontation effect evaluation matrix to obtain a first standard matrix;

[0159] The expression of the first standard calculation process is:

[0160]

[0161] Where m represents the number of scenario planning information, x ij represents the element of the ith row and jth column of the confrontation effect evaluation matrix, z ij represents the element of the i-th row and j-th column of the first standard matrix;

[0162] Performing column optimization processing on the first standard matrix to obtain an optimal solution vector;

[0163] The column optimal processing is to take out the largest number in each column to form the optimal solution vector z + ; The expression of the optimal solution vector is:

[0164]

[0165] Where n is the number of scenario information;

[0166] Performing column worst processing on the first standard matrix to obtain a worst solution vector;

[0167] The worst column processing is to take out the smallest number in each column to form the worst solution vector z - ; The expression of the worst solution vector is:

[0168]

[0169] The worst solution vector and the best solution vector are scored and calculated to obtain training effect evaluation information; the score calculation expression is:

[0170]

[0171] In the formula, ω j Assume the importance weight of the information for the jth scenario; ω j , obtained by presetting, or by calculating the variance value of each column of the first standard matrix.

[0172] The training evaluation process is performed on the confrontation effect evaluation information to obtain the training effect evaluation information, including:

[0173] Using the confrontation effect evaluation information under a number of scenario planning information and scenario assumption information, a confrontation effect evaluation information set is constructed;

[0174] Using the confrontation effect evaluation information set, a confrontation effect evaluation matrix is ​​constructed; the elements x in the i-th row and the j-th row of the confrontation effect evaluation matrix ij , is the confrontation effect evaluation information under the i-th scenario planning information and the j-th scenario assumption information;

[0175] The confrontation effect evaluation matrix is ​​used to calculate the correlation coefficient set R; the calculation expression of the correlation coefficient set R is:

[0176]

[0177] Among them, r ij is the i-th row vector x i and the j-th row vector x j The correlation coefficient of , p is the number of rows and columns of the confrontation effect evaluation matrix;

[0178] The correlation coefficient set R is processed by eigenvalue calculation to obtain the eigenvalue sequence λ1,λ2,λ p ,λ1≥λ2≥λ p ≥0;

[0179] Perform cumulative calculation on the eigenvalue sequence to obtain a weighted value;

[0180] The cumulative calculation process is calculated as follows:

[0181]

[0182] Among them, y m is the weighted value of the mth time series;

[0183] averaging each row vector of the confrontation effect evaluation matrix to obtain an average value of each row vector;

[0184] The weighted values ​​are used to perform weighted summation processing on the corresponding row vector average values ​​to obtain training effect evaluation information.

[0185] The system proposed by the present invention includes a spectrum game control subsystem (programmable target simulation, radio frequency channel simulation, complex environment simulation subsystem), a spectrum game confrontation target and environment simulation subsystem, a software-defined electronic warfare simulation subsystem (software-defined interference side simulation subsystem), and a data and resource management subsystem. The information exchange relationship between each subsystem is as follows: Figure 2 shown.

[0186] In the spectrum confrontation game control subsystem, the game engine driver module exchanges information with the target side and the interference side through the RF channel and the digital channel respectively. Based on the training data, the game engine imagines game scenarios and drives the opposing parties to conduct intelligent game confrontation in different scenarios. The game engine driver module exchanges training data and evaluation results with the model training and evaluation module through the digital channel to support the performance improvement of the intelligent model.

[0187] In the spectrum game confrontation target and environment simulation and software-defined interference side simulation subsystem, the RF channel simulation (including complex environment simulation) module provides RF analog information and digital analog information, and sends it to each simulation module through the RF interface library and the digital interface; each simulation module exchanges RF information and digital information through the RF interface and the digital interface respectively; the simulation modules of the two sides of the confrontation exchange information through the digital channel and the game engine driver of the game control subsystem to realize the closed loop of game confrontation.

[0188] In the data and resource management subsystem, various modules also transmit and exchange digital information through the digital bus; the data and resource management subsystem exchanges information with the analog subsystem and the game control subsystem through digital channels, and can realize functions such as different types of data collection, generation, distribution and access, analysis and processing, storage and retrieval, and supports the collection and storage of excellent resource / achievement information.

[0189] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An electromagnetic spectrum game intelligent training device, It is characterized in that include: Infrastructure layer and hardware resource layer; The infrastructure layer is connected to the hardware resource layer and is used to control the electromagnetic spectrum game intelligent training process and provide data resources and computing resources; The hardware resource layer is used to realize signal generation, channel simulation and environment simulation in the electromagnetic spectrum game process; The infrastructure layer includes a spectrum confrontation game control subsystem and a data and resource management subsystem; The spectrum confrontation game control subsystem is connected to the data and resource management subsystem, and is used to generate and display the simulation scene of the spectrum confrontation game, control the spectrum confrontation game training process; evaluate the spectrum confrontation game training process, and obtain confrontation effect evaluation information and training effect evaluation information; The data and resource management subsystem is used to collect and store data generated by each subsystem during the intelligent training of electromagnetic spectrum game, and to manage the hardware resource layer; store offline data sets used to train the interference decision-making evaluation model; The hardware resource layer includes a programmable target simulation subsystem, a software-defined interference side simulation subsystem, a radio frequency channel simulation subsystem, and a complex environment simulation subsystem; The programmable target simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem to generate a target signal; receive the channel target signal and the channel interference signal sent by the radio frequency channel simulation subsystem, process the channel target signal and the channel interference signal to obtain a result target signal; and send the result target signal to the spectrum confrontation game control subsystem; The software-defined interference side simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to receive the channel target signal sent by the radio frequency channel simulation subsystem, perform signal feature extraction processing on the channel target signal, and obtain signal feature information; Performing interference decision processing on the signal characteristic information to obtain a corresponding interference signal, and sending the interference signal to the radio frequency channel simulation subsystem; The radio frequency channel simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to simulate the radio frequency channel, attenuate the input signal and superimpose the environmental signal processing; The complex environment simulation subsystem is connected to the spectrum confrontation game control subsystem and the data and resource management subsystem, and is used to generate an environmental signal based on the radiation signal of the background target information, and send the environmental signal to the radio frequency channel simulation subsystem; The background target information includes the movement trajectory and radiation signal of the background target; The programmable target analog subsystem is used to generate a target signal, send the target signal to the radio frequency channel analog subsystem, receive the channel target signal and the channel interference signal sent by the radio frequency channel analog subsystem, process the channel target signal and the channel interference signal, and obtain a result target signal; The RF channel simulation subsystem is used to attenuate the target signal and superimpose the environmental signal to obtain a channel target signal; to attenuate the interference signal and superimpose the environmental signal to obtain a channel interference signal; to send the channel target signal to the programmable target simulation subsystem and the software-defined interference side simulation subsystem; and to send the channel interference signal to the programmable target simulation subsystem.

2. The electromagnetic spectrum game intelligent training device according to claim 1, It is characterized in that The software-defined interference side simulation subsystem includes a signal reconnaissance module, an interference decision module, and an interference signal generation module; The signal reconnaissance module is used to extract signal characteristics of the received signal to obtain signal characteristic information; the signal characteristic information includes carrier frequency, bandwidth, symbol rate, amplitude, and modulation type; The interference decision module is implemented using a trained interference decision evaluation model; The interference signal generating module is used to receive the interference signal parameters obtained by the interference decision module; generate a corresponding interference signal according to the interference signal parameters; the interference signal parameters include the interference signal center frequency, bandwidth, duty cycle, and modulation mode.

3. The electromagnetic spectrum game intelligent training device as claimed in claim 2, It is characterized in that The interference decision evaluation model includes an input module, an evaluation module, and a decision network; the evaluation module includes a behavior evaluation network and a target evaluation network; The input module receives data using the input terminals and outputs the data using the output terminals; The input end of the decision network is connected to the output end of the input module; The input end of the behavior evaluation network is connected to the output end of the input module; The output end of the behavior evaluation network is connected to the input end of the target evaluation network; The input end of the decision network is connected to the output end of the target evaluation network; The input end of the behavior evaluation network is connected to the output end of the decision network.

4. The electromagnetic spectrum game intelligent training device as claimed in claim 3, It is characterized in that The decision network comprises: a first input layer, a first hidden layer, a first upsampling layer, a first convolutional layer, a second hidden layer, a third hidden layer and a first output layer; The behavior evaluation network comprises: a second input layer, a fourth hidden layer, a second upsampling layer, a fifth hidden layer, a second convolutional layer, a first residual attention layer, and a second output layer; The target evaluation network comprises: a third input layer, a sixth hidden layer, a seventh hidden layer, an eighth hidden layer and a third output layer; The first to eighth hidden layers are all implemented using a fully connected network; The behavior evaluation network is used to process the input signal feature information and interference signal parameters to obtain a first estimated value; and update its parameters using the parameter update value; The target evaluation network processes the signal characteristic information and interference signal parameters to obtain an interference effect value; and updates its parameters using parameter update values; The decision network is used to process the signal characteristic information and the interference effect value to obtain interference signal parameters; and update the parameters using parameter update values.

5. The electromagnetic spectrum game intelligent training device as claimed in claim 4, It is characterized in that The training process of the interference decision evaluation model includes: S1, randomly initialize the parameters of the evaluation module and decision network of the interference decision evaluation model; initialize the training number value to 1; obtain an offline data set; the offline data set includes offline data; each offline data includes signal feature information, interference signal parameters and interference effect value; the interference effect value is obtained through adversarial effect evaluation information; S2, randomly sampling the offline data set to obtain a sampled data set; inputting the signal feature information in the sampled data set into the interference decision assessment model; S3, using the interference decision evaluation model to process the input data to obtain interference signal parameters and interference effect values ​​corresponding to the signal feature information; S4, using a first parameter updating model to update parameters of the behavior evaluation network and the target evaluation network; S5, updating the parameters of the decision network using a second parameter updating model; S6, increase the number of training times by 1, and determine whether the number of training times is greater than the set training threshold; if not, determine whether the sum of all interference effect values ​​obtained is greater than the sum of all interference effect values ​​in the sampling data set; if it is greater than the sum of all interference effect values ​​in the sampling data set, complete the training process of the interference decision evaluation model; if it is not greater than the sum of all interference effect values ​​in the sampling data set, execute S2; if it is greater than the training threshold, complete the training process of the interference decision evaluation model.

6. An intelligent training method for electromagnetic spectrum game, It is characterized in that The electromagnetic spectrum game intelligent training device used in any one of claims 1 to 5 comprises: Acquire scenario planning information and scenario assumption information; the scenario planning information includes the initial position of the target device, the initial position of the interference device, the movement information of the target device, and the movement information of the interference device; The scenario information includes the geographical environment information and background target information of the electromagnetic spectrum game intelligent training; the background target information includes the movement trajectory and radiation signal of the background target; Using the spectrum confrontation game control subsystem, the scenario planning information is loaded into the scenario assumption information to obtain simulation scenario information; Using a complex environment simulation subsystem, based on the radiation signal of the background target information, an environment signal is generated, and the environment signal is sent to the radio frequency channel simulation subsystem; Generate a target signal by using a programmable target simulation subsystem, send the target signal to a radio frequency channel simulation subsystem, receive a channel target signal and a channel interference signal sent by the radio frequency channel simulation subsystem, process the channel target signal and the channel interference signal to obtain a result target signal; send the result target signal to the spectrum confrontation game control subsystem; Utilizing the software-defined interference side simulation subsystem, receiving a channel target signal sent by the radio frequency channel simulation subsystem, performing signal feature extraction processing on the channel target signal to obtain signal feature information; performing interference decision processing on the signal feature information to obtain a corresponding interference signal, and sending the interference signal to the radio frequency channel simulation subsystem; Using the radio frequency channel simulation subsystem, the target signal is attenuated and superimposed with the environmental signal to obtain a channel target signal; the interference signal is attenuated and superimposed with the environmental signal to obtain a channel interference signal; the channel target signal is sent to the programmable target simulation subsystem and the software-defined interference side simulation subsystem; the channel interference signal is sent to the programmable target simulation subsystem; Utilizing the spectrum confrontation game control subsystem, evaluating and processing the result target signal to obtain confrontation effect evaluation information, and sending the confrontation effect evaluation information to the offline database and the software-defined interference side simulation subsystem; performing training evaluation processing on the confrontation effect evaluation information to obtain training effect evaluation information, and sending the training effect evaluation information to the offline database; The data and resource management subsystem is used to collect and store data generated by each subsystem during the intelligent training of electromagnetic spectrum game.

7. The electromagnetic spectrum game intelligent training method according to claim 6, It is characterized in that The step of evaluating the result target signal to obtain the confrontation effect evaluation information includes: The difference value between the result target signal and the corresponding target signal is calculated; and the confrontation effect evaluation information is determined according to the inverse of the difference value.

8. The electromagnetic spectrum game intelligent training method as claimed in claim 7, It is characterized in that The training evaluation process is performed on the confrontation effect evaluation information to obtain the training effect evaluation information, including: Using the confrontation effect evaluation information under a number of scenario planning information and scenario assumption information, a confrontation effect evaluation information set is constructed; The confrontation effect evaluation information set is used to construct a confrontation effect evaluation matrix; the element x in the i-th row and j-th column of the confrontation effect evaluation matrix ij , is the confrontation effect evaluation information under the i-th scenario planning information and the j-th scenario assumption information; Performing a first standard calculation process on the confrontation effect evaluation matrix to obtain a first standard matrix; The expression of the first standard calculation process is: Where m represents the number of scenario planning information, x ij represents the element of the ith row and jth column of the confrontation effect evaluation matrix, z ij represents the element of the i-th row and j-th column of the first standard matrix; Performing column optimization processing on the first standard matrix to obtain an optimal solution vector; The column optimal processing is to take out the largest number in each column to form the optimal solution vector z + ; The expression of the optimal solution vector is: Where n is the number of scenario information; Performing column worst processing on the first standard matrix to obtain a worst solution vector; The worst column processing is to take out the smallest number in each column to form the worst solution vector z - ; The expression of the worst solution vector is: The worst solution vector and the best solution vector are scored and calculated to obtain training effect evaluation information s; the score calculation is expressed as: In the formula, s is the training effect evaluation information, ω j Assume the importance weight of the information for the jth scenario; ω j , obtained by presetting, or by calculating the variance value of each column of the first standard matrix.