A method, electronic device and program product for predicting the performance of a pilot in a close air combat mission

Abstract

The application discloses a near-distance air combat pilot task performance prediction method, which predicts the task performance of a near-distance air combat pilot through a task performance prediction model, and the construction method of the task performance prediction model comprises the following steps: acquiring physiological state data, aircraft control data and air combat situation data of the pilot; extracting physiological state deep features, aircraft control deep features and air combat situation deep features from the physiological state data, the aircraft control data and the air combat situation data respectively; inputting the physiological state deep features, the aircraft control deep features and the air combat situation deep features into a cross-modal gated attention fusion module to fuse the physiological state deep features, the aircraft control deep features and the air combat situation deep features, so as to obtain cross-modal fusion features; and inputting the cross-modal fusion features into a full-connection predictor module to output a predicted pilot task performance grade, thereby realizing the prediction of the pilot task performance.

Description

Technical Field

[0001] This disclosure pertains to the field of flight control technology, and specifically relates to a method, electronic equipment, and program product for predicting the mission performance of close-range air combat pilots. Background Technology

[0002] In modern warfare, air combat is a crucial means of gaining and protecting air superiority. Air combat can be divided into close-range and medium-to-long-range air combat based on the distance between the engaging parties. Close-range air combat refers to operations conducted by pilots within visual observation range using infrared-guided munitions and cannons, while medium-to-long-range air combat refers to operations conducted by pilots outside visual observation range using active radar-guided missiles. Although beyond-visual-range (BVR) air combat has gradually become mainstream due to the extension of missile range and advancements in radar technology, close-range air combat remains the decisive stage in determining the outcome of air battles. Electronic warfare interference, sensor limitations, and the uncertainty of enemy behavior often lead to air combat devolving into close-range dogfights; therefore, close-range air combat capability is essential for gaining air superiority.

[0003] With the development of aviation technology, the modern air combat environment is becoming increasingly complex, characterized by intense competition, real-time response, incomplete information, and uncertain boundaries. This places more stringent demands on equipment performance and pilot combat skills. Air combat decision-making is the core of modern air combat. Currently, air combat decision-making mainly relies on human pilots. However, the battlefield situation in modern air combat changes rapidly, the amount of battlefield data provided to pilots has exploded, and the time available for pilots to analyze and make decisions has drastically decreased. This makes it almost impossible for pilots to make accurate tactical decisions in a timely manner, leading to a higher error rate and even affecting air combat effectiveness and mission success or failure. Therefore, a purely manned combat mode can no longer meet the mission requirements of modern air combat. Summary of the Invention

[0004] One aspect of this disclosure is a method for predicting the mission performance of close-range air combat pilots, which uses a mission performance prediction model to predict the mission performance of close-range air combat pilots. The method for constructing the mission performance prediction model includes...

[0005] Acquire pilot physiological data, aircraft control data, and air combat situation data;

[0006] Physiological state depth features, aircraft control depth features, and air combat situation depth features are extracted from the physiological state data, aircraft control data, and air combat situation data, respectively.

[0007] The physiological state depth features, aircraft control depth features, and air combat situation depth features are input into the cross-modal gating attention fusion module to fuse the physiological state depth features, aircraft control depth features, and air combat situation depth features to obtain cross-modal fused features.

[0008] The cross-modal fusion features are input into the fully connected predictor module, which outputs the predicted pilot mission performance level, thereby enabling the prediction of pilot mission performance.

[0009] This disclosure relates to the interdisciplinary field of artificial intelligence and aviation combat technology. Based on a multimodal deep learning-based pilot mission performance prediction framework, it realizes dynamic allocation of permissions in human-machine collaborative air combat systems, thereby improving air combat effectiveness and flight safety. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0011] Figure 1 A schematic diagram of a hierarchical attention module network architecture according to one embodiment of this disclosure.

[0012] Figure 2 A schematic diagram of the spatial frequency channel attention module network architecture according to one embodiment of this disclosure.

[0013] Figure 3 A schematic diagram of the network architecture of an edge-aware graph attention autoencoder network module according to one embodiment of the present disclosure.

[0014] Figure 4 A schematic diagram of a cross-modal gating attention fusion module network architecture according to one embodiment of this disclosure.

[0015] Figure 5 A fully connected predictor network architecture based on a Transformer encoder structure according to one embodiment of the present disclosure. Detailed Implementation

[0016] The development of artificial intelligence (AI) technology has prompted a shift in modern air combat from purely manned operations to human-machine collaborative operations. The core of human-machine collaborative operations lies in the allocation of human and machine authority: this requires dynamic adjustment of authority based on the pilot's future mission performance. When the pilot's future mission performance is excellent, they retain full tactical decision-making authority; as mission performance declines, the AI ​​system gradually increases its intervention in the pilot's decision-making and control; when future mission performance deteriorates significantly, the AI ​​system initiates emergency assistance or even takes over, replacing the pilot in decision-making and control, but the pilot still retains the emergency veto right. This human-machine authority adjustment strategy based on the prediction of the pilot's future mission performance can maximize flight safety and combat effectiveness. Furthermore, compared to traditional assessments and predictions of the pilot's cognitive state, mission performance is closely related to the core operational objectives of close-range air combat, making it more suitable for the operational needs of close-range air combat. In this disclosure, human-machine collaboration refers to the pilot, and the machine refers to the machine intelligence algorithm or machine intelligence agent used to control the aircraft. That is, the pilot and the machine intelligence agent collaborate, with the machine algorithm assisting the pilot. Furthermore, human-machine authority refers to the division of power and responsibilities among humans and machine intelligence in the decision-making and control chain within a human-machine collaborative combat system. In other words, it requires clarifying under what circumstances who (human or machine) makes the final decision and executes the operation. Mission performance, on the other hand, is a comprehensive and predictive assessment of a pilot's ability and effectiveness in completing a specific combat mission in the current and future period.

[0017] Currently, there is a lack of pilot mission performance prediction methods both domestically and internationally for close-range air combat. Firstly, mission performance prediction lacks specificity: existing research largely focuses on predicting flight parameters, cognitive states, or workload of civil aviation pilots. However, close-range air combat is characterized by high dynamism, strong maneuverability, and rapid decision-making under extreme pressure. Predictive indicators for civil aviation scenarios (such as flight trajectory deviation and conventional cognitive assessment) cannot meet the needs of close-range air combat. Secondly, there are limitations of single-modal information: pilot mission performance is influenced by multiple modalities, including physiological states (such as EEG and ECG), control behaviors (such as joystick, throttle, and rudder inputs), and air combat situations (such as relative positions and speeds of friendly and enemy forces). Single-modal data cannot comprehensively represent mission performance-related features. Finally, feature extraction and fusion efficiency is low: multimodal data exhibit heterogeneity (such as the high dynamic nonlinearity of physiological signals, the discreteness of control behaviors, and the spatial topological complexity of situational data), making it difficult for traditional manual feature engineering to effectively extract key information within a modality. Furthermore, static fusion methods (such as splicing and early / late fusion) cannot capture the dynamic dependencies between modalities. Therefore, there is an urgent need for a pilot mission performance prediction technology that is designed for close-range air combat scenarios, integrates multimodal information, and has efficient feature extraction and temporal modeling capabilities, so as to support the dynamic allocation of human and machine permissions.

[0018] To address the current lack of dedicated pilot mission performance prediction technology for close-range air combat, this disclosure provides a multimodal deep learning framework for pilot mission performance prediction in close-range air combat, achieving high-precision and highly generalizable mission performance prediction and supporting optimized allocation of permissions in human-machine collaborative air combat.

[0019] The core technical problem addressed by this disclosure is how to accurately predict a pilot's future mission performance in close-range air combat. This technical objective is achieved by constructing an end-to-end framework of "pilot multimodal data—intramodal feature extraction—cross-modal feature fusion—mission performance prediction," and the specific methods include:

[0020] (1) First, based on the established high-vacuum combat flight simulation environment, the physiological sensor equipment system of EEG and ECG is used to collect the pilot's physiological data (EEG and ECG signals), control data (control input of control stick, throttle stick and foot rudder) and situational data (position, speed, attitude angle, etc. of enemy and friendly aircraft) in 1-on-1 close air combat missions, forming a multimodal database of pilots.

[0021] The data was preprocessed and labeled with mission performance level tags frame by frame according to the close air combat mission performance evaluation model. The performance level is divided into four categories: excellent, good, medium and poor. The data was extracted by sliding time windows. For example, the time window length can be 30 seconds. Each time the data is slid forward by 1 second. Each historical time window (30 seconds) corresponds to the mission performance tag for the next 10 seconds, so as to form a pilot close air combat multimodal dataset. The close air combat mission performance evaluation model here adopts an existing evaluation model, such as the model proposed in reference [1].

[0022] Literature [1] Dong, Y.; He, S.; Zhao, Y.; Ai, J.; Wang, C. Development andEvaluation of Transformer-Based Basic Fighter Maneuver Decision-SupportScheme for Piloting During Within-Visual-Range Air Combat. Aerospace 2025,12, 73. https: / / doi.org / 10.3390 / aerospace12020073.

[0023] (2) Secondly, a dedicated feature extraction module was designed to address the characteristics of the three modalities: physiological state, operational behavior, and air combat situation. A cross-modal feature fusion module was also designed. A hierarchical attention module was designed to extract deep features from the physiological state data, such as... Figure 1As shown; a spatial frequency channel attention module is designed to extract deep features from manipulation behavior data, such as... Figure 2 As shown; an edge-aware graph attention autoencoder network is designed to extract deep features from air combat situational data, such as... Figure 3 As shown. Finally, a cross-modal gating attention fusion module is designed to fuse deep features from three modalities: physiological state, operational behavior, and air combat situation, outputting cross-modal fused features, such as... Figure 4 As shown.

[0024] (3) Finally, a fully connected predictor based on the Transformer encoder architecture is designed to receive the cross-modal fusion features output in step (2), learn key features through a multi-head self-attention mechanism, and map the output to the pilot's future mission performance level, thereby achieving pilot mission performance prediction, such as... Figure 5 As shown.

[0025] The disclosed method for predicting the mission performance of close-range air combat pilots based on multimodal deep learning has the following main advantages:

[0026] (1) Strong scenario-specific

[0027] This disclosure constructs a multimodal database of pilots specifically for close-range air combat, and for the first time applies multimodal deep learning technology to achieve pilot mission performance prediction adapted to close-range air combat, solving the problem of "civil aviation adaptation, air combat failure" in existing technologies.

[0028] (2) Comprehensive integration of multimodal human-machine information

[0029] This disclosure covers three key modalities: physiological state, maneuvering behavior, and air combat situation. It effectively captures core information within each modality through a dedicated feature extraction module and dynamically balances modal contributions through a cross-modal feature fusion module, thereby enhancing feature representation capabilities.

[0030] (3) Excellent prediction accuracy and generalization

[0031] This disclosure was validated through 10-fold cross-validation (average accuracy 94.43%) and leave-one-out cross-validation (average accuracy 84.73%). The model showed stable performance in predicting mission performance for different datasets and individual pilots.

[0032] According to one or more embodiments, a method for predicting the mission performance of close-range air combat pilots includes the following steps.

[0033] Step 1: Data Acquisition and Preprocessing.

[0034] This disclosure utilizes DCS World air combat simulation software to establish a highly realistic flight simulation environment, and conducts data acquisition and analysis based on this environment. The software demonstrates a high degree of realism in simulating flight dynamics, avionics systems, and fire control systems, providing an operational environment extremely close to actual flight. Simultaneously, this disclosure employs a 32-channel EEG cap (using a commercially available product, with electrodes arranged according to the international 10-20 system, a sampling frequency of 1000Hz, a maximum single-channel sampling frequency of 16kHz, and supporting real-time wireless data transmission) and a single-channel ECG smart vest sensor (using a commercially available smart sports vest with a body condition sensor, supporting real-time recording of ECG potential data) to acquire 32-channel EEG signals and a single-channel ECG signal from the pilot.

[0035] This data collection focuses on one-on-one close-range air combat, with both sides' aircraft piloted by experienced human pilots. Both sides use cannons for attacks. The data collection scenario is that the enemy and friendly aircraft are flying head-on at an initial distance of 12 km, with identical initial speed and altitude. They engage in basic fighter maneuvering (BFM) tactical combat until one side is shot down or crashes. Furthermore, to ensure fairness, both sides' aircraft are the same type of medium-sized, twin-engine, carrier-based third-generation fighter.

[0036] The collected data included: 32-channel EEG signals from the pilots, sampled at 1000 Hz; single-channel ECG signals, sampled at 256 Hz; lateral and longitudinal stick displacements, throttle displacements, and rudder displacements of the control sticks, sampled at 20 Hz; the aircraft's three-dimensional coordinates, velocity, acceleration, normal overload, and pitch, yaw, and roll attitude angles; the enemy aircraft's three-dimensional coordinates, velocity, acceleration; and the relative distance, approach rate, and velocity vector angle between the aircraft and the enemy aircraft, sampled at 20 Hz. A total of 13 pilots (10 males and 3 females, aged 18-26, with 1-4 years of flight experience) were collected for multimodal data.

[0037] For EEG signals, the following preprocessing method was adopted: First, bandpass filtering (4-45Hz) was performed on each channel; second, notch filtering (50Hz) was performed; then, independent component analysis was used to remove EEG / EMG artifacts; subsequently, EEG re-reference was performed based on the average value of the reference electrodes of both earlobes; then, downsampling was performed to 250Hz; finally, Pearson correlation analysis and principal component analysis were used sequentially to reduce the dimensionality of the number of EEG channels, and finally 11 main EEG channels were obtained, namely Fp1, Fp2, Fz, FC2, FC5, FC6, F7, C3, CP1, P4, and P8.

[0038] For the electrocardiogram (ECG) signal, the following preprocessing method is adopted: First, notch filtering (50 Hz) is performed; then, bandpass filtering (0.5-40 Hz) is performed; finally, the Pan-Tompkins algorithm is used to detect the R wave and calculate the heart rate per second.

[0039] For manipulation behavior data and air combat situation data, the original signal with a frequency of 20Hz is downsampled to 1Hz.

[0040] Step 2, Network Input and Output.

[0041] The network input includes physiological state, operational behavior, and air combat situation characteristics for each second within a historical 30-second time window. The output includes pilot mission performance rating labels (excellent, good, average, poor, or below average) for each second within a future 10-second time window. This disclosure uses a sliding time window method to segment the data samples, with a time window length of 30 seconds and a sliding step size of 1 second. Specific network input feature types are shown in Table 1.

[0042] Table 1. Description of Feature Types of Network Input

[0043]

[0044] Table 1 lists the feature descriptions for the three modality categories. Each modality contains several features, and each feature has its own feature dimension. Physiological state modalities include:

[0045] Variance (feature dimension 11) – The variance of the signal per second in each of the 11 channels of the EEG signal;

[0046] Energy (feature dimension 11) – the energy of the signal per second in each of the 11 channels of the EEG signal;

[0047] EEG Sample Entropy (Feature Dimension 11) – The sample entropy of the signal per second for each of the 11 channels of the EEG signal;

[0048] Differential entropy (feature dimension 11) – the differential entropy of the signal per second in each of the 11 channels of the EEG signal;

[0049] Spectral entropy (feature dimension 11) – the spectral entropy of the signal per second for each of the 11 channels of the EEG signal;

[0050] Heart rate (feature dimension 1) – Heart rate per second in a single ECG channel signal;

[0051] ECG sample entropy (feature dimension 1) – the sample entropy of a single ECG channel signal per second;

[0052] Skewness (feature dimension 1) – Skewness of a single-channel ECG signal per second;

[0053] Divergence (Feature Dimension 1) – The divergence of a single-channel ECG signal per second.

[0054] Manipulation behavior modalities include:

[0055] Lateral lever displacement (feature dimension 1) — the amount of lateral displacement of the joystick per second;

[0056] Longitudinal lever displacement (characteristic dimension 1) — the amount of longitudinal displacement of the joystick per second;

[0057] Rudder displacement (feature dimension 1) – the amount of displacement of the rudder per second;

[0058] Throttle lever displacement (feature dimension 1) – The amount of displacement of the throttle lever per second.

[0059] Air combat situation modes include:

[0060] My machine's position (feature dimension 3) - my machine's XYZ three-dimensional coordinates per second;

[0061] My aircraft's attitude angles (feature dimension 3) - my aircraft's pitch, yaw, and roll angles per second, in radians;

[0062] My aircraft speed (feature dimension 1) — my aircraft's actual airspeed per second;

[0063] My machine's acceleration (feature dimension 1) — My machine's acceleration per second;

[0064] Enemy aircraft position (feature dimension 3) – the enemy aircraft's XYZ three-dimensional coordinates per second;

[0065] Enemy aircraft speed (feature dimension 1) – the enemy aircraft’s actual airspeed per second;

[0066] Enemy aircraft acceleration (feature dimension 1) – the acceleration of the enemy aircraft per second;

[0067] Relative distance (feature dimension 1) – the relative distance between our aircraft and the enemy aircraft per second;

[0068] Approach rate (feature dimension 1) – the approach rate between our aircraft and the enemy aircraft per second;

[0069] Velocity Angle (Feature Dimension 1) – The angle between the velocity vectors of our aircraft and the enemy aircraft per second.

[0070] Here, the dimension of each feature represents the number of independent values ​​that feature represents in the input data.

[0071] The feature data input to the network is normalized using min-max normalization to reduce the risk of numerical instability during training.

[0072] The network output includes pilot mission performance rating labels (good, average, poor, or 4 categories) for each second within a 10-second time window. This disclosure uses a pilot mission performance evaluation method based on close-range air combat situational advantage value to label mission performance ratings. Specifically, it uses the close-range air combat integrated situational assessment method described in reference [1] to calculate the integrated situational value of our aircraft and the enemy aircraft in close-range air combat at each moment. and Then, the mission performance value of our pilots at each moment is calculated according to the following formula.

[0073] (1)

[0074] After determining the pilot's mission performance value, the quartiles (25%, 50%, 75%) of the performance value distribution were obtained using the equal-frequency binning method. The pilot's mission performance was then divided into four levels, and the corresponding mission performance value range and output labels for each level are shown in Table 2.

[0075] Table 2 Pilot Mission Performance Levels and Corresponding Value Ranges and Labels

[0076]

[0077] Here, pilot mission performance levels are categorized as excellent, good, average, and poor.

[0078] Step 2, Network Architecture and Training.

[0079] This disclosure presents a multimodal deep learning framework for predicting mission performance of close-range air combat pilots, whose network architecture includes a hierarchical attention module ( Figure 1 As shown), spatial frequency channel attention module ( Figure 2 As shown), the edge-aware graph attention autoencoder network module ( Figure 3 As shown), cross-modal gating attention fusion module ( Figure 4 (as shown) and a fully connected predictor based on the Transformer encoder architecture (as shown) Figure 5 (As shown). Specifically, the hierarchical attention module is used to extract deep features from physiological state data, the spatial frequency channel attention module is used to extract deep features from maneuver behavior data, the side-aware graph attention autoencoder network module is used to extract deep features from air combat situation data, the cross-modal gating attention fusion module is used to fuse the deep features of the above three modules and output cross-modal fused features. Finally, a fully connected predictor based on the Transformer encoder structure maps the cross-modal fused features to output as a pilot's future mission performance level label.

[0080] The network architecture parameters are shown in Table 3. During network training, the training and test sets were split using 10-fold cross-validation and leave-one-out-of-subjects cross-validation, respectively. All experiments were conducted on a Microsoft Windows 10 system equipped with an Intel Core i9-11900K processor and an NVIDIA GeForce RTX 3090 graphics card with 24GB of video memory. The Adam training algorithm was used, with an initial learning rate of 0.001 and a cosine annealing strategy to dynamically adjust the learning rate. An early stopping strategy was introduced to optimize the training process; training was terminated when the model showed no performance improvement for 20 consecutive epochs. The batch size during training was set to 64.

[0081] Table 3 Network Architecture Parameter Settings

[0082]

[0083] Table 3 shows the parameter settings for each module in the network architecture. The key parameters for the hierarchical attention module include the hidden layer dimension and the number of attention heads, with values ​​of 32 and 8, respectively. The key parameter for the spatial frequency channel attention module is the hidden layer dimension, with a value of 32. The key parameters for the edge-aware graph attention autoencoder network include the hidden layer dimension and the number of graph attention layers, with values ​​of 16 and 3, respectively. The key parameter for the cross-modal gated attention fusion module is the fusion feature dimension, with a value of 128. The key parameters for the fully connected predictor based on the Transformer encoder architecture include the number of attention heads and the number of encoder layers, with values ​​of 8 and 4, respectively. The key parameter for the training parameters module is the Dropout probability, with a value of 0.2. These parameter settings guide the construction and training process of the network model.

[0084] Step 3, Network Validation and Performance Evaluation.

[0085] This disclosure employs 10-fold cross-validation and leave-one-out-of-subjects cross-validation to evaluate the network's predictive performance and cross-subject generalization performance. In 10-fold cross-validation, all participants' experimental data are divided into 10 folds. One fold is randomly selected as the test set, and the remaining 9 folds are used as the training set. This process is repeated 10 times to train and test the network, and the average accuracy, precision, recall, and F1 score are calculated as performance metrics. The results of the 10-fold cross-validation are shown in Table 4. The network exhibits a prediction accuracy of no less than 90% in each fold test, with a final average accuracy of 94.43%, indicating that the network possesses excellent predictive performance.

[0086] Table 4. Network performance of 10-fold cross-validation

[0087]

[0088] In leave-one-subject cross-validation, all data from one subject was selected as the test set each time, while the data from the remaining 12 subjects was used as the training set. This process was repeated 13 times to train and test the network. The average accuracy, precision, recall, and F1 score were calculated as performance metrics. The results of leave-one-subject cross-validation are shown in Table 5. The network achieved an average prediction accuracy of 84.73% across all subjects, with most subjects achieving an accuracy of over 85%, indicating that the network possesses good cross-subject generalization performance.

[0089] Table 5. Network performance of leave-one-out cross-validation

[0090]

[0091] like Figure 1 The diagram shows the hierarchical attention module network architecture. EEG represents electroencephalogram (EEG) signals; ECG represents electrocardiogram (ECG) signals; FA1 and FA2 both represent feature-level attention, CA represents channel-level attention, and the three share the same network structure but differ in the number of layers and nodes; MA represents modality-level attention; GAP represents global average pooling; FC represents fully connected layers; and BN represents batch normalization.

[0092] Figure 1 The hierarchical attention module network architecture aims to process and analyze physiological signal data, particularly electroencephalogram (EEG) and electrocardiogram (ECG) signals. Specifically, it includes:

[0093] (1) Input layer, including EEG and ECG, which represent electroencephalogram (EEG) and electrocardiogram (ECG) signals respectively, as input to the network.

[0094] (2) The feature extraction layer includes FA1 and FA2 feature-level attention modules, used to extract features from EEG and ECG signals; it also includes a CA channel-level attention module for EEG, used to further extract the relationships between EEG channels on the EEG features extracted by feature-level attention, thereby enhancing the expressive power of EEG features. These three modules can share the same network structure, but their internal parameters are different. For each module, the input feature dimension is N. After passing through the first fully connected layer FC, the feature dimension is reduced to N / 2, and then the feature dimension is increased to N by the second fully connected layer FC.

[0095] (3) Modality fusion layer, including MA modality-level attention module, is used to fuse the features of EEG and ECG signals. It generates a comprehensive feature representation by comparing and weighing the features of different modalities. The EEG and ECG features are projected by a 32-node fully connected layer FC, and then activated by the ReLU function. The activated projections are added together and then fed into a 2-node fully connected layer and the Softmax function for weight calculation. Finally, the weights are multiplied by the input EEG and ECG features respectively and concatenated to obtain the final modality fused features.

[0096] (4) Feature processing layer, which includes, in sequence:

[0097] Fully connected (FC) layers are used for further processing of features, typically for non-linear transformations of features, with 32 nodes.

[0098] Batch normalization (BN) layers are used to accelerate the training process and improve the stability of the model.

[0099] ReLU activation function layers are used to introduce non-linearity and learn complex feature representations;

[0100] The Dropout layer uses regularization to prevent overfitting, and the loss probability is set to 0.1.

[0101] (5) Output layer: deep features R are obtained through a ReLU activation function layer. T×l , where T represents the length of the time series and l represents the dimension of the feature.

[0102] In the diagram, element-wise addition (⊕) and element-wise multiplication (⊗) are used for feature fusion operations, representing weighted summation and weighted product of features, respectively.

[0103] like Figure 2 The diagram shows the network architecture of the spatial-frequency channel attention module. ResBlock represents the residual block, Conv represents the convolution kernel, and 1×1 / 3×3 / 5×5 represents the kernel size; MSFC represents multi-scale frequency domain convolution, FFT represents fast Fourier transform; SENet represents compressed excitation attention network; BN represents batch normalization. This spatial-frequency channel attention module network architecture aims to process and analyze manipulation behavior data, extract spatial and frequency domain features, and enhance the expressive power of features through an attention mechanism. Specifically, it includes:

[0104] (1) Input layer R is the data of manipulation behavior, which serves as the input to the network.

[0105] (2) Feature extraction layer, including:

[0106] ResBlock residual blocks are used to extract spatial features. They are achieved by stacking multiple convolutional layers (Conv) and batch normalization layers (BN), and finally by element-wise addition (⊕) to realize residual connections.

[0107] The MSFC multi-scale frequency domain convolution module is used to extract frequency domain features. It converts the time domain signal into a frequency domain signal through Fast Fourier Transform (FFT) to obtain the amplitude, phase, real part, and imaginary part features of the frequency domain signal. After concatenating the four types of features, it upsamples them to 32 dimensions and then performs convolution operations using convolution kernels of different sizes (1×1, 3×3, 5×5) to extract frequency domain features at different scales. After these features are concatenated, they are batch normalized and activated by the ReLU function to obtain the final frequency domain features.

[0108] The features are concatenated to combine spatial and frequency domain features into a comprehensive feature representation.

[0109] (3) Attention Mechanism: The SENet layer is a compressed and stimulated attention network used to weight the concatenated features. SENet generates channel attention weights through global average pooling (GAP) and fully connected layers (FC), and then weights the features through element-wise multiplication (⊗) to enhance the expressive power of important features. The first fully connected layer reduces the feature dimension to 1 / 4 of the original dimension, and the second fully connected layer restores the feature dimension to the original dimension, with the aim of enhancing the non-linear expressive power of the features.

[0110] (4) Feature processing layer, including:

[0111] Conv convolutional layers, with 1×1 convolutional kernels, are used for non-linear transformations of features;

[0112] Batch normalization (BN) layers are used to accelerate the training process and improve the stability of the model.

[0113] The ReLU activation function is used to introduce non-linearity so that the model can learn more complex feature representations.

[0114] By using residual connections, the original manipulation behavior data is processed through Conv convolutional layers, BN batch normalization layers, and ReLU activation functions, and then concatenated with the processed spatial frequency domain fusion features to enhance the robustness of the features.

[0115] (5) Output layer, output depth features.

[0116] like Figure 3The diagram shows the network architecture of the Edge-Aware Graph Attention Network Autoencoder (EAGAT-AE) module. MLP stands for Multilayer Perceptron; FC stands for Fully Connected Layer. This EAGAT-AE network architecture aims to process and analyze air combat situational data, learning relationships between nodes through a graph attention mechanism and using an autoencoder structure to extract and reconstruct features. The network structure includes:

[0117] (1) Input layer, which contains air combat situation data;

[0118] (2) Graph construction layer, set up our side and enemy side, respectively representing the aircraft and other entities of our side and enemy side in air combat; convert the air combat situation data into a graph structure, where nodes represent aircraft, node features are the input features of enemy and friendly aircraft respectively, and edges represent the interaction or relationship between enemy and friendly aircraft.

[0119] (3) Edge-aware graph attention layer: The graph attention mechanism is used to learn the relationships between nodes in the graph. The node features of our aircraft and the enemy aircraft are projected into hidden features by a multilayer perceptron. The two multilayer perceptrons have the same number of layers and nodes (2 layers, 32 and 16 nodes respectively). The hidden features and the original edge features are input into the graph attention network layer. The topological representation of the interaction between nodes and the interaction between nodes and edges is learned based on the message passing mechanism of the graph network. The graph network has a total of 3 layers.

[0120] (4) Feature extraction layer: extracts features of our aircraft, relative features and enemy aircraft features. The relative features are the relative features between our aircraft and enemy aircraft. The relative features are extracted from the original edge features by a multilayer perceptron with 2 layers and 32 and 16 nodes.

[0121] (5) Feature fusion layer: different features are spliced ​​together to form a comprehensive feature representation. The spliced ​​features are weighted by an attention mechanism to highlight important features. The number of nodes in the fully connected layer (FC) of the attention mechanism is 32.

[0122] (6) Encoding and decoding layer, including encoder and decoder. The encoder encodes the input air combat situation data into a low-dimensional feature representation. The decoder reconstructs the original air combat situation data from the low-dimensional feature representation. The network is trained under supervision by the mean square error between the reconstructed data and the original data to optimize the network parameters.

[0123] (8) Output layer: After the encoder layer is frozen, it is used as a feature extractor. The feature extractor outputs the deep features of the air combat situation.

[0124] like Figure 4As shown in the figure, (a) represents the network architecture of the cross-modal gating attention fusion module; (b) Intra-M represents intra-modal gating; and (c) Inter-M represents inter-modal gating. Depth features representing physiological state modalities; Depth features representing the manipulation behavior modality; Indicates the depth characteristics of air combat situation modes; represents cross-modal fusion features; FC represents a fully connected layer; mean represents the average value; WS represents a weighted summation. Figure 4 The cross-modal learning structure uses a gating mechanism to fuse information from different modalities. The cross-modal gating attention fusion module combines information from different modalities. The figure shows the features of three modalities (physiological state, maneuvering behavior, and air combat situation). The features of each modality are processed through its own intra-modal gating module (Intra-M), and then fused through the inter-modal gating module (Inter-M). Each gating module contains two fully connected layers: the first fully connected layer has 64 nodes, and the second fully connected layer has 128 nodes.

[0125] Here, L_PS, L_CB, and L_CS represent the deep features of physiological state, maneuvering behavior, and air combat situation modalities, respectively. L_Fused is the final cross-modal fusion feature, obtained by weighted summation of features from different modalities. Intra-modal gating (Intra-M) weights features within a single modality to highlight the most important features and suppress unimportant ones. The inter-modal gating (Inter-M) module is responsible for feature fusion between different modalities, achieving feature fusion by calculating inter-modal correlations.

[0126] like Figure 5 As shown, Figure 5 (a) A fully connected predictor network architecture based on the Transformer encoder structure; (b) A multi-head self-attention mechanism. The fully connected predictor network architecture includes, in order:

[0127] The input embedding layer, serving as the input layer of the network, transforms the input data into embedding vectors;

[0128] The multi-head self-attention component captures different features through multiple parallel self-attention layers, with a total of 8 attention heads;

[0129] The feedforward neural network operates independently at each location after the self-attention layer, and this layer has 256 nodes.

[0130] The addition & layer normalization layer follows the self-attention layer and the feedforward network. The output is added to the input (residual connection) and then normalized.

[0131] The feature aggregation layer, following multiple Transformer encoder layers, aggregates features using a global average aggregation.

[0132] The linear layers process the aggregated features through a series of linear layers. The number of nodes in the two linear layers are 32 and 4, respectively. The layers also include a ReLU activation function and a Dropout regularization layer, with the latter having a loss probability of 0.2.

[0133] The network output task performance level is determined, and the network output prediction results are obtained.

[0134] It should be understood that in the embodiments of this disclosure, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for predicting the mission performance of close-range air combat pilots, characterized in that, A mission performance prediction model is used to predict the mission performance of close-range air combat pilots. The method for constructing the mission performance prediction model includes... Acquire pilot physiological data, aircraft control data, and air combat situation data; Physiological state depth features, aircraft control depth features, and air combat situation depth features are extracted from the physiological state data, aircraft control data, and air combat situation data, respectively. The physiological state depth features, aircraft control depth features, and air combat situation depth features are input into the cross-modal gating attention fusion module to fuse the physiological state depth features, aircraft control depth features, and air combat situation depth features to obtain cross-modal fused features. The cross-modal fusion features are input into the fully connected predictor module, which outputs the predicted pilot mission performance level, thereby enabling the prediction of pilot mission performance.

2. The method according to claim 1, characterized in that, The physiological state depth features are extracted from the physiological state data using a hierarchical attention module.

3. The method according to claim 1, characterized in that, The spatial frequency channel attention module extracts the aircraft control depth features from the aircraft control data.

4. The method according to claim 1, characterized in that, The edge-aware graph attention autoencoder module extracts deep features of the air combat situation from the air combat situation data.

5. The method according to claim 1, characterized in that, The physiological data includes electroencephalogram (EEG) and electrocardiogram (ECG) data of pilots during close-range air combat missions.

6. The method according to claim 1, characterized in that, The aircraft control data includes the pilot's input data on the control stick, throttle, and rudder during close-range air combat missions.

7. The method according to claim 1, characterized in that, The air combat situation data includes the position, speed, and attitude angle data of enemy and friendly aircraft during close-range air combat missions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images