Method and system for automatic identification of human factor critical phases in simulated flight

Abstract

The present application relates to a kind of automatic identification method and system of simulating flight human factor key stage, belong to aviation medical training, flight automatic evaluation, verification field, solve the flight action of relying on expert experience manual discrimination, there is strong subjectivity, low efficiency, difficult to quantify and real-time feedback defects, unable to meet the needs of flight training and safety evaluation on precision and real-time problem.The method comprises the following steps: obtaining a plurality of flight parameter data in simulated flight and preprocessing to obtain preprocessed flight parameter time series data;extracting the time domain features and frequency domain features of the flight parameter time series data and splicing them into corresponding global statistical features;inputting the flight parameter time series data and global statistical features into a trained human factor key stage automatic identification model to obtain a human factor key stage identification result, which is used to evaluate the flight stability of the pilot in combination with the heart rate obtained at the corresponding time.The method realizes objective, efficient and quantitative evaluation of the flight stability of the pilot in the human factor key stage.

Description

Technical Field

[0001] This invention relates to the fields of aviation medical training, automatic flight evaluation and verification, and electronic digital data processing, and particularly to a method and system for automatic identification of key human factors during simulated flight. Background Technology

[0002] The pilot's flight stability at each stage of flight is closely related to the mission execution and the flight stage itself; the mission execution, in turn, is related to the quality of completion of the critical human factors phases during flight. During flight, pilots are essentially in two states: level flight patrol and the occurrence of critical human factors phases. During level flight patrol, the pilot's overall physiological and psychological state remains relatively stable, with relatively minor changes in flight stability. However, during critical human factors phases, the pilot's physiological and psychological state changes drastically, resulting in relatively significant changes in flight stability. Therefore, studying the physiological and psychological changes during critical human factors phases in simulated flight can reflect the pilot's physical and mental state during that particular flight.

[0003] Currently, the most widely used algorithms for flight maneuver recognition rely primarily on expert experience to manually judge flight maneuvers. This approach is highly subjective, inefficient, and difficult to quantify and provide real-time feedback. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide an automatic identification method and system for key human factors during simulated flight, in order to solve the technical problems of existing methods that rely on expert experience to manually judge flight actions, which have the disadvantages of strong subjectivity, low efficiency, difficulty in quantification and real-time feedback, and cannot meet the requirements of accuracy and real-time performance for flight training and safety assessment.

[0005] The objective of this invention is mainly achieved through the following technical solutions:

[0006] This invention provides a method for automatic identification of critical human factors during simulated flight, comprising the following steps:

[0007] Acquire various flight parameter data from the simulated flight and preprocess them to obtain preprocessed flight parameter time series data;

[0008] Extract the time-domain and frequency-domain features of the flight parameter time-series data and concatenate them into the corresponding global statistical features;

[0009] The flight parameter time series data and the global statistical features are input into the trained human factor critical phase automatic identification model to obtain the human factor critical phase identification results, which are used to evaluate the pilot's flight stability in combination with the heart rate at the corresponding time.

[0010] Furthermore, various flight parameter data include flight altitude, flight speed, flight G-force, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed.

[0011] Furthermore, the time-domain features include mean, standard deviation, range, maximum and minimum values, median, sum of squares, and average absolute rate of change; the frequency-domain features include maximum frequency component and average frequency component features.

[0012] The frequency domain features of the flight parameter time series data are extracted using Fast Fourier Transform.

[0013] Furthermore, the automatic identification model for key human factors stages includes a temporal dynamic branch, a global statistical branch, and a feature fusion classification branch;

[0014] The time-series dynamic branch obtains a time-series dynamic feature vector based on the input time-series flight parameter data;

[0015] The global statistics branch, based on the input global statistical features, obtains a global statistical feature vector;

[0016] The feature fusion classification branch concatenates and fuses the temporal dynamic feature vector and the global statistical feature vector, and then inputs them into the intermediate fully connected layer for classification to obtain the human factor key stage identification result.

[0017] Furthermore, the temporal dynamic branch sequentially includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, an LSTM layer, a temporal Dropout layer, and a temporal fully connected layer;

[0018] The first convolutional layer uses 64 arrays of size 1. The convolution kernel is used to convolve the flight parameter time series data to obtain the first time series feature;

[0019] The first pooling layer uses max pooling, and the window size is [size missing]. With a step size of 2, the first time-series feature is reduced in dimensionality while retaining significant features to obtain the second time-series feature;

[0020] The second convolutional layer uses 128 arrays of size 128. The convolutional kernel captures higher-order combined features from the second temporal feature to obtain the third temporal feature;

[0021] The second pooling layer uses max pooling, with a window size of [size missing]. With a step size of 2, we further focus on the features of the key time nodes of the third time series feature to obtain the fourth time series feature.

[0022] The LSTM layer is used to obtain the 128-dimensional hidden state features of the last time step of the fourth temporal feature, and thus obtain the fifth temporal feature.

[0023] The temporal Dropout layer, based on the fifth temporal feature, randomly discards 50% of the neuron outputs to prevent overfitting, thus obtaining the sixth temporal feature;

[0024] The time-series fully connected layer is used to map the sixth time-series feature to a 64-dimensional vector and perform a ReLU activation nonlinear transformation to obtain the time-series dynamic feature vector.

[0025] Furthermore, the global statistics branch sequentially includes a global fully connected layer, a global Dropout layer, and a global output layer;

[0026] The global fully connected layer maps the global statistical features to 32 dimensions through ReLU activation nonlinear transformation, reduces the dimensionality, and learns the correlation between statistical features to obtain the first global feature.

[0027] The global Dropout layer randomly discards 30% of the neurons in the first global feature to enhance generalization ability and obtain the second global feature;

[0028] The global output layer compresses the second global feature to 16 dimensions, retains key statistical features, and obtains a global statistical feature vector.

[0029] Furthermore, the feature fusion classification branch sequentially includes a feature fusion layer, an intermediate fully connected layer, and an output layer;

[0030] The feature fusion layer is used to concatenate a 64-dimensional temporal dynamic feature vector and a 16-dimensional global statistical feature vector to obtain an 80-dimensional first fused feature.

[0031] The intermediate fully connected layer is used to map the first fused feature to 32 dimensions through ReLU activation to obtain the second fused feature;

[0032] The output layer is used to output the probability distribution of 7 types of human factors key stages by activating the 32-dimensional second fusion feature through Softmax.

[0033] Furthermore, the automatic identification model for key human factors stages is trained through the following process to obtain a well-trained automatic identification model for key human factors stages.

[0034] Acquire sample data including flight parameter time series data and corresponding global statistical features, and form a sample training set with the corresponding sample labels; wherein, the sample labels represent the corresponding different human factor key stages;

[0035] Initialize the learning rate and preset the maximum number of iterations;

[0036] Adam is selected as the optimizer, and the model parameters are iteratively optimized through forward and backward propagation until the classification cross-entropy loss function converges or the training stops after reaching the preset maximum number of iterations. The model parameters with the minimum loss are saved as the trained human factors key stage automatic recognition model.

[0037] Furthermore, the key human factors phases include takeoff, climb, navigation, level flight, mission execution, return, and landing;

[0038] The key human factors stages are converted into corresponding integers using LabelEncoder, which serve as sample labels.

[0039] This invention provides an automatic identification system for key human factors during simulated flight, including a data acquisition and preprocessing module M1, a feature extraction module M2, and a key human factors identification and evaluation module M3.

[0040] The data acquisition and preprocessing module M1 is used to acquire and preprocess various flight parameter data in the simulated flight to obtain preprocessed flight parameter time series data.

[0041] Feature extraction module M2 is used to extract the time-domain and frequency-domain features of the flight parameter time-series data and concatenate them into the corresponding global statistical features;

[0042] The human factor critical phase identification and evaluation module M3 is used to input the flight parameter time series data and the global statistical features into the trained human factor critical phase automatic identification model to obtain the human factor critical phase identification result, which is used to evaluate the pilot's flight stability in combination with the heart rate at the corresponding time.

[0043] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0044] 1. This invention identifies critical human factors stages through automated feature extraction and machine learning models, reducing reliance on expert experience and thus lowering subjectivity. By utilizing the time and frequency domain features of flight parameters, it can more accurately identify critical human factors stages during simulated flight, improving the objectivity and accuracy of critical human factor stage identification; the identified critical human factors stages are combined with the corresponding heart rate to assess the pilot's flight stability.

[0045] 2. This invention processes flight parameter data in real time, quickly extracts features, and automatically identifies key human factors at critical stages. This real-time capability enables the system to provide immediate feedback to pilots during flight missions, helping them adjust flight strategies and physiological and psychological states in a timely manner, thereby improving the efficiency of flight training and safety assessment, and enhancing real-time performance and feedback efficiency.

[0046] 3. This invention, by accurately identifying critical human factors phases in flight, can provide pilots with more targeted training, helping them better master key flight skills. Simultaneously, by assessing the pilot's flight stability during these critical phases, potential flight risks can be identified in a timely manner, thereby improving the accuracy of flight safety assessments and reducing the incidence of flight accidents.

[0047] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0048] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0049] Figure 1 This is a flowchart of the automatic identification method for key human factors during simulated flight in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram illustrating the concatenation of time-domain features and frequency-domain features to obtain the corresponding global statistical features in an embodiment of the present invention.

[0051] Figure 3 This is a schematic diagram of the structure and input / output features of the automatic identification model for key human factors stages in an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the automatic identification system module for key human factors during simulated flight in an embodiment of the present invention. Detailed Implementation

[0053] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0054] Example 1:

[0055] The critical phase of human factors analysis refers to the actions taken by pilots during flight that significantly impact flight safety, mission execution, and their own physical and mental state. These actions are typically accompanied by significant physiological and psychological changes, placing extremely high demands on the pilot's technical and psychological qualities. By automatically identifying and analyzing these actions, pilots' flight skills and physical and mental state can be better assessed, thereby improving the efficiency of flight training and flight safety.

[0056] The key human factors phases closely related to the mission during flight include takeoff, climb, navigation, level flight, mission execution, return, and landing.

[0057] Based on flight simulator research, key human factors in flight are studied, and data analysis and feature summarization are performed to obtain time-series flight parameter data and corresponding global statistical features. Based on the established automatic identification model of key human factors in simulated flight, the identification results of key human factors are obtained. Based on the automatically identified key human factors, a flight stability discrimination system is established to calculate the pilot's flight stability.

[0058] Changes in flight stability during critical human factors phases reflect a pilot's flight condition and ability. Summarizing the characteristics of these critical human factors phases, identifying them, and combining this information with heart rate data at corresponding moments to assess flight stability can provide timely warnings of when pilots need rest, thus reducing safety risks.

[0059] A specific embodiment of the present invention discloses a method for automatic identification of key human factors during simulated flight, such as... Figure 1 As shown, it includes the following steps:

[0060] Step S1: Acquire various flight parameter data from the simulated flight and preprocess them to obtain preprocessed flight parameter time series data;

[0061] Step S2: Extract the time-domain and frequency-domain features of the flight parameter time-series data and concatenate them into the corresponding global statistical features;

[0062] Step S3: Input the flight parameter time series data and the global statistical features into the trained human factor critical stage automatic identification model to obtain the human factor critical stage identification result, which is used to evaluate the pilot's flight stability in combination with the heart rate at the corresponding time.

[0063] Step S1 includes steps S11-S12.

[0064] Step S11: Obtain the pilot's flight parameter data and physiological data.

[0065] Flight simulation experiments were conducted using flight simulators to collect flight parameter data and physiological data of pilots during key stages.

[0066] Multiple flights were conducted using a flight simulator to simultaneously collect various flight parameter data and physiological data. The flight parameter data was then preprocessed to obtain preprocessed time-series flight parameter data.

[0067] Flight parameter data is acquired based on a flight simulator, collecting flight parameter data from the pilot during continuous simulated flight in real time. For example, flight parameter data is acquired using a flight parameter recorder.

[0068] Multiple flight parameter data include flight altitude, flight speed, flight G-force, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed.

[0069] The physiological data is the pilot's heart rate data. For example, the pilot's heart rate data is acquired using a dynamic electrocardiogram recorder; physiological data of the pilot is collected in real time during continuous simulated flight.

[0070] During the experiment, key stages for human flight were defined.

[0071] The key human factors phases include takeoff, climb, navigation, level flight, mission execution, return to base, and landing;

[0072] The key human factors stages are converted into corresponding integers using LabelEncoder, which serve as sample labels.

[0073] LabelEncoder is a utility class provided by scikit-learn that maps discrete category labels (such as strings or integers) to consecutive integers starting from 0, making it easier for machine learning models to process them.

[0074] The sample label list in this invention is shown in Table 1.

[0075] Table 1: Sample Label List

[0076]

[0077] Step S12: Preprocess the various flight parameter data to obtain the preprocessed flight parameter time series data.

[0078] The preprocessing includes:

[0079] Missing values ​​in the flight parameter data are padded forward and standardized to obtain standardized continuous flight parameter time series data.

[0080] The standardized continuous flight parameter time series data is divided into time series segments of a preset fixed length, and the insufficient parts are padded with the average value to obtain multiple flight parameter time series data.

[0081] The acquired flight parameter data, including flight altitude, flight speed, flight G-force, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed, are processed for missing values ​​and standardized.

[0082] For example, missing value processing adopts a forward filling method, which fills the missing values ​​with the values ​​of the previous time step in order to preserve the dynamic change trend of flight maneuvers to the greatest extent.

[0083] Standardization transforms the flight parameter data, after filling in missing values, into a distribution with a mean of 0 and a standard deviation of 1, as shown below:

[0084] Formula (1)

[0085] in, These are the standardized values; The flight parameter data obtained after filling in missing values; This represents the mean of the flight parameter data after filling in the missing values. This represents the standard deviation of the flight parameter data after filling in the missing values.

[0086] Because the dimensions and ranges of flight parameter data vary greatly—for example, flight altitude is several thousand kilometers, roll angle is between -180° and 180°, pitch angle is between -90° and 90°, and yaw angle is between -90° and 90°—if the raw flight parameter data is used directly to train the model, features with large numerical ranges, such as the large range of flight altitude changes, may dominate the learning process of the automatic identification model in critical human factors stages, while the influence of features with small numerical ranges, such as changes in angles (roll, pitch, and yaw), will be weakened.

[0087] The standardized flight parameter data have the same scale, ensuring that flight altitude, flight speed, flight overload, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed data can participate in model training with equal weight.

[0088] The preprocessed continuous flight parameter data is divided into sequences of preset fixed lengths.

[0089] For example, a fixed time step is preset to 30 seconds, but this can be revised according to specific needs in actual applications.

[0090] For time steps less than 30 seconds, the mean of continuous flight parameter data is used to fill in the gaps. Each sequence corresponds to a sample label for a key human factor stage, namely, the integer value corresponding to takeoff, climb, navigation, level flight, mission execution, return to base, or landing.

[0091] Step S1 aims to acquire various flight parameter data and physiological data. By preprocessing the various flight parameter data, including handling missing values, standardization, and segmentation into time series segments of preset fixed length, standardized flight parameter time series data is generated for subsequent training of the human factors key stage automatic identification model.

[0092] Step S2, specifically.

[0093] From the time-series data of flight parameters such as flight altitude, flight speed, flight G-force, pitch angle, roll angle, heading angle, northward speed, azimuth speed, and eastward speed, time-domain features and frequency-domain features are extracted, and the corresponding global statistical features are obtained by concatenating the time-domain features and frequency-domain features. For example... Figure 2 As shown.

[0094] The time-domain features include mean, standard deviation, range, maximum and minimum values, median, sum of squares, and average absolute rate of change; the frequency-domain features include maximum frequency component and average frequency component features.

[0095] The frequency domain features of the flight parameter time series data are extracted using Fast Fourier Transform.

[0096] Time-domain analysis is a method for directly analyzing the time-series data of original flight parameters that change over time. Its core is to capture the numerical distribution, dynamic change patterns, and key time node characteristics of flight parameters in the time dimension, thereby revealing the essential attributes of flight maneuvers.

[0097] (1) Mean, which reflects the overall trend of flight parameters. For example, the mean of flight altitude parameters during the climb phase is usually higher than that during the level flight phase.

[0098] The mean values ​​are calculated as follows: the mean values ​​of flight altitude, flight speed, flight G-force, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed are all calculated using the same method, as shown below:

[0099] Formula (2)

[0100] in, This is the mean of all data points in the time series data of flight parameters; This represents the total number of data points collected in the flight parameter time series data. This refers to the value of each collected data point in the flight parameter time series data;

[0101] (2) Standard deviation reflects the degree of fluctuation in the time series of flight parameters. For example, the standard deviation of the roll angle during the mission phase is usually greater than that during the level flight phase, reflecting a significant change in the angle.

[0102] Standard deviation The calculation is as follows:

[0103] Formula (3)

[0104] (3) Range, which reflects the maximum variation of flight parameter time series data. For example, the range of flight altitude at mission nodes will be significantly greater than that during level flight.

[0105] The range is calculated as follows:

[0106] Formula (4)

[0107] (4) Maximum and minimum values ​​reflect the extreme states of flight parameter time series data in the sequence. For example, the maximum pitch angle may correspond to the peak of the climb in the mission, and the minimum pitch angle may correspond to the lowest point of the dive in the mission.

[0108] The maximum and minimum values ​​are calculated as follows:

[0109] Formula (5)

[0110] Formula (6)

[0111] in, This represents the maximum value of all data points in the flight parameter time series data; It is the minimum value of all data points in the flight parameter time series data. These are the flight parameter time series data. The value of each data point.

[0112] (5) The median is more resistant to the influence of outliers than the mean. For example, during the execution of a mission, when there is a momentary disturbance such as flight overload, the median reflects the true trend better than the mean.

[0113] The median is calculated as follows:

[0114] Formula (7)

[0115] in, This is the median.

[0116] (6) Sum of squares, reflecting the cumulative energy of flight parameters. If a large flight overload is maintained for a long time during the mission, the sum of squares of the flight overload will be significantly higher.

[0117] The sum of squares is calculated as follows:

[0118] Formula (8)

[0119] in, It is the sum of squares.

[0120] (7) Average absolute rate of change reflects the degree of change in flight parameter data. For example, the absolute value of the difference in angles (pitch angle, roll angle, heading angle) during the mission phase is much greater than that during level flight.

[0121] The average absolute rate of change is calculated as follows:

[0122] Formula (9)

[0123] in, This represents the average absolute rate of change.

[0124] Frequency domain features include maximum frequency component and average frequency component features. Frequency domain analysis extracts the frequency domain features of the flight parameter time series data using Fast Fourier Transform (FFT).

[0125] Calculate the absolute value (modulus of the complex number) of the Fourier transform result and extract the amplitude information of the frequency components;

[0126] The maximum frequency component and average frequency component features are extracted to distinguish different types of flight maneuvers. The frequency domain features and time domain features complement each other to improve the recognition capability of the automatic recognition model for key human factors stages.

[0127] For time series data of flight parameters, the discrete Fourier transform is as follows.

[0128] Formula (10)

[0129] Formula (11)

[0130] in, No. Complex representation of each frequency domain component; For the first Index of each frequency domain component ; The imaginary unit; For complex numbers The real part; For complex numbers The imaginary part; For frequency The corresponding amplitude (energy) reflects the intensity of the frequency component.

[0131] The maximum frequency component reflects the strongest frequency fluctuation in flight data. For example, the maximum frequency component is usually smaller during level flight, while a stronger high-frequency component is generated during mission execution.

[0132] The maximum frequency component is calculated as follows:

[0133] Formula (12)

[0134] in, For the maximum frequency component, Represents frequency The corresponding signal amplitude (energy intensity) reflects the flight parameter data at a given frequency. The fluctuation energy.

[0135] The average frequency component reflects the overall activity level of flight parameter fluctuations. For example, during the level flight phase, the amplitudes of each frequency component are relatively small, and the average frequency component value is low. During the mission execution phase, multiple frequency components are excited, and the average frequency component value increases significantly.

[0136] The average frequency component is calculated as follows:

[0137] Formula (13)

[0138] in, The average frequency component; This represents a weighted summation of all positive frequency components, with the weights being the frequency values. itself; This represents the total energy of all positive frequency components (ignoring the DC component). ).

[0139] Time-domain features reflect the direct changes of flight parameters over time, while frequency-domain features reveal hidden periodic patterns. Combining time-domain and frequency-domain features yields global statistical features.

[0140] The eight time-domain feature vectors and two frequency-domain feature vectors of each flight parameter time series data are concatenated to form the corresponding global statistical features.

[0141] Time-series dynamic data and global statistical features describe flight maneuvers from different perspectives, and the joint feature vector is used as the model input.

[0142] The purpose of step S2 is to extract time-domain and frequency-domain features from the flight parameter time-series data and concatenate these features into global statistical features to comprehensively describe flight actions and provide sample data for the human factors key stage automatic identification model.

[0143] Step S3 includes steps S31-S33.

[0144] Step S31: Construct an automatic identification model for key human factors stages.

[0145] like Figure 3 As shown, the automatic identification model for key human factors stages includes a time-series dynamic branch, a global statistical branch, and a feature fusion classification branch;

[0146] The time-series dynamic branch obtains a time-series dynamic feature vector based on the input time-series flight parameter data;

[0147] The global statistics branch, based on the input global statistical features, obtains a global statistical feature vector;

[0148] The feature fusion classification branch concatenates and fuses the temporal dynamic feature vector and the global statistical feature vector, and then inputs them into the intermediate fully connected layer for classification to obtain the human factor key stage identification result.

[0149] Based on the constructed human factor key stage automatic identification model, the human factor key stage is identified in flight parameter data. Through multiple rounds of convolution and pooling operations, local features are gradually extracted and abstracted. The human factor key stage automatic identification model is essentially a classification model.

[0150] The input data for the automatic identification model of human factors critical stages is flight parameter data, specifically including: flight parameter time series data and corresponding global statistical feature vectors; the output is the probability distribution of human factors critical stages, and the human factors critical stage corresponding to the input data is taken as the maximum probability value.

[0151] The temporal dynamic branch sequentially includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, an LSTM layer, a temporal Dropout layer, and a temporal fully connected layer;

[0152] The first convolutional layer uses 64 arrays of size 1. The convolution kernel is used to convolve the flight parameter time series data to obtain the first time series feature;

[0153] The first pooling layer uses max pooling, and the window size is [size missing]. With a step size of 2, the first time-series feature is reduced in dimensionality while retaining significant features to obtain the second time-series feature;

[0154] The second convolutional layer uses 128 arrays of size 128. The convolutional kernel captures higher-order combined features from the second temporal feature to obtain the third temporal feature;

[0155] The second pooling layer uses max pooling, with a window size of [size missing]. With a step size of 2, we further focus on the features of the key time nodes of the third time series feature to obtain the fourth time series feature.

[0156] The LSTM layer is used to obtain the 128-dimensional hidden state features of the last time step of the fourth temporal feature, and thus obtain the fifth temporal feature.

[0157] The temporal Dropout layer, based on the fifth temporal feature, randomly discards 50% of the neuron outputs to prevent overfitting, thus obtaining the sixth temporal feature;

[0158] The time-series fully connected layer is used to map the sixth time-series feature to a 64-dimensional vector and perform a ReLU activation nonlinear transformation to obtain the time-series dynamic feature vector.

[0159] The input samples for the time-series dynamic branch include flight data for 30 consecutive time steps, with each time step having 9 types of parameter features (time-series flight parameter data for flight altitude, flight speed, flight overload, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed).

[0160] The sample consists of 1 sample, 30 time steps, and 9 features.

[0161] Figure 3 The input feature shape is (300, 30, 9), where 300 represents a batch of 300 samples.

[0162] (1) First convolutional layer: 64 convolutional kernels of size 3 are used to extract basic local features of time-series dynamic data. The basic local feature dimension is 30.

[0163] Input shape: (1, 30, 9), Output shape: (1, 28, 64).

[0164] The time step calculation is 30 - 3 + 1 = 28; the number of features is 64, and each kernel extracts a local pattern.

[0165] The original 6 features are transformed into 64 local features over 28 time steps by using 64 convolutional kernels with 3 time steps.

[0166] (2) First pooling layer: The convolution result of the first convolutional layer is reduced in dimensionality, each window retains the maximum value, the time length is compressed from 30 to 15, and key features are enhanced.

[0167] Input shape: (1, 28, 64), Output shape: (1, 14, 64).

[0168] The time step calculation is 28 ÷ 2 = 14; the number of features is kept at 64. One maximum value is retained every two time steps to reduce the amount of data while focusing on key features.

[0169] (3) Second convolutional layer: 128 convolutional kernels are used to process the pooled features and extract more complex combined features.

[0170] Input shape: (1, 14, 64), Output shape: (1, 12, 128).

[0171] The time step calculation is 14 - 3 + 1 = 12; the feature number is 128.

[0172] Based on the 64 local features of the first convolutional layer, features of the learned features are studied, such as the combination of flight speed change features and roll angle change features.

[0173] (4) Second pooling layer: The convolution result of the second convolution layer is reduced in dimensionality again, and the temporal length is compressed from 15 to 7, focusing on key features for key action identification.

[0174] Input shape: (1, 12, 128), Output shape: (1, 6, 128).

[0175] The time step calculation is 12 ÷ 2 = 6; the number of features is kept at 128. The time dimension is further compressed, retaining 128 combinations of features from 6 key time steps.

[0176] (5) LSTM layer: Based on the local features output by the second pooling layer, it captures the long-range dependencies of time-series feature data.

[0177] Input shape: (1, 6, 128), Output shape: (1, 128).

[0178] In this process, the time step disappears, and only the result of the last time step is output; the number of features is 128. By learning the sequential relationship of the features of the 6 time steps (such as roll before heading change), a 128-dimensional vector is output to represent the temporal pattern of the entire flight parameter time series data sequence.

[0179] (6) Temporal Dropout layer: 50% of neurons are randomly dropped during training to prevent the model from over-relying on local features and improve generalization ability.

[0180] Input shape: (1, 128), Output shape: (1, 128).

[0181] During training, 50% of the feature values ​​are randomly set to 0.

[0182] (7) Temporal Fully Connected Layer: The fully connected layer integrates the features output by the LSTM layer to output a 64-dimensional abstract feature vector.

[0183] Input shape: (1, 128), Output shape: (1, 64).

[0184] The 128-dimensional features are compressed into a 64-dimensional temporal dynamic feature vector through a temporal fully connected layer, thus extracting the most critical temporal features.

[0185] The final features of the output temporal dynamic branch are used for subsequent fusion with the output feature vectors of the global statistical feature branch. The final output is a 64-dimensional temporal dynamic feature vector.

[0186] By combining multiple layers of convolution and pooling, local features ranging from simple to complex are extracted from the original flight parameter time series data. LSTM is used to capture the dependencies in the time dimension, and finally, abstract features that can represent the dynamic characteristics of flight actions are output, providing key input for subsequent classification.

[0187] (1, 30, 9) → Input

[0188] ↓

[0189] (1, 28, 64) → First convolutional layer (extracts local features)

[0190] ↓

[0191] (1, 14, 64) → First pooling layer (compressing the time dimension)

[0192] ↓

[0193] (1, 12, 128) → Second convolutional layer (extracts complex combined features)

[0194] ↓

[0195] (1, 6, 128) → Second pooling layer (compressed again)

[0196] ↓

[0197] (1, 128) → LSTM layer (encoding time-dependent)

[0198] ↓

[0199] (1, 128) → Temporal Dropout layer (to prevent overfitting)

[0200] ↓

[0201] (1, 64) → Temporal fully connected layer (feature compression)

[0202] ↓

[0203] (1, 64) → Temporal dynamic feature vector

[0204] The global statistics branch includes, in sequence, a global fully connected layer, a global Dropout layer, and a global output layer;

[0205] The global fully connected layer maps the global statistical features to 32 dimensions through ReLU activation nonlinear transformation, reduces the dimensionality, and learns the correlation between statistical features to obtain the first global feature.

[0206] The global Dropout layer randomly discards 30% of the neurons in the first global feature to enhance generalization ability and obtain the second global feature;

[0207] The global output layer compresses the second global feature to 16 dimensions, retains key statistical features, and obtains a global statistical feature vector.

[0208] We perform dimensionality reduction and abstraction on global statistical features to enable them to be efficiently integrated with the temporal features of dynamic branches of time series.

[0209] Input the mean, standard deviation, range, maximum and minimum values, median, sum of squares, mean absolute rate of change, maximum frequency component, and mean frequency component data of the flight parameters.

[0210] Input shape: (1, 90), representing 1 sample and 90 global statistical features, mainly calculated values ​​of 10 time-domain and frequency-domain features of 9 types of flight parameters.

[0211] (1) Global Fully Connected Layer: Maps the input global statistical features to a 32-dimensional space and learns the non-linear relationships between features through 32 neurons.

[0212] Input shape: (1, 90), output shape: (1, 32).

[0213] Figure 3 The input shape for the global statistical features is (300, 90), indicating that a batch of 300 samples is input.

[0214] The 90-dimensional global statistical features are extracted into 32 dimensions to capture the correlation between features.

[0215] (2) Global Dropout layer: 30% of neurons are randomly dropped during training to prevent the model from over-relying on certain specific features and improve generalization ability.

[0216] Input shape: (1, 32), Output shape: (1, 32).

[0217] During model training, 30% of the feature values ​​are randomly set to 0 to prevent the model from over-relying on certain specific features and to enhance its generalization ability.

[0218] (3) Global output layer: The fully connected layer compresses the features to 16 dimensions and extracts the key information. This output serves as the final feature vector of this branch and is fused with the features of the temporal dynamic branch.

[0219] Input shape: (1, 32), output shape: (1, 16).

[0220] The 32-dimensional features are compressed into 16-dimensional core features, retaining the most critical statistical information for flight motion recognition.

[0221] (1, 90) → Global statistical characteristics

[0222] ↓

[0223] (1, 32) → Global Fully Connected Layer (Feature Compression and Transformation)

[0224] ↓

[0225] (1, 32) → Global Dropout layer (randomly discards data to prevent overfitting)

[0226] ↓

[0227] (1, 16) → Global Output Layer (Extracting Core Statistical Features)

[0228] The feature fusion classification branch includes, in sequence, a feature fusion layer, an intermediate fully connected layer, and an output layer;

[0229] The feature fusion layer is used to concatenate a 64-dimensional temporal dynamic feature vector and a 16-dimensional global statistical feature vector to obtain an 80-dimensional first fused feature.

[0230] The intermediate fully connected layer is used to map the first fused feature to 32 dimensions through ReLU activation to obtain the second fused feature;

[0231] The output layer is used to output the probability distribution of 7 types of human factors key stages by activating the 32-dimensional second fusion feature through Softmax.

[0232] Feature fusion layer: The feature vectors output by the two different branches (temporal dynamic branch and global statistical branch) are concatenated together column by column using a tensor concatenation function to form the first fused feature.

[0233] The input consists of time-series flight parameter data and corresponding global statistical features, and the output is the first fused feature. The time-series flight parameter data is a 64-dimensional feature vector, and the global statistical features are 16-dimensional feature vectors; concatenating them results in an 80-dimensional fused feature vector. This achieves the synergistic effect of multi-dimensional features. The time-series flight parameter data and global statistical features describe flight maneuvers from different perspectives, and the fused first feature provides a more comprehensive feature representation.

[0234] Intermediate fully connected layer: Uses 32 neurons to perform a non-linear transformation on the first fused feature. Through this 32-neuron fully connected layer, the 80-dimensional first fused feature is compressed to 32 dimensions. Input shape: (1, 80), output shape: (1, 32). This extracts the most discriminative information from the first fused feature and introduces non-linearity through the ReLU activation function, enhancing the ability to express complex feature relationships.

[0235] Output layer: The final output layer of the human factors critical stage automatic identification model, representing the specific category of the human factors critical stage. Output shape: (m, 32), where m is the number of samples; Output shape: (probability of the human factors critical stage label sequence).

[0236] Step S32: Train the human factors key stage automatic identification model to obtain the trained human factors key stage automatic identification model.

[0237] The human factors critical stage automatic identification model is trained through the following process to obtain a well-trained human factors critical stage automatic identification model.

[0238] Acquire sample data including flight parameter time series data and corresponding global statistical features, and form a sample training set with the corresponding sample labels; wherein, the sample labels represent the corresponding different human factor key stages;

[0239] Initialize the learning rate and preset the maximum number of iterations;

[0240] Adam is selected as the optimizer, and the model parameters are iteratively optimized through forward and backward propagation until the classification cross-entropy loss function converges or the training stops after reaching the preset maximum number of iterations. The model parameters with the minimum loss are saved as the trained human factors key stage automatic recognition model.

[0241] The corresponding sample labels are shown in Table 1 in step S11.

[0242] The sample data and sample labels form a sample training set, which is used to train the human factors key stage automatic identification model.

[0243] The sample training set is divided into three parts: training set, validation set, and test set. The training set comprises 64%, the validation set 16%, and the test set 20%.

[0244] For example, the maximum number of iterations is 50; the training batch size is 300, meaning that 300 samples are used for training each time; and the initial learning rate is set to 0.001.

[0245] During training, Adam was selected as the optimizer. Adam automatically adjusts the learning rate of different parameters, which has a better optimization effect on high-dimensional features and can converge to a better solution faster.

[0246] Classification cross-entropy is used as the loss function to achieve classification for key stages of human behavior identification. The loss function measures the difference between the probability distribution of the model output, such as [0.05, 0.10, 0.20, 0.05, 0.35, 0.10, 0.15], and the true action labels, such as [0,1, 0,0,0,0,0]. A smaller loss value indicates a smaller deviation between the model's predictions and the actual actions.

[0247] The classification cross-entropy loss function is as follows:

[0248] Formula (14)

[0249] in, Let C be the cross-entropy loss function, where C is the total number of categories (7). It is the one-hot encoding of the real label. If the sample belongs to the i-th class, then... ,otherwise, ; To predict the probability that a sample belongs to class i for the model, satisfying... .

[0250] Monitor the loss value on the validation set to determine if model performance has improved. If the validation set loss does not decrease after 10 consecutive training epochs, early stopping is performed. After training stops, the model parameters are automatically restored to the state when the validation set loss was minimized.

[0251] Accuracy represents the proportion of samples correctly predicted by the model out of the total sample. A higher accuracy rate indicates a higher accuracy rate in recognizing human actions. This invention requires an accuracy rate of 90% or higher.

[0252] The human factors critical stage automatic identification model is trained on the server host configured on the flight simulator; incremental training with sample data is carried out; the trained human factors critical stage automatic identification model is periodically updated to the computer host of the flight simulator.

[0253] The identified critical human factors phases during flight can be used to identify the critical human factors phases throughout the entire flight process. Based on the identified critical human factors phases during flight, combined with physiological state monitoring data, physical and mental efficacy assessments can be carried out, laying the foundation for real-time assessment of physical and mental efficacy in the air.

[0254] Step S33: Combine the results of the identification of key human factors at critical stages with the heart rate obtained at the corresponding time to assess the pilot's flight stability.

[0255] Flight parameter data is preprocessed to obtain time-series flight parameter data, which, along with corresponding global statistical features, is input into a pre-trained automatic human factor critical phase identification model to obtain human factor critical phase identification results for accurate flight phase segmentation. The physiological and psychological characteristics of each flight phase are studied to assess the pilot's flight stability at each phase. Based on the automatically identified critical phases and the patterns of physiological data changes, targeted training is conducted to improve the pilot's training capabilities.

[0256] The pilot's flight stability is calculated as follows. It is a standardized heart rate deviation index. Its core function is to quantify the degree of deviation of the heart rate at a certain moment from the range of heart rate fluctuations throughout the monitoring period. It can be used to characterize the relative fluctuation of heart rate at a specific moment.

[0257] The average flight stability of the pilots during the seven phases of takeoff, climb, navigation, level flight, mission execution, return to base, and landing was recorded.

[0258] The mean flight stability is calculated as follows:

[0259] Formula (15)

[0260] in, This represents the average flight stability. The heart rate value at a specific moment corresponding to a certain flight parameter time series data; The mean heart rate of a certain flight parameter time series data; These represent the maximum and minimum values ​​of the heart rate values ​​corresponding to multiple data points in the time series data of a certain flight parameter.

[0261] absolute value The closer it is to 0, the closer the heart rate is to the average level of the entire cycle, the smaller the heart rate fluctuation, and the more stable the heart rate is.

[0262] absolute value The closer it is to 1, the more significantly the heart rate deviates from the mean at that moment and is close to the extreme value of heart rate fluctuation, which may indicate that the heart rate is under extreme load or in an abnormal state.

[0263] During takeoff, If the indicator fluctuates within a small range of ±0.2, it indicates that the heart rate changes stably around the mean, matches the rhythm of movement, and has high dynamic stability. If the indicator fluctuates significantly in a short period of time, beyond ±0.2, it indicates that the heart rate deviates drastically from the mean, reflecting disordered regulation of the body's load or unstable psychological stress, and poor dynamic stability.

[0264] As described above, the fluctuation threshold during the climbing phase is set to ±0.3;

[0265] The fluctuation threshold during the navigation phase is set to ±0.2;

[0266] The fluctuation threshold during level flight is set to ±0.2.

[0267] The task fluctuation threshold during the execution phase is set to ±0.6;

[0268] The fluctuation threshold during the return phase is set to ±0.3;

[0269] The landing phase fluctuation threshold is set to ±0.4.

[0270] Accurately dividing flight phases and studying the physiological and psychological characteristics of each phase allows for the assessment of pilots' flight stability at each stage. Based on automatically identified critical human factors phases and the patterns of heart rate changes using physiological data, targeted training is conducted to enhance pilots' training capabilities.

[0271] The purpose of step S3 is to build and train an automatic identification model for critical phases of flight. This model combines flight parameter time series data and global statistical features to identify critical phases of flight and uses these identification results in conjunction with the pilot's heart rate data to assess the pilot's flight stability at each phase of flight.

[0272] Example 2:

[0273] A specific embodiment of the present invention discloses an automatic identification system for critical human factors during simulated flight, thereby implementing the automatic identification method for critical human factors during simulated flight described in Embodiment 1. The specific implementation methods of each module are as described in the corresponding descriptions in Embodiment 1.

[0274] like Figure 4 As shown, an automatic identification system for key human factors during simulated flight includes: a data acquisition and preprocessing module M1, a feature extraction module M2, and a key human factors identification and evaluation module M3.

[0275] The data acquisition and preprocessing module M1 is used to acquire and preprocess various flight parameter data in the simulated flight to obtain preprocessed flight parameter time series data.

[0276] Feature extraction module M2 is used to extract the time-domain and frequency-domain features of the flight parameter time-series data and concatenate them into the corresponding global statistical features;

[0277] The human factor critical phase identification and evaluation module M3 is used to input the flight parameter time series data and the global statistical features into the trained human factor critical phase automatic identification model to obtain the human factor critical phase identification result, which is used to evaluate the pilot's flight stability in combination with the heart rate at the corresponding time.

[0278] Since the system in this embodiment and the method in Embodiment 1 are related and can be referenced from each other, this description is redundant and will not be repeated here. Because this system embodiment shares the same principle as the above method embodiment, it also possesses the corresponding technical effects of the above method embodiment.

[0279] In summary, the automatic identification method and system for key human factors during simulated flight according to embodiments of the present invention has the following beneficial effects:

[0280] 1. This invention identifies critical human factors stages through automated feature extraction and machine learning models, reducing reliance on expert experience and thus lowering subjectivity. By utilizing the time and frequency domain features of flight parameters, it can more accurately identify critical human factors stages during simulated flight, improving the objectivity and accuracy of critical human factor stage identification; the identified critical human factors stages are combined with the corresponding heart rate to assess the pilot's flight stability.

[0281] 2. This invention processes flight parameter data in real time, quickly extracts features, and automatically identifies key human factors at critical stages. This real-time capability enables the system to provide immediate feedback to pilots during flight missions, helping them adjust flight strategies and physiological and psychological states in a timely manner, thereby improving the efficiency of flight training and safety assessment, and enhancing real-time performance and feedback efficiency.

[0282] 3. This invention, by accurately identifying critical human factors phases in flight, can provide pilots with more targeted training, helping them better master key flight skills. Simultaneously, by assessing the pilot's flight stability during these critical phases, potential flight risks can be identified in a timely manner, thereby improving the accuracy of flight safety assessments and reducing the incidence of flight accidents.

[0283] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0284] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatic identification of key human factors during simulated flight, characterized in that, include: Acquire various flight parameter data from the simulated flight and preprocess them to obtain preprocessed flight parameter time series data; Extract the time-domain and frequency-domain features of the flight parameter time-series data and concatenate them into the corresponding global statistical features; The flight parameter time series data and the global statistical features are input into the trained human factor critical phase automatic identification model to obtain the human factor critical phase identification results, which are used to evaluate the pilot's flight stability in combination with the heart rate at the corresponding time. The automatic identification model for key human factors stages includes a time-series dynamic branch, a global statistical branch, and a feature fusion classification branch. The time-series dynamic branch obtains a time-series dynamic feature vector based on the input time-series flight parameter data; The global statistics branch, based on the input global statistical features, obtains a global statistical feature vector; The feature fusion classification branch concatenates and fuses the temporal dynamic feature vector and the global statistical feature vector, and then inputs them into the intermediate fully connected layer for classification to obtain the human factor key stage identification result.

2. The method for automatic identification of key human factors during simulated flight as described in claim 1, characterized in that, Multiple flight parameter data include flight altitude, flight speed, flight G-force, pitch angle, roll angle, heading angle, northward speed, skyward speed, and eastward speed.

3. The method for automatic identification of key human factors during simulated flight as described in claim 1, characterized in that, The time-domain features include mean, standard deviation, range, maximum and minimum values, median, sum of squares, and average absolute rate of change; the frequency-domain features include maximum frequency component and average frequency component features. The frequency domain features of the flight parameter time series data are extracted using Fast Fourier Transform.

4. The method for automatic identification of key human factors during simulated flight as described in claim 1, characterized in that, The temporal dynamic branch sequentially includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, an LSTM layer, a temporal Dropout layer, and a temporal fully connected layer; The first convolutional layer uses 64 arrays of size 1. The convolution kernel is used to convolve the flight parameter time series data to obtain the first time series feature; The first pooling layer uses max pooling, and the window size is [size missing]. With a step size of 2, the first time-series feature is reduced in dimensionality while retaining significant features to obtain the second time-series feature; The second convolutional layer uses 128 arrays of size 128. The convolutional kernel captures higher-order combined features from the second temporal feature to obtain the third temporal feature; The second pooling layer uses max pooling, with a window size of [size missing]. With a step size of 2, we further focus on the features of the key time nodes of the third time series feature to obtain the fourth time series feature. The LSTM layer is used to obtain the 128-dimensional hidden state features of the last time step of the fourth temporal feature, and thus obtain the fifth temporal feature. The temporal Dropout layer, based on the fifth temporal feature, randomly discards 50% of the neuron outputs to prevent overfitting, thus obtaining the sixth temporal feature; The time-series fully connected layer is used to map the sixth time-series feature to a 64-dimensional vector and perform a ReLU activation nonlinear transformation to obtain the time-series dynamic feature vector.

5. The method for automatic identification of key human factors during simulated flight as described in claim 4, characterized in that, The global statistics branch includes, in sequence, a global fully connected layer, a global Dropout layer, and a global output layer; The global fully connected layer maps the global statistical features to 32 dimensions through ReLU activation nonlinear transformation, reduces the dimensionality, and learns the correlation between statistical features to obtain the first global feature. The global Dropout layer randomly discards 30% of the neurons in the first global feature to enhance generalization ability and obtain the second global feature; The global output layer compresses the second global feature to 16 dimensions, retains key statistical features, and obtains a global statistical feature vector.

6. The method for automatic identification of key human factors during simulated flight as described in claim 4, characterized in that, The feature fusion classification branch includes, in sequence, a feature fusion layer, an intermediate fully connected layer, and an output layer; The feature fusion layer is used to concatenate a 64-dimensional temporal dynamic feature vector and a 16-dimensional global statistical feature vector to obtain an 80-dimensional first fused feature. The intermediate fully connected layer is used to map the first fused feature to 32 dimensions through ReLU activation to obtain the second fused feature; The output layer is used to output the probability distribution of 7 types of human factors key stages by activating the 32-dimensional second fusion feature through Softmax.

7. The method for automatic identification of key human factors during simulated flight according to any one of claims 1-6, characterized in that, The human factors critical stage automatic identification model is trained through the following process to obtain a well-trained human factors critical stage automatic identification model. Acquire sample data including flight parameter time series data and corresponding global statistical features, and form a sample training set with the corresponding sample labels; wherein, the sample labels represent the corresponding different human factor key stages; Initialize the learning rate and preset the maximum number of iterations; Adam is selected as the optimizer, and the model parameters are iteratively optimized through forward and backward propagation until the classification cross-entropy loss function converges or the training stops after reaching the preset maximum number of iterations. The model parameters with the minimum loss are saved as the trained human factors key stage automatic recognition model.

8. The method for automatic identification of key human factors during simulated flight as described in claim 7, characterized in that, The key human factors phases include takeoff, climb, navigation, level flight, mission execution, return to base, and landing; The key human factors stages are converted into corresponding integers using LabelEncoder, which serve as sample labels.

9. An automatic identification system for key human factors during simulated flight, characterized in that, It includes a data acquisition and preprocessing module M1, a feature extraction module M2, and a human factors key stage identification and evaluation module M3; The data acquisition and preprocessing module M1 is used to acquire and preprocess various flight parameter data in the simulated flight to obtain preprocessed flight parameter time series data. Feature extraction module M2 is used to extract the time-domain and frequency-domain features of the flight parameter time-series data and concatenate them into the corresponding global statistical features; The human factor critical phase identification and evaluation module M3 is used to input the flight parameter time series data and the global statistical features into the trained human factor critical phase automatic identification model to obtain the human factor critical phase identification result, which is used to evaluate the pilot's flight stability in combination with the heart rate at the corresponding moment. The automatic identification model for key human factors stages includes a time-series dynamic branch, a global statistical branch, and a feature fusion classification branch. The time-series dynamic branch obtains a time-series dynamic feature vector based on the input time-series flight parameter data; The global statistics branch, based on the input global statistical features, obtains a global statistical feature vector; The feature fusion classification branch concatenates and fuses the temporal dynamic feature vector and the global statistical feature vector, and then inputs them into the intermediate fully connected layer for classification to obtain the human factor key stage identification result.

Images