A method for acquiring psychological stress data of a flight student based on a skin electrical signal

By using deep convolutional autoencoders and parallel neural networks to process electrodermal signals, the problems of motion artifacts and noise in flight trainee simulation training were solved, achieving high-precision psychological state assessment and feature extraction, and improving the effectiveness of flight training.

CN116421186BActive Publication Date: 2026-02-17HARBIN INST OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310235497.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-13
Publication Date
2026-02-17
Estimated Expiration
2043-03-13

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively remove motion artifacts from electrodermal signals during flight training for student pilots, resulting in insufficient extraction of electrodermal signal features. This affects the accuracy and generalization ability of psychological state recognition. Furthermore, existing algorithms are ineffective in handling noise and motion artifacts in electrodermal signals, leading to low accuracy in psychological state assessment.

Method used

A deep convolutional autoencoder is used to remove motion artifacts from the electrodermal signal. Combined with frequency domain, time domain and nonlinear feature extraction, and feature fusion is performed using a parallel neural network. Psychological stress data of flight trainees is obtained by training the model, including a combination of feature detection network and relational network. Deep convolutional neural network and LSTM network are used for feature extraction and integration.

Benefits of technology

It achieves efficient removal of motion artifacts in flight trainee simulation training, improves the accuracy of extracting electrodermal signal features and the accuracy of psychological state assessment, and can better reflect the fluctuations in the psychological state of flight trainees, providing targeted training guidance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116421186B_ABST
    Figure CN116421186B_ABST
Patent Text Reader

Abstract

The application discloses a kind of based on skin electric signal's flight student psychological stress data acquisition method, it is related to machine learning technical field.The technical points of the present application include: collecting the skin electric signal of multiple flight students when flight simulation training, as physiological signal data set;The motion artifact removal is carried out to the skin electric signal collected;The skin electric signal after removing artifact is carried out frequency domain, time domain and nonlinear feature extraction, forms skin electric signal feature set;According to skin electric signal feature set, train psychological stress data acquisition model based on parallel neural network;The skin electric signal of new flight student is input in the psychological stress data acquisition model trained when flight simulation training, and the psychological stress data of new flight student is acquired.The present application can accurately express the fluctuation of psychological state of flight student when carrying out simulation flight training, help flight student summarizes flight experience, and provides help for subsequent flight training of flight student.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a method for acquiring psychological stress data of flight trainees based on electrodermal signals. Background Technology

[0002] Flight simulation training is a crucial guarantee for ensuring pilot safety. Flight trainees endure immense psychological pressure during real flight training, facing significant challenges to their willpower and flying abilities. Their psychological state directly impacts mission completion and can even determine their lives. Combining virtual reality technology with high-dynamic simulation platforms effectively improves flight trainees' psychological resilience. Therefore, assessing psychological fluctuations during simulated flight training, presenting these fluctuations, and analyzing the varying degrees of psychological changes experienced when facing flight difficulties and challenges allows for targeted evaluation of their psychological qualities, providing guidance for subsequent training missions.

[0003] Electrodermal signals, as fundamental physiological signals of the human body, reflect the activity of the sympathetic nervous system. Their intensity varies with fluctuations in a person's psychological state, thus serving as an indicator of such fluctuations. The skin resistance on the palm surface provides the most direct feedback on psychological state, and its high density of sweat glands makes it highly sensitive, making it a common location in psychophysiological studies.

[0004] In early studies, researchers monitored changes in skin conductivity by recording the electrodermal response on areas such as the forehead and palms of the human body. This provided a more direct view of how the sympathetic nervous system regulates skin conductivity when a person experiences changes in their psychological state, such as tension or happiness.

[0005] Currently, there are many research results on emotion recognition based on dimensional emotion models using electrodermal signals (EDS). For example, multilayer perceptual mechanisms are used to distinguish different psychological states based on arousal, mediativity, and dominance; nonlinear characteristics of EDS and pulse signals, such as Lyapunov index and energy entropy, are used to classify human psychological states based on probabilistic neural networks; multidimensional psychological state studies using EDS algorithms such as decision trees and random forests are employed; other methods, such as K-nearest neighbors and support vector machines, have also been used to classify psychological states based on EDS. However, these studies have some problems: the classification of psychological states based on EDS is limited in variety and accuracy; the extraction of EDS features is insufficient and lacks representativeness; the neural network training results lack generalization ability and exhibit significant individual differences, failing to achieve the expected high accuracy; and an emotion assessment model based on valence-arousal emotion scoring patterns and EDS responses is obtained using curve fitting theory and EDS decomposition methods.

[0006] Current research on mapping psychological states using electrodermal (EDS) signals based on neural networks mainly employs methods such as support vector machines, random forests, Bayesian networks, and fuzzy logic. These methods automatically distinguish human psychological states through various features obtained in the preprocessing stage. However, the challenges lie in the limited variety of stimulus sources, the simplistic experimental methods for inducing psychological fluctuations, and the need for strong stress responses from the human body during model construction. The difficulty also stems from the generally high noise levels in EDS signals, necessitating filtering, and the fact that human activity often introduces motion trajectories, which are clearly reflected in EDS signals, making feature extraction challenging.

[0007] Due to the weak and low-frequency characteristics of EDS signals, the acquisition process is significantly affected by motion artifacts. Furthermore, during simulated flight training, student pilots interact with the virtual environment using joysticks and manual throttles; the intense muscle activity during training is a major reason why motion artifacts are difficult to correct. At the algorithmic level, one challenge in removing motion artifacts lies in the lack of pure EDS signals, making it impossible to compare signals containing motion artifacts with neural network training. Secondly, when using filtering methods for artifact removal, a reference signal needs to be constructed. Since the generation of artifacts is unpredictable, it is impossible to find a reasonable reference signal to describe the state of motion artifacts in the EDS signal.

[0008] For example, adaptive filters correct motion artifacts, but their drawback is that they require constructing a reference signal as the filter input, making it difficult to remove motion artifacts using time-spectrum methods, interpolation methods, etc. For example, wavelet decomposition, when correcting motion artifacts, requires filtering out noise in the electrodermal signal in stages, setting reasonable wavelet bases and thresholds, and removing artifact signals. Its drawback is that it is difficult to locate the noise in the electrodermal signal. Variational mode decomposition-adaptive entropy thresholding can remove motion artifacts, but it requires solving and sorting the energy entropy in stages, locating the modal components in different frequency bands, and the motion artifact removal procedure is complex. Summary of the Invention

[0009] Therefore, this invention proposes a method for acquiring psychological stress data of flight trainees based on electrodermal signals, in an attempt to solve or at least alleviate at least one of the above-mentioned problems.

[0010] A method for acquiring psychological stress data of flight trainees based on electrodermal signals includes the following steps:

[0011] S1: Collect skin electrodermal signals from multiple flight trainees during flight simulation training to create a physiological signal dataset;

[0012] S2: Motion artifact removal from the collected electrodermal signals;

[0013] S3: Extract frequency domain, time domain and nonlinear features from the artifact-removed electrodermal signal to form an electrodermal signal feature set;

[0014] S4: Train a psychological stress data acquisition model based on a parallel neural network using skin electrodermal signal feature sets;

[0015] S5: After removing motion artifacts and extracting features from the skin conduction signals of new flight trainees during flight simulation training, the data is input into the trained psychological stress data acquisition model to obtain the psychological stress data of the new flight trainees.

[0016] Furthermore, the skin electrical signals acquired in S1 specifically include:

[0017] Electrodermal signals of the distal skin of the index and middle fingers at the joystick control end were collected before and during flight simulation training.

[0018] Electrical signals from the distal skin of the index and middle fingers at the manual throttle control input terminals were collected before and during flight simulation training.

[0019] Furthermore, the flight simulation training described in S1 includes multiple flight training mission scenarios, specifically including: daily flight skills training scenarios created using virtual reality technology, flight training scenarios in severe weather, formation flight training scenarios, and enemy aircraft shooting training scenarios.

[0020] Furthermore, in S2, a deep convolutional neural network is used to remove motion artifacts from the acquired electrodermal signals, including the following steps:

[0021] Reduce the sampling frequency of the EDS signal; segment the EDS signal into non-overlapping segments; input the segmented data into a deep convolutional autoencoder for motion artifact removal.

[0022] Furthermore, the deep convolutional autoencoder described in S2 includes an encoder and a decoder. The encoder consists of three convolutional blocks, each consisting of a convolutional layer, a pooling layer, a Leaky-ReLU activation function, and a batch normalization layer. The decoder consists of three deconvolutional blocks. The first two deconvolutional blocks consist of a deconvolutional layer, a pooling layer, an optimized randomized Leaky-ReLU activation function, and a batch normalization layer, while the last deconvolutional block consists of only a deconvolutional layer.

[0023] The expression for the loss function of the depthwise convolutional autoencoder is:

[0024]

[0025] Where, x i This represents the i-th original skin electrical signal. Let represent the i-th EDS signal after artifact removal, and n represent the total number of EDS signals. The first part of the loss function is the mean square error between the EDS signal and the EDS signal after artifact removal, and the second part is the L2 regularization function added to prevent overfitting. λ represents the coefficient, which takes values ​​between [0, 0.01]. A regular function representing the electrodermal signal.

[0026] Furthermore, the expression for optimizing the stochastic Leaky-ReLU activation function described in S2 is:

[0027]

[0028] Where, x i This indicates the input skin electrical signal. u represents a uniformly distributed random number, and u has a supremum, i.e., 0. <u≤1。

[0029] Furthermore, the mid-frequency domain features of S3 include: mean, median, standard deviation, root mean square, maximum value, minimum value, ratio of maximum value to minimum value, range, average, median, standard deviation, maximum value, minimum value, ratio of maximum value to minimum value, difference, mean of first-order difference, median of first-order difference, standard deviation of first-order difference, maximum value of first-order difference, minimum value of first-order difference, ratio of maximum value of first-order difference, ratio of minimum value of first-order difference; mean of second-order difference, median of second-order difference, standard deviation of second-order difference, maximum value of second-order difference, minimum value of second-order difference, ratio of maximum value of second-order difference, ratio of minimum value of second-order difference;

[0030] Time-domain features include mean, median, standard deviation, root mean square, maximum value, minimum value, ratio of maximum value to minimum value, and range;

[0031] Nonlinear feature extraction includes wavelet decomposition of the electrodermal signal and calculation of energy entropy in different frequency bands as nonlinear features of the electrodermal signal.

[0032] Furthermore, the parallel neural network described in S4 includes a feature detection network and a relational network. The feature detection network is used for local feature extraction of the electrodermal signal, and the relational network is used for feature integration of segmented electrodermal signals.

[0033] The feature detection network comprises two convolutional modules and one weight allocation module. The convolutional modules include a one-dimensional convolutional layer, a batch normalization layer, and a Leaky-ReLU activation function.

[0034] The relational network comprises two cascaded LSTM-based attention layers that generate EDS signal features and perform a weighted summation operation to obtain the weight coefficients of key EDS signal features. The attention layer comprises three parallel bidirectional LSTM networks and two Softmax layers. To achieve the function of the attention layer, the input segmented EDS signal is treated as a key-value pair. Based on the given Query and Key matrices in the feature weight allocation, the Key and Query matrices of the bidirectional LSTM network are multiplied and output to the Softmax layer to obtain the similarity coefficients of the two matrices. These similarity coefficients are then multiplied with the Value matrix of the bidirectional LSTM network and output to the Softmax layer to obtain the corresponding weight coefficients, i.e., the attention layer weights.

[0035] Its output signal is:

[0036]

[0037] Here, Softmax is the probability distribution function, and Query, Key, and Value are different weight matrices of the bidirectional LSTM network.

[0038] Furthermore, the structure of the psychological stress data acquisition model based on parallel neural networks described in S4 also includes: a cascaded layer and a softmax layer connected in series after the parallel neural network. The cascaded layer is used to fuse the key features of the skin conductance signal obtained from the feature extraction network and the relational network. The softmax layer is used to classify the model after feature fusion to obtain different psychological stress data acquisition models.

[0039] Furthermore, the expression for the loss function of the parallel neural network is:

[0040]

[0041] Among them, y j and p j λ1 and λ2 are the training results and prediction results of the j-th layer, respectively; λ3 and λ2 are the regularization coefficients; N is the classification result of all outputs.

[0042] The beneficial technical effects of this invention are:

[0043] This invention proposes a method for acquiring psychological stress data of flight trainees based on electrodermal signals. The method involves collecting electrodermal signals from flight trainees during simulated flight training in a virtual reality environment. Motion artifacts in the electrodermal signals are removed using a data-driven deep convolutional autoencoder, and frequency and time domain features are extracted. An improved parallel feature detection and correlation neural network is then used to fuse these features. The resulting model is trained to acquire psychological stress data from flight trainees, which can be used later to assess fluctuations in their psychological state during flight simulation training. Attached Figure Description

[0044] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, which together with the following detailed description are included in and form part of this specification, and are used to further illustrate preferred embodiments of the invention and explain the principles and advantages of the invention.

[0045] Figure 1 This is a flowchart of a method for acquiring psychological stress data of flight trainees based on electrodermal signals, according to an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the structure of the psychological stress data acquisition model based on parallel neural networks in an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present invention, exemplary embodiments or examples of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments or examples are merely some, not all, of the embodiments or examples of the present invention. All other embodiments or examples obtained by those skilled in the art based on the embodiments or examples of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0048] Based on the specific scenarios of simulated flight training for flight cadets, it is believed that the psychological stress of flight cadets is extremely important for the smooth and safe execution of flight missions. Therefore, it is necessary to specifically collect electrodermal signals for feature extraction and evaluation to form a psychological fluctuation assessment model for flight cadets. Thus, this invention proposes a method for acquiring psychological stress data of flight cadets based on electrodermal signals, such as... Figure 1 As shown, the method includes the following steps:

[0049] S1: Collect skin electrodermal signals from multiple flight trainees during flight simulation training to create a physiological signal dataset;

[0050] S2: Motion artifact removal from the collected electrodermal signals;

[0051] S3: Extract frequency domain, time domain and nonlinear features from the artifact-removed electrodermal signal to form an electrodermal signal feature set;

[0052] S4: Train a psychological stress data acquisition model based on a parallel neural network using skin electrodermal signal feature sets;

[0053] S5: After removing motion artifacts and extracting features from the skin conduction signals of new flight trainees during flight simulation training, the data is input into the trained psychological stress data acquisition model to obtain the psychological stress data of the new flight trainees.

[0054] In S1, flight simulation training can be conducted by flight trainees wearing VR glasses in a virtual reality environment. The duration is between 10 and 20 minutes, depending on the training scenario. The virtual reality scenarios that flight trainees can choose from include: normal flight training in clear weather, formation flight training, flight training in bad weather, simulated combat training for multiple flight trainees, and shooting down enemy aircraft training, etc.

[0055] The electrodermal (ED) signals were collected from the distal dermal points of the index and middle fingers at the joystick control end and the distal dermal points of the index and middle fingers at the manual throttle control end. Specifically, the collected EED signals included: the distal dermal signals of the index and middle fingers at the joystick control end 5 minutes before and during flight simulation training; and the distal dermal signals of the index and middle fingers at the manual throttle control input end 5 minutes before and during flight simulation training.

[0056] In S2, motion artifact removal is required for the acquired electrodermal signals. The specific steps are as follows:

[0057] The EKS signal was downsampled, with the sampling frequency reduced from 200Hz to 10Hz. The downsampled EKS signal was then segmented into 9-second non-overlapping segments. If the segments could not be divided equally, the last ten seconds of data could be unequally divided into non-overlapping segments of more than 6 seconds. After grouping the EKS signal data, a deep convolutional autoencoder was used to remove motion artifacts from the EKS signal.

[0058] The deep convolutional autoencoder consists of an encoder and a decoder. The encoder comprises three convolutional blocks, each consisting of a convolutional layer, a pooling layer, an optimized randomized LeakyReLU activation function, and a batch normalization layer. The stride of the convolutional layer is 3, and the pooling layer reduces the dimensionality of the input data by a factor of 2. The outer pooling layer uses max pooling to downsample the segmented EEG signal, reducing the feature dimension of the EEG signal. The decoder comprises three deconvolutional blocks. The first two deconvolutional blocks consist of a deconvolutional layer, a pooling layer, an optimized randomized LeakyReLU activation function, and a batch normalization layer. The stride of the deconvolutional layer is 3, and the pooling layer expands the dimensionality of the input data by a factor of 2. The last deconvolutional block does not have a pooling layer, activation function, or batch normalization layer. Feature information is transferred between the convolutional blocks of the encoder and the symmetrical convolutional blocks of the decoder using skip connections to ensure the integrity of the reconstructed signal.

[0059] The expression for optimizing the random Leaky-ReLU activation function in a deep convolutional autoencoder is as follows:

[0060]

[0061] Where, x i This indicates the input skin electrical signal. u represents a uniformly distributed random number, and u has a supremum, i.e., 0. <u≤1。

[0062] The expression for the loss function of a depthwise convolutional autoencoder is:

[0063]

[0064] Where x i This represents the i-th original skin electrical signal. The first part of the loss function for the i-th EDS signal after artifact removal is the mean square error between the EDS signal and the EDS signal after artifact removal, and the second part is the L2 regularization function added to prevent overfitting. λ represents the regularization function of the skin electrodermal signal; λ represents the coefficient, which takes the value between [0, 0.01], and in this embodiment, the value is 0.001.

[0065] Based on the above steps, the skin electrodermal signal dataset for flight trainees in simulated flight training was constructed.

[0066] In S3, the frequency domain features of the electrodermal signal after artifact correction are extracted using fast Fourier transform to obtain statistical features such as mean, median, standard deviation, root mean square deviation, maximum value, minimum value, maximum value ratio, minimum value ratio, and range in the frequency domain of the electrodermal signal.

[0067] Temporal features were extracted from the electrodermal signal after artifact correction, including mean, median, standard deviation, maximum, minimum, maximum ratio, minimum ratio, and difference; mean, median, standard deviation, maximum, minimum, maximum ratio, and minimum ratio of the first-order difference; and mean, median, standard deviation, maximum, minimum, maximum ratio, and minimum ratio of the second-order difference; and mean, median, standard deviation, maximum, minimum, maximum ratio, and minimum ratio of the second-order difference.

[0068] Nonlinear feature extraction was performed on the electrodermal signal after artifact correction, including four-level wavelet decomposition, and the energy entropy of different frequency bands was calculated as the nonlinear feature of the electrodermal signal.

[0069] Based on the above steps, the time-domain, frequency-domain, and nonlinear features of the artifact-corrected electrodermal signal were extracted, and a feature set of electrodermal signals in flight student simulated flight training was constructed.

[0070] In S4, a psychological stress data acquisition model based on a parallel neural network is trained using a skin electrodermal signal feature set; such as... Figure 2 As shown, the parallel neural network includes a feature detection network and a relational network. The feature extraction network is used to extract local features of the electrodermal signal. Its significance lies in extracting as many meaningful features as possible from each segment of the electrodermal signal after segmentation, not limited to time domain, frequency domain and nonlinear features. The relational network is used to integrate the features of the segmented electrodermal signals to obtain the relationship between the electrodermal signal features and the fluctuation of the pilot's psychological state during simulated flight training.

[0071] The feature detection network consists of two convolutional layers and one weight allocation layer. The convolutional layers include a one-dimensional convolutional layer, a batch normalization layer, and a Leaky ReLU activation function. The one-dimensional convolutional layer is used to deeply mine local features of the EDS signal, the batch normalization layer is used to prevent overfitting within the network, which would lead to low learning efficiency, and the Leaky ReLU activation function is used to prevent the loss of effective features of the EDS signal.

[0072] The relational network consists of two cascaded LSTM-based attention layers, three parallel bidirectional LSTM networks, and two Softmax layers. It performs weighted summation and other operations on the features of the electrodermal signal to obtain the weight coefficients of key features, constructs the connections between complex features of the electrodermal signal, deepens the key feature information of the electrodermal signal, and identifies which features in the input electrodermal signal have a strong indicative role in psychological fluctuations and which features are irrelevant to generating a psychological fluctuation model.

[0073] To achieve the function of the attention layer, the input segmented electrodermal signals are treated as key-value pairs. Based on the given Query and Key matrices in the feature weight allocation, the Key and Query matrices of the bidirectional LSTM network are multiplied and output to the Softmax layer to obtain the similarity coefficient of the two matrices. Then, the similarity coefficient is multiplied with the Value matrix of the bidirectional LSTM network and output to the Softmax layer to obtain the corresponding weight coefficients, which are the attention layer weights.

[0074] Its output signal is:

[0075] O LSTM =Softmax(Softmax(Query·Key) T Value)

[0076] Where is the output of the attention layer, Softmax is the probability distribution function, and Query, Key, and Value are different weight matrices of the bidirectional LSTM network.

[0077] Furthermore, in order to realize the function of the neural network, a cascaded layer and a softmax layer need to be connected in series after the two parallel neural networks. The role of the cascaded layer is to fuse the key features of the skin electrodermal signal obtained by the feature extraction network and the relational network. The role of the softmax layer is to classify the model after feature fusion to obtain different psychological fluctuation assessment models and output the psychological fluctuation assessment results.

[0078] The loss function of a neural network is defined as follows:

[0079]

[0080] Where y j and p j λ1 and λ2 are the training results and prediction results of the j-th layer, respectively; λ2 and λ3 are the regularization coefficients; N is the classification result of all outputs.

[0081] This invention removes artifacts from the original electrodermal signal dataset in flight training simulations to obtain an electrodermal signal dataset with easily extractable features. It also extracts the time-domain, frequency-domain, and nonlinear features of the electrodermal signals to form a physiological signal feature dataset. This dataset is then input into a parallel neural network, and through parallel feature extraction neural networks and correlation neural networks, it outputs to cascaded layers and a Softmax layer to obtain the corresponding psychological fluctuation assessment level.

[0082] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A method for acquiring psychological stress data of flight trainees based on electrodermal signals, characterized in that, Includes the following steps: S1: Collect skin electrodermal signals from multiple flight trainees during flight simulation training to create a physiological signal dataset; S2: Use a deep convolutional neural network to remove motion artifacts from the acquired electrodermal signals; The method includes the following steps: reducing the sampling frequency of the electrodermal signal; segmenting the electrodermal signal into non-overlapping segmented data; inputting the segmented data into a deep convolutional autoencoder for motion artifact removal; the deep convolutional autoencoder includes an encoder and a decoder, wherein the encoder consists of 3 convolutional blocks, each convolutional block consisting of a convolutional layer, a pooling layer, a Leaky-ReLU activation function, and a batch normalization layer; the decoder consists of 3 deconvolutional blocks, the first 2 deconvolutional blocks consisting of a deconvolutional layer, a pooling layer, an optimized randomized Leaky-ReLU activation function, and a batch normalization layer, and the last deconvolutional block consisting only of a deconvolutional layer; The expression for the loss function of the depthwise convolutional autoencoder is: ; in, This represents the i-th original skin electrical signal. represents the i-th EK signal after motion artifact removal, and n represents the total number of EK signals; the first part of the loss function of the deep convolutional autoencoder is the mean square error between the EK signal and the EK signal after motion artifact removal, and the second part is the L2 regularization function added to prevent overfitting; Represents the coefficient, whose value is in between; A regularization function representing the skin electrical signal; The expression for the optimized random Leaky-ReLU activation function is: ; in, This indicates the input skin electrical signal. Let u represent a uniformly distributed random number, and u has a supremum, i.e. ; S3: Extract frequency domain, time domain and nonlinear features from the electrodermal signal after removing motion artifacts to form an electrodermal signal feature set; S4: Train a psychological stress data acquisition model based on a parallel neural network using the feature set of electrodermal signals; the parallel neural network includes a feature detection network and a relational network. The feature detection network is used for local feature extraction of electrodermal signals, and the relational network is used for feature integration of segmented electrodermal signals; wherein, the feature detection network contains two convolutional modules and one weight allocation module, the convolutional modules contain one-dimensional convolutional layers, batch normalization layers, and Leaky-ReLU activation functions; the relational network contains two cascaded LSTM-based attention layers, which generate electrodermal signal features and perform weighted summation to obtain the weight coefficients of key features of electrodermal signals; the attention layer contains three parallel bidirectional LSTM networks and two softmax layers; the input segmented electrodermal signals are treated as key-value pairs, and according to the given query and key matrices in the feature weight allocation, the key matrix and query matrix of the bidirectional LSTM network are multiplied and output to the softmax layer to obtain the similarity coefficient of the two matrices, which is then multiplied with the value matrix of the bidirectional LSTM network and output to the softmax layer to obtain the corresponding weight coefficients, i.e., the weights of the attention layer; its output signal is: ; Where Softmax is the probability distribution function, and Query, Key, and Value are different weight matrices of the bidirectional LSTM network; S5: After removing motion artifacts and extracting features from the skin conduction signals of new flight trainees during flight simulation training, the data is input into the trained psychological stress data acquisition model to obtain the psychological stress data of the new flight trainees.

2. The method for acquiring psychological stress data of flight trainees based on electrodermal signals according to claim 1, characterized in that, The skin electrodermal signals collected in S1 specifically include: Electrodermal signals of the distal skin of the index and middle fingers at the joystick control end were collected before and during flight simulation training. Electrical signals from the distal skin of the index and middle fingers at the manual throttle control input terminals were collected before and during flight simulation training.

3. The method for acquiring psychological stress data of flight trainees based on electrodermal signals according to claim 2, characterized in that, The flight simulation training described in S1 includes multiple flight training mission scenarios, specifically including: daily flight skills training scenarios created using virtual reality technology, flight training scenarios in severe weather, formation flight training scenarios, and enemy aircraft shooting training scenarios.

4. The method for acquiring psychological stress data of flight trainees based on electrodermal signals according to claim 1, characterized in that, S3 mid-frequency domain features include: mean, median, standard deviation, maximum value, minimum value, ratio of maximum value to minimum value, range, average value, median, standard deviation, maximum value, minimum value, ratio of maximum value to minimum value, difference, mean of first-order difference, median of first-order difference, standard deviation of first-order difference, maximum value of first-order difference, minimum value of first-order difference, ratio of maximum value of first-order difference, ratio of minimum value of first-order difference; mean of second-order difference, median of second-order difference, standard deviation of second-order difference, maximum value of second-order difference, minimum value of second-order difference, ratio of maximum value of second-order difference, ratio of minimum value of second-order difference. Time-domain features include mean, median, standard deviation, maximum value, minimum value, ratio of maximum value to minimum value, and range; Nonlinear feature extraction includes wavelet decomposition of the electrodermal signal and calculation of energy entropy in different frequency bands as nonlinear features of the electrodermal signal.

5. The method for acquiring psychological stress data of flight trainees based on electrodermal signals according to claim 1, characterized in that, The structure of the psychological stress data acquisition model based on parallel neural networks in S4 also includes: a cascaded layer and a softmax layer connected in series after the parallel neural network. The cascaded layer is used to fuse the key features of the skin electrophoresis signal obtained from the feature extraction network and the relational network. The softmax layer is used to classify the model after feature fusion to obtain different psychological stress data acquisition models.

6. The method for acquiring psychological stress data of flight trainees based on electrodermal signals according to claim 1, characterized in that, The expression for the loss function of the parallel neural network is: ; in, and These are the training results and prediction results of the j-th layer, respectively; and is the regularization coefficient; N is the classification result of all outputs.

Citation Information

Patent Citations

  • Psychological stress evaluation grading method based on improved genetic algorithm

    CN111150411A

  • Psychological stress prediction method and system based on multi-channel physiological data

    CN111513730A