Pilot psychological pressure prediction and early warning method
By utilizing self-supervised learning and spatiotemporal feature networks and taking advantage of the temporal dependence of pilots' physiological signals, a psychological stress prediction model was constructed. This solved the problem of predicting pilots' psychological stress in the next moment, achieving accurate prediction of psychological stress in the next moment and improving aviation flight safety.
Patent Information
- Application Number
- CN202510975721.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-15
- Publication Date
- 2025-11-21
AI Technical Summary
Existing technologies cannot effectively predict the psychological stress state of pilots in the next moment, leading to a decline in aviation flight safety performance. Furthermore, the sparse labeling of physiological signal data makes it difficult for supervised learning to achieve accurate prediction.
By employing a self-supervised learning approach, a spatiotemporal feature network is constructed. This model utilizes the temporal dependence of prior feature space and physiological signals, combined with self-supervised training and fine-tuning, to build a psychological stress prediction model that can predict the psychological stress state at the next moment.
It effectively avoids the model training difficulties caused by sparse data annotation, achieves accurate prediction of the pilot's psychological stress in the next moment, and improves aviation flight safety performance.
Smart Images
Figure CN120983037A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of physiological signal prediction, and more particularly to a method for predicting and warning of psychological stress in pilots. Background Technology
[0002] In civil aviation flight safety, pilot psychological stress can lead to distraction, decreased task processing ability, and prolonged periods of severe psychological stress, even loss of situational awareness, resulting in a decline in flight safety performance. Utilizing physiological signal data to predict pilot psychological stress levels during flight missions allows for timely detection and intervention of negative pilot conditions, further improving flight safety performance and ensuring flight safety.
[0003] Traditional methods for predicting psychological stress based on physiological signals map a segment of physiological signal to a label representing the psychological stress state at the current time. Kaur proposed a method for predicting psychological stress levels using wearable devices to collect signals in driving scenarios, while Gupta et al. used multi-source physiological signals to predict the psychological stress state of the human body. However, in practical applications, by the time these methods detect a severe level of psychological stress, the pilot is often already in that state and can only make remedial adjustments. Therefore, predicting the psychological stress state at the next moment is a real requirement for flight safety.
[0004] Predicting the psychological stress state in the next moment is essentially using Physiological signals at time predicted The psychological stress state at any given moment maps a physiological signal, invisible to the human eye, to a psychological stress state label over a subsequent time period. Unfortunately, current psychological stress data collection uses a "stimulation + labeling" method. Because psychological stress has a subjective element, data labeling requires subjects to self-label after continuous experimentation. Often, an entire physiological signal segment only has one overall label, resulting in sparse labels. This leads to a situation where, in supervised learning, the vast majority of the segmented dataset... and Equivalent to directly learning a model for predicting the next moment using supervised learning, the training results are indistinguishable from the recognition results. These challenging realities raise key issues for predicting pilots' psychological stress in the next moment: 1) Existing research lacks methods for early warning of pilots' psychological stress in the next moment, failing to meet the actual needs of aviation flight safety; 2) Limited by the paradigm of psychological stress data collection, the collected physiological signal data has sparse labeling characteristics, making it difficult to directly predict psychological stress in the next moment using supervised learning. Summary of the Invention
[0005] The present application aims at the actual demand of aviation flight safety and the problem that the model is difficult to train due to the sparsity of label system, and proposes a pilot psychological stress prediction and early warning method to solve the problem of difficult supervised learning prediction model under label sparsity, realize the prediction of next moment psychological stress, and then early warning, guarantee aviation flight safety.
[0006] The technical scheme adopted by the present application is that the pilot psychological stress prediction and early warning method comprises the following steps:
[0007] Step 1, obtain a psychological stress recognition data set, which includes psychological signals and psychological stress states;
[0008] The psychological stress state recognition model including the first feature extractor and the classifier is trained, and the trained first feature extractor is saved; wherein the input of the classifier is the first feature vector of the physiological signal output by the first feature extractor; the classifier is used for predicting the psychological stress state of the physiological signal;
[0009] Step 2, obtain a physiological signal data set of psychological stress, and build a space-time feature network including a second feature extractor and a first prediction decoder, wherein the second feature extractor is used for extracting a second feature vector of the physiological signal data, and the first prediction decoder is used for predicting the physiological signal data at the next moment based on the input second feature vector of the physiological signal data at the current moment;
[0010] Based on the physiological signal data set and the trained first feature extractor, the space-time feature network is self-supervised trained;
[0011] Step 3, obtain a physiological signal prediction data set of psychological stress, and construct a psychological stress prediction model based on the second prediction decoder and the trained second feature extractor in step 2; wherein the input of the second prediction decoder is the second feature vector of the physiological signal at the current moment, and the output is the predicted psychological stress state at the next moment;
[0012] The psychological stress prediction model is trained based on the physiological signal prediction data set, and the network parameters of the second feature extractor are fixed and unchanged during training, and only the second prediction decoder is adjusted, so as to obtain the psychological stress prediction model for the target physiological signal.
[0013] Further, the loss function of the psychological stress state recognition model during training is set as cross entropy loss.
[0014] Further, a fully connected layer is used as the classifier of the psychological stress state recognition model.
[0015] Further, the optimization target of the self-supervised training in step 2 is set as:
[0016]
[0017] in, For mean square error loss, For feature consistency loss, The balance parameter is used to balance the two types of losses;
[0018]
[0019]
[0020] in, Let K be the L2 norm, and K represent the length of the physiological signal data, i.e., the number of signal sampling points. Indicates time, Indicates the current moment. express Physiological signal data at any given time, i.e., self-supervised data labels. This indicates the output of the first predictive decoder. Physiological signal data at any given time This represents the physiological signal data at the current moment. This indicates the output of the first predictive decoder. Physiological signal data at any given time This represents the first feature vector extracted by the first feature extractor.
[0021] Furthermore, the second feature extractor is constructed based on a concatenated convolutional neural network and a Mamba neural network.
[0022] Furthermore, when training a psychological stress prediction model based on a physiological signal prediction dataset, if the prediction task of the psychological stress prediction model is a continuous index of psychological stress state, then the optimization objective during training adopts the mean squared error:
[0023]
[0024] in, This represents the training optimization objective of the psychological stress prediction model. This represents the output of the second predictive decoder. This represents the output of the second feature extractor. Represents the first in the physiological signal prediction dataset Predictive data for physiological signals, The first part of the physiological signal prediction dataset represents the first part of the dataset. The psychological stress state predicted by physiological signals This indicates the number of signal samples included in the physiological signal prediction dataset.
[0025] Furthermore, when training a psychological stress prediction model based on a physiological signal prediction dataset, if the prediction task of the psychological stress prediction model is a discrete index of psychological stress state, then the optimization objective during training adopts cross-entropy loss:
[0026]
[0027] in, This represents the training optimization objective of the psychological stress prediction model. This represents the output of the second predictive decoder. This represents the output of the second feature extractor. Represents the first in the physiological signal prediction dataset Predictive data for physiological signals, The first part of the physiological signal prediction dataset represents the first part of the dataset. The psychological stress state predicted by physiological signals This indicates the number of signal samples included in the physiological signal prediction dataset. This represents the cross-entropy loss.
[0028] The technical solution provided by this invention brings at least the following beneficial effects:
[0029] This invention acquires a prior feature space based on a psychological stress recognition task. Further, through self-supervised training using physiological signals, and by combining the prior feature space with the temporal dependence of psychological stress, the model can learn the characteristic information of physiological signals at the next moment. Fine-tuning then applies this characteristic information to the psychological stress prediction task. This prediction method effectively avoids the problem of difficulty in training the prediction model due to sparse data labeling, thus enabling the prediction of pilots' psychological stress at the next moment. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 This is a schematic diagram of the processing procedure according to an embodiment of the present invention. Detailed Implementation
[0032] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not only to limit the scope of the claimed present application, but only to represent selected embodiments of the present application.
[0033] The embodiments of the present application provide a pilot psychological stress prediction and early warning method, aiming to provide a psychological stress prediction framework, so that in the case of label annotation sparsity caused by the current psychological stress data collection method, the next moment psychological stress state can be predicted.
[0034] In one embodiment, as shown in Figure 1 , the specific implementation steps of the pilot psychological stress prediction and early warning method of the present embodiment include:
[0035] Step 1: Obtain a psychological stress recognition data set, initialize a neural network-based recognition model and train to obtain a feature space of psychological stress; wherein the recognition model includes a feature extractor and a classifier, and the trained feature extractor is saved for step 2;
[0036] Step 2: Obtain a physiological signal data set of psychological stress, build a spatio-temporal feature network, use the time series dependence characteristics of psychological stress combined with the feature space of step 1, self-supervised train the spatio-temporal feature network, and obtain a spatio-temporal feature extractor;
[0037] Step 3: Obtain a physiological signal prediction data set of psychological stress, fix the spatio-temporal feature extractor of step 2, adjust the decoder, retrain the neural network, fine-tune the decoder parameters, and realize prediction of the next moment of psychological stress.
[0038] In one embodiment, step 1 includes the following steps:
[0039] Step 1-1, obtain a psychological stress recognition data set (including physiological signals and psychological stress states), for physiological signals and the psychological stress state at this moment , construct training data ;
[0040] Step 1-2, initialize the recognition model, for example, use Resnet18 as the feature extractor , and use a fully connected layer as the classifier ;
[0041] Step 1-3, training the recognition model, taking the cross-entropy of the classification result and the true result as the loss, and optimizing the target For:
[0042]
[0043] wherein, is the cross-entropy loss, n is the number of samples, represents the output of the classifier, i.e. the predicted mental stress state.
[0044] Step 1-4, obtain the mental stress prior feature space The feature space is constructed by the feature extractor of the recognition model, satisfying . That is, the trained feature extractor is used for feature extraction of the physiological signal data set in step 2.
[0045] In one embodiment, step 2 includes the following steps:
[0046] Step 2-1, obtain a physiological signal self-training data set under mental stress, for physiological signal , construct self-supervised training data ; that is, construct self-supervised training data based on adjacent data points.
[0047] Step 2-2, initialize the self-supervised training network, which mainly includes a spatio-temporal feature extractor and a decoder for the next time physiological signal data prediction task.
[0048] In this example, the spatio-temporal feature extractor is constructed by a CNN neural network and a Mamba neural network in series. The CNN part in the network is composed of four layers of CNN, which extracts the features in the physiological signal by using the step-by-step convolution method, and refines the local features of the sample, thereby extracting high-resolution features in the physiological signal as initial features. The Mamba network part captures the long-time dependence of the initial features, selectively processes the initial features extracted in the early stage, thereby jointly extracting the spatio-temporal features of the physiological signal.
[0049] Step 2-3, train the network with consistent features combined with prior knowledge.
[0050] Psychological stress is a time-dependent psychological risk factor; two similar signals exhibit similar psychological stress responses. Combining the feature space from step 1 as prior knowledge, self-supervised training of the neural network effectively ensures the learning of stress-related features, avoiding being overwhelmed by features like heart rate, which contain a lot of information but have a weaker correlation with psychological stress. Therefore, the optimization objective of self-supervised pre-training is:
[0051]
[0052] in, For mean square error loss, It is a feature consistency loss that combines prior knowledge. Parameters are preset to balance the two types of losses. The two losses are specifically:
[0053]
[0054]
[0055] in, It is the L2 norm. For estimating self-supervised training models The signal at a given moment, i.e., the decoder The output, represents the feature vector extracted by the feature extractor; K represents the temporal length of the physiological signal.
[0056] In one embodiment, step 3 includes the following steps:
[0057] Step 3-1: Obtain the psychological stress prediction dataset for fine-tuning, i.e., for physiological signals. It has corresponding psychological stress prediction tags. This allows for the construction of fine-tuning data. ;
[0058] Step 3-2: Initialize the psychological stress prediction model; specifically, fix the spatiotemporal feature extractor from Step 2. No change, replace with a new decoder This enables the spatiotemporal feature extractor The obtained features can be mapped to the final predicted psychological stress state. In this embodiment, It is implemented using a fully linked layer.
[0059] Step 3-3: Fine-tune the model to predict psychological stress.
[0060] Fixed-time spatiotemporal feature extractor Retrain the decoder with the parameters unchanged. To the mental stress prediction task, the continuous indicators of the mental stress state are used as the prediction task, and the optimization target uses the mean square error as:
[0061]
[0062] wherein, represents the output of the decoder , that is, the predicted next moment of the mental stress prediction label.
[0063] If the discrete indicators of the mental stress state are used as the prediction task, the optimization target can use the cross-entropy loss:
[0064]
[0065] Thus, the next moment of the mental stress prediction model is trained, and the fixed spatiotemporal feature extractor ensures the prediction information of the future physiological signal, and realizes the pilot mental stress prediction and early warning through fine-tuning.
[0066] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
[0067] The above only describes some embodiments of the present application. For those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.
Claims
1. A method for predicting and warning of pilot psychological stress, characterized in that, Includes the following steps: Step 1: Obtain a psychological stress recognition dataset, which includes psychological signals and psychological stress states; A psychological stress state recognition model, including a first feature extractor and a classifier, is trained and the trained first feature extractor is saved; wherein, the input of the classifier is the first feature vector of the physiological signal output by the first feature extractor; the classifier is used to predict the psychological stress state of the physiological signal. Step 2: Obtain the physiological signal dataset of psychological stress, and build a spatiotemporal feature network including a second feature extractor and a first predictive decoder. The second feature extractor is used to extract the second feature vector of the physiological signal data, and the first predictive decoder predicts the physiological signal data at the next moment based on the second feature vector of the physiological signal data at the current moment. Based on the physiological signal dataset and the trained first feature extractor, the spatiotemporal feature network is trained in a self-supervised manner. Step 3: Obtain the physiological signal prediction dataset of psychological stress, and construct a psychological stress prediction model based on the second prediction decoder and the second feature extractor trained in Step 2; wherein, the input of the second prediction decoder is the second feature vector of the physiological signal at the current moment, and the output is the predicted psychological stress state at the next moment. The psychological stress prediction model is trained based on a physiological signal prediction dataset, with the network parameters of the second feature extractor remaining constant during training.
2. The method as described in claim 1, characterized in that, The loss function for training the psychological stress state recognition model is set to cross-entropy loss.
3. The method as described in claim 1, characterized in that, A fully connected layer is used as the classifier for the psychological stress state recognition model.
4. The method as described in claim 1, characterized in that, The optimization objective of the self-supervised training in step 2 is set as follows: ； in, For mean square error loss, For feature consistency loss, The balance parameter is used to balance the two types of losses; ； ； in, Let K be the L2 norm, and K represent the length of the physiological signal data. Indicates time, Indicates the current moment. express Physiological signal data at any given time This indicates the output of the first predictive decoder. Physiological signal data at any given time This represents the physiological signal data at the current moment. This indicates the output of the first predictive decoder. Physiological signal data at any given time This represents the first feature vector extracted by the first feature extractor.
5. The method as described in claim 1, characterized in that, The second feature extractor is constructed by concatenating a convolutional neural network and a Mamba neural network.
6. The method as described in claim 1, characterized in that, When training a psychological stress prediction model based on a physiological signal prediction dataset, if the prediction task of the psychological stress prediction model is a continuous indicator of psychological stress state, then the mean squared error is used as the optimization objective during training. ； in, This represents the training optimization objective of the psychological stress prediction model. This represents the output of the second predictive decoder. This represents the output of the second feature extractor. Represents the first in the physiological signal prediction dataset Predictive data for individual physiological signals, The first part of the physiological signal prediction dataset represents the first part of the dataset. The psychological stress state predicted by physiological signals This indicates the number of signal samples included in the physiological signal prediction dataset.
7. The method as described in claim 1, characterized in that, When training a psychological stress prediction model based on a physiological signal prediction dataset, if the prediction task of the psychological stress prediction model is a discrete index of psychological stress state, then the optimization objective during training adopts cross-entropy loss: ； in, This represents the training optimization objective of the psychological stress prediction model. This represents the output of the second predictive decoder. This represents the output of the second feature extractor. Represents the first in the physiological signal prediction dataset Predictive data for individual physiological signals, The first part of the physiological signal prediction dataset represents the first part of the dataset. The psychological stress state predicted by physiological signals This indicates the number of signal samples included in the physiological signal prediction dataset. This represents the cross-entropy loss.