Pilot high-risk psychological state recognition method based on electrocardiogram data enhancement

Abstract

The application discloses a pilot high-risk psychological state recognition method based on electrocardio data enhancement, and relates to the technical field of physiological signal recognition.The application firstly trains a conventional recognition model on all data, then extracts low information sample sections of head class samples in an unbalanced data set by the model through a feature visualization technology, splices the low information sample sections with electrocardio effective information of tail class signals, realizes data enhancement of high-risk psychological state samples, and further realizes re-balancing of psychological risk data.Finally, a psychological risk recognition model is trained on the re-balanced data set, so that the psychological state of pilots can be detected, recorded and analyzed in actual flight early warning and simulation flight training.The application is targeted to the unbalanced scene of psychological risk samples caused by insufficient pilot high-risk psychological state data, so that the high-risk psychological state can be accurately captured and effectively recognized, and the flight safety of pilots in the flight process is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of physiological signal recognition, and particularly relates to a pilot high-risk psychological state recognition method based on electrocardiogram data enhancement. BACKGROUND

[0002] Safety is the lifeline of the civil aviation industry. In today's aviation safety field where modern science and technology are gradually deepening, the main task of pilots during flight has undergone great changes. From the past two-rod one-rudder flight driving operation to the collection, arrangement, decision-making and execution of the overall flight related information during the flight process. Among the current factors that endanger civil aviation pilot safety, psychological risk factors are the most hidden but also the most dangerous factor.

[0003] The risk psychological state of civil aviation pilots during flight includes risk emotion, mental stress, cognitive fatigue, risk perception, alertness and other serious threats to flight safety pilot psychological factors. These psychological risk factors affect the information receiving ability and problem solving ability of pilots in general flight and in the event of unexpected special situations from motivation, skill level, perception of danger, etc. Highly excited panic, high mental stress and other high-risk psychological states can even swallow the pilot's reason, leading to the inability to effectively control safety risks, and ultimately causing a huge disaster.

[0004] Therefore, when identifying the psychological risk factors of pilots, the most important thing to focus on is these high-risk psychological states. However, due to the occupational adaptability of pilots, the psychological fluctuations produced when facing unexpected events and special situations are usually not too intense. Even if some unexpected situations during flight training have triggered high-intensity psychological risks, this high-risk fluctuation is only temporary and one-time, which leads to the fact that the pilot's psychological risk factors are in a high-risk state during flight training or actual flight process. It is relatively rare.

[0005] The consequence of this problem is that in the process of psychological risk data collection, high-risk psychological state is prone to be in the situation of insufficient data due to its difficulty in triggering and obtaining. Therefore, the psychological risk data set of pilots tends to be a long-tail distribution with sufficient low-risk data and insufficient high-risk data, that is, the calm and relaxed state with low attention and low risk accounts for the majority of the total data, while the high-risk state such as panic and severe mental stress accounts for a small proportion. If no measures are taken to present the long-tail distributed data, the deep learning method which highly depends on the data volume will face two challenges. The first challenge is that due to the lack of data, the generalization ability of high-risk state samples is insufficient, which makes it difficult for the model to learn the recognizable features of high-risk state, so the recognition performance of high-risk state cannot meet the needs. The second challenge is that because the data volume of the head class is much larger than that of the tail class, the training of the model will be dominated by the head class, which leads to the effective learning of the tail class being overwhelmed by the head class, and finally the accuracy in the test set and actual application scene is much lower than that in the training set. Therefore, it is necessary to study a pilot high-risk psychological state recognition method based on electrocardiogram data enhancement to overcome the two challenges of data scarcity and insufficient proportion of high-risk psychological state. SUMMARY

[0006] The application aims at the deficiencies of the prior art, and provides a pilot high-risk psychological state recognition method based on electrocardiogram data enhancement.

[0007] The technical scheme adopted by the application is:

[0008] The pilot high-risk psychological state recognition method based on electrocardiogram data enhancement comprises the following steps:

[0009] Step S1: training a low-information sample segment extraction model f on the long-tail distributed electrocardiogram data set D h ; that is, based on the pre-set first deep network model based on residual network, the model is trained on the long-tail distributed electrocardiogram data set D to obtain the low-information sample segment extraction model f h ;

[0010] The head class of the long-tail distributed electrocardiogram data set D is a low-risk psychological state, and the tail class is a high-risk psychological state;

[0011] The low-information sample segment extraction model f h is a deep network model with a backbone network of residual network (ResNet);

[0012] Step S2: For all head class samples in the ECG dataset D, use the model f through feature visualization technology h Extract each segment’s head class sample x h The sample low information segment is stored in the low information sample queue Q;

[0013] Step S3: In the part outside the peak protection segment of all tail class samples in the ECG dataset D, the low information sample segments in the low information sample queue Q are used to enhance the data of the tail class signal samples, and a re-balanced ECG dataset D' is obtained:

[0014] Step S4: Training a pilot risk psychological state recognition model f on the re-equalized ECG dataset D', wherein the pilot risk psychological state recognition model f is a deep network model whose backbone network is ResNet; that is, based on a preset second residual network-based deep network model, the model is trained on the re-equalized ECG dataset D' to obtain the pilot risk psychological state recognition model f;

[0015] Step S5: Input the pilot's real-time electrocardiogram data into the trained pilot risk psychological state recognition model f to obtain the pilot's risk psychological state recognition result.

[0016] Furthermore, in step S1, the low information sample segment extraction model f is trained h When , the loss function used is the cross entropy loss function.

[0017] Furthermore, in step S2, the low information segment of the sample is a continuous segment of sample point activation values ​​0 obtained by feature visualization technology.

[0018] Furthermore, in step S3, the peak protection segment of the tail class samples is obtained by using a peak detection algorithm.

[0019] Furthermore, in step S3, the peak detection algorithm is used to obtain the peak protection segment of the tail class samples as follows:

[0020] The peak detection algorithm is used to determine the location r of the R peak (i.e., the R wave peak) of the ECG signal in the sample, and then the preset threshold is used to (The preferred upper limit is 20, that is, 20 is the base value, and a certain upper and lower deviation is allowed) Determine the protection segment of the tail class samples:

[0021] Furthermore, in step S3, the data enhancement of the tail signal samples is implemented by using the low information sample segments in the low information sample queue Q as follows:

[0022] Extract a low-information sample from the low-information sample queue Q and will Paste to the tail class sample xt obtain a new signal sample after data enhancement obtain a new signal sample after data enhancement obtain a new signal sample after data enhancement

[0023] Further, in step S4, when the pilot risk psychological state recognition model f is trained, the loss function used is a cross-entropy loss function.

[0024] Further, the low-risk psychological state includes: calm emotion, low stress level and low fatigue degree; and the high-risk psychological state includes: panic emotion, extreme resistance emotion, high stress level and high fatigue degree.

[0025] The technical solution provided by the present application at least brings the following beneficial effects:

[0026] The present application can obtain a rebalanced electrocardiogram data set by data enhancement on sample data under the condition that the psychological risk data is in a low-risk psychological state with sufficient data and a high-risk psychological state with insufficient data, and further realize effective recognition of the psychological risk state, especially the high-risk psychological state. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0028] Figure 1 The process schematic diagram of the pilot high-risk psychological state recognition method based on electrocardiogram data enhancement provided by the embodiments of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to make a detailed and complete description of the technical scheme in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not only to limit the scope of the claimed application, but only to represent selected embodiments of the present application.

[0030] An embodiment of the present invention provides a method for identifying high-risk psychological states of pilots based on ECG data enhancement. The method first trains a conventional recognition model on all data, then uses feature visualization technology to extract low-information sample segments of head-type samples in an unbalanced data set through the model, and splices them with the effective ECG information of tail-type signals to achieve data enhancement of high-risk psychological state samples, thereby achieving rebalancing of psychological risk data. Finally, a psychological risk identification model is trained on the rebalanced data set to ensure that the psychological state of pilots in actual flight warnings and simulated flights for training can be detected, recorded, and analyzed. The method proposed in the embodiment of the present invention is targeted at scenarios where psychological risk samples are unbalanced due to insufficient data on high-risk psychological states of pilots, so that high-risk psychological states can be accurately captured and effectively identified, thereby ensuring the flight safety of pilots during flight.

[0031] As a possible implementation, see Figure 1 The specific implementation steps of the method for identifying a pilot's high-risk psychological state based on ECG data enhancement provided by the embodiment of the present invention include:

[0032] Step S1: Train the low-information sample segment extraction model f on the long-tail distribution ECG dataset D h :

[0033] Among them, the head class of the long-tail distribution ECG dataset D with sufficient data volume is the low-risk psychological state, and the tail class with scarce data volume is the high-risk psychological state;

[0034] In this embodiment, the ECG dataset D is an ECG dataset of pilot psychological risk factors collected based on a simulated flight training program for flight trainees. Low-risk psychological states include calm emotions, low stress levels, and low fatigue levels, while high-risk psychological states include panic, extreme resistance, extremely high stress levels, and extremely high fatigue levels.

[0035] Low-information sample segment extraction model f h The backbone network is a deep neural network with ResNet, preferably, a low-information sample segment extraction model f h The specific structure is ResNet-18;

[0036] Specifically, the model f is trained by cross entropy loss h , where the cross entropy loss is: Among them, n represents the number of samples participating in the training, I (i = y i ) represents an exponential function. If the predicted result is consistent with the true label, the function value is 1, otherwise it is 0; f i It is the signal feature of the sample extracted by the feature extraction network, and p() represents the classification probability of the current task output by the model.

[0037] Step S2: For all head class samples in the ECG dataset D, use feature visualization technology to use f h Extract each segment’s head class sample x h The sample low information segment is stored in the low information sample queue Q:

[0038] Among them, feature visualization technologies include Grad-CAM, Grad-CAM++, etc., and Grad-CAM++ is usually used. Preferably, Grad-CAM++ is used as the feature visualization technology in this embodiment. The low information segment of the sample is a continuous segment with a sample point activation value of 0 obtained by the feature visualization technology;

[0039] Specifically, feature visualization technology can be used to h Get a head class sample x h The activation value of each sample point in x corresponds to the possibility of the sample point being used as a feature for identifying psychological risk status. h The activation value divides the sample points into two parts: one part has an activation value of 0, and the other part has an activation value of a floating point value in the range of (0, 1]. The larger the value, the higher the possibility that the sample point can be used as an identifiable feature of the psychological risk factor. The sample segment with a continuous activation value of 0 is considered a low-information sample segment, that is, this sample segment contains no or only a very small amount of information related to the psychological risk factor category;

[0040] In this embodiment, only f is selected h This step is performed on head samples with a confidence score higher than 0.6 to ensure the reliability of the extracted psychological risk state identification features;

[0041] The low information sample queue Q is a queue storage structure for low information sample segments.

[0042] Step S3: In the part outside the peak protection segment of all tail class samples in the ECG dataset D, the low information sample segments in the low information sample queue Q are used to enhance the data of the tail class signal samples, and a re-balanced ECG dataset D' is obtained:

[0043] The peak protection segment of the tail class sample is obtained by the peak detection algorithm. First, the peak position r of the R peak in the sample is determined by peak detection, and then the threshold is used to Determine the protection segment of the tail class samples:

[0044] In this embodiment, the threshold 20 sample points are selected, that is, for each ECG signal peak, 41 sample points including the peak are used as peak protection segments;

[0045] Extract a low-information sample from the low-information sample queue Q and will Paste to the tail class sample x t Except for the protection segment I. In order to adapt to the characteristics of the ECG signal, since the non-protection segment of the tail class sample contains several segments in one sample, the non-protection segment is filled by copying the low-information sample and adding other low-information samples to obtain the new signal sample after data enhancement. Will Add the ECG dataset D to obtain the rebalanced ECG dataset D`;

[0046] Specifically, the rebalanced ECG dataset D' consists of two parts, one part is all samples in the ECG dataset D, and the other part is new tail class samples obtained by data enhancement.

[0047] Step S4: Train the pilot risk psychological state recognition model f on the re-equalized ECG dataset D':

[0048] The pilot risk psychological state recognition model f is a deep learning model with ResNet as the backbone network;

[0049] In this embodiment, the pilot risk psychological state recognition model f structure is specifically ResNet-18;

[0050] Specifically, the model f is trained by cross entropy loss h , where the cross entropy loss is:

[0051] Step S5: Input the pilot's real-time ECG data x into the trained pilot risk psychological state recognition model f to obtain the pilot's risk psychological state recognition result r(x); that is, when the preset training convergence conditions are met, such as the number of training times reaches the preset upper limit, or the set model loss converges.

[0052] Specifically, the risk psychological state identification result r(x)=f(x).

[0053] In this embodiment, during the training process of model f, the batch size is set to 64, the training epoch is set to 100, the ADAM optimizer is used, and the learning rates of all submodules in the module are set to 0.001.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

[0055] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A method for identifying pilots’ high-risk psychological states based on ECG data enhancement, characterized by: The following steps are involved: Step S1: Train the low-information sample segment extraction model f on the long-tail distribution ECG dataset D h ; Among them, the head class of the long-tail distribution ECG dataset D is low-risk psychological state, and the tail class is high-risk psychological state; Low-information sample segment extraction model f h A deep network model with a residual network as the backbone network; Step S2: For all head class samples in the ECG dataset D, use the model f through feature visualization technology h Extract each segment’s head class sample x h The sample low information segment is stored in the low information sample queue Q; Step S3: In the part outside the peak protection segment of all tail class samples in the ECG dataset D, the low information sample segments in the low information sample queue Q are used to enhance the data of the tail class signal samples, and a re-balanced ECG dataset D' is obtained: Step S4: training a pilot risk psychological state recognition model f on the re-equalized ECG dataset D', wherein the pilot risk psychological state recognition model f is a deep network model whose backbone network is a residual network; Step S5: Inputting the pilot's real-time ECG data into the trained pilot risk psychological state recognition model f to obtain the pilot's risk psychological state recognition result; Among them, in step S3, the data enhancement of the tail signal samples is realized by using the low information sample segments in the low information sample queue Q as follows: Extract a low-information sample from the low-information sample queue Q , and Paste to the tail class sample x t Except for the protection segment, the new signal sample after data enhancement is obtained ,Will The ECG dataset D is added to obtain the re-balanced ECG dataset D'.

2. The method according to claim 1, wherein In step S1, the low information sample segment extraction model f is trained h When , the loss function used is the cross entropy loss function.

3. The method according to claim 1, wherein In step S2, the low information segment of the sample is a continuous segment with a sample point activation value of 0 obtained by feature visualization technology.

4. The method according to claim 1, wherein In step S3, the peak protection segment of the tail class samples is obtained by using a peak detection algorithm.

5. The method according to claim 4, wherein In step S3, the peak detection algorithm is used to obtain the peak protection segment of the tail class samples as follows: The peak detection algorithm is used to determine the peak position r of the R wave of the ECG signal in the sample, and then the preset threshold is used to Determine the protection segment of the tail class samples: .

6. The method according to claim 5, wherein Threshold The value is set to 20±△, where △ represents the allowable deviation.

7. The method according to claim 1, wherein In step S4, when training the pilot risk psychological state recognition model f, the loss function used is the cross entropy loss function.

8. The method according to claim 1, wherein Low-risk psychological states include: calm emotions, low stress levels and low fatigue levels; high-risk psychological states include: panic, extreme resistance, high stress levels and high fatigue levels.

9. The method according to claim 1, wherein Low-information sample segment extraction model f h The pilot risk psychological state recognition model f and the pilot risk psychological state recognition model f both use ResNet-18.

Images