Pilot mental load monitoring method based on single-channel electrode electroencephalogram signals

Abstract

The invention discloses a pilot mental load monitoring method based on single-channel electrode electroencephalogram signals, which is based on objective data and avoids the limitation of a subjective scale evaluation method. Electroencephalogram signals are direct reflection of mental load, so that mental load monitoring conducted through the electroencephalogram signals is direct and more accurate, monitoring conducted on mental load through other objective indexes (such as heart rate, skin electricity and respiration) is indirect, and the accuracy is relatively low. Compared with a traditional multi-electrode electroencephalogram signal, the single-electrode electroencephalogram signal only needs to collect the electroencephalogram signal at one position, obviously has higher convenience and practicability, and can be better popularized in practical application. Compared with a multi-electrode method, single-electrode electroencephalogram collection is easy and convenient, efficient and accurate monitoring is achieved through SVM modeling after preprocessing and feature extraction, calculation is rapid, and the result is reliable.

Description

Technical Field

[0001] The present invention belongs to the technical field of mental workload monitoring, and specifically relates to a method for monitoring the mental workload of pilots based on single-channel electrode electroencephalogram (EEG) signals. Background Art

[0002] According to surveys and statistics, among many accidents that occur during flight, human-factor flight accidents account for 70%-80% of all accidents, becoming the focus of attention in the world aviation community. In the overall system of "human-aircraft-environment-task", humans are the core. Excessive mental workload of pilots is one of the key causes of accidents. The main reason is that the multi-source information in flight tasks can easily cause "information overload" for pilots, leading to an increase in pilots' mental workload, thereby having an adverse impact on their operating behaviors and posing a potential threat to flight safety. Therefore, accurately evaluating the mental workload of pilots is of great significance for realizing the intelligent control of aircraft cockpits, improving the control behaviors of crew members, and maintaining flight safety.

[0003] Currently, there are generally three means of detecting mental workload, namely subjective measurement, behavioral performance measurement, and physiological index measurement. Subjective measurement is to reflect the mental workload level of the test subjects to the experimenter by means of self-recording by the test subjects and scoring by filling out questionnaires. Behavioral performance measurement refers to using the performance of the test subjects when performing the corresponding tasks in the experiment as an evaluation index of mental workload. Physiological index measurement is to monitor the changes of various physiological signals of the test subjects in real time when they generate a certain level of mental workload. Subjective measurement technology is simple to operate and highly efficient, but the evaluation results are too subjective and there are large individual differences. EEG is a physiological measurement index and is more objective.

[0004] Most traditional methods for identifying the mental workload of pilots or other staff based on EEG signals collect data through multi-electrodes. Due to the difficulties in carrying multi-electrode collection equipment, high costs, and complex and difficult-to-operate procedures, although the accuracy is relatively high, there are certain defects in portability, universality, and practicality, and it also causes great interference to pilots who are performing flight tasks, making it difficult to achieve generalization and marketization. Therefore, this study will extract EEG signals from a single-channel electrode to monitor the mental workload of pilots. Summary of the Invention

[0005] To solve the above technical problems, the present invention designs a method for monitoring the mental workload of pilots based on single-channel electrode EEG signals, including the following steps:

[0006] S1: Set flight tasks, set as Task 1 and Task 2, and simultaneously collect EEG signals according to the flight task situation;

[0007] S2: Preprocess the collected EEG signals;

[0008] S3: Perform a fast Fourier transform on the preprocessed EEG signals, and extract the power values of the δ, θ, α, and β frequency bands of the EEG data before and after the fault during the task; and perform pairwise division on the power values of the four frequency bands to obtain six ratios: α / β, δ / α, δ / β, δ / θ, θ / α, and θ / β.

[0009] Use the power values of the four frequency bands and the six ratios obtained in S3 as data indicators; use the EEG data indicators of Task 1 as the model training data set of the support vector machine (SVM), and use the EEG data indicators of Task 2 as the test data set.

[0010] S4: Use the power values of the 4 frequency bands and the 6 ratios obtained in S3 as data indicators; use the EEG data indicators of Task 1 as the model training data set of the support vector machine (SVM), and use the EEG data indicators of Task 2 as the test data set.

[0011] Use the EEG data indicators of Task 1 as the model training data set of the support vector machine (SVM), and use the EEG data indicators of Task 2 as the test data set.

[0012] S5: Use the EEG data indicators of the flight task to start training the SVM model and perform verification.

[0013] Further, in S2, the preprocessing process of the collected EEG signals is as follows:

[0014] S2.1: Filtering, including high-pass filtering with a lower cut-off frequency of 0.1 Hz, low-pass filtering with an upper cut-off frequency of 40 Hz, and notch filtering with a range of 48 Hz - 52 Hz.

[0015] S2.2: Segmentation, divide the EEG signals obtained in each flight task into two segments, the first segment is before the fault, and the second segment is after the fault.

[0016] S2.3: Removing extreme values: Divide the EEG data into several intervals with a length of 500 milliseconds. In each 500-millisecond interval, if the EEG voltage value in this interval is outside [-100 μV, +100 μV], it is marked as an extreme value, and the interval marked as an extreme value is removed.

[0017] Further, in S5, the steps of training the SVM model and performing verification are as follows:

[0018] S5.1: Feature selection: For each of the 10 data indicators in S4, perform a one-way repeated measures analysis of variance. The factor is the flight phase, which is divided into two levels: before the fault and after the fault, and find the EEG data indicators that can distinguish between before the fault and after the fault.

[0019] S5.2: Data normalization: Convert the original values of the data indicators selected in S5.1 into data within the range of [0, 1]. The specific conversion formula is as follows:

[0020]

[0021] In the formula, represents the normalized data, represents the original data, represents the minimum value in this group of data, represents the maximum value in this group of data;

[0022] S5.3: Selection of SVM model hyperparameters: The SVM model uses the radial basis function (RBF) as the kernel function of the model, and determines two hyperparameters respectively, γ in the radial basis function and the regularization parameter C;

[0023] After determining C and γ, bring the data of the training set into the model. Mark the EEG data before the fault of Task 1 as low load (-1), and the EEG data after the fault as high load (1) for model training;

[0024] S5.5: Model test and verification: After the model is trained, normalize the EEG data of Task 2 as the test data set to test the effect of the model.

[0025] Furthermore, the method for determining hyperparameters in S5.3 is as follows:

[0026] S5.3.1: First, set the range of log2C to [-30, 30], and set the range of log2γ to [-30, 30];

[0027] S5.3.2: In the two ranges set in S5.3.1, traverse all combinations of numbers within the range with a step of 1, and bring each combination of C and γ into the SVM model;

[0028] S5.3.3: Use the 5-fold cross-validation method, that is, divide the training set into 5 parts, use 4 of them as the training set, and the remaining 1 part as the validation set. Bring each combination of C and γ, calculate the correct rate of validation, and finally select the C and γ that can obtain the highest correct rate.

[0029] The beneficial effects of the present invention are:

[0030] First of all, the present invention is based on objective data (i.e., EEG data), avoiding the limitations of the subjective scale assessment method. EEG signals are a direct reflection of mental workload. Therefore, using EEG signals for mental workload monitoring is direct and more accurate, while other objective indicators (such as heart rate, skin conductance, respiration, etc.) have indirectness in mental workload monitoring, and their accuracy is relatively low.

[0031] Secondly, compared with traditional multi-electrode EEG signals, the single-electrode EEG of the present invention only needs to collect EEG signals at one location, which obviously has greater convenience and practicability and can be better promoted in practical applications. And through preprocessing of single-electrode EEG data such as filtering, segmentation, and removal of extreme values, and then converting them into power values of each frequency band and their pairwise ratios through fast Fourier transform, and then selecting indicators that can significantly distinguish different mental workloads as the eigenvalues of SVM through variance analysis, the constructed SVM model can accurately monitor mental workload, the calculation speed of the model is very fast, and the obtained results are highly reliable. Therefore, by simulating the real task scenario of the cockpit during flight in the simulator, the present invention can specifically evaluate the mental workload of pilots during task execution, and has feasibility and good operability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below.

[0033] Figure 1 It is the ROC curve diagram in the present invention.

[0034] Figure 2 It is the flowchart of the monitoring method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] Embodiment

[0036] The present invention discloses a method for monitoring pilots' mental workload based on single-channel electrode EEG signals, including the following steps:

[0037] S1: Set flight tasks, set as Task 1 and Task 2, and at the same time collect EEG signals according to the flight task

[0038] conditions;

[0039] Specifically, 37 pilots were recruited as subjects to participate in the experiment. The average age of the subjects was 24 years old, and the average flight hours were 252.02 hours. Experimental procedure: The pilots participating in the experiment wore an EEG cap to collect EEG signals at the left prefrontal position (FP1 point of the international 10-20 system). After wearing the EEG cap, each pilot needed to complete two flight tasks on a Cessna 172R simulator.

[0040] Specific Instructions for Task 1: The initial conditions are Runway 13 of a certain airport at 14:00 pm, with high visibility, no crosswind, and clear weather. During the initial climb phase after takeoff and leaving the ground, maintain the track on one side until reaching the minimum safe altitude (500 feet, AGL) to meet the obstacle clearance requirements, and then make a 180-degree turn. After the 180-degree turn, continue to climb on both sides to the rectangular traffic pattern altitude of 2500 feet. Twenty seconds after completing the 180-degree turn, make a 90-degree turn to the downwind leg course. Complete the corresponding pre-landing items and the corresponding checklist on the downwind leg, and adjust the appropriate rectangular traffic pattern width. Ten seconds after the aircraft is tangent to the runway threshold on the downwind leg, set an engine failure. After the failure occurs, the test subjects are required to complete the corresponding in-flight engine failure checklist and the powerless emergency landing checklist, and control the aircraft to land safely.

[0041] Specific Instructions for Task 2: The first half of Task 2 is the same as Task 1 until ten seconds after the aircraft is tangent to the runway threshold on the downwind leg, when three failures, namely engine failure, altimeter failure, and airspeed indicator failure, are set to occur simultaneously. After the failures occur, the test subjects are required to complete the corresponding in-flight engine failure checklist, the powerless emergency landing checklist, and the air data system failure checklist, and control the aircraft to land safely.

[0042] S2: The electroencephalogram (EEG) signals collected during the execution of the above Tasks 1 and 2

[0043] are preprocessed; mainly including filtering, which includes high-pass filtering with a lower cutoff frequency set to 0.1 Hz, low-pass filtering with an upper cutoff frequency set to 40 Hz, and notch filtering; the range is: 48 Hz - 52 Hz; the low-pass filtering (40 Hz) and notch filtering (48 - 52 Hz) are used to remove high-frequency interference and power frequency interference. Segmentation (according to the key events in the flight task (such as before and after the failure), the EEG signals are divided into two segments for subsequent analysis), the EEG signals obtained for each flight task are divided into two segments, the first segment is before the failure, and the second segment is after the failure; Extreme value removal: The EEG data is segmented into several intervals with a length of 500 milliseconds each. In each 500-millisecond interval, if the EEG voltage value in this interval is outside [-100 μV, +100 μV], it is marked as an extreme value, and the interval marked as an extreme value is removed to ensure the data quality;

[0044] S3: Perform a fast Fourier transform on the preprocessed EEG signals, and extract the power values of the four frequency bands of δ, θ, α, and β in the EEG data before and after the failures in Tasks 1 and 2; and calculate the quotient of the power values of the four frequency bands in pairs to obtain six ratios of α / β, δ / α, δ / β, δ / θ, θ / α, and θ / β to capture the relative changes between different frequency bands and reflect the mental workload status;

[0045] S4: Use the power values of the 4 frequency bands and the 6 ratios obtained in S3 as data indicators; use the EEG data indicators of Task 1 as the model training dataset for the support vector machine (SVM), and use the EEG data indicators of Task 2 as the test dataset;

[0046] S5: Utilize the EEG data indicators of the flight mission to start training the SVM model and conduct verification. The steps for training the SVM model and conducting verification are as follows:

[0047] S5.1: Feature selection: For each of the 10 data indicators in S4, perform a one-way repeated measures analysis of variance. The factor is the flight phase, which is divided into two levels: before failure and after failure, to find the EEG data indicators that can distinguish between before failure and after failure;

[0048] S5.2: Data normalization: Convert the original values of the data indicators selected in S5.1 into data within the range of [0,1] to eliminate the influence of dimensions and improve the model training efficiency. The specific conversion formula is as follows:

[0049] S5.2: Data normalization: Convert the original values of the data indicators selected in S5.1 into data within the range of [0,1] to eliminate the influence of dimensions and improve the model training efficiency. The specific conversion formula is as follows:

[0050]

[0051] In the formula, represents the normalized data, represents the original data, represents the minimum value in this group of data, represents the maximum value in this group of data;

[0052] S5.3: Selection of SVM model hyperparameters: The SVM model uses the radial basis function (RBF) as the kernel function of the model, and determine two hyperparameters respectively, γ in the radial basis function and the regularization parameter C; first, set the range of log2C to [-30,30], and set the range of log2γ to [-30,30];

[0053] Within the set two ranges, with a step size of 1, traverse all the number combinations within the ranges, and bring each combination of C and γ into the SVM model; use the 5-fold cross-validation method, that is, divide the training set into 5 parts, use 4 of them as the training set, and the remaining 1 part as the validation set, bring each combination of C and γ, calculate the verification accuracy rate, and finally select the C and γ that can obtain the highest accuracy rate.

[0054] S5.4 After determining C and γ, bring the data of the training set into the model, mark the EEG data before failure of Task 1 as low load (-1), and mark the EEG data after failure as high load (1) for model training;

[0055] S5.5: Model Test and Verification: After the model is trained, the EEG data of Task 2 is normalized and used as the test dataset to test the model's performance. The pre-fault data of Task 2 is labeled as low load (-1), and the post-fault data is labeled as high load (1). The predicted results of the model are compared with the labeled results to construct the confusion matrix as shown in Table 1 below:

[0056] Table 1 Confusion Matrix Constructed by Comparing the Predicted Results and Labeled Results of the Model

[0057]

[0058] Based on the data in the confusion matrix, performance metrics such as the accuracy, precision, recall, and F1 value of the model can be calculated to evaluate the accuracy of the SVM model. An ROC curve can also be drawn and the area under the curve (AUC) can be calculated as a performance metric for evaluating the SVM model. The data obtained through experiments, after verification, shows that the performance of the SVM model is good. Therefore, based on the single-electrode EEG data at the left prefrontal lobe (FP1 position), the mental workload of pilots can be monitored.

[0059] Based on the actual verification of the above confusion matrix, it is specifically shown in Table 2 below:

[0060] Table 2 Confusion Matrix Constructed by Comparing the Actual Predicted Results and Labeled Results

[0061]

[0062] From the above confusion matrix, the accuracy, error rate, precision, recall, and F1 value of the model can be further calculated to obtain the performance metrics, which are specifically shown in Table 3 below:

[0063] Table 3 Model Performance Metric Values Calculated from the Confusion Matrix

[0064]

[0065] Specifically, in terms of various performances, the model has good classification ability, and from the precision perspective, it shows that the model is relatively conservative and accurate when predicting the positive class; at the same time, combined with the ROC curve in the performance metrics, specifically as Figure 1 shown, the above results indicate that the monitoring of pilots' mental workload can be achieved only based on single-electrode EEG signal data and the SVM algorithm. This method is practical, highly reliable, requires simple equipment and operations, and the model has a fast calculation speed. It has advantages compared with the existing mental workload monitoring methods using multi-electrodes, multi-types of physiological signals, and multi-dimensional features, and can be well applied to the pilot group to monitor their mental workload during actual flight.

[0066] Specifically, this application uses single-electrode electroencephalogram (EEG) signals as the monitoring basis, greatly simplifying the data acquisition process and reducing the requirements for equipment complexity and operation difficulty. This innovation not only reduces equipment costs but also improves wearing comfort, enabling pilots to maintain a natural state during long flights and reducing the additional burden caused by equipment discomfort. Moreover, as a powerful classification algorithm, Support Vector Machine (SVM) can accurately extract features closely related to mental workload from single-electrode EEG signals through its efficient feature selection and classification capabilities, and construct a monitoring model with high accuracy. The algorithm has a fast calculation speed and can maintain low latency even when processing real-time data streams, ensuring the immediacy and effectiveness of monitoring results.

[0067] Experimental results show that the accuracy of this method reaches 70.83% and the precision is 75%, indicating that the model has high reliability in distinguishing different mental workload states of pilots. At the same time, through fine preprocessing steps and optimized algorithm parameters, the error rate is further reduced, ensuring the accuracy of monitoring results. Therefore, the method for monitoring pilots' mental workload based on single-electrode EEG signals and SVM algorithm is not only technically feasible, efficient and reliable, but also has significant advantages and wide applicability in practical applications. This method provides strong technical support and solutions for the real-time monitoring of pilots' mental workload, and is of great significance for improving flight safety and optimizing flight mission arrangements.

[0068] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described.

Claims

1. A method for monitoring the mental workload of pilots based on single-channel electrode electroencephalogram signals, characterized in that, It includes the following steps: S1: Set flight tasks, set as Task 1 and Task 2, and collect EEG signals according to the flight task situation at the same time; S2: Preprocess the collected EEG signals; S3: Perform fast Fourier transform on the preprocessed EEG signals, and extract each The power values of the four frequency bands of δ, θ, α, and β of the EEG data before and after the fault in the task; and perform pairwise division on the power values of the four frequency bands to obtain six ratios of α / β, δ / α, δ / β, δ / θ, θ / α, and θ / β; S4: Use the power values of the 4 frequency bands and 6 ratios obtained in S3 as data indicators; use the EEG data indicators of Task 1 as the model training data set of the support vector machine (SVM), and use the EEG data indicators of Task 2 as the test data set; S5: Use the EEG data indicators of the flight task to start training the SVM model and conduct verification. In S2, the preprocessing process of the collected EEG signals is as follows: S2.1: Filtering, including high-pass filtering, the lower limit value of the high-pass filter is set to 0.1 Hz, low-pass filtering, the upper limit value of the low-pass filter is set to 40 Hz, and notch filtering, the range is: 48 Hz - 52 Hz; 2. The pilot mental workload monitoring method based on single-channel electrode EEG signals according to claim 1, characterized in that, S2.2: Segmentation, divide the EEG signals obtained from each flight task into two segments, the first segment is before the fault, and the second segment is after the fault; S2.3: Remove extreme values: Divide the EEG data into several intervals with a length of 500 milliseconds. In each 500-millisecond interval, if the EEG voltage value in this interval is outside [-100 μV, +100 μV], it is marked as an extreme value, and the interval marked as an extreme value is removed. In S5, the steps of training the SVM model and conducting verification are as follows:

3. The method for monitoring the mental workload of pilots based on single-channel electrode EEG signals according to claim 1, wherein S5.1: Feature selection: For each of the 10 data indicators in S4, perform one-way repeated measures analysis of variance. This factor is the flight phase, which is divided into two levels: before the fault and after the fault, and find the EEG data indicators that can distinguish between before the fault and after the fault; S5.2: Data normalization: Convert the original values of the data indicators selected in S5.1 into data within the range of [0, 1]. The specific conversion formula is as follows: S5.3: Selection of SVM model hyperparameters: The SVM model uses the radial basis function (RBF) as the kernel function of the model, and determine two hyperparameters respectively, γ in the radial basis function and the regularization parameter C; In the formula, represents the normalized data, represents the original data, represents the minimum value in this group of data, represents the maximum value in this group of data; S5.4 After determining C and γ, bring the data of the training set into the model, mark the EEG data before the fault of Task 1 as low load, and the EEG data after the fault as high load, and conduct model training; S5.5: Model test verification: After the model is trained, normalize the EEG data of Task 2 and use it as the test data set to test the effect of the model. In S5.3, the method for determining hyperparameters is as follows:

4. The method for monitoring the mental workload of pilots based on single-channel electrode EEG signals according to claim 3, characterized in that, S5.3.1: First, set the range of log2C to [-30, 30], and set the range of log2γ to [-30, 30]; S5.3.2: In the two ranges set in S5.3.1, traverse all combinations of numbers within the range with a step of 1, and bring each combination of C and γ into the SVM model; ​ S5.3.3: Apply the 5-fold cross-validation method, that is, divide the training set into 5 parts, use 4 of them as the training set, and the remaining 1 part as the validation set. Substitute each combination of C and γ, calculate the validation accuracy rate, and finally select the C and γ that can obtain the highest accuracy rate.

5. Application of the pilot mental workload monitoring method based on single-channel electrode EEG signals according to any one of claims 1-4 in a monitoring system.

Images