Fatigue recognition model training method and system based on three-lead electroencephalogram

Abstract

The invention belongs to the technical field of biomedical signal processing, and discloses a fatigue recognition model training method and system based on three-lead electroencephalogram. Existing fatigue recognition depends on multi-lead equipment, the cost is high, and dynamic time-frequency evolution information of fatigue cannot be captured during feature extraction. According to the technical scheme, the method comprises the steps that three-lead electroencephalogram signals are preprocessed and subjected to fatigue degree marking, then, time sequence features, frequency domain features and time-frequency domain features are extracted from electroencephalogram data, then, the time sequence features are utilized to train a lightweight LSTM model, the frequency domain features are utilized to train an XGBoost model, and the time-frequency domain features are utilized to train a CNN-LSTM fusion model. According to the method, through mutual cooperation of the three-lead electroencephalogram data and the reasoning model, accurate recognition of the fatigue degree of the three-lead portable equipment is achieved, and a feasible path is provided for large-scale and normalized fatigue risk monitoring of high-risk workers.

Description

Technical Field

[0001] This invention relates to the field of biomedical signal processing technology, specifically to a method and system for training a fatigue recognition model based on three-lead EEG. Background Technology

[0002] In modern industrial production and transportation, employee fatigue is one of the core contributing factors to major safety accidents. For example, during long-distance driving, driver fatigue leading to decreased attention and slowed reaction time is a major cause of traffic accidents.

[0003] Electroencephalography (EEG) records the spontaneous, rhythmic electrical activity of brain neuron clusters through scalp electrodes. It can objectively and in real time reveal changes in an individual's alertness and cognitive state and is considered a standard physiological signal for fatigue recognition.

[0004] However, applying EEG fatigue recognition technology to real-world industrial scenarios, especially for portable, low-cost, and routine monitoring of frontline workers, faces a series of severe technical challenges. Existing fatigue recognition model training methods generally suffer from the following pain points:

[0005] 1. The contradiction between equipment portability and data richness: High-precision EEG systems usually require 8, 16 or even more electrodes. The wearing process is complicated and time-consuming, requiring professional personnel to operate. Moreover, the equipment is bulky and expensive, which cannot meet the needs of on-site staff for long-term, non-intrusive wearing.

[0006] While lightweight three-lead and single-lead head-mounted devices significantly improve portability and ease of use, the reduced number of channels results in insufficient spatial dimension of the acquired EEG signals. When existing algorithm models are applied to data with few channels, incomplete feature extraction leads to a sharp drop in model recognition accuracy, making it difficult to meet the reliability requirements of practical applications.

[0007] 2. The disconnect between signal non-stationarity and feature extraction methods: Fatigue is a dynamic and non-stationary evolutionary process, and the associated EEG activity exhibits complex changes in both the time and frequency domains. However, existing methods disconnect time and frequency features, failing to fully capture the dynamic evolution of fatigue.

[0008] Existing methods often suffer from one-sidedness in feature extraction: they either rely solely on time-domain statistical analysis, which is extremely sensitive to noise interference; or they simply use Fourier transform to extract global frequency domain features, which can reflect rhythm intensity but completely loses information about temporal evolution.

[0009] 3. Imbalance between model complexity and deployment platform computing power: In pursuit of high accuracy, existing methods often use deep neural networks with a huge number of parameters. Their computing and storage requirements far exceed the carrying capacity of the embedded hardware built into portable head-mounted devices, making it impossible to achieve local real-time inference.

[0010] At the same time, if traditional lightweight models such as logistic regression and support vector machines are used to accommodate the computing power of embedded hardware, it is difficult to effectively model the inherent long-term time series dependencies and nonlinear relationships in EEG signals, resulting in reduced accuracy.

[0011] Therefore, there is an urgent need for an innovative technical solution that, using only a three-lead portable device, can fully extract deep information from limited channel data through innovative signal processing and feature engineering methods. Furthermore, a matching model architecture and training strategy that balances high accuracy and low complexity should be designed to achieve stable, real-time, and highly reliable identification of fatigue states on resource-constrained embedded devices. Summary of the Invention

[0012] The purpose of this invention is to provide a training method and system for a fatigue recognition model based on three-lead EEG, so as to solve the problems mentioned in the background art.

[0013] The main design concept of this invention is as follows:

[0014] The primary objective of this invention is to overcome the shortcomings of existing technologies, such as incomplete feature extraction due to a small number of device leads, loss of dynamic information due to one-sided feature representation, and inability to run in real time on portable devices due to bulky models.

[0015] To this end, the present invention provides a systematic solution that aims to build a high-precision, lightweight, and highly generalizable fatigue recognition model based on a low-cost, easy-to-wear three-lead head-mounted device through an innovative data cleaning, feature fusion, and model adaptation training process.

[0016] This model can accurately distinguish between three fatigue levels: "normal," "fatigue," and "dangerous." It achieves sub-second real-time inference on embedded devices, with a core accuracy of no less than 90% and a false negative rate of less than 3% for dangerous conditions. This provides a practical and usable technical tool for fatigue risk early warning in high-risk positions.

[0017] To achieve the above objectives, the present invention provides the following technical solution:

[0018] First, this invention provides a method for training a fatigue recognition model based on three-lead EEG, comprising the following steps:

[0019] The subjects' three-lead EEG and ECG data were collected, including FP1, O1, and C3 EEG data.

[0020] Fatigue levels were annotated using three-lead EEG data based on subjects' subjective reports and ECG data;

[0021] The labeled three-lead EEG data were divided into multiple data segments according to time windows, and time-series features, frequency domain features, and time-frequency domain features were extracted for each data segment.

[0022] Lightweight LSTM models are trained using temporal features, XGBoost models are trained using frequency domain features, and CNN-LSTM fusion models are trained using time-frequency domain features.

[0023] More preferably, the three-lead EEG data are preprocessed before annotation, specifically by:

[0024] The three-lead EEG data were subjected to bandpass filtering and power frequency notch filtering respectively to obtain the filtered data.

[0025] Independent component analysis was used to separate and remove the artifact components of electrooculography (EOG) in the filtered data to obtain the FP1 EEG data after EOG removal.

[0026] The C3 EEG data in the filtered data were subjected to moving average filtering to obtain the processed C3 EEG data.

[0027] Invalid data were removed from the FP1 EEG data after removing electrooculograms, the O1 EEG data in the filtered data, and the invalid data in the processed C3 EEG data.

[0028] More preferably, the method for removing invalid data from the FP1 EEG data after removing electrooculograms, the O1 EEG data in the filtered data, and the processed C3 EEG data includes:

[0029] The signal quality index within the sliding window is calculated by iterating through the FP1 EEG data after electrooculogram removal, the O1 EEG data in the filtered data, and the processed C3 EEG data in the sliding window.

[0030] The signal quality index is compared with a quality threshold. If the signal quality index is less than the quality threshold, the data in the sliding window is invalid and is removed.

[0031] The signal quality index is:

[0032]

[0033] in, Indicates the signal quality index. and Indicates weight, This indicates the proportion of sampling points within a time window where the voltage amplitude of any given EEG data exceeds the range of the voltage standard deviation of that EEG data. This indicates the proportion of the voltage standard deviation of the EEG data within the time window that is greater than the set mean voltage standard deviation of the EEG data.

[0034] More preferably, the method for labeling fatigue levels in three-lead EEG data based on the subject's subjective report and ECG data includes:

[0035] Subjects reported their subjective fatigue levels at different time intervals;

[0036] Heart rate variability index is calculated using electrocardiogram data, and the objective fatigue level of the subject during that period is determined by the heart rate variability index.

[0037] The fatigue level of the subjects during that period was determined and labeled based on subjective and objective fatigue levels.

[0038] More preferably, the method for extracting time-frequency domain features for each data segment is to perform a three-scale continuous wavelet transform using complex Morlet wavelets;

[0039] The time-frequency domain features are three-dimensional time-frequency domain feature tensors, where the first dimension is the data segment index, the second dimension is the time frame, and the third dimension is the data from the three leads respectively. , , and Energy distribution across the four frequency bands.

[0040] More preferably, the fatigue recognition model training method further includes evaluating the basic metrics, generalization ability, and real-time performance of the trained lightweight LSTM model, XGBoost model, and CNN-LSTM fusion model, and optimizing the model based on the evaluation results;

[0041] The basic performance indicators include: classification accuracy, macro-average F1 score, Cohen-Kappa coefficient, and false negative rate for hazardous conditions.

[0042] The generalization ability assessment includes cross-subject verification and noise immunity testing.

[0043] More preferably, the CNN-LSTM fusion model includes:

[0044] The CNN module, consisting of two one-dimensional convolutional layers and a pooling layer, is used to receive time-frequency domain features and extract local time-frequency patterns.

[0045] The LSTM module contains 64 hidden units and is used to receive the output of the CNN module and output the fatigue recognition result.

[0046] Secondly, this invention provides a fatigue recognition model training system based on three-lead EEG, comprising the following modules:

[0047] The data acquisition module is used to collect three-lead EEG data and electrocardiogram data of the subjects. The three-lead EEG data includes FP1 EEG data, O1 EEG data and C3 EEG data.

[0048] The data annotation module is used to annotate the fatigue level of three-lead EEG data based on the subjects' subjective reports and ECG data;

[0049] The feature extraction module is used to divide the labeled three-lead EEG data into multiple data segments according to time windows, and extract time-series features, frequency domain features and time-frequency domain features for each data segment.

[0050] The model training module is used to train a lightweight LSTM model using temporal features, an XGBoost model using frequency domain features, and a CNN-LSTM fusion model using time-frequency domain features.

[0051] Furthermore, this invention also provides a fatigue level recognition method based on three-lead EEG, comprising the following steps:

[0052] Collect the user's three-lead EEG data, which includes FP1 EEG data, O1 EEG data and C3 EEG data;

[0053] The collected three-lead EEG data is inferred using any one of the lightweight LSTM model, XGBoost model, and CNN-LSTM fusion model trained by the fatigue recognition model training method, and the user fatigue recognition result is output.

[0054] Finally, this invention provides a fatigue recognition system based on three-lead EEG, comprising the following modules:

[0055] A portable data acquisition module, including a three-lead EEG head-mounted device with electrodes located at FP1, O1 and C3, is used to acquire three-lead EEG data from the user.

[0056] An embedded processing module is integrated into the three-lead EEG head-mounted device and is equipped with a lightweight LSTM model trained by the fatigue recognition model training method. This model receives three-lead EEG data transmitted from the portable data acquisition module and outputs fatigue recognition results after inference by the lightweight LSTM model.

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] 1. This invention achieves a high information capture rate with limited leads, acquiring over 80% of the fatigue-related feature information of traditional multi-lead systems through three leads. It fundamentally solves the industry problem of incomplete feature extraction and low model performance caused by the limited number of channels in portable devices. Combined with a trained inference model, it makes it possible for workers in high-risk positions to routinely wear these devices for fatigue monitoring.

[0059] 2. This invention constructs a complete feature system characterizing the dynamic evolution of fatigue. Through triple features—original time series, global frequency domain, and local time-frequency—it describes the fatigue state from three different dimensions: waveform details, steady-state rhythms, and dynamic evolution, overcoming the limitations of single-type feature representation. In particular, the wavelet time-frequency feature can clearly reveal the occurrence, duration, and decay of fatigue-related rhythms along the time axis.

[0060] 3. This invention achieves the optimal matching design of model and scenario, deploys a lightweight LSTM model suitable for real-time inference on a front-end portable device for real-time fatigue inference, deploys a CNN-LSTM fusion model with higher accuracy and time-frequency information on the server side, and provides an API service interface for fatigue recognition and classification to third parties, which greatly expands its application scenarios and reduces the cost of implementation and application. Attached Figure Description

[0061] Figure 1 This is a flowchart of the fatigue recognition model training method of the present invention;

[0062] Figure 2 The original waveform of the three-lead EEG of this invention Figure 1 ;

[0063] Figure 3 The original waveform of the three-lead EEG of this invention Figure 2 ;

[0064] Figure 4 This is a normalized graph showing the fatigue and characteristics of the present invention.

[0065] Figure 5 These are waveform feature diagrams for the five frequency bands of this invention;

[0066] Figure 6 The diagram shows the waveform characteristics and fatigue reasoning results of the five frequency bands of this invention.

[0067] Figure 7 The present invention contains / and / The fatigue inference results of the indicators are shown in the figure. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] In the description of this invention, it should be noted that the terms "upper," "lower," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, the terms "installation" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0070] Example 1

[0071] like Figure 1 As shown, this embodiment provides a method for training a fatigue recognition model based on three-lead EEG, including the following steps:

[0072] The subjects' three-lead EEG and ECG data were collected, including FP1, O1, and C3 EEG data.

[0073] Fatigue levels were annotated using three-lead EEG data based on subjects' subjective reports and ECG data;

[0074] The labeled three-lead EEG data were divided into multiple data segments according to time windows, and time-series features, frequency domain features, and time-frequency domain features were extracted for each data segment.

[0075] Lightweight LSTM models are trained using temporal features, XGBoost models are trained using frequency domain features, and CNN-LSTM fusion models are trained using time-frequency domain features.

[0076] Specifically, the training method for this fatigue recognition model includes the following steps:

[0077] 1. EEG data of subjects were collected using a three-lead head-mounted device.

[0078] This embodiment uses a wireless three-lead EEG head-mounted device, powered by a rechargeable lithium battery. Its key parameters are as follows: sampling frequency 256Hz, input impedance greater than 100MΩ, common-mode rejection ratio (CMRR) greater than 100dB, and support for Bluetooth 5.0 or higher wireless transmission.

[0079] The three electrodes of the three-lead system are precisely fixed in the corresponding positions of the head-mounted device through a flexible arm. Specifically, the three electrodes are selected in the core brain regions of FP1 (left prefrontal cortex), O1 (left occipital cortex) and C3 (left central region), and are set in strict accordance with the standard positions of the international 10-20 system.

[0080] FP1 is located in the left prefrontal cortex, which is closely related to higher cognitive control and attention maintenance. In a state of fatigue, theta wave (4-8 Hz) activity in this area is significantly enhanced, making it a core indicator of mental relaxation and drowsiness.

[0081] O1 is located in the left occipital lobe, a major region of the visual cortex. Alpha wave (8-13 Hz) activity is most pronounced in this region when the eyes are closed or during periods of fatigue. Fatigue can lead to alpha wave rhythm disturbances, peak frequency shifts, or abnormal power variations.

[0082] C3 is located in the left central region, corresponding to the sensorimotor cortex. This area is associated with physical preparation and execution of movement, and its beta wave (13-30 Hz) activity is positively correlated with the body's alertness and activation levels. Beta wave power typically decreases during fatigue.

[0083] The FP1-O1-C3 combination forms a triangle spatially covering the anterior, middle, and posterior brain regions, enabling simultaneous monitoring of the three most fatigue-related rhythm changes with a minimal number of channels. The reference electrode is located at the mastoid process behind the ear, and the ground electrode is located at the center of the forehead (near Fpz).

[0084] In addition, a medical-grade portable heart rate monitor (such as Polar H10) was used to simultaneously acquire the subjects' electrocardiogram (ECG) signals for subsequent calculation of heart rate variability (HRV). HRV is an objective indicator for assessing autonomic nervous system function and reflecting physiological fatigue.

[0085] 2. Design data acquisition experiments and data acquisition procedures.

[0086] For the target profession (such as professional driver), 50 to 100 healthy participants with no history of neuropsychiatric disorders will be recruited. Participants will be required to abstain from caffeine, alcohol, or other substances that affect the central nervous system for 24 hours prior to the experiment and maintain a normal daily routine.

[0087] Each subject underwent continuous work sessions lasting 4 to 6 hours, with EEG signals collected throughout the entire cycle of "awake → mild fatigue → severe fatigue" in real or simulated scenarios, and ECG signals were collected simultaneously. For example, this could involve prolonged driving on a highway in a real-world environment.

[0088] When collecting EEG and ECG signals from the subjects, they periodically reported on a fatigue self-assessment scale, for example, every 30 minutes. The fatigue self-assessment scale includes three items: "No drowsiness" (corresponding to "normal"), "Somewhat drowsy but able to persevere" (corresponding to "fatigue"), and "Very drowsy, unable to stay awake" (corresponding to "dangerous"). Subjects selected from these options based on their own situation.

[0089] 2. Preprocess the collected three-lead EEG and ECG data.

[0090] The raw EEG data collected from each subject, i.e., the time-series data of the three leads, were preprocessed separately. The EEG data preprocessing included the following steps:

[0091] 2.1 Bandpass filtering: Baseline drift (low frequency) and ultra-high frequency noise are removed using a zero-phase digital bandpass filter. In this embodiment, an FIR filter ranging from 0.5 Hz (high-pass) to 100 Hz (low-pass) is used. This step aims to eliminate ultra-low frequency drift (<0.5 Hz) caused by sweating or breathing, as well as unwanted high-frequency noise above 100 Hz.

[0092] 2.2 Power Frequency Notch Filtering: A narrowband notch filter is used to eliminate power frequency interference and its harmonics from the mains power supply, while minimizing the impact on adjacent frequency bands. The notch filter's attenuation depth is no less than 40dB, its bandwidth does not exceed 2Hz, and the frequency is set according to the actual local power grid frequency. In this embodiment, a 50Hz or 60Hz notch filter is used.

[0093] 2.3 EOG Removal in Lead FP1: Independent Component Analysis (ICA) is used to remove EOG from the filtered Lead FP1 data. This embodiment uses the FastICA algorithm for ICA. The correlation coefficient between each isolated independent component and a preset EOG template is calculated, and the 1-2 components with the highest correlation coefficients are identified as EOG components. After setting these components to zero, the FP1 channel signal is reconstructed. This step has no effect on the Lead O1 and Lead C3 data.

[0094] 2.4 A moving average filter is applied to lead C3. The C3 electrode is located on a protruding area at the top of the head, making it susceptible to poor contact due to hair obstruction, sweat, or slight head movements, resulting in instantaneous changes in resistance and sudden spikes. A moving average filter is applied to the filtered C3 lead signal to effectively smooth out such sudden noise while preserving the main EEG rhythm characteristics. In this embodiment, the moving window is set to 0.1 seconds, with 25 points at a sampling frequency of 256Hz.

[0095] 2.5 Removing invalid data from three-lead EEG data based on signal quality index

[0096] Define the Signal Quality Index (SQI), set the sliding window parameters, and calculate the SQI for each sliding window by iterating through the three leads of EEG data processed in the above steps, using the sliding window as a unit. The expression for this SQI is:

[0097]

[0098] in, This represents the signal quality index based on a time window. and Indicates weight, This indicates that within this sliding window, the voltage amplitude of any lead's EEG signal exceeds... The proportion of sampling points in the range to the total number of sampling points. This indicates that within this sliding window, the standard deviation of the EEG signal voltage is greater than [value missing]. The proportion of leads in the three leads.

[0099] in, and These represent the minimum and maximum values ​​of the standard deviation of the EEG signal voltage in a certain lead, respectively. This represents the mean standard deviation of the voltage of the three-lead EEG signal. Used to detect artifacts and electrode detachment caused by large vibrations. Used to detect high-frequency electromyographic noise or persistent interference.

[0100] In this embodiment, the sliding window is set to 1 second (256 sampling points). It is 0.6. It is 0.4. (Three leads) and Both are -80 and 80 Three leads Both are 40 In practical applications, each lead's , and Configure according to the actual scenario.

[0101] Furthermore, set a quality threshold. The signal quality index calculated for each sliding window is compared with a quality threshold. If... If the current sliding window is deemed invalid, it is marked and removed to prevent it from entering subsequent processing stages. In this embodiment, an empirical value is used to set the data. It is 0.8.

[0102] If the signal quality index of any lead within a sliding window is less than the quality threshold, then all three lead data points corresponding to that sliding window are discarded. Furthermore, for any consecutive 2-second data segment, if any 1-second sub-window within it is marked as invalid, the entire 2-second data segment is considered invalid data.

[0103] 3. Fatigue level labeling was performed on preprocessed EEG data based on subjects' subjective reports and heart rate variability indices.

[0104] In this embodiment, 30 minutes is used as the basic annotation unit, the fatigue self-assessment form fed back by the subjects within the annotation unit is used as input, and the fatigue level is annotated by referring to the heart rate variability index calculated based on electrocardiogram data.

[0105] Extract all normal sinus beat (NN) interval sequences from the electrocardiogram signals within the time period corresponding to the labeled unit, and further calculate the standard deviation of the normal sinus beat (NN) interval (SDNN index) to represent heart rate variability (HRV).

[0106] In this embodiment, the correspondence between the SDNN index and the fatigue state is set as follows: when SDNN > 80 ms, it is judged as "normal"; when 60 ms ≤ SDNN ≤ 80 ms, it is judged as "fatigue"; and when SDNN < 60 ms, it is judged as "dangerous".

[0107] When the fatigue self-assessment scale results reported by the subjects are inconsistent with the HRV index results, the data in the labeled cell is marked and removed, or arbitration is conducted through video of the subjects' behavior during that period to determine the final fatigue level.

[0108] Once the fatigue level of the labeled unit is determined, all valid three-lead EEG data within that time period are labeled with that unified fatigue level label. The specific fatigue level labels are set as follows: 0 indicates normal, 1 indicates fatigue, and 2 indicates danger.

[0109] 4. Cluster-based proactive data filtering and dataset partitioning

[0110] For all preprocessed and labeled valid three-lead EEG data, frequency domain features are rapidly extracted, and then a clustering algorithm is used to automatically identify high-density regions and outliers. After clustering, the data is divided into several core clusters, as well as boundary points and noise points. This embodiment uses the DBSCAN clustering algorithm.

[0111] Discard all samples marked as "noise points" by the DBSCAN algorithm, as well as "boundary point" samples located between any two core clusters of different fatigue levels (defined by calculating the ratio of the distances from the sample to the nearest two different cluster centers).

[0112] Multiple experiments have shown that such ambiguous samples account for approximately 20% of the total. Discarding them can purify the dataset, allowing the model to focus more on learning the clearest patterns for each category, which helps improve the clarity of decision boundaries and the model's generalization ability.

[0113] For the remaining 80% of "clear" data, stratified sampling was used to randomly divide the data into training, validation, and test sets in a ratio of 64%:16%:20%. During the partitioning, it was ensured that the sample proportions of the "normal," "fatigue," and "dangerous" categories in the three subsets were approximately consistent with the proportions in the entire set.

[0114] 5. Perform multi-class feature extraction on the labeled full three-lead EEG data.

[0115] The raw, three-lead EEG data with labels was segmented by lead according to a set time window, resulting in multiple data segments of uniform length. The size of the time window was determined based on a combination of factors, including the heart rate during data transmission from the three-lead head-mounted device, network bandwidth, and data packet size.

[0116] In this embodiment, the three-lead head-mounted device used for data acquisition uploads a data packet every 2 seconds, therefore the time window is set to 2 seconds. Each data segment after this process contains 512 sampling points (sampling frequency of 256Hz). Assuming there are N data segments after segmentation, each segment is 2 seconds × 3 leads, i.e., a 512 × 3 dimensional matrix.

[0117] 5.1 Standardize the data segments to extract time-series features

[0118] Each 512×3 matrix is ​​Z-score normalized along the time axis to eliminate individual amplitude differences. Ultimately, all samples constitute a three-dimensional temporal feature tensor X_time with dimensions (N, 512, 3). The first dimension is the sample index, i.e., the data segment index; the second dimension is the 512 time steps, i.e., the product of the sampling frequency and the time window; and the third dimension represents the three leads (FP1, O1, C3). This three-dimensional temporal feature tensor fully preserves the temporal waveform information of the EEG signal.

[0119] 5.2 Extract frequency domain features by performing Fourier transform on the data segment

[0120] For each time-series data segment, a Fourier transform is performed. This invention uses the Welch method to perform the Fourier transform and calculate the power spectral density (PSD) of the EEG signal. The parameters are set as follows: the 512 sampling points are divided into 4 sub-segments, each with 128 points, with 50% overlap, and a Hamming window is used to generate the power spectral density of the signal.

[0121] Then, on the generated power spectral density curve, five standard frequency bands were analyzed. (0.5-4Hz) (4-8Hz) (8-13Hz) (13-30Hz) Numerical integration was performed within the range of 30-100Hz to calculate the corresponding absolute power. To further eliminate individual and global power differences, the relative power of each frequency band (i.e., the ratio of single-band power to total band power) was calculated.

[0122] For each frequency band, the arithmetic mean of the relative powers calculated from the three leads of that data band is taken to obtain a 5-dimensional feature vector for that data band. ,in , , , and These represent the data segments respectively. , , , and The relative power of the frequency bands. Finally, all samples constitute a (N, 5) two-dimensional frequency domain feature tensor X_freq, where N represents the sample index and 5 represents the relative power of the five frequency bands described for that sample.

[0123] 5.3 Perform wavelet transform on the data segment to extract time-frequency domain features

[0124] For each time-series data segment, a continuous wavelet transform (CWT) is performed. This invention uses complex Morlet wavelets for the CWT. Three different wavelet scales are selected, with corresponding center frequencies of 6Hz (center of the theta wave band), 10Hz (center of the alpha wave band), and 20Hz (center of the beta wave band). By adjusting the scale, the wavelet transform can provide optimal time-frequency focusing at different frequencies.

[0125] Extract the time-frequency energy matrix, and take the square of the modulus of the wavelet coefficients for each lead, each time series data segment, and each scale to obtain the time-frequency energy distribution E(3, 3, 512) of that lead at that scale. The first dimension represents the 3 leads, the second dimension represents the 3 wavelet scales, and the third dimension represents the 512 time steps, which is the product of the sampling frequency and the time window.

[0126] Time warping and frequency band fusion were performed to reduce data dimensionality and adapt to model input, aggregating 512 sampling points into 30 time frames. Specifically, the 2-second time window was divided into 30 different frames, each approximately 66.7 milliseconds (approximately 17 original sampling points). The average energy at each lead and each scale within each frame was calculated to obtain a feature matrix (3 channels, 3 scales, 30 time frames).

[0127] Construct a 12-dimensional time-frequency feature vector and map the three wavelet scales to... , , and Four frequency bands. Scale 1 (high-frequency end) corresponds to... Band, scale 2 corresponding Band, Scale 3 (low frequency end) corresponds to Band.

[0128] The frequency band energy can be approximated by additional calculations at the low-frequency end of scale 3 or by using a separate low-pass filter channel. For simplicity, this embodiment contributes the energy of scale 3 simultaneously to... frequency band and Frequency band.

[0129] Finally, for each time frame, a 12-dimensional feature vector is obtained, which can be represented as:

[0130]

[0131] Each dimension represents a specific lead (FP1, O1, C3) in a specific frequency band. , , , The average energy in that time frame.

[0132] For a time-series data segment, the feature matrices of all 30 time frames are concatenated in chronological order to obtain a (30, 12) matrix. All N samples constitute a (N, 30, 12) three-dimensional time-frequency domain feature tensor X_timefreq, where the first dimension is the sample index, i.e., the data segment index, the second dimension represents the time frame, and the third dimension represents the 12-dimensional average energy feature vector composed of the three leads and four frequency bands. This feature tensor perfectly characterizes the dynamic process of the evolution of different rhythmic energies in different brain regions over time within 2 seconds.

[0133] 6. Train different inference models for the extracted feature tensors of different categories.

[0134] Different models are selected and trained based on the characteristics of different use cases and deployment goals. All models use the same training and validation sets, and the loss function is selected as either cross-entropy loss or mean squared error loss depending on the task. The optimizer used is Adam.

[0135] 6.1. For the X_time 3D temporal feature tensor, a lightweight unidirectional LSTM algorithm is adopted, suitable for real-time front-end classification. The model input is (512, 3), the network structure is an LSTM layer with 32 hidden units, followed by a Dropout layer with a Dropout rate of 0.2, and finally a fully connected layer with 3 units connected to a Softmax activation function. The output is the probability of three fatigue levels, and the model parameters are approximately 5000. The training objective is to minimize the classification cross-entropy loss.

[0136] 6.2. For the X_freq two-dimensional frequency domain feature tensor, the XGBoost classifier is used, suitable for offline task analysis on the server side. The XGBoost classifier takes 5-dimensional features as input. The hyperparameters are set as follows: number of trees 100, maximum depth 5, and learning rate 0.1. This model trains quickly and can output feature importance (e.g., displaying...). and (Wave ratio), suitable for post-accident debriefing analysis.

[0137] 6.3. For the X_timefreq three-dimensional time-frequency domain feature tensor, a CNN-LSTM fusion algorithm is adopted, which is suitable for real-time monitoring and analysis on the server side. The input of the CNN-LSTM fusion algorithm is (30, 12).

[0138] The CNN part first treats the input as 30 time steps, with 12 feature channels per step. It uses two one-dimensional convolutional layers: the first layer has a kernel size of 3 and 16 output channels, activated by ReLU; the second layer has a kernel size of 3 and 32 output channels, activated by ReLU.

[0139] Each convolutional layer is followed by a max-pooling layer (pooling size 2). After passing through the CNN, the data is compressed and local time-frequency patterns are extracted. Then, the CNN output is flattened and reshaped into a new sequence, which is then input into an LSTM.

[0140] The LSTM part is an LSTM layer with 64 hidden units, used to capture the temporal dependencies of high-level features encoded by CNN. The output of the last time step of the LSTM is mapped to 3D through a fully connected layer, followed by a Softmax output probability. The model has approximately 20,000 parameters and can achieve millisecond-level inference on a server.

[0141] 7. Perform multidimensional performance evaluation on each trained model.

[0142] The trained models were comprehensively evaluated using the aforementioned test set. Model evaluation consisted of three dimensions, all of which had to be met:

[0143] Basic performance dimensions include a classification accuracy of ≥90%, a macro average F1 score of ≥0.88, and a Cohen's Kappa coefficient of ≥0.85. Furthermore, a confusion matrix is ​​used to ensure a recall rate of ≥97% for the "dangerous" category (i.e., a false negative rate of less than 3%), and the highest possible precision for the "normal" category to minimize false alarms.

[0144] Among them, classification accuracy is defined as the proportion of the number of samples correctly predicted by the model to the total number of samples; when calculating the macro average F1 score, the F1 score (the harmonic mean of precision and recall) of each lead is calculated first, and then its arithmetic mean is calculated; the Cohen-Kappa coefficient represents the consistency between the model prediction results and the actual results.

[0145] Generalization ability dimension: includes cross-pass validation and noise robustness testing. Cross-pass validation: testing is performed on the validation set data reserved during data partitioning, requiring that the absolute difference between its accuracy and the test set accuracy is less than 5%.

[0146] Noise robustness test: Add Gaussian white noise with a signal-to-noise ratio (SNR) of 10% to 15% to the original EEG signals of all samples in the test set, re-extract features, and pass the model. The accuracy reduction after adding noise should be less than 8%.

[0147] Real-time performance dimension: The lightweight LSTM model is converted and quantized (INT8) and deployed to the target embedded platform. The test starts with reading a 2-second segment of raw data, completing all data preprocessing and model inference, and the total time (end-to-end latency) to outputting the classification result. This latency must be less than 500ms.

[0148] Iterative optimization path: If any dimension's metrics fail to meet the target, optimize according to the following priorities and path, and retrain and evaluate the model:

[0149] Feature and data level optimization: Adjust wavelet transform parameters (such as scale number, center frequency of wavelet basis), fine-tune data cleaning thresholds (such as quality threshold). The value was reduced from 0.8 to 0.75 to retain more data, but caution is needed regarding the introduction of noise.

[0150] Model structure optimization: For lightweight LSTM models, try reducing the number of hidden units from 32 to 24 or 16; for CNN-LSTM models, reduce the number of convolutional layers or channels.

[0151] Example 2

[0152] like Figure 1 As shown, this embodiment provides a fatigue recognition model training system based on three-lead EEG, including the following modules:

[0153] The data acquisition module is used to collect three-lead EEG data and electrocardiogram data of the subjects. The three-lead EEG data includes FP1 EEG data, O1 EEG data and C3 EEG data.

[0154] The data preprocessing module is used to preprocess the three-lead EEG data, including bandpass filtering, power frequency notch filtering, removal of ocular artifacts from FP1 EEG data, moving average filtering of C3 EEG data, and removal of invalid data from the three-lead EEG data processed in the aforementioned steps based on the signal quality index.

[0155] The data annotation module is used to annotate the fatigue level of three-lead EEG data based on the subjects' subjective reports and ECG data;

[0156] The feature extraction module is used to divide the labeled three-lead EEG data into multiple data segments according to time windows, and extract time-series features, frequency domain features and time-frequency domain features for each data segment.

[0157] The model training module is used to train a lightweight LSTM model using temporal features, an XGBoost model using frequency domain features, and a CNN-LSTM fusion model using time-frequency domain features.

[0158] Example 3

[0159] like Figure 1 As shown, this embodiment provides a fatigue level recognition method based on three-lead EEG, including the following steps:

[0160] Collect the user's three-lead EEG data, which includes FP1 EEG data, O1 EEG data and C3 EEG data;

[0161] The collected three-lead EEG data were preprocessed to improve data quality and remove impurities.

[0162] After the preprocessed three-lead EEG data is trained and optimized using the fatigue recognition model training method described in Example 1, any one of the following models—the lightweight LSTM model, the XGBoost model, and the CNN-LSTM fusion model—is used for inference, the user fatigue recognition result is output.

[0163] Example 4

[0164] like Figure 1 As shown, this embodiment provides a fatigue recognition system based on three-lead EEG, including the following steps:

[0165] A portable data acquisition module, including a three-lead EEG head-mounted device with electrodes located at FP1, O1 and C3, is used to acquire three-lead EEG data from the user.

[0166] An embedded processing module is integrated into the three-lead EEG head-mounted device and is equipped with a lightweight LSTM model trained and optimized by the fatigue recognition model training method described in Example 1. The model is used to receive three-lead EEG data transmitted by the portable data acquisition module, and then output the fatigue recognition result after inference by the lightweight LSTM model.

[0167] The remote management module includes an enterprise server that has deployed an XGBoost model or a CNN-LSTM fusion model trained and optimized by the fatigue recognition model training method described in Example 1. The server is used to receive EEG data uploaded by the portable data acquisition module and output fatigue recognition results after accurate inference by the model.

[0168] More preferably, it may also include a cloud service module, including a cloud server that deploys a CNN-LSTM fusion model trained and optimized by the fatigue recognition model training method described in Example 1, for providing fatigue recognition and classification services based on three-lead EEG data to third parties in a service interface mode.

[0169] Specifically, the model trained and optimized according to the method described in Example 1 is deployed to different computing nodes based on its characteristics and performance to form a collaborative system.

[0170] Front-end embedded device deployment: Deploy a lightweight LSTM binary classification model quantized with INT8 (merging "fatigue" and "danger" into the "fatigue" state), or in extreme cases, deploy a logistic regression model.

[0171] The model is deployed on the microprocessor of a three-lead head-mounted device (such as an ARM Cortex-A7 embedded chip). The device collects the user's EEG signals in real time, performs data filtering and standardization on the chip, and then calls the local model in real time to perform an inference.

[0172] If the inference result is "fatigue" three times consecutively, a local warning is triggered (e.g., a flashing red LED, a short buzzer, or a micro-motor vibration) to remind the user to perk up. Raw data or features are packaged, encrypted, and wirelessly transmitted to the server at regular intervals (e.g., every minute). End-to-end latency is <300ms.

[0173] Server-side job clustering model deployment, deploying pre-trained XGBoost or CNN-LSTM fusion models. It can receive EEG data uploaded from multiple front-end devices. Running a more accurate model, it performs fine-grained classification of each person as "normal / fatigued / dangerous".

[0174] Furthermore, cluster analysis can be performed on the fatigue status of all drivers in the same position (such as the entire fleet) to generate a heatmap of group fatigue and statistical reports (such as "At 3 pm today, 30% of drivers are fatigued"). Finally, early warning information and statistical reports are pushed to the monitoring screen of management personnel to support management decisions.

[0175] Server-side general-purpose model deployment, featuring high-performance general-purpose CNN-LSTM fusion models or lightweight Transformer models. Providing a fatigue recognition API in the form of Model-as-a-Service (MaaS). Supporting calls from different client terminals across various industries, outputting recognition results in standard JSON format, and providing long-term historical fatigue trend analysis reports.

[0176] Example 5

[0177] like Figures 1 to 7 As shown, this embodiment uses a portable three-lead EEG head-mounted device, with the three electrodes located at the standard positions of FP1, O1, and C3. The three-lead EEG head-mounted device is equipped with a lightweight LSTM model trained and optimized to meet the standards by the method described in Embodiment 1.

[0178] Select frontline workers in high-risk industries, such as long-distance transport drivers, and have them wear and turn on the aforementioned three-lead EEG headset during their work to collect three-lead data from the user during work. After inference by the aforementioned lightweight LSTM model, the fatigue recognition result is output.

[0179] A remote management server is established, and multiple three-lead EEG headsets communicate with the remote management server via wireless network technology. The three-lead EEG headsets upload the collected real-time three-lead EEG data of the users to the remote management server according to preset time windows.

[0180] The remote management server is equipped with the CNN-LSTM fusion model trained and optimized according to the method described in Example 1. The received three-lead EEG data is transmitted to the CNN-LSTM fusion model for inference, and finally the fatigue recognition result is output for remote monitoring and management.

[0181] The evolution of user fatigue was observed using a remote management server, as shown in the attached figure. Figure 2 The image shows three-lead EEG data collected from a user at a specific time period. The horizontal axis represents time in milliseconds, and the vertical axis represents voltage in microvolts. In the image, the top section shows the EEG waveform of lead FP1, the middle section shows the EEG waveform of lead O1, and the bottom section shows the EEG waveform of lead C3. As can be seen from the image, the three-lead EEG waveforms of the user are changing normally, and the voltage amplitude is also fluctuating within the normal range.

[0182] As attached Figure 3 The image shows three-lead EEG data collected from another user during a specific time period. As can be seen from the figure, the EEG waveform in lead O1 is abnormal at most sampling points, with sudden, extremely large voltage peaks appearing at multiple sampling points, while the waveform returns to normal fluctuations in the final time period. This suggests that the abnormal waveform in lead O1 may be due to the user adjusting the three-lead head-mounted device. This abnormal data will be filtered out using a filter or removed based on signal quality index.

[0183] To better demonstrate the fatigue recognition and inference performance of the CNN-LSTM fusion model, the preset fatigue index was first normalized to the range [0,1]. (See attached image) Figure 4 As shown, the horizontal axis represents time, and the vertical axis represents the normalized value of the fatigue index. The fatigue index is divided into low fatigue threshold, medium fatigue threshold, and high fatigue threshold.

[0184] As attached Figure 5 As shown, the collected three-lead EEG data of a user was preprocessed, feature extracted, mapped to five frequency bands, and then normalized to the range [0,1]. It can be seen that... , , The characteristic changes of the three frequency bands are quite consistent. and The frequency band characteristics show relatively consistent changes. However, there are two significant fluctuations in the waveform, which may be due to changes in the collected data caused by user head movement.

[0185] Based on the appendix Figure 5 The fatigue index, derived from the data through a CNN-LSTM fusion model and normalized to the range [0,1], is shown in the attached figure. Figure 6 As shown in the figure, the real-time fatigue index of the inference is basically consistent with the waveform changes of the three-lead frequency band, indicating that this fusion model can accurately infer the user's fatigue index based on the three-lead EEG data.

[0186] Furthermore, the parameters of the CNN-LSTM fusion model also integrate / ratio, and / The two indicators, the ratio, are shown in the appendix. Figure 7 As shown. / The ratio index is used to distinguish between "depth of fatigue" and "active relaxation" and to accurately determine whether fatigue is accompanied by "drowsy tendency". Its ratio directly reflects the balance between alertness and relaxation and drowsiness.

[0187] / The ratio index is used to quantify "alertness" and "fatigue resistance," and more accurately reflects the conflict between "awake drive" and "fatigue drive." It aligns more with the intuition that "the higher the value, the more awake one is," making it suitable for "preventive fatigue detection."

[0188] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0189] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for training a fatigue degree recognition model based on three-lead electroencephalogram, characterized in that, Includes the following steps: The subjects' three-lead EEG and ECG data were collected, including FP1, O1, and C3 EEG data. Fatigue levels were annotated using three-lead EEG data based on subjects' subjective reports and ECG data; The labeled three-lead EEG data were divided into multiple data segments according to time windows, and time-series features, frequency domain features, and time-frequency domain features were extracted for each data segment. Lightweight LSTM models are trained using temporal features, XGBoost models are trained using frequency domain features, and CNN-LSTM fusion models are trained using time-frequency domain features.

2. The method of claim 1, wherein the method comprises: The three-lead EEG data were preprocessed before annotation. The specific method was as follows: The three-lead EEG data were subjected to bandpass filtering and power frequency notch filtering respectively to obtain the filtered data. Independent component analysis was used to separate and remove the artifact components of electrooculography (EOG) in the filtered data to obtain the FP1 EEG data after EOG removal. The C3 EEG data in the filtered data were subjected to moving average filtering to obtain the processed C3 EEG data. Invalid data were removed from the FP1 EEG data after removing electrooculograms, the O1 EEG data in the filtered data, and the invalid data in the processed C3 EEG data.

3. The fatigue recognition model training method based on three-lead EEG according to claim 2, characterized in that, The method for removing invalid data from the FP1 EEG data after ophthalmoscopy removal, the O1 EEG data in the filtered data, and the invalid data in the processed C3 EEG data includes: The signal quality index within the sliding window is calculated by iterating through the FP1 EEG data after electrooculogram removal, the O1 EEG data in the filtered data, and the processed C3 EEG data in the sliding window. The signal quality index is compared with the quality threshold. If the signal quality index is less than the quality threshold, the data in the sliding window is invalid and is removed. The signal quality index is: in, Indicates the signal quality index. and Indicates weight, This indicates the proportion of sampling points within a time window where the voltage amplitude of any given EEG data exceeds the range of the voltage standard deviation of that EEG data. This indicates the proportion of the voltage standard deviation of the EEG data within the time window that is greater than the set mean voltage standard deviation of the EEG data.

4. The fatigue recognition model training method based on three-lead EEG according to claim 1, characterized in that, The method for labeling fatigue levels in three-lead EEG data based on subjects' subjective reports and ECG data includes: Subjects reported their subjective fatigue levels at different time intervals; Heart rate variability index is calculated using electrocardiogram data, and the objective fatigue level of the subject during that period is determined by the heart rate variability index. The fatigue level of the subjects during that period was determined and labeled based on subjective and objective fatigue levels.

5. The fatigue recognition model training method based on three-lead EEG according to claim 1, characterized in that, The method for extracting time-frequency domain features for each data segment is to perform a three-scale continuous wavelet transform using complex Morlet wavelets. The time-frequency domain features are three-dimensional time-frequency domain feature tensors, where the first dimension is the data segment index, the second dimension is the time frame, and the third dimension is the data from the three leads respectively. , , and Energy distribution across the four frequency bands.

6. The fatigue recognition model training method based on three-lead EEG according to claim 1, characterized in that, It also includes evaluating the basic metrics, generalization ability, and real-time performance of trained lightweight LSTM models, XGBoost models, and CNN-LSTM fusion models, and optimizing the models based on the evaluation results; The basic performance indicators include: classification accuracy, macro-average F1 score, Cohen-Kappa coefficient, and false negative rate for hazardous conditions. The generalization ability assessment includes cross-subject verification and noise immunity testing.

7. The fatigue recognition model training method based on three-lead EEG according to claim 1, characterized in that, The CNN-LSTM fusion model includes: The CNN module, consisting of two one-dimensional convolutional layers and a pooling layer, is used to receive time-frequency domain features and extract local time-frequency patterns. The LSTM module contains 64 hidden units and is used to receive the output of the CNN module and output the fatigue recognition result.

8. A fatigue recognition model training system based on three-lead EEG, characterized in that, Includes the following modules: The data acquisition module is used to collect three-lead EEG data and electrocardiogram data of the subjects. The three-lead EEG data includes FP1 EEG data, O1 EEG data and C3 EEG data. The data annotation module is used to annotate the fatigue level of three-lead EEG data based on the subjects' subjective reports and ECG data; The feature extraction module is used to divide the labeled three-lead EEG data into multiple data segments according to time windows, and extract time-series features, frequency domain features, and time-frequency domain features for each data segment. The model training module is used to train a lightweight LSTM model using temporal features, an XGBoost model using frequency domain features, and a CNN-LSTM fusion model using time-frequency domain features.

9. A fatigue level recognition method based on three-lead EEG, characterized in that, Includes the following steps: Collect the user's three-lead EEG data, which includes FP1 EEG data, O1 EEG data and C3 EEG data; After the collected three-lead EEG data is inferred by any one of the lightweight LSTM model, XGBoost model, and CNN-LSTM fusion model trained by the method described in claim 1, the user fatigue recognition result is output.

10. A fatigue recognition system based on three-lead EEG, characterized in that, Includes the following modules: A portable data acquisition module, including a three-lead EEG head-mounted device with electrodes located at FP1, O1 and C3, is used to acquire three-lead EEG data from the user. An embedded processing module is integrated into the three-lead EEG head-mounted device and is equipped with a lightweight LSTM model trained by the method described in claim 1. The model receives three-lead EEG data transmitted by a portable data acquisition module and outputs fatigue recognition results after inference by the lightweight LSTM model.

Images