Special equipment operator electroencephalogram fatigue detection method based on sample compensation and hybrid model

Abstract

The invention discloses a special equipment operator electroencephalogram fatigue detection method based on sample compensation and a hybrid model, and the method comprises the steps: collecting an electroencephalogram signal of a special equipment operator, and carrying out the preprocessing of the electroencephalogram signal, and obtaining a preprocessed electroencephalogram signal; performing multi-domain feature extraction and fusion on the preprocessed electroencephalogram signals to obtain a multi-dimensional feature set; performing compensation processing on the electroencephalogram signal and the multi-dimensional feature set by adopting a sample compensation method to obtain a compensated sample feature set; wherein the sample compensation method comprises a characteristic level compensation method and a signal level compensation method; training a hybrid model through the compensated sample feature set to obtain a trained hybrid model; wherein the hybrid model comprises a deep learning model and a traditional machine learning model; and performing electroencephalogram fatigue detection on the special equipment operator through the trained hybrid model to obtain a detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fatigue detection of special equipment operators, and particularly to a fatigue detection method for special equipment operators based on sample compensation and a hybrid model. BACKGROUND

[0002] In the field of special equipment operation, such as heavy machinery operation, high-altitude operation, and long-distance transportation, the cognitive state of the operator, especially the degree of fatigue, is a key factor in ensuring the safety of the operation. The decline in alertness and the slowness of reaction caused by fatigue are one of the main causes of major safety accidents. Electroencephalogram (EEG) as a physiological monitoring technology that can non-invasively and real-time reflect the neural activity of the cerebral cortex provides a direct neurophysiological basis for objective and quantitative assessment of fatigue state.

[0003] However, the existing EEG-based fatigue detection methods still face the following technical bottlenecks: 1. Limitation of feature representation: Traditional fatigue detection methods mostly rely on single-dimensional feature extraction, such as analyzing only the power spectral density of specific frequency bands such as alpha waves and theta waves or simple time-domain statistics. Such methods ignore the complex dynamic information contained in the EEG signal, such as the nonlinear dynamic characteristics of the signal, the functional connectivity across channels, and the spatiotemporal dynamic evolution pattern. This one-sidedness of feature extraction leads to insufficient representation of the complex process of brain functional network reconstruction under fatigue state, limiting the upper limit of the accuracy of the detection model.

[0004] 2. Intrinsic imbalance of data distribution: In actual acquisition scenarios, due to the natural distribution of physiological states, the amount of available data samples in the wakeful state is usually much larger than that in the fatigue state. This inherent class imbalance problem leads the model to tend to the majority class samples during training, resulting in low sensitivity to key fatigue state recognition, and seriously affecting the generalization ability of the model and its reliability in actual application.

[0005] 3. Generalization performance bottleneck of single classification model: The classification models currently used, whether traditional machine learning models or deep learning networks, all have their own limitations. Traditional methods rely on expert knowledge for manual feature design, which has poor adaptability; while deep learning models can automatically learn features, but are prone to overfitting on small samples and strong individual differences in EEG data, resulting in poor robustness and generalization performance of the model under different individuals or different test environments.

[0006] 4. The conflict between computational complexity and real-time performance: In order to pursue higher detection accuracy, existing research often uses complex feature engineering and deep network structure, which brings high computational overhead. The significant computational delay generated in the model training and online inference stage becomes a key bottleneck for deploying this technology in resource-constrained embedded devices or wearable monitoring systems, which restricts its practical application in real-time early warning scenarios.

[0007] Therefore, developing a new fatigue detection method that can deeply integrate multi-dimensional neurophysiological features, effectively cope with data imbalance, and balance model accuracy, generalization ability and computational efficiency is a technical problem that needs to be solved in the field. SUMMARY

[0008] The present application provides a special equipment operator EEG fatigue detection method based on sample compensation and hybrid model to solve the problems of insufficient feature extraction, insufficient data imbalance processing, poor model robustness and generalization ability, and low computational efficiency in the prior art.

[0009] To achieve the above purpose, the present application provides a special equipment operator EEG fatigue detection method based on sample compensation and hybrid model, comprising: Collecting the EEG signals of the special equipment operator and preprocessing to obtain the preprocessed EEG signals; Extracting and fusing multi-domain features from the preprocessed EEG signals to obtain a multi-dimensional feature set; Compensating the EEG signals and the multi-dimensional feature set using a sample compensation method to obtain a compensated sample feature set; wherein the sample compensation method includes a feature-level compensation method and a signal-level compensation method; Training the hybrid model using the compensated sample feature set to obtain a trained hybrid model; wherein the hybrid model includes a deep learning model and a traditional machine learning model; Performing fatigue detection on the special equipment operator using the trained hybrid model to obtain the detection result.

[0010] Optionally, the preprocessing of the EEG signal includes: Collecting the EEG signal, filtering, denoising and segmenting the EEG signal to obtain the preprocessed EEG signal.

[0011] Optionally, the process of extracting and fusing multi-domain features from the preprocessed EEG signal includes: Extracting time domain features: quantifying the mean, standard deviation, root mean square, skewness, kurtosis, zero-crossing rate and activity of the preprocessed EEG signal to represent the statistical distribution and dynamic characteristics of the signal on the time axis; Frequency domain features were extracted: the average power, total power, spectral centroid, and peak frequency of the preprocessed EEG signal were calculated using Fast Fourier Transform. Extracting key fatigue ratio features: Calculating multiple sets of predefined inter-band power ratios that are highly correlated with fatigue state evolution; Spatial feature extraction: Statistical analysis of signal activity in specific brain regions such as the whole brain, frontal lobe, posterior region, and left and right hemispheres is performed to capture spatial distribution changes in brain activity under fatigue, such as the hemispheric asymmetry index. The calculation formula is as follows: in, P band,Right and P band,Left These represent the average band power calculated from selected electrode groups in the right and left hemispheres, respectively. This index can reveal the impact of fatigue on the functional dominance of the brain hemispheres. Extracting time-frequency features: Calculating time variability using the sliding window method. The calculation formula is as follows: in, F w It is in the w The feature values ​​calculated in each window μ F It is the mean of the feature sequence; this indicator reflects the stability of the brain state during task execution and is used to capture non-stationary dynamic changes caused by fatigue. A multidimensional feature set is obtained by combining the extracted time-domain features, frequency-domain features, fatigue key ratio features, spatial features, and time-frequency features.

[0012] Optionally, the process of compensating EEG signals and multidimensional feature sets using sample compensation methods includes: Quantitative analysis was performed on EEG signals and multidimensional feature sets to obtain minority class EEG signals and minority class multidimensional feature sets. Noise enhancement and improvement are performed on minority class EEG signals using a signal-level compensation method to obtain compensated EEG signal samples. The calculation formula is as follows: in, x synthetic To compensate for the EEG signal samples, x i EEG signals, ϵ Random noise conforming to a specific distribution; The compensated EEG signal samples are preprocessed and multi-domain features are extracted to obtain signal compensation samples; The minority class oversampling based on linear interpolation is performed on the minority class multidimensional feature set using a feature-level compensation method to obtain feature-compensated samples. The calculation formula is as follows: in, x new For feature compensation samples, x i For minority class samples, x j and x i A neighboring sample, λ It is a random number in the range [0, 1].

[0013] The feature-compensated samples, signal-compensated samples, and multidimensional feature sets are combined to obtain the compensated sample feature set.

[0014] Optionally, the training process for the hybrid model includes: The convolutional neural network model and the heterogeneous ensemble model are subjected to 10-fold cross-validation using the compensated sample feature set. The probability outputs of the two models are then integrated to generate the final output of the hybrid model and complete the model training, resulting in the trained hybrid model.

[0015] Optionally, the process of obtaining the heterogeneous integration model includes: The predictive contribution of each independent traditional machine learning classifier is determined by its performance-oriented weights. W i The weight is determined based on the overall performance metric of the traditional machine learning classifier on the cross-validation training set, which combines accuracy and F1 score. The formula for this metric is as follows: in, W i For the first i The weights of each classifier, Accuracy i Its accuracy on the training set, F1_score i Its F1 score on the training set, α This is the balancing coefficient. Then, all weights are normalized to ensure that the sum of the weights is 1, thereby focusing on the classifier with better performance.

[0016] Optionally, the process of integrating the probability outputs of the two types of models includes: The probability outputs of the deep learning model and the traditional machine learning model are integrated using a linear weighted fusion method, and the calculation formula is as follows: in,P hybrid is a final mixed prediction probability, P ML and P CNN are prediction probabilities of a traditional machine learning model and a convolutional neural network model, respectively, w ML and w CNN are corresponding preset weights, and w ML + w CNN = 1.

[0017] Optionally, the process of detecting the brain fatigue of the special equipment operator comprises the following steps: inputting the multi-dimensional feature set obtained by pre-processing and multi-dimensional feature extraction of the brain electrical signals of the special equipment operator into the trained mixed model to perform fatigue detection, and obtaining a detection result.

[0018] Compared with the prior art, the present application has the following advantages and technical effects: The present application comprehensively and deeply captures the complex information related to fatigue in the brain electrical signals through multi-dimensional feature engineering, significantly enhances the expression ability of the features, and thus effectively improves the accuracy of fatigue detection; the class imbalance problem commonly existing in the brain fatigue data is effectively solved through the sample compensation method, and the recognition accuracy and robustness of the model for the minority class samples such as fatigue are greatly improved; the advantages of the two types of models are complemented by constructing a mixed model combining the deep learning model and the traditional machine learning model, and higher detection accuracy, stability and generalization ability are provided through the performance-oriented fusion strategy; the model training and prediction time is significantly shortened through parallel computing, making it more suitable for real-time or quasi-real-time application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their description together with the drawings serve to explain the application. In the drawings: Figure 1 is a schematic diagram of the overall method flow of the embodiment of the present application; Figure 2 is a schematic diagram of the brain electrical signal preprocessing flow of the embodiment of the present application; Figure 3 is a schematic diagram of the multi-dimensional feature extraction flow of the brain electrical signal of the embodiment of the present application; Figure 4 is a schematic diagram of the sample compensation flow of the embodiment of the present application; Figure 5 is a schematic diagram of the mixed model training flow of the embodiment of the present application; Figure 6 The flowchart of the special equipment operator's EEG fatigue prediction method is shown in the figure. DETAILED DESCRIPTION

[0020] It should be noted that the embodiments in the present application and the technical features therein can be combined with each other to form other embodiments without conflict.

[0021] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings.

[0022] As shown in the figure, the special equipment operator's EEG fatigue detection method based on sample compensation and mixed model provided by the present application specifically includes the following steps: Figure 1 Step one: EEG signal acquisition A portable multi-channel EEG acquisition device is used to monitor the real-time EEG of the special equipment operator. The acquisition device is configured with not less than 8 electrode channels arranged in the frontal lobe, central and occipital lobe regions according to the international 10-20 system to capture the EEG activity related to fatigue. A fixed sampling rate of 256Hz or 512Hz is set during the acquisition process, and a low-impedance electrode configuration is used to ensure electrode contact stability, so as to continuously record the EEG signals of the operator under actual or simulated working conditions. The acquired EEG signals are sent in real time to the data processing end through a wireless transmission module or a wired connection method. As shown in the figure, step two: EEG signal preprocessing

[0023] The acquired EEG signals are filtered, denoised and segmented to obtain the preprocessed EEG signals. Specifically, it includes: Figure 2 The EEG signal is subjected to a 0.5Hz-45Hz band-pass filter to remove low-frequency drift and high-frequency noise and retain the effective EEG components related to fatigue state. The effective EEG components related to fatigue state are subjected to independent component analysis for denoising to eliminate interference components such as eye movement, electromyogram and motion artifacts, and obtain pure EEG signals.

[0024] The pure EEG signals are segmented by a sliding window with a window length of 0.5 seconds and an overlap rate of 20% to obtain the preprocessed EEG signals.

[0025] The pure EEG signals are segmented by a sliding window with a window length of 0.5 seconds and an overlap rate of 20% to obtain the preprocessed EEG signals.

[0026] As shown in the figure, step three: multi-dimensional feature extraction of EEG signal Figure 3 The preprocessed EEG signals are subjected to multi-domain feature extraction and fusion to obtain a comprehensive multi-dimensional feature set representing fatigue. Specifically, it includes: ​​The time domain, frequency domain, fatigue key ratio, space and time-frequency multi-dimensional features are extracted from the preprocessed electroencephalogram signals to reflect the fluctuation trend of electroencephalogram activity, frequency band energy distribution, fatigue sensitive index change and the synergistic relationship between brain regions in multiple levels, and a multi-dimensional feature set fully representing fatigue is obtained.

[0027] The multi-dimensional features extracted from the electroencephalogram signals include: Time domain features: the time distribution of the signal is reflected by calculating the mean, standard deviation, skewness and other statistical quantities of the signal; Frequency domain features: the power of each frequency band and the spectral centroid are extracted based on the fast Fourier transform power spectrum; Fatigue key ratio features: the frequency band power ratio highly related to fatigue is calculated; Spatial features: used to analyze the activity difference between different brain regions or brain hemispheres to reveal the spatial dynamic change of neural activity under fatigue state. The hemispheric asymmetry index quantifies the functional asymmetry between brain hemispheres by comparing the power difference between the left and right hemispheres in specific frequency bands such as α, β and θ. The calculation formula is as follows: Wherein, is the hemispheric asymmetry index, P band,Right and P band,Left represent the average frequency band power calculated from the selected electrode group of the right hemisphere and the left hemisphere respectively, and the index can reveal the influence of fatigue on the functional advantage of brain hemispheres; Time-frequency features: in electroencephalogram signal analysis, the time variability of time-frequency features can be calculated by sliding window method: set a fixed length window and step, slide the window and calculate the target feature in turn, such as α band power P α , generate a feature time sequence { F 1 ,F 2 ,…,F W}, wherein, W is the total number of windows, and finally the standard deviation or variance of the sequence is taken as the quantitative index of time variability to reflect the dynamic fluctuation characteristics of the feature in time dimension, and the calculation formula is: Wherein, F w is the feature value calculated in the w th window, μ F is the mean of the feature sequence. The index reflects the stability of brain state during task execution.

[0028] All these extracted features combine into a multi-dimensional feature vector that comprehensively characterizes fatigue.

[0029] As shown in Figure 4 Step four: sample compensation The collected electroencephalogram signals and the extracted multi-dimensional feature set solve the class imbalance problem through a signal-level compensation method and a feature-level compensation method to obtain a compensated sample set. Specifically, it includes: Calculate the imbalance ratio of the electroencephalogram signal and the multi-dimensional feature set class. If the value indicates that the class is greater than 1.2, then the minority sample compensation is performed.

[0030] The signal-level compensation method obtains compensated electroencephalogram signal samples through random selection of noise enhancement and other methods, and the calculation formula is Among them, x synthetic is the compensated electroencephalogram signal sample, x i is the electroencephalogram signal, ϵ is a random noise that meets the Gaussian distribution.

[0031] The compensated electroencephalogram signal sample is preprocessed and multi-domain feature extraction is performed to obtain a signal compensation sample; The feature-level compensation method obtains feature compensation samples through random selection of SMOTE improved version and other methods, and the core interpolation formula is: Among them, x new is the feature compensation sample, x i is the minority class sample, x j and x i one of its neighboring samples, λ is a random number in the range of [0, 1].

[0032] The feature compensation sample, the signal compensation sample, and the multi-dimensional feature set are combined, and a random permutation function is used to shuffle to ensure randomness and generalization, to obtain a compensated sample feature set.

[0033] As shown in Figure 5 Step five: mixed model training The compensated sample set is used to train a deep learning model and a traditional machine learning integrated model in parallel to obtain a trained mixed model.

[0034] The deep learning model training adopts a four-layer convolutional neural network composed of convolutional layers, batch normalization layers, activation layers, max pooling layers, and Dropout layers, and is trained using an Adam optimizer.

[0035] The traditional machine learning model training includes an ensemble model of multiple classifiers such as optimized linear discriminant analysis, enhanced logistic regression, Gaussian naive Bayes, enhanced random forest, enhanced support vector machine, quadratic discriminant analysis, and K nearest neighbor classifier. After each classifier is trained, its performance is calculated and a weight is assigned according to the calculation formula: wherein, W i is the weight of the i th classifier, Accuracy i is its accuracy, and F1_score i is its F1 score. All original weights W i will be normalized to obtain the final ensemble weight.

[0036] The probability outputs of the deep learning model and the traditional machine learning model are integrated by a linear weighting fusion method, and the calculation formula is: wherein, P hybrid is the final mixed prediction probability, P ML and P CNN are the prediction probabilities of the traditional machine learning model and the convolutional neural network model, respectively, w ML and w CNN are the corresponding preset weights, and w ML + w CNN = 1.

[0037] As shown in Figure 6 , step six: fatigue detection of special equipment operator According to the trained hybrid model, the electroencephalogram fatigue detection of the special equipment operator is completed. Specifically, it includes: The multi-dimensional feature set obtained by preprocessing and multi-dimensional feature extraction of the electroencephalogram signal of the special equipment operator is input into the trained hybrid model for fatigue detection.

[0038] The above merely describes the preferred embodiments of the present application, and is not intended to limit the scope of protection of the present application. Any equivalent replacement or improvement made by those skilled in the art without departing from the spirit of the present application shall be considered to fall within the scope of protection of the present application. The scope of protection of the present application is defined by the appended claims.

Claims

1. A method for detecting EEG fatigue in special equipment operators based on sample compensation and a hybrid model, characterized in that, include: Collect and preprocess the electroencephalogram (EEG) signals of special equipment operators to obtain preprocessed EEG signals; Multi-domain feature extraction and fusion are performed on the preprocessed EEG signals to obtain a multi-dimensional feature set; A sample compensation method is used to compensate the EEG signal and the multidimensional feature set to obtain the compensated sample feature set; wherein, the sample compensation method includes a feature-level compensation method and a signal-level compensation method; The hybrid model is trained by using the compensated sample feature set to obtain the trained hybrid model; wherein, the hybrid model includes a deep learning model and a traditional machine learning model; The fatigue detection results were obtained by using a trained hybrid model to detect fatigue in special equipment operators.

2. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 1, characterized in that, The preprocessing of EEG signals includes: EEG signals are acquired, and the EEG signals are filtered, denoised, and segmented to obtain preprocessed EEG signals.

3. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 1, characterized in that, The process of multi-domain feature extraction and fusion of preprocessed EEG signals includes: Extracting temporal features: quantifying the mean, standard deviation, root mean square, skewness, kurtosis, zero crossover rate, and activity of the preprocessed EEG signal to characterize the statistical distribution and dynamic characteristics of the signal on the time axis; Frequency domain features were extracted: the average power, total power, spectral centroid, and peak frequency of the preprocessed EEG signal were calculated using Fast Fourier Transform. Extracting key fatigue ratio features: Calculating multiple sets of predefined inter-band power ratios that are highly correlated with fatigue state evolution; Spatial feature extraction: Statistical analysis of signal activity in specific brain regions such as the whole brain, frontal lobe, posterior region, and left and right hemispheres is performed to capture spatial distribution changes in brain activity under fatigue, such as the hemispheric asymmetry index. The calculation formula is as follows: in, P band,Right and P band,Left These represent the average band power calculated from selected electrode groups in the right and left hemispheres, respectively. This index can reveal the impact of fatigue on the functional dominance of the brain hemispheres. Extracting time-frequency features: Calculating time variability using the sliding window method. The calculation formula is as follows: in, F w It is in the w The feature values ​​calculated in each window μ F It is the mean of the feature sequence; this indicator reflects the stability of the brain state during task execution and is used to capture non-stationary dynamic changes caused by fatigue. A multidimensional feature set is obtained by combining the extracted time-domain features, frequency-domain features, fatigue key ratio features, spatial features, and time-frequency features.

4. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 1, characterized in that, The process of compensating EEG signals and multidimensional feature sets using sample compensation methods includes: Quantitative analysis was performed on EEG signals and multidimensional feature sets to obtain minority class EEG signals and minority class multidimensional feature sets. Noise enhancement and improvement are performed on minority class EEG signals using a signal-level compensation method to obtain compensated EEG signal samples. The calculation formula is as follows: in, x synthetic To compensate for the EEG signal samples, x i EEG signals, ϵ Random noise conforming to a specific distribution; The compensated EEG signal samples are preprocessed and multi-domain features are extracted to obtain signal compensation samples; The minority class oversampling based on linear interpolation is performed on the minority class multidimensional feature set using a feature-level compensation method to obtain feature-compensated samples. The calculation formula is as follows: in, x new For feature compensation samples, x i For minority class samples, x j and x i A neighboring sample, λ A random number in the range [0, 1]; The feature-compensated samples, signal-compensated samples, and multidimensional feature sets are combined to obtain the compensated sample feature set.

5. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 1, characterized in that, The training process for hybrid models includes: The convolutional neural network model and the heterogeneous ensemble model are subjected to 10-fold cross-validation using the compensated sample feature set. The probability outputs of the two models are then integrated to generate the final output of the hybrid model and complete the model training, resulting in the trained hybrid model.

6. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 5, characterized in that, The process of obtaining the heterogeneous integration model includes: The predictive contribution of each independent traditional machine learning classifier is determined by its performance-oriented weights. W i The weight is determined based on the overall performance metric of the traditional machine learning classifier on the cross-validation training set, which combines accuracy and F1 score. The formula for this metric is as follows: in, W i For the first i The weights of each classifier, Accuracy i Its accuracy on the training set, F1_score i Its F1 score on the training set, α This is the balancing coefficient. Then, all weights are normalized to ensure that the sum of the weights is 1, thereby focusing on the classifier with better performance.

7. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 5, characterized in that... The process of integrating the probability outputs of the two types of models includes: The probability outputs of the deep learning model and the traditional machine learning model are integrated using a linear weighted fusion method, and the calculation formula is as follows: in, P hybrid For the final mixed prediction probability, P ML and P CNN These are the predicted probabilities of a traditional machine learning model and a convolutional neural network model, respectively. w ML and w CNN For the corresponding preset weights, and w ML + w CNN =1.

8. The method for detecting EEG fatigue of special equipment operators based on sample compensation and hybrid models according to claim 1, characterized in that, The process of conducting EEG fatigue detection on special equipment operators includes: The multidimensional feature set obtained by preprocessing and multidimensional feature extraction of the EEG signals of special equipment operators is input into the trained hybrid model for fatigue detection, and the detection results are obtained.

Images