Brain fatigue detection method, system and device based on electroencephalogram signals and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a brain fatigue detection method, system and device based on electroencephalogram signals and a storage medium. The brain fatigue detection method based on the electroencephalogram signals comprises the following steps: acquiring an electroencephalogram signal sequence of a subject; inputting the electroencephalogram signal sequence into a feature extraction network, respectively extracting spatial features and time features of each electroencephalogram signal in the electroencephalogram signal sequence by the feature extraction network, and fusing the spatial features and the time features to obtain space-time fusion features of the electroencephalogram signals; differential entropy features of the electroencephalogram signals are calculated according to the space-time fusion features; and determining a brain fatigue state detection result of the electroencephalogram signal of the subject according to the differential entropy characteristics. According to the invention, the identified brain fatigue state detection result can be more accurate.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, device, and storage medium for detecting brain fatigue based on electroencephalogram (EEG) signals. Background Technology

[0002] Currently, fatigue state detection technology based on electroencephalogram (EEG) signals is widely used in fields such as driving safety and medical rehabilitation. This technology quantifies the degree of brain fatigue by analyzing the characteristics of brain electrical activity. The process of brain fatigue detection methods includes signal acquisition, feature extraction, and fatigue classification modeling (such as classifying fatigue states through machine learning or deep learning) to obtain brain fatigue detection results.

[0003] Currently, fatigue state detection technology based on electroencephalogram (EEG) signals has many technical shortcomings. Specific research findings are as follows: The patent application with announcement number WO2004028362A1 and patent name "EEG-based fatigue detection" adopts the traditional frequency domain analysis method to identify fatigue state through power changes in the frequency band. However, this method has a high error rate and requires additional electrodes to monitor multiple physiological signals, resulting in high system complexity and making it difficult to promote in practical applications.

[0004] Although the patent application with publication number CN102657526A, entitled "Method for Evaluating the Power Spectrum and R-value of EEG Signals Caused by Watching 3D Images", proposes an EEG evaluation method based on power spectrum analysis, it is only applicable to specific visual fatigue scenarios and lacks the ability to effectively identify general fatigue states. Furthermore, the feature extraction method is singular and cannot capture the complex nonlinear characteristics of EEG signals.

[0005] The patent application with publication number CN108256629B, entitled "An Unsupervised Feature Learning Method for EEG Signals Based on Convolutional Networks and Autoencoders," attempts to introduce deep learning technology and uses convolutional autoencoders for feature learning. However, it only uses a single unsupervised learning architecture, lacks effective learning of discriminative features between different fatigue states, and does not consider individual differences among subjects, resulting in a serious lack of cross-individual generalization ability.

[0006] Furthermore, patent application CN118303883A, entitled "A Fatigue Detection Method Based on Feature Fusion of EEG and EOG Signals," proposes a method for fatigue detection that fuses EEG and EOG signals and employs multi-level feature extraction and feature concatenation, including spatial and temporal convolution. While this method represents an improvement in feature extraction, it relies on the simultaneous acquisition of multiple physiological signals, resulting in a complex system structure. Moreover, this patent application primarily achieves binary or coarse classification, which is insufficient to meet the demands of refined, multi-level fatigue state identification in practical applications.

[0007] The above-mentioned existing technologies generally have the following key problems: (1) Feature extraction relies heavily on traditional signal processing methods or simple convolutional structures, which cannot adaptively learn complex EEG patterns and have limited feature expression capabilities; (2) There is a complete lack of effective processing mechanisms for individual differences of subjects, resulting in huge performance differences between different individuals; (3) The classification accuracy is generally low and unstable, making it difficult to accurately identify multi-level fatigue states and failing to meet the needs of practical applications. Summary of the Invention

[0008] This application aims to at least solve the technical problems existing in the prior art, and to provide a method, system, device and storage medium for detecting brain fatigue based on electroencephalogram (EEG) signals.

[0009] In a first aspect, the present invention provides a method for detecting brain fatigue based on electroencephalogram (EEG) signals, the method comprising: Obtain the subject's electroencephalogram (EEG) signal sequence; The EEG signal sequence is input into the feature extraction network, which extracts the spatial and temporal features of each EEG signal in the sequence, and then fuses the spatial and temporal features to obtain the spatiotemporal fusion features of the EEG signal. Calculate the differential entropy characteristics of EEG signals based on spatiotemporal fusion features; The results of brain fatigue detection based on the differential entropy characteristics of the subject's electroencephalogram (EEG) signals were determined.

[0010] By adopting the above technical solution, this application uses only scalp electroencephalography (EEG) signals for fatigue state identification, without the need to simultaneously collect other physiological signals such as electrooculography, greatly simplifying the system structure, reducing hardware costs, and facilitating practical promotion and large-scale application, unlike complex systems that rely on multimodal signals. It extracts multi-level features (spatial and temporal features) from the EEG signals and uses the spatiotemporal fusion features for calculating differential entropy features. By calculating the differential entropy features of the spatiotemporal fusion features of the EEG signals, an enhanced feature representation is obtained, which can fully exploit the high-order, nonlinear, and spatiotemporally coupled features automatically learned by deep networks. The statistical measurement of differential entropy further enhances the discriminative power of the features, making the detected brain fatigue state more accurate.

[0011] Optionally, acquiring the subject's electroencephalogram (EEG) signal sequence includes: Collect raw data streams of the subject's electroencephalogram (EEG) signals; The raw EEG signal data stream was sequentially processed by bandpass filtering, power frequency notch filtering, and independent component analysis to obtain the preprocessed data stream. The preprocessed data stream is divided into several EEG data segments using a time window to obtain EEG signal sequences.

[0012] By adopting the above technical solution, preprocessing the raw EEG signal data stream can effectively remove artifacts in the EEG signal and improve the signal-to-noise ratio of the EEG signal.

[0013] Optionally, the method further includes: Obtain the training dataset; the training dataset includes positive samples and negative samples. Positive samples are EEG signals of the same subject in the same fatigue state, and negative samples are EEG signals of the same subject in different fatigue states and / or EEG signals of different subjects. Construct the network structure for the feature extraction network; The feature extraction network is iteratively trained using the training dataset until the training termination condition is met. In each training iteration, the feature extraction network extracts the spatiotemporal features of positive samples and negative samples respectively. Based on the spatiotemporal features of positive and negative samples, the normalized temperature-scaled cross-entropy loss of the feature extraction network is determined. Then, self-supervised contrastive learning pre-training is performed based on the normalized temperature-scaled cross-entropy loss to optimize the network parameters of the feature extraction network and obtain the final feature extraction network.

[0014] By adopting the above technical solution, a spatiotemporal convolutional neural network architecture is built based on the training dataset using contrastive learning technology for contrastive learning training. This enables the feature extraction network to learn fatigue-related neural activity patterns and state differences. After training, the EEG signal to be tested is input into the pre-trained feature extraction network to extract features. Combined with differential entropy and multilayer perceptron classification, accurate cognition of cross-subject brain fatigue state is achieved.

[0015] Optionally, for EEG signals from different subjects as negative samples, after extracting the spatiotemporal features of the negative samples, the spatiotemporal feature distribution of the negative samples is subjected to hierarchical normalization processing to obtain standardized spatiotemporal features of the negative samples. Based on the spatiotemporal characteristics of positive samples and the standardized spatiotemporal characteristics of negative samples, self-supervised contrastive learning pre-training is performed using normalized temperature-scaled cross-entropy loss to optimize the parameters of the feature extraction network.

[0016] By adopting the above technical solution, the hierarchical normalization strategy can be used to standardize the features to ensure training stability and significantly improve the generalization ability and robustness of the model under cross-subject and cross-scenario conditions. This enables the method to adapt to changes in EEG signals in different individuals and various practical application environments, and effectively reduce performance fluctuations caused by individual differences and environmental noise.

[0017] Optionally, the calculation formula for hierarchical normalization is: in, Original features The mean of the subject's electroencephalogram (EEG) signal sequence data. The standard deviation of the subject's electroencephalogram (EEG) signal sequence data. This is a small constant used to prevent division by zero.

[0018] By adopting the above technical solution, the specific processing steps of hierarchical normalization were clarified.

[0019] Optionally, the loss function expression for the normalized temperature-scaled cross-entropy loss is: ; in, , and They represent different subjects. and They represent different stimuli. For temperature parameters, The cosine similarity function is used. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the number of stimuli received by the subjects within a batch; Indicates an indicator function, when When the value is 1, The value is 0 at that time.

[0020] By adopting the above technical solution, the specific calculation method of the loss function has been clarified.

[0021] Optionally, the brain fatigue state detection results include at least two fatigue state levels and the probability distribution corresponding to each fatigue state level.

[0022] Secondly, the present invention provides a brain fatigue detection system based on electroencephalogram (EEG) signals, the system comprising: The acquisition module is used to acquire the subject's electroencephalogram (EEG) signal sequence; The feature extraction module is used to input the EEG signal sequence into the feature extraction network. The feature extraction network extracts the spatial and temporal features of each EEG signal in the EEG signal sequence, and fuses the spatial and temporal features to obtain the spatiotemporal fusion features of the EEG signal. The differential entropy feature calculation module is used to calculate the differential entropy features of EEG signals based on spatiotemporal fusion features; The output module is used to determine the brain fatigue state detection results of the subject's EEG signal based on the differential entropy characteristics.

[0023] Thirdly, the present invention provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the brain fatigue detection method based on electroencephalogram (EEG) signals described above.

[0024] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the brain fatigue detection method based on electroencephalogram (EEG) signals described above.

[0025] In summary, this application has the following beneficial technical effects: This application uses only scalp electroencephalography (EEG) signals for fatigue state recognition, without the need to simultaneously collect other physiological signals such as electrooculography, which greatly simplifies the system structure, reduces hardware costs, and facilitates practical promotion and large-scale application, unlike complex systems that rely on multimodal signals. Multi-level features (spatial and temporal features) of EEG signals are extracted, and the spatiotemporal fusion features are used to calculate differential entropy features. By calculating the differential entropy features of the spatiotemporal fusion features of EEG signals, an enhanced feature representation is obtained, which can fully exploit the high-order, nonlinear, and spatiotemporally coupled features automatically learned by deep networks. The discriminative power of the features is further enhanced by the statistical measurement of differential entropy, making the detection results of brain fatigue state more accurate. Based on the training dataset, a spatiotemporal convolutional neural network architecture was built using contrastive learning techniques for contrastive learning training. This enabled the feature extraction network to learn fatigue-related neural activity patterns and state differences. After training, the EEG signals to be tested were input into the pre-trained feature extraction network to extract features. Combined with differential entropy and multilayer perceptron classification, accurate recognition of cross-subject brain fatigue states was achieved. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a brain fatigue detection method based on electroencephalogram (EEG) signals according to an embodiment of the present invention. Figure 2 The waveform of the raw EEG signal data stream of the subject; Figure 3 This is a waveform diagram of the electroencephalogram (EEG) signal sequence obtained after preprocessing. Figure 4 This is a network system structure diagram of a brain fatigue state cognition method based on contrastive learning and scalp EEG signals provided in an embodiment of the present invention. Figure 5 This is a visualization of the differential entropy features extracted by the encoder trained through comparative learning in this invention. Figure 6 A diagram showing the results of the classification confusion matrix; Figure 7 This is a schematic diagram of the structure of an electronic device that implements the brain fatigue detection method based on electroencephalogram (EEG) signals, according to an embodiment of the present invention.

[0027] Reference numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface.

[0028] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0030] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0031] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal communication between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0032] Reference Figure 1 The diagram shown is a flowchart illustrating a brain fatigue detection method based on electroencephalogram (EEG) signals according to an embodiment of the present invention. In this embodiment, the brain fatigue detection method based on EEG signals includes: S1. Obtain the subject's electroencephalogram (EEG) signal sequence.

[0033] Specifically, the subjects are individuals participating in the experiment. Subjects provide scalp EEG signals by wearing an EEG acquisition device (such as an electrode cap). Before collecting the subject's EEG signals using the device, the subject needs to clean the hair near the scalp to ensure the quality of the acquired signals. The EEG signals include current neural activity information and frequency domain characteristics related to fatigue status.

[0034] The frequency domain features related to neural activity and fatigue state correspond to the features of emotion-related brain regions, namely the activation intensity and distribution patterns of different brain regions (such as the temporal lobe, frontal lobe, and occipital lobe) under emotional stimulation. By analyzing the frequency domain features related to neural activity and fatigue state of pre-subjects, the fatigue state of the subject's brain can be determined.

[0035] In this embodiment, the step of acquiring the subject's electroencephalogram (EEG) signal sequence includes: S11. Collect the raw data stream of the subject's electroencephalogram (EEG) signals.

[0036] S12. The raw EEG signal data stream is sequentially processed by bandpass filtering, power frequency notch filtering, and independent component analysis to obtain the preprocessed data stream.

[0037] In one example of this embodiment, the preprocessing of the raw EEG signal data stream is as follows: First, the raw EEG signal data stream of the current subject is stably and accurately acquired through a programming interface; then, the raw EEG signal data stream is passed through a bandpass filter with a bandpass filtering frequency range of 0.5–100 Hz; subsequently, the EEG signal data stream output from the bandpass filter is subjected to power frequency notch processing at a frequency of 50 Hz; finally, the power frequency notch-processed EEG signal data stream is imported into an independent component analysis tool (such as EEGLAB, Electroencephalogram Laboratory), and a channel location information template matching the recorded data (such as the standard-10-5-cap385 template of the international 10-5 system) is loaded to clarify the spatial distribution of the electrodes; the independent component analysis tool decomposes the multi-channel EEG signal into statistically independent components, each component containing temporal processes and topographic distribution, separating artifacts such as blinking, eye drift, and power frequency interference, thereby effectively removing artifacts in the EEG signal and improving the signal-to-noise ratio of the EEG signal.

[0038] S13. Using a time window, the preprocessed data stream is divided into several EEG data segments to obtain an EEG signal sequence.

[0039] In one example of this embodiment, the time window length is 2.5 seconds, corresponding to 1250 EEG signal sampling points, and the EEG signal data segments are segmented with a 50% overlap rate. (Refer to...) Figure 2 and Figure 3 , Figure 2 This is a waveform diagram of the raw EEG signal data stream from the subject. Figure 3 The image shows the waveform of the EEG signal sequence after preprocessing. Preprocessing the raw EEG signal data stream can effectively remove artifacts from the EEG signal and improve the signal-to-noise ratio of the EEG signal.

[0040] This application uses only scalp electroencephalography (EEG) signals for fatigue state identification, without the need to simultaneously collect other physiological signals such as electrooculography, which greatly simplifies the system structure, reduces hardware costs, and facilitates practical promotion and large-scale application, unlike complex systems that rely on multimodal signals.

[0041] S2. Input the EEG signal sequence into the feature extraction network. The feature extraction network extracts the spatial and temporal features of each EEG signal in the EEG signal sequence, and fuses the spatial and temporal features to obtain the spatiotemporal fusion features of the EEG signal.

[0042] refer to Figure 4The feature extraction network is a deep neural network. This invention adopts an end-to-end deep neural network structure to automatically complete feature learning and classification. The model structure is compact and efficient, suitable for real-time deployment on wearable devices and other terminals, and facilitates continuous monitoring and feedback in real-world scenarios.

[0043] The feature extraction network consists of a spatial convolutional layer and a temporal convolutional layer connected sequentially. Preprocessed EEG signal data is input into the spatial convolutional layer, which uses 16 63×1 convolutional kernels to extract spatial features from the spatial distribution of the EEG signal sequence. The spatial convolution result output from the spatial convolutional layer is input into the temporal convolutional layer, which uses 16 1×60 convolutional kernels to extract temporal features from the temporal sequence of the EEG signal. The spatial and temporal features are then fused, and the temporal convolution result is output. Average pooling is performed on the temporal convolution result output from the temporal convolutional layer, with a pooling window size of 1×30. An activation function is then used to apply a nonlinear transformation to the neuron output. Finally, a 2x channel expansion factor is used to flatten the feature map processed by the average pooling operation and activation function, outputting a 9216-dimensional feature vector, thus obtaining the spatiotemporal fusion features of the EEG signal.

[0044] In this embodiment, the calculation formula for spatial convolution is: in, This represents the output of the i-th spatial filter, where i represents the index of the spatial filter. This represents the activation function. Represents the spatial convolution kernel weights. This represents the EEG signal of the j-th electrode, where j represents the index of the electrode used by the EEG signal acquisition device to acquire the EEG signal. Indicates the number of electrode channels. This represents the bias term of the i-th spatial filter.

[0045] The formula for calculating the temporal convolution operation is: in, Indicates the first The output of a time filter Indicates the index of the spatial filter. Indicates the temporal convolution kernel weights. Indicates the index of the time step. Indicates the time offset. Indicates the first The bias term of a time filter.

[0046] S3. Calculate the differential entropy characteristics of EEG signals based on spatiotemporal fusion features.

[0047] The formula for calculating the differential entropy characteristic of EEG signals is: in, This indicates the spatiotemporal fusion characteristics of EEG signals. Spatiotemporal fusion characteristics of EEG signals The corresponding differential entropy features, Let be the variance of the spatiotemporal fusion features of the EEG signal in the time dimension, and e be the natural constant.

[0048] The feature extraction network automatically extracts multi-level features from EEG signals using spatial and temporal convolutional layers, and uses the features from the intermediate layers of the deep network (i.e., the extracted spatiotemporal fusion features) for calculating differential entropy features. This method differs from traditional differential entropy feature extraction methods, as it can fully exploit the high-order, nonlinear, spatiotemporal fusion features automatically learned by the deep network, and further enhance the discriminative power of the features through the statistical measurement of differential entropy, making the detected brain fatigue state results more accurate.

[0049] Reference Figure 5 This application integrates deep neural networks with statistical features such as differential entropy, which can fully explore the high-order nonlinearity and spatiotemporal fusion features of EEG signals, improve the accuracy and expressive power of fatigue state recognition, and overcome the problems of reliance on artificial feature design and limited expressive power of traditional methods.

[0050] S4. Determine the brain fatigue state detection results of the subject's EEG signals based on the differential entropy characteristics.

[0051] The brain fatigue state detection result includes at least two fatigue state levels and a probability distribution corresponding to each fatigue state level. In this embodiment, there are four brain fatigue state levels: mild fatigue, moderate fatigue, severe fatigue, and extreme fatigue. In this embodiment, differential entropy features are input into a classifier for inference, and the classifier outputs the probability distributions of the four fatigue states.

[0052] Reference Figure 5 This application achieves a four-level refined classification of mild fatigue, moderate fatigue, severe fatigue, and extreme fatigue, which meets the needs of practical applications and overcomes the limitations of existing technologies in accurately identifying multi-level fatigue states. Its overall performance is significantly better than existing methods, meeting the needs of practical applications for fatigue classification.

[0053] Furthermore, in the preferred embodiment of this invention, the present invention adopts a compact classifier structure, which avoids the overfitting risk brought about by complex network structures, fully adapts to the statistical characteristics of EEG differential entropy features, has good generalization ability and computational efficiency, and is easy to deploy and apply in real-world scenarios.

[0054] The differential entropy features are input into the classifier for inference; the output includes a probability distribution of four types of fatigue states, thus completing the automatic identification of fatigue states across subjects.

[0055] A complete classification pipeline can be represented as: in, The input vector represents the raw EEG signal. `Preprocess()` indicates preprocessing, `CNN()` represents spatiotemporal convolution feature extraction, `DE()` represents differential entropy feature calculation, the `softmax` function converts the input vector into probability scores, and the `argmax` function obtains the index of the largest element in the vector, commonly used in classification tasks to determine the final category; `p` represents the fatigue state level index. This is the final fatigue state prediction result.

[0056] In this embodiment, to verify the effectiveness of the proposed method, detailed experimental validation was conducted on a multi-subject EEG fatigue dataset. Sixty-three effective electrodes (excluding the reference electrode) were used, with a sampling frequency of 500 Hz, and approximately two hours of continuous EEG recordings were performed for each subject, covering four fatigue states: mild fatigue, moderate fatigue, severe fatigue, and extreme fatigue. Detailed performance for each fatigue state is shown in Table 1 and... Figure 6 As shown in Table 1, the performance of various fatigue states is classified. Figure 6 The classification confusion matrix results are shown in detail, illustrating the comparison between the model's predictions and the true labels.

[0057] Table 1 Classification of various fatigue states and performance

[0058] Compared to the high error rate of traditional frequency domain analysis methods, the feature extraction network in this application uses a spatial-temporal convolutional network to automatically extract multi-level, spatiotemporally coupled deep features, and combines it with differential entropy calculation, which can fully capture the high-order nonlinear dynamic features of EEG signals and improve the accuracy of fatigue state classification.

[0059] In another embodiment of this application, the brain fatigue detection method based on electroencephalogram (EEG) signals further includes a step of training a feature extraction network, specifically as follows: S21. Obtain the training dataset.

[0060] Specifically, the training dataset can utilize commonly used scalp EEG signal datasets, such as the clinical application dataset (Temple University Hospital EEG Corpus Database, TUH EEG dataset) and the epilepsy research dataset (CHB-MIT Scalp EEG dataset, the full English name of CHB-MIT Scalp EEG dataset is Children's Hospital Boston-Massachusetts Institute of Technology Scalp EEG Database). The TUH EEG dataset contains tens of thousands of clinical records, covering conditions such as epilepsy and sleep disorders. The data includes basic patient information and diagnostic information, making it suitable for neuroscience research and machine learning model training. The CHB-MIT Scalp EEG dataset contains long-term scalp EEG records (sampling rate 256Hz) of 23 pediatric epilepsy patients, covering the ictal, interictal, and preictal phases. Alternatively, scalp EEG signals from different subjects can be collected using EEG acquisition equipment as the training dataset; this embodiment does not impose any restrictions.

[0061] When using EEG acquisition equipment to collect scalp EEG signals from different subjects as training datasets, the subjects whose EEG signals were collected in the training dataset include healthy volunteers of different ages, genders, and dominant hands, as well as patients with specific diseases (such as stroke patients and amyotrophic lateral sclerosis patients). Alternatively, EEG signals collected from subjects under different physical conditions or fatigue states can be selected as training sample data. This embodiment does not impose any restrictions.

[0062] The training dataset includes positive and negative samples. Positive samples are EEG signals of the same subject under the same fatigue state, while negative samples are EEG signals of the same subject under different fatigue states and / or EEG signals of different subjects.

[0063] In a preferred embodiment of this example, after acquiring the raw EEG signal data streams of the same subject under different fatigue states and / or different subjects, the raw EEG signal data streams are preprocessed by filtering, noise reduction, and removal of electrooculogram (EOG) signals. Then, a training dataset containing positive and negative samples is constructed using the preprocessed EEG signal data. The positive samples are EEG signal segments of the same subject under similar fatigue states, and the negative samples are EEG signal segments of different fatigue states or different subjects (such as signals from waking and fatigue states).

[0064] S22. Construct the network structure of the feature extraction network.

[0065] In this embodiment, the network structure of the feature extraction network includes a spatial convolutional layer, a temporal convolutional layer, and a processing module connected in sequence. The spatial convolutional layer is used to extract the spatial features of the EEG signal, the temporal convolutional layer is used to extract the temporal features of the EEG signal, and the processing module is used to fuse the temporal and spatial features, as well as to perform average pooling and activation function processing on the spatiotemporal fused features.

[0066] S23. Use the training dataset to iteratively train the feature extraction network until the training termination condition is met.

[0067] Based on the training dataset, a spatiotemporal convolutional neural network architecture was built using contrastive learning techniques for contrastive learning training. This enabled the feature extraction network to learn fatigue-related neural activity patterns and state differences. After training, the EEG signals to be tested were input into the pre-trained feature extraction network to extract features. Combined with differential entropy and multilayer perceptron classification, accurate recognition of cross-subject brain fatigue states was achieved.

[0068] Specifically, in each training iteration, the feature extraction network extracts the spatiotemporal features of both positive and negative samples. Based on these features, the normalized temperature-scaled cross-entropy loss of the feature extraction network is determined. Self-supervised contrastive learning pre-training is then performed using this loss to optimize the encoder parameters, enabling the network to learn discriminative feature representations that differentiate between different fatigue states. An optimization algorithm is used to train the model, distinguishing features of different fatigue states in the feature space. Finally, features are extracted from the tested EEG signals using the pre-trained model, and combined with differential entropy for classification via a multilayer perceptron, achieving accurate cross-subject fatigue state identification.

[0069] Figure 5 This paper presents the visualization results of the differential entropy features extracted by the encoder trained through contrastive learning according to the present invention. The brain fatigue detection method based on EEG signals in this embodiment enhances feature discriminativeness through contrastive learning and combines the physiological interpretability of differential entropy, effectively solving the generalization problem of cross-subject EEG fatigue state recognition.

[0070] When using EEG acquisition equipment to collect scalp EEG signals from different subjects as a training dataset, the raw EEG signal data streams were preprocessed to obtain EEG signal sequences. Then, two different enhanced views were generated based on each EEG data segment in the EEG signal sequence. The methods for generating the enhanced views included data augmentation of the raw EEG signals, including time shifting, noise addition, and amplitude scaling. The time shift range was ±300 sampling points, the Gaussian noise addition noise level was 0.25 times the data standard deviation, the amplitude scaling range was 0.3-1.7 times, the application probability of each enhancement operation was 0.8, and the enhancement intensity parameter was set to 2.5 to control the intensity of the enhancement effect.

[0071] Through contrastive learning training, the model learns subject-invariant characteristics by maximizing the similarity of EEG representations of the same emotional stimulus among different subjects and minimizing the similarity of representations of different stimuli. Contrastive learning ensures that the model's learned representations satisfy the following: minimum cross-subject representation distance for the same stimulus and maximum distance between different stimuli.

[0072] Furthermore, unlike traditional convolutional networks for feature extraction, our model architecture consists of two key stages: a contrastive learning stage and a prediction stage.

[0073] In the contrastive learning phase: the encoder transforms the raw EEG data into a spatiotemporal representation aligned across subjects, and then maps it to the latent space through a nonlinear projector in order to calculate the similarity between samples.

[0074] In this embodiment, two different enhanced views are generated for each EEG data segment, wherein the time shift range is ±300 sampling points, the noise level added by Gaussian noise is 0.25 times the data standard deviation, the amplitude scaling range is 0.3-1.7 times, the application probability of each enhancement operation is 0.8, the enhancement intensity parameter is set to 2.5 to control the intensity of the enhancement effect, and a hierarchical normalization strategy is used to standardize the features to ensure training stability.

[0075] For negative samples consisting of EEG signals from different subjects, after extracting the spatiotemporal features of the negative samples, the spatiotemporal feature distribution of the negative samples is subjected to hierarchical normalization to obtain standardized spatiotemporal features of the negative samples. Based on the spatiotemporal features of the positive samples and the standardized spatiotemporal features of the negative samples, self-supervised contrastive learning pre-training is performed using normalized temperature scaling cross-entropy loss to optimize the parameters of the feature extraction network. This enables the feature extraction network to automatically adjust the scale and distribution of features when faced with EEG signals from different individuals.

[0076] The calculation formula for hierarchical normalization is: in, Original features The mean of the subject's electroencephalogram (EEG) signal sequence data. The standard deviation of the subject's electroencephalogram (EEG) signal sequence data. This is a small constant used to prevent division by zero.

[0077] The use of hierarchical normalization strategy to standardize features ensures training stability. Through the above-mentioned diverse data augmentation strategies and hierarchical normalization, the generalization ability and robustness of the model under cross-subject and cross-scenario conditions are significantly improved, enabling the method to adapt to changes in EEG signals in different individuals and various practical application environments, and effectively reduce performance fluctuations caused by individual differences and environmental noise.

[0078] In addition, we innovatively employ adaptive feature normalization for the differential entropy features during the prediction phase, dynamically updating the mean and variance of the test data (based on the initial values ​​of the training data and the real-time weighted average of the test data). This allows for rapid adaptation without accessing new subject data, improving the feasibility of practical applications. The specific formula is as follows: in, This represents the new mean obtained after dynamic updating. This represents the initial mean of the training data. This represents the real-time mean of the test data. This represents the weighting coefficients; a portion of the EEG signal sequences from the training dataset is used as training data, and another portion of the EEG signal sequences from the training dataset is used as test data. The training data is the dataset used to train the model, which learns features, patterns, and parameters by analyzing this data; the test data is the dataset used to evaluate the model's generalization ability.

[0079] In a preferred embodiment of this example, reference is made to Figure 4 The spatiotemporal fusion feature vector is processed using a fully connected classifier. Specifically, the spatiotemporal fusion feature vector is input into a three-layer fully connected projection head, with the first layer having dimensions from 9216 to 256, the second layer from 256 to 256, and the third layer from 256 to 128. The Adam optimizer and cosine annealing scheduler are used to learn discriminative feature representations that distinguish different fatigue states by minimizing the normalized temperature-scaling cross-entropy loss.

[0080] The fully connected classifier design includes: first, layer normalization of the input differential entropy features to reduce the gap between subjects; then, mapping the features to a 256-dimensional space through an input projection layer; feature extraction using a two-layer fully connected network (256→128 dimensions), with each layer sequentially using linear transformation, batch normalization, ReLU activation function, and Dropout regularization; finally, outputting the logits values ​​of four fatigue states through a classification head containing a 64-dimensional intermediate layer, and using the cross-entropy loss function for supervised learning.

[0081] Define homologous stimulus pairs ( (positive example) and heterologous stimulus pair ( (Negative examples) The loss function is constructed by normalizing the similarity using Softmax normalization based on temperature τ. In this embodiment, the loss function expression for the normalized temperature-scaled cross-entropy loss is: ; in, , and They represent different subjects. and These represent different stimuli. In EEG signal processing, a stimulus is a standardized external input that can induce a specific labelable psychological state (such as emotion, fatigue, etc.) in the subject. When collecting the subject's scalp EEG signals, this application will show the subject stimulating materials to intervene in the subject's brain neural activity through the stimulating materials. In this embodiment, the stimulating materials can be pictures of several different scenes or a video material with changing scenes. When the subject watches the stimulating materials, the scalp EEG signals will change with the different scenes displayed.

[0082] For temperature parameters, The cosine similarity function is used. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. This indicates the number of stimuli received by the subjects within a batch. Indicates an indicator function, when When the value is 1, The value is 0 at that time.

[0083] In some examples of this embodiment, the temperature parameter τ is set to 0.07, the learning rate to 0.0005, and the weight decay to 0.0001. A cosine annealing scheduler is used to decay the learning rate from its initial value to the minimum learning rate. It was trained for 1000 rounds to learn discriminative feature representations that distinguish different fatigue states.

[0084] For each pair of subjects, positive sample pairs (EEG signal segments from the same stimulus) and negative sample pairs (EEG signal segments from different stimuli) are constructed. The sample size increases quadratically with the number of subjects, enhancing the model's robustness. Cross-subject representation alignment is achieved by minimizing the total loss. For the first time, normalized temperature-scaled cross-entropy loss is applied to scalp EEG signal fatigue recognition, improving the model's generalization ability across subject scenarios and effectively learning similar EEG signal data from different subjects.

[0085] Based on the same inventive concept, one embodiment of the present invention provides a brain fatigue detection system based on electroencephalogram (EEG) signals.

[0086] The brain fatigue detection system of the present invention can be installed in an electronic device. Depending on the functions implemented, the brain fatigue detection system includes: The acquisition module is used to acquire the subject's electroencephalogram (EEG) signal sequence; The feature extraction module is used to input the EEG signal sequence into the feature extraction network. The feature extraction network extracts the spatial and temporal features of each EEG signal in the EEG signal sequence, and fuses the spatial and temporal features to obtain the spatiotemporal fusion features of the EEG signal. The differential entropy feature calculation module is used to calculate the differential entropy features of EEG signals based on spatiotemporal fusion features; The output module is used to determine the brain fatigue state detection results of the subject's EEG signal based on the differential entropy characteristics.

[0087] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0088] The various variations and specific examples of the brain fatigue detection method based on EEG signals provided in the above embodiments are also applicable to the brain fatigue detection system of this embodiment. Through the foregoing detailed description of the brain fatigue detection method based on EEG signals, those skilled in the art can clearly understand the implementation method of the brain fatigue detection system in this embodiment. For the sake of brevity, it will not be described in detail here.

[0089] This application also discloses an electronic device, such as Figure 7 The diagram shown is a schematic representation of an electronic device for a brain fatigue detection method based on electroencephalogram (EEG) signals according to an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 communicatively connected to the at least one processor, a communication bus 12, and a communication interface 13. It may also include a computer program, such as a program for a brain fatigue detection method based on EEG signals, stored in the memory 11 and executable on the processor 10.

[0090] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., methods for detecting brain fatigue) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0091] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a method for detecting brain fatigue, but also to temporarily store data that has been output or will be output.

[0092] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0093] Communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0094] Figure 7 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 7 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0095] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.

[0096] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0097] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0098] This application provides a computer-readable storage medium, including, for example, any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM). The computer-readable storage medium stores a computer program capable of being loaded by a processor and executing the brain fatigue detection method based on electroencephalogram (EEG) signals described in the above embodiments.

[0099] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0100] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting brain fatigue based on electroencephalogram (EEG) signals, characterized in that, The method includes: Obtain the subject's electroencephalogram (EEG) signal sequence; The EEG signal sequence is input into the feature extraction network, which extracts the spatial and temporal features of each EEG signal in the sequence, and then fuses the spatial and temporal features to obtain the spatiotemporal fusion features of the EEG signal. Calculate the differential entropy characteristics of EEG signals based on spatiotemporal fusion features; The results of brain fatigue detection based on the characteristics of differential entropy were determined from the EEG signals of the subjects. The method further includes: Obtain the training dataset; the training dataset includes positive samples and negative samples. Positive samples are EEG signals of the same subject in the same fatigue state, and negative samples are EEG signals of the same subject in different fatigue states and / or EEG signals of different subjects. Construct the network structure for the feature extraction network; The feature extraction network is iteratively trained using the training dataset until the training termination condition is met. In each training iteration, the feature extraction network extracts the spatiotemporal features of positive samples and negative samples respectively. Based on the spatiotemporal features of positive and negative samples, the normalized temperature-scaled cross-entropy loss is determined. Then, self-supervised contrastive learning pre-training is performed based on the normalized temperature-scaled cross-entropy loss to optimize the network parameters of the feature extraction network and obtain the final feature extraction network.

2. The brain fatigue detection method based on electroencephalogram (EEG) signals as described in claim 1, characterized in that, The acquisition of the subject's electroencephalogram (EEG) signal sequence includes: Collect raw data streams of the subject's electroencephalogram (EEG) signals; The raw EEG signal data stream was sequentially processed by bandpass filtering, power frequency notch filtering, and independent component analysis to obtain the preprocessed data stream. The preprocessed data stream is divided into several EEG data segments using a time window to obtain EEG signal sequences.

3. The brain fatigue detection method based on electroencephalogram (EEG) signals as described in claim 1, characterized in that, For negative samples consisting of EEG signals from different subjects, after extracting the spatiotemporal features of the negative samples, the spatiotemporal feature distribution of the negative samples is subjected to hierarchical normalization to obtain standardized spatiotemporal features of the negative samples. Based on the spatiotemporal characteristics of positive samples and the standardized spatiotemporal characteristics of negative samples, self-supervised contrastive learning pre-training is performed using normalized temperature-scaled cross-entropy loss to optimize the parameters of the feature extraction network.

4. The brain fatigue detection method based on electroencephalogram (EEG) signals as described in claim 3, characterized in that, The calculation formula for hierarchical normalization is: in, Original features The mean of the subject's electroencephalogram (EEG) signal sequence data. The standard deviation of the subject's electroencephalogram (EEG) signal sequence data. This is a small constant used to prevent division by zero.

5. The brain fatigue detection method based on electroencephalogram (EEG) signals as described in any one of claims 1 to 4, characterized in that, The loss function expression for normalized temperature-scaled cross-entropy loss is: ； in, , and They represent different subjects. and They represent different stimuli. For temperature parameters, The cosine similarity function is used. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the subject In the The feature vector of the electroencephalogram (EEG) signal generated under a stimulus. Indicates the number of stimuli received by the subjects within a batch; Indicates an indicator function, when When the value is 1, The value is 0 at that time.

6. The brain fatigue detection method based on electroencephalogram (EEG) signals as described in claim 1, 2, 3, or 4, characterized in that, The brain fatigue state detection results include at least two fatigue state levels, and the probability distribution corresponding to each fatigue state level.

7. A brain fatigue detection system based on electroencephalogram (EEG) signals, used to implement the brain fatigue detection method based on EEG signals as described in any one of claims 1 to 6, characterized in that, include: The acquisition module is used to acquire the subject's electroencephalogram (EEG) signal sequence; The feature extraction module is used to input the EEG signal sequence into the feature extraction network. The feature extraction network extracts the spatial and temporal features of each EEG signal in the EEG signal sequence, and fuses the spatial and temporal features to obtain the spatiotemporal fusion features of the EEG signal. The differential entropy feature calculation module is used to calculate the differential entropy features of EEG signals based on spatiotemporal fusion features; The output module is used to determine the brain fatigue state detection results of the subject's EEG signal based on the differential entropy characteristics.

8. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10) to enable the at least one processor (10) to perform the brain fatigue detection method based on electroencephalogram (EEG) signals as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the brain fatigue detection method based on electroencephalogram (EEG) signals as described in any one of claims 1 to 6.

Images