An electroencephalogram signal fatigue monitoring method, system, electronic device and storage medium

Abstract

The application discloses a brain electrical signal fatigue monitoring method and system, an electronic device and a storage medium. The method extracts multi-dimensional brain electrical features from multi-channel brain electrical signals, constructs an adjacency matrix based on three-dimensional coordinates of multiple electrodes, and constructs multiple fatigue labels through eye movement signals. The multi-dimensional brain electrical features, the adjacency matrix and the multiple fatigue labels are constructed into a test set and a training set. A regularized graph neural network model is constructed. Cross-entropy classification loss, domain adversarial loss and regularization loss are weighted and summed to obtain a total loss function. Based on the test set and the training set, the regularized graph neural network model is trained through the total loss function to obtain a trained regularized graph neural network model. The brain electrical signal to be detected and the three-dimensional coordinates of the electrode to be detected are preprocessed and input into the trained regularized graph neural network model to obtain a fatigue monitoring result. The application can improve the accuracy of fatigue monitoring and improve the robustness of the model.

Description

Technical Field

[0001] This application relates to the field of fatigue monitoring technology, and in particular to a method, system, electronic device and storage medium for monitoring fatigue by electroencephalogram (EEG) signals. Background Technology

[0002] Traditional fatigue monitoring methods, such as convolutional neural networks (CNNs) and support vector machines (SVMs), treat electroencephalogram (EEG) signals as independent Euclidean spatial data, neglecting the spatial distribution of electrodes on the scalp and the biological topological characteristics of brain functional regions. This results in insufficient modeling of local anatomical connectivity and global functional coupling between channels. Brain neural activity exhibits significant spatial dependence, with interactions between adjacent or functionally related brain regions. Existing models struggle to effectively model this non-Euclidean topology, leading to inadequate capture of inter-channel relationships.

[0003] Existing deep learning models rely on large amounts of labeled data, making them prone to overfitting to the characteristics of specific subjects during training. When applied to new, unseen subjects, their performance drops significantly, failing to meet the needs of cross-individual monitoring in practical applications. This leads to a sharp decline in the accuracy of fatigue monitoring across subjects and in scenarios with few samples. Furthermore, existing deep learning models are highly susceptible to overfitting when there are few samples or noisy labels, resulting in a significant reduction in classification accuracy and robustness.

[0004] In summary, existing fatigue monitoring technologies suffer from low accuracy and low model robustness. Summary of the Invention

[0005] This application aims to propose a method, system, electronic device, and storage medium for monitoring fatigue using electroencephalogram (EEG) signals, which can improve the accuracy of fatigue monitoring and enhance the robustness of the model.

[0006] In a first aspect, embodiments of this application provide a method for monitoring brainwave signal fatigue, the method comprising: Acquire eye movement signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes; Multidimensional EEG features are extracted from the multi-channel EEG signals, an adjacency matrix is ​​constructed based on the three-dimensional coordinates of the multiple electrodes, and various fatigue labels are constructed using the eye movement signals. The multidimensional EEG features, the adjacency matrix, and the various fatigue labels are used to construct a test set and a training set. A regularized graph neural network model is constructed, comprising a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The signal diffuser includes a graph convolutional layer, and the domain discriminator includes a gradient inversion layer and multiple fully connected layers. The total loss function is obtained by weighted summation of cross-entropy classification loss, domain adversarial loss, and regularization loss. The regularization loss includes adjacency matrix regularization and model weight regularization. Based on the test set and the training set, the regularized graph neural network model is trained using the total loss function to obtain a trained regularized graph neural network model. The EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected are preprocessed and then input into the trained regularized graph neural network model. Fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain fatigue monitoring results.

[0007] In some embodiments, extracting multidimensional EEG features from the multichannel EEG signals includes: The multi-channel EEG signal was processed by moving average power spectral density, linear dynamic system power spectral density, moving average differential entropy, and linear dynamic differential entropy to obtain four EEG features. The four types of EEG features are spliced ​​together to obtain multidimensional EEG features.

[0008] In some implementations, constructing the adjacency matrix based on the three-dimensional coordinates of the plurality of electrodes includes: Calculate the spatial distance between the two electrodes in three-dimensional coordinates; The initial weights are calculated using the spatial distance; Based on the initial weights, an adjacency matrix is ​​constructed.

[0009] In some implementations, the construction of multiple fatigue labels using the eye-tracking signals includes: Based on the eye movement signals, the eyelid closure percentage value is calculated; Based on the stated eyelid closure percentage, various fatigue labels are constructed.

[0010] In some implementations, training the regularized graph neural network model using the total loss function based on the test set and the training set to obtain a trained regularized graph neural network model includes: The test set and the training set are randomly mixed, and the mixed dataset is input into the regularized graph neural network model to obtain the classifier prediction result and the domain discriminator prediction result; wherein: The multidimensional EEG features and adjacency matrix from the mixed dataset are input into the signal diffuser to obtain the output of the signal diffuser; The feature extractor extracts features from the output of the signal diffuser to obtain an abstract feature representation. The abstract feature representation and multiple fatigue labels in the mixed dataset are input into the classifier, and the abstract feature representation is input into the domain discriminator. The classifier and the domain discriminator are then subjected to adversarial training using the total loss function to obtain the classifier prediction result and the domain discriminator prediction result. In backpropagation, the gradient of each loss in the total loss function is calculated based on the prediction results of the classifier and the prediction results of the domain discriminator. The parameters in the regularized graph neural network model are updated using the gradient, and the model is trained again using the updated parameters until the termination condition is met, resulting in a well-trained regularized graph neural network model.

[0011] In some implementations, calculating the gradient of each loss in the total loss function based on the classifier prediction results and the domain discriminator prediction results includes: Obtain the first original gradient corresponding to the adjacency matrix regularization and the second original gradient corresponding to the model weight regularization; Based on the classifier prediction results, calculate the first gradient corresponding to the cross-entropy classification loss; Based on the prediction results of the domain discriminator, the second gradient corresponding to the domain adversarial loss is calculated; Based on the first original gradient, calculate the third gradient corresponding to the adjacency matrix regularization; Based on the second original gradient, calculate the fourth gradient corresponding to the model weight regularization.

[0012] In some implementations, updating the parameters in the regularized graph neural network model via the gradient includes: The first gradient corresponding to the training set samples is backpropagated in the classifier by controlling the domain label, so as to update the parameters of the classifier by the first gradient corresponding to the training set samples; The second gradient corresponding to the training set samples and the second gradient corresponding to the test set samples are backpropagated in the domain discriminator by controlling the domain label, so as to update the parameters of the domain discriminator by the second gradient corresponding to the training set samples and the second gradient corresponding to the test set samples; When backpropagating to the feature extractor, the second gradient corresponding to the domain discriminator is inverted and added to the first gradient corresponding to the classifier to obtain the added gradient; the parameters of the feature extractor are updated using the added gradient. The adjacency matrix is ​​updated using the third gradient. The weights in the regularized graph neural network model are updated using the fourth gradient.

[0013] Secondly, embodiments of this application also provide a brainwave signal fatigue monitoring system, the system comprising: The data acquisition module is used to acquire eye movement signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes; The data processing module is used to extract multidimensional EEG features from the multi-channel EEG signals, construct an adjacency matrix based on the three-dimensional coordinates of the multiple electrodes, and construct various fatigue labels through the eye movement signals. The dataset construction module is used to construct the multidimensional EEG features, the adjacency matrix, and the various fatigue labels into a test set and a training set; The model building module is used to build a regularized graph neural network model including a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The signal diffuser includes a graph convolutional layer, and the domain discriminator includes a gradient inversion layer and multiple fully connected layers. The weighted summation module is used to sum the cross-entropy classification loss, domain adversarial loss, and regularization loss to obtain the total loss function. The regularization loss includes adjacency matrix regularization and model weight regularization. The model training module is used to train the regularized graph neural network model based on the test set and the training set, using the total loss function, to obtain a trained regularized graph neural network model. The fatigue monitoring module is used to preprocess the EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected, and then input them into the trained regularized graph neural network model. The fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain the fatigue monitoring results.

[0014] Thirdly, embodiments of this application also provide an electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform an EEG signal fatigue monitoring method as described above.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the electroencephalogram (EEG) signal fatigue monitoring method described above.

[0016] Compared with related technologies, this application has the following advantages: This application acquires eye-tracking signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes; extracts multi-dimensional EEG features from the multi-channel EEG signals, constructs an adjacency matrix based on the three-dimensional coordinates of multiple electrodes, and constructs various fatigue labels using eye-tracking signals; constructs test and training sets using the multi-dimensional EEG features, adjacency matrix, and various fatigue labels; constructs a regularized graph neural network model including a signal diffuser, feature extractor, classifier, and domain discriminator, where the signal diffuser includes graph convolutional layers, and the domain discriminator includes gradient inversion layers and multiple fully connected layers; and incorporates cross-entropy classification loss, ... The domain adversarial loss and the regularization loss are weighted and summed to obtain the total loss function. The regularization loss includes adjacency matrix regularization and model weight regularization. Based on the test set and the training set, the regularized graph neural network model is trained using the total loss function to obtain the trained regularized graph neural network model. The EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected are preprocessed and then input into the trained regularized graph neural network model. Fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain the fatigue monitoring results.

[0017] Thus, by constructing an adjacency matrix using the three-dimensional coordinates of multiple electrodes and using this matrix as a learnable parameter in training, the model can adaptively learn the optimal connection pattern while preserving the biological prior that "the connection strength between adjacent brain regions decreases with distance," thus solving the problem of insufficient capture of inter-channel relationships in existing related technologies. Adversarial training using a classifier and domain discriminator forces the feature extractor to learn fatigue features unrelated to the subject's identity, improving the model's robustness and addressing the significant performance drop when applied to unfamiliar subjects, while also preventing overfitting during training. By comprehensively considering cross-entropy classification loss, domain adversarial loss, and regularization loss, and training the model using the total loss function, followed by fatigue monitoring using the trained model, the accuracy of fatigue monitoring can be improved. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic flowchart of an embodiment of the brainwave signal fatigue monitoring method provided in this application; Figure 2 This is a schematic diagram of the training process of the regularized graph neural network model in the preferred embodiment of the brainwave signal fatigue monitoring method provided in this application; Figure 3 This is a schematic diagram of the signal acquisition parameters and signal preprocessing method in the preferred embodiment of the EEG signal fatigue monitoring method provided in this application; Figure 4This is a schematic diagram of the regularized graph neural network model architecture in the preferred embodiment of the brainwave signal fatigue monitoring method provided in this application; Figure 5 This is a schematic diagram of an embodiment of the EEG signal fatigue monitoring system provided in this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0020] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0021] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0022] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0023] To address the issues of low accuracy and low robustness of fatigue monitoring models in related technologies, this application proposes a method, system, electronic device, and storage medium for monitoring fatigue using electroencephalogram (EEG) signals.

[0024] Reference Figure 1 This application provides a schematic flowchart of a brainwave signal fatigue monitoring method. This brainwave signal fatigue monitoring method is applied to an electronic device, which may be a server or a mobile terminal, etc. Figure 1 As shown, the brainwave signal fatigue monitoring method may include the following steps S101 to S107.

[0025] Step S101: Acquire eye movement signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes.

[0026] Specifically, the three-dimensional coordinates of 17 electrodes were acquired based on the international 10-20 system, resulting in multiple electrode three-dimensional coordinates. A 17-channel electroencephalogram (EEG) was acquired, covering the frontal, central, parietal, temporal, and occipital lobes; frequency coverage... (1 to 4 Hz) (4 to 8 Hz) (8 to 14 Hz) (14 to 31 Hz) and Five frequency bands (31 to 50 Hz) were used; each subject was sampled continuously for 2 hours, with an 8-second interval, for a total of 885 samples; a total of 23 subjects were included; this yielded multi-channel EEG signals with a data dimension of 17×885×5×23. Eye movement signals can be acquired via a camera, or other techniques known to those skilled in the art can be used to acquire eye movement signals; this embodiment does not impose specific limitations on this.

[0027] Step S102: Extract multidimensional EEG features from multi-channel EEG signals, construct an adjacency matrix based on the three-dimensional coordinates of multiple electrodes, and construct various fatigue labels through eye movement signals.

[0028] In this step, the multi-channel EEG signal is processed by moving average power spectral density, linear dynamic system power spectral density, moving average differential entropy, and linear dynamic differential entropy to obtain four EEG features; the four EEG features are then spliced ​​together to obtain multidimensional EEG features.

[0029] Specifically, Differential Entropy (DE) is a feature specifically designed for EEG emotion and fatigue analysis, and studies have shown that it outperforms traditional Power Spectral Density (PSD) in characterizing the energy distribution of brain activity. Linear Dynamic System (LDS) is a classic temporal smoothing algorithm. In EEG processing, it is commonly used to filter out random fluctuations in feature sequences while preserving long-term trends related to the evolution of fatigue states. Moving Average (MA) is the most basic temporal smoothing technique, reducing high-frequency noise by averaging through a sliding window.

[0030] Therefore, the four types of features in this embodiment—moving average power spectral density (psd_movingAve), linear dynamic system power spectral density (psd_LDS), moving average differential entropy (de_movingAve), and linear dynamic differential entropy (de_LDS)—are formed by combining the above techniques. Among them, PSD and DE are fundamental features characterizing EEG energy and complexity; moving average (MA) and linear dynamic system (LDS) are temporal smoothing strategies used to remove physiological interference.

[0031] The spatial distance between the three-dimensional coordinates of the two electrodes is calculated; initial weights are calculated using this spatial distance; and an adjacency matrix is ​​constructed based on these initial weights. Specifically, the spatial distance between the three-dimensional coordinates of the two electrodes is calculated using normalized Euclidean distance, and the initial weights are obtained by dividing the spatial distance by a scaling factor. The calculated initial weights are then filled into a diagonal matrix, and symmetry constraints are applied to generate a complete learnable adjacency matrix.

[0032] Based on eye movement signals, the percentage of eyelid closure is calculated; based on the percentage of eyelid closure, multiple fatigue labels are constructed. Specifically, based on eye movement signals, the percentage of eyelid closure is calculated, and multiple fatigue labels are classified according to the percentage of eyelid closure. For example, the multiple fatigue labels are classified as: (0, 0.35) for no fatigue, (0.35, 0.7) for mild fatigue, and (0.7, 1) for severe fatigue.

[0033] Step S103: Construct a test set and a training set from the multidimensional EEG features, adjacency matrix, and various fatigue labels.

[0034] Specifically, the dataset containing multidimensional EEG features, adjacency matrix, and various fatigue labels is divided into a test set and a training set according to a certain ratio. Both the test set and the training set contain multidimensional EEG features, adjacency matrix, and various fatigue labels.

[0035] Step S104: Construct a regularized graph neural network model including a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The signal diffuser includes a graph convolutional layer, and the domain discriminator includes a gradient inversion layer and multiple fully connected layers.

[0036] Specifically, the constructed regularized graph neural network model includes the following structure: The signal diffuser includes a graph convolutional layer (GC). The graph convolutional layer transforms the input 17×4×5 features into 17×2×5 features. LeakyReLU activation and L2 regularization are used to achieve hierarchical feature extraction.

[0037] The feature extractor consists of two fully connected layers (FC) responsible for reducing the dimensionality of high-dimensional graph sequences to a 16-dimensional abstract representation.

[0038] The domain discriminator consists of a gradient inversion layer (GR) and three fully connected layers (FC). During forward propagation, it performs an identity mapping, and during backward propagation, it multiplies the gradient by the negative gradient (which can be -). After being activated by Softmax, the output consists of two types of domain label values ​​(i.e., the domain discriminator prediction results).

[0039] The classifier consists of three fully connected (FC) layers, which receive a 16-dimensional abstract feature representation from the feature extractor and output the three-class fatigue probability (i.e., the classifier prediction result) after being activated by Softmax.

[0040] Step S105: The cross-entropy classification loss, domain adversarial loss, and regularization loss are weighted and summed to obtain the total loss function. The regularization loss includes adjacency matrix regularization and model weight regularization.

[0041] Specifically, the cross-entropy classification loss, domain adversarial loss, and regularization loss are weighted and summed to obtain the total loss function, which is calculated as follows: ; in, For the total loss function, Cross-entropy classification loss, For domain confrontation losses, The regularization loss includes adjacency matrix L1 regularization (i.e., adjacency matrix regularization) and model weight L2 regularization (i.e., model weight regularization)). and These are the weighting coefficients.

[0042] Step S106: Based on the test set and training set, train the regularized graph neural network model using the total loss function to obtain the trained regularized graph neural network model.

[0043] In this step, the test set and training set are randomly mixed, and the mixed dataset is input into a regularized graph neural network model to obtain the classifier prediction results and the domain discriminator prediction results. Wherein: The multidimensional EEG features and adjacency matrix from the mixed dataset are input into a signal diffuser to obtain the output of the signal diffuser. The output of the signal diffuser is then processed by a feature extractor to obtain an abstract feature representation. The abstract feature representation and various fatigue labels from the mixed dataset are input into a classifier, and the abstract feature representation is input into a domain discriminator. The classifier and the domain discriminator are then trained adversarially using a total loss function to obtain the prediction results of the classifier and the domain discriminator.

[0044] In backpropagation, the gradient of each loss in the total loss function is calculated based on the prediction results of the classifier and the domain discriminator.

[0045] Specifically, the first original gradient corresponding to adjacency matrix regularization and the second original gradient corresponding to model weight regularization are obtained; the first gradient corresponding to cross-entropy classification loss is calculated based on the classifier prediction results; the second gradient corresponding to domain adversarial loss is calculated based on the domain discriminator prediction results; and the third gradient corresponding to adjacency matrix regularization is calculated based on the first original gradient. Based on the second original gradient, calculate the fourth gradient corresponding to the model weight regularization.

[0046] Based on the classifier's prediction results, the first gradient corresponding to the cross-entropy classification loss is calculated. ,include: ; in, Represents the cross-entropy classification loss. This represents the output of the last fully connected layer of the classifier. This represents the classifier's prediction result. Indicates the first A standard unit vector with one component of 1 and the rest of 0.

[0047] Based on the prediction results of the domain discriminator, the second gradient corresponding to the domain adversarial loss is calculated. ,include: ; in, Indicates domain adversarial loss, This represents the output of the last fully connected layer of the domain discriminator. Indicates the prediction result of the domain discriminator. Let z be a unit vector whose z-th component is 1.

[0048] The above calculation of the third gradient corresponding to the adjacency matrix regularization based on the first original gradient includes: ; in, This is the first original gradient. It is an adjacency matrix. For symbolic functions, The third gradient, This represents the regularization coefficient.

[0049] The above calculation of the fourth gradient corresponding to the model weight regularization based on the second original gradient includes: ; in, For the second original gradient, It is the fourth gradient. This represents the weight matrix.

[0050] The parameters in the regularized graph neural network model are updated by gradients, and the model is then trained again with the updated parameters until the termination condition is met, resulting in a well-trained regularized graph neural network model.

[0051] Specifically, the first gradient corresponding to the training set samples is backpropagated in the classifier through domain labels to update the classifier parameters. The second gradient corresponding to the training set samples and the second gradient corresponding to the test set samples are backpropagated in the domain discriminator through domain labels to update the domain discriminator parameters. When backpropagating to the feature extractor, the second gradient corresponding to the domain discriminator is inverted and added to the first gradient corresponding to the classifier to obtain the added gradient. The parameters of the feature extractor are updated using the added gradient. The adjacency matrix is ​​updated using the third gradient. The weights (which can be weight matrices) in the regularized graph neural network model are updated using the fourth gradient.

[0052] Step S107: After preprocessing the EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected, input them into the trained regularized graph neural network model. Fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain fatigue monitoring results.

[0053] Specifically, once the regularized graph neural network model is trained on the test and training sets, fatigue monitoring can be performed using only the signal diffuser, feature extractor, and classifier from the trained model. The EEG signal to be detected and the three-dimensional coordinates of the electrodes to be detected are preprocessed (using the same processing method as during model training for multi-channel EEG signals and multiple electrode three-dimensional coordinates, which will not be repeated here) to obtain the multi-dimensional EEG features to be detected and the adjacency matrix to be detected. Then, the multi-dimensional EEG features to be detected and the adjacency matrix to be detected are input into the trained regularized graph neural network model for fatigue monitoring, yielding the fatigue monitoring results.

[0054] To facilitate understanding by those skilled in the art, a set of preferred embodiments is provided below: Existing fatigue monitoring technologies based on electroencephalography (EEG) can be mainly divided into two categories.

[0055] The first category is traditional machine learning methods, whose process typically includes: preprocessing the raw EEG signal (such as filtering and artifact removal), manually designing and extracting time-frequency domain features (such as power spectral density and differential entropy), and finally using classifiers such as Support Vector Machines (SVM) and Random Forests for fatigue state identification. For example, some researchers have combined power spectral density features with SVM for fatigue classification. The second category is deep learning-based methods, which utilize Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), or their variants (such as Long Short-Term Memory Networks (LSTMs)) to automatically learn features from EEG signals and perform classification. These methods typically treat multi-channel EEG signals as images (channels as different rows of an image) or sequential data, extracting deep features by stacking convolutional or recurrent layers, resulting in improved classification accuracy compared to traditional methods. However, existing related technologies have the following drawbacks: Disadvantage 1: Insufficient modeling of biological topological relationships between EEG signal channels.

[0056] Traditional methods (such as CNN and SVM) treat EEG signals as independent Euclidean spatial data, neglecting the spatial distribution of electrodes on the scalp and the biological topological characteristics of brain functional regions. This results in insufficient modeling of local anatomical connectivity and global functional coupling between channels. Neural activity in the brain exhibits significant spatial dependence, with interactions between adjacent or functionally related brain regions. However, existing models struggle to effectively model this non-Euclidean topology, leading to inadequate capture of inter-channel relationships.

[0057] Specifically, the following are the manifestations: (1) The topological structure of the EEG channels is not effectively utilized to learn more discriminative EEG representations; (2) The local and global relationships between EEG signal channels cannot be effectively captured; and (3) Spatial information is lost during processing, affecting classification performance.

[0058] Disadvantage 2: Insufficient generalization ability in cross-subject scenarios.

[0059] Significant individual differences exist in electroencephalogram (EEG) signals, with varying signal morphology, frequency band energy, and other characteristics among different subjects. Existing deep learning models rely on large amounts of labeled data, which can easily lead to overfitting to the characteristics of specific subjects during training. When applied to new, unseen subjects, their performance drops drastically, making it difficult to meet the needs of cross-individual monitoring in practical applications. This results in a sharp decline in classification accuracy across subjects and in scenarios with few samples.

[0060] Specifically, the differences are as follows: (1) There are significant differences in EEG signals among different subjects, which hinders the universality of the trained classifier in classification settings unrelated to the subjects; (2) The fatigue level detection and analysis method of single-channel EEG signals has certain limitations; (3) How to improve the accuracy of EEG signal feature fusion and improve the correlation between different features is a difficult point in the research of multi-channel EEG signals.

[0061] Disadvantage 3: Weak ability to handle noise tags and complex fatigue conditions.

[0062] Acquiring large amounts of accurately labeled fatigue EEG data is costly and time-consuming. Furthermore, fatigue is a continuous and gradual process, and the labeling of boundary states such as mild and moderate fatigue is ambiguous, leading to label noise. Hard-label classification methods struggle to accurately model the probability distribution. Existing models are prone to overfitting when there are few samples or noisy labels, resulting in significantly reduced classification accuracy and robustness.

[0063] Specifically, the fatigue state is continuously changing and there is a transition phase; hard labels force the continuous state to be discretized, resulting in labeling noise; and the classification boundary ambiguity problem affects the model performance.

[0064] The purpose of this embodiment is to provide a fatigue monitoring method for electroencephalogram (EEG) signals based on regularized graph neural networks, in order to solve the following three core technical problems existing in related technologies: First, this invention addresses the problem of insufficient modeling of the biological topological relationships between EEG signal channels. Existing technologies treat multi-channel EEG signals as independent Euclidean spatial data, neglecting the irregular three-dimensional spatial distribution of electrodes on the scalp surface and the biological topological characteristics of brain functional zones. This embodiment aims to explicitly model the non-Euclidean topological relationships between electrode channels using graph neural networks, fully utilizing prior knowledge of electrode spatial distribution.

[0065] Specifically, existing technologies (such as CNN and SVM) treat multichannel EEG signals as regular grid data or independent vectors, fundamentally because they lack a computational framework capable of explicitly modeling non-Euclidean topological structures. Brain electrode channels naturally form a graph structure, a fact that current technologies fail to utilize.

[0066] Therefore, this embodiment introduces a graph neural network as the core computational framework, treating each EEG electrode as a graph node. To ensure that the adjacency matrix encodes meaningful biological priors, a bio-inspired adjacency matrix construction method based on the three-dimensional spatial coordinates of the electrodes is designed: electrode coordinates are obtained based on the international 10-20 system, and normalized Euclidean distance formula is used... Calculate the initial connection weights, where You can set it to 50 and apply a symmetric constraint. The adjacency matrix serves as a learnable parameter in training, preserving the biological prior that "the connection strength between adjacent brain regions decreases with distance" while allowing the model (in this embodiment, the model can be a regularized graph neural network model) to adaptively learn the optimal connection pattern.

[0067] Second, this invention addresses the issue of insufficient model generalization ability in cross-subject scenarios. Significant individual differences exist in the EEG signals of different subjects, causing existing deep learning models to experience a substantial performance drop in cross-subject scenarios. This embodiment aims to learn fatigue state feature representations independent of subject identity through an adversarial training strategy.

[0068] Specifically, the fundamental problem with existing related technologies is that the training objective only includes classification loss and does not impose constraints on the "subject invariance" of feature representations, which makes it easy for the model to learn "shortcut features" that are related to the subject's identity but not to fatigue, and thus cannot generalize to new subjects.

[0069] Therefore, this embodiment introduces a node-level domain adversarial training strategy. A gradient reversal layer is introduced at the intermediate layer output of the fully connected feature extractor and connected to the domain discriminator branch. The gradient reversal layer performs an identity mapping during forward propagation and multiplies the gradient from the domain discriminator by a negative coefficient (which can be...) during backward propagation. The domain discriminator aims to minimize the domain classification loss (accurately distinguishing the subject's origin), while the feature extractor, receiving the optimization signal through gradient inversion, aims to maximize the domain classification loss (generating features that the domain discriminator cannot distinguish). This adversarial game forces the feature extractor to learn fatigue features unrelated to the subject's identity.

[0070] Third, this paper addresses the issue of insufficient robustness of models under a single feature perspective. Existing methods typically employ only a single feature extraction approach, resulting in a limited feature perspective. This embodiment aims to comprehensively characterize the time-frequency characteristics of EEG signals from multiple angles through a multi-feature fusion strategy, thereby enhancing the richness of input information.

[0071] Specifically, the fundamental problem with existing related technologies is that single feature extraction methods can only capture one aspect of the time-frequency characteristics of EEG signals. Power spectral density reflects energy distribution, while differential entropy reflects signal complexity; the two are complementary.

[0072] Therefore, this embodiment designs a multi-feature fusion preprocessing strategy, employing four complementary methods: moving average power spectral density (psd_movingAve), linear dynamic system power spectral density (psd_LDS), moving average differential entropy (de_movingAve), and linear dynamic differential entropy (de_LDS). Starting from the two feature dimensions of power spectral density and differential entropy, and the two temporal smoothing strategies of moving average and linear dynamic system, a 4×17×5 multi-view feature matrix is ​​constructed to provide rich input for the graph neural network.

[0073] The technical solution of this embodiment specifically includes the following: I. Overall System Architecture.

[0074] like Figure 2As shown, the overall workflow of the system in this embodiment includes three levels: 1. Input Layer: Receives three types of input. 1) Eye movement signals are processed to generate fatigue labels based on eyelid closure percentage (PERCLOS) values ​​as supervision signals; 2) Multi-channel EEG signals are sent to the preprocessing module; 3) Electrode 3D coordinates are used to construct an adjacency matrix.

[0075] 2. Core Processing Layer: Contains two key sub-modules. The graph convolutional neural network signal diffuser (i.e., the signal diffuser) receives preprocessed multidimensional EEG features and an adjacency matrix, propagating information and aggregating features between nodes through graph convolution to capture spatial topological dependencies. The fully connected neural network feature extractor (i.e., the feature extractor) performs nonlinear transformations and dimensionality reduction on the graph convolution output (i.e., the output of the signal diffuser) to extract high-level abstract features.

[0076] 3. Output Layer: Contains two parallel fully connected networks. The fully connected neural network classifier (i.e., the classifier) ​​receives the output of the feature extractor (i.e., the abstract feature representation) and the fatigue label, and outputs the classification prediction result (i.e., the classifier's prediction result). The fully connected neural network domain discriminator (i.e., the domain discriminator) receives the output of the feature extractor and attempts to distinguish which subject the features originated from. The two networks work against each other through a gradient inversion layer: the classifier learns fatigue-related but subject-independent features, while the domain discriminator attempts to identify the subject's origin. This adversarial mechanism forces the feature extractor to produce subject-invariant feature representations.

[0077] II. Data Acquisition and Preprocessing Module.

[0078] This embodiment uses the publicly available SEED-VIG driver fatigue dataset.

[0079] Signal acquisition parameters (such as) Figure 3 (As shown on the left): 17-channel EEG, covering the frontal, central, parietal, temporal, and occipital lobes; frequency coverage... (1 to 4 Hz) (4 to 8 Hz) (8 to 14 Hz) (14 to 31 Hz) and Five frequency bands (31 to 50 Hz); each subject was sampled continuously for 2 hours, with an 8-second interval, for a total of 885 samples; a total of 23 subjects; thus, multi-channel EEG signals were obtained, with a data dimension of 17×885×5×23.

[0080] Signal preprocessing methods (such as) Figure 3(As shown on the right): The original signal is processed in four ways: moving average power spectral density (psd_movingAve), linear dynamic system power spectral density (psd_LDS), moving average differential entropy (de_movingAve), and linear dynamic differential entropy (de_LDS). After processing, the data dimension is expanded to 4×17×885×5×23, that is, four feature dimensions are added, resulting in multidimensional EEG features.

[0081] Fatigue value label generation: Based on the PERCLOS value calculated from eye movement signals, the values ​​are divided into three categories: (0, 0.35) for no fatigue, (0.35, 0.7) for mild fatigue, and (0.7, 1) for severe fatigue.

[0082] It should be noted that the PERCLOS value calculated based on eye movement signals in this embodiment can be obtained by methods known to those skilled in the art, and this embodiment will not describe it in detail.

[0083] Differential entropy (DE) is a feature specifically designed for EEG emotion and fatigue analysis. Studies have shown that it outperforms traditional power spectral density (PSD) in characterizing the energy distribution of brain activity. Linear Dynamic System (LDS) is a classic temporal smoothing algorithm. In EEG processing, it is commonly used to filter out random fluctuations in feature sequences while preserving long-term trends related to the evolution of fatigue states. Moving Average (MA) is the most basic temporal smoothing technique, reducing high-frequency noise by averaging through a sliding window.

[0084] Therefore, the four types of features in this embodiment—moving average power spectral density (psd_movingAve), linear dynamic system power spectral density (psd_LDS), moving average differential entropy (de_movingAve), and linear dynamic differential entropy (de_LDS)—are formed by combining the above techniques. Among them, PSD and DE are fundamental features characterizing EEG energy and complexity; moving average (MA) and linear dynamic system (LDS) are temporal smoothing strategies used to remove physiological interference.

[0085] III. Adjacency Matrix Construction Module.

[0086] This embodiment innovatively designs a bio-inspired adjacency matrix construction method based on the three-dimensional coordinates of electrodes: The three-dimensional coordinates of 17 electrodes were obtained based on the international 10-20 system. Adjacency relationships between the electrode three-dimensional coordinates were defined based on physical layout and neuroanatomy. For connected electrode pairs, initial weights were calculated using normalized Euclidean distance based on their respective electrode three-dimensional coordinates. ,in, Electrode and electrodes Spatial distance and scaling factor in the international 10-20 system The weight is set to 50. For electrode pairs without physical or anatomical connections, the weight is reset to 0. The closer the physical distance, the greater the weight, encoding the prior that "connection strength decreases with distance." The normalized Euclidean distance in this embodiment is a distance calculation method known to those skilled in the art, and will not be described in detail in this embodiment.

[0087] Then forced This aligns with the physiological characteristics of functional symmetry in the cerebral hemispheres, and the parameter values ​​are adjusted from... Reduce to Initialize a 17×17 diagonal matrix, fill in the calculated weights, and apply symmetric constraints to generate a complete learnable adjacency matrix.

[0088] IV. Regularized Graph Neural Network Model Architecture.

[0089] like Figure 4 As shown, the core architecture and hyperparameters of the model are as follows: Hyperparameters include: learning rate 0.0005, batch size 16, L2 regularization coefficient 0.0001, and Adam optimizer (…). ), 100 training rounds.

[0090] The adjacency matrix (17×17) and preprocessed data (4×17×5 per sample) are used as inputs to the regularized graph neural network model. The regularized graph neural network model includes the following structure: Signal diffuser: a single graph convolutional layer (GC). Graph convolutional layer operation: transforms the input 17×4×5 features into 17×2×5 features. LeakyReLU activation function and L2 regularization are used to achieve hierarchical feature extraction.

[0091] Feature extractor: It consists of two fully connected layers (FC) responsible for reducing the dimensionality of high-dimensional graph sequences to a 16-dimensional abstract representation.

[0092] Domain discriminator (for gradient inversion layer (GR) and domain adversarial training): Contains a gradient inversion layer (GR) and three fully connected layers (FC). It performs an identity mapping during forward propagation and multiplies the gradient by the negative gradient (which can be -) during backward propagation. After being activated by Softmax, the output consists of two types of domain label values ​​(i.e., the domain discriminator prediction results).

[0093] Classifier: It consists of three fully connected (FC) layers, which receive the 16-dimensional abstract feature representation output by the feature extractor and output the three-class fatigue probability (i.e. the classifier prediction result) after Softmax activation.

[0094] V. Loss Function and Joint Optimization Strategy.

[0095] (1) Forward propagation - Improved Softmax activation function.

[0096] Let the output of the last fully connected layer of the classifier be... (C is the number of categories; C=3 for three categories). To avoid exponent explosion, a modified Softmax is used: ; in, for The maximum value of each component in the middle. This represents the output of the last fully connected layer of the classifier. Each component value. This formula is mathematically equivalent to the standard Softmax, but by subtracting the maximum value, all exponential inputs are made non-positive, and the output falls within the (0,1] interval, thus avoiding numerical overflow.

[0097] (2) Cross-entropy classification loss function.

[0098] Let the true class of the sample be y (hard label, The corresponding classifier output vector is Then the cross-entropy classification loss is: ; That is, only the components corresponding to the actual categories are taken. The logarithmic probability of the group is used, while the other groups are not included in the calculation.

[0099] (3) The gradient corresponding to the cross-entropy classification loss (i.e., the first gradient).

[0100] set up For the first A standard unit vector (one-hot vector) with one component of 1 and the rest of 0, then right The gradient is: ; in, This is the output vector of the Softmax classifier (i.e., the prediction result of the classifier). This indicates the transpose. The gradient can be computed in-place, requiring only... The Each component is reduced by 1, eliminating the need for additional storage of intermediate variables, and the memory usage equals the size of the output dimension.

[0101] (4) Prevent gradient propagation on the test set.

[0102] Using the domain label z to control gradient propagation in the classifier branches: ; When z=0 (test set sample), the gradient is zero and does not participate in classifier training; when z=1 (training set sample), the gradient propagates normally.

[0103] (5) The gradient corresponding to the domain adversarial loss (i.e., the second gradient).

[0104] Let the output of the last fully connected layer of the domain discriminator be (Binary classification, (i.e., distinguishing whether a sample belongs to the test set or the training set), the real domain label is... Similarly: ; ; in, This indicates the component corresponding to the real domain label. This is the Softmax probability vector output by the domain discriminator (i.e., the domain discriminator prediction result). Let z be a unit vector with the z-th component being 1. In the domain discriminator, the gradients of both the training and test set samples propagate normally (unaffected by z-shading).

[0105] (6) Gradient merging and inversion.

[0106] When the two gradients propagate to the feature extractor (i.e., at the output of the fully connected layer 32→16), the gradient corresponding to the domain discriminator is inverted by the gradient inversion layer and added to the gradient corresponding to the classifier, jointly driving the feature extractor update: ; in, This is the 16-dimensional feature vector output by the feature extractor. This mechanism forces the feature extractor to learn features that the domain discriminator cannot distinguish from fatigue-related features originating from the subject.

[0107] (7) Regularized gradient.

[0108] Let the weight matrix of a certain layer be... The original gradient is The gradient after L1 / L2 regularization is The regularization coefficient is ,but: L1 regularization of adjacency matrix: This leads to sparsification of the adjacency matrix; among which, This is the first original gradient. It is an adjacency matrix. For symbolic functions, It is the third gradient.

[0109] Model weight L2 regularization: To prevent the model from overfitting; among them, For the second original gradient, It is the fourth gradient.

[0110] Based on the above, the specific process steps of this embodiment are as follows: S1. Data Acquisition and Multi-Feature Preprocessing. Acquire multi-channel EEG and eye-tracking signals. Extract features from the EEG signals using four methods (psd_movingAve, psd_LDS, de_movingAve, and de_LDS), covering... , , , as well as Five frequency bands were used, and each sample generated a multi-view node feature matrix (i.e., multi-dimensional EEG features) with a dimension of 4×17×5. The eyelid closure percentage (PERCLOS) value was calculated based on eye movement signals and divided into three hard labels (i.e., fatigue labels) according to the intervals (0, 0.35), (0.35, 0.7), and (0.7, 1), which served as the supervision signal for the classifier.

[0111] S2. Construct a bio-inspired adjacency matrix. Obtain the three-dimensional coordinates of 17 electrodes, calculate the initial weights using the method described above, and apply symmetric constraints to generate a 17×17 learnable adjacency matrix.

[0112] S3. Construct a regularized graph neural network model. The regularized graph neural network model consists of four parts: a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The overall architecture is as follows: Figure 2 As shown.

[0113] (1) Signal diffuser: Consists of a single graph convolutional layer (graph convolutional layer: 17×4×5→17×2×5, LeakyReLU activation, L2 regularization). The graph convolutional layer receives the node feature matrix from step S1 and the adjacency matrix from step S2, and aggregates information between nodes through the following operations: ; in, The output of the graph convolutional layer. The input node feature matrix (which can be the adjacency matrix from step S2) ), For adjacency matrices with self-loops, It is the identity matrix. For degree matrix, This is a learnable weight matrix.

[0114] (2) Feature extractor: It consists of two fully connected layers. The first fully connected layer (fully connected layer 170→32, LeakyReLU activation, L2 regularization) flattens and reduces the dimensionality of the output of the graph convolutional layer (i.e. the output of the signal diffuser); the second fully connected layer (fully connected layer 32→16, LeakyReLU activation, L2 regularization) processes the output of the first fully connected layer and outputs a 16-dimensional abstract feature representation.

[0115] (3) Classifier: It consists of three fully connected layers (fully connected layers 16→8→4→3, LeakyReLU activation, L2 regularization), which receive the 16-dimensional features output by the feature extractor, and output the three-class fatigue probability after Softmax activation, corresponding to three hard label categories.

[0116] (4) Domain Discriminator: Composed of a gradient reversal layer (GR) and three fully connected layers (fully connected layers 16→8→4→2, LeakyReLU activation, L2 regularization), which output a 2D vector after Softmax activation to distinguish whether a sample comes from the training set (domain label z=1) or the test set (domain label z=0). The gradient reversal layer performs an identity mapping during forward propagation and multiplies the gradient by the negative gradient (which can be -) during backward propagation. Then send it back.

[0117] S4. Introduce a node-level domain adversarial training module. Building upon step S3, a gradient inversion layer (GR) is introduced at the output of the second layer (fully connected layer 32→16) of the feature extractor, connecting to a three-layer fully connected domain discriminator (fully connected layer 16→8→4→2, LeakyReLU activation, L2 regularization). The output is a 2D vector used to distinguish the subject source of a sample, i.e., whether it comes from the test set or the training set. During training, training and test set samples are randomly mixed and input simultaneously. In the classifier branch, only the gradient corresponding to the training set samples (i.e., the first gradient) participates in backpropagation (controlled by the domain label z; gradient propagation occurs when z=1, and propagation is blocked when z=0). In the domain discriminator branch, the gradients corresponding to both training and test set samples (i.e., the second gradient) propagate normally and are updated in an adversarial manner after passing through the gradient inversion layer. During training, regularized gradients (including the third and fourth gradients) are also used to update the adjacency matrix and model weights.

[0118] S5. Construct a joint optimization framework. Building upon step S4, integrate the cross-entropy classification loss, domain adversarial loss, and regularization loss into a unified framework for end-to-end training: ; in, For the total loss function, Cross-entropy classification loss, For domain confrontation losses, The regularization loss includes adjacency matrix L1 regularization (i.e., adjacency matrix regularization) and model weight L2 regularization (i.e., model weight regularization). The Adam optimizer is used. , (Learning rate: 0.005, batch size: 32) End-to-end joint optimization of all learnable parameters, training 100 epochs.

[0119] It should be noted that the cross-entropy classification loss and domain adversarial loss in this embodiment are calculated using loss functions known to those skilled in the art, and will not be described in detail in this embodiment. Adjacency matrix L1 regularization is calculated by performing L1 regularization on the adjacency matrix, and model weight L2 regularization is calculated by performing L2 regularization on the model weights.

[0120] S6. Model Evaluation and Deployment. Based on step S5, leave-one-subject-out (LOSO) method is used for 23-fold cross-validation to evaluate the cross-subject generalization ability of the regularized graph neural network model.

[0121] Reference Figure 5 This application also provides a brainwave signal fatigue monitoring system, which includes: The data acquisition module 100 is used to acquire eye movement signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes. The data processing module 200 is used to extract multidimensional EEG features from multi-channel EEG signals, construct an adjacency matrix based on the three-dimensional coordinates of multiple electrodes, and construct various fatigue labels through eye movement signals. Dataset building module 300 is used to construct test and training sets from multidimensional EEG features, adjacency matrices, and various fatigue labels; The model building module 400 is used to build a regularized graph neural network model that includes a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The signal diffuser includes a graph convolutional layer, and the domain discriminator includes a gradient inversion layer and multiple fully connected layers. The weighted summation module 500 is used to perform a weighted summation of the cross-entropy classification loss, domain adversarial loss, and regularization loss to obtain the total loss function. The regularization loss includes adjacency matrix regularization and model weight regularization. The model training module 600 is used to train a regularized graph neural network model based on the test set and training set, using the total loss function, to obtain a trained regularized graph neural network model. The fatigue monitoring module 700 is used to preprocess the EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected, and then input them into a trained regularized graph neural network model. The fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain the fatigue monitoring results.

[0122] It should be noted that since the EEG signal fatigue monitoring system in this embodiment is based on the same inventive concept as the EEG signal fatigue monitoring method described above, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0123] Reference Figure 6 This application also provides an electronic device, which includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the brainwave signal fatigue monitoring method described above in this disclosure.

[0124] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0125] The electronic devices according to embodiments of this application will now be described in detail.

[0126] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and called by the processor 1600 to execute the EEG signal fatigue monitoring method of the embodiments of this disclosure.

[0127] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0128] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described electroencephalogram (EEG) signal fatigue monitoring method.

[0129] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0131] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0134] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0135] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to existing related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The embodiments of this application have been described in detail above with reference to the accompanying drawings, but this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

[0140] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for monitoring fatigue using electroencephalogram (EEG) signals, characterized in that, The method includes: Acquire eye movement signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes; Multidimensional EEG features are extracted from the multi-channel EEG signals, an adjacency matrix is ​​constructed based on the three-dimensional coordinates of the multiple electrodes, and various fatigue labels are constructed using the eye movement signals. The multidimensional EEG features, the adjacency matrix, and the various fatigue labels are used to construct a test set and a training set. A regularized graph neural network model is constructed, comprising a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The signal diffuser includes a graph convolutional layer, and the domain discriminator includes a gradient inversion layer and multiple fully connected layers. The total loss function is obtained by weighted summation of cross-entropy classification loss, domain adversarial loss, and regularization loss. The regularization loss includes adjacency matrix regularization and model weight regularization. Based on the test set and the training set, the regularized graph neural network model is trained using the total loss function to obtain a trained regularized graph neural network model. The EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected are preprocessed and then input into the trained regularized graph neural network model. Fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain fatigue monitoring results.

2. The method for monitoring brainwave signal fatigue according to claim 1, characterized in that, The extraction of multidimensional EEG features from the multichannel EEG signals includes: The multi-channel EEG signal was processed by moving average power spectral density, linear dynamic system power spectral density, moving average differential entropy, and linear dynamic differential entropy to obtain four EEG features. The four types of EEG features are spliced ​​together to obtain multidimensional EEG features.

3. The method for monitoring brainwave signal fatigue according to claim 1, characterized in that, The construction of the adjacency matrix based on the three-dimensional coordinates of the multiple electrodes includes: Calculate the spatial distance between the two electrodes in three-dimensional coordinates; The initial weights are calculated using the spatial distance; Based on the initial weights, an adjacency matrix is ​​constructed.

4. The method for monitoring brainwave signal fatigue according to claim 1, characterized in that, The method of constructing multiple fatigue labels using the eye-tracking signals includes: Based on the eye movement signals, the eyelid closure percentage value is calculated; Based on the stated eyelid closure percentage, various fatigue labels are constructed.

5. The method for monitoring brainwave signal fatigue according to claim 1, characterized in that, The step of training the regularized graph neural network model based on the test set and the training set using the total loss function to obtain a trained regularized graph neural network model includes: The test set and the training set are randomly mixed, and the mixed dataset is input into the regularized graph neural network model to obtain the classifier prediction result and the domain discriminator prediction result; wherein: The multidimensional EEG features and adjacency matrix from the mixed dataset are input into the signal diffuser to obtain the output of the signal diffuser; The feature extractor extracts features from the output of the signal diffuser to obtain an abstract feature representation. The abstract feature representation and multiple fatigue labels in the mixed dataset are input into the classifier, and the abstract feature representation is input into the domain discriminator. The classifier and the domain discriminator are then subjected to adversarial training using the total loss function to obtain the classifier prediction result and the domain discriminator prediction result. In backpropagation, the gradient of each loss in the total loss function is calculated based on the prediction results of the classifier and the prediction results of the domain discriminator. The parameters in the regularized graph neural network model are updated using the gradient, and the model is trained again using the updated parameters until the termination condition is met, resulting in a well-trained regularized graph neural network model.

6. The method for monitoring brainwave signal fatigue according to claim 5, characterized in that, The step of calculating the gradient of each loss in the total loss function based on the prediction results of the classifier and the domain discriminator includes: Obtain the first original gradient corresponding to the adjacency matrix regularization and the second original gradient corresponding to the model weight regularization; Based on the classifier prediction results, calculate the first gradient corresponding to the cross-entropy classification loss; Based on the prediction results of the domain discriminator, the second gradient corresponding to the domain adversarial loss is calculated; Based on the first original gradient, calculate the third gradient corresponding to the adjacency matrix regularization; Based on the second original gradient, calculate the fourth gradient corresponding to the model weight regularization.

7. The method for monitoring brainwave signal fatigue according to claim 6, characterized in that, Updating the parameters in the regularized graph neural network model through the gradient includes: The first gradient corresponding to the training set samples is backpropagated in the classifier by controlling the domain label, so as to update the parameters of the classifier by the first gradient corresponding to the training set samples; The second gradient corresponding to the training set samples and the second gradient corresponding to the test set samples are backpropagated in the domain discriminator by controlling the domain label, so as to update the parameters of the domain discriminator by the second gradient corresponding to the training set samples and the second gradient corresponding to the test set samples; When backpropagating to the feature extractor, the second gradient corresponding to the domain discriminator is inverted and added to the first gradient corresponding to the classifier to obtain the added gradient; the parameters of the feature extractor are updated using the added gradient. The adjacency matrix is ​​updated using the third gradient. The weights in the regularized graph neural network model are updated using the fourth gradient.

8. A brainwave signal fatigue monitoring system, characterized in that, The system includes: The data acquisition module is used to acquire eye movement signals, multi-channel EEG signals, and three-dimensional coordinates of multiple electrodes; The data processing module is used to extract multidimensional EEG features from the multi-channel EEG signals, construct an adjacency matrix based on the three-dimensional coordinates of the multiple electrodes, and construct various fatigue labels through the eye movement signals. The dataset construction module is used to construct the multidimensional EEG features, the adjacency matrix, and the various fatigue labels into a test set and a training set; The model building module is used to build a regularized graph neural network model including a signal diffuser, a feature extractor, a classifier, and a domain discriminator. The signal diffuser includes a graph convolutional layer, and the domain discriminator includes a gradient inversion layer and multiple fully connected layers. The weighted summation module is used to sum the cross-entropy classification loss, domain adversarial loss, and regularization loss to obtain the total loss function. The regularization loss includes adjacency matrix regularization and model weight regularization. The model training module is used to train the regularized graph neural network model based on the test set and the training set, using the total loss function, to obtain a trained regularized graph neural network model. The fatigue monitoring module is used to preprocess the EEG signal to be detected and the three-dimensional coordinates of the electrode to be detected, and then input them into the trained regularized graph neural network model. The fatigue monitoring is performed through the signal diffuser, feature extractor and classifier in the trained regularized graph neural network model to obtain the fatigue monitoring results.

9. An electronic device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the electroencephalogram (EEG) signal fatigue monitoring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the electroencephalogram (EEG) signal fatigue monitoring method as described in any one of claims 1 to 7.

Images