Cross-user vigilance monitoring method based on subspace distance

By constructing a cross-user vigilance monitoring method based on subspace distance, and utilizing deep feature fusion and transfer learning of EEG and EEG signals, the problems of insufficient feature extraction and new user labeled data collection were solved, achieving high-precision and stable vigilance monitoring.

CN118885797BActive Publication Date: 2026-11-10XIDIAN UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411018976.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2026-11-10
Estimated Expiration
2044-07-29

AI Technical Summary

Technical Problem

In existing technologies, the feature extraction of EEG and EEG signals is insufficient, and spatial distribution differences are ignored during modality fusion, resulting in insufficient accuracy of alertness monitoring. Furthermore, new users require time-consuming and laborious data collection and annotation, making it difficult to achieve convenient cross-user alertness monitoring.

Method used

Deep EEG features are extracted using multi-scale spatial adaptive feature modulation and convolutional neural network modules, while deep EOG features are extracted using outer product embedding and convolutional neural network modules. Feature fusion is performed using an explicit spatial mapping module, and subspace distance and base mismatch penalty are calculated through singular value decomposition. A cross-user alertness monitoring model is constructed and trained using existing source domain data.

Benefits of technology

It improves the accuracy and stability of cross-user vigilance monitoring, eliminates the need for new user-labeled data, enables convenient cross-user vigilance monitoring, and enhances the model's feature representation ability and transfer learning performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118885797B_ABST
    Figure CN118885797B_ABST
Patent Text Reader

Abstract

This invention discloses a cross-user alertness monitoring method based on subspace distance, solving the problems of single modal data, insufficient correlation, and large error fluctuations in cross-user alertness monitoring. The implementation scheme includes: data acquisition and preprocessing; construction of a cross-user alertness monitoring model; model training; and user alertness monitoring. This invention uses source and target domain features to calculate subspace distance, base mismatch penalty, and root mean square error as the total model loss. The model is trained by minimizing the total loss, and the model is used to monitor users to obtain alertness values. This invention improves the accuracy and stability of cross-user alertness monitoring by using multimodal data fusion and a subspace distance-based transfer learning algorithm, reducing alertness monitoring errors. It can be used for real-time continuous monitoring of user alertness in fields such as human-computer interaction, traffic driving, and aviation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction technology, and mainly relates to alertness monitoring, specifically a cross-user alertness monitoring method based on subspace distance. It can be applied to research fields such as traffic driving, aviation flight, and engineering operations, specifically for real-time continuous monitoring of subject alertness during human-computer interaction. Background Technology

[0002] During prolonged continuous driving or engineering operations, a user's alertness gradually declines with increasing operation time, posing a significant safety hazard. To address this issue, various research methods have been proposed to monitor alertness levels and prevent serious accidents caused by insufficient alertness. Among these, electroencephalogram (EEG) and electrooculogram (EOG) signals are widely used for alertness monitoring due to their objectivity and robustness.

[0003] Currently, there are many challenges in methods for monitoring alertness based on EEG and EEG signals. For example, the extraction of deep alertness-related features from EEG and EEG is insufficient, leading to inadequate accuracy of alertness monitoring network models and making it difficult to achieve precise alertness monitoring. Furthermore, because EEG and EEG physiological signals are easily affected by individual differences, pre-trained models may perform well on training user data but show significant errors on new users who differ greatly from the training users, indicating poor generalization and robustness. Therefore, designing a cross-user alertness monitoring method based on EEG and EEG signals has become a meaningful and challenging task.

[0004] The University of Electronic Science and Technology of China disclosed a method for monitoring cross-user alertness of EEG signals based on multi-source domain adaptation in its patent application document "A method for monitoring cross-user alertness of EEG signals based on multi-source domain adaptation" (patent application number: CN202211330654.1, authorization number: CN115470830A). This method uses EEG signals from multiple users for preprocessing and as multiple source domains. For each source domain, a feature extractor is set to extract spatial and temporal features of the EEG signals. Two task classifiers are set to classify the features. An adversarial domain adaptation strategy is used to train the feature extractor and task classifier to optimize the distribution of the extracted features and reduce the impact of confusion at the feature decision boundary caused by cross-user interaction. This method improves the accuracy of monitoring users' alertness status in cross-user contexts. However, the method still has some shortcomings. A single EEG signal only reflects the alertness status from brain activity information, resulting in a single information dimension and poor robustness. Furthermore, this method only performs qualitative classification analysis of alertness, transforming the continuous state of alertness into discrete categories, which leads to the loss of subtle information about alertness and makes it difficult to capture its dynamic changes.

[0005] Xinyu Cheng et al., in their paper "VigilanceNet: Decouple Intra-and Inter-Modality Learning for Multimodal Vigilance Estimation in RSVP-Based BCI," disclosed a continuous alertness monitoring method based on EEG and EOG signals. This method uses EEG and EOG signals to perform regression monitoring of alertness, primarily employing intramodal representation learning. It captures complementary information between EEG and EOG signals through a cross-attention-based transformer to achieve continuous regression monitoring of alertness. This method shows improvements over existing methods in terms of regression error and correlation evaluation metrics. However, a remaining drawback is the time-consuming and laborious need for labeled data collection when new users adopt the method, hindering their convenient and quick use. Therefore, designing an algorithm that can effectively adapt to target subjects using source domain data is crucial, maximizing the use of existing source domain data to provide new users with a more accurate and convenient method.

[0006] Existing technologies suffer from insufficient feature extraction and neglect of spatial distribution differences between different modalities during modality fusion, resulting in inadequate model accuracy and monitoring stability. Furthermore, new users often need to perform time-consuming and laborious labeled data collection when using alertness monitoring models, which hinders their ability to easily and quickly perform alertness monitoring. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of the prior art by proposing a cross-user alertness monitoring method based on subspace distance that integrates two modal data, EEG and EEG signals, to achieve high alertness monitoring accuracy and eliminates the need for new user-labeled data.

[0008] This invention is a cross-user vigilance monitoring method based on subspace distance. Its key features include: extracting deep EEG features through multi-scale spatial adaptive feature modulation and a convolutional neural network module; extracting deep EEG features through outer product embedding and a convolutional neural network module; fusing the deep EEG and deep EEG features using an explicit spatial mapping module; calculating the subspace distance RSD and base mismatch penalty BMP using singular value decomposition on the fused feature matrix; calculating the root mean square error RMSE using a multilayer perceptron; and training the cross-user vigilance monitoring model using the subspace distance, base mismatch penalty, and RMSE as the total loss through backpropagation. The method comprises the following steps:

[0009] (1) Data acquisition and preprocessing: EEG and EEG signal data of users, i.e., traffic drivers, are acquired and preprocessed, including downsampling, artifact removal, time window division, and shallow feature extraction. Differential entropy of 17 channels in the EEG signal with an 8s time window and a 2Hz step size is obtained as shallow EEG features. Blinking, saccade, and fixation time in the EEG signal with an 8s time window are obtained as shallow EEG features. The data is divided into training set and test set, with the training set divided into source domain data and target domain data.

[0010] (2) Constructing a cross-user vigilance monitoring model W: Using source domain data and target domain data from the training set, deep features are extracted from EEG and EOS data through parallel branches. EEG data is processed through an EEG deep feature extraction module, which includes multi-scale spatial adaptive feature modulation and a convolutional neural network, to obtain deep EEG features. EOS data is processed through an EOS deep feature extraction module, which includes outer product embedding and a convolutional neural network, to obtain deep EOS features. The two deep features are fused using a spatial explicit mapping module to obtain source domain features and target domain features. The source domain features and target domain features in the fused features are then compared using a domain-invariant feature extraction module to calculate the subspace distance and base mismatch penalty as L. RSD Loss and L BMP The loss is calculated by comparing the root mean square error (RMSE) of the monitored values ​​obtained from the source domain features through the alertness regression monitoring module based on a multilayer perceptron with the true values. RMSE loss;

[0011] The deep feature extraction branch of EEG is sequentially connected to a multi-scale spatial adaptive feature modulation and a convolutional neural network, while the deep feature extraction branch of EOS is sequentially connected to an outer product embedding and a convolutional neural network.

[0012] The spatial explicit mapping module realizes feature fusion, maps the deep features of EOG to the deep feature space of EEG, and performs feature fusion through convolution operation to obtain source domain fused features and target domain fused features;

[0013] In the domain-invariant feature extraction module, the source domain fusion features and target domain fusion features are calculated using singular value decomposition to obtain the orthogonal basis of the source domain subspace and the orthogonal basis of the target domain subspace. The subspace distance RSD and the base mismatch penalty BMP are then calculated using these orthogonal basis bases as L... RSD Loss and L BMP loss;

[0014] In the alertness regression monitoring module, the source domain fusion features are used to obtain the monitored alertness value through a multilayer perceptron containing two hidden layers. The root mean square error (RMSE) between the monitored value and the actual value is calculated as L. RMSE loss;

[0015] (3) Model training: Using the training set, the subspace distance loss L RSD Base mismatch penalty loss L BMP and root mean square error loss L RMSE The sum of these is used as the total loss L of the model. total Minimize the total loss L using backward iterative propagation total , until the loss converges, and the trained cross-user vigilance monitoring model W is obtained;

[0016] (4) User vigilance monitoring: Input the test set into the trained cross-user vigilance monitoring model W and output vigilance monitoring values.

[0017] This invention mainly addresses the problems of insufficient use of EEG and EEG feature extraction in current technologies, neglect of spatial distribution differences during feature fusion, and insufficient accuracy of traditional transfer learning methods in alertness regression monitoring due to changes in feature scale.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] This invention integrates two modalities of EEG and EEG signals: By constructing deep feature extraction and feature fusion modules for EEG and EEG, it captures global and multi-scale information in EEG data through multi-scale spatial feature modulation and captures the intrinsic relationships in EEG data through outer product embedding, thereby enriching the feature expression of EEG and EEG data. Furthermore, it maps EEG features to the EEG feature space through a spatial explicit mapping method, avoiding insufficient feature fusion caused by the differences in the spatial distribution of EEG and EEG features, and solving the problem of insufficient accuracy and stability of alertness monitoring in existing technologies.

[0020] No new user labeled data required: This invention uses a transfer learning method based on subspace distance, which can train the model using existing labeled source domain data and new user data without collecting labeled data. This avoids the need for new users to use the vigilance monitoring model without labeled data, making it convenient for new users to use the vigilance monitoring model.

[0021] Efficient transfer learning through subspace distance: This invention constructs a domain-invariant feature extraction module and calculates the subspace distance between the source and target domains through singular value decomposition. By minimizing the subspace distance, the source and target domains can be aligned without changing the feature scale. Aligning the source and target domains through subspace distance achieves efficient transfer learning performance and solves the problem of insufficient accuracy in cross-user vigilance monitoring using transfer learning in existing technologies. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention;

[0023] Figure 2 This is a block diagram illustrating the construction of the cross-user vigilance monitoring model of the present invention;

[0024] Figure 3 This is a block diagram illustrating the feature extraction and fusion structure of the cross-user alertness monitoring model of the present invention.

[0025] Figure 4 This is a block diagram of the cross-user alertness monitoring model of the present invention, which performs transfer learning based on subspace distance. The invention will now be described in detail with reference to the accompanying drawings: Detailed Implementation

[0026] Example 1

[0027] Alertness monitoring holds significant safety importance and has broad market prospects in the field of traffic safety driving. However, current research methods suffer from insufficient stability and accuracy due to inadequate feature extraction, and new users cannot easily and quickly perform alertness monitoring. To address these issues, this invention conducts research and experiments, proposing a cross-user alertness monitoring method based on subspace distance.

[0028] This invention is a cross-user alertness monitoring method based on subspace distance. It extracts deep EEG features through multi-scale spatial adaptive feature modulation and a convolutional neural network module, and extracts deep EEG features through outer product embedding and a convolutional neural network module. The deep EEG and deep EEG features are fused using an explicit spatial mapping module. The fused feature matrix is ​​then used to calculate the subspace distance (RSD) and base mismatch penalty (BMP) through singular value decomposition, and the root mean square error (RMSE) is calculated using a multilayer perceptron. The subspace distance, base mismatch penalty, and RMSE are used as the total loss to train a cross-user alertness monitoring model through backpropagation. See [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart of the present invention, including the following steps:

[0029] (1) Data acquisition and preprocessing: EEG and EOS signal data of users, i.e., traffic drivers, are acquired and preprocessed, including downsampling, artifact removal, time window division, and shallow feature extraction. Differential entropy of 17 channels in the EEG signal with an 8s time window and a 2Hz step size is obtained as shallow EEG features. Blinking, saccade, and fixation time in the EOS signal with an 8s time window are obtained as shallow EOS features. The data is divided into training set and test set, with the training set divided into source domain data and target domain data.

[0030] (2) Constructing a cross-user vigilance monitoring model W: See Figure 2 , Figure 2 This is a block diagram of the cross-user alertness monitoring model constructed in this invention. The invention uses source and target domain data from the training set, extracting deep features from EEG and EEG data through parallel branches. Specifically, EEG data is processed by a deep feature extraction module, which includes multi-scale spatial adaptive feature modulation and a convolutional neural network, to obtain deep EEG features; EEG data is processed by a deep feature extraction module, which includes outer product embedding and a convolutional neural network, to obtain deep EEG features. The two deep features are then fused using a spatial explicit mapping module to obtain source and target domain features. The source and target domain features in the fused features are then compared using a domain-invariant feature extraction module to calculate the subspace distance and base mismatch penalty as L. RSD Loss and L BMP The loss is calculated by comparing the root mean square error (RMSE) of the monitored values ​​obtained from the source domain features through the alertness regression monitoring module based on a multilayer perceptron with the true values. RMSE loss.

[0031] See Figure 3 , Figure 3 This is a block diagram of the feature extraction and fusion structure of the cross-user alertness monitoring model of the present invention. Brain and eye data are processed through their respective deep feature extraction branches to obtain deep features of the brain and eyes, which are then input into the spatial explicit mapping module to achieve feature fusion and obtain the fused features of the source domain and the target domain. Figure 3 The upper part is the deep feature extraction module for EEG data. This module cascades and stacks multi-scale spatial adaptive feature extraction and convolutional neural networks three times. In the multi-scale spatial adaptive feature extraction part, global information is captured by convolving EEG data at different scales, and local information is captured by convolutional neural networks, thus achieving the fusion of global and local information. Figure 3 The lower part is the deep feature extraction module for electrooculogram (EOG) data, which includes outer product embedding and convolutional neural networks. In the outer product embedding, the high-dimensional cross relationships in the EOG data are captured by the outer product of the EOG data, and then the deep feature extraction of the EOG data is achieved by the convolutional neural network. Figure 3 The right side shows the feature fusion section. It obtains the mapping relationship between EEG features and the EEG feature space through concatenation and convolution operations. The EEG features and the mapping relationship are then weighted and summed to map them into the EEG feature space. Finally, concatenation and convolution operations are used to achieve feature fusion within the same feature space. See also... Figure 4 , Figure 4 This is a block diagram of the cross-user alertness monitoring model of the present invention, which performs transfer learning based on subspace distance. Figure 4 The left side shows the characteristics and fusion of EEG and EEG data. Figure 4On the right, the upper part uses a multilayer perceptron for alertness monitoring, while the lower part uses singular value decomposition to calculate the subspace distance and base mismatch penalty between the source and target domains. This subspace distance and base mismatch penalty are used to align the source and target domains, achieving transfer learning. Specifically, the source domain fusion feature is calculated using the alertness regression monitoring module, which uses the root mean square error as L. RMSE The loss, L, is calculated by the subspace distance loss between the source domain fusion feature and the target domain fusion feature through the domain-invariant feature extraction module. RSD and base mismatch penalty as L BMP Loss, L RMSE L RSD L BMP The sum of these is the total loss L total The model is trained by minimizing the total loss using backpropagation until the loss converges, resulting in a well-trained alertness monitoring network model.

[0032] The deep EEG feature extraction branch is sequentially connected to a multi-scale spatial adaptive feature modulation and a convolutional neural network, while the deep OEO feature extraction branch is sequentially connected to an outer product embedding and a convolutional neural network. The explicit spatial mapping module performs feature fusion, mapping deep EOG features to the deep EEG feature space and fusing features through convolution operations to obtain source domain fused features and target domain fused features. In the domain-invariant feature extraction module, the source domain fused features and target domain fused features are used to calculate the orthogonal basis of the source domain subspace and the orthogonal basis of the target domain subspace through singular value decomposition. The subspace distance RSD and the base mismatch penalty BMP are then calculated using these orthogonal basis bases as L... RSD Loss and L BMP Loss. In the alertness regression monitoring module, the source domain fusion features are used to obtain the monitored alertness value through a multilayer perceptron containing two hidden layers. The root mean square error (RMSE) between the monitored value and the actual value is calculated as L. RMSE loss.

[0033] (3) Model training: Using the training set, the subspace distance loss L RSD Base mismatch penalty loss L BMP and root mean square error loss L RMSE The sum of these is used as the total loss L of the model. total Minimize the total loss L using backward iterative propagation total The training continues until the loss converges, resulting in a well-trained cross-user vigilance monitoring model W.

[0034] (4) User vigilance monitoring: Input the test set into the trained cross-user vigilance monitoring model W and output vigilance monitoring values.

[0035] Existing technologies suffer from insufficient extraction of EEG and EEG features, neglect of spatial distribution differences in feature fusion, and insufficient accuracy in alertness regression monitoring due to changes in feature scale in traditional domain adaptation transfer learning methods. To address these issues, this invention aims to achieve feature fusion by fusing data from both EEG and EEG modalities and using a method that aligns the spatial distribution of different modalities. Furthermore, it employs a method that aligns the source and target domain distributions without altering the feature scale, thereby improving the overall accuracy and stability of the alertness monitoring model.

[0036] This invention is a comprehensive technical solution that uses multi-scale spatial adaptive feature modulation and convolutional neural networks to extract deep EEG features, and uses outer product embedding and convolutional neural networks to capture deep EEG features. It achieves feature fusion by explicitly mapping deep EEG and EEG features in space, and aligns the source and target domains by calculating subspace distance without changing the feature scale. This solves the problems of insufficient feature extraction and fusion in traditional methods and poor performance of transfer learning methods, and improves the accuracy and stability of cross-user alertness monitoring models.

[0037] Example 2, the cross-user vigilance monitoring method based on subspace distance is the same as in Example 1. The data acquisition and preprocessing steps of step (1) of this invention include the following steps:

[0038] (1a) Data acquisition: The SEED-VIG dataset containing EEG, EEG signals and alertness labels of vehicle drivers was obtained from a public website. The dataset contains EEG and EEG data of 21 drivers. The EEG signals were collected through electrode channels in the temporal and parietal lobes of the driver's brain, and the EEG data were collected through the EEG channel in the frontal lobe. The electrode channels for acquiring the EEG data can be adjusted. For example, for the convenience of data acquisition, EEG and EEG data can be collected simultaneously in the frontal lobe. The EEG and EEG data are then separated and preprocessed separately.

[0039] (1b) Preprocessing the dataset:

[0040] (1b1) EEG signal preprocessing: EEG data was downsampled to 200Hz to reduce computational complexity. A 1-75Hz bandpass filter was used to eliminate artifacts. Using a 2Hz frequency band and an 8s time window, the differential entropy of 25 frequency bands across 17 channels of the EEG data was calculated as a superficial EEG feature. Studies have shown that differential entropy performs well in alertness monitoring from low to high frequency energies and is considered one of the most effective features for alertness monitoring. While differential entropy can be replaced by the energy spectral entropy of different frequency bands in the EEG signal, this approach suffers from reduced accuracy in the alertness monitoring model.

[0041] (1b2) Preprocessing of electrooculogram (EOG) signals: The EOG data is downsampled to 200Hz and bandpass filtering of 1-75Hz is used to eliminate artifacts. With 8s as the time window, the blink, saccade, and fixation time within 8s are calculated as shallow features of the EOG using the peak detection method. The peak detection algorithm can be used to identify local maximum values ​​or peak points in the signal and is widely used in signal processing and time series analysis.

[0042] (1b3) Obtaining Alertness Labels: The PERCLOS values ​​from the dataset are used as alertness labels. PERCLOS, the eye-closing rate, is one of the most widely recognized and applied indicators of alertness. It refers to the proportion of time the eyes remain closed within a given time unit, and its mean is calculated. Each preprocessed dataset contains EEG, superficial EEG features, and alertness labels. Because PERCLOS is highly objective compared to estimates of a user's subjective alertness, it is used as the user's alertness label in this invention, i.e., the true value of the user's alertness.

[0043] (1c) Dataset partitioning: The preprocessed dataset is divided into a training set and a test set in a 4:1 ratio. In the training set, one subject is selected as the target domain S, and the remaining subjects are selected as the source domain T. Different methods can be used for dataset partitioning, such as a 2:1 or 3:1 ratio. Different experiments may employ different methods for training and test set partitioning.

[0044] This invention obtains and preprocesses electroencephalogram (EEG) and electrooculogram (EOG) data. Differential entropy is used as a superficial feature of the EEG, and fixation, saccade, and blink times obtained from the EOG signal are used as superficial features of the EOG. These features have a high correlation with alertness, which facilitates the further extraction of deeper features by the cross-user alertness monitoring model proposed in this invention. The specific data given in this example can be adjusted and modified according to specific circumstances; for example, the time window size and frequency step size can be adjusted.

[0045] Example 3, the cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-2. The construction of the cross-user vigilance monitoring model W in step (2) is described in [reference needed]. Figure 3 , Figure 4 It includes the following steps:

[0046] (2a) Constructing a deep EEG feature extraction module: Constructing a deep EEG feature extraction module that includes multi-scale spatial adaptive feature modulation and convolutional neural network (CNN). Since the differential entropy of different frequency ranges of EEG signals contains different information about alertness, this invention proposes multi-scale spatial adaptive feature modulation to capture global and multi-scale information of EEG data, thereby enhancing the EEG feature extraction capability.

[0047] (2b) Constructing a deep feature extraction module for electrooculogram (EOG): Constructing a deep feature extraction module for EOG that includes outer product embedding and a convolutional neural network (CNN). Since deep relationships exist in the superficial features of EOG, for example, the total blink time can be calculated by multiplying the average blink time by the number of blinks, this invention uses outer product embedding to capture the multiplicative relationships in the EOG data, thus enriching the expression of EOG features.

[0048] (2c) Constructing a feature fusion module: Constructing a feature fusion module based on spatial display mapping to obtain source domain features and target domain features. Considering that EEG and EEG are data of different modalities, although their features are related to alertness, their feature spatial distribution still has certain differences. Therefore, this invention uses the spatial display mapping method to map EEG features to the EEG feature space to achieve feature fusion, thereby better capturing the complementarity of deep EEG and EEG features and improving the feature extraction and expression capabilities of the model.

[0049] (2d) Constructing a domain-invariant feature learning module: Construct a domain-invariant feature learning module based on subspace distance, and use source domain features and target domain features to calculate the subspace distance RSD and base mismatch penalty BMP through this module;

[0050] (2e) Constructing an alertness regression monitoring module: Construct an alertness regression monitoring module based on a multilayer perceptron, and use source domain features to calculate the root mean square error (RMSE) between the monitored value and the actual value.

[0051] This invention achieves deep feature extraction of EEG and EEG by cascading multi-scale spatial adaptive feature modulation and convolutional neural networks, stacking them three times; it achieves deep feature extraction of EEG by cascading outer product embedding and convolutional neural networks; and it achieves feature fusion by spatial explicit mapping. Through the proposed multi-scale spatial adaptive feature modulation, outer product embedding, and spatial explicit mapping methods, this invention realizes deep feature extraction and fusion of EEG and EEG, giving the model a stronger feature representation capability than existing methods.

[0052] Example 4, the cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-3. The deep EEG feature extraction module described in step (2a) is composed of multi-scale spatial adaptive feature modulation and convolutional neural network (CNN).

[0053] (2a1) Multi-scale spatial adaptive feature modulation: See [link] Figure 3The EEG data was divided into three parallel parts according to channels. One part was downsampled by one-quarter using the nearest neighbor interpolation algorithm, another part was downsampled by half using the leading-edge interpolation algorithm, and the third part remained unchanged. Each of the three parallel parts was passed through a 1x3 convolutional layer, and then upsampled by four times and two times using the nearest neighbor interpolation algorithm to restore the data size. Data that had not been downsampled was not upsampled. Since feature maps of different sizes have different receptive fields, using convolutional kernels of the same size can capture global features of the EEG. The three data parts were concatenated along the channel dimension and then sequentially connected to a 1x1 convolutional layer and a GELU activation function to obtain the output. The output was then dot-producted with the input features and added back to the input features to obtain the output features modulated by multi-scale spatial adaptive feature modulation. Multi-scale spatial adaptive feature modulation captures multi-scale information from the EEG data, facilitating the model's extraction of deep alertness features, enhancing feature representation ability and model robustness.

[0054] (2a2) Constructing a Convolutional Neural Network (CNN): See [link] Figure 3 The constructed convolutional neural network (CNN) consists of a cascaded convolutional layer with a kernel size of 1*3, a batch normalization layer, and a ReLU activation function. The CNN can capture local information from features obtained through multi-scale spatial adaptive feature modulation, enriching the feature representation. The kernel size, stride, and activation function selected in this CNN are optimal choices obtained through experiments.

[0055] (2a3) Constructing a deep EEG feature extraction module: See [link] Figure 3 The EEG feature extraction module is obtained by cascading and stacking multi-scale spatial adaptive feature modulation and convolutional neural network (CNN) three times. The number of stacks depends on the size of the training data. When the amount of training data is large, it is appropriate to stack it further, but underfitting due to too many layers should be avoided.

[0056] This invention combines multi-scale spatial adaptive feature modulation and convolutional neural networks to effectively capture global and local information in EEG data. Multi-scale spatial adaptive feature modulation adjusts the feature responses at different scales, enabling the network model to focus on subtle changes and important patterns in EEG data. Meanwhile, the convolutional neural network fully extracts local information from the EEG data. The combination of the two can effectively capture deep features related to alertness in EEG data, facilitating further feature fusion and improving the overall performance of the model.

[0057] Example 5, the cross-user alertness monitoring method based on subspace distance is the same as in Examples 1-4. The electrooculography feature extraction module described in step (2b) consists of outer product embedding and a convolutional neural network (CNN):

[0058] (2b1) Outer product embedding: see [link] Figure 3 The input features of electrooculography (EOG) are passed through two parallel linear layers. The first linear layer is cascaded with a sigmoid activation function. The two output features are then multiplied together to form the output features of the outer product embedding. The outer product embedding can capture the inherent multiplicative relationships in the EOG data. Capturing the multiplicative relationships between EOG features through the outer product embedding can help with alertness monitoring.

[0059] (2b2) ​​Convolutional Neural Networks (CNN): See [link] Figure 3 The model comprises four convolutional layers with a kernel size of 1*3. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The batch normalization layer accelerates model training and provides regularization to some extent, reducing overfitting. The ReLU activation function enhances the non-linearity of features and improves feature representation. The kernel size parameter performs optimally in this invention and can be adjusted according to different datasets, acquisition devices, and other objective factors.

[0060] (2b3) Constructing the electrooculography feature extraction module: See [link] Figure 3 By cascading the outer product embedding with the convolutional neural network, an electrooculography feature extraction module is obtained.

[0061] This invention utilizes an outer product embedding module and a convolutional neural network to extract deep features related to alertness associated with electrooculography (EOG). Outer product embedding effectively enriches feature representation by increasing feature diversity and complexity. Simultaneously, the convolutional neural network leverages its powerful feature extraction capabilities to extract deep features related to alertness. This feature extraction method not only enhances the expressive power of EOG data but also provides a foundation for subsequent feature fusion, promoting the overall generalization and robustness of the model.

[0062] Example 6: The cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-5. In step (2c), a feature fusion module based on explicit spatial mapping is constructed, as follows:

[0063] (2c1) Feature fusion based on explicit spatial mapping: see Figure 3 The feature fusion part in the module. It will extract the deep brainwave features (F) obtained from the EEG feature extraction module. eeg And the deep electrooculography features obtained from the electrooculography feature extraction module, namely F eog The features are concatenated along the channel dimension, and the concatenated features are used as F. concat , splicing feature Fconcat After passing through a convolutional layer with a kernel size of 1*3, the mapping relationship between EEG and EEG features can be obtained, and the EEG feature F... eog By weighted summation of EEG features, the mapping from EEG features to the EEG feature space is achieved, thus obtaining F'. eog Then F eeg With F' eog Feature F is obtained by concatenating along the channel dimension. Feature F is then passed sequentially through a convolutional layer with a kernel size of 1*3, a batch normalization layer, and a ReLU activation function to obtain the fused features of the source domain data and the target domain data.

[0064] This invention achieves feature fusion through a spatial explicit mapping method, which avoids the problems of poor fusion effect and insufficient feature fusion caused by differences in feature distribution between modalities. The feature fusion method based on spatial explicit mapping can capture the complementary relationship of features in EEG and EEG, which is convenient for subsequent use of EEG and EEG features for further alertness monitoring and transfer.

[0065] Example 7, the cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-6. The step (2d) constructs a domain-invariant feature learning module based on subspace distance, which includes the following steps:

[0066] (2d1) Calculate the orthogonal basis of the source and target domain subspaces: First, obtain the orthogonal basis of the source and target domain subspaces through Singular Value Decomposition (SVD). Singular Value Decomposition is a widely used decomposition method in matrix analysis. It can decompose any matrix into the product of three special matrices, thereby revealing the structural characteristics of the matrix, that is, revealing the spatial distribution characteristics of the characteristic matrix. S is the target domain, and T is the source domain. After the source domain data and target domain data are processed by the above-mentioned EEG and EEG deep feature extraction and feature fusion modules, the fused feature F is obtained. s and F t Using singular value pairs F s and F t The breakdown is as follows:

[0067] F s =U s Σ s (V s ) T (1)

[0068] F t =U t Σ t (V t ) T (2)

[0069] Matrix U s Let U represent the orthogonal basis composition of the subspace of the target domain S, and matrix U. tLet ∑ denote the orthogonal basis composition of the source domain T subspace. s ∑ represents the singular values ​​obtained by performing singular value decomposition on the source domain S. t This represents the singular values ​​obtained by performing singular value decomposition on the target domain T. Performing singular value decomposition on the source and target domains can obtain orthogonal bases for the subspaces of the source and target domains without changing their feature scales. Aligning the source and target domains using orthogonal bases can solve the problem of poor performance of traditional domain-adaptive transfer learning algorithms in regression problems due to changes in feature scales.

[0070] (2d2) Calculate the subspace distance RSD: Using the protagonist Θ as the similarity measure between the orthogonal basis of the source domain subspace and the orthogonal basis of the target domain subspace, the subspace distance RSD between the source domain S subspace and the target domain T subspace is defined as the sum of the sine values ​​of all protagonists, as follows:

[0071]

[0072] in, Represents the subspace distance RSD between the defined source domain S and the target domain T. This indicates that the sine value of the protagonist is calculated using both the source and target subspaces. Let represent the sum of the sine values ​​of all protagonists in the m-dimensional subspace, where m is the dimension of the source subspace.

[0073] The subspace distance RSD is calculated using the following formula:

[0074]

[0075] Among them, (U) s ) T U t Let P represent the product of the orthogonal basis of the target subspace and the orthogonal basis of the source subspace. Singular value decomposition is then performed on this product to obtain the weight matrix P of the source subspace. s The weight matrix P of the target domain subspace t And the cosine values ​​of the main characters distributed on the orthogonal matrix. Using sine and cosine transformations, we can obtain the sine values ​​of all the main characters. Adding the sine values ​​of all the main characters together yields the subspace distance RSD between the source domain S and the target domain T. The subspace distance RSD is then used as L. RSD loss.

[0076] (2d3) Calculate the base mismatch penalty (BMP): The base mismatch penalty is defined as follows:

[0077]

[0078] in, Let P represent the base mismatch penalty (BMP) and the weight matrix P of the source domain subspace. s The weight matrix P of the target domain subspace t The L2 norm of the difference in absolute values. Using BMP as the loss L... BMP Considering the geometric properties between orthogonal bases, constraint through base mismatch penalty can minimize the interference with migration performance caused by mismatches between different orthogonal bases.

[0079] (2d4) Obtain the domain-invariant feature learning module: The subspace distance and base mismatch penalty are calculated by the source domain and the target domain to obtain the domain-invariant feature learning module.

[0080] This invention utilizes singular value decomposition to calculate the orthogonal basis of the source and target domain subspaces without altering the feature scale, and to calculate the subspace distance between the target and source domains. By minimizing the subspace distance, the distributions of the source and target domains can be aligned, and domain-invariant features can be extracted for migration from the source to the target domain. Furthermore, considering the orthogonal basis order of the source and target domains to maintain the geometric features of their depth representation, a base mismatch penalty is proposed to preserve this property. Through these steps, this invention can complete the migration from the source to the target domain without changing the feature scale, solving the problem of poor cross-user alertness monitoring performance caused by changing the feature scale in traditional domain adaptation methods, and improving the accuracy and efficiency of alertness monitoring.

[0081] Example 8: The cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-7. The vigilance regression monitoring module constructed in step (2e) has the following structure:

[0082] (2e1) Construction of the regression monitoring module: This module contains a first hidden layer of size 128*80, a second hidden layer of size 80*1, and a Tanh activation function layer. The source domain fusion features are input into this module, and the root mean square error (RMSE) between the output and the true label values ​​is calculated as the loss L. RMSE The calculation formula is as follows:

[0083]

[0084] n represents the number of samples, y i Let y represent the actual value of the i-th sample. i ' represents the monitoring value of the i-th sample obtained by the vigilance regression monitoring module, and the sum of the squares of the differences between the actual values ​​and the monitoring values ​​of the n samples is used as the loss L. RMSE The alertness regression monitoring module was obtained.

[0085] The above-mentioned hierarchical feature representation is achieved through a multilayer perceptron. The two hidden layers can balance computational and expressive power and avoid the risk of overfitting. The Tanh activation function can capture complex nonlinear relationships, thereby refining the alertness monitoring results and making the alertness monitoring values ​​more accurate.

[0086] Example 9: The cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-8. The vigilance monitoring model W described in step (3) is trained, and the training method includes:

[0087] (3a) Training the model using the total loss: using the loss L obtained from the domain-invariant feature learning module. RSD L BMP L obtained from the alertness regression monitoring module RMSE As the total loss L total :

[0088] L total =L RSD +L BMP +L RMSE (8)

[0089] Using the training set, minimize the total loss L through backpropagation. total We continuously adjust the model parameters to optimize performance.

[0090] During training, the gradient of the loss is calculated to guide parameter updates, ensuring that the model gradually approaches the global optimum. This process continues until the model converges, ultimately yielding a well-trained cross-user alertness monitoring model W. This model can achieve cross-user alertness monitoring with high accuracy and stability.

[0091] The technical advantages of this invention are verified and illustrated below through simulation and data:

[0092] Example 10: The cross-user vigilance monitoring method based on subspace distance is the same as in Examples 1-9.

[0093] I. Simulation conditions:

[0094] The hardware test platform for the simulation experiment is: CPU is Intel(R) Core(TM) i7-8700 with a main frequency of 3.2GHz, memory is 16GB, and GPU is NVIDIA GeForce GT 710.

[0095] The software testing platform for the simulation experiment is: Windows 11 operating system and Python 3.7.

[0096] The dataset used in the simulation experiment: SEED-VIG is a publicly available dataset containing simulated vehicle driving data from 21 subjects, including EEG, EEG data, and PERCLOS eyelid closure time as the true value of alertness.

[0097] II. Simulation Content:

[0098] The simulation uses three existing comparison methods, the sources of which are:

[0099] H.Li et al. proposed a method for cross-user vigilance monitoring in their published paper "Multimodal vigilance estimation with adversarial domain adaptation networks".

[0100] In their paper "Capsule attention for multimodal EEG and EOG spatiotemporal representation learning with application to drivervigilance estimation", GYzhang et al.

[0101] This paper proposes a method for cross-user vigilance monitoring.

[0102] Meihong Zhang et al. proposed a method for cross-user vigilance monitoring in their paper "Multimodal Vigilance Estimation with Modality-Pairwise Contrastive Loss".

[0103] Table 1 shows the average correlation coefficient and average root mean square error for different methods.

[0104] Average correlation coefficient Mean root mean square error The method of the present invention 0.858±0.053 0.139±0.065 Existing Method 1 0.811±0.109 0.176±0.109 Existing Method 2 0.824±0.081 0.185±0.095 Existing Method 3 0.844±0.085 0.162±0.098

[0105] Using the present invention and the three existing methods described above, the alertness of users in the SEED-VIG dataset is monitored, and the average correlation coefficient and average root mean square error between the monitored alertness values ​​and the actual values ​​are calculated. Table 1 shows the average correlation coefficient and average root mean square error between the present invention and the three different methods.

[0106] As shown in Table 1, the average correlation coefficient between the cross-user alertness monitoring values ​​and the true values ​​of this invention is higher than that of the three existing methods. Specifically, this invention improves the correlation by 0.047 compared to existing method 1, 0.034 compared to existing method 2, and 0.114 compared to existing method 3, significantly improving the correlation between the alertness monitoring values ​​and the true values. The root mean square error (RMSE) between the alertness monitoring values ​​and the true values ​​of this invention is lower than that of existing methods. Specifically, it reduces the error by 0.037 compared to existing method 1, 0.046 compared to existing method 2, and 0.023 compared to existing method 3, significantly improving the accuracy of cross-user alertness monitoring and solving the problem of insufficient accuracy in cross-user alertness monitoring in existing technologies. Furthermore, the transfer learning method proposed in this invention can be used without collecting labeled data from new users, making it easier for new users to use the alertness monitoring model. In addition, the high efficiency of the feature extraction and fusion module proposed in this invention has also been demonstrated through comparative experiments. This improves the reliability of accurate alertness measurement during user traffic driving, thereby effectively avoiding major safety accidents caused by insufficient alertness in vehicle driving, aviation, or engineering operations.

[0107] Simulation results and data show that the performance of this invention in cross-user alertness monitoring is significantly better than existing methods. The alertness monitoring results are more accurate and realistic, and it can be applied to relevant research on alertness monitoring in traffic driving. It allows new users to use the alertness monitoring model without collecting labeled data. Furthermore, the model's feature representation ability is improved by using data from two modalities and new feature extraction and fusion methods. The transfer learning method based on subspace distance also improves the transfer performance compared with traditional domain adaptation transfer learning methods without changing the feature scale. Finally, the accuracy and stability of the obtained alertness monitoring model are significantly improved.

[0108] In summary, the subspace distance-based cross-user alertness monitoring method of this invention solves the problems of single modal data, insufficient correlation, and large error fluctuations in existing cross-user alertness monitoring technologies. The implementation scheme is as follows: EEG and EEG signals of users are collected and preprocessed; EEG and EEG data are input into a deep feature extraction module, and deep feature fusion of EEG and EEG is achieved using explicit spatial mapping; the subspace distance between the source domain feature space and the target domain feature space, as well as the prediction loss of the source domain, are calculated; the model is trained by minimizing the loss using backpropagation; and alertness is detected using the trained model. This invention, by using multimodal data fusion and a subspace distance-based transfer learning algorithm, significantly improves the linear correlation between monitored and true values ​​during cross-user alertness monitoring and significantly reduces monitoring errors. It can be used for real-time continuous monitoring of user alertness in research fields such as human-computer interaction, traffic driving, and aviation flight.

[0109] It should be noted that the step numbers in the specification and claims of this invention are only for the purpose of clearly describing the embodiments of this invention and facilitating understanding, and their order is not limited.

Claims

1. A method for monitoring cross-user vigilance based on subspace distance, characterized in that, Deep EEG features are extracted using multi-scale spatial adaptive feature modulation and a convolutional neural network module. Deep EEG features are extracted using an outer product embedding and a convolutional neural network module. The deep EEG and deep EEG features are then fused using an explicit spatial mapping module. The fused feature matrix is ​​then used to calculate the subspatial distance (RSD) and base mismatch penalty (BMP) using singular value decomposition, and the root mean square error (RMSE) is calculated using a multilayer perceptron. The subspatial distance, base mismatch penalty, and RMSE are used as the total loss to train a cross-user vigilance monitoring model via backpropagation. The process includes the following steps: (1) Data acquisition and preprocessing: EEG and electrooculogram (EOG) signal data of users, i.e., traffic drivers, are acquired and preprocessed, including downsampling, artifact removal, time window division, and shallow feature extraction. Differential entropy of 17 channels in the EEG signal with an 8s time window and a step size of 2Hz is obtained as shallow EEG features. Blinking, saccade, and fixation time in the EOG signal with an 8s time window are obtained as shallow EOG features. The data is divided into training set and test set, with the training set divided into source domain data and target domain data. (2) Constructing a cross-user vigilance monitoring model W: Using source domain data and target domain data in the training set, deep features are extracted from the EEG data and EEG data through parallel branches. The EEG data is extracted through the EEG deep feature extraction module, which includes multi-scale spatial adaptive feature modulation and convolutional neural network (CNN), as follows: (2a1) Multi-scale spatial adaptive feature modulation: The EEG data is divided into three parallel parts according to the channels. One part is downsampled by one-quarter using the nearest neighbor interpolation algorithm, another part is downsampled by one-half using the leading interpolation algorithm, and the third part remains unchanged. The three parallel parts are passed through a convolutional layer with a kernel size of 1*3, and then upsampled by four times and two times using the nearest neighbor interpolation algorithm to restore the data size. Data that has not been downsampled is not upsampled. The three parts of data are concatenated along the channel dimension and then connected to a convolutional layer with a kernel size of 1*1 and the GELU activation function in sequence to obtain the output. The output is then dot-producted with the input features and then added to the input features to obtain the output features after multi-scale spatial adaptive feature modulation. (2a2) Constructing a Convolutional Neural Network (CNN): The constructed CNN consists of a cascaded convolutional layer with a kernel size of 1*3, a batch normalization layer, and a ReLU activation function; (2a3) Constructing a deep EEG feature extraction module: Multi-scale spatial adaptive feature modulation and convolutional neural network (CNN) are cascaded and stacked three times to obtain the EEG feature extraction module; deep EEG features are obtained; Electroocular data is processed through an electroocular deep feature extraction module to obtain deep electroocular features; The electrooculogram (EOG) feature extraction module is constructed, including outer product embedding and a convolutional neural network (CNN), as detailed below: (2b1) External product embedding: The electrooculogram input features are passed through two parallel linear layers, the first of which is cascaded with a sigmoid activation function. The two output features are then externally producted as the output features of the external product embedding. (2b2) ​​Convolutional Neural Network (CNN): It contains four convolutional layers with a kernel size of 1*3. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. (2b3) Constructing the electrooculogram feature extraction module: The outer product embedding is cascaded with the convolutional neural network to obtain the electrooculogram feature extraction module; The feature fusion module based on explicit spatial mapping is constructed as follows: (2c1) Feature fusion based on explicit spatial mapping: deep EEG features obtained from the EEG feature extraction module and deep electrooculogram features obtained from the electrooculogram feature extraction module The concatenation is performed along the channel dimension to obtain the concatenated features. splicing features After passing through a convolutional layer with a kernel size of 1*3, and combining it with deep electrooculography features... We get the weighted sum. Then and Features are obtained by concatenating along the channel dimension. , will feature The source and target domain data are obtained by sequentially passing through a convolutional layer with a kernel size of 1*3, a batch normalization layer, and a ReLU activation function; Two deep features are fused using a spatial explicit mapping module to obtain source domain features and target domain features. The source domain features and target domain features from the fused features are then used to calculate the subspace distance RSD and base mismatch penalty BMP using a domain-invariant feature extraction module as L. RSD Loss and L BMP The loss is calculated by comparing the root mean square error (RMSE) between the monitored values ​​obtained from the source domain features through the alertness regression monitoring module based on a multilayer perceptron and the true values. RMSE loss; The deep feature extraction branch of EEG is sequentially connected to a multi-scale spatial adaptive feature modulation and a convolutional neural network, while the deep feature extraction branch of EOS is sequentially connected to an outer product embedding and a convolutional neural network. The spatial explicit mapping module realizes feature fusion, maps the deep features of EOG to the deep feature space of EEG, and performs feature fusion through convolution operation to obtain source domain fused features and target domain fused features; In the domain-invariant feature extraction module, the source domain fusion feature and the target domain fusion feature are calculated by singular value decomposition to obtain the source domain subspace orthogonal basis and the target domain subspace orthogonal basis. The subspace distance RSD and the base mismatch penalty BMP are calculated using the source domain subspace orthogonal basis and the target domain subspace orthogonal basis. In the alertness regression monitoring module, the source domain fusion features are used to obtain the monitored alertness value through a multilayer perceptron containing two hidden layers, and the root mean square error (RMSE) between the monitored value and the actual value is calculated. (3) Training of the alertness monitoring model W: Using the training set, the subspace distance loss L RSD Base mismatch penalty loss L BMP and root mean square error loss L RMSE The sum of these is used as the total loss L of the model. total Minimize the total loss L using backward iterative propagation total , until the loss converges, and the trained cross-user vigilance monitoring model W is obtained; (4) User vigilance monitoring: Input the test set into the trained cross-user vigilance monitoring model W and output the vigilance monitoring value.

2. The cross-user vigilance monitoring method based on subspace distance according to claim 1, characterized in that, Step (1), data acquisition and preprocessing, includes the following steps: (1a) Data Acquisition: The SEED-VIG dataset containing EEG, EOG signals, and alertness labels of vehicle drivers was obtained from publicly available websites; (1b) Preprocess the dataset: (1b1) Preprocessing of EEG signals: The EEG data was downsampled to 200Hz to reduce computational complexity. Bandpass filtering of 1-75Hz was used to eliminate artifacts. With 2Hz as the frequency band and 8s as the time window, the differential entropy of 25 frequency bands on 17 channels of EEG data was calculated as the shallow features of EEG. (1b2) Preprocessing of electrooculogram (EOG) signals: The EOG data was downsampled to 200Hz, and bandpass filtering of 1-75Hz was used to eliminate artifacts. With 8s as the time window, the peak detection method was used to calculate the blink, saccade, and fixation time within 8s as the superficial features of the EOG. (1b3) Obtaining alertness labels: The PERCLOS values ​​in the dataset are used as alertness labels. Each preprocessed dataset contains superficial features of EEG and EEG, as well as alertness labels. (1c) Dataset partitioning: The preprocessed dataset is divided into a training set and a test set in a ratio of 4:

1. In the training set, one subject is selected as the target domain S, and the remaining subjects are selected as the source domain T.

3. The cross-user vigilance monitoring method based on subspace distance according to claim 1, characterized in that, The construction of the cross-user vigilance monitoring model W in step (2) includes the following steps: (2a) Constructing a deep EEG feature extraction module: Constructing a deep EEG feature extraction module that includes multi-scale spatial adaptive feature modulation and a convolutional neural network (CNN); (2b) Constructing a deep feature extraction module for electrooculography: Constructing a deep feature extraction module for electrooculography that includes outer product embedding and a convolutional neural network (CNN); (2c) Constructing a feature fusion module: Constructing a feature fusion module based on spatial display mapping to obtain source domain features and target domain features; (2d) Constructing a domain-invariant feature learning module: Construct a domain-invariant feature learning module based on subspace distance, and use source domain features and target domain features to calculate subspace distance RSD and base mismatch penalty BMP through this module; (2e) Constructing an alertness regression monitoring module: Construct an alertness regression monitoring module based on a multilayer perceptron, and use source domain features to calculate the root mean square error (RMSE) between the monitored value and the actual value.

4. The cross-user vigilance monitoring method based on subspace distance according to claim 3, characterized in that, The construction domain-invariant feature learning module described in step (2d) includes the following steps: (2d1) Calculate the orthogonal basis of the source and target domain subspaces: After the target domain S and source domain T are processed by the above-mentioned EEG and EEG deep feature extraction and feature fusion modules, the fused features are as follows: and ,Will and As input to the domain-invariant feature learning module, an orthogonal basis for the source and target domain subspaces is first obtained through singular value decomposition (SVD): , , In equations (1) and (2), the matrix Composed of orthogonal bases of the target domain S subspace, the matrix It consists of orthogonal bases of the source domain T subspace. This represents the singular values ​​obtained by performing singular value decomposition on the source domain S. This represents the singular values ​​obtained by singular value decomposition of the target domain T; (2d2) Calculate subspace distance RSD: using the protagonist As a similarity measure between orthogonal bases of the source domain subspace and the target domain subspace, the subspace distance RSD between the source domain S subspace and the target domain T subspace is defined as the sum of the sine values ​​of all protagonists, as follows: , in, Represents the subspace distance RSD between the defined source domain S and the target domain T. This indicates that the sine value of the protagonist is calculated using both the source and target subspaces. This represents the sum of the sine values ​​of all the main characters in the m-dimensional subspace; The subspace distance RSD is calculated using the following formula: , , in, Let represent the product of the orthogonal basis of the target subspace and the orthogonal basis of the source subspace. Singular value decomposition is then performed on this product to obtain the weight matrix of the source subspace. Weight matrix of the target domain subspace And the cosine values ​​of the main characters distributed on the orthogonal matrix. Using sine and cosine transformations, we can obtain the sine values ​​of all the main characters. Adding the sine values ​​of all the main characters together yields the subspace distance RSD between the source domain S and the target domain T, which is used as the loss L. RSD ; (2d3) Calculate the base mismatch penalty (BMP): The base mismatch penalty is defined as follows: , in, Let S be the base mismatch penalty between the source domain S and the target domain T, and let T be the weight matrix of the source domain subspace. Weight matrix of the target domain subspace The L2 norm of the absolute difference between the base mismatches is used as the loss L. BMP ; (2d4) Obtain the domain-invariant feature learning module: The subspace distance and base mismatch penalty are calculated by the source domain and the target domain to obtain the domain-invariant feature learning module.

5. The cross-user vigilance monitoring method based on subspace distance according to claim 3, characterized in that, The alarm regression monitoring module described in step (2e) has the following structure: (2e1) Constructing a regression monitoring module: This module contains a first hidden layer of size 128*80, a second hidden layer of size 80*1, and a Tanh activation function layer; the source domain fusion features are input into this module, and the root mean square error between the output and the true label value is calculated as the loss L. RMSE The calculation formula is as follows: , Where n represents the number of samples. This represents the actual value of the i-th sample. L represents the monitoring value of the i-th sample obtained by the vigilance regression monitoring module. The sum of the squares of the differences between the actual values ​​and the monitoring values ​​of the n samples is used as the loss L. RMSE The alertness regression monitoring module was obtained.

6. The cross-user vigilance monitoring method based on subspace distance according to claim 1, characterized in that, The training of the alertness monitoring model W in step (3) includes the following training methods: Training the model using total loss: L obtained using the domain-invariant feature learning module. RSD L BMP L obtained from the alertness regression monitoring module RMSE As the total loss L total : , Using the training set, minimize the total loss L through backpropagation. total Train the model until it converges to obtain a trained cross-user alertness monitoring model W.

Citation Information

Patent Citations

  • Multi-source domain adaptation-based electroencephalogram signal cross-user alertness monitoring method

    CN115470830A

  • A method for cross-user vigilance monitoring of EEG signals based on multi-source domain adaptation

    CN115470830B

  • System and method for driver alertness intelligent monitoring based on fusion of eye movement and brain waves

    CN103340637A

  • Method for realizing a multi-channel convolutional recurrent neural network EEG emotion recognition model using transfer learning

    US20230039900A1