Method and system for vigilance detection based on uncertainty measure and cross-modal knowledge distillation

Abstract

This invention discloses an alertness detection method and system based on uncertainty measurement and cross-modal knowledge distillation, belonging to the fields of physiological signal processing and deep learning. Specifically, the method first preprocesses and extracts features from raw EEG and EOG signals to obtain structured EEG and EOG feature matrices, and generates alertness state labels based on the subject's behavioral reaction time index. Then, a teacher model is constructed and trained using EEG-EOG bimodal data as input; simultaneously, a student model is constructed using EOG unimodal data as input. Finally, using the trained teacher model as the knowledge source, the student model undergoes uncertainty-guided cross-modal knowledge distillation training. This method can effectively improve the alertness detection performance of the EOG unimodal student model under limited sample conditions.

Description

Technical Field

[0001] This invention relates to the fields of physiological signal processing, deep learning, and knowledge distillation, specifically to an alertness detection method and system based on uncertainty measurement and cross-modal knowledge distillation. Background Technology

[0002] Vigilance refers to an individual's ability to maintain sustained attention on a specific task or stimulus over a prolonged period, and is an important component of human cognitive function. In safety-critical scenarios such as driving, aviation, rail transportation, and industrial production, a decline in operator vigilance often signifies slower reaction times, weakened judgment, and an increased risk of operational errors, potentially leading to serious safety accidents. Therefore, accurate and real-time detection of vigilance status has significant theoretical and practical value.

[0003] Among physiological signals, electroencephalography (EEG) and electrooculography (EOG) are the two most commonly used signals for alertness detection. EEG signals directly reflect the neural electrical activity of the cerebral cortex and are highly sensitive to cognitive changes related to alertness. EOG signals record potential changes generated by eye movements, containing rich information on blinking and saccades, and are closely related to an individual's alertness state. The two are naturally complementary in terms of information dimensions; therefore, methods based on EEG-EOG multimodal fusion have become an important approach to improve the performance of alertness detection.

[0004] However, in practical applications, EEG signals require multi-channel electrode caps for acquisition, which are complex to wear and have poor stability, making them unsuitable for long-term deployment. In contrast, EOG signals can be acquired with only a few periocular electrodes, making them more suitable for real-world applications. Therefore, how to maintain high detection performance while using only EOG single-modal signals has become an important issue in current alertness detection research.

[0005] Knowledge distillation offers a feasible solution to the aforementioned problems. Its core idea is to transfer knowledge from complex teacher models to lightweight student models, enabling student models to maintain high performance even with fewer parameters or simpler input modalities. However, most existing cross-modal distillation methods employ deterministic feature or output alignment strategies, neglecting the uncertainties in teacher model predictions. In scenarios where physiological signals exhibit significant non-stationarity and individual differences, the confidence levels of teacher model predictions for different samples fluctuate significantly. Indiscriminate distillation may introduce noisy knowledge and weaken the learning effectiveness of student models.

[0006] Furthermore, in practical applications, labeled data is usually very limited, and there is still a lack of systematic research on how to screen high-quality distilled samples and achieve effective cross-modal knowledge transfer under conditions of few samples.

[0007] Therefore, the existing technology has the following shortcomings: (1) cross-modal distillation ignores the uncertainty of teacher model prediction, resulting in low-quality knowledge contaminating student training; (2) feature alignment lacks constraints at the probability distribution level and cannot effectively convey the structural information of the teacher feature space; (3) the high-quality knowledge distillation mechanism in the few-sample scenario is not yet perfect. Summary of the Invention

[0008] To address the problems existing in current technologies, this invention proposes an alertness detection method, system, and medium based on uncertainty measurement and cross-modal knowledge distillation. This invention constructs an EEG-EOG bimodal teacher model and transfers its knowledge to a student model using only EOG signals, thereby reducing system acquisition complexity while maintaining high alertness detection performance.

[0009] To achieve the objectives of this invention, the technical solution provided by this invention is as follows: First aspect This invention provides an alertness detection method based on uncertainty measurement and cross-modal knowledge distillation, comprising the following steps: Step S1: Collect raw EEG and EOG signals from the subjects during the sustained attention task; Step S2: Preprocess and extract features from the acquired raw EEG and EOG signals to obtain structured EEG feature matrices and EOG feature matrices, and assign an alertness status label to each sample based on the subject's behavioral reaction time index. Step S3: Using the EEG feature matrix and EOG feature matrix as input, construct and train a bimodal teacher model; Step S4: Construct an EOG single-modal student model using the EOG feature matrix as input; Step S5: Using the trained teacher model as the knowledge source, perform cross-modal knowledge distillation training on the student model based on uncertainty guidance; Specifically, the trained teacher model is used to infer for each sample in the few-sample support set, and its predicted probability distribution is obtained as a soft label. The confidence index of the sample is calculated based on the output probability distribution of the teacher model. The distilled samples are screened according to a preset confidence threshold, and only samples with confidence higher than the threshold are retained to participate in knowledge distillation training, so as to reduce the interference of unreliable knowledge contained in low-confidence samples on the training of student models. For the screened samples, the output distribution of the student model is constrained to move closer to the predicted distribution of the teacher model through soft label distillation loss, so that the student model learns the inter-class relationship information contained in the output probability distribution of the teacher model.

[0010] Step S6: Deploy the trained student model and perform alertness state detection inference based on the EOG single-modal input.

[0011] Furthermore, in step S2, the preprocessing and feature extraction of the EEG signal specifically includes: The original EEG signal is sequentially processed by bandpass filtering, downsampling, line noise suppression, average rereference, and artifact removal to obtain a high-quality EEG signal. The preprocessed EEG signal is segmented using a sliding time window, and spatial features are extracted in each frequency band. The features of multiple time segments are averaged to form a structured EEG feature matrix. The preprocessing and feature extraction of EOG signals are as follows: The original EOG signal is processed sequentially by differential calculation, bandpass filtering, downsampling and line noise suppression to obtain a high-quality eye movement signal; blinking events are detected within the time window, and time, amplitude and velocity features reflecting blinking behavior are extracted. The statistical values ​​of each feature are summarized to form a structured EOG feature matrix. The generation of alertness status labels is specifically as follows: The subjects' behavioral reaction time was standardized within the subjects to eliminate the influence of individual differences. A dual-threshold discrimination mechanism was used to divide the standardized behavioral performance indicators into alertness and fatigue states. Samples in the transition interval between the two thresholds were removed, and the two types of samples were balanced to obtain a balanced alertness state label.

[0012] Furthermore, in step S3, the bimodal teacher model includes the following: an input layer, a sequence embedding layer, a Transformer self-attention encoder and a cross-modal fusion encoder, a neural adaptive uncertainty estimation module, a projection head, and a classification head; The input layer is used to input the EEG feature matrix and the EOG feature matrix; The sequence embedding layer includes an EEG encoding branch and an EOG encoding branch, wherein the EEG encoding branch and the EOG encoding branch respectively perform sequence encoding on their respective input feature matrices; The Transformer self-attention encoder is used to extract deep features from EEG and EOG sequences, respectively. The cross-modal fusion encoder is used to fuse bimodal information through a cross-modal attention mechanism, using EEG as the query and EOG as the key and value, to obtain a multimodal fusion representation; The neural adaptive uncertainty estimation module is used to model the encoded representation of each modality as a probability distribution, output the distribution parameters, and obtain a randomized representation through reparameterization sampling. Specifically, for the encoded feature representation, the mean parameter and variance parameter of the feature distribution are output through the mean mapping layer and the variance mapping layer, respectively, to model the feature representation as a Gaussian probability distribution. The distribution is sampled using the reparameterization technique to obtain a randomized feature representation that can participate in gradient backpropagation, so that the uncertainty parameters can be effectively optimized in end-to-end training.

[0013] The projection head is used to map the EEG and EOG representations and their randomized sampling results to a low-dimensional common feature space through a nonlinear projection layer, which is used for feature-level alignment during the distillation stage. The classification head is used to take the CLS token, which is a multimodal representation after cross-modal fusion, as input, and output binary classification prediction probability through Dropout and a linear layer. During the training phase, it is optimized in a supervised manner using cross-entropy loss, and during the inference phase, it outputs the final alertness state judgment result.

[0014] Furthermore, in step S4, the student model includes the following: an input layer, an encoding module, an uncertainty estimation module, and a classification module; The input layer is used to input the EOG feature matrix; The encoding module includes a sequence embedding layer and a Transformer encoder, which are used to perform sequence encoding on the input EOG feature matrix and extract the single-modal context representation; The uncertainty estimation module is used to model the encoded representation as a probability distribution and sample it; The classification module includes a feature adapter and a classification head; the feature adapter is used to perform a nonlinear transformation on the encoded representation and map it to a common feature space shared with the teacher model to support cross-modal feature alignment during the distillation process; the classification head is used to output the predicted probability of the alertness state based on the encoded representation.

[0015] Furthermore, in step S5, the total training loss of the student model It consists of multiple losses: ; in, Cross-entropy classification loss, To screen for soft-label distillation loss with confidence levels, Feature-level alignment loss; For the uncertainty distribution alignment loss, , , The balancing hyperparameters for each loss term; in the few-shot training setting, k samples are sampled from each class, k∈{5,10,20,50} to form the support set, and 5-fold cross-validation is used for evaluation; Among them, confidence level screening for soft-label distillation loss The calculation formula is as follows: ; in, S This represents the set of samples with a confidence level higher than a set threshold, where |S| is the size of the set. The confidence level being higher than the set threshold means that a threshold is set based on the confidence level predicted by the teacher model, and only samples with a confidence level higher than the threshold are selected to participate in distillation training. T This is a temperature coefficient used to determine the probability distribution of softening and to amplify the relative relationships between classes. The gradient scaling factor is introduced to compensate for temperature scaling; σ represents the Softmax function; and These represent the student model and the teacher model respectively for the first... i The output logits of each sample; KL () represents the KL divergence between two distributions; Among them, feature-level alignment loss The calculation formula is as follows: ; in, The reference feature vector is obtained by mapping the CLS token of the teacher model through the projection head; The feature vector obtained by mapping the student model's CLS token through a feature adapter; The L2 norm is represented. The specific implementation of feature-level alignment is as follows: the feature representation of the EOG encoding branch of the teacher model is mapped to the common feature space through the projection head to obtain the teacher reference feature; the encoded feature representation of the student model is mapped to the same common feature space through its feature adaptation module to obtain the student feature; with the teacher reference feature as the supervision target, the distance loss between teacher and student features is calculated to guide the feature representation of the student model to move closer to the feature space of the teacher model.

[0016] Among them, uncertainty distribution alignment loss The calculation formula is as follows: ; in, L Indicates the sequence length; lFor the token's location index; and The uncertainty estimation module of the student model is in the first... l The mean and variance of the output for each token position; and The teacher model uncertainty module is in the first... l The mean and variance of the output for each token position; N ( μ,σ 2 ) indicates that the mean is μ variance is σ 2 The Gaussian distribution is used. The distribution alignment for uncertainty perception specifically involves treating the distribution parameters output by the teacher model's uncertainty estimation module and the student model's uncertainty estimation module as two separate probability distributions. The KL divergence of the student distribution relative to the teacher distribution is calculated as the distribution alignment loss, forcing the student model to approximate the teacher model in terms of probability distribution shape in the feature space, thereby conveying the uncertainty structure information inherent in the teacher's feature representation. The total training loss of the student model is a weighted combination of classification loss, soft label distillation loss, feature alignment loss, and distribution alignment loss.

[0017] In step S5, the specific setup for few-shot distillation training is as follows: construct a few-shot support set with k samples for each class; use the trained teacher model to infer the support set samples and obtain soft labels and uncertainty distribution parameters as distillation supervision signals; the student model is jointly optimized under few-shot conditions using the above-mentioned multiple distillation losses, and after training, it can independently complete alertness detection inference by relying only on EOG single-modal input without the participation of the teacher model.

[0018] Second aspect Corresponding to the above method, the present invention provides an alertness detection system based on uncertainty measurement and cross-modal knowledge distillation, comprising the following units: a data acquisition unit, a preprocessing and extraction unit, a bimodal teacher model construction unit and a student model construction unit, a cross-modal knowledge distillation training unit, and an inference unit; The acquisition unit is used to acquire the raw EEG and EOG signals of the subject during the sustained attention task. The preprocessing and extraction unit is used to preprocess and extract features from the acquired raw EEG and EOG signals respectively, to obtain structured EEG feature matrices and EOG feature matrices, and to assign an alertness status label to each sample based on the subject's behavioral reaction time index. The bimodal teacher model construction unit is used to construct and train a bimodal teacher model using the EEG feature matrix and the EOG feature matrix as input. The student model building unit is used to construct an EOG single-modal student model using the EOG feature matrix as input. The cross-modal knowledge distillation training unit is used to perform uncertainty-guided cross-modal knowledge distillation training on the student model using the trained teacher model as the knowledge source. The inference unit is used to deploy the trained student model and perform alertness state detection inference based on the EOG single-modal input.

[0019] Furthermore, the preprocessing and feature extraction of EEG signals specifically involve: The original EEG signal is sequentially processed by bandpass filtering, downsampling, line noise suppression, average rereference, and artifact removal to obtain a high-quality EEG signal. The preprocessed EEG signal is segmented using a sliding time window, and spatial features are extracted in each frequency band. The features of multiple time segments are averaged to form a structured EEG feature matrix. The preprocessing and feature extraction of EOG signals are as follows: The original EOG signal is processed sequentially by differential calculation, bandpass filtering, downsampling and line noise suppression to obtain a high-quality eye movement signal; blinking events are detected within the time window, and time, amplitude and velocity features reflecting blinking behavior are extracted. The statistical values ​​of each feature are summarized to form a structured EOG feature matrix. The generation of alertness status labels is specifically as follows: The subjects' behavioral reaction time was standardized within the subjects to eliminate the influence of individual differences. A dual-threshold discrimination mechanism was used to divide the standardized behavioral performance indicators into alertness and fatigue states. Samples in the transition interval between the two thresholds were removed, and the two types of samples were balanced to obtain a balanced alertness state label.

[0020] Furthermore, the bimodal teacher model includes the following: an input layer, a sequence embedding layer, a Transformer self-attention encoder and a cross-modal fusion encoder, a neural adaptive uncertainty estimation module, a projection head, and a classification head; The input layer is used to input the EEG feature matrix and the EOG feature matrix; The sequence embedding layer includes an EEG encoding branch and an EOG encoding branch, wherein the EEG encoding branch and the EOG encoding branch respectively perform sequence encoding on their respective input feature matrices; The Transformer self-attention encoder is used to extract deep features from EEG and EOG sequences, respectively. The cross-modal fusion encoder is used to fuse bimodal information through a cross-modal attention mechanism, using EEG as the query and EOG as the key and value, to obtain a multimodal fusion representation; The neural adaptive uncertainty estimation module is used to model each modality encoding representation as a probability distribution, output distribution parameters, and obtain a randomized representation through reparameterization sampling; The projection head is used to map the EEG and EOG representations and their randomized sampling results to a low-dimensional common feature space through a nonlinear projection layer, which is used for feature-level alignment during the distillation stage. The classification head is used to take the CLS token, which is a multimodal representation after cross-modal fusion, as input, and output binary classification prediction probability through Dropout and a linear layer. During the training phase, it is optimized in a supervised manner using cross-entropy loss, and during the inference phase, it outputs the final alertness state judgment result.

[0021] Furthermore, the student model includes the following components: an input layer, an encoding module, an uncertainty estimation module, and a classification module. The input layer is used to input the EOG feature matrix; The encoding module includes a sequence embedding layer and a Transformer encoder, which are used to perform sequence encoding on the input EOG feature matrix and extract the single-modal context representation; The uncertainty estimation module is used to model the encoded representation as a probability distribution and sample it; The classification module includes a feature adapter and a classification head; the feature adapter is used to perform a nonlinear transformation on the encoded representation and map it to a common feature space shared with the teacher model to support cross-modal feature alignment during the distillation process; the classification head is used to output the predicted probability of the alertness state based on the encoded representation.

[0022] Furthermore, when the cross-modal knowledge distillation training unit is executed, the total training loss of the student model is... It consists of multiple losses: ; in, Cross-entropy classification loss, To screen for soft-label distillation loss with confidence levels, Feature-level alignment loss; For the uncertainty distribution alignment loss, , , The balancing hyperparameters for each loss term; in the few-shot training setting, k samples are sampled from each class, k∈{5,10,20,50} to form the support set, and 5-fold cross-validation is used for evaluation; Among them, confidence level screening for soft-label distillation loss The calculation formula is as follows: ; in,S This represents the set of samples with a confidence level higher than a set threshold, where |S| is the size of the set. The confidence level being higher than the set threshold means that a threshold is set based on the confidence level predicted by the teacher model, and only samples with a confidence level higher than the threshold are selected to participate in distillation training. T This is a temperature coefficient used to determine the probability distribution of softening and to amplify the relative relationships between classes. The gradient scaling factor is introduced to compensate for temperature scaling; σ represents the Softmax function; and These represent the student model and the teacher model respectively for the first... i The output logits of each sample; KL () represents the KL divergence between two distributions; Among them, feature-level alignment loss The calculation formula is as follows: ; in, The reference feature vector is obtained by mapping the CLS token of the teacher model through the projection head; The feature vector obtained by mapping the student model's CLS token through a feature adapter; Represents the L2 norm; Among them, uncertainty distribution alignment loss The calculation formula is as follows: ; in, L Indicates the sequence length; l For the token's location index; and The uncertainty estimation module of the student model is in the first... l The mean and variance of the output for each token position; and The teacher model uncertainty module is in the first... l The mean and variance of the output for each token position; N ( μ,σ 2 ) indicates that the mean is μ variance is σ 2 The Gaussian distribution.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention proposes a confidence-based distillation method. The teacher model infers from the training samples to obtain the predicted probability distribution and its confidence level. A confidence threshold is set based on the uncertainty of the teacher model's prediction, and only samples with confidence levels higher than the threshold are selected to participate in distillation training to reduce the noise knowledge introduced by low-confidence samples; (2) This invention models the feature representation as a probability distribution and aligns the distribution parameters of teachers and students through KL divergence to achieve feature distribution alignment for uncertainty perception. Compared with deterministic feature alignment, it can convey richer structural information. (3) This invention realizes effective cross-modal knowledge transfer from EEG-EOG bimodal teacher model to EOG unimodal student model. Under the condition of few samples, the performance of EOG unimodal student model is significantly better than the baseline of direct training, effectively reducing the complexity of the collection equipment of the alertness detection system and promoting its application in practical scenarios. Attached Figure Description

[0024] Figure 1 A schematic diagram of the alertness detection method based on uncertainty measurement and cross-modal knowledge distillation provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the teacher model (EEG-EOG dual-modal Transformer) structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of the student model (EOG single-modal) structure in an embodiment of the present invention; Figure 4 This is a schematic diagram of a knowledge distillation framework based on uncertainty guidance in an embodiment of the present invention; it includes confidence-based distillation, feature-level alignment, and uncertainty-aware distribution alignment; Figure 5 This is a schematic diagram showing the performance comparison results of different distillation strategies under a few-sample setting in the embodiments of the present invention; it shows the accuracy of each distillation strategy combination under different few-sample settings (k=5, 10, 20, 50). Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be noted that the acquisition of data and collection of information in this application are legal, compliant, or obtained with the consent of the subject of the data collection.

[0027] like Figure 1 As shown, the alertness detection method based on uncertainty measurement and cross-modal knowledge distillation provided in this embodiment of the invention includes the following steps: Step S1: Collect raw EEG and EOG signals from the subjects during the sustained attention task; The experimental data in this embodiment came from a multimodal physiological signal dataset collected by the laboratory based on the continuous attention test (PVT) paradigm. A total of 21 subjects were included, and each subject performed three PVT experiments lasting 33 minutes, and a total of multimodal physiological signal records were collected.

[0028] Step S2: Preprocess and extract features from the acquired raw EEG and EOG signals to obtain structured EEG feature matrices and EOG feature matrices, and assign an alertness status label to each sample based on the subject's behavioral reaction time index. Specifically, EEG feature extraction involves standard preprocessing steps on the original 64-channel EEG signal (retaining 59 channels), including bandpass filtering (1-50Hz), downsampling (250Hz), line noise suppression, average rereference, and artifact removal based on independent component analysis (ICA). The EEG signal within a 60-second window for each sample is uniformly divided into 20 3-second segments. For each segment, the power (equivalent differential entropy feature) of the 59 channels in the five frequency bands δ (1-4Hz), θ (4-8Hz), α (8-13Hz), β (13-30Hz), and γ (30-50Hz) is calculated. The average of the 20 segments yields an EEG feature matrix of dimension (59, 5).

[0029] Electroocular (EOG) feature extraction: Preprocessing of the raw EOG signal includes differential calculation (constructing a vertical EOG channel VEOG), channel filtering, bandpass filtering (1-50Hz), downsampling (250Hz), and line noise suppression. Using the BLINKER tool, all blink events were detected within a 60-second window, and 11 blink features were extracted, including continuous blink interval, full blink duration, half blink duration, blink closure duration, blink closure duration, blink closure duration, blink closure duration, peak amplitude, blink closure peak velocity, blink closure peak velocity, blink closure amplitude-velocity ratio, and blink closure amplitude-velocity ratio. The mean, standard deviation, median, and interquartile range of each feature were calculated to obtain a 44-dimensional EOG feature vector, which was then segmented to form an EOG feature matrix of dimension (11, 4).

[0030] Alertness status labeling: The mean reaction time after Z-score standardization within the subjects was used as the behavioral performance indicator. A dual threshold discrimination mechanism (alertness threshold -0.3597, fatigue threshold +0.3597) was used to divide the samples into alert and fatigue states. After removing samples in the transition interval, strict class balance processing was performed, and finally 6300 effective labeled samples with 3150 samples in each class were obtained, with complete class balance.

[0031] Step S3: Using the EEG feature matrix and EOG feature matrix as input, construct and train a bimodal teacher model; like Figure 2 As shown, the teacher model takes the EEG feature matrix and EOG feature matrix as input and includes the following core modules: (1) Input layer; (2) Sequence embedding layer: The EEG feature matrix is ​​independently linearly projected along the time step dimension (5 steps) to map the 59-dimensional channel features of each time step to the model dimension (model_dim=128), and a CLS token and learnable position encoding are added to the beginning of the sequence. Finally, after Dropout, the embedded sequence with shape (B, 6, 128) is output. The EOG encoding branch adopts the same structure to map the EOG features of (11, 4) to the sequence of (B, 5, 128).

[0032] (3) Transformer self-attention encoder: a multi-head self-attention mechanism (n_heads=4, head_dim=32) and a feedforward network (feed_dim=512) are used to aggregate features within the sequence and to extract deep features from EEG and EOG sequences respectively.

[0033] (4) Cross-modal fusion encoder: EEG encoding is used to represent the query, and EOG encoding is used to represent the key and value. Cross-modal interactive modeling is achieved through the cross-modal Transformer encoder, and the fused multimodal representation is output. Its CLS token is used for binary classification prediction.

[0034] (5) Neural Adaptive Uncertainty Estimation Module: For the encoded sequences of EEG and EOG respectively, the distribution parameters (μ, log σ) of each token position are output through the mean mapping layer and the log-variance mapping layer respectively. 2 The randomized representation z is obtained through reparameterized sampling. The aforementioned uncertainty parameters will provide a distribution alignment objective for the student model during the distillation phase.

[0035] (6) Projection head: The EEG and EOG representations and their randomized sampling results are mapped to a low-dimensional common feature space (projection_dim=128) through a nonlinear projection layer for feature-level alignment during the distillation stage.

[0036] (7) Classification head: The CLS token, which is a multimodal representation after cross-modal fusion, is used as input. The binary classification prediction probability is output through Dropout and linear layer. During the training phase, supervised optimization is performed using cross-entropy loss. During the inference phase, the final alertness state judgment result is output.

[0037] The teacher model uses full labeled training data for 5-fold cross-validation training with cross-entropy loss and adopts an optimal checkpoint saving strategy based on training set balancing accuracy.

[0038] Step S4: Construct an EOG single-modal student model using the EOG feature matrix as input; like Figure 3 As shown, the Student Model uses the EOG feature matrix as its sole input, and its core modules include: an input layer, an encoding module, an uncertainty estimation module, and a classification module. The input layer is used to input the EOG feature matrix; The encoding module includes a sequence embedding layer and a Transformer encoder, which are used to perform sequence encoding on the input EOG feature matrix and extract the single-modal context representation. It should be noted that the EOG encoding branches of the student model and the teacher model use the same sequence embedding layer and Transformer encoding structure, and output the EOG sequence representation containing the CLS token.

[0039] The uncertainty estimation module has the same structure as the teacher model uncertainty estimation module, and calculates the distribution parameters (μ_S, log σ) for the EOG sequence representation. 2 _S) and reparameterized sampling, providing a student-side distribution estimate for alignment of uncertainty distributions during training.

[0040] The classification module includes a feature adapter and a classification head. The feature adapter comprises a non-linear transformation layer (two linear transformation layers + ReLU) for the CLS tokens, and a feature adapter consisting of a linear layer, batch normalization, and ReLU. It is responsible for mapping the CLS token representation of the student model to a common feature space shared with the teacher model, supporting feature-level distillation alignment. The classification head outputs a binary classification prediction based on the original CLS token representation after Dropout and a linear layer, independently determining the alertness state during the inference phase.

[0041] Step S5: Using the trained teacher model as the knowledge source, perform uncertainty-guided cross-modal knowledge distillation training on the student model to achieve effective knowledge transfer from bimodal teachers to unimodal students; like Figure 4 As shown, the distillation training framework includes three complementary loss terms: (1) Confidence level screening for soft-label distillation loss: During the distillation training phase, the teacher model parameters are first fixed, and forward inference is performed on a small support set to obtain the predicted probability distribution and corresponding prediction confidence for each sample. The confidence is represented by the maximum class probability in the teacher model's output probability distribution and is used to measure the reliability of the teacher model's prediction for that sample.

[0042] Considering the varying prediction stability of the teacher model for different samples in physiological signal scenarios, directly using all samples for distillation might introduce noisy knowledge from low-confidence samples, thus affecting the learning performance of the student model. Therefore, this invention introduces a confidence screening mechanism: a threshold is set based on the prediction confidence of the teacher model, and only samples with confidence levels higher than the threshold are selected for distillation training, thereby ensuring the reliability of the distilled knowledge.

[0043] For a sample set S that meets the confidence screening criteria, the student model learns the category relationship information contained in the soft labels of the teacher model through KL divergence loss with a temperature coefficient T. The calculation formula is as follows: ; Where S represents the set of samples with confidence levels higher than a set threshold, |S| is the size of the set, and T is a temperature coefficient used to soften the probability distribution and amplify the relative relationships between classes. The gradient scaling factor is introduced to compensate for temperature scaling; σ represents the Softmax function; and These represent the student model and the teacher model respectively for the first... i The output logits of each sample; KL() represents the KL divergence between the two distributions.

[0044] (2) Feature-level alignment loss: The teacher reference features are obtained by mapping the CLS token representation of the teacher model through the projector head. Simultaneously, the student model's CLS token representation is mapped to student features via a feature adapter. By constraining the difference between the two through mean squared error loss, the alignment of teacher and student feature representations in the common feature space is achieved. The calculation formula is as follows: ; in, The reference feature vector is obtained by mapping the CLS token of the teacher model through the projection head; The feature vector obtained by mapping the student model's CLS token through a feature adapter; This represents the L2 norm.

[0045] (3) Uncertainty distribution alignment loss: The distribution parameters output by the uncertainty module of the EOG branch of the teacher model ( μ T ,logσ T 2 ) and the output of the student model uncertainty estimation module ( μ S ,logσ S2 The distribution parameters of ) represent two Gaussian probability distributions. Alignment at the feature probability distribution level is achieved by calculating the KL divergence between the student distribution and the teacher distribution. The loss is calculated token-by-token and then averaged over the entire sequence; its form is: ; in, L Indicates the sequence length. l For the token's location index; and The uncertainty estimation module of the student model is in the first... l The mean and variance of the output for each token position; and The teacher model uncertainty module is in the first... l The mean and variance of the output at each token position; N(μ,σ) 2 ) represents a mean of μ and a variance of σ. 2 The Gaussian distribution.

[0046] The total training loss of the student model is composed of multiple losses: ; in, Cross-entropy classification loss, To screen for soft-label distillation loss with confidence levels, Feature-level alignment loss; For the uncertainty distribution alignment loss, , , The balancing hyperparameters for each loss term. In the few-shot training setting, k samples are sampled from each class (k∈{5,10,20,50}) to form the support set, which is evaluated using 5-fold cross-validation.

[0047] Step S6: Deploy the trained student model and perform alertness state detection inference based on the EOG single-modal input.

[0048] After training, the student model is deployed to the target application scenario. During the inference phase, only an EOG feature matrix with dimensions of (11, 4) needs to be input. Through the EOG encoding branch and classification head of the student model, the binary classification prediction result of the alertness state (alertness / fatigue) can be directly output. No EEG equipment or teacher model is required, which effectively reduces the complexity of the system's data acquisition equipment and is suitable for practical deployment scenarios such as driving.

[0049] Experimental results On the aforementioned dataset, using accuracy (Acc) and F1 score as evaluation metrics, ablation experiments were conducted to compare the proposed method with several baseline methods under different few-sample settings (k=5, 10, 20, 50), and the results are shown in Table 1.

[0050]

[0051] Table 1. Detection performance of different distillation methods under small sample conditions With a very small sample size (k=5), the baseline method using only cross-entropy loss (CE only) achieved an accuracy of 66.8% and an F1 score of 65.9%. Introducing standard soft-label distillation did not significantly improve performance (66.3% / 65.5%), indicating that conventional distillation methods are insufficient to consistently improve model performance with very few samples. Further introducing a confidence-based distillation strategy improved model performance to 68.1% / 67.3%. Adding a feature-level alignment mechanism further increased accuracy to 68.9% and F1 score to 68.1%. The proposed complete method achieved optimal performance of 69.4% / 68.8% with k=5, representing an improvement of approximately 2.6 percentage points compared to the baseline method using only cross-entropy training. This demonstrates that the uncertainty-guided distillation strategy can effectively improve model performance in scenarios with very few samples.

[0052] With the increase in the number of samples, the overall performance of each method showed an improving trend. When k=10, the accuracy of cross-entropy was 70.9%, while the method of this invention reached 74.0% / 73.4%; when k=20, the accuracy of this invention further improved to 76.8% / 76.5%, significantly better than other distillation strategies; when the number of samples increased to k=50, the performance of each method tended to stabilize, with the cross-entropy at 74.2% / 74.1%, while the method of this invention reached 76.9% / 76.4%, still maintaining the best results.

[0053] Overall, the proposed method outperforms other comparative methods in all few-sample settings, and its advantage persists as the sample size increases. Experimental results show that confidence-based distillation effectively suppresses noisy knowledge introduced by the teacher model's low-confidence predictions, feature-level alignment further enhances the student model's learning of teacher feature representations, and distribution alignment for uncertainty perception conveys richer structural information at the probability distribution level. The synergistic effect of these three mechanisms enables the student model to achieve discriminative capabilities approaching those of a multimodal teacher model, even when relying solely on EOG unimodal input, thus achieving a significant performance improvement in few-sample alertness detection tasks.

[0054] Finally, it should be noted that the above embodiments are merely illustrative and explanatory of the present invention, and are not intended to limit the present invention to the scope of the described embodiments. Furthermore, those skilled in the art will understand that the present invention is not limited to the above embodiments, and many more variations and modifications can be made based on the teachings of the present invention, all of which fall within the scope of protection claimed by the present invention.

Claims

1. An alertness detection method based on uncertainty measurement and cross-modal knowledge distillation, characterized in that, Includes the following steps: Step S1: Collect raw EEG and EOG signals from the subjects during the sustained attention task; Step S2: Preprocess and extract features from the acquired raw EEG and EOG signals to obtain structured EEG feature matrices and EOG feature matrices, and assign an alertness status label to each sample based on the subject's behavioral reaction time index. Step S3: Using the EEG feature matrix and EOG feature matrix as input, construct and train a bimodal teacher model; Step S4: Construct an EOG single-modal student model using the EOG feature matrix as input; Step S5: Using the trained teacher model as the knowledge source, perform cross-modal knowledge distillation training on the student model based on uncertainty guidance; Step S6: Deploy the trained student model and perform alertness state detection inference based on the EOG single-modal input.

2. The alertness detection method based on uncertainty measurement and cross-modal knowledge distillation according to claim 1, characterized in that, In step S2, the preprocessing and feature extraction of the EEG signal specifically involves: The original EEG signal is sequentially processed by bandpass filtering, downsampling, line noise suppression, average rereference, and artifact removal to obtain a high-quality EEG signal. The preprocessed EEG signal is segmented using a sliding time window, and spatial features are extracted in each frequency band. The features of multiple time segments are averaged to form a structured EEG feature matrix. The preprocessing and feature extraction of EOG signals are as follows: The original EOG signal is processed sequentially by differential calculation, bandpass filtering, downsampling and line noise suppression to obtain a high-quality eye movement signal; blinking events are detected within the time window, and time, amplitude and velocity features reflecting blinking behavior are extracted. The statistical values ​​of each feature are summarized to form a structured EOG feature matrix. The generation of alertness status labels is specifically as follows: The subjects' behavioral reaction time was standardized within the subjects to eliminate the influence of individual differences. A dual-threshold discrimination mechanism was used to divide the standardized behavioral performance indicators into alertness and fatigue states. Samples in the transition interval between the two thresholds were removed, and the two types of samples were balanced to obtain a balanced alertness state label.

3. The alertness detection method based on uncertainty measurement and cross-modal knowledge distillation according to claim 1, characterized in that, In step S3, the bimodal teacher model includes the following: an input layer, a sequence embedding layer, a Transformer self-attention encoder and a cross-modal fusion encoder, a neural adaptive uncertainty estimation module, a projection head, and a classification head; The input layer is used to input the EEG feature matrix and the EOG feature matrix; The sequence embedding layer includes an EEG encoding branch and an EOG encoding branch, wherein the EEG encoding branch and the EOG encoding branch respectively perform sequence encoding on their respective input feature matrices; The Transformer self-attention encoder is used to extract deep features from EEG and EOG sequences, respectively. The cross-modal fusion encoder is used to fuse bimodal information through a cross-modal attention mechanism, using EEG as the query and EOG as the key and value, to obtain a multimodal fusion representation; The neural adaptive uncertainty estimation module is used to model each modality encoding representation as a probability distribution, output distribution parameters, and obtain a randomized representation through reparameterization sampling; The projection head is used to map the EEG and EOG representations and their randomized sampling results to a low-dimensional common feature space through a nonlinear projection layer, which is used for feature-level alignment during the distillation stage. The classification head is used to take the CLS token, which is a multimodal representation after cross-modal fusion, as input, and output binary classification prediction probability through Dropout and a linear layer. During the training phase, it is optimized in a supervised manner using cross-entropy loss, and during the inference phase, it outputs the final alertness state judgment result.

4. The alertness detection method based on uncertainty measurement and cross-modal knowledge distillation according to claim 1, characterized in that, In step S4, the student model includes the following: an input layer, an encoding module, an uncertainty estimation module, and a classification module; The input layer is used to input the EOG feature matrix; The encoding module includes a sequence embedding layer and a Transformer encoder, which are used to perform sequence encoding on the input EOG feature matrix and extract the single-modal context representation; The uncertainty estimation module is used to model the encoded representation as a probability distribution and sample it; The classification module includes a feature adapter and a classification head; The feature adapter is used to perform a nonlinear transformation on the encoded representation, mapping it to a common feature space shared with the teacher model, in order to support cross-modal feature alignment during the distillation process; The classification head is used to predict the probability of outputting the alertness state based on the encoded representation.

5. The alertness detection method based on uncertainty measurement and cross-modal knowledge distillation according to claim 1, characterized in that, In step S5, the total training loss of the student model It consists of multiple losses: ； in, Cross-entropy classification loss, To screen for soft-label distillation loss with confidence levels, Feature-level alignment loss; For the uncertainty distribution alignment loss, , , The balancing hyperparameters for each loss term; in the few-shot training setting, k samples are sampled from each class, k∈{5,10,20,50} to form the support set, and 5-fold cross-validation is used for evaluation; Among them, confidence level screening for soft-label distillation loss The calculation formula is as follows: ； in, S This represents the set of samples with a confidence level higher than a set threshold, where |S| is the size of the set. The confidence level being higher than the set threshold means that a threshold is set based on the confidence level predicted by the teacher model, and only samples with a confidence level higher than the threshold are selected to participate in distillation training. T This is a temperature coefficient used to determine the probability distribution of softening and to amplify the relative relationships between classes. The gradient scaling factor is introduced to compensate for temperature scaling; σ represents the Softmax function; and These represent the student model and the teacher model respectively for the first... i The output logits of each sample; KL () represents the KL divergence between two distributions; Among them, feature-level alignment loss The calculation formula is as follows: ； in, The reference feature vector is obtained by mapping the CLS token of the teacher model through the projection head; The feature vector obtained by mapping the student model CLStoken through the feature adapter; Represents the L2 norm; Among them, uncertainty distribution alignment loss The calculation formula is as follows: ； in, L Indicates the sequence length; l For the token's location index; and The uncertainty estimation module of the student model is in the first... l The mean and variance of the output for each token position; and The teacher model uncertainty module is in the first... l The mean and variance of the output for each token position; N ( μ,σ 2 ) indicates that the mean is μ variance is σ 2 The Gaussian distribution.

6. An alertness detection system based on uncertainty measurement and cross-modal knowledge distillation, characterized in that, Includes the following units: The system includes a data acquisition unit, a preprocessing and extraction unit, a bimodal teacher model construction unit and a student model construction unit, a cross-modal knowledge distillation training unit, and a reasoning unit. The acquisition unit is used to acquire the raw EEG and EOG signals of the subject during the sustained attention task. The preprocessing and extraction unit is used to preprocess and extract features from the acquired raw EEG and EOG signals respectively, to obtain structured EEG feature matrices and EOG feature matrices, and to assign an alertness status label to each sample based on the subject's behavioral reaction time index. The bimodal teacher model construction unit is used to construct and train a bimodal teacher model using the EEG feature matrix and the EOG feature matrix as input. The student model building unit is used to construct an EOG single-modal student model using the EOG feature matrix as input. The cross-modal knowledge distillation training unit is used to perform uncertainty-guided cross-modal knowledge distillation training on the student model using the trained teacher model as the knowledge source. The inference unit is used to deploy the trained student model and perform alertness state detection inference based on the EOG single-modal input.

7. The alertness detection system based on uncertainty measurement and cross-modal knowledge distillation according to claim 6, characterized in that, The preprocessing and feature extraction of EEG signals are as follows: The original EEG signal is sequentially processed by bandpass filtering, downsampling, line noise suppression, average rereference, and artifact removal to obtain a high-quality EEG signal. The preprocessed EEG signal is segmented using a sliding time window, and spatial features are extracted in each frequency band. The features of multiple time segments are averaged to form a structured EEG feature matrix. The preprocessing and feature extraction of EOG signals are as follows: The original EOG signal is processed sequentially by differential calculation, bandpass filtering, downsampling and line noise suppression to obtain a high-quality eye movement signal; blinking events are detected within the time window, and time, amplitude and velocity features reflecting blinking behavior are extracted. The statistical values ​​of each feature are summarized to form a structured EOG feature matrix. The generation of alertness status labels is specifically as follows: The subjects' behavioral reaction time was standardized within the subjects to eliminate the influence of individual differences. A dual-threshold discrimination mechanism was used to divide the standardized behavioral performance indicators into alertness and fatigue states. Samples in the transition interval between the two thresholds were removed, and the two types of samples were balanced to obtain a balanced alertness state label.

8. The alertness detection system based on uncertainty measurement and cross-modal knowledge distillation according to claim 6, characterized in that, The bimodal teacher model includes the following components: an input layer, a sequence embedding layer, a Transformer self-attention encoder and a cross-modal fusion encoder, a neural adaptive uncertainty estimation module, a projection head, and a classification head. The input layer is used to input the EEG feature matrix and the EOG feature matrix; The sequence embedding layer includes an EEG encoding branch and an EOG encoding branch, wherein the EEG encoding branch and the EOG encoding branch respectively perform sequence encoding on their respective input feature matrices; The Transformer self-attention encoder is used to extract deep features from EEG and EOG sequences, respectively. The cross-modal fusion encoder is used to fuse bimodal information through a cross-modal attention mechanism, using EEG as the query and EOG as the key and value, to obtain a multimodal fusion representation; The neural adaptive uncertainty estimation module is used to model each modality encoding representation as a probability distribution, output distribution parameters, and obtain a randomized representation through reparameterization sampling; The projection head is used to map the EEG and EOG representations and their randomized sampling results to a low-dimensional common feature space through a nonlinear projection layer, which is used for feature-level alignment during the distillation stage. The classification head is used to take the CLS token, which is a multimodal representation after cross-modal fusion, as input, and output binary classification prediction probability through Dropout and a linear layer. During the training phase, it is optimized in a supervised manner using cross-entropy loss, and during the inference phase, it outputs the final alertness state judgment result.

9. The alertness detection system based on uncertainty measurement and cross-modal knowledge distillation according to claim 6, characterized in that, The student model consists of the following components: an input layer, an encoding module, an uncertainty estimation module, and a classification module. The input layer is used to input the EOG feature matrix; The encoding module includes a sequence embedding layer and a Transformer encoder, which are used to perform sequence encoding on the input EOG feature matrix and extract the single-modal context representation; The uncertainty estimation module is used to model the encoded representation as a probability distribution and sample it; The classification module includes a feature adapter and a classification head; The feature adapter is used to perform a nonlinear transformation on the encoded representation, mapping it to a common feature space shared with the teacher model, in order to support cross-modal feature alignment during the distillation process; The classification head is used to predict the probability of outputting the alertness state based on the encoded representation.

10. The alertness detection system based on uncertainty measurement and cross-modal knowledge distillation according to claim 6, characterized in that, When the cross-modal knowledge distillation training unit is executed, the total training loss of the student model is... It consists of multiple losses: ； in, Cross-entropy classification loss, To screen for soft-label distillation loss with confidence levels, Feature-level alignment loss; For the uncertainty distribution alignment loss, , , The balancing hyperparameters for each loss term; in the few-shot training setting, k samples are sampled from each class, k∈{5,10,20,50} to form the support set, and 5-fold cross-validation is used for evaluation; Among them, confidence level screening for soft-label distillation loss The calculation formula is as follows: ； in, S This represents the set of samples with a confidence level higher than a set threshold, where |S| is the size of the set. The confidence level being higher than the set threshold means that a threshold is set based on the confidence level predicted by the teacher model, and only samples with a confidence level higher than the threshold are selected to participate in distillation training. T This is a temperature coefficient used to determine the probability distribution of softening and to amplify the relative relationships between classes. The gradient scaling factor is introduced to compensate for temperature scaling; σ represents the Softmax function; and These represent the student model and the teacher model respectively for the first... i The output logits of each sample; KL () represents the KL divergence between two distributions; Among them, feature-level alignment loss The calculation formula is as follows: ； in, The reference feature vector is obtained by mapping the CLS token of the teacher model through the projection head; The feature vector obtained by mapping the student model CLStoken through the feature adapter; Represents the L2 norm; Among them, uncertainty distribution alignment loss The calculation formula is as follows: ； in, L Indicates the sequence length; l For the token's location index; and The uncertainty estimation module of the student model is in the first... l The mean and variance of the output for each token position; and The teacher model uncertainty module is in the first... l The mean and variance of the output for each token position; N ( μ,σ 2 ) indicates that the mean is μ variance is σ 2 The Gaussian distribution.

Images