A brain fatigue dynamic early warning method and device, computer equipment and storage medium

Abstract

The application provides a brain fatigue dynamic early warning method and device, computer equipment and a storage medium, and belongs to the field of traffic safety. The method comprises the following steps: obtaining a pupil diameter entropy time sequence, a gaze entropy time sequence, time characteristics and a preset task stage label of a driver; preprocessing the above characteristics respectively, splicing the preprocessing results, and obtaining a joint feature matrix; performing time sequence feature extraction on the joint feature matrix, and generating shared hidden features; generating distribution parameters of cognitive load, fatigue state and alertness state from the shared hidden features respectively, and then determining state values of the fatigue state and the alertness state from the distribution parameters through reparameterization sampling; and performing early warning according to the state values of the fatigue state and the alertness state in combination with a driving scene. In this way, the application realizes determination of the brain fatigue state through the pupil state by means of multi-modal fusion, mechanism constraint decoupling and dynamic risk assessment, and can reduce the traffic accident rate caused by fatigue in the field of driving safety.

Description

A method, device, computer equipment, and storage medium for dynamic early warning of brain fatigue. Technical Field

[0001] This invention belongs to the field of traffic safety, and specifically relates to a method, device, computer equipment, and storage medium for dynamic early warning of brain fatigue. Background Technology

[0002] Traffic safety is a crucial part of building a modern traffic safety prevention and control system. According to data, more than 20% of traffic accidents each year are caused by fatigued driving.

[0003] The causes of driver fatigue are complex and diverse, primarily including fatigue caused by circadian rhythms and sleep debt, as well as task-related fatigue (active and passive fatigue). Active fatigue is caused by prolonged, high-intensity driving activities. Drivers must maintain high concentration and perform frequent driving maneuvers when dealing with complex road conditions, leading to excessive consumption of physical and mental resources and resulting in fatigue. Passive fatigue, on the other hand, occurs because the driving task is monotonous and lacks sufficient stimulation, causing the brain to operate at low load for extended periods, making it difficult for drivers to concentrate and gradually leading to symptoms such as drowsiness and slowed reaction time. These four types of fatigue have different causes, and their warning thresholds and intervention strategies also differ. Dynamically monitoring and differentiating these factors during driving is crucial for targeted warnings and appropriate interventions.

[0004] However, existing common technologies cannot distinguish between different types of fatigue. Behavioral tests such as the PVT (psychomotor alertness task) and the McWhorst clock test rely on peripheral muscle activity, and their results are easily affected by muscle factors; the PERCLOS method can quantify the degree of eyelid closure, but skeletal muscle activity affects its accuracy; EEG and MRI equipment are difficult to apply to driving scenarios due to their large size and demanding operating conditions; and HRV (heart rate variability) cannot specifically reflect the state of brain alertness due to significant individual differences.

[0005] The locus coeruleus (LC, or simply locus coeruleus) acts as the "alertness control center," regulating global alertness. Changes in pupil diameter are highly synchronized with locus coeruleus activity, providing an important window for studying brain fatigue. Traditional static models cannot analyze dynamic neural mechanisms such as the "exploration-exploitation" strategy transition and locus coeruleus activation. Traditional models typically assume that the interactions between factors are linear and constant, but in reality, the interactions of various factors in the cognitive process are complex and variable, exhibiting nonlinear and dynamic characteristics. This prevents traditional models from deeply exploring the rich neurophysiological information represented by fluctuations in the PCI (pupil-locus complexity index), limiting the progress in accurate modeling and mechanism analysis of cognitive states. Therefore, determining brain fatigue states based on pupil state is crucial for targeted early warning and appropriate intervention. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a method, device, computer equipment, and storage medium for dynamic early warning of brain fatigue.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A dynamic early warning method for brain fatigue, the method comprising:

[0009] Real-time acquisition of the driver's pupil diameter entropy time series and gaze entropy time series; determination of time features based on the current time; acquisition of the preset task stage label corresponding to the current moment;

[0010] The pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels are concatenated to obtain a joint feature matrix.

[0011] Temporal features are extracted from the joint feature matrix to generate shared hidden features; distribution parameters for cognitive load, fatigue state, and alertness state are generated from the shared hidden features, and then the state values ​​of fatigue state and alertness state are determined from the distribution parameters through reparameterized sampling.

[0012] Warnings are issued based on the fatigue and alertness status values ​​combined with the driving scenario.

[0013] Optionally, before splicing the pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels, preprocessing is performed, including:

[0014] The pupil diameter entropy time series E(t) and the gaze entropy time series G(t) are standardized respectively, using the following formulas:

[0015] ;

[0016] ;

[0017] in, and These are the mean and standard deviation of the pupil diameter entropy time series, respectively. and These are the mean and standard deviation of the gaze entropy time series, respectively.

[0018] Encode the time feature h(t) into a Fourier base with a 24-hour period. and Generate time-coded features The specific formula is as follows:

[0019] ;

[0020] Where k represents the first and second harmonics, and h(t) represents the time characteristic;

[0021] Preset task stage labels are used for one-hot encoding to generate task feature vectors. The specific formula is as follows:

[0022] ;

[0023] in, v represents the number of task stages. i It is the i-th element in the one-hot encoded vector.

[0024] Optionally, the step of concatenating the pupil diameter entropy time series, the gaze entropy time series, the time features, and the preset task stage labels to obtain a joint feature matrix includes:

[0025] Preprocessed and The joint feature matrix is ​​constructed by concatenating the features along the time dimension, as shown in the following formula:

[0026] ;

[0027] in, The joint characteristic matrix at time t has the following shape: T is the time step. The dimension of the input feature. This is the symbol for the correlation coefficient matrix.

[0028] Optionally, temporal feature extraction is performed on the joint feature matrix using a TCN network, wherein the TCN network includes dilated convolutional layers, defined as follows:

[0029] ;

[0030] in, The shared hidden features output by the dilated convolutional layer. Let L be the input joint feature matrix, and L be the number of dilated convolutional layers.

[0031] Optionally, after determining the state values ​​of fatigue and alertness, the method further includes:

[0032] Determine the state value of cognitive load;

[0033] The pupil diameter entropy signal and the gaze entropy signal are reconstructed based on the state values ​​of cognitive load, fatigue state, and alertness state, respectively.

[0034] The reconstruction formula for the pupil diameter entropy signal is as follows:

[0035] ;

[0036] in, To reconstruct the generated pupil diameter entropy signal, These are cognitive load, fatigue state, and alertness state, respectively. The weight matrix for generating the pupil diameter entropy branch. This is the bias term for the pupil diameter entropy generation branch;

[0037] The reconstruction formula for the gaze entropy signal is:

[0038] ;

[0039] in, These represent the state values ​​of cognitive load, fatigue state, and alertness state at time step t, respectively. This represents the feature vector of the task stage corresponding to time step t. Let be the hidden state of the LSTM network at time step t−1; To reconstruct the generated gaze entropy signal, The weight matrix for generating branches of gaze entropy. The bias term for the gaze entropy generation branch;

[0040] according to and Determine the pupil diameter entropy signal error based on and The gaze entropy signal error is determined, and the fatigue state and alertness state are verified based on the pupil diameter entropy signal error and the gaze entropy signal error. The real-time risk index is then calculated based on the verified fatigue state and alertness state.

[0041] Optionally, before reconstructing the pupil diameter entropy signal and the gaze entropy signal, dynamic prior constraints are applied to cognitive load, fatigue state, and alertness state, including:

[0042] Cognitive load Perform piecewise stability priors; for fatigue states Perform Brownian motion prior with drift ; state of alertness Perform periodic normal prior Synchronize with the 24-hour biological clock; among which, This is the preset fatigue state drift amount. This represents the standard deviation of the fatigue state fluctuation.

[0043] Optionally, the step of issuing a warning based on the state values ​​of the fatigue state and alertness state combined with the driving scenario is as follows:

[0044] A risk index is generated based on the state values ​​of fatigue and alertness, using the following formula:

[0045] ;

[0046] in, The state value of the fatigue state at time t after verification. The verified alert state value at time t. and These are the variable parameters;

[0047] Warnings are issued based on the risk index and the risk thresholds corresponding to different driving scenarios.

[0048] A dynamic early warning device for brain fatigue, the device comprising:

[0049] The acquisition module is used to collect the time series of the driver's pupil diameter entropy and gaze entropy in real time; and to determine the time features based on the current time.

[0050] The preprocessing module is used to concatenate the pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels to obtain a joint feature matrix.

[0051] The determination module is used to extract temporal features from the joint feature matrix and generate shared hidden features; the distribution parameters of cognitive load, fatigue and alertness state are generated from the shared hidden features respectively, and then the state values ​​of fatigue state and alertness state are determined from the distribution parameters through reparameterized sampling;

[0052] The early warning module is used to issue early warnings based on the state values ​​of the fatigue state and alertness state combined with the driving scenario.

[0053] A computer-readable storage medium, characterized in that the storage medium stores a computer program, which, when executed by a processor, implements the aforementioned dynamic early warning method for brain fatigue.

[0054] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned dynamic early warning method for brain fatigue.

[0055] The dynamic early warning method for brain fatigue provided by this invention has the following beneficial effects:

[0056] This invention uses real-time acquired pupil diameter entropy time series, which is directly correlated with locus coeruleus neural activity, to capture fatigue signals caused by the accumulation of metabolic products. The gaze entropy time series reflects the stability of eye-tracking strategies and indirectly characterizes attention allocation and cognitive load fluctuations. Combining temporal features with preset task stage labels, a joint feature matrix is ​​generated through feature splicing, effectively integrating neural metabolism, behavioral patterns, and environmental context information. This overcomes the limitation of single signals being susceptible to noise interference and significantly enhances the robustness of fatigue representation. Then, the deep temporal dependencies of the joint feature matrix are extracted to generate shared hidden features. By generating distribution parameters for fatigue and alertness states, state values ​​are determined, achieving the determination of brain fatigue state through pupil state and decoupling the physiological interpretability of fatigue state. Finally, based on fatigue and alertness states, warnings are issued in conjunction with driving scenarios. Thus, this invention upgrades the traditional fatigue monitoring system’s “single-dimensional threshold alarm” to an intelligent prevention and control system based on “neuro-metabolic-behavioral-environmental collaborative analysis” that determines brain fatigue status based on pupil state through multimodal fusion, mechanism constraint decoupling and dynamic risk assessment. In the field of driving safety, this can reduce the traffic accident rate caused by fatigue and provide quantitative basis for personalized health management. Attached Figure Description

[0057] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 is a flowchart illustrating a dynamic early warning method for brain fatigue according to an exemplary embodiment of the present invention.

[0059] Figure 2 is a model block diagram of an MDD-VAE provided by the present invention according to an exemplary embodiment.

[0060] Figure 3 is a schematic diagram of cross-modal feature fusion provided by the present invention according to an exemplary embodiment.

[0061] Figure 4 is a flowchart of a driving scenario consistency verification according to an exemplary embodiment of the present invention.

[0062] Figure 5 is a schematic diagram of a closed-loop security control system integration provided by the present invention according to an exemplary embodiment.

[0063] Figure 6 is a block diagram of a dynamic early warning device for brain fatigue provided by the present invention according to an exemplary embodiment. Detailed Implementation

[0064] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0065] This invention focuses on analyzing the dynamic interaction mechanism of cognitive states through dual-modal signals of pupil diameter entropy (E(t)) and gaze entropy (G(t)). Pupil diameter entropy directly characterizes the neural activity of the locus coeruleus noradrenergic system (LC-NE), and its dynamic changes are regulated by multiple factors: it reflects the immediate level of cognitive load (related to task difficulty, individual experience, and reward mechanisms), exhibits the cumulative fatigue effect over time, and is also affected by fluctuations in alertness caused by circadian rhythms and environmental stimuli. While gaze entropy, as an indicator of attention allocation controlled by oculomotor muscles, can indirectly reflect cognitive load and fatigue levels, its signal characteristics are significantly affected by the periodic modulation of alertness levels. The core scientific challenge lies in decoupling four key latent variables from two types of temporal signals with different physiological sources (autonomic nervous system vs. skeletal muscle system) but a coupling relationship: ① task-driven cognitive load (modulated by task complexity, experience accumulation, and stimulus intensity); ② cumulative fatigue that monotonically increases over time (manifested as the nonlinear accumulation of neural metabolites); ③ fluctuations caused by the switching of exploration / exploitation modes; and ④ alertness levels influenced by the combined effects of the biological clock and environmental stimuli (exhibiting approximately 24-hour periodic fluctuations). This dynamic decoupling of multidimensional states has significant theoretical value for achieving accurate cognitive state assessment. Based on the decoded states, targeted interventions can be implemented. For example, active fatigue caused by high cognitive load requires methods to reduce task load, decreased alertness caused by low cognitive load can be addressed through remote call communication to improve alertness, and fatigue caused by sleep debt can be addressed by forced rest.

[0066] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0067] First, this invention provides a dynamic early warning method for brain fatigue, as shown in Figure 1, which includes the following steps:

[0068] S101. Real-time acquisition of the driver's pupil diameter entropy time series and gaze entropy time series; determination of time features based on the current time; acquisition of the preset task stage label corresponding to the current moment.

[0069] In this step, an infrared camera or eye tracker can be used to capture images of the driver's eyes at a high frame rate (e.g., 100Hz), or the pupil position can be precisely located using central pupillary reflection (PCCR) technology, and the pupil diameter can be calculated. Simultaneously, the coordinates of the gaze point are recorded to form raw time-series data.

[0070] Then, the pupil diameter time series is segmented by a sliding window, and the sample entropy or approximate entropy of the signal in each window is calculated to obtain the pupil diameter entropy time series, which is used to reflect the activity of the blue spot nucleus; based on the gaze point coordinate sequence, the randomness of the gaze distribution is quantified by a Markov chain or information entropy model to obtain the gaze entropy time series, which represents the eye movement strategy.

[0071] For the time feature, h(t) represents the hour value of the current time. The hour value of the current time (24-hour format) can be read from the clock module in real time. For example, h(t) = 9 at 9:00 AM and h(t) = 22 at 10:00 PM. Alternatively, the cumulative number of hours since the start of the driving task at each time point can be recorded synchronously (for example, h(t) = 3 when driving continues for 3 hours) to analyze the cumulative effect of fatigue.

[0072] S102. The pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels are concatenated to obtain a joint feature matrix.

[0073] In this step, the four features obtained in the previous steps need to be preprocessed, specifically including the following steps.

[0074] First, the pupil diameter entropy time series E(t) and the gaze entropy time series G(t) are standardized respectively, using the following formulas:

[0075] ;

[0076] ;

[0077] in, This is the standardized pupil diameter entropy time series. The standardized gaze entropy time series, and These are the mean and standard deviation of the pupil diameter entropy time series, respectively. and These are the mean and standard deviation of the gaze entropy time series, respectively.

[0078] Secondly, the time features are encoded into Fourier bases with a 24-hour cycle. and Generate time-coded features The specific formula is as follows:

[0079] ;

[0080] Where k represents the 1st and 2nd harmonics, and h(t) is the time feature, representing the current moment in hours. This encoding maps discrete hourly values ​​to continuous periodic signals, enhancing the model's ability to model circadian rhythms (such as the 24-hour fluctuations in alertness levels). It is a key input for characterizing circadian rhythms and is transformed into a periodic signal that the model can analyze through Fourier encoding, providing temporal context support for decoupling fatigue states.

[0081] Then, one-hot encoding is performed on the preset task stage labels to generate task feature vectors. The specific formula is as follows:

[0082] ;

[0083] in, v represents the number of task stages. i It is the i-th element in the one-hot encoded vector.

[0084] Finally, the preprocessed results of the four features are concatenated to obtain the preprocessed result. and The joint feature matrix is ​​constructed by concatenating the features along the time dimension, as shown in the following formula:

[0085] ;

[0086] in, The joint characteristic matrix at time t has the following shape: T is the time step. The dimension of the input feature. This is the symbol for the correlation coefficient matrix.

[0087] S103. Perform temporal feature extraction on the joint feature matrix to generate shared hidden features; generate distribution parameters for cognitive load, fatigue state, and alertness state from the shared hidden features, and then determine the state values ​​of fatigue state and alertness state from the distribution parameters through reparameterized sampling.

[0088] In this step, a multimodal dynamic decoupled variational autoencoder (MDD-VAE) is constructed, which includes a TCN network and a latent variable branch network. After the joint feature matrix is ​​input into the MDD-VAE, the TCN network extracts temporal features from the joint feature matrix to generate shared hidden features. The independent multilayer perceptron (MLP) in the latent variable branch network generates distribution parameters for cognitive load, fatigue, and alertness states from the shared hidden features, and then determines the state values ​​of fatigue, alertness, and cognitive load based on the distribution parameters.

[0089] In addition, to improve the accuracy of the MDD-VAE during training, and to perform signal denoising and feature enhancement, the MDD-VAE may also include a dual-branch decoder. This dual-branch decoder includes a pupil diameter entropy generation branch and a gaze entropy generation branch. The pupil diameter entropy signal and gaze entropy signal are reconstructed based on the state values ​​of cognitive load, fatigue state, and alertness state, respectively. The pupil diameter entropy generation branch is implemented through a dynamic generation mechanism using stacked TCN layers, with the specific formula as follows:

[0090] ;

[0091] in, To reconstruct the generated pupil diameter entropy signal, These are cognitive load, fatigue state, and alertness state, respectively. The weight matrix for generating the pupil diameter entropy branch is used to map the output of the decoder's TCN layer to the reconstructed signal space. The time-encoded feature is the bias term of the pupil diameter entropy generation branch. Adjust the baseline of the reconstructed signal.

[0092] The gaze entropy generation branch is an LSTM-based temporal decoder, and the specific formula is as follows:

[0093] ;

[0094] in, These are the latent variables at time step t, specifically corresponding to the state values ​​of cognitive load, fatigue state, and alertness state. The feature vector of the task stage corresponding to time step t is generated by one-hot encoding. Let be the hidden state of the LSTM network at time step t−1; To reconstruct the generated gaze entropy signal, The weight matrix for generating branches of gaze entropy. This is the bias term for the gaze entropy generation branch.

[0095] according to and Determine the pupil diameter entropy signal error based on and Determine the gaze entropy signal error, and validate the latent variables based on the pupil diameter entropy signal error and the gaze entropy signal error.

[0096] Therefore, in this step, it is first necessary to construct a multimodal dynamic decoupled variational autoencoder (MDD-VAE). This model aims to build a multimodal fusion framework based on variational autoencoders, and achieve the following core objectives by dynamically decoupling the interaction between cognitive load, cumulative fatigue, and alertness level: 1) State decoupling—separating latent variables with independent representation capabilities from the mixed signals of pupil diameter entropy (autonomic neural regulation) and gaze entropy (motor control drive), ensuring that the temporal characteristics of cognitive load, fatigue, and alertness are not confused; 2) Physiological interpretability—through KL divergence regularization and dynamic prior constraints (such as the monotonically increasing nature of the fatigue state and the 24-hour periodic synchronicity of the alertness state), the latent variables strictly follow the neurophysiological mechanism; 3) Cross-modal complementary enhancement—using the covariate nucleus sensitivity of pupil signals (high load response) and the eye-movement strategy characteristics of gaze signals (alertness modulation), a cross-modal alignment loss function is designed to improve the joint reasoning ability for complex cognitive states.

[0097] The model architecture is based on a multimodal dynamic decoupled variational autoencoder (MDD-VAE). By jointly encoding the temporal signals of pupil diameter entropy and gaze entropy, it separates cognitive load, fatigue, and alertness in a structured latent space and injects dynamic prior constraints (fatigue monotonicity and alertness periodicity). Finally, it reconstructs physiological signals through a dual-branch decoder to achieve cross-modal complementary physiological interpretable state decoupling. As shown in Figure 2, Pupil_Entropy, Gaze_Entropy, Time_Features, and Task_Labels represent the time series of pupil diameter entropy, gaze entropy, time features, and preset task stage labels, respectively. Preprocess is the preprocessing step, generating the joint feature matrix Combined_Features: [E, G:Time, Task]. Subsequently, the hidden features are shared by the feature extraction network TCN, which generates the distribution parameters of cognitive load, fatigue state, and alertness state. Then, the state values ​​of cognitive load, fatigue state, and alertness state are generated through resampling. Finally, the time series signals of pupil diameter entropy and gaze entropy are reconstructed based on the state values, and the reconstruction loss is determined to minimize the loss and complete the training.

[0098] In one embodiment, the MDD-VAE includes the following modules and functions.

[0099] Feature extraction (TCN network). A Temporal Convolutional Network (TCN) is used as the feature extraction network to model temporal dependencies. Its dilated convolutional layers are defined as follows:

[0100] ;

[0101] in, The shared hidden features output by the dilated convolutional layer. Let L be the input joint feature matrix, and L be the number of dilated convolutional layers, L=8, with the dilation coefficient of each layer increasing exponentially. The purpose of using dilated convolution is to expand the receptive field, and this method has significant advantages in parameter efficiency, long-range dependency modeling, and resolution preservation. The output of each layer is... Layer normalization (LayerNorm) is applied to the output of each layer.

[0102] Latent variable branching. The distribution parameters of the latent variables are generated using independent multilayer perceptrons (MLPs):

[0103] ;

[0104] in, .

[0105] MLP structure: Fully connected layer ( ) + GELU activation function.

[0106] Reparameterization sampling. This technique uses reparameterization to sample latent variables from a Gaussian distribution. :

[0107] ;

[0108] in, The central value of the latent variable, As a measure of uncertainty for latent variables, This is standard normal noise.

[0109] Dual-branch decoder. The pupil diameter entropy generation branch is dynamically generated through stacked TCN layers:

[0110] ;

[0111] in, To reconstruct the generated pupil diameter entropy signal, These are cognitive load variables, fatigue state, and alertness state, respectively. The weight matrix for generating the pupil diameter entropy branch. For time-encoded features, This is the bias term for the pupil diameter entropy generation branch.

[0112] Latent variables and time characteristics Concatenate along the feature dimension. 4 layers of dilated convolution (dilation coefficient) ), Output dimensions of each layer The projection layer transforms the high-dimensional features of the hidden layer through a linear transformation. Mapped to one-dimensional real space This generates the final output value.

[0113] The gaze entropy generation branch is an LSTM-based time-series decoder:

[0114] ;

[0115] in, These are the latent variables at time step t, specifically corresponding to cognitive load, fatigue state, and alertness state. The feature vector of the task stage corresponding to time step t is generated by one-hot encoding. Let be the hidden state of the LSTM network at time step t−1; To reconstruct the generated gaze entropy signal, The weight matrix for generating branches of gaze entropy. This is the bias term for generating the gaze entropy branch. The LSTM layer consists of two stacked layers, with the hidden state dimension... Latent variables and task characteristics Hidden state Initialized by the global average pooling features of the TCN encoder.

[0116] according to and Determine the pupil diameter entropy signal error based on and Determine the gaze entropy signal error, and validate the latent variables based on the pupil diameter entropy signal error and the gaze entropy signal error.

[0117] Furthermore, to ensure the physiological significance of the latent variables, the model approximates the distribution to a pre-defined prior using KL divergence; that is, before reconstructing the pupil diameter entropy signal and gaze entropy signal, dynamic prior constraints are applied to the latent variables. For cognitive load variables... Perform segmented, stable priors and match task difficulty labels; for fatigue states... Perform Brownian motion prior with drift Forced monotonically increasing trend; for alert state Perform periodic normal prior Synchronize with the 24-hour biological clock. Among them, This represents the fatigue state at time step t. This is the preset fatigue state drift amount. The standard deviation of the fatigue state fluctuation is used to control the stability of the increasing trend.

[0118] Finally, the constructed model needs further training. The model's optimization objective consists of three parts: reconstruction loss to ensure the fidelity of signal generation; KL divergence term to force the distribution of latent variables to approximate physiological priors; and dynamic regularization term to introduce domain knowledge constraints. The total loss function is defined as:

[0119] ;

[0120] in These are the KL divergence weighting coefficients. These are the strength parameters for each regularization term. To reconstruct the loss function, For the KL divergence term, This is a dynamic regularization term.

[0121] Reconstruction loss. Reconstruction loss measures the difference between the decoder output and the original signal. A weighted mean square error is used to account for the different characteristics of pupil and gaze signals.

[0122] ;

[0123] The default weight is set to , This reflects a higher signal-to-noise ratio for pupil diameter entropy than for gaze entropy in relation to locus coeruleus activity.

[0124] KL divergence term. KL divergence constrains the distribution of latent variables. With pre-set priors Alignment, designing differentiated priors for different physiological mechanisms:

[0125] Cognitive load: Standard normal prior KL is calculated as follows:

[0126] .

[0127] Fatigue: Brownian motion prior with drift Its KL item is:

[0128] .

[0129] Alert: Fourier Cycle Prior KL form is the same as cognitive load.

[0130] Dynamic regularization terms. Two regularization constraints based on physiological knowledge are introduced:

[0131] fatigue monotonicity: a penalized, non-increasing state of fatigue, with a forced cumulative metabolic effect.

[0132] .

[0133] Alert cycle synchronization: ensuring Synchronize with the diurnal rhythm:

[0134] .

[0135] Training strategy. KL annealing, The weights are linearly increased from 0.1 to 1.0, with the transition completed in the first 20% of training cycles to avoid initial suppression of reconstruction learning by KL terms. Regularized weights. , The optimality is determined through grid search. The optimizer uses AdamW, and the learning rate is... Weight decay Prevent overfitting.

[0136] S104. Issue a warning based on the state values ​​of the fatigue state and alertness state combined with the driving scenario.

[0137] For example, a real-time risk index can be calculated based on the fatigue and alertness states combined with pre-labeled driving scenario tags, and a warning can be issued based on the risk index's tiered thresholds. Alternatively, a risk index can be generated based on the fatigue and alertness state values, using the following formula:

[0138] ;

[0139] in, The state value of the fatigue state at time t after verification. The verified alert state value at time t. and These are variable parameters; warnings are issued based on the risk index and the risk thresholds corresponding to different driving scenarios.

[0140] This invention employs a three-stage approach—"data-driven hierarchical structure, knowledge-guided optimization, and closed-loop feedback verification"—to construct a mapping rule base. The specific process is as follows:

[0141] Fatigue type grading criteria are established, with thresholds for potential variables. Based on the statistical distributions (mean ± standard deviation) of the decoupled fatigue factor F(t) and alertness factor A(t), fatigue states and types are categorized. A dynamic mapping rule base of "cognitive state - intervention strategy" is constructed. Active fatigue (high cognitive load): triggering task degradation (e.g., extended ACC following distance, voice assistant taking over simple tasks). Passive fatigue (low alertness): initiating environmental stimuli (5000K cold light, mint aromatherapy) and remote communication activation. Sleep debt accumulation: graded rest strategies (15-minute nap → mandatory service station stop). Dynamic priority mechanism: based on driving scenario (highway / urban road) and real-time risk index. Adjust the strategy trigger threshold, where, The state value of the fatigue state at time t after verification. The state value of the alert state at time t after verification. and These are the variable parameters.

[0142] This invention also includes multimodal data fusion and model interpretability verification. First, multimodal data acquisition and spatiotemporal alignment are performed. Data sources cover multiple aspects. Core signals include pupil diameter sampled at 100Hz and gaze coordinates sampled at 100Hz. Physiological indicators include heart rate variability (HRV) sampled at 1Hz and EEG (θ / β wave power ratio) sampled at 128Hz. Driving behavior data involves steering wheel angle variance and lane departure frequency sampled at 10Hz. Subjective assessment data is also included, obtained through immediate post-task scoring using the NASA-TLX scale. In the preprocessing workflow, the issue of different sampling rates across multiple devices is addressed by using Dynamic Time Warping (DTW) for time alignment. After time alignment, feature extraction is performed, and the extracted features consist of multiple components. Next, cross-modal feature fusion and enhancement are performed, as shown in Figure 3. This invention employs multiple methods to improve interpretability verification. First, canonical correlation analysis (CCA) is used, which maximizes the cross-modal correlation between physiological signals and behavioral indicators (lane departure), uncovering potential connections between different modalities.

[0143] Next comes adversarial training, which uses a domain discriminator to constrain the consistency of feature distributions across different driving scenarios (highway and city scenarios). The loss function for adversarial training... Defined as This allows the model to learn more general features across different scenarios. Finally, there are data augmentation strategies. On one hand, temporal adversarial generation is employed, using Wasserstein GAN to generate synthetic data for extreme driving scenarios (such as continuous curves with strong light interference), enriching data diversity and enhancing the model's adaptability to complex scenarios. On the other hand, a feature masking strategy is implemented, randomly discarding 20% ​​of the gaze data to simulate a brief eye-tracking failure, enabling robust training and improving the model's stability in the face of data anomalies.

[0144] Finally, an interpretability verification method, namely a physiological consistency test, was performed. First, subjective-objective alignment was checked: the Pearson correlation coefficient between the latent variables and the NASA-TLX score was calculated. ,k∈{cognitive load, fatigue, alertness}, where, NASA-TLX rating NASA k The standard deviation reflects the dispersion of subjective ratings. Subjective ratings obtained using the NASA-TLX scale were used to quantify the driver's cognitive load, fatigue, or alertness level. Neural mechanism validation: Alertness status was validated using EEG microstate analysis (K-means clustering). Spatiotemporal correlation with alpha wave suppression. Driving scenario decoupling visualization: t-SNE projection reduces the latent space to 2D, labeling distribution clusters for different driving scenarios (nighttime highway, rainy city, etc.); SHAP value analysis: quantifying the contribution of each modal feature to state classification. Where N represents the set of all input features, and S represents a subset of N that does not include feature i. Next is the driving scenario consistency verification, as shown in Figure 4. Six typical scenarios (long straight roads, continuous curves, tunnels with strong light, etc.) were constructed on the CARLA platform, and 50 subjects aged 26-40 with 3 or more years of driving experience were recruited to collect cross-scenario data; 10 test vehicles equipped with the in-vehicle system collected a total of 10,000 kilometers of natural driving data.

[0145] This invention employs a three-stage approach—"layered architecture design, real-time optimization, and fault tolerance verification"—to construct a highly reliable embedded closed-loop safety control system integration, as shown in Figure 5. Data is collected using an infrared eye tracker or high-precision GNSS, and then transmitted to the processor based on the data transmission protocol (RS-422). Signal preprocessing is then performed using the FPGA and TCN accelerator IP core. Brain fatigue recognition is performed based on the QNX Neutrino RTOS and dynamic priority task scheduling within the PCIe platform, and feedback is provided based on the recognition results, such as for haptic feedback seats, fragrance / light environment control, and autonomous driving domain controllers.

[0146] The specific process of the system design is as follows: In the heterogeneous computing architecture design, in terms of hardware platform selection, the main control unit adopts NVIDIA Jetson AGX Orin, which has 64GB RAM and 200 TOPS of AI computing power; the real-time coprocessor adopts TI TDA4VM, which has dual-core Cortex-R5F and lockstep core redundancy; the communication bus adopts CAN FD (with a rate of up to 8Mbps) and Ethernet AVB (time-sensitive network).

[0147] Using the above method, the real-time acquired pupil diameter entropy time series is directly correlated with the neural activity of the locus coeruleus, and can capture fatigue signals caused by the accumulation of metabolic products. The gaze entropy time series reflects the stability of eye-tracking strategies and indirectly characterizes the fluctuations in attention allocation and cognitive load. Combining time features and preset task stage labels, a joint feature matrix is ​​generated through feature splicing, which effectively integrates neural metabolism, behavioral patterns, and environmental context information, solving the limitation of single signals being susceptible to noise interference and significantly enhancing the robustness of fatigue representation. Then, the deep temporal dependencies of the joint feature matrix are extracted to generate shared hidden features, and the state values ​​are determined by generating distribution parameters of fatigue and alertness states. This realizes the determination of brain fatigue state through pupil state and decouples the physiological interpretability of fatigue state. Finally, based on fatigue state and alertness state, warnings are issued in combination with driving scenarios. Thus, this invention upgrades the traditional fatigue monitoring system’s “single-dimensional threshold alarm” to an intelligent prevention and control system based on “neuro-metabolic-behavioral-environmental collaborative analysis” that determines brain fatigue status based on pupil state through multimodal fusion, mechanism constraint decoupling and dynamic risk assessment. In the field of driving safety, this can reduce the traffic accident rate caused by fatigue and provide quantitative basis for personalized health management.

[0148] Secondly, the present invention also provides a dynamic early warning device for brain fatigue, as shown in Figure 6, comprising:

[0149] The acquisition module 201 is used to collect the time series of the driver's pupil diameter entropy and gaze entropy in real time; and to determine the time characteristics based on the current time.

[0150] The preprocessing module 202 is used to concatenate the pupil diameter entropy time series, the gaze entropy time series, the time features, and the preset task stage labels to obtain a joint feature matrix.

[0151] The determination module 203 is used to extract temporal features from the joint feature matrix and generate shared hidden features; the distribution parameters of cognitive load, fatigue and alertness state are generated from the shared hidden features respectively, and then the state values ​​of fatigue state and alertness state are determined from the distribution parameters through reparameterized sampling.

[0152] The early warning module 204 is used to issue early warnings based on the state values ​​of the fatigue state and alertness state combined with the driving scenario.

[0153] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the steps of the dynamic early warning method for brain fatigue provided in FIG1.

[0154] This invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the steps of the dynamic early warning method for brain fatigue provided in Figure 1.

[0155] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0157] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0159] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the patent of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A dynamic early warning method for brain fatigue, characterized in that, The method includes: real-time acquisition of the driver's pupil diameter entropy time series and gaze entropy time series; determination of time features based on the current time; acquisition of a preset task stage label corresponding to the current moment; concatenation of the pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels to obtain a joint feature matrix; extraction of temporal features from the joint feature matrix to generate shared hidden features; generation of distribution parameters for cognitive load, fatigue state, and alertness state from the shared hidden features, and determination of the state values ​​of fatigue state and alertness state from the distribution parameters through reparameterized sampling; issuing a warning based on the state values ​​of fatigue state and alertness state combined with the driving scenario; and preprocessing before concatenating the pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels, including: standardization of the pupil diameter entropy time series E(t) and gaze entropy time series G(t), with the specific formula as follows: ； ;in, and These are the mean and standard deviation of the pupil diameter entropy time series, respectively. and These represent the mean and standard deviation of the gaze entropy time series, respectively; the time feature h(t) is encoded as a Fourier basis with a 24-hour period. and Generate time-coded features The specific formula is as follows: Where k represents the 1st and 2nd harmonics, and h(t) represents the time feature; pre-defined task stage labels are used for one-hot encoding to generate task feature vectors. The specific formula is as follows: ;in, v represents the number of task stages. i The i-th element in the one-hot encoded vector; the step of concatenating the pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels to obtain the joint feature matrix includes: preprocessing the... and The joint feature matrix is ​​constructed by concatenating the features along the time dimension, as shown in the following formula: ;in, The joint characteristic matrix at time t has the following shape: T is the time step. The dimension of the input feature. This is the symbol for the correlation coefficient matrix.

2. The method for dynamic early warning of brain fatigue according to claim 1, characterized in that, Temporal feature extraction is performed on the joint feature matrix using a TCN network, which includes dilated convolutional layers and is defined as follows: ;in, The shared hidden features output by the dilated convolutional layer. Let L be the input joint feature matrix, and L be the number of dilated convolutional layers.

3. The method for dynamic early warning of brain fatigue according to claim 1, characterized in that, After determining the state values ​​of fatigue and alertness, the method further includes: determining the state value of cognitive load; reconstructing the pupil diameter entropy signal and the gaze entropy signal based on the state values ​​of cognitive load, fatigue, and alertness, respectively; wherein the reconstruction formula for the pupil diameter entropy signal is: ;in, To reconstruct the generated pupil diameter entropy signal, These are cognitive load, fatigue state, and alertness state, respectively. The weight matrix for generating the pupil diameter entropy branch. This is the bias term for the pupil diameter entropy generation branch; the reconstruction formula for the gaze entropy signal is: ;in, These represent the state values ​​of cognitive load, fatigue state, and alertness state at time step t, respectively. This represents the feature vector of the task stage corresponding to time step t. Let be the hidden state of the LSTM network at time step t−1; To reconstruct the generated gaze entropy signal, The weight matrix for generating branches of gaze entropy. The bias term for the gaze entropy generation branch; according to and Determine the pupil diameter entropy signal error based on and The gaze entropy signal error is determined, and the fatigue state and alertness state are verified based on the pupil diameter entropy signal error and the gaze entropy signal error. The real-time risk index is then calculated based on the verified fatigue state and alertness state.

4. The method for dynamic early warning of brain fatigue according to claim 3, characterized in that, Before reconstructing the pupil diameter entropy signal and the gaze entropy signal, dynamic prior constraints were applied to cognitive load, fatigue state, and alertness state, including: for cognitive load Perform piecewise stability priors; for fatigue states Perform Brownian motion prior with drift ; state of alertness Perform periodic normal prior Synchronize with the 24-hour biological clock; among which, This is the preset fatigue state drift amount. This represents the standard deviation of the fatigue state fluctuation.

5. A method for dynamic early warning of brain fatigue according to any one of claims 4, characterized in that, The step of issuing a warning based on the state values ​​of fatigue and alertness combined with the driving scenario includes: generating a risk index based on the state values ​​of fatigue and alertness, using the following formula: ;in, The state value of the fatigue state at time t after verification. The verified alert state value at time t. and These are variable parameters; warnings are issued based on the risk index and the risk thresholds corresponding to different driving scenarios.

6. A dynamic early warning device for brain fatigue, characterized in that, The device is applied to the method of claim 1, and the device includes: an acquisition module for real-time acquisition of the driver's pupil diameter entropy time series and gaze entropy time series; determining time features based on the current time; a preprocessing module for concatenating the pupil diameter entropy time series, gaze entropy time series, time features, and preset task stage labels to obtain a joint feature matrix; a determination module for performing temporal feature extraction on the joint feature matrix to generate shared hidden features; generating distribution parameters of cognitive load, fatigue, and alertness state from the shared hidden features, and then determining the state values ​​of fatigue state and alertness state from the distribution parameters through reparameterized sampling; and an early warning module for issuing early warnings based on the state values ​​of fatigue state and alertness state combined with the driving scenario.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 5.

8. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 5.

Images