Mental fatigue evaluation method and system based on multi-mode nerve physiological signals

Abstract

The invention discloses a brain fatigue evaluation method and system based on multi-mode nerve physiological signals, and relates to the field of biological signal processing and neural engineering.The brain fatigue evaluation method comprises the following steps that S1, signals are collected, specifically, multi-lead electroencephalogram signals are synchronously collected when a subject executes a persistent cognitive task, the task is an AX type continuous performance task for adjusting a stimulation interval and a secondary task probability based on real-time accuracy, and the secondary task is introduced in the later period of the task to form a double-task normal form. According to the method, an AX type continuous performance task of a stimulation interval and a secondary task probability is adjusted based on real-time accuracy, a secondary task is introduced to form a double-task normal form, a set of dynamic and self-adaptive cognitive load applying mechanism is established, the task difficulty can be dynamically adjusted according to real-time performance of a subject through a closed-loop control strategy, and the cognitive load applying efficiency is improved. It is ensured that different individuals can be induced to be in a stable and controllable fatigue state, and the defect that the induction effect of a traditional fixed-time-length task is not consistent is overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biological signal processing and neural engineering, in particular to a method and system for evaluating mental fatigue based on multi-modal neuro-physiological signals. BACKGROUND

[0002] Mental fatigue is a state of reduced brain function caused by prolonged or high-intensity cognitive activities, characterized by distraction, reduced reaction inhibition, reduced working memory capacity, and slower decision-making speed. In fields such as aerospace, intelligent driving, surgery, and power dispatching, operator mental fatigue is a major potential factor leading to human error and causing major safety accidents. Therefore, developing an objective, accurate, and early warning mental fatigue evaluation system has become an urgent need in the fields of human factors engineering and neural engineering.

[0003] The existing mental fatigue evaluation based on neuro-physiological signals has the following defects: 1. Patent document CN115985464B discloses a muscle fatigue degree classification method and system based on multi-modal data fusion, "The present application relates to the field of muscle fatigue detection, especially to a muscle fatigue degree classification method and system based on multi-modal data fusion. The method comprises: acquiring multiple physiological signals and IMU signals during rehabilitation training; preprocessing the acquired physiological signals and IMU signals; converting the preprocessed physiological signals and IMU signals into grayscale images; combining the grayscale images obtained from the same signal to form an RGB image; extracting features of the physiological signals and IMU signals and their RGB images through a deep learning network model; the present application is based on the research of fatigue degree at the present stage, and a multi-input parallel neural network model is proposed to extract features based on the synchronous acquisition of sEMG, EEG, ECG, and IMU signals, to avoid errors in single signal detection", but the method and system in the above document have the technical problem of unadjustable cognitive load intensity, which cannot be adaptively adjusted according to the real-time state of the subject. SUMMARY

[0004] The present application aims to provide a method and system for evaluating mental fatigue based on multi-modal neuro-physiological signals to solve the technical problems raised in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: a method for evaluating mental fatigue based on multi-modal neuro-physiological signals, comprising the following steps: S1. Signal acquisition: Multi-lead EEG signals were simultaneously acquired during the subjects' performance of a continuous cognitive task. The task was an AX-type continuous performance task that adjusted the stimulus interval and the probability of the secondary task based on real-time accuracy. A secondary task was introduced in the later stage of the task to form a dual-task paradigm. S2. Signal preprocessing: Filtering and artifact removal are performed on the raw EEG signal; S3. Feature Extraction: Extract the first set of EEG signals representing the continuous state of the brain from the preprocessed signal, and extract multi-dimensional cognitive features reflecting attention, inhibitory control, and working memory from the event-related potential waveform as the second set of features, including correlation negative variable amplitude, No-Go N2 amplitude, No-Go P3 amplitude, P3b amplitude and latency; S4. Feature Fusion and Fatigue Assessment: The feature set is input into a two-branch neural network model containing a fully connected layer and an attention fusion module. The first branch receives EEG signal features and outputs a state fatigue score, while the second branch receives event-related potential features and outputs a cognitive fatigue score. The two scores are fused to generate a mental fatigue index and fatigue level.

[0006] Preferably, it also includes dynamic difficulty adjustment and adopts a closed-loop control strategy, which adaptively adjusts the stimulus interval and the probability of secondary tasks based on the comparison between the subject's real-time performance and the individualized baseline, so as to maintain the subject in a controllable cognitive load state.

[0007] Preferably, the first feature set of the EEG signal includes frequency domain features and nonlinear features extracted from continuous EEG signals; The nonlinear features include the sample entropy of the EEG signals in the prefrontal leads.

[0008] Preferably, in the feature fusion and fatigue evaluation step, the fusion module of the dual-branch neural network model adopts an attention mechanism to dynamically assign fusion weights to the state fatigue score and the cognitive fatigue score.

[0009] Preferably, after the feature extraction step, a feature optimization step is also included: using the maximum relevance and minimum redundancy algorithm implemented based on the scikit-learn library, setting the number of features to be retained to 12, selecting the feature subset with the greatest discriminative power to fatigue state and the minimum redundancy between each other from the first feature set of EEG signals and the second feature set of event-related potentials, and then inputting it into the dual-branch neural network model.

[0010] Preferably, it also includes an individualized baseline calibration step: before the formal task begins, the subject's EEG signals and event-related potential data are collected in the resting state and under the baseline task, and the extracted features are used as individual benchmarks to normalize the features extracted in subsequent tasks.

[0011] Preferably, a multimodal neurophysiological signal-based mental fatigue assessment system includes: The EEG signal acquisition module is used to acquire raw EEG signals; The stimulus presentation and task control module is used to present continuous cognitive tasks and has an integrated adaptive control unit for performing the dynamic difficulty adjustment based on the closed-loop control strategy. A signal processing and feature extraction module is used to perform preprocessing and feature extraction. This module is configured to calculate nonlinear features of EEG signals and perform the feature optimization steps. The fatigue assessment model module includes a dual-branch neural network, which internally stores and runs a dual-branch neural network model equipped with an attention mechanism fusion module to calculate the fatigue index and level. The results display and early warning module is used to output evaluation results and issue an alarm when the threshold is reached.

[0012] Preferably, the adaptive control unit in the stimulus presentation and task control module is configured to synchronously adjust the task difficulty in two dimensions: stimulus interval and secondary task probability, based on real-time calculated behavioral performance indicators.

[0013] Preferably, the signal processing and feature extraction module integrates an individualized baseline calibration unit, which stores individual baseline features and performs online normalization processing on the features extracted in real time.

[0014] Preferably, the result display and early warning module outputs the overall mental fatigue index, and simultaneously outputs the state fatigue score and cognitive fatigue score to provide decompositional diagnostic information about the source of fatigue.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention establishes a dynamic and adaptive cognitive load application mechanism by adjusting the stimulus interval and the probability of the secondary task based on real-time accuracy of the AX-type continuous performance task and introducing the secondary task to form a dual-task paradigm. This closed-loop control strategy can dynamically adjust the task difficulty according to the real-time performance of the subjects, ensuring that different individuals can be induced to a stable and controllable fatigue state. It overcomes the defects of inconsistent induced effects of traditional fixed-duration tasks and provides a high-quality data foundation for subsequent accurate evaluation. 2. This invention integrates EEG signal features characterizing the brain's persistent background state with event-related potential features characterizing specific cognitive functions (including negative correlation variables reflecting working memory, N2 for conflict monitoring, P3 for response inhibition, and P3b for attention allocation) to construct a multi-level feature evaluation system. This system can accurately locate the impact of fatigue from multiple independent cognitive subprocesses such as attention, inhibitory control, and working memory. The evaluation results not only reflect "whether there is fatigue" but also reveal "what kind of cognitive function declines," greatly improving the interpretability and diagnostic value of the results. 3. This invention uses a dual-branch neural network model and attention mechanism to fuse model-level features. This invention can adaptively weigh the differential contributions of state indicators and cognitive function indicators to fatigue state. Among them, the event-related potential indicator, which is extremely sensitive to fluctuations in cognitive resources, can detect the decline in neural efficiency before the brain's behavioral performance shows obvious decline, thereby achieving early warning. This method fundamentally overcomes the inherent defects of subjective evaluation being lagging and biased, as well as the one-sidedness and poor robustness of single physiological indicators. 4. The system provided by this invention integrates the entire process from signal acquisition, task control, data processing to model evaluation and result output, forming a complete automated closed loop. By introducing individualized baseline calibration and feature optimization steps, the system can effectively eliminate physiological differences between individuals and improve the generalization ability among different users. At the same time, the system outputs comprehensive index and decomposed diagnostic information in parallel, which not only provides an overall evaluation but also indicates the source of fatigue, providing direct and powerful decision support for subsequent personalized interventions (such as rest or task switching). It has broad prospects for industrial transformation. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall process of the method of the present invention; Figure 2 This is a schematic diagram of the system structure of the present invention; Figure 3 This is a schematic diagram of the dynamic AX-type continuous performance task flow of the present invention; Figure 4 This is a schematic diagram of the dual-branch neural network architecture of the present invention; Figure 5 This is a schematic diagram of the individualized baseline calibration and feature optimization process of the present invention; Figure 6 This is a flowchart of the system early warning and result display of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: Please refer to Figure 1 , Figure 2 , Figure 3 and Figure 4 A method and system for evaluating mental fatigue based on multimodal neurophysiological signals: This embodiment uses driver mental fatigue monitoring as an example to illustrate the basic implementation of the present invention: 1. Specific implementation of the system The mental fatigue assessment system includes the following modules: (1) EEG signal acquisition module: The 32-lead ActiveTwo system (BioSemi) was used, the sampling rate was set to 1000Hz, the electrode arrangement followed the international 10-20 system, and the signals of the prefrontal lobe (Fp1, Fp2, Fz, F3, F4), central area (Cz, C3, C4) and parietal lobe (Pz, P3, P4) were collected. The electrode impedance was kept below 10kΩ. (2) Stimulus presentation and task control module: Implemented using a computer with MATLAB Psychtoolbox installed, equipped with a 21-inch LCD display for presenting dynamic AX-type continuous performance tasks. This module has a built-in adaptive control unit for performing dynamic difficulty adjustment based on closed-loop control strategy. (3) Signal processing and feature extraction module: The workstation is equipped with an Intel Core i7 processor and 16GB of memory, and runs a signal processing program based on the EEGLAB toolbox and a custom MATLAB algorithm. This module is configured to calculate the nonlinear features of EEG signals. (4) Fatigue evaluation model module: It embeds a dual-branch neural network model developed based on Python 3.8 and TensorFlow 2.4 framework. The model is built using Keras high-level API and includes a 3-layer fully connected network structure and an additive attention fusion module. (5) Result display and early warning module: integrates a voice broadcast system and a visual display interface. The display interface includes a fatigue index display dashboard (0-100 scale), a trend change curve, and three-color early warning status indicator lights (green, yellow, and red); 2. Specific Implementation of the Method The method for assessing mental fatigue includes the following steps: S1. Signal acquisition steps: Multi-lead EEG signals were simultaneously acquired during the subjects' performance of a continuous cognitive task. The task was an AX-type continuous performance task with dynamic difficulty adjustment, with a total duration of 60 minutes. The basic parameters of the task were: fixation presentation 500ms, cue stimulus presentation 500ms, delay period 1500ms, probe stimulus presentation 1000ms. At the 30-minute mark, digit parity judgment was introduced as a secondary task, forming a dual-task paradigm. The specific rules of the AX-type continuous performance task are as follows: After fixation and cue stimulation, when the probe stimulus is the letter "X" and the previous cue stimulus is "A" (i.e., AX sequence), the subject is required to press the "F" key on the keyboard with the right index finger as the target response. For all other cue-probe combinations (such as AY, BX, BY), the subject is required to suppress the key press response. The system collects behavioral accuracy and reaction time data by recording the pressing of the "F" key. S2. Signal preprocessing steps: The acquired raw EEG signals are processed as follows: (1) Use 0.5Hz high-pass filter and 45Hz low-pass filter to remove low-frequency drift and high-frequency noise; (2) Perform 50Hz notch filtering to eliminate power frequency interference; (3) The independent component analysis method is used to automatically identify and remove physiological artifacts such as electrooculography and electromyography; (4) Baseline correction is performed on the signal, with the first 200ms of stimulation as the baseline.

[0019] S3. Feature extraction step: Extract the following feature set from the preprocessed signal: (1) First feature set of EEG signals: Extracted from continuous EEG signals Frequency domain characteristics: The power spectral density was calculated using the Welch method, and the relative power in the Theta band (4-7Hz) of the prefrontal cortex and the Alpha band (8-13Hz) of the parietal-occipital region was extracted. Relative power was defined as the ratio of the power in a specific frequency band to the total power across the entire frequency band (1-45Hz). Nonlinear characteristics: Calculate the sample entropy of the signals in leads Fz, F3, and F4 of the prefrontal cortex, with parameters set to m=2 and r=0.2×signal standard deviation.

[0020] (2) Second feature set of event-related potentials: extracted from the event-related potential waveforms induced by the task. Relevance negative variable amplitude: the average amplitude within a time window of 500-1500ms after cue stimulus presentation; No-Go N2 amplitude: the peak amplitude within a time window of 200-300ms after the presentation of the No-Go stimulus; No-Go P3 amplitude: the peak amplitude within a time window of 300-500ms after the presentation of the No-Go stimulus; P3b amplitude and latency: the peak amplitude and corresponding latency within a time window of 300-600ms after the presentation of the target stimulus; S4. Feature Fusion and Fatigue Evaluation Steps Input the feature set into the two-branch neural network model: (1) The first branch of the model receives the first feature set of the EEG signal, passes through a 3-layer fully connected neural network (64, 32 and 16 nodes respectively), and outputs a state fatigue score (range 0-1). (2) The second branch of the model receives the second feature set of event-related potentials, passes through a 3-layer fully connected neural network (with 64, 32 and 16 nodes respectively), and outputs a cognitive fatigue score (range 0-1). (3) The model's fusion module adopts an additive attention mechanism to dynamically allocate fusion weights to the state fatigue score and the cognitive fatigue score. The attention weights are calculated by the output features of the penultimate layer of the two branch networks through a fully connected layer and a softmax function. (4) The weighted scores are mapped to generate a mental fatigue index of 0-100 through the sigmoid function, and divided into three fatigue levels according to the preset threshold: normal (0-30), fatigue (30-60), and severe fatigue (60-100). The dynamic difficulty adjustment adopts a closed-loop control strategy based on the proportional-integral (PI) algorithm: the system monitors the subject's task accuracy in real time. When the accuracy of the most recent 20 trials is higher than 85%, the stimulus interval is shortened by 100ms or the probability of the secondary task is increased by 10%. When the accuracy is lower than 70%, the stimulus interval is extended by 100ms or the probability of the secondary task is reduced by 10%. The control parameters are adaptively adjusted according to the accuracy deviation to maintain the subject in a controllable cognitive load state.

[0021] Example 2: Please refer to Figure 5 A method and system for evaluating mental fatigue based on multimodal neurophysiological signals: This embodiment, based on Embodiment 1, focuses on explaining the specific implementation of the feature optimization step: System supplementary implementation Based on the system in Example 1, the signal processing and feature extraction module is further configured with a feature selection unit, which integrates the feature selection function based on the maximum correlation minimum redundancy algorithm. This unit uses the SelectKBest function of the Python scikit-learn library combined with mutual information scoring to implement the maximum correlation minimum redundancy algorithm. Supplementary implementation of the method Based on the method of Example 1, a feature optimization step is added after the feature extraction step S3: An algorithm based on maximum correlation and minimum redundancy is used to select the feature subset with the highest discriminative power and minimum redundancy from the first feature set of EEG signals and the second feature set of event-related potentials. The specific implementation process is as follows: (1) Calculate the mutual information between each feature and the fatigue state label (based on the binarization of the NASA-TLX subjective scores of the same period) as a measure of the correlation between the feature and the target; (2) Calculate the mutual information between each pair of features as a measure of feature redundancy; (3) Iteratively select features using the maximum relevance and minimum redundancy criterion: In the first round, select the feature with the highest relevance to the target; in subsequent rounds, select the feature that maximizes the following expression:

[0022] in As candidate features, For the target variable, For the selected feature set, Represents mutual information, For the selected feature set The number of features contained therein For the selected feature set Summing all features (4) Set the number of features to be retained to 12 to form the optimal feature subset; (5) Input the optimized feature subset into the dual-branch neural network model for further processing.

[0023] Example 3: Please refer to Figure 5 A method and system for evaluating mental fatigue based on multimodal neurophysiological signals: This embodiment, based on Embodiment 1, focuses on illustrating the specific implementation of individualized baseline calibration: 1. Supplementary Implementation of the System Based on the system in Example 1, the signal processing and feature extraction module integrates an individualized baseline calibration unit. This unit is equipped with a dedicated storage area for storing individual baseline features and has real-time normalization calculation capabilities. 2. Supplementary Implementation of the Method Based on the method in Example 1, an individualized baseline calibration step is added before the formal task begins: (1) Resting state data acquisition: Subjects were asked to sit quietly with their eyes closed for 5 minutes to collect EEG signals in the resting state; (2) Baseline task execution: Execute the standard AX type continuous performance task for 10 minutes with fixed task parameters (stimulation interval fixed at 2000ms, no secondary task). (3) Baseline feature extraction: Extract the same set of features as the formal task from the resting state and baseline task data; (4) Baseline establishment: Calculate the mean and standard deviation of each feature and establish an individual feature baseline database; (5) Real-time normalization: During the formal task, the features extracted in real time are normalized using Z-score based on the individual baseline.

[0024] in These are the original eigenvalues. and These are the mean and standard deviation of the feature at the baseline period, respectively.

[0025] Example 4: Please refer to Figure 6 A method and system for evaluating mental fatigue based on multimodal neurophysiological signals: This embodiment focuses on illustrating the optimization implementation details of each module of the system: 1. System optimization implementation (1) The adaptive control unit in the stimulus presentation and task control module is configured to adjust the task difficulty in two dimensions, stimulus interval and secondary task probability, in real time according to the behavioral performance indicators calculated in real time. The specific implementation includes: Establish a two-dimensional difficulty adjustment mechanism: the stimulus interval is adjustable from 800 to 2000 ms, and the probability of secondary tasks occurring is adjusted from 5% to 40%. Set up a performance evaluation window: use the performance of the most recent 20 trials as the basis for difficulty adjustment; To achieve a smooth transition: the difficulty level is adjusted gradually, with each adjustment not exceeding 20% ​​of the current value, to avoid sudden changes interfering with the participants; (2) The results display and early warning module outputs the overall mental fatigue index, and simultaneously outputs the state fatigue score and cognitive fatigue score to provide decomposition and diagnostic information about the source of fatigue.

[0026] The specific implementation includes: Design a multi-dimensional information display interface: simultaneously display the comprehensive index, state score, cognitive score, and their respective levels; Establish a tiered early warning mechanism: when the state score exceeds the baseline value by 2 standard deviations and the cognitive score is lower than the baseline value by 1 standard deviation, prompt "State fatigue, rest is recommended"; When the cognitive score exceeds the baseline value by 2 standard deviations and the state score is lower than the baseline value by 1 standard deviation, the message "Cognitive fatigue, it is recommended to switch tasks" is displayed. When both exceed the baseline value by 2 standard deviations, it indicates "severe fatigue, rest immediately"; Provide targeted recommendations: Based on the ratio between state scores and cognitive scores, provide personalized intervention recommendations; 2. Corresponding Implementation of the Method Implement the corresponding methods: (1) Dynamic adjustment of difficulty in two dimensions, and more precise control of cognitive load based on multi-dimensional performance indicators; (2) Multi-level information output provides users with a more comprehensive fatigue state analysis; (3) Personalized early warning strategy: adopt targeted early warning methods according to different fatigue patterns.

[0027] Technical effect verification To verify the technical effect of the present invention, we organized 50 healthy adult subjects (aged 25-45 years, half male and half female) to conduct system tests. The tests were conducted in a standard laboratory environment (soundproof room, temperature 25±2℃, humidity 50±10%). The equipment used was a 32-lead EEG acquisition system. Each person completed two 60-minute dynamic AX-type continuous performance tasks. The Adam optimizer and cross-entropy loss function were used to train the model. The training cycle was 100 rounds, and the batch size was 32. The verification results show: Accuracy assessment: The correlation coefficient between the mental fatigue index calculated by this invention and the Pearson correlation coefficient of the NASA-TLX subjective score was tested using a two-tailed t-test. The significance level was p < 0.01, and the correlation coefficient reached 0.87 (p < 0.01), which was significantly higher than that of the single EEG signal feature method (r = 0.62) and the single event-related potential feature method (r = 0.58). Early warning capability: Based on a significant increase in behavioral reaction time (more than 2 standard deviations from the baseline), this invention can detect fatigue 18±3 minutes earlier than traditional behavioral indicators; Individual adaptability: By comparing the intragroup correlation coefficients predicted by the model before and after individualized calibration, it was confirmed that the inter-individual differences were reduced by 35% after individualized baseline calibration; System stability: The system operates continuously for 12 hours with a 100% failure rate, and all modules work together stably and reliably.

[0028] The working principle involves an AX-type continuous performance task that adjusts the stimulus interval and the probability of secondary tasks based on real-time accuracy. This dual-task paradigm, incorporating secondary tasks, establishes a dynamic and adaptive cognitive load application mechanism. This closed-loop control strategy dynamically adjusts task difficulty based on the subject's real-time performance, ensuring that different individuals can be induced to a stable and controllable state of fatigue. This overcomes the inconsistent effects of traditional fixed-duration tasks, providing a high-quality data foundation for subsequent accurate evaluation. By integrating EEG signal features representing the brain's continuous background state with event-related potential features representing specific cognitive functions (including negative correlation reflecting working memory, N2 for conflict monitoring, P3 for response inhibition, and P3b for attention allocation), a multi-level feature evaluation system is constructed. This allows for precise localization of the impact of fatigue from multiple independent cognitive subprocesses such as attention, inhibitory control, and working memory. The evaluation results not only reflect "whether fatigue exists" but also reveal "what cognitive function declines," greatly enhancing the interpretability of the results. This invention, through the use of a dual-branch neural network model and attention mechanism for model-level feature fusion, adaptively balances the differentiated contributions of state indicators and cognitive function indicators to fatigue. Event-related potential (ERP) indicators, which are highly sensitive to fluctuations in cognitive resources, can detect a decline in neural efficiency before a significant decline in brain behavior, thus providing early warning. This method fundamentally overcomes the inherent defects of subjective evaluation (lagging and biased) and the one-sidedness and poor robustness of single physiological indicators. The system integrates the entire process from signal acquisition, task control, data processing to model evaluation and result output, forming a complete automated closed loop. By introducing individualized baseline calibration and feature optimization steps, the system can effectively eliminate physiological differences between individuals and improve generalization ability among different users. Simultaneously, the system outputs a comprehensive index and decomposed diagnostic information in parallel, providing not only an overall evaluation but also indicating the source of fatigue, offering direct and powerful decision support for subsequent personalized interventions (such as rest or task switching). The system has broad prospects for industrial transformation.

[0029] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

Claims

1. A method for evaluating mental fatigue of a multi-modal neuro-physiological signal, characterized in that: Comprise the following steps: S1, signal acquisition: synchronously collect multi-lead electroencephalogram signals during the subject performs a sustained cognitive task, the task is an AX type continuous performance task based on real-time accuracy adjustment of stimulus interval and secondary task probability, a secondary task is introduced in the later stage of the task to constitute a dual task paradigm; S2, signal preprocessing: filtering and artifact removal are performed on the original electroencephalogram signals; S3, feature extraction: a first feature set of electroencephalogram signals representing the sustained state of the brain is extracted from the preprocessed signals, and a second feature set reflecting attention, inhibition control, and working memory is extracted from the event-related potential waveform as a second feature set, including correlation negative wave amplitude, No-Go N2 wave amplitude, No-Go P3 wave amplitude, P3b wave amplitude, and latency; S4, feature fusion and fatigue evaluation: input the feature set into a double-branch neural network model, the first branch receives the electroencephalogram signal feature to output a state fatigue score, the second branch receives the event-related potential feature to output a cognitive fatigue score, and the two scores are fused to generate a mental fatigue index and a fatigue grade. 2.The brain fatigue evaluation method of multi-modal neuro-physiological signals according to claim 1, characterized in that: It also includes dynamic difficulty adjustment and adopts a closed-loop control strategy, which adaptively adjusts the stimulus interval and secondary task probability based on the comparison of the real-time performance of the subject and the individualized baseline to maintain the subject in a controllable cognitive load state. 3.The brain fatigue evaluation method of multi-modal neuro-physiological signals according to claim 1, characterized in that: The first feature set of electroencephalogram signals includes frequency domain features and nonlinear features extracted from continuous electroencephalogram signals; The nonlinear features include sample entropy of frontal lobe lead electroencephalogram signals. 4.The brain fatigue evaluation method of multi-modal neuro-physiological signals according to claim 1, characterized in that: In the feature fusion and fatigue evaluation step, the fusion module of the double-branch neural network model adopts an attention mechanism to dynamically allocate fusion weights to the state fatigue score and the cognitive fatigue score.

5. The method of claim 1, wherein: After the feature extraction step, a feature optimization step is further included: using the maximum relevance minimum redundancy algorithm based on the scikit-learn library, setting the feature retention number to 12, selecting a feature subset with the highest discriminability and the smallest redundancy from the first feature set of electroencephalogram signals and the second feature set of event-related potentials, and then inputting it to the double-branch neural network model.

6. The method of claim 1, wherein: It also includes an individualized baseline calibration step: before the formal task starts, collect the electroencephalogram signals and event-related potential data of the subject under the resting state and the baseline task, and use the extracted features as individual benchmarks for normalizing the features extracted in subsequent tasks.

7. A mental fatigue evaluation system of multi-modal neuro-physiological signals, adapted to the mental fatigue evaluation method of multi-modal neuro-physiological signals according to claims 1-6, characterized in that: Comprise: An electroencephalogram signal acquisition module for acquiring raw electroencephalogram signals; A stimulus presentation and task control module for presenting a sustained cognitive task, and an adaptive control unit is built in for performing dynamic difficulty adjustment based on the closed-loop control strategy; A signal processing and feature extraction module for performing preprocessing and feature extraction, which is configured to calculate electroencephalogram nonlinear features and perform the feature optimization step; A fatigue evaluation model module including a double-branch neural network, which internally stores and runs a double-branch neural network model equipped with an attention mechanism fusion module for calculating fatigue index and grade; A result display and warning module for outputting evaluation results and issuing an alarm when a threshold is reached.

8. The mental fatigue evaluation system of multi-modal neuro-physiological signals according to claim 7, characterized in that: The stimulation presentation and task control module comprises an adaptive control unit configured to synchronously adjust the task difficulty of the two dimensions of the stimulus interval and the secondary task probability according to the real-time calculated behavioral performance indicators.

9. The mental fatigue evaluation system of multi-modal neuro-physiological signals according to claim 7, characterized in that: The signal processing and feature extraction module is integrated with an individualized baseline calibration unit which stores individual baseline features and performs online normalization processing on the real-time extracted features.

10. The mental fatigue evaluation system of multi-modal neuro-physiological signals according to claim 7, characterized in that: The result display and early warning module outputs the comprehensive mental fatigue index, and simultaneously outputs the state fatigue score and the cognitive fatigue score.

Images