Fatigue state detection method, system and device and storage medium

Abstract

The invention discloses a fatigue state detection method, system and device and a storage medium, and relates to the technical field of fatigue detection.The method comprises the steps that electroencephalogram, electrocardio, myoelectricity and blood oxygen physiological signals of a user are collected; a Morlet wavelet transform method is adopted to extract time-frequency characteristics in the electroencephalogram signals; the method comprises the following steps: calculating a heart rate signal and heart rate variability by detecting the position of an R wave in an electrocardiosignal, and extracting a time sequence characteristic of the heart rate signal and a heart rate variability characteristic in the heart rate variability; utilizing an LSTM network to respectively extract electromyographic activity features and blood oxygen level features in the electromyographic signals and the blood oxygen signals; inputting the plurality of extracted features into a feature fusion network for feature fusion; the fused features are converted into fatigue indexes through a regression network; detecting the fatigue state of the user according to the fatigue index; by integrating a plurality of physiological signals, real-time monitoring and accurate evaluation of the fatigue state of the pilot are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue detection, and particularly relates to a fatigue state detection method, system, device and storage medium. Background Art

[0002] In civil aviation flight missions, the high concentration and precise decision-making of pilots are the keys to ensuring flight safety and mission success. However, factors such as long flights, night operations, flights in different time zones, and ever-changing meteorological conditions can all have a negative impact on the physiological and psychological states of pilots, leading to the accumulation of fatigue. Excessive fatigue not only reduces cognitive and executive abilities, but may also trigger wrong judgments and reactions, thus posing a potential threat to the safety of flight missions.

[0003] Traditional pilot fatigue monitoring methods include: using single physiological information detection, such as heart rate detection, to evaluate the fatigue level of pilots by monitoring their heart rates; for example, a pilot fatigue detection method based on electroencephalogram signals proposed in the invention patent CN108272463A, which uses a deep contractive autoencoder network to learn electroencephalogram signals to identify the fatigue state of pilots; using multi-physiological signal fusion detection, such as fusing the electroencephalogram, electrocardiogram, electromyogram, and blood oxygen physiological signals collected from users to detect the fatigue degree.

[0004] However, during the process of using multi-physiological signal fusion detection, the detection of heart rate signals only evaluates the fatigue level of pilots by simply monitoring their heart rates, resulting in a low accuracy rate for current fatigue detection of pilots. Summary of the Invention

[0005] Aiming at the deficiency that in the prior art, during the process of using multi-physiological signal fusion detection, the detection of heart rate signals only evaluates the fatigue level of pilots by simply monitoring their heart rates, resulting in a low accuracy rate for current fatigue detection, the present invention provides a fatigue state detection method, system, device and storage medium. By using multi-source physiological information such as electroencephalogram, electrocardiogram, electromyogram, and blood oxygen, it comprehensively reflects the physiological state of users. At the same time, a deep learning neural network is used for feature extraction and fusion, and high-dimensional feature representations are automatically learned and extracted, thus solving the problems existing in the prior art.

[0006] A fatigue state detection method includes the following steps:

[0007] Collect the electroencephalogram signal, electrocardiogram signal, electromyogram signal and blood oxygen physiological signal of the user, and extract the time-frequency features, electromyogram activity features and blood oxygen level features of the electroencephalogram signal;

[0008] By extracting the heartbeat timestamps in the electrocardiogram (ECG) signal, the positions of the R waves in the ECG signal are obtained; the heart rate signal and its heart rate variability are calculated based on the time intervals between adjacent R waves, and the temporal features of the heart rate signal and the heart rate variability features in the heart rate variability are extracted through an LSTM network;

[0009] The time-frequency features of the extracted electroencephalogram (EEG) signal, the temporal features of the heart rate signal, the heart rate variability features, the electromyogram (EMG) activity features, and the blood oxygen level features are subjected to feature fusion through a channel attention network;

[0010] The fatigue state of the user is detected based on the fused features.

[0011] Further, after collecting the user's EEG signal, the EEG signal is analyzed by using a band-pass filter of 0.1 - 45 Hz and independent component analysis (ICA) to remove the electrooculogram (EOG), ECG, and EMG components in the EEG signal; after collecting the user's ECG signal, the signal is filtered and baseline drift is removed; after collecting the EMG signal and blood oxygen signal, the signals are filtered and denoised respectively.

[0012] Further, it also includes using the Morlet wavelet transform method to convert the EEG signal from the time domain to the frequency domain to generate a time-frequency diagram, which specifically includes the following steps:

[0013] Time-frequency analysis is realized through the Morlet wavelet transform, the EEG signal is converted from the time domain to the frequency domain, and the energy information in time and frequency is provided, and the corresponding time-frequency diagram is generated;

[0014] Define the input EEG signal as x(t), the Morlet wavelet as ψ(t), and its frequency as f, then the Morlet wavelet transform X f (τ) is expressed as:

[0015]

[0016] where ψ f,τ (t) is the form of the Morlet wavelet at frequency f and time shift τ, which is defined as:

[0017]

[0018] where, where A is the normalization coefficient, f 0 is the central frequency of the wavelet, σ is the parameter controlling the wavelet width, and i is the imaginary unit.

[0019] Further, a convolutional neural network (CNN) is used to extract the time-frequency features of the EEG signal, which specifically includes the following steps:

[0020] When using a convolutional layer to capture the local features of the time-frequency map, it perceives the EEG activities at different frequency bands and time points;

[0021] The captured feature map is dimensionally reduced through a pooling layer to obtain the high-dimensional features of the EEG.

[0022] Furthermore, calculating the heart rate signal and its heart rate variability according to the time interval between adjacent R waves specifically includes the following steps:

[0023] Extract the heartbeat timestamps in the ECG signal, that is, the R-R interval; where the R wave corresponds to each contraction of the heart;

[0024] Use the sliding window method to detect the R wave in the ECG signal;

[0025] Calculate the heart rate HR according to the time interval between adjacent R waves, which is expressed as:

[0026]

[0027] where Δt is the time interval and N is the number of heartbeats per unit time;

[0028] By calculating the time interval between R waves, a heart rate time series is obtained, and the heart rate variability is calculated according to the heart rate time series; the heart rate variability parameters include the standard deviation SDNN, the standard deviation of adjacent RR intervals SDRR, and the root mean square difference RMSSD, which are expressed as:

[0029]

[0030] where t represents the time series index.

[0031] Furthermore, fusing the time-frequency features of the extracted EEG signal, the temporal features of the heart rate signal, the heart rate variability features, the EMG activity features, and the blood oxygen level features through a channel attention network specifically includes the following steps:

[0032] Represent the time-frequency features of the EEG signal, the temporal features of the heart rate, the heart rate variability features, the EMG features, and the blood oxygen level features with x 1 ,x 2 ,x 3 ,x 4 ,x 5 respectively;

[0033] Fuse x 1 ,x 2 ,x 3 ,x 4 ,x 5 through a channel attention network for feature fusion; let the attention weight of feature x i be αi Then the fused feature h is expressed as:

[0034]

[0035] where z i = W i ·x i + b i and z i is a linear combination of the feature x i , W i are the weight parameters of the attention network, and b i is the bias term. represents the exponential function.

[0036] Furthermore, before detecting the fatigue state of the user according to the fused feature, the fused feature is converted into a fatigue index, and the fatigue index is expressed as:

[0037]

[0038] where f(·) is the mapping function of the regression network, and the regression network is a multi-layer perceptron which includes multiple hidden layers and activation functions.

[0039] The present invention also includes a fatigue state detection system, comprising:

[0040] An acquisition module, configured to acquire the electroencephalogram signal, electrocardiogram signal, electromyogram signal and blood oxygen physiological signal of the user, and extract the time-frequency feature of the electroencephalogram signal, the electromyogram activity feature and the blood oxygen level feature;

[0041] A feature extraction module, configured to obtain the position of the R wave in the electrocardiogram signal by extracting the heartbeat timestamp in the electrocardiogram signal; calculate the heart rate signal and its heart rate variability according to the time interval between adjacent R waves, and extract the time series feature of the heart rate signal and the heart rate variability feature in the heart rate variability through the LSTM network;

[0042] A feature fusion module, configured to fuse the time-frequency feature of the extracted electroencephalogram signal, the time series feature of the heart rate signal, the heart rate variability feature, the electromyogram activity feature and the blood oxygen level feature through a channel attention network;

[0043] A detection module, configured to detect the fatigue state of the user according to the fused feature.

[0044] The present invention also includes a fatigue state detection computer device, comprising: a memory, a processor, and a computer program stored in the memory, and when the processor executes the computer program, the steps of the fatigue state detection method are implemented.

[0045] The present invention further includes a readable storage medium storing a computer program, the computer program including program instructions which, when executed by a processor, are used to execute the steps of the fatigue state detection method described above.

[0046] The present invention provides a fatigue state detection method, system, device and storage medium, having the following

[0047] Beneficial effects:

[0048] By extracting the time-frequency features of electroencephalogram (EEG) signals, the present invention can effectively capture local features and sense EEG activities at different frequency bands and time points; by detecting the position of R waves in electrocardiogram (ECG) signals to calculate the heart rate signal and its heart rate variability, the rhythm and frequency of heartbeats can be captured, and by extracting the time series features of the heart rate signal and the heart rate variability features in the heart rate variability, the changes in heart activities can be effectively captured; by extracting the electromyogram (EMG) activity features and blood oxygen level features in EMG signals and blood oxygen signals, the time series dynamics of the signals are effectively retained; by fusing multiple different features, high-dimensional feature representations are automatically learned and extracted, thereby improving the real-time monitoring and accurate assessment of the fatigue state of pilots. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of a multi-source physiological information fatigue detection framework in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0051] The present invention proposes a fatigue detection method, which extracts and fuses features by using multi-source physiological information such as EEG, ECG, EMG, and blood oxygen, in combination with a deep learning neural network, to output the fatigue index of a pilot. This method uses the Morlet wavelet transform method to extract time-frequency features from EEG signals, and the Morlet wavelet transform X f (τ) is expressed as: According to the position of R waves in the ECG signal, the heart rate and heart rate variability are calculated from the ECG signal, and Calculate the heart rate and simultaneously calculate the heart rate variability related parameters, including the standard deviation of NN (SDNN), the standard deviation of adjacent RR intervals (SDRR), and the root mean square difference (RMSSD). Extract the electromyogram activity characteristics from the electromyogram signal and the blood oxygen level characteristics from the blood oxygen signal. Input the characteristics of these five physiological information into the neural network for feature extraction and fusion. Add a personalized attention module during the fusion process to dynamically adjust the attention weights of the features for different civil aviation pilots to obtain a personalized high-dimensional feature representation. Through a regression algorithm, convert the high-dimensional feature representation into the fatigue index of the pilot; the method specifically includes the following steps:

[0052] S1. Data acquisition: (1) For the electroencephalogram signal, first preprocess the collected 64-channel electroencephalogram data, including filtering and independent component analysis (ICA), to remove the noise from the environment, physiological artifacts, and other electromagnetic interferences to obtain a purer electroencephalogram signal. The filtering uses a band-pass filter of 0.1 - 45 Hz, and ICA analyzes the electroencephalogram signal to delete the electrooculogram, electrocardiogram, and electromyogram components in the electroencephalogram. Perform wavelet transform on the preprocessed signal. The Morlet wavelet is used, which is a complex wavelet with a Gaussian-shaped real part and a sine-shaped imaginary part and is suitable for capturing the time-frequency characteristics of the signal. The input preprocessed electroencephalogram signal is x(t), and the Morlet wavelet is ψ(t), with its frequency being f. Then the Morlet wavelet transform X f (τ) is expressed as:

[0053]

[0054] where ψ f,τ (t) is the form of the Morlet wavelet at the frequency f and time translation τ, defined as:

[0055]

[0056] Here, ψ(t) is the basic Morlet wavelet, defined as: where A is the normalization coefficient, f 0 is the central frequency of the wavelet, and σ is the parameter controlling the wavelet width.

[0057] Use the Morlet wavelet transform to achieve time-frequency analysis, convert the signal from the time domain to the frequency domain, and provide the energy information in time and frequency, and generate the time-frequency diagrams corresponding to 64 electrodes. The time-frequency diagrams reflect the electroencephalogram energy values at different times and different frequencies.

[0058] After obtaining the time-frequency maps corresponding to 64 electrodes, a convolutional neural network (CNN) is used to further extract features from the EEG time-frequency maps. The convolutional layer can effectively capture local features and perceive EEG activities at different frequency bands and time points, while the pooling layer helps to reduce the dimensionality of the feature map, reduce computational complexity, and retain key information, finally obtaining high-dimensional features of the EEG.

[0059] (2) For electrocardiogram signals, first, through filtering, removing baseline drift, etc., to ensure accurate and reliable electrocardiogram signals. The preprocessed electrocardiogram signals are divided into multiple small segments of data, and the heart rate and heart rate variability are obtained by calculating each small segment of the electrocardiogram signal.

[0060] Through reasonable analysis of the electrocardiogram signals, the present invention can calculate the frequency of heartbeats, that is, the heart rate. Heartbeat timestamps, that is, R-R intervals, are extracted from the preprocessed electrocardiogram signals. This step is achieved by detecting the positions of R waves in the electrocardiogram signals, and the R wave corresponds to each contraction of the heart. By calculating the time intervals between adjacent R waves, a heart rate time series is obtained. The calculation of the heart rate involves analyzing the electrocardiogram signals in the time domain to capture the rhythm and frequency of heartbeats. The sliding window method is used to detect R waves in the electrocardiogram signals, and the R wave corresponds to each contraction of the heart. The heart rate is calculated based on the time intervals between adjacent R waves. The formula for calculating the heart rate is:

[0061]

[0062] where Δt is the time interval (for example, within one minute), and N is the number of heartbeats per unit time. The heart rate signal is input into a long short-term memory network (LSTM) for feature extraction. LSTM is a recurrent neural network suitable for processing time series data. The LSTM network is used to capture the temporal information in the heart rate signal, including the fluctuations and trends of the heart rate. The output of the LSTM is a high-dimensional feature representation after multiple time steps, which contains the dynamic information in the heart rate signal. This feature representation has strong expressive ability and can effectively capture the changes in heart activities.

[0063] Heart rate variability (HRV) refers to the variation in heartbeat intervals and can reveal information about cardiac regulatory function and overall health status. Heart rate variability is calculated from the heart rate time series, and three heart rate variability parameters, namely the standard deviation of all normal-to-normal intervals (SDNN), the standard deviation of adjacent RR intervals (SDRR), and the root mean square of successive differences (RMSSD), are focused on. Among them, the standard deviation of all normal-to-normal intervals (SDNN), that is, the standard deviation of all heartbeat intervals, is calculated by the following formula:

[0064]

[0065] The standard deviation of adjacent RR intervals (SDRR) is the standard deviation of the differences between adjacent heartbeat intervals and is calculated by the following formula:

[0066]

[0067] The root mean square of successive differences (RMSSD) is the root mean square value of the differences between adjacent RR intervals and is calculated by the formula:

[0068]

[0069] In the feature extraction stage, the present invention inputs the three parameter time series of the calculated heart rate variability into the LSTM model, extracts the key features in the heart rate variability sequence, and represents them as high-dimensional feature vectors, such as calculating the heart rate and heart rate variability for a segment of electrocardiogram signals in Table 1.

[0070] Table 1 Calculating heart rate and heart rate variability for a segment of electrocardiogram signals

[0071] Heart rate SDNN SDRR RMSSD 75 0.028 0.028 0.014

[0072] (3) For electromyography signals, after filtering and denoising, the signals are input into a long short-term memory network (LSTM) to better understand the time-domain characteristics of electromyographic activities, including actions such as muscle contraction and relaxation. In the LSTM network, the present invention trains the network to learn and extract the key features in the electromyography signals. Through iterations of multiple time steps, the LSTM network can capture the long-term dependencies in the electromyography signals and effectively retain the temporal dynamics of the signals. The output of this step is the high-dimensional feature representation of the LSTM network after training, which contains an abstract expression of the electromyographic activities.

[0073] (4) For blood oxygen information, after filtering and denoising in the same way, the blood oxygen signal is input into a long short-term memory network (LSTM) for feature extraction to better capture the temporal changes in blood oxygen levels and the dynamic features related to the fatigue state. The LSTM network outputs the high-dimensional feature representation of the blood oxygen signal after training.

[0074] S2. Feature Fusion: After extracting the physiological information features of EEG, ECG, EMG, and blood oxygen modalities, in order to comprehensively and accurately evaluate the fatigue state of pilots, the present invention adopts a feature fusion method. This step aims to organically combine the abstract features from different physiological information modalities to provide more comprehensive information for the final fatigue index calculation. The process of feature fusion needs to consider the correlation between different modalities and the importance of each modality's features in expressing the fatigue state. The highly abstract time-frequency features extracted from the EEG modality, the temporal features of heart rate, heart rate variability features, EMG features, and blood oxygen features are input into the feature fusion network, and channel attention is used for fusion extraction. These five physiological information features are represented by x 1 , x 2 , …, x 5 respectively, and they are fused through a channel attention network. The attention network can be used to dynamically learn the weights of each feature to better fuse the features. Let the attention weight of feature x i be α i , then the fused feature h can be expressed as:

[0075]

[0076] where z i = W i · x i + b i , z i is a linear combination of feature x i , W i is the weight parameter of the attention network, and b i is the bias term.

[0077] S3. The fused feature h passes through a regression network to obtain the fatigue index, and the fatigue index ranges from 0 to 10. Assuming the mapping function of the regression network is f(·), then the predicted fatigue index can be expressed as:

[0078]

[0079] The regression network is a Multilayer Perceptron (MLP), which contains multiple hidden layers and activation functions.

[0080] Table 2 Predicted Fatigue Index and Accuracy in Fatigue Detection of Each Subject

[0081] Subject 1 2 3 4 5 6 7 8 9 Fatigue index 6 7 6 5 0 2 3 4 10

[0082] According to the predicted subject fatigue index in Table 2, it can reflect the overall fatigue level of pilots to a certain extent, providing an intuitive and comprehensive evaluation index for pilots and monitoring personnel.

[0083] During the process of model training, in the first stage, the feature extraction networks of each modality and the attention mechanism feature fusion module are trained simultaneously to dynamically update all parameters in the network. In the second stage, the feature extraction networks of each modality are frozen, and for different civil aviation pilot data, the personalized attention mechanism feature fusion modules are trained respectively, and the parameters of the relevant feature fusion modules of different pilots are saved. In the application of fatigue detection, the personalized fusion module parameters are used to improve the fatigue detection accuracy of civil aviation pilots.

[0084] Based on the same inventive concept, the present invention proposes a fatigue state detection system, including:

[0085] An acquisition module, configured to acquire the electroencephalogram, electrocardiogram, electromyogram, and blood oxygen physiological signals of a user.

[0086] A feature extraction module, configured to use the Morlet wavelet transform method to convert the electroencephalogram signal from the time domain to the frequency domain to generate a time-frequency diagram; use a convolutional neural network (CNN) to extract the time-frequency features in the time-frequency diagram; calculate the heart rate signal and its heart rate variability by detecting the position of the R wave in the electrocardiogram signal, and extract the time series features of the heart rate signal and the heart rate variability features in the heart rate variability through an LSTM network; use the LSTM network to extract the electromyogram activity features and blood oxygen level features in the electromyogram signal and the blood oxygen signal respectively.

[0087] A feature fusion module, configured to input the time-frequency features of the extracted electroencephalogram signal, the time series features of the heart rate, the heart rate variability features, the electromyogram features, and the blood oxygen level features into a feature fusion network for feature fusion.

[0088] A conversion module, configured to convert the fused features into a fatigue index through a regression network.

[0089] A detection module, configured to detect the fatigue state of a user according to the fatigue index.

[0090] Based on the same inventive concept, the present invention also proposes a computer device for fatigue state detection, including: a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, the steps of the fatigue state detection method are implemented.

[0091] Based on the same inventive concept, the present invention also proposes a readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, they are used to execute the steps of the fatigue state detection method.

[0092] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. A fatigue state detection method, characterized in that: The following steps are involved: Collect the user's EEG signals, ECG signals, EMG signals and blood oxygen physiological signals, and extract the time-frequency characteristics of the EEG signals, the EMG activity characteristics and the blood oxygen level characteristics; By extracting the heartbeat timestamp in the ECG signal, the position of the R wave in the ECG signal is obtained; the heart rate signal and its heart rate variability are calculated according to the time interval between adjacent R waves, and the time series features of the heart rate signal and the heart rate variability features in the heart rate variability are extracted through the LSTM network; The extracted time-frequency features of the EEG signal, the time series features of the heart rate signal, the heart rate variability features, the myoelectric activity features and the blood oxygen level features are fused through a channel attention network; The user's fatigue state is detected based on the fused features.

2. A fatigue state detection method according to claim 1, characterized in that: After collecting the user's EEG signal, the EEG signal is analyzed by using a 0.1-45 Hz bandpass filter and independent component analysis (ICA) to remove the electrooculogram, electrocardiogram and myoelectric components in the EEG signal; after collecting the user's electrocardiogram signal, the signal is filtered to remove baseline drift; After collecting the electromyographic signal and blood oxygen signal, the signal is filtered and denoised respectively.

3. A fatigue state detection method according to claim 1, characterized in that: The method further includes converting the EEG signal from the time domain to the frequency domain using the Morlet wavelet transform method to generate a time-frequency graph, which specifically includes the following steps: Time-frequency analysis is achieved through Morlet wavelet transform, which converts EEG signals from time domain to frequency domain, provides energy information in time and frequency, and generates corresponding time-frequency graphs; Define the input EEG signal as x(t), the Morlet wavelet as ψ(t), and its frequency as f, then the Morlet wavelet transform X f (τ) is expressed as: where ψ f,τ (t) is the form of the Morlet wavelet at frequency f and time translation τ, defined as: in, Where A is the normalization coefficient, f0 is the center frequency of the wavelet, σ is the parameter that controls the width of the wavelet, and i is the imaginary unit.

4. A fatigue state detection method according to claim 3, characterized in that: The convolutional neural network (CNN) is used to extract the time-frequency features of the EEG signal, which specifically includes the following steps: The convolutional layer is used to capture the local features of the time-frequency graph and perceive the EEG activities in different frequency bands and time points; The captured feature map is reduced in dimension through the pooling layer to obtain the high-dimensional features of EEG.

5. A fatigue state detection method according to claim 1, characterized in that: The method of calculating the heart rate signal and its heart rate variability according to the time interval between adjacent R waves specifically includes the following steps: Extract the heartbeat timestamp from the ECG signal, i.e., the RR interval, where the R wave corresponds to each contraction of the heart; The sliding window method is used to detect the R wave in the ECG signal; The heart rate HR is calculated based on the time interval between adjacent R waves, which is expressed as: Among them, Δt is the time interval, and N is the number of heartbeats per unit time; The heart rate time series is obtained by calculating the time interval between R waves, and the heart rate variability is calculated according to the heart rate time series; the heart rate variability parameters include the standard deviation SDNN, the standard deviation SDRR of adjacent RR intervals, and the root mean square deviation RMSSD, which are expressed as: Where t represents the time series index.

6. A fatigue state detection method according to claim 1, characterized in that: The extracted time-frequency features of the EEG signal, the time series features of the heart rate signal, the heart rate variability features, the myoelectric activity features and the blood oxygen level features are subjected to feature fusion through a channel attention network, specifically including the following steps: The time-frequency characteristics of the EEG signal, the time series characteristics of the heart rate, the heart rate variability characteristics, the myoelectric activity characteristics, and the blood oxygen level characteristics are represented by x1, x2, x3, x4, and x5 respectively; Pass x1, x2, x3, x4, and x5 through a channel attention network to perform feature fusion; let feature x i The attention weight is α i , then the fused feature h is expressed as: Among them, z i =W i ·x i +b i , z i is feature x i The linear combination of i is the weight parameter of the attention network, b i is the bias term, Represents an exponential function.

7. A fatigue state detection method according to claim 6, characterized in that: The method also includes converting the fused features into a fatigue index before detecting the fatigue state of the user according to the fused features. It is expressed as: Wherein, f(·) is the mapping function of the regression network, and the regression network is a multi-layer perceptron, which includes multiple hidden layers and activation functions.

8. A fatigue state detection system, characterized in that: include: The acquisition module is used to collect the user's EEG signals, ECG signals, EMG signals and blood oxygen physiological signals, and extract the time-frequency characteristics of the EEG signals, the EMG activity characteristics and the blood oxygen level characteristics; The feature extraction module is used to obtain the position of the R wave in the ECG signal by extracting the heartbeat timestamp in the ECG signal; calculate the heart rate signal and its heart rate variability according to the time interval between adjacent R waves, and extract the time series features of the heart rate signal and the heart rate variability features in the heart rate variability through the LSTM network; The feature fusion module is used to fuse the extracted time-frequency features of the EEG signal, the time series features of the heart rate signal, the heart rate variability features, the myoelectric activity features and the blood oxygen level features through a channel attention network; The detection module is used to detect the user's fatigue state based on the fused features.

9. A fatigue state detection computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory, wherein the processor implements the steps of the fatigue state detection method according to any one of claims 1 to 7 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, they are used to execute the steps of the fatigue state detection method according to any one of claims 1 to 7.

Images