Fatigue state recognition method based on learnable filter bank and joint regularization

Abstract

The application discloses a fatigue state recognition method based on a learnable filter bank and joint regularization, and belongs to the field of fatigue state monitoring, and the steps are as follows: acquiring an original physiological signal, performing basic interference removal preprocessing on the original physiological signal, organizing the processed data into a three-dimensional input tensor in a multi-channel format, adding a learnable filter bank layer before model input, the learnable filter bank layer is trained jointly with the model, and the frequency band is dynamically adjusted according to the task requirement; a fatigue state recognition model is constructed based on a CNN feature extraction module, an LSTM sequence modeling module and an attention weighting module; a joint optimization objective function is constructed based on a filter bank sparse regularization term, a time sequence transfer consistency regularization term and an adaptive time-frequency consistency regularization term; and the fatigue state is recognized according to the trained model. The application not only improves the accuracy of fatigue recognition, but also significantly improves the smoothness and cross-individual robustness of the prediction result.

Description

Technical Field

[0001] This invention belongs to the field of fatigue state monitoring technology, specifically relating to a fatigue state identification method based on learnable filter banks and joint regularization. Background Technology

[0002] With the development of intelligent monitoring technology, fatigue state recognition is gradually evolving from traditional manual observation to an automated intelligent recognition system based on physiological signals and behavioral characteristics. Existing research has made some progress in areas such as physiological parameters (e.g., ECG, EEG, EOG), visual behavioral characteristics (e.g., eye posture, head posture), and multimodal fusion, but overall there are still significant limitations.

[0003] On the one hand, fatigue monitoring methods based on physiological signals often employ fixed preprocessing filter banks and manually designed frequency domain features, making it difficult to adapt to individual differences and physiological frequency band variations under different task loads. In 2022, Liu Jie et al. used an ECG-based SA-BiGRU model to extract temporal features, but its processing flow still relied on fixed bandpass filtering and manual preprocessing. In 2021, Xu Junli et al. pointed out in EEG-eye-tracking fusion detection that the stability of EEG frequency bands is significantly affected by noise and working conditions, and fixed frequency bands are difficult to take into account cross-subject differences. In 2023, Su Ruizhi et al. also indicated that existing physiological signal fatigue monitoring algorithms generally fail to solve the problem of "optimal frequency bands varying from person to person," lack adaptive filtering mechanisms, and limit the model's generalization ability in complex scenarios.

[0004] On the other hand, many fatigue detection studies rely on visual behavioral features, such as eye closure, facial landmarks, and head posture. While these methods perform well in specific scenarios, they are susceptible to external factors such as changes in lighting, occlusion, and posture shifts, exhibiting low robustness and difficulty in stable operation in complex environments such as mines and logistics parks. Furthermore, visual methods cannot directly characterize the intensity of physiological fatigue and struggle to depict deeper patterns of fatigue evolution.

[0005] In the field of deep learning, various learnable filter models have been proposed in recent years to replace the difficult-to-interpret convolutional kernels in traditional CNNs. Among them, Mirco Ravanelli and Yoshua Bengio achieved learnable upper and lower cutoff frequencies of bandpass filters through Sinc function parameterization, significantly improving the interpretability of the filters. However, this type of method still has limitations: for example, training instability can lead to frequency band drift or excessive contraction, a problem explicitly pointed out by Helena Peic Tukuljac et al.; in addition, Junyi Peng et al. pointed out in "Learnable Sparse Filterbank for SpeakerVerification" that learnable filter banks generally suffer from insufficient sparsity, frequency band repetition, and unstable bandwidth structure, making it difficult for these methods to maintain robust frequency band division capabilities when processing complex and noisy physiological signals.

[0006] On the other hand, fatigue, as a physiological process that accumulates gradually over time, requires strong temporal continuity in its identification. However, existing deep time series models typically lack temporal consistency constraints on the evolution of fatigue states. For example, Hsia g-Fu Yu et al. pointed out that to improve the stability of time series predictions, sequence smoothing and transition constraints need to be introduced in the time dimension; Arnab Ghosh et al. also showed that appropriate temporal regularization can effectively reduce prediction jumps caused by short-term noise and maintain the consistency of the model over continuous time. However, existing fatigue identification methods mostly focus only on feature extraction and lack modeling of the gradual evolution of fatigue levels, which can easily lead to problems such as jitter and unreasonable short-term jumps in prediction results.

[0007] In summary, current fatigue state recognition technology still has prominent problems in the following aspects:

[0008] ① The frequency band is not adaptive, and it is difficult for a fixed filter to match the optimal physiological frequency band for different individuals and working conditions;

[0009] ② The filter has insufficient stability; learnable filters suffer from problems such as frequency band drift and bandwidth degradation.

[0010] ③ The model has weak time-series consistency modeling, lacks continuity constraints in the time dimension, and the prediction results are prone to fluctuations;

[0011] ④ It has limited cross-individual generalization ability and its performance degrades significantly in complex scenarios with multiple subjects.

[0012] Therefore, there is an urgent need for an identification method that can simultaneously achieve frequency band adaptation, bandwidth and boundary control, enhanced timing consistency, and is applicable to physiological signal fatigue monitoring under complex working conditions. Summary of the Invention

[0013] To address the aforementioned problems in existing technologies, this invention proposes a fatigue state identification method based on learnable filter banks and joint regularization. This method is rationally designed, overcomes the shortcomings of existing technologies, and achieves good results.

[0014] To achieve the above objectives, the present invention adopts the following technical solution:

[0015] The fatigue state identification method based on learnable filter banks and joint regularization includes the following steps:

[0016] Step 1: Obtain the raw physiological signals of the personnel;

[0017] Step 2: Perform basic interference removal preprocessing on the original physiological signals, organize the processed data into a three-dimensional input tensor in a multi-channel format, add a learnable filter bank layer before the model input, and train the learnable filter bank layer together with the model. The frequency band is dynamically adjusted according to the task requirements to improve the recognition robustness across individuals and in complex environments.

[0018] Step 3: Construct a fatigue state recognition model based on the CNN feature extraction module, LSTM sequence modeling module, and attention weighting module, and complete the fatigue state determination through the classification output module;

[0019] Step 4: Construct a joint optimization objective function based on the filter bank sparse regularization term, the time-series transition consistency regularization term, and the adaptive time-frequency consistency regularization term; and jointly train the learnable filter bank layer and the model according to the joint optimization objective function.

[0020] Step 5: Identify fatigue state based on the trained model.

[0021] Furthermore, in step 2, the basic interference removal preprocessing specifically involves: sequentially filtering specific effective frequency bands through bandpass filtering, suppressing high-frequency noise and retaining target features through wavelet denoising; then processing the model input to adapt to the data, using the sliding window method to cut the continuous time series signal into fixed-length sample segments to increase the number of samples and time series generalization ability, and then unifying the data scale through z-score standardization or min-max normalization to avoid model bias caused by amplitude differences;

[0022] The learnable filter bank layer constructs the Sinc kernel function by parameterizing the upper and lower cutoff frequencies, and automatically optimizes the parameters through backpropagation during training, so that the model can adaptively adjust the optimal frequency band according to different individuals and different working conditions.

[0023] The learnable filter bank layer consists of multiple parallel learnable filters, with the number of filters being K; each filter has independent upper and lower cutoff frequency parameters ( , The filters are defined as acting on different frequency bands of the input signal; the convolution outputs of each filter are concatenated along the channel direction to form an overall output feature tensor, which is used as input to the subsequent convolutional neural network.

[0024] No. A learnable filter is defined as:

[0025] ;

[0026] in, For the first The impulse response of a learnable filter For the first The lower cutoff frequency of the filter. For the first The upper cutoff frequency of the filter;

[0027] Different types of signals in multimodal raw physiological signals are mapped to different channels, and the first... Each channel signal is denoted as ,right Filtered output for:

[0028] ;

[0029] To ensure stable training and the injection of physiological priors, regularization and boundary constraints are introduced into the filter kernel:

[0030] ;

[0031] in, For filter bank sparse regularization terms, , These are the weighting coefficients. >0, for The discretized vector form, i.e., the filter kernel weights, is used to constrain the energy of the filter kernel. and For the global frequency boundary, For minimum bandwidth, >0.

[0032] Furthermore, in step 2, the CNN feature extraction module includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input is the output of the learnable filter bank layer, and the output is a sequence of feature maps.

[0033] The LSTM sequence modeling module is used to extract long-term temporal dependencies, process the feature map sequence, and output the hidden state sequence.

[0034] The attention weighting module applies attention weights to the hidden state sequence, highlighting key moment features and outputting a weighted global feature vector. ;

[0035] In the classification output module The predicted probability vector of fatigue state is mapped through a fully connected layer and a Softmax classification layer, and its calculation form is as follows:

[0036] ;

[0037] in, For the first The predicted probability vector at each time step. This is the weight matrix. is the bias term, and L is the number of fatigue state categories.

[0038] Furthermore, the temporal transition consistency regularization term is based on the fatigue state transition matrix. Establish prior constraints to reduce frequent jumps in prediction results; apply consistency constraints to adjacent time steps, expressed as:

[0039] ;

[0040] in, For time-series transition consistency regularization, The regularization coefficient is . , For the first The predicted probability distribution vector at time t. This represents the total number of time steps for the time series samples. KL divergence is used to measure the difference between two probability distributions; Let be the fatigue state transition matrix, and its th... OK Indicates from the first The conditional probability distribution of transitions from one state to another; Indicates matrix transpose; Represents the current predicted probability distribution vector The estimated distribution for the next time step is obtained through the state transition prior;

[0041] Introducing smoothing indicators The number of class switching occurrences in the predicted sequence is statistically analyzed to evaluate the constraint effect of the time series transition consistency regularization term. The expression is as follows:

[0042] ;

[0043] in, For the The predicted probability vector at time step Take the index with the highest probability. For indicator functions, if the first... The prediction category at time and the first If the predicted categories at different times are not equal, the function value is 1; otherwise, the function value is 0. The lower the smoothing index value, the better the constraint effect of the time series transition consistency regularization term.

[0044] Furthermore, the adaptive time-frequency consistency regularization term By dynamically coupling the smoothness of time series prediction with the frequency band energy variation pattern, consistency optimization across the time and frequency domains is achieved, as expressed in the following expression:

[0045] ;

[0046] in, No. A learnable filter at time... Average frequency band energy, , These are the time-domain and frequency-domain regularization weights, respectively. To prevent constants with a denominator of zero, KL divergence is a measure of the difference between two distributions.

[0047] Joint optimization objective function for:

[0048] ;

[0049] in, For cross-entropy loss, , , These are the weighting coefficients.

[0050] The beneficial technical effects of this invention are as follows:

[0051] 1. Improve cross-individual robustness;

[0052] The "dual-layer filtering structure" proposed in this invention introduces a learnable filter bank layer on top of traditional fixed bandpass filtering. This allows the model to automatically select the optimal frequency band based on different individuals and operating conditions, thereby enhancing its adaptability to individual differences and complex environments. Compared to existing methods that rely solely on fixed filtering, this invention can more effectively capture fatigue-related characteristic signals and significantly improve cross-individual generalization ability.

[0053] 2. Improve the smoothness and rationality of time series forecasts;

[0054] This invention introduces a "temporal transition consistency regularization" into the output of a Long Short-Term Memory (LSTM) network, forcing the prediction results to conform to the physiological progression of "awake → mild fatigue → severe fatigue." This design effectively avoids the prediction jitter and unreasonable abrupt switching common in existing methods, ensuring the stability and reliability of fatigue state identification results over time.

[0055] 3. An adaptive time-frequency consistency dynamic coupling regularization term is proposed;

[0056] This invention proposes for the first time an "Adaptive Temporal–Spectral Consistency Regularization" (ATSCR) mechanism, introducing dual constraints of temporal smoothness and frequency-domain energy stability into the joint optimization objective. This mechanism dynamically couples state transition probabilities with frequency band energy changes for the first time, enabling the model to maintain predictive asymptoticity in the time dimension and filter response continuity in the frequency dimension. This innovation overcomes the limitations of existing regularization methods that only perform one-dimensional regularization, significantly improving the robustness and physiological plausibility of fatigue identification.

[0057] 4. Achieve multi-objective joint optimization;

[0058] The joint optimization objective function designed in this invention not only includes the traditional classification loss, but also introduces filter constraint regularization, temporal consistency regularization, and adaptive time-frequency consistency regularization, forming a multi-objective collaborative optimization mechanism of "classification accuracy + feature robustness + temporal rationality". This mechanism ensures that the model maintains recognition accuracy while further improving the stability and practicality of prediction.

[0059] In summary, this invention not only improves the accuracy of fatigue identification, but also significantly improves the smoothness and cross-individual robustness of the prediction results. It can better meet the real-time monitoring needs in high-risk scenarios such as driving safety and mining operations, and has outstanding application value and promotion prospects. Attached Figure Description

[0060] Figure 1 This is a flowchart of the fatigue state recognition method based on learnable filter banks and joint regularization in this invention.

[0061] Figure 2 This is a heatmap of the attention mechanism in this invention.

[0062] Figure 3 This figure shows the comparison results of the predicted label time series using the method of this invention and the baseline model.

[0063] Figure 4 This is a heatmap of the filter's frequency band response.

[0064] Among them, (a) is the band response heatmap of a fixed bandpass filter; (b) is the band response heatmap of a learnable filter bank.

[0065] Figure 5 A comparison chart of frequency band energy timing "jumps".

[0066] Figure 6 This is a comparison chart of time-frequency consistency indices.

[0067] Figure 7 This is a comparison chart of confusion matrices;

[0068] Wherein, (a) is the baseline confusion matrix; and (b) is the confusion matrix of the present invention.

[0069] Figure 8 This is a training curve diagram of the method of the present invention;

[0070] Wherein, (a) is the accuracy curve of the method of the present invention; (b) is the loss curve of the method of the present invention.

[0071] Figure 9 The graph shows the ATSCR weight γ versus Macro-F1, number of category switching, and TSCI.

[0072] Among them, (a) is a curve of ATSCR weight γ versus Macro-F1 and TSCI; (b) is a curve of ATSCR weight γ versus the number of category switching. Detailed Implementation

[0073] The specific embodiments of the present invention will be further described below with reference to specific examples:

[0074] Fatigue state identification methods based on learnable filter banks and joint regularization, such as Figure 1 As shown, it includes the following steps:

[0075] Step 1: Acquire the raw physiological signals of the personnel, including electrocardiogram (ECG) signals, electrodermal conductance (EDA) signals, and electroencephalogram (EEG) signals;

[0076] Step 2: Perform basic interference removal preprocessing on the original physiological signals, organize the processed data into a three-dimensional input tensor in a multi-channel format, add a learnable filter bank layer before the model input, and train the learnable filter bank layer and the model together. Dynamically adjust the frequency band according to the task requirements to improve the recognition robustness across individuals and in complex environments.

[0077] The basic interference removal preprocessing is as follows: first, specific effective frequency bands are selected by bandpass filtering; second, high-frequency noise is suppressed by wavelet denoising while retaining target features; then, the input of the model is adapted, and the continuous time series signal is cut into sample segments of fixed length by sliding window method to improve the number of samples and time series generalization ability. Finally, the data scale is unified by z-score standardization or min-max normalization to avoid model bias caused by amplitude differences.

[0078] Specifically, non-target frequency band interference components in the original signal need to be removed through filtering. Based on the spectral characteristics of different signals, appropriate bandpass filters are selected to retain the effective frequency bands, and wavelet denoising or smoothing filters are used to suppress high-frequency noise. Taking EEG signals as an example, a 0.5–45Hz bandpass filter is typically used to retain the main brainwave rhythms; a 0.5–40Hz bandpass filter can be used for ECG to retain the low-frequency components of QRS and fatigue; and a 0.05–5Hz smoothing filter can be used for EDA to suppress fast noise.

[0079] Taking EEG signals as an example, a band-pass filter is typically used to preserve brain activity in a specific frequency band (such as the common 0.5Hz–45Hz range), and wavelet denoising is employed: the signal is decomposed into multiple scales through wavelet transform to suppress high-frequency noise components and preserve low-frequency fatigue characteristics; the filtered signal can be represented as:

[0080] ;

[0081] in, This indicates the bandpass filter function used. The original signal, This is the filtered signal;

[0082] To accommodate the fixed-length input format required for convolution operations while preserving temporal dynamics, continuous time-series data needs to be segmented into windows. A sliding window method is used for this segmentation.

[0083] ;

[0084] in, For the first A sample fragment, To indicate the first The starting time point (sampling time) of each time window. Window length (e.g., 2 seconds), step size The overlap ratio can be set to 50%. Sliding window truncation can increase the number of samples, enhance the model's temporal generalization ability, and reduce the impact of short-term disturbances.

[0085] The amplitude of signals varies significantly among different individuals (subjects). The original signals need to be scaled uniformly before being input into the model to ensure more stable model training. Generally, z-score normalization or min-max normalization is used.

[0086] The z-score normalization expression is:

[0087] ;

[0088] in, For the standardized signal, The mean, Standard deviation;

[0089] The min-max normalization expression is:

[0090] ;

[0091] in, The signal after normalization and These are the maximum and minimum values ​​in the segment, respectively.

[0092] Standardization or normalization can accelerate the network convergence process while avoiding bias in the model for high-amplitude channels.

[0093] The preprocessed data needs to be organized into a uniform tensor structure to adapt to CNN input. If the acquired signal is a multi-channel EEG, it can be organized into a three-dimensional input tensor. : For the sample size, The number of signal channels (e.g., EEG16 channels). The time step length for each signal segment (e.g., 256 points).

[0094] The learnable filter bank layer constructs the Sinc kernel function by parameterizing the upper and lower cutoff frequencies, and automatically optimizes the parameters through backpropagation during training, enabling the model to adaptively adjust the optimal frequency band according to different individuals and different working conditions;

[0095] The learnable filter bank layer consists of multiple parallel learnable filters, with the number of filters being K; each filter has independent upper and lower cutoff frequency parameters ( , The filters are defined as acting on different frequency bands of the input signal; the convolution outputs of each filter are concatenated along the channel direction to form an overall output feature tensor, which is used as input to the subsequent convolutional neural network.

[0096] The k-th learnable filter is defined as:

[0097] ;

[0098] in, For the first The impulse response of a learnable filter For the first The lower cutoff frequency of the filter. For the first The upper cutoff frequency of the filter. The normalized sine function is defined as follows: ;

[0099] Different types of signals in multimodal raw physiological signals are mapped to different channels, and the first... Each channel signal is denoted as (t), for The filtered output of (t) for:

[0100] ;

[0101] To ensure stable training and the injection of physiological priors, regularization and boundary constraints are introduced into the filter kernel:

[0102] ;

[0103] in, For filter bank sparse regularization terms, , These are the weighting coefficients. >0, for The discretized vector form, i.e., the filter kernel weights, is used to constrain the energy of the filter kernel. and As the global frequency boundary, a physiologically reasonable range is selected (e.g., 0.5–45 Hz for EEG). For minimum bandwidth, >0.

[0104] Unlike fixed bandpass, the above , During training, the network is jointly optimized through backpropagation, enabling it to automatically select discriminative frequency bands under specific tasks and individual differences, forming a two-layer filter ("fixed filter + learnable filter bank") to improve cross-subject robustness.

[0105] In this way, the model can effectively suppress interference from irrelevant frequency bands while enhancing the feature representation of fatigue-related frequency bands.

[0106] Step 3: Construct a fatigue state recognition model based on the CNN feature extraction module, LSTM sequence modeling module, and attention weighting module, and complete the fatigue state determination through the classification output module;

[0107] The CNN feature extraction module includes an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The input is the output of the learnable filter bank layer, and the output is a sequence of feature maps.

[0108] The preprocessed signal data is input into the CNN in a multi-channel format, where each channel can represent a different physiological signal or a different feature dimension.

[0109] Convolutional layers are used to extract low-level features from the data. By sliding convolutional kernels across the input physiological signal data, they extract features within a local timeframe. These features may include fluctuations within a specific frequency range, periodic changes in the signal, etc. These feature maps are processed layer by layer to progressively extract higher-level features, enabling the identification of complex patterns related to fatigue states; the expression is:

[0110] ;

[0111] in, For activation function, For the convolutional layer The set of input channels corresponding to each output feature map Represents convolution operation. Indicates the convolutional layer's... The first in the layer One input feature map, The weight matrix representing the convolution kernel. The bias matrix, For the first The first layer of convolutional network Each output feature map;

[0112] Pooling layers are used to prevent overfitting by reducing the dimensionality of the data. They downsample the feature maps generated by convolutional layers, reducing the spatial dimension of the features while retaining the most salient feature information. This not only reduces the computational complexity of the model but also helps enhance its robustness to noise. Through pooling operations, CNNs have the ability to aggregate overall features, better capturing overall fatigue-related features.

[0113] The pooling process is represented by the following formula:

[0114] ;

[0115] in, This represents the selected sampling function. This is the pooling function.

[0116] Fully connected layers combine convolutional and pooling layers to obtain the basic characteristics and information of features. They typically employ the Rectified Linear Unified Array (ReLU) function. The formula is as follows:

[0117] ;

[0118] This represents an element-wise nonlinear mapping function of the output feature map of a convolutional or pooling layer, whose input... These are the feature values ​​output by the convolutional or pooling layer. The output layer obtains the maximum probability solution based on the information from the fully connected layer.

[0119] In the LSTM sequence modeling module, the hidden layers of the Long Short-Term Memory network are composed of storage units, and each storage unit consists of an input gate, a forget gate, and an output gate.

[0120] The forget gate receives the hidden state from the previous moment. and current input The output of the forget gate is obtained after the sigmoid activation function. Its value ranges from 0 to 1 and is used to control cell state. The retention ratio is calculated using the following formula:

[0121] ;

[0122] in, , Indicates weight, Indicates bias. This represents the activation function sigmoid. This indicates the cell state at the previous moment.

[0123] The input gate processes the input at the current position, determining which information to update in the cell state. At this point, the neuron's input... and the hidden state of the previous neuron First, process it using the sigmoid function to obtain... ,also , Also entered The activation function obtains the current neuron state. ,at last and The two factors are multiplied to update the cell state, and the calculation formula is as follows:

[0124] ;

[0125] ;

[0126] The output gate determines the model's output and the cell's state. Similar to the input gate, and The output after passing through the sigmoid activation function is obtained , Depend on After the activation function output and Multiplication yields the hidden state , The calculation is shown in the formula:

[0127] ;

[0128] ;

[0129] in, , , Both represent bias terms. , , , , , All represent weights.

[0130] In the classification process, LSTM based on time-series memory features can introduce the time-varying features of fatigue features into the prediction results, so that the prediction results can be integrated with the time-series features, which not only matches the fatigue mechanism of workers (such as drivers, operators, miners, etc.) but also better reveals the fatigue state of miners.

[0131] Attention Weighting Module: This module transforms the input feature matrix into a query vector Q, a key vector K, and a value vector V using different linear transformations (e.g., WQ, WK, WV). It calculates the dot product score between the query and key vectors and normalizes these scores using Softmax to obtain attention weights. The value vectors are then weighted and summed using these attention weights to obtain the weighted output vector. The multi-head attention mechanism divides the query, key, and value into multiple subspaces, each learning an independent attention pattern. Finally, the outputs of all attention heads are concatenated and a linear transformation is applied to obtain the final output MultiHead(Q,K,V).

[0132] MultiHead(Q,K,V)=Concat(head1,…,head h )WO;

[0133] Where h is the number of attention heads, and WO is the linear transformation matrix.

[0134] Final output It is obtained through weighted summation, reflecting the weighted characteristics of important parts of the input, such as... Figure 2As shown, the horizontal axis (0~29) represents the time steps (30 sliding windows or frames in total), and the vertical axis (0~4) has 5 feature dimensions (such as alpha wave, beta wave, ECG signal, EDA signal, and EEG signal). The color intensity represents the feature strength after attention weighting. The darker the color, the more the model pays attention to this "time step × feature" combination.

[0135] In the classification output module, The predicted probability vector of fatigue state is mapped through a fully connected layer and a Softmax classification layer, and its calculation form is as follows:

[0136] ;

[0137] in, Let be the predicted probability vector at time step t. This is the weight matrix. For bias terms, This represents the number of fatigue state categories (e.g., alert, mild fatigue, severe fatigue).

[0138] Step 4: Construct a joint optimization objective function based on the filter bank sparse regularization term, the time-series transition consistency regularization term, and the adaptive time-frequency consistency regularization term; and jointly train the learnable filter bank layer and the model according to the joint optimization objective function.

[0139] Joint optimization objective function for:

[0140] ;

[0141] in, For cross-entropy loss, This is a sparse regularization term for the filter bank, used to constrain the stability of band learning. This is a time-series consistency regularization term used to constrain the smoothness and reasonableness of predictions. , , These are the weighting coefficients. During training, { , The parameters of the CNN, LSTM, attention, and classification layers are jointly backpropagated, and a validation set is used for selection. , , (e.g., grid search {10}) −4 ~10 −1}).

[0142] Through end-to-end training, the model can maintain smooth predictions in the time domain and continuous filtering response in the frequency domain, significantly reducing the impact of cross-individual noise and short-term abrupt changes.

[0143] The timing transition consistency regularization term is based on the fatigue state transition matrix. Establish prior constraints to reduce frequent jumps in prediction results; apply consistency constraints to adjacent time steps, expressed as:

[0144] ;

[0145] in, For time-series transition consistency regularization, The regularization coefficient is . , For the first The predicted probability distribution vector at time t. The total number of time steps in the time series sample is t, which is the total number of discrete moments contained in the segment. The regularization term only applies to two adjacent time points, so the summation range is from t=1 to T−1. KL divergence is used to measure the difference between two probability distributions; Let i be the fatigue state transition matrix, and let i be the row number of the matrix. Indicates from the first The conditional probability distribution of transitions from one state to another; Indicates matrix transpose; Represents the current predicted probability distribution vector The estimated distribution for the next time step is obtained through the state transition prior;

[0146] Introducing smoothing indicators The number of class switching occurrences in the predicted sequence is statistically analyzed to evaluate the constraint effect of the time series transition consistency regularization term. The expression is as follows:

[0147] ;

[0148] in, For the The predicted probability vector at time step Take the index with the highest probability. For indicator functions, if the first... The prediction category at time and the first If the predicted categories at different times are not equal, the function value is 1; otherwise, the function value is 0. The lower the smoothing index value, the better the constraint effect of the time series transition consistency regularization term.

[0149] To further improve the stability and physiological rationality of the model in both time and frequency dimensions, this invention proposes for the first time an "Adaptive Temporal–Spectral Consistency Regularization" (ATSCR). The adaptive time-frequency consistency regularization term... By dynamically coupling the smoothness of time series prediction with the frequency band energy variation pattern, consistency optimization across the time and frequency domains is achieved, as expressed in the following expression:

[0150] ;

[0151] in, No. A learnable filter at time... Average frequency band energy, , These are the time-domain and frequency-domain regularization weights, respectively. To prevent constants with a denominator of zero, KL divergence is a measure of the difference between two distributions.

[0152] The ATSCR regularization term in this invention takes into account:

[0153] Temporal consistency (by constraining the predicted distribution of adjacent time steps using KL divergence).

[0154] Band stability (the smoothness of energy changes in the same band at adjacent times constrained by the squared difference).

[0155] Step 5: Identify fatigue state based on the trained model.

[0156] To verify the effectiveness and superiority of the method proposed in this invention, comparative experiments and ablation analyses were designed and conducted. Performance verification was performed primarily from three dimensions: prediction smoothness, frequency band adaptability, and classification performance. Specifically, the following were included:

[0157] The evaluation metrics used include: Classification accuracy: overall recognition accuracy. Macro-F1 score: comprehensively measures the recognition performance of each class, avoiding the impact of class imbalance. AUC (Area Under Curve): reflects the overall classification discrimination. Recall for Severe Fatigue: focuses on the recognition ability of critical states. Temporal smoothness metrics: including the number of predicted switches (SwitchCount, the smaller the better) and the variance of the predicted probability distribution (…). (The smaller the better).

[0158] Design ablation experiments:

[0159] The proposed method is progressively compared with comparative models of different configurations to verify the contributions of each module:

[0160] (a) Baseline model: CNN + LSTM + Attention.

[0161] (b) Baseline + Learnable Filter Bank: Validating the contribution of band adaptation.

[0162] (c) Baseline + Temporal Consistency Regularization: Verify the effect of temporal smoothing constraints.

[0163] (d) This invention (two-layer filtering + timing regularization + joint optimization).

[0164] The experimental results are as follows: Figure 3 As shown, without the introduction of ATSCR, the predicted labels exhibited frequent jitter and skipped switching; after the addition of ATSCR, the sequence became significantly smoother, and the number of switching times decreased from 16 to 2, which is consistent with the physiological progression of "awake → mild → severe", indicating that the temporal consistency regularization achieved the expected effect.

[0165] like Figure 4 As shown in (a) and (b), a fixed bandpass filter has narrow and rigid coverage, while a learnable filter bank can form multi-band adaptive coverage; combined with Figure 5 , Figure 6 It is evident that this design, in conjunction with ATSCR, significantly improves frequency domain stability and time-frequency consistency.

[0166] Figure 5 The statistical behavior of the band energy difference Δ(t) with and without ATSCR was compared: with ATSCR, μ decreased from 0.0878 to 0.0096, and the variance decreased from 8.03 × 10⁻⁴ to 1.1 × 10⁻⁵, with a significant reduction in the 95th percentile. The results indicate that ATSCR effectively suppresses transient jumps in the frequency domain, making the feature evolution more consistent with the natural dynamics of physiological signals.

[0167] like Figure 6 As shown, TSCI improved from 0.850 to 0.906 after the introduction of ATSCR, and the difference was statistically significant (p=0.0003 in the example). This indicator directly quantifies the improvement in time-frequency dynamic coupling consistency.

[0168] like Figure 7 As shown in (a) and (b), the present invention significantly improves the recall rate in the severe fatigue category, and the confusion matrix is ​​more concentrated on the diagonal, indicating that it is more robust in identifying critical safety conditions.

[0169] In addition, such as Figure 8 As shown in (a), adding ATSCR results in faster convergence and higher validation set accuracy, indicating that the joint objective improves training stability and generalization ability. Further... Figure 8 As shown in (b), the training / validation loss decreases more smoothly and with less fluctuation, compared to Figure 9 The SwitchCount in (b) shows a consistent decreasing trend, reflecting the benefits of this invention in terms of output stability.

[0170] like Figure 9 As shown in (a) and (b), when the ATSCR weight γ increases from 0 to 0.1, the Macro-F1 and TSCI increase synchronously (error bars represent the standard deviation / confidence interval of repeated experiments), indicating that the joint objective of this invention continuously improves time-frequency consistency without sacrificing recognition accuracy. Simultaneously, as γ increases, SwitchCount decreases monotonically (e.g., 35→27), indicating that ATSCR can effectively suppress short-term oscillations and jumps, further verifying the advantages of this invention in time-domain stability.

[0171] Compared to the baseline model, this invention demonstrates superior performance in the following aspects: Enhanced cross-individual robustness: The two-layer filtering structure improves consistency among different subjects. Temporal prediction rationality: Transition consistency regularization suppresses abrupt switching that does not conform to physiological patterns. Improved critical state identification: The recall rate of the severe fatigue category is significantly improved, ensuring safety monitoring needs are met. Overall performance improvement: Accuracy and Macro-F1 scores are both significantly higher than the baseline model.

[0172] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A fatigue state identification method based on learnable filter banks and joint regularization, characterized in that, Includes the following steps: Step 1: Obtain the raw physiological signals of the personnel; Step 2: Perform basic interference removal preprocessing on the original physiological signals, organize the processed data into a three-dimensional input tensor in a multi-channel format, add a learnable filter bank layer before the model input, and train the learnable filter bank layer together with the model. The frequency band is dynamically adjusted according to the task requirements to improve the recognition robustness across individuals and in complex environments. Step 3: Construct a fatigue state recognition model based on the CNN feature extraction module, LSTM sequence modeling module, and attention weighting module, and complete the fatigue state determination through the classification output module; Step 4: Construct a joint optimization objective function based on the filter bank sparse regularization term, the time-series transition consistency regularization term, and the adaptive time-frequency consistency regularization term; and jointly train the learnable filter bank layer and the model according to the joint optimization objective function. Step 5: Identify fatigue states based on the trained model; The time-series transition consistency regularization term is based on the fatigue state transition matrix. Establish prior constraints to reduce frequent jumps in prediction results; apply consistency constraints to adjacent time steps, expressed as: ； in, For time-series transition consistency regularization, The regularization coefficient is . , For the first The predicted probability distribution vector at time t. This represents the total number of time steps for the time series samples. KL divergence is used to measure the difference between two probability distributions; Let be the fatigue state transition matrix, and its th... OK Indicates from the first The conditional probability distribution of transitions from one state to another; Indicates matrix transpose; Represents the current predicted probability distribution vector The estimated distribution for the next time step is obtained through the state transition prior; Introducing smoothing indicators The number of class switching occurrences in the predicted sequence is statistically analyzed to evaluate the constraint effect of the time series transition consistency regularization term. The expression is as follows: ； in, For the first The predicted probability vector at time step Take the index with the highest probability. For indicator functions, if the first... The prediction category at time and the first If the predicted categories at different times are not equal, the function value is 1; otherwise, the function value is 0. The lower the smoothing index value, the better the constraint effect of the time series transition consistency regularization term. The adaptive time-frequency consistency regularization term By dynamically coupling the smoothness of time series prediction with the frequency band energy variation pattern, consistency optimization across the time and frequency domains is achieved, as expressed in the following expression: ； in, No. A learnable filter at time... Average bandwidth energy, , These are the time-domain and frequency-domain regularization weights, respectively. To prevent constants with a denominator of zero, KL divergence is a measure of the difference between two distributions. Joint optimization objective function for: ； in, For cross-entropy loss, , , These are the weighting coefficients.

2. The fatigue state identification method based on learnable filter banks and joint regularization according to claim 1, characterized in that, In step 2, the basic interference removal preprocessing specifically involves: sequentially filtering specific effective frequency bands through bandpass filtering, and suppressing high-frequency noise while retaining target features through wavelet denoising. The input to the model is then processed to adapt it. The sliding window method is used to cut the continuous time series signal into sample segments of fixed length to increase the number of samples and the time series generalization ability. Then, the data scale is unified by z-score standardization or min-max normalization to avoid model bias caused by amplitude differences. The learnable filter bank layer constructs the Sinc kernel function by parameterizing the upper and lower cutoff frequencies, and automatically optimizes the parameters through backpropagation during training, so that the model can adaptively adjust the optimal frequency band according to different individuals and different working conditions. The learnable filter bank layer consists of multiple parallel learnable filters, with the number of filters being K; each filter has independent upper and lower cutoff frequency parameters ( , The filters are defined as acting on different frequency bands of the input signal; the convolution outputs of each filter are concatenated along the channel direction to form an overall output feature tensor, which is used as input to the subsequent convolutional neural network. No. A learnable filter is defined as: ； in, For the first The impulse response of a learnable filter For the first The lower cutoff frequency of the filter. For the first The upper cutoff frequency of the filter; Different types of signals in multimodal raw physiological signals are mapped to different channels, and the first... Each channel signal is denoted as ,right Filtered output for: ； To ensure stable training and the injection of physiological priors, regularization and boundary constraints are introduced into the filter kernel: ； in, For filter bank sparse regularization terms, , These are the weighting coefficients. >0, for The discretized vector form, i.e., the filter kernel weights, is used to constrain the energy of the filter kernel. and For the global frequency boundary, For minimum bandwidth, >

0.

3. The fatigue state identification method based on learnable filter banks and joint regularization according to claim 1, characterized in that, In step 2, the CNN feature extraction module includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input is the output of the learnable filter bank layer, and the output is a sequence of feature maps. The LSTM sequence modeling module is used to extract long-term temporal dependencies, process the feature map sequence, and output the hidden state sequence. The attention weighting module applies attention weights to the hidden state sequence, highlighting key moment features and outputting a weighted global feature vector. ; In the classification output module The predicted probability vector of fatigue state is mapped through a fully connected layer and a Softmax classification layer, and its calculation form is as follows: ； in, For the first The predicted probability vector at each time step. This is the weight matrix. is the bias term, and L is the number of fatigue state categories.

Images