A human physiological fatigue detection method based on electrocardiogram and respiratory signals

Abstract

The present invention proposes a human physiological fatigue detection algorithm based on ECG signals and respiratory signals. After obtaining human physiological signals through wearable devices, pure signals are obtained through preprocessing such as noise removal. The corresponding features of ECG signals and respiratory signals are extracted based on experience. The physiological signals at the current moment are classified and judged based on the ELM model to obtain the current fatigue state, and the results are recorded for "post-processing". The recorded historical fatigue state is added to the evaluation, and training is performed based on the long short-term memory network model to obtain the fatigue scene of the current human body. Then, the current fatigue state is secondary evaluated and corrected according to the human fatigue scene. This process is called "post-processing" to obtain fatigue detection results with higher accuracy.

Description

Technical Field

[0001] The present invention provides a method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals, belonging to the technical field of wearable devices. Background Art

[0002] The 5th International Conference on Sports Biochemistry in 1982 unified the concept of "fatigue": the body's physiological process cannot maintain the predetermined exercise intensity and the motor function temporarily decreases, which can be restored after proper rest. Fatigue can be divided into physiological fatigue and psychological fatigue: physiological fatigue is also called physical fatigue, which is generally caused by physiological overload. It refers to the phenomenon that when a person exercises, the amount of exercise or workload exceeds the load, resulting in physical weakness, muscle soreness, etc. Psychological fatigue is also called mental fatigue, which refers to a subjective state of lack of alertness and motivation. Long-term engagement in monotonous and mechanical work activities will cause the central local nerve cells to be inhibited due to continuous tension, resulting in a significant decrease in people's enthusiasm and interest in work and life, until boredom, etc. The present invention mainly studies the detection method of human physiological fatigue.

[0003] When people are engaged in physical work or physical exercise, they will experience physiological fatigue. Physiological fatigue will make the body's physiological process unable to maintain the predetermined exercise intensity, thus causing a temporary decline in motor function. For example, athletes, soldiers and other occupations will experience physiological fatigue during training. If they do not get proper rest, it may cause danger; physical workers will also experience varying degrees of physiological fatigue after long-term work, which may cause irreparable damage to life and property. The time when physiological fatigue occurs is determined by many factors, including personal physique, exercise intensity, physiological condition, external environment, etc. Different individuals will also show different performance when physiological fatigue occurs, so physiological fatigue is difficult to accurately predict.

[0004] There are two main types of fatigue detection methods at present: the first is fatigue detection based on human behavior, such as using inertial sensors to detect students’ dozing off in class, and using eye trackers to observe the eye movement characteristics of drivers while driving, to determine whether they are in a fatigue state; the second is fatigue detection based on human physiological signals, that is, by detecting physiological signals such as human electrocardiogram signals, respiratory signals, brain waves, and epidermal electromyography signals, to determine whether the current human body is fatigued. The present invention adopts the second fatigue detection method.

[0005] In 2019, Northeastern University proposed a method to judge the physiological fatigue of miners based on human ECG signals and epidermal EMG signals (Chen S, Xu K, Yao X, et al. Information fusion and multi-classifier system for miner fatigue recognition in plateau environments based on electrocardiography and electromyography signals. [J]. Computer Methods and Programs in Biomedicine, 2021, 211: 106451.) The team took ECG signals and epidermal EMG signals in fatigue and non-fatigue states from 55 miners as the basic signals for fatigue analysis, and performed filtering, R wave monitoring and other preprocessing. The team performed time domain analysis and frequency domain analysis on the ECG signals, extracted RR cycle sequence, R wave peak and power spectrum parameters as features, and mainly used time-frequency analysis methods for epidermal EMG signals, extracting median frequency, maximum frequency, average frequency, average power and total power as features. The fatigue recognition model was established by constructing classifiers based on ECG and EMG indicators using support vector machine, random forest and XGBoost machine learning algorithms. The accuracy of the classifiers constructed by the three machine learning algorithms were 89.47%, 78.95% and 89.47% respectively.

[0006] However, this method has disadvantages:

[0007] (1) The epidermal electromyographic signal used requires that all four limbs be worn. If real-time monitoring by wearable devices is achieved, it will have a significant impact on the subject's physical activities;

[0008] (3) The algorithm of this technology is not accurate enough. It divides the human body into two states: "fatigue" and "non-fatigue". It only determines whether fatigue occurs, but cannot classify the fatigue state more accurately. Summary of the invention

[0009] The present invention designs a method for monitoring human physiological fatigue based on electrocardiogram and respiratory signals, uses wearable devices to monitor human physiological data in real time, and uses machine learning algorithms to analyze the data to determine the current physiological fatigue state of the human body. Wearable physiological monitoring equipment is relatively light and can be worn during human exercise. It monitors the human body without affecting work and training. Combined with the physiological fatigue assessment algorithm, it monitors the human body state, thereby reducing or avoiding the losses caused by physiological fatigue.

[0010] To solve the following problems:

[0011] (1) Reduce the impact of wearable devices on subjects’ activities without affecting the accuracy of physiological fatigue detection

[0012] Prior art 1 uses a data collection and analysis method based on ECG signals and epidermal electromyography signals. The equipment used in this method has a great impact on the subject's physical activities. Prior art 2 only uses ECG signals as the basis for judging physiological fatigue, and the accuracy is relatively low. The present invention proposes to judge human physiological fatigue based on ECG signals and respiratory signals by combining multi-source physiological data fusion technology. This not only reduces the impact of wearable devices on the subject's movement, but also improves the accuracy of single ECG signal detection.

[0013] (2) Algorithm Optimization

[0014] On the basis of the prior art, the present invention optimizes the physiological fatigue detection algorithm, classifies the physiological characteristics based on the ELM network, and then judges the current fatigue scene of the subject based on the LSTM network model according to the historical fatigue state changes. The fatigue scene includes "physical energy period" (no fatigue), "exertion period" (fatigue gradually deepens), "exhaustion period" (continuous fatigue), and "recovery period" (fatigue gradually slows down). According to the scene, the physiological fatigue detection result is "post-processed" to correct the result of the physiological fatigue detection of the ELM network and increase the accuracy of physiological fatigue detection. Not only is it judged whether fatigue occurs, but also the results of the classification algorithm are set to three categories: "no fatigue", "general fatigue", and "very fatigue", so as to accurately grasp the current fatigue state of the subject.

[0015] The specific technical solution of the present invention is:

[0016] A method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals comprises the following steps:

[0017] S1. Data preprocessing

[0018] ECG noise mainly includes baseline drift, power frequency noise and motion artifacts. Baseline drift and power frequency noise are removed by Butterworth filter and notch filter. To remove motion artifacts, the motion artifact noise must be located first. The location is divided into two steps: motion artifact noise intensity estimation and motion artifact position detection.

[0019] The noise intensity detection algorithm is used to analyze and evaluate the motion artifact noise intensity of the ECG signal. The intensity evaluation result ranges from [0 to 1]. Values ​​close to 0 and 1 indicate slight and strong noise interference, respectively.

[0020] The motion artifact noise intensity estimation method is

[0021]

[0022] Among them, σ noise represents the estimated value of motion artifact noise intensity, f s represents the sampling frequency of the ECG signal, L represents the decomposition level of the wavelet transform, σ i Represents the i-th level wavelet detail coefficient The absolute median difference estimator of .

[0023] The calculation method of wavelet transform L is:

[0024]

[0025] Among them, f n is the noise frequency estimate, which is 14 Hz.

[0026] σ i The calculation formula is:

[0027]

[0028] According to the above noise intensity detection algorithm, the motion artifact noise intensity estimation value σ of the current detection information is obtained: noise , based on this value, the motion artifact noise is located.

[0029] Motion artifact position detection uses signal variance transform SVT to transform the signal, and then uses an adaptive threshold to locate and analyze the window where the noise signal may exist. The specific method is as follows:

[0030] Based on the noise intensity estimate σ noise , calculate the window length w of SVT transform

[0031] w=σ noise ×(w u -w l )+w l (4)

[0032] Among them, the value range of the window length w is [w l , w u ], in milliseconds.

[0033] Divide the detected ECG signal into windows and perform SVT transformation:

[0034]

[0035] Among them, y t represents the measured value of the ECG signal at time t, and μ represents the average of the measured values ​​of the ECG signal in the time period [i, i+w].

[0036] Select two upper and lower thresholds (upper threshold T u, lower threshold T l ) Select the window of motion artifacts, set M c represents all motion artifact windows, namely:

[0037] M C ={m j |T l <v j <T u , j = 1, 2, ..., N}

[0038] The methods for determining the upper and lower adaptive thresholds are as shown in (6) and (7).

[0039] T l =μ v -γ l *σ v , T u =μ v +γ u *σ v (6)

[0040] γ l =σ noise *(0.45-0.15)+0.15

[0041] γ u =σ noise *(1.5-0.75)+0.75 (7)

[0043] Among them, μ v and σ v They represent the mean and standard deviation of the SVT signal v respectively.

[0044] The SVT signal window in the adaptive interval will be marked as a motion artifact and added to the set M c In the subsequent filtering process.

[0045] The Savitzky-Golay filtering method is used to fit the original signal X[n] based on the least squares fitting method to obtain the noise fitting polynomial:

[0046]

[0047] The fitted polynomial acts as a smooth filter for the noisy signal. The noisy fitted signal is subtracted from the original signal, and the fitting residual is:

[0048]

[0049] Among them, M is the number of unilateral signals, N is the polynomial order, in order to make the fitting residual C N The minimum, its partial derivative with respect to the parameter is zero, so we can get:

[0050]

[0051] Right now:

[0052]

[0053] From formula (10), we can know that by determining the variables M, N and the original signal data X[n], we can get the noise fitting polynomial p(n). The difference between the original signal X[n] and the polynomial p(n) can get the pure ECG signal.

[0054] S2. Feature extraction

[0055] Based on the noise processing methods such as filtering in the previous step, pure ECG and respiratory signals are extracted from the raw data collected on site. Based on expert experience, manual feature extraction is performed on the signal, and feature comparison and screening are performed. The final retained features are shown in Table 1 below.

[0056] Table 1 Feature names and corresponding descriptions

[0057]

[0058]

[0059] There are seven features in Table 1. The first six features are extracted from the ECG signal, and the last one is extracted from the respiratory signal. The R peak is the peak point with the highest potential value in the electrical signal generated by a heartbeat; the RR interval represents the time interval between two adjacent R peaks.

[0060] S3. Model building and training

[0061] The construction of the physiological fatigue detection model is divided into two parts:

[0062] The first part classifies and judges the physiological signals at the current moment based on the ELM model to obtain the current physiological fatigue state. The features are classified into three possible fatigue conditions: "not tired", "generally tired", and "very tired", and the results are recorded for "post-processing". This process only makes judgments based on the physiological signals at the current moment, and the accuracy is relatively low.

[0063] The second part adds the recorded historical fatigue status to the evaluation. Based on the long-short term memory network model, the current fatigue scene of the human body can be obtained through training. The fatigue scene is also divided into four situations: "full physical strength period" (no fatigue), "exertion period" (fatigue gradually deepens), "exhaustion period" (continuous fatigue), and "recovery period" (fatigue gradually eases). Then, the current physiological fatigue state is evaluated and corrected again according to the human fatigue scene. This process can be called "post-processing".

[0064] 1) Extreme Learning Machine Model Training Method

[0065] The feature vectors extracted from fatigue data of different individuals are used as samples and divided into training set and test set. The training set is used for ELM model training, and based on the generalization ability of the neural network, the test set is classified to verify the reliability of the classification model results. The specific steps of model training and testing are as follows:

[0066] (1) Establishing an artificial neural network structure model

[0067] Analysis sample data structure: There are N training samples, the feature dimension is n, and the sample input matrix X

[0068]

[0069] The sample output dimension is m, and the output matrix is ​​T

[0070]

[0071] According to the sample input parameter feature dimension n, the number of input layer neurons is determined to be n. This model is a multi-classification problem, and the probability of fatigue data results is 3, so the number of output layer neurons is set to 3. :

[0072] (2) Setting the number of hidden layer nodes and regularization parameters

[0073] Select different numbers of hidden layer nodes L and regularization parameters C, perform multiple model training, and compare and select the most appropriate parameters.

[0074] (3) Set the input weight and input bias of each hidden layer node

[0075] Randomly generate several weight matrices W and input biases b, and select the best model. The hidden layer input weight matrix W is

[0076]

[0077] The hidden layer input bias matrix b is

[0078] b=(b l … b L ) 1×L (15)

[0079] (4) Calculate the hidden layer output H

[0080] By selecting a suitable activation function g(x), the hidden layer output H can be obtained.

[0081]

[0082] (5) Calculate the hidden layer output weight β

[0083] According to the output weight matrix calculation method of RELM, the hidden layer output weight β is calculated

[0084]

[0085] By changing the number of hidden layer nodes L and the regularization parameter C and regenerating the weight matrix W and the input bias b, multiple independent experiments are conducted to obtain different hidden layer output weights β to form complete neural network parameters. The sample test set is brought into the network for testing, and the accuracy and error of the samples for different models are calculated. The model with the best effect can be used for preliminary detection of physiological fatigue.

[0086] 2) Long Short-Term Memory Network Model Training Method

[0087] Based on the LSTM network model, the current fatigue scenario of the individual is inferred, namely, "full physical strength period", "exertion period", "continuous fatigue period" or "physical recovery period", and the evaluation result of the ELM model is corrected according to the result;

[0088] The subjects' data were divided into windows of ten minutes in length. The window data labels were calibrated according to the subjects' subjective feelings about the fatigue scenario and used as samples for model training.

[0089] The specific steps of model training and testing are as follows:

[0090] (1) Determine LSTM network model parameters

[0091] The establishment of the LSTM model requires the determination of key parameters: input dimension n, output data dimension m, and number of hidden nodes in the neural network L. The input of the network model is the historical fatigue state, and the data is in the form of a digital sequence of length n, which represents the change in the fatigue state of the subject from several minutes ago to the present moment. The output contains two pieces of information, one is the fatigue scene the subject is currently in, and the other is the current fatigue state of the subject after "post-processing". The number of hidden nodes is trained through multiple experiments to select the most appropriate value.

[0092] (2) Training LSTM network model

[0093] According to the subjective feelings of the subjects, each part of the collected physiological data is marked with the corresponding fatigue scene and fatigue state to make training samples. It is divided into training set and test set. The training set is used to train the LSTM network model, and the test set is brought into the trained model to obtain the accuracy to evaluate the quality of the network model.

[0094] (3) Adjust the network model

[0095] By adjusting the input historical fatigue state duration, the number of hidden nodes and other values, the network accuracy is adjusted to obtain the network structure with the highest accuracy.

[0096] The present invention provides a method for detecting human physiological fatigue using electrocardiogram signals and respiratory signals. The method uses wearable devices to collect physiological signals and detect human physiological fatigue in real time. This can prevent athletes, soldiers and other professionals from developing physiological fatigue during training and causing danger due to lack of proper rest. It can also prevent drivers, doctors and precision instrument operators from developing fatigue after long-term work and causing irreparable losses to life and property. The specific effects of the invention are as follows:

[0097] 1. The present invention abandons the more complicated epidermal electromyography signal acquisition equipment used in the prior art, and uses wearable devices based on electrocardiogram signals and respiratory signals to collect information and analyze the physiological fatigue generated by the human body during exercise, thereby reducing the discomfort caused by the subjects when wearing the wearable devices.

[0098] 2. The present invention adopts the structure of ELM network plus LSTM network "post-processing" and optimizes the algorithm, so that the fatigue detection accuracy of the three-classification model of "not tired", "generally tired" and "very tired" is the same as that of the two-classification model, reaching 89%, and the detected human fatigue state is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 This is the overall flow chart of physiological fatigue detection based on wearable devices of the present invention;

[0100] Figure 2 This is the motion artifact noise intensity estimation process of the present invention;

[0101] Figure 3 is the original ECG signal in the embodiment;

[0102] Figure 4 is the signal after SVT transformation in the embodiment;

[0103] Figure 5 Motion artifacts in the ECG are located for thresholding in an embodiment;

[0104] Figure 6 The electrocardiogram during running in the embodiment;

[0105] Figure 7 The electrocardiogram after noise removal in the embodiment;

[0106] Figure 8 This is a flow chart of physiological fatigue detection in the embodiment;

[0107] Fig. 9 This is the general model diagram of the ELM;

[0108] Fig.10 It is a structural diagram of the ELM model in the embodiment;

[0109] Fig.11 The test equipment in the embodiment includes an ECG signal acquisition device (left), a respiratory acquisition device (middle) and a Bluetooth router (right);

[0110] Fig.12 It is the confusion matrix of the prediction result of the ELM model in the embodiment;

[0111] Fig.13 It is the confusion matrix of the LSTM model prediction result in the embodiment. DETAILED DESCRIPTION

[0112] The method of the present invention can provide a method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals. According to the physiological signals collected by the wearable device, the physiological fatigue generated by the human body during exercise is analyzed. The overall flow chart of the method is shown in FIG. Figure 1 The invention scheme is introduced below from the aspects of data preprocessing, feature extraction, model training, experimental verification, data analysis and conclusion.

[0113] A method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals comprises the following steps:

[0114] S1. Data preprocessing

[0115] Since the respiratory signal is the thoracic pressure value, the effective information extracted from it is the respiratory rate, and the noise has little effect, so the main content is to remove the noise in the ECG signal data. ECG noise mainly includes baseline drift, power frequency noise and motion artifacts, among which baseline drift and power frequency noise can be removed by Butterworth filter and notch filter, so the denoising content of the present invention is the removal of motion artifacts.

[0116] When the wearable device monitors the ECG signal, the subject may be in motion or at rest, so the motion artifact noise must be located first. The location is divided into two steps: motion artifact noise intensity estimation and motion artifact position detection.

[0117] The noise intensity detection algorithm is used to analyze and evaluate the motion artifact noise intensity of the ECG signal. The intensity evaluation result ranges from [0 to 1]. Values ​​close to 0 and 1 indicate slight and strong noise interference, respectively. It is robust to noisy ECG signals with different sampling frequencies and noise levels.

[0118] Figure 2 Motion artifact noise estimation process.

[0119] The motion artifact noise intensity estimation method is:

[0120]

[0121] Among them, σ noise represents the estimated value of motion artifact noise intensity, f s represents the sampling frequency of the ECG signal, L represents the decomposition level of the wavelet transform, σ i Represents the i-th level wavelet detail coefficient The absolute median difference estimator of .

[0122] The calculation method of wavelet transform L is:

[0123]

[0124] Among them, f n It is the estimated value of noise frequency, which is generally taken as 14Hz.

[0125] σ i The calculation formula is

[0126]

[0127] According to the above noise intensity detection algorithm, the motion artifact noise intensity estimation value σ of the current detection information can be obtained: noise , based on this value, the motion artifact noise is located.

[0128] Motion artifact position detection mainly uses signal variance transform (SVT) to transform the signal, and then uses adaptive threshold to locate and analyze the window where the noise signal may exist. The specific method is as follows:

[0129] Based on the noise intensity estimate σ noise , calculate the window length w of SVT transform

[0130] w=σ noise ×(w u -w l )+w l (4)

[0131] Among them, the value range of the window length w is [w l , w u ], in this experiment, the value is [25, 50], the unit is milliseconds.

[0132] Divide the detected ECG signal into windows and perform SVT transformation

[0133]

[0134] Among them, y t represents the measured value of the ECG signal at time t, and μ represents the average of the measured values ​​of the ECG signal in the time period [i, i+w].

[0135] Figure 3 and Figure 4 This is a typical example of the original ECG signal and the signal after SVT transformation.

[0136] Select two upper and lower thresholds (upper threshold T u , lower threshold T l ) Select the window of motion artifacts, set M C represents all motion artifact windows, i.e.

[0137] M C ={m j |T l <v j <T u , j = 1, 2, ..., N}

[0138] The methods for determining the upper and lower adaptive thresholds are as shown in (6) and (7).

[0139] T l =μ v -γ l *σ v , T u =μ v +γ u *σ v (6)

[0140] γ l =σ noise *(0.45-0.15)+0.15

[0141] γ u =σ noise *(1.5-0.75)+0.75 (7)

[0143] Among them, μ v and σ v They represent the mean and standard deviation of the SVT signal v respectively.

[0144] The threshold selects the signal window of motion artifacts, such as Figure 5 shown.

[0145] The SVT signal window in the adaptive interval will be marked as a motion artifact and added to the set M C In the subsequent filtering process.

[0146] The Savitzky-Golay filtering method has lower time complexity than the Kalman filtering method and can adapt to signal data with changing window length. It is more suitable to use this method to remove motion artifacts in the signal.

[0147] The core idea of ​​Savitzky-Golay filtering is to fit the original signal X[n] based on the least squares fitting method to obtain the noise fitting polynomial:

[0148]

[0149] The fitting polynomial acts as a smooth noise signal filter. Subtracting the noise fitting signal from the original signal can quickly and easily remove motion artifact noise in the ECG signal. The fitting residual is:

[0150]

[0151] Among them, M is the number of unilateral signals, N is the polynomial order, in order to make the fitting residual C N The minimum, its partial derivative with respect to the parameter is zero, so we can get:

[0152]

[0153] Right now:

[0154]

[0155] From formula (10), we can know that by determining the variables M, N and the original signal data X[n], we can get the noise fitting polynomial p(n). The original signal X[n] is subtracted from the polynomial p(n) to get a pure ECG signal. It can also be seen that the Savitzky-Golay filter can have any length, so it is beneficial for processing ECG signals with lower sampling frequencies.

[0156] The following pictures are the electrocardiograms while running Figure 6 and the ECG after noise removal Figure 7 contrast.

[0157] S2. Feature extraction

[0158] Based on the above-mentioned noise processing methods such as filtering, pure ECG and respiratory signals are extracted from the original data collected on site. According to expert experience, manual feature extraction is performed on the signal, and feature comparison and screening are performed. The final retained features are shown in Table 1 below.

[0159] Table 1 Feature names and corresponding descriptions

[0160]

[0161] There are seven features in Table 1. The first six features are extracted from the ECG signal, and the last one is extracted from the respiratory signal. The R peak is the peak point with the highest potential value in the electrical signal generated by a heartbeat; the RR interval represents the time interval between two adjacent R peaks.

[0162] S3. Model training

[0163] Physiological fatigue detection flow chart Figure 8 shown.

[0164] The physiological fatigue detection model of the present invention is built in two parts:

[0165] The first part classifies and judges the physiological signals at the current moment based on the ELM model to obtain the current physiological fatigue state. The features are classified into three possible fatigue conditions: "not tired", "generally tired", and "very tired", and the results are recorded for "post-processing". This process only makes judgments based on the physiological signals at the current moment, and the accuracy is relatively low.

[0166] The second part adds the recorded historical fatigue status to the evaluation. Based on the long-short term memory network model, the current fatigue scene of the human body can be obtained through training. The fatigue scene is also divided into four situations: "full physical strength period" (no fatigue), "exertion period" (fatigue gradually deepens), "exhaustion period" (continuous fatigue), and "recovery period" (fatigue gradually eases). Then, the current physiological fatigue state is evaluated and corrected again according to the human fatigue scene. This process can be called "post-processing".

[0167] 1) Extreme Learning Machine Model Training Method

[0168] In 2004, Professor Huang Guangbin proposed the Extreme Learning Machine (ELM) model to solve the problems faced by traditional neural networks. ELM is a single hidden layer feedforward neural network, which consists of three layers: input layer, hidden layer, and output layer. Its basic structure is as follows: Fig. 9 As shown in the figure, the parameters of the hidden layer are randomly generated, and the parameters of the output layer are obtained based on the least squares method. This method is iteration-free and avoids the disadvantage of back propagation falling into the local optimum. It also has the advantages of faster speed and stronger generalization ability.

[0169] Traditional ELM is based on the empirical risk minimization criterion (Empirical Risk Minimization, ERM), which approaches the minimum value of the true expected risk by solving the minimum value of the empirical risk. In actual application, the model based on the empirical risk minimization criterion has the problem of inconsistency between learning accuracy and prediction accuracy, and may have the problem of overfitting. This model can easily mislead the final judgment. In response to this, Professor Deng Wanyu and others proposed to add a regularization term to the ELM to represent the complexity of the model, reduce overfitting, and make the training model more generalizable, that is, Regularized ELM (Regularized ELM, RELM).

[0170] The output weight matrix representation of RELM is shown in Formula 11:

[0171]

[0172] Among them, H represents the hidden layer output matrix, C represents the regularization parameter, L represents the number of hidden layer neurons, N represents the number of training samples, T is the expected output vector, and I represents the identity matrix.

[0173] This experiment uses the feature vectors extracted from fatigue data of different individuals as samples and divides them into training sets and test sets. The training set is used for ELM model training, and based on the generalization ability of the neural network, the test set is classified to verify the reliability of the classification model results. The specific steps of model training and testing are as follows:

[0174] (1) Establishing an artificial neural network structure model

[0175] Analysis sample data structure: There are N training samples, the feature dimension is n, and the sample input matrix X

[0176]

[0177] The sample output dimension is m, and the output matrix is ​​T

[0178]

[0179] According to the sample input parameter feature dimension n, the number of input layer neurons is set to n. This model is a multi-classification problem, and the probability of fatigue data results is 3, so the number of output layer neurons is set to 3. Fig.10 As shown:

[0180] (2) Setting the number of hidden layer nodes and regularization parameters

[0181] Select different numbers of hidden layer nodes L and regularization parameters C, perform multiple model training, and compare and select the most appropriate parameters.

[0182] (3) Set the input weight and input bias of each hidden layer node

[0183] Randomly generate several weight matrices W and input biases b, and select the best model. The hidden layer input weight matrix W is

[0184]

[0185] The hidden layer input bias matrix b is

[0186] b=(b l … b L ) 1×L (15)

[0187] (4) Calculate the hidden layer output H

[0188] By selecting a suitable activation function g(x), the hidden layer output H can be obtained.

[0189]

[0190] (5) Calculate the hidden layer output weight β

[0191] According to the output weight matrix calculation method of RELM, the hidden layer output weight β is calculated

[0192]

[0193] By changing the number of hidden layer nodes L and the regularization parameter C and regenerating the weight matrix W and the input bias b, multiple independent experiments are conducted to obtain different hidden layer output weights β to form complete neural network parameters. The sample test set is brought into the network for testing, and the accuracy and error of the samples for different models are calculated. The model with the best effect can be used for preliminary detection of physiological fatigue.

[0194] 2) Long Short-Term Memory Network Model Training Method

[0195] Long Short Term Memory (LSTM) is a type of recurrent neural network used for time series data prediction. LSTM is mainly used to solve the long-term dependency problem of traditional RNN. For traditional RNN, as the sequence interval lengthens, due to problems such as gradient explosion or gradient disappearance, the model becomes unstable during training or cannot be effectively learned at all.

[0196] According to the historical fatigue development trend, the present invention infers the individual's current fatigue scenario based on the LSTM network model, namely the "physical full period", "exertion period", "continuous fatigue period" or "physical recovery period", and based on the result, corrects the evaluation result of the ELM model, which can improve the accuracy of physiological fatigue detection results.

[0197] The subjects' data were divided into windows of ten minutes in length. The window data labels were calibrated according to the subjects' subjective feelings about the fatigue scenario and used as samples for model training.

[0198] The specific steps of model training and testing are as follows:

[0199] (1) Determine LSTM network model parameters

[0200] The establishment of the LSTM model requires the determination of key parameters: input dimension n, output data dimension m, and number of hidden nodes in the neural network L. The input of the network model is the historical fatigue state, and the data is in the form of a digital sequence of length n, which represents the change in the fatigue state of the subject from several minutes ago to the present moment. The output contains two pieces of information, one is the fatigue scene the subject is currently in, and the other is the current fatigue state of the subject after "post-processing". The number of hidden nodes is trained through multiple experiments to select the most appropriate value.

[0201] (2) Training LSTM network model

[0202] According to the subjective feelings of the subjects, each part of the collected physiological data is marked with the corresponding fatigue scene and fatigue state to make training samples. It is divided into training set and test set. The training set is used to train the LSTM network model, and the test set is brought into the trained model to obtain the accuracy to evaluate the quality of the network model.

[0203] (3) Adjust the network model

[0204] By adjusting the input historical fatigue state duration, the number of hidden nodes and other values, the network accuracy is adjusted to obtain the network structure with the highest accuracy.

[0205] Experimental verification:

[0206] 1) Experimental subjects

[0207] The present invention selected 10 male students from Beijing University of Science and Technology as research subjects and collected data through field tests. The basic information of the subjects is as follows: the average age is 24±2 years old, the average height is 173.2±5.1cm, and the average weight is 68.2±8.2kg. In addition, the subjects have no history of cardiovascular disease, are in good health, and have maintained at least eight hours of adequate sleep the day before the experiment. Make the subjects familiar with the experimental process to ensure the objectivity of the data. In order to complete the two contents of noise processing and human physiological fatigue detection under physiological data motion, the subjects are required to complete two tests. Subsequent processing is completed based on the signals obtained from the two tests.

[0208] 2) Experimental equipment

[0209] The equipment used in this experiment is the wearable ECG data acquisition device model BW-ECG-CHA from Shanxi Yikang Xinyue Co., Ltd., the wearable chest strap respiratory data acquisition device model FHYDCSH1 from the same company, and the Cassia S2000 Bluetooth router developed by Beijing Guihua Network Technology Co., Ltd. Fig.11 shown.

[0210] 3) Experimental process

[0211] The overall experimental process is divided into the following steps:

[0212] (1) Explain the experimental objectives and procedures to the subjects and ensure that they understand the content and significance of each step.

[0213] (2) Collect basic information of all subjects, including age, height, weight, physical health, and history of cardiovascular disease.

[0214] (3) Collect physiological signals at rest. Have the subjects wear the experimental equipment, sit quietly in a chair for ten minutes, and collect the corresponding data.

[0215] (4) Collect physiological signals during exercise. Ask the subject to repeat the sit-down-stand-up action and collect data for five minutes. After resting for twenty minutes, ask the subject to jog and collect data for five minutes.

[0216] (5) Collect physiological signals of different fatigue conditions. The subjects were asked to sit quietly for ten minutes and then run on the playground. The subjects were asked about their subjective physiological fatigue feelings every five minutes. The fatigue feelings were divided into three categories: "no fatigue", "general fatigue", and "extreme fatigue". The fatigue feelings were recorded as data labels.

[0217] Data Analysis and Conclusion:

[0218] The accuracy of the physiological fatigue detection results of the model of the present invention is divided into the accuracy of the model based on the ELM network to detect human physiological fatigue and the accuracy of physiological fatigue detection using the LSTM network "post-processing", and compared with the accuracy of the methods used in other studies. The present invention adopts the analysis method of drawing the confusion matrix, and calculates the accuracy, precision, recall and specificity of the model based on the confusion matrix.

[0219] The confusion matrix of physiological fatigue detection classification results of ELM model and LSTM model is as follows Fig.12 and Fig.13 shown.

[0220] The performance of the model is shown in Table 2

[0221] Table 2 Precision, recall and specificity of physiological fatigue detection results

[0222]

[0223] In order to compare the accuracy of the results, the classification methods of the present invention are compared with those of other studies, as shown in Table 3. It should be noted that the classification method of the present invention is a three-classification method, while most other studies only conduct a two-classification study of fatigue and non-fatigue.

[0224] Table 3 Accuracy of different classification models

[0225]

[0226] The results show that the classification method of the present invention has a low accuracy rate of 78.75% in the preliminary physiological fatigue detection based on the ELM model, but after the "post-processing" of the LSTM network, the detection accuracy rate can reach 88.89%, which is similar to the accuracy of the two-classification results of other studies and higher than the accuracy of the three-classification results of other studies. Overall, it has a good physiological fatigue recognition effect.

Claims

1. A method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals, characterized in that: The following steps are involved: S1. Data Preprocessing ECG noise includes baseline drift, power frequency noise and motion artifacts. Baseline drift and power frequency noise are removed by Butterworth filter and notch filter. To remove motion artifacts, the motion artifact noise must be located first and then filtered. S2. Feature extraction Based on the S1 data preprocessing method, pure ECG and respiratory signals are extracted from the raw data collected on site, and the signal is manually feature extracted based on expert experience to retain feature: SDNN, milliseconds, standard deviation of all RR intervals; RMSSD, milliseconds, the root mean square value of the difference between adjacent RR intervals; NN50, counts / min, the number of pairs of consecutive RR intervals separated by more than 50 ms; RR, milliseconds, mean RR interval; HR, beats / min, average heart rate; RH, millivolt, mean R peak height; RESP, breaths / min, average respiratory rate, number of breaths per minute; The first six features are extracted from the ECG signal, and the last one is extracted from the respiratory signal; the R peak is the peak point with the highest potential value in the electrical signal generated by a heartbeat; the RR interval represents the time interval between two adjacent R peaks; S3. Model building and training The construction of physiological fatigue detection model is divided into two parts: The first part classifies and judges the physiological signals at the current moment based on the ELM model to obtain the current physiological fatigue state. The features are classified into three possible fatigue conditions: "not fatigued", "generally fatigued", and "very fatigued", and the results are recorded for "post-processing". This process makes judgments based only on the physiological signals at the current moment. The second part adds the recorded historical fatigue status to the evaluation, and trains based on the long-short-term memory network model to obtain the current fatigue scene of the human body. The fatigue scene is also divided into four situations: "full physical strength period", "exertion period", "exhaustion period", and "recovery period". Then, the current physiological fatigue state is secondary evaluated and corrected according to the human body fatigue scene. This process is called "post-processing". The training adopts the extreme learning machine model training method and the long short-term memory network model training method.

2. The method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals according to claim 1, characterized in that: In S1, the motion artifact noise is located, including motion artifact noise intensity estimation and motion artifact position detection; The noise intensity detection algorithm is used to analyze and evaluate the noise intensity of motion artifacts in ECG signals. The intensity evaluation result ranges from [0 to 1]. Values ​​close to 0 and 1 indicate slight and strong noise interference, respectively. The motion artifact noise intensity estimation method is: Among them, σ noise represents the estimated value of motion artifact noise intensity, f s represents the sampling frequency of the ECG signal, L represents the decomposition level of the wavelet transform, σ i Represents the i-th level wavelet detail coefficient The absolute median difference estimator of ; The calculation method of wavelet transform L is: Among them, f n is the noise frequency estimate, which is 14 Hz; σ i The calculation formula is: According to the above noise intensity detection algorithm, the motion artifact noise intensity estimation value σ of the current detection information is obtained noise , based on this value, the motion artifact noise is located; Motion artifact position detection uses signal variance transform SVT to transform the signal, and then uses an adaptive threshold to locate and analyze the window where the noise signal may exist. The specific method is as follows: Based on the noise intensity estimate σ noise , calculate the window length w of the SVT transform: w=σ noise ×(w u -w l )+w l (4) Among them, the value range of the window length w is [w l , w u ], in milliseconds; Divide the detected ECG signal into windows and perform SVT transformation: Among them, y t represents the measured value of the ECG signal at time t, and μ represents the average value of the measured value of the ECG signal in the time period [i, i+w]; Select two upper and lower thresholds, namely the upper threshold T u , lower threshold T l , select the window of motion artifacts, set M C represents all motion artifact windows, namely: M C ={m j |T l <v j <T u ,j=1,2,...,N} Among them, the methods for selecting the upper and lower adaptive thresholds are as shown in (6) and (7); T l =μ v -c l *s v ,T u =μ v +g u *s v (6) c l =s noise *(0.455-0.15))+0.15 c u =s noise *(1.55-0.75))+0.75 (7) Among them, μ v and σ v Respectively represent the mean and standard deviation of the SVT signal v; The SVT signal window in the adaptive interval will be marked as a motion artifact and added to the set M C In the subsequent filtering process.

3. The method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals according to claim 2, characterized in that: In S1, the Savitzky-Golay filtering method is used to fit the original signal X[n] based on the least squares fitting method to obtain the noise fitting polynomial: The fitted polynomial acts as a smooth filter for the noisy signal. The noisy fitted signal is subtracted from the original signal, and the fitting residual is: Among them, M is the number of unilateral signals, N is the polynomial order, in order to make the fitting residual C N The minimum, its partial derivative with respect to the parameter is zero, so we can get: Right now: From formula (10), we can know that by determining the variables M, N and the original signal data X[n], we can obtain the noise fitting polynomial p(n); the pure ECG signal can be obtained by subtracting the original signal X[n] from the polynomial p(n).

4. The method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals according to claim 1, characterized in that: The ELM model training method described in S3 is specifically as follows: The feature vectors extracted from fatigue data of different individuals are used as samples and divided into training set and test set. The training set is used for ELM model training, and the test set is classified based on the generalization ability of the neural network to verify the reliability of the classification model results. The specific steps of model training and testing are as follows: (1) Establishing an artificial neural network structure model Analysis sample data structure: There are N training samples, the feature dimension is n, and the sample input matrix X is: The sample output dimension is m, and the output matrix is ​​T: According to the characteristic dimension n of the sample input parameter, the number of neurons in the input layer is determined to be n. This model is a multi-classification problem, and the probability of fatigue data results is 3, so the number of neurons in the output layer is set to 3; (2) Setting the number of hidden layer nodes and regularization parameters Select different numbers of hidden layer nodes L and regularization parameters C, perform multiple model training, and compare and select the most appropriate parameters; (3) Set the input weight and input bias of each hidden layer node Randomly generate several weight matrices W and input bias b, and select the model with the best effect; The hidden layer input weight matrix W is: The hidden layer input bias matrix b is: b=(b1 … b L ) 1×L (15) (4) Calculate the hidden layer output H By selecting a suitable activation function g(x), the hidden layer output H can be obtained; (5) Calculate the hidden layer output weight β According to the output weight matrix calculation method of RELM, the hidden layer output weight β is calculated: By changing the number of hidden layer nodes L and the regularization parameter C and regenerating the weight matrix W and the input bias b, multiple independent experiments are conducted to obtain different hidden layer output weights β to form a complete neural network parameter; the sample test set is brought into the network for testing, and the accuracy and error of the samples for different models are calculated; the model with the best effect is selected and can be used for preliminary detection of physiological fatigue.

5. The method for detecting human physiological fatigue based on electrocardiogram signals and respiratory signals according to claim 1, characterized in that: The long short-term memory network model training method described in S3 is specifically as follows: Based on the LSTM network model, the current fatigue scenario of the individual is inferred, that is, "full physical strength period", "exertion period", "continuous fatigue period" or "physical recovery period", and the evaluation results of the ELM model are corrected according to the results; The data of the subjects are divided into windows of ten minutes in length. The window data labels are calibrated according to the subjective feelings of the subjects in the fatigue scenario and used as samples for model training. The specific steps of model training and testing are as follows: (1) Determine LSTM network model parameters The establishment of LSTM model requires the determination of key parameters: input dimension n, output data dimension m, and number of hidden nodes in the neural network L; the input of the network model is the historical fatigue state, and the data is in the form of a digital sequence of length n, which represents the change of the fatigue state of the subject from several minutes ago to the present moment; the output contains two pieces of information, one is the fatigue scene the subject is currently in, and the other is the current fatigue state of the subject after "post-processing"; the number of hidden nodes is trained through multiple experiments to select the most appropriate value; (2) Training LSTM network model According to the subjective feelings of the subjects, each part of the collected physiological data is marked with the corresponding fatigue scene and fatigue state to make a training sample; divided into a training set and a test set, where the training set is used to train the LSTM network model, and the test set is brought into the trained model to obtain the accuracy rate to evaluate the quality of the network model; (3) Adjust the network model By adjusting the input history fatigue state duration, the number of hidden nodes, and the network accuracy, the network structure with the highest accuracy can be obtained.

Images