Electric power operator fatigue state detection method based on multiple physiological signals

Abstract

The invention discloses a method for detecting the fatigue state of an electric power operator based on multiple physiological signals. The method belongs to the technical field of electric power protection, and comprises the following steps: based on a CNN network model, electroencephalogram signal and electro-oculogram signal fatigue features; inputting the extracted features of the electroencephalogram signals and the extracted features of the electro-oculogram signals into a Transform model, and fusing the physiological signal fatigue features. According to the method for detecting the fatigue state of the electric power operating personnel based on the multiple physiological signals, the accuracy and effectiveness of feature extraction are improved, and the defects of traditional deep learning in the aspect of fatigue detection are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power protection, and in particular to a method for detecting fatigue status of electric power workers based on multiple physiological signals. Background Art

[0002] The power industry is a high-risk industry. Power workers not only need to deal with high-voltage electricity, dangerous equipment, equipment operation and maintenance, but also need to ensure the stability of power supply. Therefore, power workers need to maintain a high level of alertness and concentration. However, long working hours, night shifts and high-intensity workloads can cause fatigue among workers, which seriously endangers their safety and productivity.

[0003] 1) Traditional fatigue state detection methods are mostly based on facial video images, extracting facial features such as blinking and yawning to characterize fatigue, but face-based methods are easily affected by lighting, and performance may drop by 30% in the case of sunlight reflection and glasses reflection. In addition, objects or masks that block the face may cause the failure of facial video image detection. At the same time, facial activities are subjective, so these methods have certain limitations.

[0004] 2) Fatigue detection based on a single physiological signal may lead to reduced accuracy and increased risk of false positives and false negatives. Fatigue is a multidimensional physiological and psychological phenomenon, and a single physiological signal cannot fully characterize the complexity of the fatigue state. Summary of the invention

[0005] The purpose of the present invention is to provide a method for detecting fatigue status of power workers based on multiple physiological signals, which improves the accuracy and effectiveness of feature extraction and makes up for the shortcomings of traditional deep learning in fatigue detection.

[0006] To achieve the above object, the present invention provides a method for detecting fatigue status of electric power workers based on multiple physiological signals, the steps comprising:

[0007] S1. Extract fatigue features of multiple physiological signals based on a CNN network model, where the multiple physiological signals include electroencephalogram signals and electrooculogram signals;

[0008] S2. Input the extracted features of EEG and EOG signals into the Transformer model to fuse the fatigue features of physiological signals.

[0009] Preferably, the specific contents of step S1 include:

[0010] S11, inputting the EEG signal and the EOG signal into the CNN network as two modal data, wherein the size of the EEG signal is 1416000*17; and the size of the EOG signal is 885000*7;

[0011] S12, extracting the spatial features of EEG and EOG signals through spatial scale convolution;

[0012] S13. Use group convolution to perform time scale convolution to extract the timing features of EEG signals and EOG signals.

[0013] Preferably, the step 2 of fusing fatigue features of physiological signals based on the Transformer model includes:

[0014] S21, the first layer is the input layer. The EEG fatigue features and the EEG fatigue features obtained in step S1 are concatenated in the first dimension before being input into the model to form a new vector, which is input into the model;

[0015] S22, the second layer is the coding layer. The input sequence is transformed into three matrices Q, K, and V through linear transformation. The calculation formula is:

[0016] Q=W Q X#(1)

[0017] K=W K X#(2)

[0018] V=W V X#(3)

[0019] Calculate the dot product of Q and K to get the weight matrix, multiply the weight matrix by the value matrix V to get the output after weighted summation. The formula is

[0020]

[0021] Perform convolution operation on the EEG signal and the EOG signal, convert the output tensor dimension, and then concatenate them, and repeat step S21;

[0022] S23, set the encoder layer value and output its hidden layer representation;

[0023] S24, introducing residual connection and layer normalization technology, and then performing average pooling on the first dimension to transform the features of the EEG signal and the EOG signal into a vector to obtain a feature vector;

[0024] S25. Map the data dimension of the obtained feature vector through a multi-layer perceptron or a linear layer.

[0025] Preferably, the step S21 includes setting the hidden layer dimension and the number of encoding layers of the fully connected network when inputting the model into the input layer.

[0026] Preferably, the data dimensions in step S25 include dimensions of two states: normal state and fatigue state.

[0027] Therefore, the present invention adopts the above-mentioned method for detecting fatigue status of power workers based on multiple physiological signals, extracts the features of EEG signals and EOG signals, then splices the features of the two modalities, and puts the spliced ​​multimodal features into the Transformer fusion model, thereby realizing local interaction between the two modalities, and obtaining multimodal fusion features containing features of different physiological signals for fatigue classification, thereby improving the accuracy and effectiveness of feature extraction and making up for the shortcomings of traditional deep learning in fatigue detection.

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of a method for fusion of fatigue features of multiple physiological signals based on CNN-Transformer according to an embodiment of a method for detecting fatigue status of electric power workers based on multiple physiological signals of the present invention;

[0030] Figure 2 It is a feature fusion graph based on Transformer in an embodiment of a method for detecting fatigue state of electric power workers based on multiple physiological signals of the present invention;

[0031] Figure 3 This is a comparison result after removing the feature fusion module in an embodiment of a method for detecting fatigue status of electric power workers based on multiple physiological signals of the present invention. DETAILED DESCRIPTION

[0032] Example

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0034] The present invention provides a method for detecting fatigue status of electric power workers based on multiple physiological signals. Figure 1 , the steps include:

[0035] S1. Extract fatigue features of eye and brain signals based on CNN network model, including:

[0036] S11, inputting the EEG signal and the EOG signal into the CNN network as two modal data, wherein the size of the EEG signal is 1416000*17; and the size of the EOG signal is 885000*7;

[0037] S12, extracting the spatial features of EEG and EOG signals through spatial scale convolution;

[0038] S13. Use group convolution to perform time scale convolution to extract the timing features of EEG signals and EOG signals.

[0039] S2. Input the extracted EEG and EOG features into the Transformer model and fuse the fatigue features of physiological signals. Figure 2 , including:

[0040] S21, the first layer is the input layer, and the EEG fatigue features and the EEG fatigue features obtained in step S1 are concatenated on the first dimension before being input into the model to form a new vector, which is input into the model. When inputting into the model, the hidden layer dimension and the number of encoding layers of the fully connected network are also set;

[0041] S22, the second layer is the coding layer. The input sequence is transformed into three matrices Q, K, and V through linear transformation. The calculation formula is:

[0042] Q=W Q X#(5)

[0043] K=W K X#(6)

[0044] V=W V X#(7)

[0045] Calculate the dot product of Q and K to get the weight matrix, multiply the weight matrix by the value matrix V to get the output after weighted summation. The formula is

[0046]

[0047] Perform convolution operation on the EEG signal and the EOG signal, convert the output tensor dimension, and then concatenate them, and repeat step S21;

[0048] S23, set the encoder layer value and output its hidden layer representation;

[0049] S24, introducing residual connection and layer normalization technology, and then performing average pooling on the first dimension to transform the features of the EEG signal and the EOG signal into a vector to obtain a feature vector;

[0050] S25. Mapping the obtained feature vector to data dimensions through a multi-layer perceptron or a linear layer, where the data dimensions include dimensions of two states: normal state and fatigue state.

[0051] Example 1

[0052] A set of data sets is selected, 70% of the data is used as the training set, and 30% of the data is used as the test set. The experimental group and the control group use the same environment configuration and are trained until convergence. The test results on the SEED-VIG data set are shown in Table 1.

[0053] Table 1 Comparison of model results

[0054]

[0055] It has been verified that the method proposed in this paper has higher accuracy, recall rate and F1Score than other methods, which means that this method can more accurately identify and predict targets in classification or evaluation tasks, improving the performance and effect of the model.

[0056] Example 2

[0057] Design an ablation experiment. In the ablation experiment, the fusion module is removed for comparison. The experimental results refer to Figure 3 ,According to the comparison results after removing the feature fusion module, ,the accuracy of the model is improved by 13.41% after using this method to process ,multiple physiological signals. ,The precision is improved by 15.89%.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for detecting fatigue status of power workers based on multiple physiological signals, characterized in that the steps include: S1. Extract fatigue features of multiple physiological signals based on a CNN network model, where the multiple physiological signals include electroencephalogram signals and electrooculogram signals; S2. Input the extracted features of EEG and EOG signals into the Transformer model to fuse the fatigue features of physiological signals.

2. The method for detecting fatigue status of electric power workers based on multiple physiological signals according to claim 1, characterized in that: The specific contents of step S1 include: S11, inputting EEG signals and EOG signals into the CNN network as two modality data; S12, extracting the spatial features of EEG and EOG signals through spatial scale convolution; S13. Use group convolution to perform time scale convolution to extract the timing features of EEG signals and EOG signals.

3. The method for detecting fatigue status of electric power workers based on multiple physiological signals according to claim 1, characterized in that: The step 2 of fusing fatigue features of physiological signals based on the Transformer model includes: S21, the first layer is the input layer. The EEG fatigue features and the EEG fatigue features obtained in step S1 are concatenated in the first dimension before being input into the model to form a new vector, which is then input into the model. S22, the second layer is the coding layer. The input sequence is transformed into three matrices Q, K, and V through linear transformation. The calculation formula is: Q=W Q X#(1) K=W K X#(2) V=W V X#(3) Calculate the dot product of Q and K to get the weight matrix, multiply the weight matrix by the value matrix V to get the output after weighted summation. The formula is Perform convolution operation on the EEG signal and the EOG signal, convert the output tensor dimension, and then concatenate them, and repeat step S21; S23, set the encoder layer value and output its hidden layer representation; S24, introducing residual connection and layer normalization technology, and then performing average pooling on the first dimension to transform the features of the EEG signal and the EOG signal into a vector to obtain a feature vector; S25. Map the data dimension of the obtained feature vector through a multi-layer perceptron or a linear layer.

4. The method for detecting fatigue status of electric power workers based on multiple physiological signals according to claim 3 is characterized in that: The step S21 includes setting the hidden layer dimension and the number of encoding layers of the fully connected network when inputting the model into the input layer.

5. The method for detecting fatigue status of electric power workers based on multiple physiological signals according to claim 3 is characterized in that: The data dimensions in step S25 include dimensions of two states: normal state and fatigue state.

Images