Electrocardio sleep staging method based on electroencephalogram and electrocardio coupling learning

Abstract

The invention discloses an electrocardio sleep staging method based on electroencephalogram and electrocardio coupling learning, which is used for improving a UNet basic feature extraction structure so that the UNet basic feature extraction structure can extract multi-level and multi-scale richer information. Through a pooling layer, a convolution layer, a normalization layer and an activation layer, fusion processing and scale normalization are respectively carried out on basic electrocardio and electroencephalogram features, and a mean square error similarity function is adopted to supervise coupling learning, so that features learned by electrocardio are similar to electroencephalogram as much as possible, and more effective features used for electrocardio sleep staging are extracted. And the accuracy and efficiency of (electrocardio) sleep staging are effectively improved.

Description

Technical Field

[0001] The invention belongs to the field of sleep staging prediction, and in particular relates to an electrocardiographic sleep staging method based on electroencephalogram and electrocardiographic coupling learning. Background Art

[0002] Various physiological parameters of the human body change with different sleep states. The process of dividing different sleep states according to the characteristics of these vital signs is called sleep staging. Sleep staging is a key link in sleep monitoring, sleep structure analysis and sleep state assessment.

[0003] Among many physiological signals, EEG signals are the most obvious signals that change during sleep, but the acquisition of EEG signals is not convenient and the cost is high. Signal acquisition equipment is expensive and can only be used in hospitals or professional laboratories, which greatly limits the research on sleep stages and the promotion of home care.

[0004] Compared with EEG detection, ECG monitoring is cheaper and more convenient. Many new wearable ECG devices have been developed, but in actual use, the accuracy of ECG sleep staging is low and cannot effectively predict sleep staging.

[0005] In summary, the existing technology has the problem of not being able to take into account both applicability and accuracy when predicting sleep stages. Summary of the invention

[0006] The purpose of the present invention is to provide an electrocardiographic sleep staging method based on electroencephalogram and electrocardiographic coupling learning to solve the problem that the sleep staging prediction method in the prior art cannot take into account both applicability and accuracy.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:

[0008] An electrocardiogram (ECG) sleep staging method based on EEG and ECG coupling learning comprises the following steps:

[0009] Step 1, collecting data sets of ECG signals and EEG signals, deleting meaningless segments in the data sets of ECG signals and EEG signals, and resampling to obtain data sets of resampled ECG signals and EEG signals;

[0010] Perform signal phase division on the resampled data sets of the ECG signal and the EEG signal to obtain the data sets of the initial ECG signal and the EEG signal;

[0011] Step 2, inputting the data sets of the initial ECG signal and EEG signal into the ECG sleep staging model, and using the basic feature extraction structure of the ECG sleep staging model to extract ECG basic features and EEG basic features from the data sets of the initial ECG signal and EEG signal;

[0012] Step 3, using the ECG sleep staging model to couple the ECG basic features with the EEG basic features using a coupled learning structure to extract similar features between the ECG basic features and the EEG basic features;

[0013] Step 4: Use similar features and a loss function to perform supervised training on the ECG sleep staging model to obtain a trained ECG sleep staging model.

[0014] Step 5: Input the ECG signal to be predicted into the trained ECG sleep staging model to obtain a prediction result.

[0015] The present invention also has the following features:

[0016] Furthermore, in step 1:

[0017] The meaningless segments include awake segments of pre-sleep monitoring and segments without sleep stage labels;

[0018] The resampled ECG and EEG data sets were divided into signal stages according to the AASM standards.

[0019] Further, step 2 includes the following sub-steps:

[0020] Step 21, inputting the data sets of the initial ECG signal and the EEG signal into the basic feature extraction structure of the ECG sleep staging model; and performing pooling and normalization on the two-layer convolution of the basic feature extraction structure of the ECG sleep staging model;

[0021] Step 22, using the attention structure for decoding, and performing feature processing on the data in the data set of the initial ECG signal and EEG signal;

[0022] Step 23, using a skip connection structure to connect information between different layers in the data sets of the initial ECG signal and the EEG signal;

[0023] Step 24, using the normalized basic feature extraction structure to perform feature extraction, to obtain basic ECG features and basic EEG features.

[0024] Furthermore, the basic feature extraction structure is UNet.

[0025] Furthermore, in step 21, when the two layers of convolution of the basic feature extraction structure are normalized after pooling, the first layer of convolution uses a 1*5 convolution sheet; the second layer of convolution uses a 1*1 convolution sheet;

[0026] In step 24, when the basic feature extraction structure is used for feature extraction, three convolution kernels of different scales, 1*1, 1*3 and 1*5, are used.

[0027] Further, step 3 includes the following sub-steps:

[0028] Step 31, pooling the ECG basic feature graph of the ECG basic feature, and pooling the EEG basic feature graph of the EEG basic feature;

[0029] The pooled ECG basic feature map and EEG basic feature map are fused and scale normalized;

[0030] Step 32, using an activation function to optimize the basic feature extraction structure;

[0031] Step 33, using the ECG sleep staging model, conduct coupling learning on the ECG basic feature map and the EEG basic feature map after fusion processing and scale normalization based on the coupling learning structure to obtain similar features of the ECG basic features and the EEG basic features.

[0032] Further, in step 31, a pooling layer with a size of (1, 10) is used to pool the ECG basic feature map of the ECG basic feature and the EEG basic feature map of the EEG basic feature respectively;

[0033] A 1*1 convolution and normalization layer is used to fuse and scale normalize the pooled ECG basic feature map and EEG basic feature map.

[0034] In step 32, the activation function uses Leaky ReLU;

[0035] In step 33, in the process of using the ECG sleep staging model to carry out coupling learning on the fused and scale-normalized ECG basic feature map and the EEG basic feature map, the mean square error similarity function is used to supervise the coupling learning. The mean square error similarity function is as follows:

[0036]

[0037] Where m represents the number of sleep stages in a batch;

[0038] f eeg Represents the features extracted from the EEG signal for each sleep stage period;

[0039] f ecg Represents the features extracted from the ECG signal for each sleep stage period.

[0040] Further, step 4 includes the following sub-steps:

[0041] Step 41, using the coupled learned ECG sleep staging model obtained in step 33 to extract ECG features and EEG features from the data sets of the initial ECG signals and EEG signals;

[0042] Step 42, sending the extracted ECG features and EEG features to the fully connected layer of the ECG sleep staging model after coupling learning to obtain the prediction data results of the ECG sleep staging;

[0043] Step 43, compare the predicted data results with the real labels of the initial ECG signal and EEG signal data, substitute them into the loss function to calculate the prediction error value, perform iterative training, and obtain the trained ECG sleep staging model.

[0044] Furthermore, the loss function in step 43 adopts CrossEntropyLoss.

[0045] Compared with the prior art, the present invention has the following technical effects:

[0046] (I) The ECG sleep staging method based on EEG and ECG coupled learning of the present invention improves the UNet basic feature extraction structure so that it can extract more abundant information at multiple levels and scales. Through the pooling layer, convolution layer, normalization layer and activation layer, the basic ECG and EEG features are fused and scale normalized respectively, and the mean square error similarity function is used to supervise the coupled learning so that the features learned by the ECG are as similar as possible to the EEG, thereby extracting more effective features for ECG sleep staging, effectively improving the accuracy and applicability of sleep staging.

[0047] (II) The ECG sleep staging method based on EEG and ECG coupling learning of the present invention can be applied to ECG detection devices. There are many wearable ECG devices in the prior art. After applying the method of the present invention, a synergistic effect can be produced, which is conducive to the further development of sleep research and lays a good foundation for the promotion of home sleep devices.

[0048] (III) The ECG sleep staging method based on EEG and ECG coupling learning of the present invention simplifies the sleep staging process. It does not require manual feature extraction and complex feature engineering. Instead, it can fully capture the spatiotemporal information and semantic features in the EEG and ECG signal data through a deep learning network, automatically learn and extract ECG signal features, and simplify the sleep staging process. Experiments show that the classification method of the present invention has a significantly improved classification accuracy in the ECG sleep staging task compared with other existing models, and obtains a higher staging accuracy in the ECG sleep staging task, which is suitable for large-scale use and promotion in industry. DETAILED DESCRIPTION

[0049] It should be noted that, unless otherwise specified, all methods in the present invention adopt methods known in the prior art.

[0050] Specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical solution of this application fall within the protection scope of the present invention.

[0051] An electrocardiogram (ECG) sleep staging method based on EEG and ECG coupling learning comprises the following steps:

[0052] Step 1, collecting data sets of ECG signals and EEG signals, deleting meaningless segments in the data sets of ECG signals and EEG signals, and resampling to obtain data sets of resampled ECG signals and EEG signals;

[0053] Perform signal phase division on the resampled data sets of the ECG signal and the EEG signal to obtain the data sets of the initial ECG signal and the EEG signal;

[0054] Step 2, using the UNet basic feature extraction structure to extract basic ECG features and basic EEG features from the initial ECG signal and EEG signal data sets;

[0055] Step 3: Based on the coupled learning structure, the ECG sleep staging model is used to couple the ECG basic features with the EEG basic features, and then similar features of the ECG basic features and the EEG basic features are extracted; the purpose of this step is to make the ECG signal have the same accuracy as the EEG signal in sleep staging;

[0056] Step 4: Use similar features and loss functions to perform supervised training on the ECG sleep staging model to obtain the UNet basic feature extraction structure after training.

[0057] Step 5: Input the ECG signal to be predicted into the trained ECG sleep staging model to obtain a prediction result.

[0058] Furthermore, the meaningless segments in step 1 include awake segments monitored during the pre-sleep period and segments without sleep stage labels;

[0059] When performing signal staging on the resampled ECG and EEG data sets, the signal staging is performed according to the AASM standards.

[0060] Further, step 2 includes the following sub-steps:

[0061] Step 21, inputting the data sets of the initial ECG signal and the EEG signal into the basic feature extraction structure of the ECG sleep staging model; and performing pooling and normalization on the two-layer convolution of the basic feature extraction structure of the ECG sleep staging model;

[0062] Step 22, using the attention structure for decoding, and performing feature processing on the data in the data set of the initial ECG signal and EEG signal; it should be noted that the attention decoding structure mainly includes Squeeze and Excitation operations.

[0063] The Squeeze operation compresses the spatial dimension of the input feature map through global pooling, thereby converting it into a feature vector of channel dimension. This feature vector represents the global feature importance of each channel. This operation extracts the global representation of the current feature while reducing the dimension of the feature map.

[0064] The Excitation operation uses a fully connected neural network to learn the relationship between channels and generate a channel attention weight vector. This weight vector is normalized by the activation function Sigmoid to ensure that the weight is between 0 and 1. In this way, the attention weight of each channel represents its importance in the entire feature map. By adaptively adjusting the channel weights in this way, relevant feature channels can be enhanced and irrelevant feature channels can be suppressed, thereby improving the expressiveness of the UNet basic feature extraction structure.

[0065] Step 23, using a skip connection structure to connect the information between different layers in the dataset of the initial ECG signal and EEG signal; the skip connection structure connects the information between different layers in the dataset of the initial ECG signal and EEG signal. The purpose of this step is to promote the flow and sharing of information, so that the basic feature extraction structure can learn residual information more easily, which helps to alleviate the gradient disappearance and gradient explosion problems and accelerate the training process. Through the skip connection, shallow features can be directly transferred to the deep layer, so that the network can better utilize the association between low-level and high-level features and improve the learning ability of complex patterns and abstract features.

[0066] Step 24, using the normalized basic feature extraction structure to perform feature extraction, to obtain basic ECG features and basic EEG features.

[0067] Furthermore, in step 21, when the two layers of convolution of the basic feature extraction structure are pooled and normalized, the first layer of convolution uses a 1*5 convolution sheet; the second layer of convolution uses a 1*1 convolution sheet; the purpose of this step is to compress the expanded feature dimensions, remove features with low task relevance, and retain the most important feature information, so as to reduce the amount of calculation and enhance the robustness of the UNet basic feature extraction structure.

[0068] Normalizing the data allows the underlying feature extraction structure to learn scaling and offset parameters, thereby improving stability and convergence speed.

[0069] When using the basic feature extraction structure for feature extraction, three convolution kernels of different scales, 1*1, 1*3, and 1*5, are used.

[0070] Further, step 3 includes the following sub-steps:

[0071] Step 31, pooling the ECG basic feature graph of the ECG basic feature, and pooling the EEG basic feature graph of the EEG basic feature;

[0072] The pooled ECG basic feature map and EEG basic feature map are fused and scale normalized;

[0073] Step 32, using an activation function to optimize the basic feature extraction structure;

[0074] Step 33, using the ECG sleep staging model, conduct coupling learning on the ECG basic feature map and the EEG basic feature map after fusion processing and scale normalization based on the coupling learning structure to obtain similar features of the ECG basic features and the EEG basic features.

[0075] Furthermore, in step 31, a pooling layer with a size of (1, 10) is used to pool the ECG basic feature map and the EEG basic feature map of the ECG basic feature and the EEG basic feature;

[0076] A 1*1 convolution and normalization layer is used to fuse and scale normalize the pooled ECG basic feature map and EEG basic feature map.

[0077] The basic features of ECG and EEG are fused and scale normalized respectively, with the aim of eliminating scale differences between data, improving the consistency of data processing and analysis, enhancing the comprehensiveness of features, revealing the correlation between data, and providing more representative features, thereby improving the accuracy and efficiency of data analysis and modeling.

[0078] After the UNet basic feature extraction structure is optimized in step 2, the shape of the basic ECG and EEG feature maps extracted is (Batchsize, 32, 1, 60000). In order to reduce the amount of calculation and reduce the impact of different basic feature unit scales between modalities, a pooling layer with a size of (1, 10) is used to reduce the spatial dimension of the feature map, reduce the computational complexity, and help make the features translation invariant. Secondly, a 1*1 convolution and normalization layer are used to fuse the features, normalize the scale, help improve the stability of training, accelerate convergence, reduce overfitting, and help improve performance.

[0079] In step 32, the activation function uses Leaky ReLU;

[0080] The use of activation layers can learn more complex and abstract feature representations, improve the model's adaptability to complex data, and enable it to better fit variable sleep data, thereby improving its ability to capture and express data features.

[0081] Among them, the use of Leaky ReLU can solve the problem of neuron death that may occur in the ReLU activation function. By introducing a non-zero slope, Leaky ReLU allows negative inputs to have gradients. It also helps to alleviate the gradient vanishing problem and improve the training effect and expression ability of the neural network.

[0082] In step 33, in the process of using the ECG sleep staging model to carry out coupling learning on the fused and scale-normalized ECG basic feature map and the EEG basic feature map, the mean square error similarity function is used to supervise the coupling learning. The mean square error similarity function is as follows:

[0083]

[0084] Where m represents the number of sleep stages in a batch;

[0085] f eeg Represents the features extracted from the EEG signal for each sleep stage period;

[0086] f ecg Represents the features extracted from the ECG signal for each sleep stage period.

[0087] It should be noted that the mean square error similarity function is used to supervise the coupled learning, so that the EEG signal assists the ECG signal training, and the features learned from the ECG signal are as similar as possible to the features learned from the EEG signal, so as to extract more effective features for ECG sleep staging, thereby further improving the accuracy of sleep staging. In the end, only ECG is used to further improve the efficiency of sleep staging.

[0088] Mean square error (MSE) is a commonly used loss function and evaluation metric used to measure the difference between the predicted value and the true value. It calculates the average square distance between the predicted value and the true value to evaluate the accuracy of the model. MSE is used to compare the difference between EEG and ECG signals. By minimizing the MSE loss function, the features extracted from EEG and ECG can be made more similar. This helps to reduce the modality-specific differences between EEG and ECG, prompting ECG signals to learn features that are more similar to EEG signals, and extracting feature information contained in ECG signals that is more effective for sleep staging.

[0089] Further, step 4 includes the following sub-steps:

[0090] Step 41, using the coupled learned ECG sleep staging model obtained in step 33 to extract ECG features and EEG features from the data sets of the initial ECG signals and EEG signals;

[0091] Step 42, sending the extracted ECG features and EEG features to the fully connected layer of the ECG sleep staging model after coupling learning to obtain the prediction data results of the ECG sleep staging;

[0092] Step 43, compare the predicted data results with the real labels of the initial ECG signal and EEG signal data, substitute them into the loss function to calculate the prediction error value, perform iterative training, and obtain the trained ECG sleep staging model.

[0093] Furthermore, the loss function in step 43 adopts CrossEntropyLoss.

[0094] The CrossEntropyLoss loss function has a built-in Softmax layer, which records the error value in real time and feeds it back to the model. Through multiple rounds of data iterative training, the weights and biases of the network nodes are continuously optimized and updated, making the final classification result closer to the true label. After completing the above parameter training, a decision response is made to the test data.

[0095] In order to verify the feasibility and effectiveness of this scheme, the results of the ECG sleep staging model were evaluated by accuracy and MF1 score.

[0096] This solution mainly focuses on the three-stage tasks of W, NREM, and REM and the five-stage tasks of W, N1, N2, N3, and REM. The evaluation indicators include accuracy and F1 score.

[0097] The accuracy rate is the ratio of the number of sleep stages predicted correctly by the model to the total sleep stage data.

[0098] The F1 score is an indicator used to measure the accuracy of a classification model. It takes into account both the accuracy and recall of the classification model and can be regarded as a weighted average of the model's accuracy and recall.

[0099] The MF1 score can help evaluate the overall performance of the model in multi-class classification problems without being affected by the imbalance in the number of class samples.

[0100] MASS comes from the Montreal Sleep Study Archives and has five subsets, SS1-SS5, which are arranged in order according to their studies and acquisition protocols.

[0101] Table 1 is a performance indicator comparison table of this scheme and other methods, showing the 31-fold cross-validation classification results on the MASS-SS3 data set, among which the three-stage results and the five-stage results reached 79.61% and 68.28% respectively, which are better than other comparison models, demonstrating the significant advantages of the method of the present invention in ECG sleep staging.

[0102] Table 1: Performance comparison of this solution and other methods

[0103] The method used Staging ACC MF1 TinySleepNet 3 71.70 50 SalientSleepNet 3 70.86 52.74 TransSleep 3 71.06 49.47 This program 3 79.61 70.68 TinySleepNet 5 46 31.8 SalientSleepNet 5 56.17 43.79 TransSleep 5 51.64 36.51 This program 5 68.28 58.52

[0104] Among them, TinySleepNet, SalientSleepNet and TransSleep are all names of existing methods known in the prior art, and those skilled in the art can know the methods corresponding to the above names.

[0105] Table 2 and Table 3 are respectively a comparison table of three-stage performance indicators before and after coupled learning using this scheme and a comparison table of five-stage performance indicators before and after coupled learning using this scheme; they respectively show the performance comparison of each stage task before and after the model of the present invention couples EEG and ECG signals.

[0106] The results show that the performance after coupling is better than before coupling. With the auxiliary training of EEG, the accuracy of ECG sleep staging has been effectively improved.

[0107] Table 2: Comparison of the three-stage performance indicators before and after coupling learning of this scheme

[0108]

[0109] Table 3: Comparison of the five-stage performance indicators before and after coupling learning of this scheme

[0110]

Claims

1. A method for electrocardiographic sleep staging based on electroencephalogram and electrocardiogram coupling learning, characterized in that: The following steps are involved: Step 1, collecting data sets of ECG signals and EEG signals, deleting meaningless segments in the data sets of ECG signals and EEG signals, and resampling to obtain data sets of resampled ECG signals and EEG signals; Perform signal phase division on the resampled data sets of the ECG signal and the EEG signal to obtain the data sets of the initial ECG signal and the EEG signal; Step 2, inputting the data sets of the initial ECG signal and EEG signal into the ECG sleep staging model, and using the basic feature extraction structure of the ECG sleep staging model to extract ECG basic features and EEG basic features from the data sets of the initial ECG signal and EEG signal; Step 3, using the ECG sleep staging model to couple the ECG basic features with the EEG basic features using a coupled learning structure to extract similar features between the ECG basic features and the EEG basic features; Step 4: Use similar features and a loss function to perform supervised training on the ECG sleep staging model to obtain a trained ECG sleep staging model. Step 5: Input the ECG signal to be predicted into the trained ECG sleep staging model to obtain a prediction result.

2. The ECG sleep staging method based on EEG and ECG coupling learning as claimed in claim 1, characterized in that: In step 1: The meaningless segments include awake segments of pre-sleep monitoring and segments without sleep stage labels; The resampled ECG and EEG data sets were divided into signal stages according to the AASM standards.

3. The electrocardiographic sleep staging method based on electroencephalogram and electrocardiogram coupling learning as claimed in claim 1, characterized in that: Step 2 includes the following sub-steps: Step 21, inputting the data sets of the initial ECG signal and the EEG signal into the basic feature extraction structure of the ECG sleep staging model; and performing pooling and normalization on the two-layer convolution of the basic feature extraction structure of the ECG sleep staging model; Step 22, using the attention structure for decoding, and performing feature processing on the data in the data set of the initial ECG signal and EEG signal; Step 23, using a skip connection structure to connect information between different layers in the data sets of the initial ECG signal and the EEG signal; Step 24, using the normalized basic feature extraction structure to perform feature extraction, to obtain basic ECG features and basic EEG features.

4. The ECG sleep staging method based on EEG and ECG coupling learning as claimed in claim 3, characterized in that: The basic feature extraction structure is UNet.

5. The ECG sleep staging method based on EEG and ECG coupling learning as claimed in claim 4, characterized in that: In step 21, when the two layers of convolution of the basic feature extraction structure are pooled and normalized, the first layer of convolution uses a 1*5 convolution sheet; the second layer of convolution uses a 1*1 convolution sheet; In step 24, when the basic feature extraction structure is used for feature extraction, three convolution kernels of different scales, 1*1, 1*3 and 1*5, are used.

6. The electrocardiographic sleep staging method based on electroencephalogram and electrocardiogram coupling learning as claimed in claim 1, characterized in that: Step 3 includes the following sub-steps: Step 31, pooling the ECG basic feature graph of the ECG basic feature, and pooling the EEG basic feature graph of the EEG basic feature; The pooled ECG basic feature map and EEG basic feature map are fused and scale normalized; Step 32, using an activation function to optimize the basic feature extraction structure; Step 33, using the ECG sleep staging model, conduct coupling learning on the ECG basic feature map and the EEG basic feature map after fusion processing and scale normalization based on the coupling learning structure to obtain similar features of the ECG basic features and the EEG basic features.

7. The electrocardiographic sleep staging method based on electroencephalogram and electrocardiogram coupling learning as claimed in claim 6, characterized in that ； In step 31, a pooling layer with a size of (1, 10) is used to pool the ECG basic feature map of the ECG basic feature and the EEG basic feature map of the EEG basic feature respectively; A 1*1 convolution and normalization layer is used to fuse and scale normalize the pooled ECG basic feature map and EEG basic feature map. In step 32, the activation function uses Leaky ReLU; In step 33, in the process of using the ECG sleep staging model to carry out coupling learning on the fused and scale-normalized ECG basic feature map and the EEG basic feature map, the mean square error similarity function is used to supervise the coupling learning. The mean square error similarity function is as follows: Where m represents the number of sleep stages in a batch; f eeg Represents the features extracted from the EEG signal for each sleep stage period; f ecg Represents the features extracted from the ECG signal for each sleep stage period.

8. The ECG sleep staging method based on EEG and ECG coupling learning as claimed in claim 7, characterized in that: Step 4 includes the following sub-steps: Step 41, using the coupled learned ECG sleep staging model obtained in step 33 to extract ECG features and EEG features from the data sets of the initial ECG signals and EEG signals; Step 42, sending the extracted ECG features and EEG features to the fully connected layer of the ECG sleep staging model after coupling learning to obtain the prediction data results of the ECG sleep staging; Step 43, compare the predicted data results with the real labels of the initial ECG signal and EEG signal data, substitute them into the loss function to calculate the prediction error value, perform iterative training, and obtain the trained ECG sleep staging model.

9. The electrocardiographic sleep staging method based on EEG and ECG coupling learning as claimed in claim 8, characterized in that: The loss function in step 43 adopts CrossEntropyLoss.

Images