A reliability evaluation method for sleep staging results

Abstract

A method for evaluating the credibility of sleep staging results comprises the following steps: step 1, calculating the original signal interference R for original sleep data; obtaining a model input by extracting features from credible data selected based on a limited interference value; selecting two classifiers with different architectures for training to obtain two trained classification models; step 2, obtaining prediction results of two classification views using the two classifiers trained in step 1; step 3, calculating the Kappa measure based on the prediction results of the two classifiers in step 2; step 4, outputting the original signal interference R and the Kappa measure, and calculating and outputting the credibility of the sleep staging result; the present invention is applicable to actual sleep monitoring scenarios, has the ability to improve the generalization of classification for massive unlabeled sleep data, can improve the credibility of the automatic sleep staging method, and is authentic, reliable, and easy to understand.

Description

Technical Field

[0001] The present invention belongs to the technical field of sleep monitoring, and in particular relates to a method for evaluating the credibility of sleep staging results. Background Art

[0002] Studying sleep helps us understand the activities of the brain's central nervous system, thereby helping humans to carry out targeted intervention and treatment of related sleep disorders.

[0003] Currently, the "gold standard" for clinical sleep monitoring is based on polysomnography (PSG), which collects multiple physiological signals from subjects throughout a night's sleep, including electroencephalogram (EEG), electromyogram (EMG), electro-oculogram (EOG), and electrocardiography (ECG). Professional physicians use a 30-second interval to interpret sleep periods according to the American Academy of Sleep Medicine (AASM) standards, which categorize sleep activity into five states: wake (Wake), non-rapid eye movement (NREM) stage I (N1), non-rapid eye movement (NREM) stage II (N2), non-rapid eye movement (NREM) stage III (N3), and rapid eye movement (REM). In recent years, many researchers have begun to focus on the use of low-cost wearable sleep monitoring devices and artificial intelligence technology to conduct automatic sleep staging technology research due to the high cost of PSG measurement, the need for subjects to wear a large number of sensors while sleeping, the need to set up professional sleep laboratories and professional doctors to interpret the results.

[0004] With the rapid development of artificial intelligence technology, AI-based automatic sleep staging technology has achieved great success. For example, the patent with publication number 2022101415154.4 discloses an automatic sleep signal staging method based on an improved deep learning algorithm model. However, due to the "black box characteristics" and "unexplainability" of AI, whether people can fully trust the AI-based sleep staging results is a great challenge. The credibility evaluation method of sleep staging results will provide important support for the development of automatic sleep staging technology and has important application value. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the purpose of the present invention is to provide a method for evaluating the credibility of sleep staging results. The method aims to evaluate the credibility of the original data and the credibility of the staging model output results from two aspects, the original data and the model results, based on the dual-view concept of two classifiers, to evaluate the credibility of the final staging results, which is authentic, reliable and easy to understand.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for evaluating the credibility of sleep staging results comprises the following steps:

[0008] Step 1: First, the signal credibility of the collected raw sleep data is evaluated, that is, the raw signal interference R is calculated. If the signal interference R is greater than the threshold, the raw data is considered unsuitable for further analysis and the process is terminated. Otherwise, the raw signal is considered credible and the credible sleep data is selected for the next step of feature extraction as the model input data.

[0009] Step 2: Use the two trained sleep stage classifiers to perform model predictions on the model input data and obtain prediction results for two classification views;

[0010] Step 3: Calculate the Kappa measure based on the prediction results of the two classifiers in step 2;

[0011] Step 4: Use the original signal interference R and Kappa measure to calculate and output the credibility of the sleep staging results.

[0012] The step 1 is specifically as follows:

[0013] After the collected original signal is preliminarily analyzed from the perspective of time domain and frequency domain, the abnormal segments are marked after setting the threshold, and the original signal interference R is calculated using formula (1);

[0014]

[0015] Where i is the fragment number of a sample, r i Is a fragment an abnormal signal? If so, then r i =1, R represents the abnormal fragment ratio of a sample;

[0016] If R < threshold, the signal is considered credible and selected for subsequent model training and testing; otherwise, the signal is unreliable, that is, R = 1, and the signal is unreliable, which means the sleep staging result is unreliable.

[0017] Credible sleep data was selected to extract one-dimensional and two-dimensional features from the time domain and frequency domain as model input. Two classifiers with different architectures were selected and trained based on self-training semi-supervised learning to obtain two classification models after training.

[0018] The step 2 is specifically as follows:

[0019] Use the two classifiers trained in step 1 to predict the sleep data to be analyzed. The sleep data is represented by S(j), where i = 1, 2, 3, ..., M, and M represents the data length. S(j) is divided into N sleep periods according to a fixed time interval to obtain S'(j), where j = 1, 2, 3, ..., N. Input the original signal of S'(j) into classifier 1 to obtain the prediction results x for all sleep periods. j , where j = 1, 2, 3, ..., N; S'(j) is input into classifier 2 after two-dimensional wavelet transform to obtain the prediction results y for all sleep periods j , where j = 1, 2, 3, ..., N.

[0020] The step three is specifically as follows:

[0021] The prediction results x of the two models are obtained through step 2 j ,y j , where j = 1, 2, 3, ..., N, and the Kappa measure between the two models is calculated using formula (2), which is defined as follows:

[0022]

[0023] In the formula, i is the category, x is ii For each category in a sample, the two models predict the number of consistent fragments, N is the number of fragments in this sample, x i is the number of segments in this sample that are classified as class i by classifier 1, y i is the number of segments in this sample that are classified as class i by classifier 2;

[0024] When K>0, it is statistically significant and can be used as an evaluation indicator for the reliability of the model prediction results.

[0025] The step four is specifically as follows:

[0026] According to the interference degree R in step 1 as the credibility of the original signal, and the Kappa measure in step 3 as the credibility of the prediction result, the credibility C of the sleep stage result is calculated using formula (3), which is defined as follows:

[0027]

[0028] Where R is the interference degree of the original signal of the data, and K is the Kappa measure obtained by the prediction results of the two models.

[0029] Compared with the prior art, the advantages of the present invention are:

[0030] 1. As can be seen from step 2, the present invention proposes using two trained sleep stage classifiers to perform model predictions on the model input data separately, obtaining prediction results from two classification views. This method, based on the dual-view concept, is not restricted by the type of input data and classifiers, and is suitable for credibility evaluation of sleep stage results from various physiological signals and classifiers in large-scale datasets.

[0031] 2. The confidence values output by the prediction results of the two classification views assist semi-supervised learning in automatically labeling massive amounts of unlabeled sleep data to update the training set size; and self-training using massive amounts of unlabeled data, adopting a repeated labeling strategy, improves classifier accuracy.

[0032] 3. The sleep segmentation data S'(j) obtained from the sleep data S(j) in step 2 of the present invention can be obtained by directly segmenting X(i) or by extracting some time domain, frequency domain and other feature vectors after segmenting S(j). It has the characteristics and advantages of significant changes in relevant information during sleep, and the appropriate data collection method can be selected according to the specific usage scenario.

[0033] 4. The classification tasks of the two classifiers in the present invention can be flexibly selected according to the specific application. For example, when applied to family sleep evaluation, three or four classifiers can be selected.

[0034] In summary, the feature of the present invention is that it proposes a sleep staging credibility evaluation method based on the dual-view concept of two classifiers. It is applied to actual sleep monitoring scenarios, improves the generalization ability of classification for massive unlabeled sleep data, and provides a basis for the credibility of the results of the automatic sleep staging method, which has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Flow chart of the method of the present invention.

[0036] Figure 2 Schematic diagram of two classifier structures according to an embodiment of the present invention, wherein: Figure 2 (a) in the figure is the classifier of the RAW-BiLSTM model structure. Figure 2 (b) in the figure is the classifier of the CWT-TCN model structure.

[0037] Figure 3 The sleep staging results and result credibility evaluation of the embodiment of the present invention; wherein, Figure 3(a) is the result of the classifier of the RAW-BiLSTM model structure. Figure 3 (b) in the figure is the result of the classifier of the CWT-TCN model structure. DETAILED DESCRIPTION

[0038] The present invention will be described in detail below with reference to the accompanying drawings.

[0039] Reference Figure 1 The present invention, in conjunction with the accompanying drawings, provides a specific embodiment of a method for automatically evaluating the credibility of sleep signal staging based on a large-scale dataset. The method is used to evaluate sleep staging results obtained by a sleep staging model constructed using a bidirectional long short-term memory (Bi-LSTM) deep neural network and a time-series convolutional neural network (TCN) using a single-channel EEG signal. The method comprises the following steps:

[0040] Step 1: First, the signal credibility of the collected raw sleep data is evaluated, that is, the raw signal interference R is calculated. If the signal interference R is greater than the threshold, the raw data is considered unsuitable for further analysis and the process is terminated. Otherwise, the raw signal is considered credible and the credible sleep data is selected for the next step of feature extraction as the model input data.

[0041] The step 1 is specifically as follows:

[0042] Considering the application scenarios of wearable EEG devices, high-quality signals are the basis for our next research. After preliminary analysis of the collected raw signals from the perspectives of time and frequency domains, statistical analysis is performed through experiments. After setting thresholds, abnormal segments are marked and the raw signal interference R is calculated using formula (1).

[0043]

[0044] Where i is the fragment number of a sample, r i Is a fragment an abnormal signal? If so, then r i =1, R represents the abnormal fragment ratio of a sample.

[0045] In the present invention, R is set to be less than the threshold, and the signal is considered to be credible and selected into the subsequent model training and testing; otherwise, the signal is unreliable, that is, R=1, and the unreliable signal can be determined as the sleep staging result is unreliable.

[0046] Combined with the needs of clinical sleep staging, refer to Figure 2In (a) and (b), five classifiers are first constructed based on the RAW-BiLSTM network and the CWT-TCN network, respectively. One-dimensional and two-dimensional features are extracted from the sleep data in the time and frequency domains as the overall feature input model. A pre-trained CNN module extracts local features for each sleep period. Finally, the extracted local features and labels are used to train and predict the two classifiers. The two classifiers are trained using self-training semi-supervised learning to obtain two trained classification models.

[0047] Step 2: Use the two trained sleep stage classifiers to perform model predictions on the model input data and obtain prediction results for two classification views.

[0048] A 7-hour 5-minute EEG signal data S(j) is divided into 850 sleep periods at 30-second intervals to obtain S'(j), where i = 1, 2, 3, ..., 25500s and j = 1, 2, 3, ..., 850. Input S'(j) into the classifier trained in step 1 to obtain the prediction results x for all sleep periods. j , where j = 1, 2, 3, ..., N; S'(j) is input into the TCN classifier after two-dimensional wavelet transform to obtain the prediction results y for all sleep periods j , where j = 1, 2, 3, ..., N, the results are as shown in the reference Figure 3 As shown in (a).

[0049] A 7-hour 5-minute EEG signal data S(i) is divided into 850 sleep periods at 30-second intervals to obtain S'(j), where i = 1, 2, 3, ..., 25500s and j = 1, 2, 3, ..., 850. The original data of S'(j) is input into the BiLSTM classifier to obtain the prediction results y for all sleep periods j , where j = 1, 2, 3, ..., N, the results are as shown in the reference Figure 3 (b) shown.

[0050] Step 3: Based on the idea of consistency, the present invention uses two different features and corresponding different models to obtain two different perspectives. The consistency of the two views is used to make a consistency judgment on the prediction results of the model to obtain the credibility of the prediction results. Usually, the Euclidean distance is used to measure the distance between two prediction results. However, EEG data is an unbalanced data, and it is inappropriate to use Euclidean distance to measure the distance between model predictions. The Kappa measure can effectively evaluate unbalanced data and determine whether the consistency is inevitable or accidental. The theoretical minimum value of the Kappa measure is -1 and the maximum value is 1. The higher the Kappa measure, the more consistent the outputs of the two models are.

[0051] Use step 2 to get the prediction results x of the two classifiers j 、y j , where j = 1, 2, 3, ..., N. According to the Kappa measure calculation formula (2), calculate the Kappa measure value K of all sleep periods:

[0052]

[0053] In the formula, i is the category, x is ii For each category in a sample, the two models predict the number of consistent fragments, N is the number of fragments in this sample, x i is the number of segments in this sample classified as category i by the RAW-BiLSTM model, y i is the number of segments in this sample that are classified as category i by the CWT-TCN model. The Kappa measurement results are shown in the reference Figure 3 As shown in (a) and (b).

[0054] When K>0, it is statistically significant and can be used as an evaluation indicator for the reliability of the model prediction results.

[0055] Step 4: Output the original signal interference R in step 1, as shown in the following example: Figure 3 As shown in (a) and (b) in the figure. At the same time, the figure includes j 、y j The sleep staging results of professional physicians are compared, and the Kappa measure obtained in step 3 is output as the credibility of the prediction result. The credibility of the sleep staging result is calculated by formula (3):

[0056]

[0057] Get, C RAW-BiLSTM =0.902,C CWT-TCN =0.463.

[0058] Where R is the interference degree of the original signal of the data, and K is the Kappa measure obtained by the prediction results of the two models.

Claims

1. A method for evaluating the credibility of sleep staging results, characterized in that: The following steps are involved: Step 1: First, the signal credibility of the collected raw sleep data is evaluated, that is, the raw signal interference R is calculated. If the signal interference R is greater than the threshold, the raw data is considered unsuitable for further analysis and the process is terminated. Otherwise, the raw signal is considered credible and the credible sleep data is selected for the next step of feature extraction as the model input data. Step 2: Use the two trained sleep stage classifiers to perform model predictions on the model input data and obtain prediction results for two classification views; Step 3: Calculate the Kappa measure based on the prediction results of the two classifiers in step 2; Step 4: Calculate and output the credibility of the sleep staging results using the original signal interference R and Kappa measure; The step 1 is specifically as follows: After the collected original signal is preliminarily analyzed from the perspective of time domain and frequency domain, the abnormal segments are marked after setting the threshold, and the original signal interference R is calculated using formula (1); Where i is the fragment number of a sample, r i Is a fragment an abnormal signal? If so, then r i =1, R represents the abnormal fragment ratio of a sample; If R < threshold, the signal is considered credible and selected for subsequent model training and testing; otherwise, the signal is unreliable, that is, R = 1, and the signal is unreliable, which means the sleep staging result is unreliable. Reliable sleep data was selected to extract one-dimensional and two-dimensional features from the time and frequency domains as model inputs. Two classifiers with different architectures were selected and trained based on self-training semi-supervised learning to obtain two trained classification models. The step four is specifically as follows: According to the interference degree R in step 1 as the credibility of the original signal, and the Kappa measure in step 3 as the credibility of the prediction result, the credibility C of the sleep stage result is calculated using formula (3), which is defined as follows: Where R is the interference degree of the original signal of the data, and K is the Kappa measure obtained by the prediction results of the two models.

2. A method for evaluating the credibility of sleep staging results according to claim 1, characterized in that: The step 2 is specifically as follows: Use the two classifiers trained in step 1 to predict the sleep data to be analyzed. The sleep data is represented by S(j), where i = 1, 2, 3, ..., M, and M represents the data length. S(j) is divided into N sleep periods according to a fixed time interval to obtain S'(j), where j = 1, 2, 3, ..., N. Input the original signal of S'(j) into classifier 1 to obtain the prediction results x for all sleep periods. j , where j = 1, 2, 3, ..., N; S'(j) is input into classifier 2 after two-dimensional wavelet transform to obtain the prediction results y for all sleep periods j , where j = 1, 2, 3, ..., N.

3. The method for evaluating the credibility of sleep staging results according to claim 1, wherein: The step three is specifically as follows: The prediction results x of the two models are obtained through step 2 j ,y j , where j = 1, 2, 3, ..., N, and the Kappa measure between the two models is calculated using formula (2), which is defined as follows: In the formula, i is the category, x is ii For each category in a sample, the two models predict the number of consistent fragments, N is the number of fragments in this sample, x i is the number of segments in this sample that are classified as class i by classifier 1, y i is the number of segments in this sample that are classified as class i by classifier 2; When K>0, it is statistically significant and can be used as an evaluation indicator for the reliability of the model prediction results.

Images