A pseudo-twin network-based heterogeneous sleep data staging method

Abstract

The present application belongs to the technical field of sleep health management, and specifically relates to a heterogeneous sleep staging method based on a pseudo-twin network architecture. The method comprises the following steps: collecting, staging, and labeling original electroencephalogram (EEG) and electrooculogram (EOG) signals, and performing pretreatment; aligning the pretreated signals with corresponding staging labels to obtain labeled sleep staging data; training a heterogeneous data sleep staging model based on a pseudo-twin network architecture using the sleep staging data, inputting the EEG and EOG signals simultaneously during the training, performing similarity calculation after feature extraction; enhancing the feature similarity of different signals by learning network parameters to obtain a heterogeneous data sleep staging network; and identifying sleep stages using the heterogeneous data sleep staging network. The present application can improve the performance of the sleep staging model by using more information during the training stage, realizes automatic sleep stage staging for heterogeneous data, and has a wide clinical application prospect.

Description

Technical Field

[0001] This invention belongs to the field of sleep health management technology, specifically relating to a method for sleep staging of heterogeneous data. Background Technology

[0002] In sleep health management, it is often necessary to manually observe data such as electroencephalograms (EEGs) and electrooculograms (EOGs) recorded by polysomnography (PSG) to determine sleep stages, thereby better understanding sleep quality and diagnosing related sleep disorders. However, manual sleep staging is time-consuming and labor-intensive, and the varying clinical experience of different experts can introduce some bias into the analysis results.

[0003] In recent years, deep learning technology has attracted much attention in the field of sleep staging. Researchers have successively built automatic sleep staging models using convolutional neural networks and recurrent neural networks to reduce the workload of doctors and specialists. Many existing automatic sleep staging methods include neural network models based on single-channel signals as well as models based on a combination of multiple modal signals. Among single-modal methods, models based on electroencephalogram (EEG) signals can achieve better classification performance because changes in EEG signals are highly correlated with sleep activity, providing more effective information. In contrast, methods using electrooculogram (EOG) signals cannot achieve the high accuracy of EEG-based methods. Furthermore, models based on single-channel signals actually perform worse than those using multimodal signals. This is because multimodal signal-based models comprehensively consider multiple information dimensions, summarizing more deep sleep-related features compared to the limited information provided by single-channel signals. Multimodal signal models can achieve higher predictive performance, but they also increase the requirements for input data. However, there are various devices for collecting EEG and EOG signals, and signals collected by different devices may have inconsistent or mismatched channels. In practical applications, there are problems such as inconsistent acquisition devices or data loss due to noise contamination leading to missing channels. Current multimodal models based on electrooculography (EOG) do not perform well in recognition and cannot handle heterogeneous data. When input information is lost, the model performance drops significantly, which is an urgent problem to be solved.

[0004] Current methods often propose network architectures that only consider a single scenario, such as the collected signals containing only EEG signals. They are trained for this specific input modality and cannot handle channel inconsistencies. Siamese neural networks (SNNs) are architectures containing two subnetworks with identical structures and weights, typically used to measure input similarity and applied in image processing, natural language processing, and other fields to compare the similarity of images or text. Pseudo-Siamese neural networks (PSNNs) are a variant used to describe two networks with similar structures but different weights. Compared to SNNs, Pseudo-Siamese networks are often used to handle scenarios with significant input differences. For the morphological differences in EEG and EEG signals, Pseudo-Siamese neural networks can be used to measure and compare their similarity, extracting common characteristics of time-series signals and improving the model's recognition performance. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art, improve the sleep staging effect using single-channel signals, and solve the problem of heterogeneous data from multiple sources and devices in the field of sleep. A heterogeneous sleep data staging method based on pseudo-twin networks is proposed.

[0006] The proposed method for heterogeneous sleep data staging based on pseudo-twin networks includes constructing a heterogeneous sleep staging model based on a pseudo-twin network architecture, utilizing deep learning to achieve automatic staging of heterogeneous data, and strengthening the model's perception of different modal signal features by calculating and learning the feature similarity of EEG and EEG signals, obtaining a general pattern of time-series signal features, and improving the model's recognition performance for single signals. The specific steps are as follows:

[0007] (1) Obtain raw EEG and EEG signals of N subjects in the sleep state, and label the corresponding signals of each subject for the sleep stage; use the stage labeling results of the raw signals as the standard stage results; the sleep stages are divided into: wakefulness (W), rapid eye movement (REM) and non-rapid eye movement (N1, N2, N3); generally, N is greater than or equal to 20.

[0008] (2) Filter the original signal obtained in step (1) to remove background noise, and then downsample the filtered signal to obtain the preprocessed signal.

[0009] The specific operation is as follows: For the EEG and EEG signals marked during the sleep stage, a Butterworth bandpass filter is used to filter out background noise and attenuate signal activity less than 0.3Hz and greater than 35Hz; then a notch filter is used to filter out artifacts caused by motion, electrocardiographic activity and electrode interference.

[0010] (3) Align the preprocessed EEG and EEG signals obtained in step (2) with the stage labels of the original signals in step (1) to obtain labeled sleep stage training data.

[0011] (4) Construct a heterogeneous data sleep staging model based on a pseudo-twin network architecture;

[0012] The heterogeneous data sleep staging model based on a pseudo-twin network architecture consists of two parts: a spatial mapping module and a classifier; wherein:

[0013] The spatial mapping module comprises two identical convolutional neural networks, which are responsible for extracting features from the signal. Each convolutional neural network includes a dual-scale convolutional network and a sequeeze-and-excitation block. Specifically, the dual-scale convolutional networks each contain the following network layers: convolutional layer, pooling layer, activation function, and batch normalization. The kernel and stride of the convolutional layers are different.

[0014] The convolutional layer extracts different feature information by convolving the input data with multiple convolutional kernels.

[0015] The pooling layer performs a pooling operation on the output of the convolutional layer, compressing feature maps of different scales into vectors of a fixed size.

[0016] The activation function uses nonlinear transformation to map features to a high-dimensional scale, thereby improving the model's ability to process nonlinear data.

[0017] The batch normalization process calculates the mean and variance of the current batch based on the output data of the previous layer network and performs a normalization operation to prevent the network from overfitting.

[0018] The compressed excitation module is a convolutional network containing residual connections. It compresses the channel-dimensional features into one dimension and multiplies them with the features obtained from further convolution to obtain enhanced features with certain weights.

[0019] In the spatial mapping module, the outputs of the two convolutional neural network branches are features extracted from two different modal signals, and both feature maps are fed into the subsequent classifier module.

[0020] The classifier module mainly consists of two fully connected layers and one activation function; the aforementioned feature vectors are output as the sleep phase after passing through the classifier module.

[0021] (5) Train a heterogeneous data sleep staging model with a pseudo-Siamese network architecture using sleep staging training data; the specific process is as follows:

[0022] (5-1) The EEG and EEG signals obtained in step (3) are sent into the two convolutional neural network branches in the spatial mapping module for feature extraction, and the signals are mapped to a high-dimensional space for subsequent similarity learning.

[0023] (5-2) Obtain two feature vectors from different modal signals from step (5-1), calculate the similarity between the feature vectors, and use similarity measurement methods to reduce the distance between the feature vectors; similarity measurement methods include cosine similarity, Euclidean distance, and maximum mean difference;

[0024] (5-3) Two feature vectors from different modal signals are obtained from step (5-1), and are respectively fed into the spatial mapping module to obtain the classification result; the cross-entropy loss function is used for parameter update in this classification problem;

[0025] During training, each subject's preprocessed signals, including EEG and EEG signals, were fed into two separate convolutional neural networks for feature extraction. The extracted features were then analyzed using a similarity metric to increase similarity and reduce differences between features, enabling the model to handle heterogeneous data with inconsistent or missing channels. Features from two different modalities, after the aforementioned processing, maintained their unique characteristics while increasing similarity, and were then fed into the same classifier. The classifier learned from diverse samples and output the corresponding sleep stage category. After training, a trained heterogeneous data sleep stage network based on a pseudo-Twin network architecture was obtained.

[0026] (6) Use the heterogeneous data sleep staging network trained in step (5) to perform sleep staging on the heterogeneous signals to be staging; the specific process is as follows:

[0027] (6-1) Obtain the electroencephalogram (EEG) or electrooculogram (EOG) signal of a subject during sleep, and perform the preprocessing in step (2) on the signal. The signal to be staged is similar to the signal obtained in step (1). The input may contain only EEG signal, only EOG signal, or both signals at the same time.

[0028] (6-2) Input the sleep staging network trained in step (5) into the signal to be staged. The network outputs the sleep staging result of the signal to be staged.

[0029] In this invention, the input signal during training includes EEG and EEG signals, and the test can handle not only cases where both signals are present, but also cases where only a single channel EEG or EEG signal is present.

[0030] Corresponding to the above-mentioned heterogeneous sleep data staging method, this invention also provides a heterogeneous sleep data staging system. This system includes the following modules: a module for acquiring and labeling raw EEG and EEG signals during sleep; a module for preprocessing raw signals; a training data processing module; a heterogeneous sleep staging model module based on a pseudo-twin network architecture; a model training module; and a module for using the model to perform sleep staging. These six modules respectively execute the six steps of the heterogeneous sleep data staging method.

[0031] The features and beneficial effects of this invention are as follows:

[0032] This invention performs noise filtering and downsampling on raw EEG and EEG signals; aligns the preprocessed signals with corresponding stage labels to obtain labeled sleep stage data; trains a pseudo-Twin network using the sleep stage data, simultaneously inputting EEG and EEG signals during training, extracting features and calculating similarity; enhances the feature similarity of different signals by learning network parameters, increasing feature diversity and further improving the robustness of the classifier, thus obtaining a heterogeneous data sleep stage network; uses the heterogeneous data sleep stage network to identify sleep stages; this invention uses more information during the training phase, which can improve the performance of sleep stage models based on single-channel signals, and can handle channel inconsistencies and missing channels caused by inconsistent acquisition devices or noise pollution in sleep data, improving the ability to identify single-channel signals, and enabling the discrimination of heterogeneous data analysis, with broad clinical application prospects. Attached Figure Description

[0033] Figure 1 This is an overall flowchart of the heterogeneous data sleep staging method based on pseudo-twin network architecture of the present invention.

[0034] Figure 2 This is a diagram of the pseudo-twin network architecture of the present invention.

[0035] Figure 3 This is a schematic diagram of the input and output of the pseudo-twin network model in the application stage of the present invention. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0037] The sleep staging method for heterogeneous data proposed in this invention has the following overall process: Figure 1 As shown, it includes the following steps:

[0038] (1) Obtain raw EEG and EEG signals, with a signal duration of at least 2 hours, and have experts label each signal with sleep stages to obtain labeled sleep stage training data corresponding to the signals.

[0039] In this embodiment, the raw EEG signals acquired by the medical EEG acquisition device do not contain classification labels for sleep stages. In order for the computer to learn the classification features of these sleep stages, medical sleep experts need to manually label the corresponding sleep stages in each signal segment. Specifically, the experts need to label the sleep stages contained in each signal segment according to the rules promulgated by the American Academy of Sleep Medicine (AASM). These labeled data can be used to train deep learning models.

[0040] The American Academy of Sleep Medicine divides adult sleep into five stages: wakefulness (W), rapid eye movement (REM), non-rapid eye movement (N1), non-rapid eye movement (N2), and non-rapid eye movement (N3).

[0041] (2) Filter the original signal obtained in step (1) and downsample the original signal after filtering out background noise to obtain the preprocessed signal.

[0042] (3) In this embodiment, the sleep stage labeling of the original signal has been obtained in step (1). At this time, it is only necessary to align the preprocessed signal obtained in step (2) with the stage labeling of the original EEG in step (1) to obtain labeled sleep stage training data.

[0043] (4) A heterogeneous data sleep staging network based on a pseudo-twin network architecture was trained using sleep staging training data;

[0044] The sleep staging network model is a network model composed of a feature extraction module and a classifier module based on a pseudo-twin network architecture;

[0045] In general, multimodal signal training models utilize both EEG and EEG signals. However, due to differences in equipment and noise contamination, data channels may be missing or inconsistent. To address this heterogeneous data with potential channel gaps, this embodiment employs a pseudo-Twin neural network architecture to calculate and enhance the feature similarity between EEG and EEG signals. During training, the sleep staging model's input is the preprocessed signal from each subject, with EEG and EEG signals fed into two separate convolutional network branches. The output is the corresponding staging training data. After training, a trained heterogeneous data sleep staging network is obtained.

[0046] The trained sleep staging network obtained in this step takes unlabeled EEG and EEG signals from the sleep process as input and outputs the automatic sleep staging results corresponding to the input signals predicted by the network.

[0047] (5) Use the heterogeneous data sleep staging network obtained after training in step (4) to perform sleep staging on the signal to be staged; the specific steps are as follows:

[0048] (5-1) Obtain the electroencephalogram (EEG) or electrooculogram (EOG) signal of a subject during sleep, or obtain both signals simultaneously, and perform the preprocessing in step (2) on the signals. The signals to be staged are similar in characteristics to the signals obtained in step (1).

[0049] (5-2) In this embodiment, the heterogeneous data sleep staging network obtained in step (4) can be a labeled preprocessed EEG signal or EEG signal, or both signals can be included as input, and the output network can predict the sleep staging result corresponding to the signal.

[0050] Table 1 shows the various indicators obtained by using EEG and EEG signals through a heterogeneous data sleep staging network. It can be seen that the method of the present invention can effectively realize automatic sleep staging of heterogeneous signals.

[0051] Compared to existing sleep staging methods, the method of this invention improves portability across different devices, ease of use, readability, and addresses the issue of missing channels, while effectively achieving sleep staging. Furthermore, the invention employs a deep learning model, which significantly reduces the time and workload of manual sleep staging.

[0052] Table 1

[0053]

Claims

1. A method for staging heterogeneous sleep data based on pseudo-twin networks, characterized in that, This includes constructing a heterogeneous data sleep staging model based on a pseudo-twin network architecture, utilizing deep learning to achieve automatic staging of heterogeneous data; and enhancing the model's perception of different modal signal features by calculating and learning the feature similarity of EEG and EEG signals, obtaining a general pattern of time-series signal features, and improving the model's recognition performance for single signals. The specific steps are as follows: (1) Obtain raw EEG and EEG signals of N subjects in the sleep state, and label the corresponding signals of each subject with the sleep stage; use the stage labeling results of the raw signals as the standard stage results; the sleep stages are divided into: wakefulness (W), rapid eye movement (REM) and non-rapid eye movement (NREM); N is greater than or equal to 20; (2) Filter the original signal obtained in step (1) to remove background noise, and then downsample the filtered signal to obtain the preprocessed signal; (3) Align the preprocessed EEG and EEG signals obtained in step (2) with the stage labels of the original signals in step (1) to obtain labeled sleep stage training data; (4) Construct a heterogeneous data sleep staging model based on a pseudo-twin network architecture; The heterogeneous data sleep staging model consists of two parts: a spatial mapping module and a classifier; wherein: The spatial mapping module contains two identical convolutional neural networks, which are responsible for extracting features from the signal. Each convolutional neural network contains a dual-scale convolutional network and a compressed activation module. The dual-scale convolutional network contains the following network layers: convolutional layer, pooling layer, activation function, and batch normalization. The kernel and stride of the convolutional layer are different. The convolutional layer extracts different feature information by convolving the input data with multiple convolutional kernels. The pooling layer performs a pooling operation on the output of the convolutional layer, compressing feature maps of different scales into vectors of a fixed size. The activation function uses nonlinear transformation to map features to a high-dimensional scale, thereby improving the model's ability to process nonlinear data. The batch normalization calculates the mean and variance of the current batch based on the output data of the previous layer network and performs a normalization operation to prevent the network from overfitting. The compressed excitation module is a convolutional network containing residual connections. It compresses the channel-dimensional features into one dimension and multiplies them with the features obtained from further convolution to obtain enhanced features with certain weights. In the spatial mapping module, the outputs of the two convolutional neural network branches are features extracted from two different modal signals, and both feature maps are fed into the subsequent classifier module. The classifier module includes two fully connected layers and one activation function; the aforementioned feature map outputs the sleep phase after passing through the classifier module. (5) Use sleep staging training data to train a heterogeneous data sleep staging model with a pseudo-twin network architecture; During training, preprocessed EEG and EEG signals from each subject were fed into two separate convolutional neural networks for feature extraction. The extracted features were then analyzed using a similarity metric to increase similarity and reduce differences between features, enabling the model to handle heterogeneous data with inconsistent or missing channels. Features from two different modalities were processed to increase similarity while retaining their unique characteristics, and then fed into the same classifier. The classifier learned from the diverse samples and output the corresponding stage category for each EEG and EEG signal. After training, a trained heterogeneous data sleep staging network based on a pseudo-Twin network architecture was obtained. The operation procedure for step (5) is as follows: (5-1) The EEG and EEG signals obtained in step (3) are sent into the two convolutional neural network branches in the spatial mapping module for feature extraction, and the signals are mapped into a high-dimensional space for subsequent similarity learning. (5-2) Obtain two feature vectors from different modal signals from step (5-1), calculate the similarity between the feature vectors, and use similarity measurement methods to reduce the distance between the feature vectors; similarity measurement methods include cosine similarity, Euclidean distance, and maximum mean difference; (5-3) Two feature vectors from different modal signals are obtained from step (5-1), and are respectively sent into the spatial mapping module to obtain the classification result; the cross-entropy loss function is used to update the parameters of the classification result; (6) Use the heterogeneous data sleep staging network trained in step (5) to perform sleep staging on the heterogeneous signals to be staging.

2. The method for staging heterogeneous sleep data according to claim 1, characterized in that, The specific operation of step (2) is as follows: For the EEG and EEG signals marked through the sleep stage, use a Butterworth bandpass filter to filter out background noise and attenuate signal activity less than 0.3Hz and greater than 35Hz; then use a notch filter to filter out artifacts caused by movement, electrocardiographic activity and electrode interference.

3. The method for staging heterogeneous sleep data according to claim 1, characterized in that, The operation procedure for step (6) is as follows: (6-1) Obtain the electroencephalogram (EEG) and electrooculogram (EOG) signals of a subject during sleep, and perform the preprocessing in step (2) on the signals. The signals to be staged are similar to the signals obtained in step (1). The input contains both signals. (6-2) Input the sleep staging network trained in step (5) into the signal to be staged. The network outputs the sleep staging result of the signal to be staged.

Images