Flexible sleep staging method and device based on multi-channel configuration unified processing

Abstract

The invention discloses a flexible sleep staging method based on multi-channel configuration unified processing, which belongs to the field of sleep monitoring, and comprises the following steps: acquiring original multi-channel electroencephalogram signals of a plurality of users, and labeling the original multi-channel electroencephalogram signals based on sleep stages to form a data set; and constructing a deep neural network model, wherein the model comprises a signal blocking module, a feature extraction module, a feature fusion module and a prediction module. The invention further provides a flexible sleep staging device. According to the method provided by the invention, the generalization performance of the sleep staging model in a multi-center data set can be improved, different computing power environments are adapted by adjusting the number of channels, and diversified clinical application requirements are met.

Description

Technical Field

[0001] The present invention belongs to the field of sleep monitoring, and in particular relates to a flexible sleep staging method and device based on multi-channel configuration and unified processing. Background Art

[0002] Clinically, sleep staging using full-night sleep data (physiological signals such as EEG, electrooculography, and electromyography) recorded by polysomnography (PSG) is the gold standard for sleep monitoring and related disease diagnosis. Traditionally, clinicians or technicians manually stage PSG records based on the AASM sleep staging rules with 30 seconds as a sleep segment. The full-night PSG recording time is about 7 to 9 hours, and it takes an average of 2 hours for doctors to complete sleep staging. This process is not only time-consuming and inefficient, but also easily affected by the doctor's subjective experience.

[0003] In the past few decades, a large number of computer-based automatic sleep staging methods have been proposed. These methods can be divided into two categories according to the algorithms used: 1) Automatic sleep staging methods based on traditional machine learning: preprocessing EEG data to extract relevant EEG features; selecting the extracted features; and classifying the selected EEG features using support vector machines, Bayesian models, random forests and other models.

[0004] 2) Automatic sleep staging method based on deep learning: pre-process the EEG data, train a deep neural network to model the mapping relationship between signals and sleep stages, and perform classification in an end-to-end manner.

[0005] The feature extraction process of the sleep staging method based on machine learning relies heavily on domain knowledge, and because the features are manually designed, it is difficult to extract comprehensive sleep features, so the accuracy and robustness are poor. Although the automatic sleep staging method based on deep learning has achieved an accuracy similar to that of human experts on large-scale data sets, the generalization performance across centers is poor. This is because independent and identically distributed data is a priori assumption of deep learning, but PSG data collection lacks a unified standard, resulting in large differences in data distribution between data sets from different centers, specifically manifested in differences in subject groups, electrode settings, number of channels, sampling rate, signal-to-noise ratio, etc.

[0006] Patent document CN118614893A discloses an end-to-end sleep staging monitoring method based on heart rate variability, wherein the method includes: extracting HRV through unconstrained acquisition of BCG signals, and synchronously acquiring polysomnography signals (PSG), using an end-to-end deep learning network for five-stage training, selecting the most suitable network model by comparing the performance of different networks, and using the selected network model to achieve low-cost home sleep monitoring.

[0007] Patent document CN116269212A discloses a multimodal sleep stage prediction method based on deep learning, including: step 1: obtaining the original PSG signal; dividing the signal into 30s segments for preprocessing; step 2: using a class-balanced data set to perform supervised pre-training on two CNNs of different sizes of the model to prevent overfitting of the sleep stage; step 3: inputting the signal into two pre-trained convolutional neural networks, one large and one small, respectively, and training the filter representing learning after four convolutions and two poolings; step 4: fusing the features obtained by the two convolutional neural networks in step 2, inputting them into the residual learning network, and fusing them again with the features after two bidirectional LSTM modules; step 5: the output obtained in step 3 is passed through a softmax layer to obtain the sleep stage predicted by the model, and the softmax function is combined with the cross entropy loss as the loss function of the model to train the model; and the model is evaluated using the SleepEDF data set. Summary of the invention

[0008] The purpose of the present invention is to provide a flexible sleep staging method and device based on multi-channel configuration and unified processing, which effectively improves the generalization across data sets in the sleep staging process.

[0009] In order to achieve the first objective of the present invention, the following technical solution is provided: a flexible sleep staging method based on multi-channel configuration and unified processing, comprising the following steps:

[0010] Obtaining raw data, which includes raw multi-channel EEG signals of multiple users during sleep, and labeling the EEG signals per unit time step according to the sleep stage, and forming a data set with the raw multi-channel EEG signals and the labels;

[0011] Construct a deep neural network, which includes a signal segmentation module, a feature extraction module, a feature fusion module and a prediction module;

[0012] The signal segmentation module divides the original multi-channel EEG signal of each user into unit steps with time and channel as horizontal and vertical axes to generate time×channel EEG images;

[0013] The feature extraction module extracts features from each EEG image of the user through a convolutional neural network encoder to construct a first feature sequence corresponding to the original multi-channel EEG signal;

[0014] The feature fusion module includes the following steps:

[0015] Channel coding and time series coding are respectively performed on the obtained first feature sequence, and the channel vector and time series vector obtained by coding are concatenated with the first feature sequence to obtain a second feature sequence;

[0016] For the obtained second feature sequence, set the category token corresponding to the time coordinate along the time axis direction, and add the category token to the head of the second feature sequence to construct the corresponding third feature sequence;

[0017] Performing padding and masking operations on the third feature sequence, extracting features in the third feature sequence through a transformer network encoder to obtain a fourth feature sequence;

[0018] The prediction module performs prediction according to the obtained fourth feature sequence to output a sleep staging result of a corresponding category token;

[0019] Training the deep neural network using the data set to obtain a sleep prediction model for predicting sleep stages;

[0020] The original multi-channel EEG signal of the user to be monitored for sleep is input into the sleep prediction model to obtain the corresponding sleep staging results.

[0021] The present invention processes the original sleep EEG data of multiple channels and multiple sleep segments into data blocks of single sleep segments under multiple single channels, extracts features from the data blocks using a convolutional neural network (CNN) encoder to form a first feature sequence, then performs vector encoding on the first feature sequence and concatenates the encoding result with the first feature sequence to obtain a second feature sequence, then introduces a category token into the second feature sequence to form a third feature sequence, and finally performs padding and masking operations on the third feature sequence and performs feature extraction using a transformer network (Transformer) encoder to construct a fourth feature sequence;

[0022] A deep neural network is trained based on a dataset consisting of raw multi-channel EEG signals and sleep stage labels to obtain a high-quality sleep prediction model.

[0023] Specifically, the sleep stages include wakefulness, non-rapid eye movement stage I, non-rapid eye movement stage II, non-rapid eye movement stage III and rapid eye movement stage.

[0024] Specifically, after the original data is obtained, it needs to be preprocessed, and the preprocessing includes data filtering and standard normalization.

[0025] Specifically, the deep neural network is constructed based on a convolutional neural network and a transformer network framework.

[0026] Specifically, the construction process of the second feature sequence is as follows:

[0027] Channel coding and time coding are performed on the first feature sequence to obtain a channel vector list {ce 1,ce 2 ,…,ce |C|} and a list of time series vectors {te 1 ,te 2 ,…,te L}, where |C| represents the maximum number of channels in the original multi-channel EEG signal, and L represents the maximum time sequence in the original multi-channel EEG signal;

[0028] Each feature in the first feature sequence is concatenated with the corresponding channel vector and time series vector, and the corresponding second feature sequence is constructed with the concatenated structure.

[0029] Specifically, the construction process of the third feature sequence is as follows:

[0030] L category tokens are set for the time sequence of the second feature sequence, where L represents the largest time sequence in the original multi-channel EEG signal;

[0031] The L category tokens are added to the head of the second feature sequence to construct a third feature sequence.

[0032] Specifically, the process of the filling and masking operation is as follows:

[0033] For the third characteristic sequences corresponding to all original multi-channel EEG signals, all the third characteristic sequences are filled using the longest characteristic sequence length as a template;

[0034] Mask processing is performed on the filled third feature sequence to generate a corresponding binary mask, where the mask is used to mark the filled positions.

[0035] Specifically, the expression generated by the binary mask is as follows:

[0036]

[0037] Among them, M i,j =1 means that the jth feature block of the i-th sequence is a filling feature, M i,j = 0 means that the jth feature block of the i-th sequence is a valid feature, C i represents the number of channels of the i-th sequence, and L represents the maximum time series in the original multi-channel EEG signal.

[0038] In order to achieve the second objective of the present invention, the following technical solution is provided: a flexible sleep staging device, used to execute the steps of the above-mentioned flexible sleep staging method based on multi-channel configuration and unified processing.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) Unify sleep EEG datasets with different channel configurations and channel numbers into the same model framework, and then incorporate many public datasets into hybrid training to improve the cross-center generalization of the final sleep prediction model.

[0041] (2) After the sleep prediction model is trained and deployed, the accuracy-rate balance can be flexibly adjusted by adjusting the number of channels of input data to meet the needs of platforms with different computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a process flow of a flexible sleep staging method based on multi-channel configuration and unified processing provided in this embodiment;

[0043] Figure 2 A schematic diagram of filling and masking operations of the third characteristic sequence provided in this embodiment;

[0044] Figure 3 A graph showing test results using different EEG channel combinations on the HQ dataset provided in this embodiment;

[0045] Figure 4 Confusion matrix of sleep staging results on the HQ dataset and SYSU center dataset provided in this embodiment;

[0046] Figure 5 The sleep stage shuffle matrix of different subjects on the HQ dataset provided in this embodiment;

[0047] Figure 6 The sleep stage shuffle matrix of different subjects on the SYSU dataset provided in this embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0049] like Figure 1As shown, a flexible sleep staging method based on unified processing of multi-channel configuration is provided in this embodiment. The method is a deep neural network model based on the convolutional neural network (CNN) + transformer network (Transformer) architecture. For the original sleep EEG data of multiple channels and multiple sleep segments, it is first divided into multiple single-channel and single sleep segment data blocks. The EEG data block is input into the sleep segment encoder module (CNN) to extract the features of the single channel and the single sleep segment, and then the obtained feature sequence is input into the sleep sequence encoder module (Transformer) to extract the features between the multi-channel and multiple sleep segments. In order to process batch data with different channel configurations, channel numbers and durations, before the feature sequence is input into the Transformer, each feature in the sequence is first channel encoded and time-series encoded, and then several classification tokens are added to the head of the feature sequence, and then padding and masking operations are performed to make the same batch sequence length the same. The added category token is used as the final classification feature after passing through the Transformer, and a linear layer classifier outputs several sleep stages.

[0050] The specific steps are as follows:

[0051] The original data is obtained, which includes the original multi-channel EEG signals of multiple users during sleep, and the EEG signals per unit time step are labeled according to the sleep stage. The original multi-channel EEG signals and the sleep stage labels constitute a data set.

[0052] For each user's original multi-channel EEG signal, the original multi-channel EEG signal is divided into unit steps with time and channel as the horizontal and vertical axes to generate time×channel EEG images. is a sleep sequence consisting of L sleep segments, where is the number of channels C i , a 30-second sleep segment, f s represents the sampling rate. First, i Block along the time and channel dimensions to obtain L×C i EEG data blocks

[0053] The convolutional neural network (CNN) encoder is used to extract features from each EEG image of the user to construct a first feature sequence corresponding to the original multi-channel EEG signal. i A single-channel, single-sleep segment EEG data block The sleep segment encoder is used to extract features within a single channel or a single sleep segment. Each data block in the sequence passes through the sleep segment encoder in turn and outputs the corresponding features. The sleep segment encoder outputs a length of L×C i The first characteristic sequence e i .

[0054] Channel coding and time sequence coding are performed on the obtained first feature sequence respectively, and the channel vector and time sequence vector obtained by coding are concatenated with the first feature sequence to obtain a second feature sequence. In this embodiment, the above steps obtain a feature sequence with a length of L×C i The first characteristic sequence e i The key point of the present invention is to use the Transformer network to process EEG data with different channel numbers and channel configurations. Since the Transformer lacks a mechanism to capture sequence position information, when the first feature sequence e i Before entering the Transformer, the sequence e i Each feature in is channel coded and time-series coded. Specifically, a channel coding list CE = {ce 1 ,ce 2 ,…,ce |C|} and a temporal encoding list TE = {te 1 ,te 2 ,…,te L}, where the channel encoding vector ce c (c=1,…,|C|) and temporal coding te l The dimensions of (l=1,…,L) are d ce and d te , |C| and L represent the maximum number of channels and the maximum time series in the dataset, respectively. i Each block feature According to the channel and time sequence position, the corresponding encoding vector is taken out from the channel encoding list and the time sequence encoding list respectively, and concatenated with the block feature to obtain the encoded block feature. This process is expressed as:

[0055]

[0056] Among them, ⊕ represents the vector concatenation operation. The channel encoding list CE and the temporal encoding list TE are learnable parameters.

[0057] For the obtained second feature sequence, set the category token corresponding to the time coordinate along the time axis direction, and add the category token to the head of the second feature sequence to form a third feature sequence. The generation method of the category token described in this embodiment is as follows: the category token is a randomly initialized vector, the dimension of each vector is the same as the dimension of each feature in the second feature sequence, and the number of category tokens is equal to the number of times. The category token is part of the neural network parameter, which is iterated during the training process and remains fixed after the training is completed. In the second feature sequence Add L learnable category tokens to the head, with a composition length of L×(C i +1)'s third characteristic sequence

[0058] Perform padding and masking operations on the third characteristic sequence. Figure 2 As shown, the process described in the above steps is for a single EEG sequence sample. The batch data input in the cross-dataset mixed training process includes multiple EEG sequence samples from different data sets. The number of channels of these samples is different, so the length of the EEG feature sequence is different. Due to the limitation of the underlying hardware, the computer cannot perform matrix operations on such indefinite length data. In order to solve this problem, the present invention uses the method of filling and blocking. Specifically, for the same batch of data B represents the batch size, and its corresponding feature sequence is The length of the third characteristic sequence of the i-th sample is L·(C i +1), take the maximum sequence length L·(C max +1), and pad all sequences to the maximum sequence length. After padding, we generate a binary mask Its value is given by:

[0059]

[0060] Among them, M i,j =1 means that the jth feature block of the i-th sequence is a filling feature, M i,j = 0 means that the jth feature block of the ith sequence is a valid feature. This mask plays a masking role in two places in the training process. First, it is used as an attention mask and input into the Transformer together with the padded feature sequence, so that it ignores the padded feature blocks when calculating attention; second, when calculating the loss function, only the loss of valid data is calculated, and the padded data is not calculated.

[0061] The fused features are extracted through the transformer network (Transformer) encoder to generate the corresponding fourth feature sequence. In this embodiment, the role of the sleep sequence encoder is to encode the third feature sequence after padding and masking, and extract the features in the sleep sequence (multi-channel, multiple sleep segments). The sleep sequence encoder is based on the Transformer architecture and consists of N Transformer modules. Each Transformer module consists of a multi-head attention layer and a feedforward neural network layer, and layer normalization is performed between the two layers. Assume that the input batch feature sequence is E 0 , the output feature sequence is E out , then this process can be expressed as:

[0062]

[0063]

[0064] E out =LN(E N )

[0065] The feature sequence output by Transformer has the same shape as the input feature sequence, and the first L feature blocks of the output feature sequence (features corresponding to the category tokens) are used as the fourth feature sequence E cls , that is, E cls =E out [1:L].

[0066] The function of the classifier is to classify the fourth feature sequence E output by the encoder (including the sleep segment encoder and the sleep sequence encoder) cls Classify and output the probability of each sleep stage. The classifier consists of a linear layer, and the number of output neurons is equal to the number of sleep stage categories. The output of the linear layer is activated by the softmax function to obtain the category probability distribution, and then the argmax function is used to obtain the classification result. Classification feature E cls The L feature blocks in are passed through the classifier in sequence, and finally the classification results of L sleep segments are obtained.

[0067] Training the deep neural network using the data set to obtain a sleep prediction model for predicting sleep stages;

[0068] The original multi-channel EEG signal of the user to be monitored for sleep is input into the sleep prediction model to obtain the corresponding sleep staging results.

[0069] In addition, the third feature sequence is input into the transformer network to extract sequence features. The features of different channels at the same time will be fused into the category token corresponding to the time, and the category token will be used as the input feature of the classifier for sleep stage classification. Other feature blocks except the category token will be discarded.

[0070] This embodiment also provides a flexible sleep staging device, which is used to execute the steps of the flexible sleep staging method based on multi-channel configuration and unified processing provided in the above embodiment.

[0071] In order to better illustrate the effect of the technical solution provided by the present invention, examples are given for illustration.

[0072] We collected 10 public datasets as source domains for training and tested them on 2 target center datasets. The configuration of the public datasets is shown in Table 2, and the configuration of the private datasets is shown in Table 3.

[0073] Table 2

[0074]

[0075] The APPLES (The Apnea Positive Pressure Long-term Efficacy Study) dataset is a multicenter dataset used to study the effect of sleep apnea treatment. It contains all-night PSG records of 1,104 patients with obstructive sleep apnea syndrome (OSAS). The dataset contains a total of 6 EEG / EOG channels, and uses the R&K standard for sleep stage annotation.

[0076] Table 3

[0077]

[0078] The DCSM (Danish Center for Sleep Medicine) dataset

[32] is a collection of 255 PSG records randomly selected from patients with non-specific sleep-related diseases who visited the Danish Center for Sleep Medicine. The dataset contains eight EEG / EOG channels and uses the AASM standard for sleep stage annotation.

[0079] The DOD (Dreem Open Dataset) dataset consists of two subsets: DOD-H and DOD-O. DOD-H comes from the French Armed Forces Biomedical Research Institute (IRBA) and contains PSG records of 25 healthy volunteers; DOD-O comes from the Stanford Sleep Medicine Center and contains PSG records of 56 OSAS patients. DOD-H and DOD-O have 12 and 8 EEG channels respectively. We only selected 5 and 7 channels that overlap with the target center dataset for experiments. Both have 2 EOG channels, and both use the AASM standard for sleep stage annotation.

[0080] The HMC (Haaglanden Medical Center) dataset consists of 151 PSG records from Haaglanden Medical Center in the Netherlands. It was randomly selected from heterogeneous patients who underwent PSG examinations for different sleep disorders in 2018. It contains 6 EEG / EOG channels and uses the AASM standard for sleep stage annotation.

[0081] The ISRUC (Institute of Systems and Robotics, University of Coimbra) dataset is composed of 126 PSG records from the Sleep Medicine Center of the University Hospital of Coimbra, Portugal. It consists of three different data sets: the first set of data is 100 PSG records of 100 patients with confirmed sleep disorders; the second set of data is 16 PSG records of 8 patients with confirmed sleep disorders; the third set of data is 10 PSG records of 10 healthy volunteers. The dataset contains 8 EEG / EOG channels and uses the AASM standard for sleep stage annotation.

[0082] The CAP (Cyclic Alternating Pattern) dataset is a collection of 108 PSG records from the Sleep Disorders Center of the Ospedale Maggiore of Parma, Italy, covering subjects such as healthy people, patients with bruxism, insomnia, narcolepsy, nocturnal frontal lobe epilepsy, periodic lower limb movements, REM sleep behavior disorder, and sleep-disordered breathing. The dataset contains 4 EEG / EOG channels and is annotated based on the R&K sleep staging standard.

[0083] The SVUH (St. Vincent's University Hospital) dataset is a PSG record of 25 patients suspected of sleep breathing disorders from St. Vincent's University Hospital in Ireland. It contains 4 EEG / EOG channels and uses the R&K standard for sleep stage annotation.

[0084] The P2018 (You Snooze You Win: The PhysioNet / Computing in Cardiology Challenge 2018) dataset comes from the 2018 open challenge and is provided by the Massachusetts General Hospital Computational Clinical Neurophysiology Laboratory (CCNL) and the Clinical Data Animation Laboratory (CDAC). The data consists of 994 training data and 989 test data, of which only the training data provides sleep stage annotations. We used labeled training data, including 7 EEG / EOG channels, and used the AASM standard for sleep stage annotation.

[0085] The STAGES (Stanford Technology Analytics and Genomics in Sleep) dataset is a prospective cross-sectional multicenter study involving 20 data collection sites in six centers including Stanford University. The dataset contains 1833 full-night PSG records of 1500 adult / adolescent patients with sleep disorders, including 8 EEG / EOG channels, annotated using the AASM sleep staging standards.

[0086] The SHHS (Sleep Heart Health Study) dataset comes from a multicenter cohort study conducted by the National Heart, Lung, and Blood Institute (NHLBI) of the United States, which aims to determine the cardiovascular and other consequences of sleep-disordered breathing. The dataset is divided into two subsets, SHHS-1 and SHHS-2, according to the collection time. SHHS-1 contains 5793 PSG records from 1995 to 1998, and SHHS-2 contains 2651 PSG records from 2001 to 2003. The SHHS dataset contains 4 EEG / EOG channels, which are annotated based on the R&K sleep stage standard.

[0087] In summary, we collected about 13,000 full-night PSG records from public datasets as training data, with a sleep record duration of about 115,000 hours. In order to evaluate the system of this embodiment, we used private data from two different centers. The data of these two centers are introduced as follows:

[0088] The HQ dataset comes from Hangzhou Seventh People's Hospital (Mental Health Center Affiliated to Zhejiang University School of Medicine), and contains PSG records of 127 subjects, including 33 healthy subjects, 51 narcolepsy patients, and 43 depression patients. The dataset contains 8 EEG / EOG channels with a sampling rate of 512Hz, and is annotated by physicians using the AASM sleep staging standard.

[0089] The SYSU dataset comes from the Sleep Laboratory of Sun Yat-sen University, and includes 80 PSG records from 21 healthy volunteers and 24 PSG records from 24 depressed patients. The dataset contains 8 EEG / EOG channels, and the sampling rates for healthy subjects and depressed patients are 500Hz and 256Hz, respectively. The PSG records were scored by two well-trained sleep technicians according to the AASM criteria.

[0090] For all PSG datasets, we selected the 8-channel EEG / EOG recommended by the AASM standard as the basis for sleep staging, and other channels and other forms of data in the dataset were not used. These 8 channels are F3-M2, F4-M1, C3-M2, C4-M1, O1-M2, O2-M1, E1-M2, and E2-M1.

[0091] In raw data processing, we first filtered all EEG / EOG channels with a fourth-order bandpass filter of 0.3-35 Hz; then resampled the data with different sampling rates to 100 Hz; finally, Z-score normalized each channel of each PSG data:

[0092]

[0093] Where x represents an example of original data, C represents the channel set of the data, mean and std represent the mean and standard deviation respectively.

[0094] In terms of label processing, we use the current AASM sleep staging standard. For the dataset using the R&K standard, we use the common practice to merge N3 and N4 into N3. In addition, we delete unlabeled sleep segments, which often mean sensor detachment, sleep interruption or other abnormal conditions, and divide the data into two segments before and after the deletion.

[0095] In order to quantitatively evaluate the sleep staging effect of the system of this embodiment on the target center data, accuracy, macro-F1 score, kappa coefficient and category F1 score are selected as evaluation indicators. is the predicted label of the i-th sample, y i is the corresponding true label, then the accuracy is defined as:

[0096]

[0097] where Ι(·) is the indicator function and N is the number of samples.

[0098] The F1 score is the harmonic average of precision and recall. The macro F1 score is the average of the F1 scores for each category and is defined as:

[0099]

[0100] in,

[0101]

[0102] TP i , FP i and FN i They represent the number of true positives, false positives, and false negatives of the ith category, respectively, and K is the number of categories.

[0103] The kappa coefficient compares the consistency between the model's predicted labels and the true labels and is defined as:

[0104]

[0105] Where P e It is the prediction accuracy under random classification. The Kappa coefficient is a value between -1 and 1. A coefficient higher than 0.8 is generally considered to have good consistency; a coefficient lower than 0 means no consistency.

[0106] In this example, the sleep segment encoder is based on the CNN (Convolution Neural Network) architecture and uses a dual-branch design with 1D convolution kernels of different sizes to extract features of different frequency bands. Branch 1 uses a smaller convolution kernel to extract high-frequency features, and branch 2 uses a large convolution kernel to extract low-frequency features. The specific parameters of the two branches are shown in Table 5, where the parameters after Conv1d represent the number, size, step size and padding of convolution kernels, and the parameters after MaxPool represent the pooling kernel size, step size and padding. The outputs of the two branches are spliced ​​along the feature dimension, and finally output 512-dimensional block features.

[0107] The sleep sequence encoder is based on the Transformer architecture and consists of 4 Transformer modules. The dimension is d = 512 + 32 (channel encoding vector) + 64 (temporal encoding vector) = 608, the number of multi-head attention heads is h = 8, and the hidden layer dimension of the feedforward neural network is d fft =d=608.

[0108] The classifier consists of a linear layer and a softmax activation layer. The number of input neurons is equal to the dimension of the sleep sequence encoder output, that is, 608; the number of output neurons is equal to the sleep stage category to be classified. This embodiment uses sleep stage 5 classification. The parameters of the sleep segment encoder model are shown in Table 4.

[0109] Table 4

[0110]

[0111] In this example, the weighted cross entropy function is used as the loss function of the training process, and the L2 regularization technique is combined to prevent overfitting.

[0112] For the K classification problem, given a sample x i and its corresponding one-hot encoded label in Let the model output after the softmax layer be in Represents sample x i The probability of being predicted as the kth class, then the weighted cross entropy loss function can be expressed as

[0113]

[0114] where w k is the weight of each category. In this embodiment, the normalized value of the reciprocal of the number of samples in each category is used as the category weight.

[0115] The essence of L2 regularization is to add a penalty term based on the model parameter norm in the loss function to limit the model complexity. Specifically, let the model parameter be p, then the L2 norm penalty term is

[0116]

[0117] The overall loss function expression is:

[0118]

[0119] Where λ≥0 is an adjustable hyperparameter used to control the regularization strength. In this embodiment, λ is set to 0.001.

[0120] This embodiment uses Python as the programming language, Pytorch as the deep learning framework, and MATLAB as the data processing and visualization tool. Table 5 lists the relevant software and hardware environment configurations.

[0121] Table 5

[0122] condition Related parameters operating system Ubuntu 20.04.4LTS CPU Intel(R)Xeon(R)Gold 6330CPU@2.00GHz RAM 500GB GPU 2×NVIDIA A800 80GB Graphics Driver nVidia ver.525.105.17 CUDA ver.12.0 Pytorch ver.2.0.0 MATLAB R2022b

[0123] In this embodiment, 10 public datasets are mixed as source domain datasets, and HQ and SYSU datasets are used as target center datasets. The sleep stage model is trained on the source domain and then tested on the target center dataset.

[0124] In this embodiment, the category weight of the weighted cross entropy loss function is set to w = [0.09, 0.44, 0.06, 0.23, 0.19], which is the normalized value of the inverse of the category ratio in the source domain data. In addition, the sleep sequence length L is set to 20, that is, the sequence length is 10 minutes.

[0125] In addition, in the hybrid training process, in order to prevent the model from over-relying on certain channels or relying on the features of too many channels, this embodiment applies a data augmentation method of random channel discarding: for a channel number of C i Sample x i , where each channel has a probability of p being discarded, and p is set to 0.5.

[0126] This example uses AdamW as the optimizer, and the initial learning rate lr 0 The L2 regularization coefficient λ is set to 1e-4, the L2 regularization coefficient λ is set to 1e-3, and the batch size is set to 256. The total number of training rounds is set to 80, of which the warm-up phase is 15 rounds. The learning rate in the warm-up phase increases linearly from 0. After the warm-up phase, the learning rate decays according to the cosine annealing strategy, and the minimum learning rate lr min Set to 1e-6. The learning rate of this training process is expressed as

[0127]

[0128] Where e is the current round number.

[0129] After training, we tested the two target center data HQ and SYSU to evaluate the generalization ability of the model of the present invention. Since the model of the present invention can input EEG signal samples with any number of channels, we tested different combinations of channels. The experimental results below use all 8 channels of data by default unless otherwise specified. The final results are shown in Table 1.

[0130] Table 1 shows the test results of the embodiment at two centers, HQ and SYSU. Benchmark 1 represents the result of training on the data of one center and testing on the data of the other center, which is used as the reference value of the lower bound of the performance of the present invention; Benchmark 2 represents the average result of five-fold cross-validation on the local center data, which is used as the reference value of the upper bound of the performance of the present invention. The results in Table 1 show that the model and method provided by the present invention achieve similar performance to the local training model on the data that has never been seen (relative to benchmark 2) through mixed training on large-scale public data sets, showing good cross-center generalization performance.

[0131] Table 1

[0132]

[0133] like Figure 3 As shown in FIG. 2 , the results of testing using different EEG channel combinations on the HQ dataset are shown. When all 8 sleep EEG channels are used, the embodiment achieves the highest accuracy. As the number of channels used decreases, the accuracy of the embodiment decreases slightly, but the inference speed increases exponentially.

[0134] Figure 4 Shown is the confusion matrix of the sleep staging results of this embodiment on the HQ and SYSU datasets. Figure 5 Shown is the sleep stage confusion matrix of healthy subjects, depression patients and narcolepsy patients on the HQ dataset in this embodiment. Figure 6 Shown is the sleep stage confusion matrix of healthy subjects and depression patients on the SYSU dataset in this embodiment.

[0135] in Figure 4 The confusion matrix of the sleep staging results of the embodiment of the present invention on the HQ dataset and the SYSU dataset is shown. The results show that the prediction accuracy of the model on both the HQ and SYSU datasets is high, and the values ​​of the diagonal position are significant, indicating that the model has a good recognition effect on each sleep stage and a high classification accuracy. At the same time, the confusion between different sleep stages is low. The confusion matrix verifies the generalization ability of the embodiment of the present invention on data from different data centers, that is, it can still maintain a high staging accuracy in cross-dataset testing.

[0136] Figure 5The confusion matrix of sleep staging results for different subject groups (healthy subjects, patients with depression, and patients with narcolepsy) on the HQ dataset is shown. It can be seen that the staging accuracy of the model in these three groups is consistent, especially in the staging of healthy and narcolepsy patients, the classification accuracy of the diagonal position is high, indicating that the model can adapt to the EEG characteristics of these subjects and perform accurate staging. In the group of patients with depression, although the accuracy rate dropped slightly, the overall performance is still relatively stable. Figure 5 It was verified that the model of the present invention has strong adaptability to subjects with different sleep disorders.

[0137] Figure 6 The confusion matrix of the sleep staging results of healthy subjects and patients with depression on the SYSU dataset is shown. It can be observed from the confusion matrix that the staging accuracy of healthy subjects and patients with depression in different sleep stages is relatively close, indicating that the model can still maintain efficient cross-population staging accuracy when dealing with subjects with different physiological characteristics. This result shows that the embodiments of the present invention show good generalization when dealing with different individual characteristics.

[0138] Table 6

[0139]

[0140] Table 6 shows the comparison of the sleep staging method provided by the present invention with the results of other state-of-the-art methods on HQ and SYSU datasets. The biggest innovation of the method of the present invention is that it can process datasets with different channel configurations, which is not available in existing methods. For comparison, in this embodiment, the intersection of sleep EEG channels in all datasets (C3-M2, E1-M2) is used to train the existing method. In Table 6, 2C represents the above-mentioned 2 common channels, and 8C represents all 8 channels.

[0141] The beneficial effects of the present invention include two aspects:

[0142] (1) The present invention breaks the limitation of channel configuration differences between different datasets in the field of sleep EEG staging, unifies sleep EEG datasets with different channel configurations and channel numbers into a model framework, and then can incorporate many public datasets into hybrid training, thereby improving the cross-center generalization of sleep staging methods.

[0143] (2) After training and deployment, the model provided by the present invention can flexibly adjust the accuracy-rate balance by adjusting the number of channels of input data to meet the needs of clinical platforms with different computing resources.

Claims

1. A flexible sleep staging method based on multi-channel configuration and unified processing, characterized in that: include: Obtaining raw data, which includes raw multi-channel EEG signals of multiple users during sleep, and labeling the EEG signals per unit time step according to the sleep stage, and forming a data set with the raw multi-channel EEG signals and the labels; Construct a deep neural network, which includes a signal segmentation module, a feature extraction module, a feature fusion module and a prediction module; The signal segmentation module divides the original multi-channel EEG signal of each user into unit steps with time and channel as horizontal and vertical axes to generate time×channel EEG images; The feature extraction module extracts features from each EEG image of the user through a convolutional neural network encoder to construct a first feature sequence corresponding to the original multi-channel EEG signal; The feature fusion module includes the following steps: Channel coding and time series coding are respectively performed on the obtained first feature sequence, and the channel vector and time series vector obtained by coding are concatenated with the first feature sequence to obtain a second feature sequence; For the obtained second feature sequence, set the category token corresponding to the time coordinate along the time axis direction, and add the category token to the head of the second feature sequence to construct the corresponding third feature sequence; Performing padding and masking operations on the third feature sequence, extracting features in the third feature sequence through a transformer network encoder to obtain a fourth feature sequence; The prediction module performs prediction according to the obtained fourth feature sequence to output a sleep staging result of a corresponding category token; Training the deep neural network using the data set to obtain a sleep prediction model for predicting sleep stages; The original multi-channel EEG signal of the user to be monitored for sleep is input into the sleep prediction model to obtain the corresponding sleep staging results.

2. The flexible sleep staging method based on multi-channel configuration and unified processing according to claim 1, characterized in that: After the raw data is acquired, it needs to be preprocessed, and the preprocessing includes data filtering and standard normalization.

3. The flexible sleep staging method based on multi-channel configuration and unified processing according to claim 1, characterized in that: The deep neural network is constructed based on a convolutional neural network and a transformer network framework.

4. The flexible sleep staging method based on multi-channel configuration and unified processing according to claim 1, characterized in that: The construction process of the second feature sequence is as follows: Channel coding and time series coding are performed on the first feature sequence to obtain a channel vector list {ce1, ce2, …, ce |C| } and a list of time series vectors {te1,te2,…,te L }, where |C| represents the maximum number of channels in the original multi-channel EEG signal, and L represents the maximum time sequence in the original multi-channel EEG signal; Each feature in the first feature sequence is concatenated with the corresponding channel vector and time series vector, and the corresponding second feature sequence is constructed with the concatenated structure.

5. The flexible sleep staging method based on multi-channel configuration and unified processing according to claim 1, characterized in that: The construction process of the third feature sequence is as follows: L category tokens are set for the time sequence of the second feature sequence, where L represents the largest time sequence in the original multi-channel EEG signal; The L category tokens are added to the head of the second feature sequence to construct a third feature sequence.

6. The flexible sleep staging method based on multi-channel configuration and unified processing according to claim 1, characterized in that: The process of the filling and masking operation is as follows: For the third characteristic sequences corresponding to all original multi-channel EEG signals, all the third characteristic sequences are filled using the longest characteristic sequence length as a template; Mask processing is performed on the filled third feature sequence to generate a corresponding binary mask, where the mask is used to mark the filled positions.

7. The flexible sleep staging method based on multi-channel configuration and unified processing according to claim 6, characterized in that: The expression generated by the binary mask is as follows: Among them, M i,j =1 means that the jth feature of the i-th sequence is a filling feature, M i,j = 0 means that the jth feature of the i-th sequence is a valid feature, C i represents the number of channels of the i-th sequence, and L represents the maximum time series in the original multi-channel EEG signal.

8. A flexible sleep staging device, characterized in that: The method is used to execute the steps of the flexible sleep staging method based on multi-channel configuration and unified processing as described in any one of claims 1 to 7.

Images