Double-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation

Abstract

The double-teacher sleep staging feature migration method based on knowledge distillation and domain adaptation belongs to the field of signal processing and pattern recognition. First, the sleep electroencephalogram and electrooculogram signals are preprocessed to obtain a plurality of multi-modal sleep signal data samples. Next, each channel contained in each sample of the source domain and target domain data is sequentially extracted using Morlet wavelet transform of different resolutions to obtain time-frequency features, and then input into the source domain teacher and target domain teacher for pre-training. In the training and optimization of the student, the two teachers with frozen feature extractors are introduced for guidance to constrain the student to learn the source domain and target domain general features and the domain-specific features of the target domain. Experiments prove that the model proposed in the application fully utilizes the features of the data for feature migration, and good results can be obtained when the amount of target domain data is small, which can effectively solve the problem that the existing automatic sleep staging method has a decreased accuracy when facing new data sets.

Description

Technical Field

[0001] This invention relates to EEG signal processing technology, feature extraction, and deep learning, belonging to the technical fields of signal processing and pattern recognition. Specifically, it relates to a dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation. Background Technology

[0002] Sleep staging involves determining a person's sleep state using polysomnography (PSG) recorded during sleep. Analyzing PSG is a time-consuming and labor-intensive process, and due to the need for continuous evaluation over extended periods, it is prone to human error. In recent years, data-driven deep learning methods have been developed, relying on the availability of large amounts of labeled data for training. Many studies have proposed using key signals containing features of each stage for automatic sleep stage classification. These techniques fall into three categories: 1) manual feature extraction using automated decision-making algorithms; 2) applying extracted features to deep learning methods; and 3) end-to-end training using deep learning methods, including convolutional neural networks (CNNs) or recurrent neural networks (RNNs). However, many sleep laboratories still use manual scoring, primarily because there are significant discrepancies between public training data and data generated in the sleep laboratory. These discrepancies are often due to domain shift problems caused by differences in data distribution between training (source) and test (target) data due to variations in acquisition equipment, channels, and environments. Domain shift problems prevent models trained in the source domain from achieving good results in the target domain. Therefore, effective transfer learning methods are needed to address the decrease in sleep staging accuracy when transferring models to different domains.

[0003] Currently, the application of transfer learning to sleep studies addresses potential issues within the same dataset: accuracy can decline when sleep models are compatible with new data or when handling variations in data distribution caused by differences in the number and location of EEG channels, sampling frequency, experimental protocols, and participants. These studies fine-tune the model to serve each participant, with each subject having data recorded twice a night: the first night for model fine-tuning and the second night for evaluation. The application of transfer learning to sleep staging models has also ushered in a new phase in the research of sleep staging algorithms.

[0004] However, fine-tuning methods require large source datasets to improve generalization ability, and fine-tuning uses data segments from the target domain to improve the robustness of deep learning models to individual differences. Models trained using this method still suffer from weak generalization when directly tested on new data. In machine learning problems where two domains have different data distributions but the same task, domain adaptation methods can be used to transfer a high-accuracy model trained on source domain data to a target domain with less data. Domain adaptation methods can overcome the challenge of requiring large source datasets to improve generalization ability. Some research has proposed using a fully shared model for domain alignment, which addresses the problem from the perspective of shared domain features, but may lose information about specific domains during feature extraction. Knowledge distillation can be seen as a special case of transfer learning, a supervised learning method that assists the training process of a smaller network (student) under the supervision of a larger network (teacher). This simplifies the student model and improves its performance and generalization ability. A common training method for knowledge distillation is to have the student model fit the probability of each category output by the teacher model, similar to training a student with basic facts while ignoring the importance of intermediate processes. Feature distillation can increase the amount of information transferred during the distillation process, thereby improving the performance of the student network. Current domain-adaptive transfer learning methods are unsupervised and ignore data labeling information. This invention uses a combination of domain adaptation and feature distillation for feature transfer, which can effectively learn features from different domains, thus achieving good accuracy in sleep staging within the target domain.

[0005] At the feature level, sleep staging data is time-series data, and time-frequency features are its effective features. This invention uses continuous wavelet transform to extract its time-frequency features, and then uses a deep learning network to automatically learn higher-order features. Label information is crucial for feature learning in sleep staging data. The feature transfer model of this invention fully considers the label information of the data during training, making it a supervised domain adaptation method. For sleep staging tasks involving two sleep staging data domains with different data distributions, the data features have domain-aligned and domain-specific components. However, research on domain adaptation only considers the shared features between the source and target domains, ignoring the specific features of the target domain. Single-teacher-student models exist for domain adaptation methods, but these models tend to make the student learn features of the target domain closer to the source domain, potentially losing knowledge of the target domain. This invention, based on a feature knowledge distillation framework, designs a cross-domain dual-teacher model to guide the student in learning the feature representations of the source and target domains. Furthermore, during feature distillation, the features extracted by the student and teacher networks are high-dimensional and redundant; therefore, key features must be extracted for learning during feature distillation. This invention, from the perspective of feature transfer, designs a method using a dual-teacher knowledge distillation approach to guide students in extracting domain-general and domain-specific features from source and target domain sleep stage data. The method mainly comprises three steps: (a) The first teacher trains using the source domain data, applying partial distance constraints in the intermediate layers of the network to allow students to extract features similar to those in the source domain; (b) The second teacher trains using a small amount of data from the target domain, applying orthogonal loss constraints in the intermediate layers of the network to allow students to extract target domain-specific features; (c) The distillation loss for intermediate features is designed to be coordinated across multiple aspects: teacher transformation, student transformation, encoder extraction of key features from the intermediate layers, and feature difference calculation. The first teacher is the source domain teacher, a complex model trained using source domain data. It guides the student in learning general feature representations of both the source and target domains; specifically, in the intermediate feature layers, it enables the student to learn similar feature representations in both domains. The second teacher is the target domain-trained model, trained using partial target domain data. It guides the student in learning specific target domain features, specifically sleep stage features that differ from the source domain's feature representations, in the intermediate feature layers. Furthermore, it proposes using feature transformations to extract key features from the intermediate layers of the network during constrained feature extraction. This collaboration between the two teachers allows the student to learn features from both the source and target domains, ensuring a smooth training process and achieving good results. Summary of the Invention

[0006] To address the issue of decreased model accuracy with new datasets and the challenge of efficiently utilizing features from different domains for feature transfer, this invention proposes a dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation. This method enables the model to fully leverage information from the source domain for feature transfer and incorporates a distillation loss design. The feature transfer method consists of two parts: first, training two teacher models; and then, training the student model using the teacher models and the designed distillation loss.

[0007] The main idea behind this invention is as follows: The model is trained from the perspective of data features, including time-frequency features of time-series sleep data and features automatically extracted by the neural network. Based on a teacher-student knowledge distillation framework, two teacher models are designed, using domain adaptation to guide the student network's feature learning. This enables the student network to learn general features of sleep stages in both the source and target domains, as well as domain-specific features of the target domain data. The source domain teacher model is trained using the source domain sleep dataset, and the target domain teacher model is trained using a small amount of sleep stage data from the target domain. The student model is the final model used to predict the target domain, and its structure is simpler than that of the teacher models. The loss function for the student model during training is designed as the sum of classification loss and feature distillation loss. Finally, the effectiveness of using knowledge distillation and domain adaptation for feature transfer on the target domain dataset is analyzed and validated using a public dataset.

[0008] A dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation includes the following steps:

[0009] Step 1: Acquire sleep physiological signals from the source and target domains and perform data preprocessing:

[0010] Publicly available sleep data is used as source domain samples, and the data from the dataset to be tested is used as target domain samples. Assume that for this method framework, there are two sets of source domain training data (SourceTrainData1 and SourceTrainData2), two sets of target domain training data (TargetTrainData1 and TargetTrainData2), one set of target domain test data (TargetTestData), and one set of target domain data (TargetFinetuneData) for fine-tuning. Based on the polysomnography, one sleep EEG channel and one EEG channel signal are selected from the original signal containing M signal channels. Then, the selected sleep data with a sampling rate of fs is downsampled to f. Next, a sliding window of length 30 seconds is used to divide the data into N non-overlapping sample data segments. Each sample data contains 2 channels, and the data length of each channel is 30*f. In this invention, f is taken as 100Hz. Then, the sliced ​​data matrix X = [x1, x2, ..., x...]. NX is N×2×3000.

[0011] Step 2, Time-Frequency Feature Extraction of Sample Data:

[0012] For each sample data x obtained in step one i The time-frequency features of the signal are extracted by sequentially applying Morlet continuous wavelet transform (1Hz-30Hz, with a center frequency ω0 every 0.5Hz) to each channel to represent the local features of the signal. Simultaneously, one extracted time-frequency feature is retained every 50 features to reduce data dimensionality. The resulting time-frequency matrix after time-frequency transformation of the above sample data is then obtained. W X The dimensions are N×2×60×60. Time-frequency features are extracted from the training and testing data sets in step one to obtain the time-frequency matrix.

[0013] Step 3, Training the dual-teacher feature extractor:

[0014] This invention designs two teacher models: a source domain teacher and a target domain teacher. The time-frequency matrix extracted from the training domain in step two is used as input to the teacher's deep learning network for training. (The time-frequency matrix of the source domain training data is shown in the original text.) The source domain teacher data is fed in and the real labels of the training data are used to train the feature extractor F for the source domain teacher data. t The time-frequency matrix of the target domain training data. The target domain teacher is fed in, and the real labels of the training data are used to train the feature extractor F′ for the target domain teacher. t .

[0015] Step 4, Knowledge Distillation and Domain-Adaptive Feature Transfer:

[0016] Then, knowledge distillation and domain adaptation methods are used to transfer the features from the two teachers to the student model. During the training of the student model, the feature extractors F from the two teacher models trained in step three are imported. t and F′ t Then, its parameters are frozen and not updated during the student model training process. The teacher model uses its prior knowledge from training to extract features from the input and then uses the loss to optimize the student model. The training of the student model is divided into two parts. First, the time-frequency matrix of the source domain data extracted in step two is used... The student model is trained using the input, followed by the time-frequency matrix of the target domain data. The student model continues to be trained as input. Input data is simultaneously fed into the student model and two teacher models. Then, the feature knowledge extracted by the intermediate layer of the teacher network's feature extractor is transferred to the student network. The feature transfer process uses distillation loss to constrain the student network to learn domain-general features of the source and target domains, as well as domain-specific features of the target domain, from the two teacher models.

[0017] Step 5: The student model classifies the samples in the target domain.

[0018] The time-frequency features of the target domain samples obtained in step two. Input the student network obtained after training in step four. The features extracted by the feature extractor of the student network are then fed into the linear classifier to obtain the classification result.

[0019] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0020] This invention creatively combines knowledge distillation, domain adaptation, and feature transfer to propose a dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation. In the sleep staging task, combining continuous wavelet time-frequency feature extraction and domain feature extraction methods, the method first learns the diagnostic approach of identifying time-frequency information in data during the sleep staging process, simulating a physician's process. Then, from the perspective of feature transfer, it learns the source and target domains, fully considering the characteristics of the feature distributions of the source and target domains. When extracting domain features, unlike domain adaptation methods that use a completely shared model for domain alignment, this invention proposes a source domain teacher and student using non-shared feature extractors, with the teacher guiding the student network to learn domain-general features. The proposed target domain teacher guides the student network to learn domain-specific features of the target domain. Simultaneously, when constraining features, considering the complexity of the network and features, it solves the problem of excessive memory consumption for loss calculation by encoding transformations and feature dimensionality reduction. Finally, different loss functions are used to constrain the feature extraction of the student network. This invention fully considers the features of the data, enabling the student network to learn the data features more comprehensively, resulting in better data classification results. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method involved in the present invention;

[0022] Figure 2 This is a diagram of the overall network framework of the present invention;

[0023] Figure 3 This invention relates to a target domain teacher-guided student distillation process and feature encoding module for a target domain specific feature;

[0024] Figure 4This invention describes the process of using a source domain teacher to guide students in performing a domain-general feature distillation process.

[0025] Figure 5 This is a schematic diagram comparing the results of the classification method designed in this invention with the results of manual classification by professional physicians. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0027] A dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation is illustrated in the flowchart below. Figure 1 As shown, the overall network framework diagram is as follows: Figure 2 As shown.

[0028] Step 1: Obtain raw sleep physiological signals from the source and target domains and perform data preprocessing:

[0029] Publicly available sleep data is used as the source domain sample, and the data from the test dataset is used as the target domain sample. The data samples are divided as follows: for this method framework, there are two sets of source domain training data (SourceTrainData1 and SourceTrainData2), two sets of target domain training data (TargetTrainData1 and TargetTrainData2), and one set of target domain test data (TargetTestData). The amount of data used for training in the source domain training data is D. strain1 and D strain2 The target domain training data contains D amounts of data used for training. ttrain1 and D ttrain2 The target domain test data contains D data points used for testing. ttest Each sleep stage data point and its label are annotated by sleep physicians according to the international AASM standard, in 30-second segments. The raw sleep data contains M signal channels, obtained by continuous acquisition at a sampling frequency of fs for a duration of T. The size of the raw data is w×h, where h is the length of the data time series, h = fs×T. The acquisition values ​​of fs in different laboratories are 100Hz, 1000Hz, 128Hz, and 256Hz. T is the continuous sleep of a normal adult in one night, which varies from person to person and is approximately 8 hours.

[0030] Data X is composed of one EEG channel signal and one EOG channel signal selected from the original data. dataFor the selected two channels, for signal samples with a sampling frequency greater than 100Hz, a sample point is extracted every fs / 100 points to reduce the sampling frequency to 100Hz. The original data size then becomes 2×h1, where h1 = 100×T. Next, the second dimension h1 of the original data is segmented into non-overlapping data segments with a time window length of 30 seconds, resulting in N segments of sample data. The dimension of the resulting data matrix is ​​then N×2×(30*100). The segmentation process can be represented as follows: For SourceTrainData1 and SourceTrainData2, after preprocessing, we can obtain... and For TargetTrainData1 and TargetTrainData2, after preprocessing, we can obtain... and For TargetTestData, after preprocessing, we can obtain... x i The data is a pre-divided sub-segment of sample data, with a data dimension of 2×3000.

[0031] In the implementation of this invention, the experimental data used involves three datasets. The first dataset is the publicly available Sleep-EDF dataset, in which multimodal signals were used as source domain samples: single-channel EEG signals (electrode positions at Pz-Oz) and electrooculogram (EOG) signals (horizontal), with a sampling frequency of 100Hz. The second dataset is the publicly available CAP-Database dataset, which also uses multimodal signals as target domain samples: single-channel EEG signals (electrode positions at C4-A1) and EOG signals, with different sampling frequencies for different individuals, including 512Hz, 256Hz, and 128Hz. The third dataset involved in this invention is a real dataset collected in a hospital, in which multimodal signals were used as target domain samples: single-channel sleep EEG data (electrode positions at F4-A1) and EOG channel data, with a sampling frequency of 1000Hz.

[0032] Step 2: Extraction of time-frequency features from sample data.

[0033] For any data segment x in step 1 k It contains 2 channels. For each of its channel data The Morlet continuous wavelet algorithm, which uses multiple frequencies as center frequencies ω0, is used to extract the corresponding time-frequency features. Where p∈[1,2] represents the channel of the current sample data. The selection rule for multiple center frequencies in this invention is: frequencies of 0.5 every 1Hz-30Hz are selected, i.e., ω0=[0.5,1,…29.5,30].

[0034] Performing a continuous wavelet transform on single-channel signal data with a specified wavelet basis function and center frequency requires the following processing:

[0035] The continuous wavelet algorithm is used to process single-channel signals. The signal is decomposed into time signals of multiple frequencies. This decomposition is performed based on the predetermined wavelet basis function ψ(t) and the wavelet transform scale s. This process can be expressed as:

[0036]

[0037] in, Let ψ(t) represent the single-channel signal to be decomposed, ψ(t) represent the wavelet basis function used in the wavelet transform, s refers to the scale of the wavelet transform (the inverse function of frequency), and t is the time axis of the signal.

[0038] For the wavelet basis function ψ(t), this invention uses the Morlet wavelet basis, whose functional expression is as follows:

[0039]

[0040] Where ψ(t) is the wavelet basis function to be defined, e is a natural constant, j represents the imaginary part, ω0 represents the center frequency of the wavelet basis function used, and t is the time axis.

[0041] After performing continuous Morlet wavelet transforms at multiple frequencies, the dimension of a single channel data segment changes from 3000 to a time-frequency matrix dimension of 30×3000. Then, by taking a sample every 50 values ​​in the last dimension, the time-frequency matrix dimension of a data segment becomes 60×60. The calculated results constitute a time-frequency feature. The dimensions are 2×60×60. The time-frequency features of the N data segments divided in step 1 are... The dimensions are N×2×60×60. Time-frequency features are extracted from the training and testing data sets in step 1 to obtain the time-frequency matrix.

[0042] Step 3, training the dual-teacher feature extractor.

[0043] Training of the dual-teacher feature extractor, such as Figure 2 As shown in Stage I, this invention designs two teacher models. Both models have the same structure, using ResNet110, but the training data differs. The source domain CWT time-frequency data after the above processing... and target domain CWT time-frequency data The data is randomly shuffled, and then a minibatch of 128 data points is taken as input to the source domain teacher and target domain teacher models. The models are trained sequentially by a feature extractor and a classifier, combined with the true labels of the input data. The feature extractor consists of a 3×3 2D convolutional layer, a batch normalization layer, a ReLU activation layer, and three residual modules. Each residual module consists of NUM residual blocks (NUM is 18 in this invention). Each residual block consists of a 3×3 2D convolutional layer, a batch normalization layer, a ReLU activation layer, a 3×3 2D convolutional layer, and a batch normalization layer. The classifier consists of average pooling and a linear classification layer with 5 output neurons. Here, the two teacher models are trained from time-frequency data in the source and target domains, respectively. After 200 iterations as specified in this invention, the loss of the teacher model no longer decreases and tends to stabilize. At this point, the teacher has fully learned the data features of the training domain. Then, the teacher model structure is frozen, meaning that the teacher model is no longer trained in subsequent steps, and the parameters in the network are no longer changed.

[0044] Step 4: Knowledge distillation and domain-adaptive feature transfer.

[0045] Then, knowledge distillation and domain adaptation methods are used to transfer the features of the two teachers to the student model. The framework for training the student model is as follows: Figure 2 Stage II is divided into target domain-specific feature distillation and domain-general feature distillation. During student model training, the teacher model, which was frozen after training in Step 3, is introduced. The feature extractors Ft and F′t of the two teacher models are used to guide the student model in learning domain-general and target domain-specific features. During training, the student network uses wavelet time-frequency feature data extracted in Step 2. and The feature extractor and classifier of the student network are fed into the model respectively, and two teacher models are also fed into the model. The two teacher models observe and constrain the output of the student model. The feature extractor of the student model is designed the same as that of the teacher model, except that each residual module consists of two residual blocks (ResBlock). That is, in this invention, the NUM of the student model is 2. As input, the source domain teacher and student models form a feature distillation method for training a compressed model. The source domain teacher will use a similarity loss L... similarity The constrained compression student model learns feature representations of the source domain from the source domain teacher, while the target domain teacher uses a differential loss Li. difference The student model is constrained to perform feature distillation specific to the target domain, thereby learning the feature representation of the target domain. As input, the source domain teacher will use similarity loss L. similarity The student model is constrained to learn feature representations from the target domain features that are similar to those from the source domain features. Similarity loss L similarity and difference loss L difference The feature distillation process for extracting domain-general features and target domain-specific features by the student model will be defined in the following steps:

[0046] (1) Distillation of specific features of the target domain

[0047] The student model learns specific features of the target domain under the guidance of the target domain teacher. When the target domain teacher is input, the target domain teacher uses its trained convolutional layer parameters to extract key features of the target domain from the input, and then uses the differential loss Li. difference This ensures that the source domain features extracted by the student differ from the key features of the target domain, thereby enabling the student model to extract domain-specific features of the target domain. Details of target domain-specific feature distillation using a teacher-constrained student feature extractor are as follows: Figure 3 As shown. The design of the loss function includes feature transformation and feature distance measurement. The feature transformation and feature distance measurement processes for teachers and students in the target domain are as follows:

[0048] When using a target domain teacher to guide student model training, considering the high computational cost of calculating the higher-order features of the time-frequency features extracted from each residual block of the source domain teacher model and the higher-order features of the time-frequency features extracted from each residual block of the student model, this invention uses a feature encoding module (TE) to extract and reduce the dimensionality of wavelet time-frequency features. The feature encoding module (TE) consists of Input Embedding, Multi-Head Attention, Add & Norm, FeedForward, Add & Norm, and Reduce & Norm. Input Embedding consists of a convolutional layer with a kernel size of 1152×1152 and a batch processing layer, and also includes a Flatten layer to stretch the data into a vector. Multi-Head Attention consists of a linear layer, a Dropout layer, and a softmax function, used to focus on the key parts of the features. In the intermediate layer of the feature extractor for the target domain teacher and student models, a size of N will be obtained. batc h ×ω sout The higher-order features of ×D×E, where N batc h The input is a minibatch size, which is 128 in this invention, ω sout For each residual module in the teacher model, ω represents the output channel of the convolutional layer. soutLet F be 16, 32, and 64 respectively, and D and E be the feature dimensions extracted by the residual modules. The D and E values ​​for the three residual modules are (60, 60), (30, 30), and (15, 15) respectively. sq and F′ tq Let F represent the features extracted by the q-th residual module of the three residual modules in the student and target domain teacher networks, respectively. sq and F′ tq After being input into the feature encoding module TE, the data first undergoes sequence encoding via Input Embedding to prevent the model from losing earlier information as the sequence grows. This positional information is then incorporated into the Multi-Head Attention structure to form relative positional encoding. Multi-Head Attention divides the model into four heads, forming four subspaces that allow the model to focus on feature information at different positions. The values ​​input into the residual block are then summed before and after computation. This is followed by Feed Forward propagation, another residual block, and finally Reduce & Norm calculation of the average value along the second-to-last dimension to reduce data dimensionality. After this encoding module, the feature representation is TE′. sq and TE′ tq .

[0049] Then, the features of the student model and the target domain teacher after the encoding module are used to apply the differential loss L. difference This constrains students to learn specific features of the target domain. The feature distance metric is positioned before the last ReLU activation layer in each residual module because the ReLU layer causes neurons with inputs less than 0 to output 0. This invention aims to ensure that the features are not distorted during distillation, hence the placement before the activation layer. The difference loss uses an orthogonal function to amplify the feature difference, specifically expressed as follows:

[0050]

[0051] in‖·‖ F It is the F-norm, that is, for a matrix A, ||A|| F It is the square root of the sum of squares of each element of A. Similar to the similarity loss, n represents the number of residual modules; in this invention, n is taken as 3. TE′ sq and TTE′ tq The size extracted for the q-th residual module is N. batc h ×ω q out The high-order wavelet time-frequency features of ×D×E are represented by the feature transformation after being processed by the feature encoding module TE, where q=[1,2,3], and ω q outIn the three residual modules, the values ​​are 16, 32, and 64 respectively, and D and E are (60, 60), (30, 30), and (15, 15) respectively.

[0052] (2) Domain-wide characteristic distillation

[0053] The student model, after being trained by (1), then extracts the D from step 2. ttrain2 Wavelet time-frequency features As input, continue to extract domain-general features from the source and target domains. When the input is fed to the source domain teacher, the trained source domain teacher uses its convolutional layer parameters to extract key source domain features from the input, and then uses L... similarity The student model is constrained to extract the same target domain features as the teacher, thus enabling it to extract domain-general features common to both the source and target domains. Details of domain-general feature distillation using the source domain teacher-constrained student feature extractor are as follows... Figure 4 As shown. The design of the loss function includes feature transformation and feature distance measurement. The feature transformation and feature distance measurement processes for the source domain teacher and student models are as follows:

[0054] When the source domain teacher guides the student's model training, The high-order feature F output by a residual module of the feature extractor of the student model and the source domain teacher model. sc and F tc The size is N batc h ×ω gout ×D×E,c=[1,2,3],F sc and F tc It is high-dimensional data with four dimensions, where N batc h The input is a minibatch size, which is 128 in this invention, ω g out For each residual module in the student model, the output channel of the convolutional layer is ω, and for the three residual modules... g out The feature dimensions D and E extracted by the three residual modules (16, 32, and 64) are (60, 60), (30, 30), and (15, 15), respectively. In this invention, the features extracted by each residual module of the source domain teacher model are used to constrain the residual modules of the student model. Before using distance metric to constrain the features, an output channel ω is used. gout The feature transformation module TS, composed of 1×1 convolutions, enables the student model to extract higher-order features F. sc To the higher-order features F of the teacher model tc Perform feature alignment.

[0055] Then, the student features after feature transformation and the intermediate features of the teachers are compared using the similarity loss L. similarityStudents are constrained to extract domain-general features from both the source and target domains. The feature distance metric is still placed before the last ReLU activation layer in each residual module. Margin-ReLU is used to calculate the feature distance; Margin-ReLU removes negative features before ReLU. For negative features of the teacher, if the student's value is less than the teacher's negative value, the feature distance is calculated as 0, because ReLU blocks negative values ​​regardless of their magnitude. The loss function is expressed as follows:

[0056]

[0057] Where F tm and F sm These refer to the wavelet time-frequency features extracted by the m-th residual module of the source domain teacher and student feature extractors, respectively, where m = [1, 2, 3] and the dimension is N. batc h ×ω gout ×D×E, ω of the three residual modules gout The feature dimensions D and E extracted by the three residual modules are taken as 16, 32, and 64 respectively, and are (60, 60), (30, 30), and (15, 15). TS(·) refers to the transformation of the features, i.e., the 1×1 convolution transformation mentioned above, and n represents the number of residual modules. In this invention, n is taken as 3. a (F tm TS(F) sm )) is used to calculate F tm and TS(F) sm The partial L2 distance of ) is specifically expressed as:

[0058]

[0059] Where N batc h wHC corresponds to N batc h ×ω gout ×D×E, i.e., ω gout The data channels represent the features, D and E represent the features extracted by the three residual modules, and ω represents the features extracted by the three residual modules when m takes the values ​​1, 2, and 3. gout Let H be 16, 32, and 64 respectively, and let D and E be (60, 60), (30, 30), and (15, 15) respectively. H represents the feature length D, and C represents the feature width E. and It is feature F tm and TS(F) sm The r-th eigenvalue of ), r = [1, 2, ..., N] batc h ×ω gout ×D×E].

[0060] Step 5: The student model classifies the samples in the target domain.

[0061] The student model is the final model that classifies the target domain. It uses the time-frequency features obtained from step 2 on the target domain samples. Input the trained student network, The features extracted by the feature extractor of the student network are then fed into the classifier to obtain the classification results. The classifier is connected after the feature extractor. According to the international standard AASM sleep staging criteria, this invention sets up a linear classification layer with 5 neurons to classify the extracted features into sleep stages, namely wakefulness (W), non-rapid eye movement (NREM, which includes N1, N2 and N3 stages), and rapid eye movement (REM).

[0062] Training to optimize student networks

[0063] During the experiment, the two teachers were trained and saved using sleep stage data from subjects in the source and target domains, respectively. The student network received the extracted wavelet time-frequency signals. The network's feature extraction layer consisted of multiple residual modules. Multiple convolutional layers were used to extract features from the input wavelet time-frequency features. Finally, the student network features, optimized as described in step 4, were input into the classifier for classification. During network optimization, the distance loss function described in step 4 was used to constrain the feature extraction of the intermediate feature layers of the student network. A multi-class cross-entropy loss function was used as the classification task loss to constrain the effectiveness of the features extracted by the student network for classification. These three loss functions were used together to correct and adjust the weights and biases in the network. Task loss L task The definition is as follows:

[0064]

[0065] For labels u and y su This indicates whether the prediction result is the same as the label; 1 indicates the same result, and 0 indicates different result. su This refers to the network's predicted output.

[0066] During the experiment, a loss function will be used to constrain the wavelet time-frequency high-order features extracted by each residual module of the student network. The loss function is used to correct and adjust the weights and biases in the student network. The overall loss function is L. task L similarity and L difference The sum, i.e., the total loss, is defined as follows:

[0067] loss = L task +L similarity +L difference

[0068] The network is trained using minibatch data for optimization. All sample data in the training or test set is divided into K minibatches, with each minibatch containing 1 / K of the total sample data. During training, the network learns from and updates the gradient parameters using the sample data in each minibatch. One epoch is completed when the network has trained on all minibatch data. In this invention, the minibatch size is set to 128, and the model training uses the EEG and EOG channels of the data, with 200 epochs selected. Considering that the model requires a high learning rate at the beginning to reduce large errors, while simultaneously needing a low learning rate to avoid getting trapped in local minima during training, an SGD optimizer with an initial decay learning rate of 0.1 and a decay rate of 0.1 is chosen. The learning rate decay occurs at 100 epochs and 150 epochs. After optimizing the parameters, the network parameters tend to stabilize after 200 epochs of iterative updates.

[0069] The network parameter settings are shown in Table 1. In the experiment, the number of ResBlocks NUM for the two teachers was set to 18, and the number of ResBlocks NUM for the students was set to 2.

[0070] Table 1 Network framework parameter settings of the method of the present invention

[0071]

[0072]

[0073] The feature transformation and distance metric design adopted by the features extracted from the intermediate layer of the student network and the teacher network are shown in Table 2. In this invention, TE is the encoding module, which is defined in step 4(1) of this invention. The distance function is defined in step 4 of this invention.

[0074] Table 2 shows the design of the feature transformation and distance function of the intermediate layer in this invention when teachers constrain students' guidance.

[0075]

[0076] The design of the feature transformation module and parameters of the intermediate layer of the student network and teacher network when calculating distillation loss is shown in Table 3.

[0077] Table 3 Design of the TE feature transformation module in this invention

[0078]

[0079]

[0080] Network performance evaluation

[0081] To evaluate the effectiveness of the proposed method, a confusion matrix was calculated in the following results to show the difference between the results given by the proposed method and the expert scores.

[0082]

[0083] any value S hl The labeled sleep segment h is divided into sleep stages l. The values ​​of h or l are 1, 2, 3, 4, and 5, representing the wakefulness period (W), REM stage I (N1), REM stage II (N2), REM stage III (N3), and non-REM stage, respectively.

[0084] Classification performance is reflected in classification accuracy (ACC), and Coens' kappa coefficient is used to measure the consistency of classification estimates across the dataset. ACC and kappa are calculated using the true positive (TP), false positive (FP), true negative (TN), and false negative (FN) values ​​for each class, which can be obtained from the confusion matrix.

[0085] Classification accuracy refers to the proportion of all correctly predicted samples out of all samples.

[0086]

[0087] The Kappa coefficient is a form of correlation coefficient used to measure the consistency of classification estimates for a dataset.

[0088]

[0089] Among them, P e The calculation is also based on the confusion matrix, and it is calculated as follows:

[0090]

[0091] Experimental Section

[0092] The experimental part of this invention is to fully verify the effectiveness of the dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation.

[0093] During training, two public datasets (Sleep-EDF, CAP-Database) were used. The performance of the method was tested on ins subjects in the target domain CAP-Database and on a private dataset. Comparisons were also made with sleep staging methods and transfer learning methods currently being implemented on the same subjects in the same target domain.

[0094] Baseline: The model uses ResNet110, and the model trained in the source domain is directly used to test sleep staging in the target domain.

[0095] DeepSleepNet: An advanced sleep staging model that uses convolutional kernels of different sizes for feature extraction and Bi-LSTM to learn the temporal information of sleep stages.

[0096] MetaSleepLearner: Based on Model-Agnostic Meta-Learning (MAML), it transfers the knowledge of sleep stages acquired from a large dataset to new individuals, enabling the identification of sleep stages by using only a few stages labeled by clinicians instead of the entire record.

[0097] OFD: Using the feature distillation method, the features extracted by the student network and the teacher network are transformed in the intermediate feature extraction layer of the network. Then, distance constraints are used to make the features of the student network as similar as possible to the features of the teacher network.

[0098] DANN: Unsupervised Domain Adaptive Algorithm, which uses a large amount of labeled data from the source domain and a large amount of unlabeled data from the target domain for training to achieve accurate identification of target domain data.

[0099] Comparative analysis

[0100] Table 4. Accuracy comparison of the method of this invention and mainstream sleep staging transfer learning methods on the CAP-Ins test set.

[0101]

[0102] Table 4's "transfer" column describes the methods used in transfer learning. DT represents direct transfer, DA represents domain adaptation, KD represents knowledge distillation, and Pre-train & Fine-tune refers to the pre-training and fine-tuning methods. The comparison algorithms Baseline and DeepSleepNet use the direct transfer method, without constraints on source and target domain features, and are evaluated using ACC and Kappa. MetaSleepLearner uses a pre-training-fine-tuning method, treating each subject's data phase as a task, training the model on each task, and then fine-tuning the trained model using several data segments from the test set. During training, K segments from different phases are selected, with K set to 5, 10, and 15, and a learning rate of 10%. -1 10 -2 10 -3 Compare the experimental results. DANN is an unsupervised domain adaptive method based on the idea of ​​adversarial learning. It designs a domain discriminator and a classifier, and uses backpropagation to update the network parameters θ.

[0103] The invention was compared with other methods on a public dataset. Training of the invention on the selected public dataset included training of teacher and student models: 1) Teacher training: The model training set consisted of two nights' data from the first 40 individuals in the Sleep-EDF dataset, totaling 80 nights' data. An additional 10 nights were selected as the validation set. The target domain CAP-Database was used, and ins1, ins3, and ins6 (where EEG electrodes were removed) were selected. Three individuals (approximately 3000 segments) were selected for training the target domain teacher group, and one ins subject was selected as the validation set. 2) Student training: The model used the first 20 nights' data from the Sleep-EDF dataset as the training set, and the validation set consisted of data from one target domain subject. The student network was the final model. During testing, data was input into the student network to observe the classification performance. Three additional ins subjects from the CAP-Database public dataset, whose data did not overlap with the training and validation sets, were selected for testing sequentially. The evaluation metrics used in Table 4 are as described in step 5. This invention records a schematic diagram of a subject's sleep score over a consecutive night using this invention, as shown below. Figure 5 As shown in the figure, this invention exhibits good generalization ability to new target domains on small datasets, and the identification results for sleep periods remain stable. Results show that the proposed model has high classification performance for sleep period identification of ins subjects in the target domain CAP-Database, achieving an average classification accuracy of 80.56% on the test set, nearly 10% higher than state-of-the-art methods. Furthermore, direct testing on ins subjects collected from hospitals also yields satisfactory experimental results, further demonstrating the effectiveness and robustness of the model.

Claims

1. A dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation, characterized in that, Includes the following steps: Step 1: Acquire sleep physiological signals from the source and target domains and perform data preprocessing; Publicly available sleep data is used as source domain samples, and the data from the dataset to be tested is used as target domain samples. It is assumed that for this method framework, there are two sets of source domain training data (SourceTrainData1 and SourceTrainData2), two sets of target domain training data (TargetTrainData1 and TargetTrainData2), and one set of target domain test data (TargetTestData). Based on the polysomnography, for any dataset containing... The original signal samples from each signal channel were selected from one sleep EEG channel signal and one electrooculogram (EOG) channel signal, and then the selected sampling rate was... Sleep data downsampled to Next, a sliding window of 30 seconds is used to divide the data into N non-overlapping sample data segments, each containing... There are 1 channel and the data length of each channel is 1. The sliced ​​data matrix , The size is ; Step 2: Extraction of time-frequency features from sample data; For any data segment in step 1 Include One channel, For each of its channel data Use multiple frequencies, i.e., 1Hz-30Hz, spaced 0.5Hz apart, as the center frequency. The Morlet continuous wavelet transform represents the local features of a signal, thereby extracting the corresponding time-frequency features. ,in This represents the channels of the current sample data; simultaneously, it represents the extracted time-frequency features. By keeping one out of every 50 data points to reduce data dimensionality, the time-frequency features extracted from a data segment can be... , dimension ; The time-frequency characteristics of the N data segments divided from step 1. , dimension Extract time-frequency features from the training and test data sets in step 1 to obtain the time-frequency matrix. , , , , ; Step 3, training the dual-teacher feature extractor; The time-frequency matrix extracted from the training set in step 2 is used as input to the teacher network for training; two teacher models are designed: a source domain teacher and a target domain teacher; the time-frequency matrix of the source domain training data... The source domain teacher is fed in, and the real labels of the training data are used to train the feature extractor for the source domain teacher. The time-frequency matrix of the target domain training data The target domain teacher is fed in, and the real labels of the training data are used to train the feature extractor for the target domain teacher. ; Step 4: Knowledge distillation and domain-adaptive feature transfer; During the training of the student model, the feature extractor from the teacher model trained in step 3 is imported. and Then, its parameters are frozen and not updated during the training of the student model; the teacher model uses its prior knowledge from training to extract features from the input and then uses the loss to optimize the student model; the training of the student model is divided into two parts, firstly, the time-frequency matrix of the source domain data extracted in step 2 is used... As the training input for the student model, the time-frequency matrix of the target domain data is then used. The input data continues to be used as the input for training the student model. The input data is simultaneously fed into the student model and the two teacher models. Then, the feature knowledge extracted by the intermediate layer of the teacher network feature extractor is transferred to the student network. The feature transfer process uses distillation loss to constrain the student network to learn the domain-general features of the source and target domains as well as the domain-specific features of the target domain from the two teacher models. Step 5: The student model classifies the samples in the target domain. The time-frequency features obtained from the target domain samples in step 2. Input the student model obtained after training in step 4. The features extracted by the feature extractor of the student network are then fed into a linear classifier with 5 neurons to obtain the classification result.

2. The dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation according to claim 1, characterized in that: Step 1, which involves acquiring and preprocessing the raw EEG and EEG signals from the source and target domains, includes the following steps: Regarding the data sample partitioning, assume that for this method framework there are two sets of source domain training data SourceTrainData1 and SourceTrainData2, two sets of target domain training data TargetTrainData1 and TargetTrainData2, and one set of target domain test data TargetTestData; based on the polysomnography, the data includes... The raw data from each signal channel was selected from one sleep EEG channel signal and one electrooculogram (EOG) channel signal to form the data. The data for each channel uses a sampling rate. Continuous acquisition time The data dimensions were subsequently obtained as follows: , The length of the data time series. , It varies depending on the individual's sleep pattern, but is approximately 8 hours. Values ​​collected in different laboratories were 100Hz, 1000Hz, 128Hz, and 256Hz; then, signal samples with frequencies greater than 100Hz were collected at intervals... Downsampling is performed by extracting one point from each point. Hz; Next, a non-overlapping sliding window of length 30 seconds is used to divide the data, resulting in N segments of sample data. The piecewise process is represented as For SourceTrainData1 and SourceTrainData2, the processed data is obtained as follows: and For TargetTrainData1 and TargetTrainData2, the processed data is obtained as follows: and For TargetTestData, after processing, we obtain , The sample data is divided into segments, and its size is... That is, each segment of sample data contains two channels, and the data size of each channel is 3000.

3. The dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation according to claim 1, characterized in that: The training method for the dual-teacher feature extractor described in step 3 includes the following steps: Two teacher models were designed to guide the learning of student models. The two teacher models have the same structure, both using ResNet110, but different training data. Both teacher models consist of a feature extractor and a linear classifier; the feature extractor consists of a convolutional kernel with a size of [missing information]. It consists of a 2D convolutional layer, a batch normalization layer, a ReLU activation layer, and three residual modules; each residual module consists of 18 residual blocks (ResBlocks), and each residual block has a convolutional kernel of size [missing information]. A 2D convolution, a batch normalization layer, a ReLU activation layer, and a convolution kernel size of [missing information]. It consists of a 2D convolution and a batch normalization layer; the classifier consists of average pooling and a linear classification layer with 5 output neurons. 1) Training of source domain teachers Source domain CWT time-frequency data after step 2 processing Then, a minibatch of 128 data points is taken as the input for the source domain teachers. The model is trained by combining the real labels of the input data with the teacher's feature extractor and classifier. The training process is set to 200 iterations. 2) Training of teachers in the target domain The target domain CWT time-frequency data after step 2 processing The data is randomly shuffled, and then a minibatch of 128 data points is taken as the input to the target domain teacher model. The model is trained by combining the real labels of the input data with the teacher's feature extractor and classifier in turn; 200 iterations are set during training.

4. The dual-teacher sleep staging feature transfer method based on knowledge distillation and domain adaptation according to claim 1, characterized in that: Step 4, knowledge distillation and domain-adaptive feature transfer, includes the following steps: Import the two teacher models trained in step 3, and then freeze their parameters so they are not updated during the training of the student model. The feature extractor in the student model is identical to that of the teacher models, except that each residual module consists of two ResBlocks. The wavelet time-frequency feature data obtained in step 2... and The feature extractor and classifier of the student network are fed into the student network as inputs, and are also fed into two teacher models. The two teacher models observe and constrain the output of the student model. As input, the source domain teacher and student models form a feature distillation method for training a compressed model. The source domain teacher will use a similarity loss. The constrained compression student model learns feature representations of the source domain from the source domain teacher, while the target domain teacher uses differential loss. The student model is constrained to perform target domain-specific feature distillation to learn the feature representation of the target domain from the source domain data; When used as input, the source domain teacher will use similarity loss. The constrained compressed student model performs domain-general feature distillation to learn feature representations similar to source domain features from target domain features. Additionally, the target domain teacher still uses differential loss. The constraints cause the student model to extract domain-general features that differ from the target domain features; similarity loss. and difference loss The feature distillation process for extracting domain-general features and target domain-specific features by the student model will be defined in the following steps: 1) Target domain specific feature distillation Target domain-specific feature distillation is performed by the target domain teacher-guided student model; When the target domain teacher is input, the target domain teacher will use its trained convolutional layer parameters to extract key features of the target domain from the input, and then use differential loss. The goal is to ensure that the source domain features extracted by the student differ from the key features of the target domain, thereby enabling the student model to extract domain-specific features of the target domain. The design of the loss function includes feature transformation and feature distance measurement. The feature transformation and feature distance measurement processes for the teacher and student in the target domain are as follows: When using a target domain teacher to guide student model training, the feature encoding module is used. Then, feature extraction and dimensionality reduction are performed on the wavelet time-frequency features, followed by distance measurement; the feature encoding module... It consists of Input Embedding, Multi-Head Attention, Add & Norm, Feed Forward, Add & Norm, and Reduce & Norm; Input Embedding consists of a single convolutional kernel. It consists of a convolutional layer and a batch processing layer, and also includes a Flatten layer to pull the data into a vector; Multi-Head Attention consists of a linear layer, a Dropout layer, and a softmax function; The feature extractor for the target domain teacher and student models will yield a size of... The higher-order features, among which Given a minibatch of size 128 as input, For each residual module in the teacher model, the output channels of the convolutional layers are: [list of three residual modules]. Let 16, 32, and 64 be the values ​​of D and E, respectively, representing the feature dimensions extracted by the residual modules. The D and E values ​​for the three residual modules are (60, 60), (30, 30), and (15, 15), respectively. Let... and These represent the first and second residual modules in the student and target domain teacher networks, respectively. Features extracted by each residual module , and Input into feature encoding module First, sequence encoding is performed using Input Embedding to prevent the model from losing earlier information as the sequence grows. Then, positional information is incorporated into the Multi-Head Attention structure to form relative positional encoding. Multi-Head Attention divides the model into four heads, forming four subspaces that allow the model to focus on feature information at different positions. The values ​​input into the residual block are then propagated forward via Feed Forward, and then the residual block and Reduce & Norm calculate the average value on the penultimate dimension of the data to reduce the data dimensionality. After this encoding module, the features are represented as follows: and ; Subsequently, the features of the student model and the target domain teacher after passing through the encoding module are used with differential loss. The learning is constrained to specific features of the target domain; the feature distance metric is positioned before the last ReLU activation layer in each residual module; the dissimilarity loss uses an orthogonal function to amplify the feature gap, as specifically expressed below: ； in It is the F-norm, that is, for a matrix , yes The square root of the sum of squares of each element; similar to similarity loss, Represents the number of residual modules. Take 3; and For the first The size extracted by each residual module is The higher-order wavelet time-frequency features are processed by the feature encoding module. The feature representation after feature transformation, where In the three residual modules, the values ​​are 16, 32, and 64 respectively, and D and E are (60, 60), (30, 30), and (15, 15) respectively in the three residual modules. 2) Domain-wide characteristic distillation The feature distillation of the common features of the source and target domains extracted by the students is guided by the source domain teacher model; the student model trained by (1) then extracts the wavelet time-frequency features obtained in step 2. As input, the student model is further trained to extract domain-general features of the source and target domains; When the input is fed to the source domain teacher, the source domain teacher will use its trained convolutional layer parameters to extract key features from the source domain, and then use... The student model is constrained to extract the same target domain features as the target domain model, thus enabling it to extract domain-general features from both the source and target domains. The loss function design includes feature transformation and feature distance measurement. The feature transformation and feature distance measurement processes for the source domain teacher and student models are as follows: When the source domain teacher guides the student's model training, The high-order features output by a residual module of the feature extractor from the student model and the source domain teacher model. and It is a high-dimensional data feature with four dimensions. Its size is ,in This indicates a minibatch with an input size of 128. For each residual module in the student model, D and E are the output channels of the convolutional layer, and D and E are the feature dimensions extracted by the residual module. The three residual modules... The values ​​are 16, 32, and 64 respectively, and D and E are (60, 60), (30, 30), and (15, 15) respectively. The features extracted from each residual module of the source domain teacher model are used to constrain the residual modules of the student model. Before using distance metric to constrain the features, an output channel is used... Feature transformation module composed of 1×1 convolution kernels Higher-order features extracted by the student model Feature alignment; Subsequently, the student features after feature transformation and the intermediate features of the teachers were compared using similarity loss. Students are constrained to extract domain-general features from both the source and target domains; the feature distance metric is still placed before the last ReLU activation layer in each residual module; Margin-ReLU is used to calculate the feature distance, which removes negative features before ReLU. For negative features of the teacher, if the student's value is less than the teacher's negative value, the feature distance is calculated as 0, because ReLU will block negative numbers regardless of their magnitude; the loss function is expressed as follows: ； in and These refer to the first and second feature extractors of the source domain teachers and students, respectively. The high-order data features of wavelet time-frequency features extracted by each residual module , dimension The three residual modules Take values ​​of 16, 32, and 64 respectively; This refers to transforming the features, specifically a 1x1 convolution transformation. Represents the number of residual modules. Take 3; Used for calculation and Part Distance, specifically expressed as: ； in correspond ,Right now The data channels represent the features, and D and E are the feature dimensions extracted by the residual module. When taking 1, 2, 3 Take values ​​of 16, 32, and 64 respectively, and take values ​​of (60, 60), (30, 30), and (15, 15) respectively for D and E. The length D of the feature is represented. The width E of the feature is represented. It is a feature and The 1 eigenvalue, .

Images