Sleep stage classification method and system based on multi-granularity feature fusion
Through the multi-grained feature fusion method, combined with 3D-CNN and three-branch network, the higher-order features of EEG signals, EEG signals and EEM signals are extracted, which solves the problem of low classification accuracy in sleep stages in the existing technology, and achieves the improvement Accurate sleep phase recognition.
Patent Information
- Application Number
- CN202410902644.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-08
AI Technical Summary
The prior art lacks screening and fusion of multi-signal and multi-dimensional features in sleep stage classification, resulting in low classification accuracy and difficult to accurately collect and denoise weak signals such as EEG signals.
Using a multi-particle feature fusion method, the EEG signal, ophthalmic signal and EMG signal are obtained through sensors, data preprocessing and time-frequency feature extraction are performed, and higher-order features are extracted using 3D-CNN and three-branch networks, combined with attention Perception mechanism and mutual information filtering method realize the fusion of global and local features of the signal.
It improves the accuracy and performance of classification in the sleep stage, avoids the problem of low accuracy in the REM and NREM stages, retains multi-dimensional information, reduces noise interference, obtains more relevant feature information, and improves the classification effect.
Smart Images

Figure CN118436317B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a sleep stage classification and recognition method, and in particular to a sleep stage classification method and system based on multi-granularity feature fusion, belonging to the technical field of signal processing and medical care information recognition. Background Art
[0002] Sleep is a necessary physiological activity for humans. Sufficient sleep can help people recover from fatigue and keep them in good condition. At present, with the rapid development of the information age, the pressure of study, life and work has increased, and people's sleep problems have become increasingly prominent and gradually become a social focus. Sleep stage is a very important step in the process of scoring sleep quality or diagnosing sleep-related diseases. Professional physicians need to observe the patient's sleep data all night and classify and evaluate the sleep stage, which is undoubtedly a time-consuming and laborious diagnostic work. With the continuous development of artificial intelligence, research on using computer-assisted diagnosis and treatment technology to improve doctors' work efficiency is also deepening. For a heavy workload task such as sleep staging, the development and application of automated sleep EEG signal analysis methods are very necessary to reduce the workload of doctors. At present, the research on sleep stage classification mainly focuses on the relevant signal processing and classification of single EEG signals or non-EEG signals, which will lead to incomplete feature signal extraction and errors.
[0003] EEG signals are electrical signals generated by the activity of brain neurons. Neurons are connected to each other through synapses to form a complex neural network. When neurons are activated, bioelectric phenomena are generated. These electrical signals can be captured by placing electrodes on the scalp or implanted directly into the brain. EEG signals are relatively weak and easily interfered by other external signals. Therefore, the collection and noise filtering of EEG signals are relatively difficult. Similarly, electromyographic signals and electrooculographic signals also face the problem of difficulty in collection and denoising. In addition, most sleep biological signals use a single dimensional feature without considering the correlation between multiple dimensions. In order to accurately classify sleep stages, it is necessary to screen and fuse multi-signal and multi-dimensional feature signals. Summary of the invention
[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a sleep stage classification method based on multi-granularity feature fusion, which can effectively improve the classification accuracy and classification performance of the sleep stages.
[0005] The technical solution adopted by the present invention is:
[0006] The sleep stage classification method based on multi-granularity feature fusion includes the following steps:
[0007] S1. Obtain physiological signals of the user during sleep through sensors, including EEG, EOG and EMG;
[0008] S2. Preprocess the EEG, EOG, and EMG data collected during the user's sleep;
[0009] S3. After the preprocessed physiological signals are subjected to empirical mode decomposition, the time-frequency features are extracted using Hilbert transform to obtain a time-frequency matrix with representation information, which is then input into the feature learning network ETN built based on 3D-CNN to extract the sleep high-order features of EEG signals, EOG signals, and EMG signals;
[0010] S4. Input the high-order features into the parallel three-branch network global channel feature extraction network, local feature extraction network, and global spatial feature extraction network to extract local and global features of EOG, EMG, and EEG signals to obtain more comprehensive feature information;
[0011] S5. Use the filtering method to filter the features of each channel, and use the attention perception mechanism network to fuse the features output from the three-branch network;
[0012] S6. The fused features are input into a classification network consisting of a fully connected and softmax function to achieve the classification of sleep stages.
[0013] In the above method, the preprocessing described in step S2 is to perform signal segmentation, remove noise and artifacts, and remove baseline drift preprocessing on the EEG signal, the EOG signal, and the EMG signal.
[0014] The time-frequency feature extraction described in step S3 is to divide the EEG, EOG, and EMG signals into multiple sub-segments, and use the time-frequency analysis method Hilbert-Huang transform HHT to transform these data sub-segments. m ( t ) are converted into time-frequency matrix data respectively. The Hilbert-Huang transform (HHT) step includes empirical mode decomposition (EMD) and Hilbert transform (HT). The specific steps are as follows:
[0015] (1) Find the signal m ( t ), the upper and lower envelopes are obtained by cubic spline fitting, and the average value is calculated. ;
[0016] (2) Get the first component , check whether it satisfies the modal component ( IMF )conditions: a. h 1( t ) has a maximum value point and a zero-crossing point number that differ by no more than 1; b. h 1(t ) is always 0; if it is not satisfied, repeat steps (1) and (2) until it is satisfied IMF Conditional modal components c 1( t );
[0017] (3) Subtract the first modal component from the original signal to obtain the signal ,Will r 1( t ) as the new "original signal" and repeat the above steps until r n ( t ) is a monotonic function or has only one extreme point, the original signal is divided into several empirical mode components c i ( t ) and a residual signal, the original signal is expressed as ;
[0018] (4) For each IMF Quantity c i ( t ) Find its Hilbert transform (HT): ,
[0019] According to the following formula, the corresponding IMF The instantaneous frequency of the component a i ( t ) and instantaneous amplitude w i ( t ),
[0020] ,
[0021] ,
[0022] in, c i ( t )express m ( t ) IMF Quantity, represents the Hilbert transform of the input signal, d() / dt represents differentiation, and arctan represents the arctan function; the original signal is represented as:
[0023] ,
[0024] In passing n After the second EMD decomposition, the residual signal is a constant or a monotonic function, which has no substantial effect on signal extraction and is therefore discarded.
[0025] The feature learning network ETN described in step S3 learns the signal time-frequency feature information within the same receptive field at different time scales through the convolution operation of 3DCNN. The constructed ETN network contains four convolution layers and three pooling layers. The convolution layers are used to learn the high-order features of the data. Three different sizes of 3D convolution kernels are set, with sizes of 2 5 5.2 3 3.2 1 1, and the sizes of the convolution kernels in the network are 2 5 5.2 3 3.2 3 3.2 1 1. In addition, the three convolutional layers Conv3D_1, Conv3D_3 and Conv3D_4 in the network are also used to increase the number of feature maps.
[0026] The global channel feature extraction network described in step S4 first reshapes the input features through the residual convolution module to obtain feature vectors F e and F f , then F e and F f Perform matrix cross multiplication to get the matrix R C ,matrix R C After the pooling layer is averaged, a one-dimensional vector representing the global relationship between each channel and all channels is obtained. The Softmax function is used to convert all global relationships into the weight of each channel, and the weight is multiplied by the input feature to output the global channel feature. The residual convolution module has two convolution layers, one is 1 1 convolutional layer, one is 3 3 convolutional layers, the signal after two convolutions is added to the input and then enters the activation function.
[0027] Input features of the global channel feature extraction network , C Indicates the number of channels of the input signal, H , W Represents the width and length of a single data channel passing through the residual convolution module. The specific calculation steps are as follows:
[0028] (1) Input signal inpute ( z ), first do 1 1. Convolution reduces dimension, improves the expressiveness of the model, and reduces 3 3 The number of parameters required for the convolution kernel, and then 3 3. Convolution increases the receiving domain and reduces the number of parameters. Then the input is added to the signal after two convolutions to obtain F e ;
[0029] (2) Input signal and then input f ( z ), f ( z ) calculation and e ( z ) is the same, the input signal is f ( z ) and then get F f , then F e and F f Perform matrix cross multiplication and get ;
[0030] (3) Apply relational average pooling to the matrix R C , and get a vector , r The elements in represent the global relationship between each channel and all channels. r The formula for calculating the elements in is:
[0031] ,
[0032] in, k express r The first k elements, j Indicates j channels;
[0033] (4) The Softmax function is used to convert all global relationships into the weight of each channel. The calculation formula is:
[0034] ,
[0035] in, p represents the weight of the channel,
[0036] (5) Multiply the weights by the input features to output global channel features F c :
[0037] .
[0038] The local feature extraction network is to perform an average pooling operation on the input features before entering feature segmentation. Feature segmentation is to segment the feature signals of the three signals of EEG, EOG, and EMG. They are first divided into three feature vectors through a granular segmentation function, and then classified through a fully connected layer (FC) to generate three feature vectors F1, F2, and F3.
[0039] The global spatial feature extraction network uses the output feature F of step S3 as the input of the local module, and first performs feature annular segmentation. The annular segmentation uses different rectangular windows to cut the input feature signal into n non-overlapping feature layers F. n , and then use the average pooling layer to convert each ring feature layer into a column feature descriptor f n .
[0040] The filtering process described in step S5 is:
[0041] Use the calculation formula of mutual information to calculate the mutual information value for each feature F and target variable Y:
[0042] ,
[0043] in, p ( f , y ) indicates characteristics f and the target variable y The joint probability distribution of p ( f )and p ( y ) represent the characteristics f and the target variable y The marginal probability distribution of
[0044] According to the calculated mutual information value, all features are sorted from large to small according to their mutual information values with the target variable, and the first n features with the highest mutual information values are selected based on the sorting results.
[0045] The process of using the attention perception mechanism to fuse features in step S5 is as follows:
[0046] (1) The feature signal is input into the attention perception mechanism layer. First, the input data is spliced in the granularity channel direction, and a global average pooling GAP operation is used to retain the number of granularity channels of the input feature. The feature is normalized to obtain the directional information on different granularity features:
[0047] ,
[0048] Among them, MG represents the number of granularity channels, M×N represents the size of the feature map, Represents the value of each pixel on the feature map;
[0049] (2) Two linear layers are set to learn the information value weights of features of different granularity. The activation functions of the two linear layers are ReLU function and sigmoid function respectively:
[0050] ,
[0051] in, W 1 and W 2 represents the weights of the two linear layers, δ (•) and σ(•) represent the ReLU function and the sigmoid function, respectively;
[0052] (3) Assign attention weights to the original input data and sum them along the granularity channel dimension to obtain the fused output Fusion :
[0053] ,
[0054] in, Fusion Represents the data of a certain granularity feature after scaling based on the attention weight. weight MG represents the data before the attention weight is applied to a certain granularity feature (i.e., the feature data spliced in the granularity channel direction), MG represents the number of granularity channels, x MG Represents the signal of the MG channel.
[0055] Another object of the present invention is to provide a sleep stage classification system based on multi-granularity feature fusion for implementing the above method, comprising:
[0056] A sleep data acquisition unit, used to obtain the user's electroencephalogram (EEG) signals, electrooculogram (EOG) signals and electromyography (EMG) signals during sleep;
[0057] A signal preprocessing unit is used to segment the collected EEG signals, electrooculographic signals and electromyographic signals of the user during sleep, filter out interference noise, and remove baseline drift and interference artifacts;
[0058] Feature extraction unit, used for extracting time-frequency features of EEG signals, electrooculogram signals, and electromyography signals, extracting high-order sleep features, and extracting local and global features;
[0059] A feature fusion unit is used to fuse the extracted local and global features;
[0060] Classification unit, used to implement the classification of sleep stages.
[0061] The beneficial effects of the present invention are:
[0062] (1) The present invention uses EEG signals and non-EEG signals (electromyographic signals and electrooculographic signals) as input signals, thus avoiding the problem of low accuracy of EEG signals in distinguishing between rapid eye movement (REM) and non-rapid eye movement (NREM) periods.
[0063] (2) The present invention uses HHT to extract the time-frequency domain feature signals of EEG, EOG and EMG, and inputs the extracted feature signals into the ETN sub-network built based on 3DCNN to obtain high-order tensors. As a multidimensional data structure, the high-order tensor can fully represent these multidimensional information in EEG, EOG and EMG signals, thereby retaining more original data details. In addition, as a nonlinear analysis tool, the high-order tensor can extract nonlinear features in EEG signals, thereby better understanding and analyzing the essence of brain activity;
[0064] (3) The present invention adopts a three-branch network to obtain local features and global features. The local features provide detailed information, while the global features provide an overall overview. The combination of these two features can more comprehensively understand and analyze the relevant signals during sleep.
[0065] (4) This method uses mutual information filtering and attention perception mechanism to fuse features. The mutual information filtering method is used to remove the features with weak correlation in each channel, and then the attention perception mechanism is used to fuse the filtered feature signals. This avoids the interference of irrelevant feature signals, thereby obtaining feature information with higher correlation, making the classification of sleep stages more accurate.
[0066] The method of the present invention first uses HHT to extract time-frequency features to obtain a time-frequency matrix with representation information, further extracts sleep high-order feature tensors of multi-channel data of EEG signals, EOG signals, and EMG signals through the ETN network, and further uses a three-branch network to obtain more comprehensive feature information for global and local features; finally, in the fusion classification part, the features of each channel are screened by the self-information filtering method, and the feature information between different channels is fused using the attention perception module. The scheme of the present invention can effectively improve the classification accuracy and classification performance of sleep stages. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 is a flow chart of the method of the present invention;
[0068] Figure 2 ENT network structure diagram in the method of the present invention;
[0069] Figure 3 Hilbert-Huang transform HHT structure diagram in the method of the present invention;
[0070] Figure 4 A diagram of a three-branch network structure in the method of the present invention;
[0071] Figure 5 The structure diagram of the residual module in the method of the present invention;
[0072] Figure 6 The global channel feature extraction network block diagram in the method of the present invention;
[0073] Figure 7 The local feature extraction network architecture diagram in the method of the present invention;
[0074] Figure 8 The global spatial feature extraction network architecture diagram in the method of the present invention;
[0075] Fig. 9 Schematic diagram of the structure of the attention perception fusion module of the method of the present invention;
[0076] Fig.10 Schematic diagram of the model network structure of the method of the present invention. DETAILED DESCRIPTION
[0077] The present invention is further described below with reference to specific embodiments.
[0078] Embodiment 1 A sleep stage classification method based on multi-granularity feature fusion comprises the following steps (eg Figure 1 )as follows:
[0079] S1. Obtain physiological signals during sleep through sensors, including EEG, EOG and EMG:
[0080] During the implementation of the present invention, the experimental data set used is the public data set SLEEP_EDF, which contains 197 PolySomnoGraphic sleep records of 20 people throughout the night. The experiment uses a sleep dual-channel EEG, and the positions of the EEG electrodes are Fpz_Cz and Pz_Oz respectively. The EOG, EMG and EEG signals are sampled at 100 Hz, and the event marker is 1 Hz.
[0081] S2. Preprocess the EEG, EOG, and EMG data collected during the user's sleep:
[0082] First, the EEG signals, EOG signals, and EMG signals were manually segmented into segments of 30 seconds each.
[0083] Independent component analysis was used to remove interference artifacts caused by user movement, swallowing and other actions. Next, a wavelet cascade adaptive filter was used to remove baseline drift; a 0.5-30Hz bandpass filter was set to filter out interference noise.
[0084] S3. After the preprocessed physiological signals are subjected to empirical mode decomposition, the time-frequency features are extracted using Hilbert transform to obtain a time-frequency matrix with representation information. The matrix is then input into the feature learning network ETN built based on 3D-CNN to extract the sleep high-order features of EEG signals, EOG signals, and EMG signals:
[0085] The input signal is decomposed into several IMFs and a residual component using empirical mode decomposition (EMD). And the IMF must meet the following two conditions:
[0086] (1) In the entire signal component IMF, the number of extreme values and the number of zeros must be equal to or differ by at most 1;
[0087] (2) In the entire signal component IMF, the local average of the upper envelope defined by the IMF maximum and the lower envelope defined by the IMF minimum is zero, so the input signal can be further expressed as:
[0088] ,
[0089] in, S ( t ) represents the input signal to be decomposed, IMF i ( t ) represents the eigenmode function IMFs ,and R n ( t ) represents the decomposition residue of the signal;
[0090] The Hilbert transform is used to obtain the IMF The instantaneous frequency a i ( t ) and instantaneous amplitude w i ( t ), the calculation formulas of the two variables are:
[0091] ,
[0092] ,
[0093] in, c i ( t ) represents the original signal, is the Hilbert transform of the input signal, d () / dt represents the differential, arctan() represents the arctan function, and the Hilbert transform process is expressed as .
[0094] The 30-second EEG, EOG, and EMG signals were further divided into six 5-second segments, and the Hilbert-Huang transform (HHT) was used to analyze the data. m ( t ) are converted into time-frequency matrix data respectively. The Hilbert-Huang transform (HHT) step includes empirical mode decomposition (EMD) and Hilbert transform (HT). The time-frequency matrix data is stacked into a form suitable for the input of 3DCNN. The time-frequency features are finally obtained. The proposed ETN network learns the signal time-frequency feature information within the same receptive field at different time scales through the convolution operation of 3DCNN. The constructed ETN network contains four convolution layers and three pooling layers. The convolution layers are used to learn the high-order features of EEG, EOG, and EMG data. Three different sizes of 3D convolution kernels are set, with sizes of 2 5 5.2 3 3.2 1 1, and the sizes of the convolution kernels in the network are 2 5 5.2 3 3.2 3 3.2 1 1. The three convolutional layers Conv3D_1, Conv3D_3 and Conv3D_4 in the network are also used to increase the number of feature maps.
[0095] S4. The high-order features are input into the parallel three-branch network global channel feature extraction network, local feature extraction network, and global spatial feature extraction network to extract local and global features of EOG, EMG, and EEG signals to obtain more comprehensive feature information:
[0096] The global channel feature extraction network is as follows Figure 6 As shown in the figure, the input features are first reshaped by the residual convolution module to obtain the feature vectors F e and F f , then F e and F f Perform matrix cross multiplication to get the matrix R C ,matrix R CAfter the pooling layer is averaged, a one-dimensional vector representing the global relationship between each channel and all channels is obtained. The Softmax function is used to convert all global relationships into the weight of each channel, and the weight is multiplied by the input feature to output the global channel feature. The residual convolution module has two convolution layers, one is 1 1 convolutional layer, one is 3 3 convolutional layers, the signal after two convolutions is added to the input and then enters the activation function.
[0097] Input Features , C Indicates the number of channels of the input signal, H , W Represents the data width and length of a single channel. The specific calculation steps are as follows:
[0098] (1) Input signal input e ( z ), first do 1 1. Convolution reduces dimension, improves the expressiveness of the model, and reduces 3 3 The number of parameters required for the convolution kernel, and then 3 3. Convolution increases the receiving domain and reduces the number of parameters. Then the input is added to the signal after two convolutions to obtain F e ;
[0099] (2) Input signal and then input f ( z ), f ( z ) calculation and e ( z ) is the same, the input signal is f ( z ) and then get F f , then F e and F f Perform matrix cross multiplication and get ;
[0100] (3) Apply relational average pooling to the matrix R C , and get a vector , r The elements in represent the global relationship between each channel and all channels. r The formula for calculating the elements in is:
[0101] ,
[0102] in, k express rThe first k elements, j Indicates j channels;
[0103] (4) The Softmax function is used to convert all global relationships into the weight of each channel. The calculation formula is:
[0104] ,
[0105] in, p represents the weight of the channel,
[0106] (5) Multiply the weights by the input features to output global channel features F c :
[0107] .
[0108] Local feature extraction network, such as Figure 7 As shown, the output feature F of step S3 is used as the input of the local module, and the steps are as follows:
[0109] (1) After the average pooling operation, the redundancy of feature information is removed and converted into a 3072-dimensional feature vector;
[0110] (2) The granularity segmentation function of the feature layer is divided into three 1024-dimensional feature vectors;
[0111] (3) Classification is performed through the fully connected layer (FC) to generate three 512-dimensional feature vectors F1, F2, and F3.
[0112] The calculation formula is:
[0113] ,
[0114] in, F split represents the feature segmentation operation, F LB represents the local feature network operation, F AP represents the average pooling operation, F 1 / 2 Represents two feature vectors after feature segmentation.
[0115] The global spatial feature extraction network is as follows Figure 8 As shown, the output feature F of step S3 is used as the input of the local module, and the steps are as follows:
[0116] (1) First, slice the output features of the backbone network according to the ring segmentation strategy and cut them into n non-overlapping feature layers F n ;
[0117] (2) Use the average pooling layer to convert each ring feature layer into a column feature descriptor f n , n =1,2,3,4.
[0118] S5. Use the filtering method to filter the features of each channel, and use the attention-aware mechanism to fuse the features output from the three-branch network:
[0119] (1) Use the mutual information calculation formula (as follows) to calculate the mutual information value for each feature (set as F) and the target variable (set as Y):
[0120] ,
[0121] in, p ( f , y ) indicates characteristics f and the target variable y The joint probability distribution of p ( f )and p ( y ) represent the characteristics f and the target variable y The marginal probability distribution of
[0122] According to the calculated mutual information value, all features are sorted from large to small according to the mutual information value with the target variable, and the first n features with the highest mutual information value are selected according to the sorting result;
[0123] (2) The feature signal is input into the attention perception mechanism layer for fusion. First, the input data is concat-joined in the granularity channel direction, and a global average pooling (GAP) operation is used to retain the number of granularity channels of the input feature. The size of the feature map is set to 1 to obtain directional information on features of different granularities:
[0124] ,
[0125] Among them, MG represents the number of granularity channels, M×N represents the size of the feature map, Represents the value of each pixel on the feature map;
[0126] (3) Two linear layers are set to learn the information value weights of features of different granularity. The activation functions of the two linear layers are ReLU function and sigmoid function respectively:
[0127] ,
[0128] in, W1 and W 2 represents the weights of the two linear layers, δ (•)and σ (•) represent ReLU function and sigmoid function respectively;
[0129] (4) Assign attention weights to the original input data and sum them along the granularity channel dimension to obtain the fused output Fusion:
[0130] ,
[0131] in, Fusion Represents the data of a certain granularity feature after scaling based on the attention weight. weight MG Represents the data before the attention weight is applied to a certain granular feature, MG Indicates the number of granularity channels, x MG Represents the signal of the MG channel.
[0132] S6. The fused features are input into a classification network consisting of a full connection and a Softmax function to classify the sleep stages:
[0133] The fused feature signal of step S5 is used as the fully connected input signal, and the probability of the input sample belonging to each category is calculated by the Softmax function. At the end of the entire model framework, the present invention sets a linear classification layer with 4 neurons as the staging criterion, and classifies the extracted features into 4 sleep stages, namely wakefulness (W), non-rapid eye movement (NREM), rapid eye movement (REM), and wakefulness (M). In addition, the number and category of sleep stages are not uniform standards, and the number of classification neurons in the present invention can be changed according to actual needs and specific circumstances.
[0134] The present invention uses several common evaluation indicators in the field of machine learning involved in model performance evaluation, including classification accuracy and F1 value. These evaluation indicators are calculated from true positive (TP), true negative (TN), false positive (FP) and false negative (FN).
[0135] Classification accuracy refers to the ratio of the number of correctly predicted samples to the total number of predicted samples:
[0136] ,
[0137] The F1 value is used to comprehensively consider the two indicators of precision and recall, and is used to comprehensively reflect the overall indicators of the model:
[0138] ,
[0139] Among them, the two indicators of precision and recall are defined as:
[0140] ,
[0141] .
[0142] Table 1 shows the comparison of the results of the method of the present invention and other existing model methods. Compared with other methods, the method of the present invention can achieve better classification accuracy under the experimental premise of the same data set.
[0143] Table 1
[0144] .
[0145] Embodiment 2 is a sleep stage classification system based on multi-granularity feature fusion for implementing the method described in Embodiment 1, comprising:
[0146] A sleep data acquisition unit, used to obtain the user's electroencephalogram (EEG) signals, electrooculogram (EOG) signals and electromyography (EMG) signals during sleep;
[0147] A signal preprocessing unit is used to segment the collected EEG signals, electrooculographic signals and electromyographic signals of the user during sleep, filter out interference noise, and remove baseline drift and interference artifacts;
[0148] Feature extraction unit, used for extracting time-frequency features of EEG signals, electrooculogram signals, and electromyography signals, extracting high-order sleep features, and extracting local and global features;
[0149] A feature fusion unit is used to fuse the extracted local and global features;
[0150] Classification unit, used to implement the classification of sleep stages.
[0151] The above is a detailed description of the present invention in combination with the embodiments, and the protection scope of the present invention is not limited thereto.
Claims
1. A sleep stage classification method based on multi-granularity feature fusion, characterized by: The steps include: S1. Obtain physiological signals of the user during sleep through sensors, including EEG, EOG and EMG; S2. Preprocess the EEG, EOG, and EMG data collected during the user's sleep; S3. After the preprocessed physiological signal is subjected to empirical mode decomposition, the time-frequency feature is extracted using Hilbert transform to obtain a time-frequency matrix with representation information, which is then input into a feature learning network ETN built based on 3D-CNN to extract high-order sleep features of EEG signals, EOG signals, and EMG signals; the feature learning network ETN learns the time-frequency feature information of signals within the same receptive field at different time scales through the convolution operation of 3DCNN. The constructed ETN network contains four convolutional layers and three pooling layers, and the convolutional layers are all used to learn high-order features of the data; S4. Input the high-order features into the parallel three-branch network global channel feature extraction network, local feature extraction network, and global spatial feature extraction network to extract local and global features of EOG, EMG, and EEG signals to obtain more comprehensive feature information; The global spatial feature extraction network uses the output feature F of step S3 as the input of the local module, and first performs feature ring segmentation. Ring segmentation uses different rectangular windows to cut the input feature signal into n non-overlapping feature layers F. n , and then use the average pooling layer to convert each ring feature layer into a column feature descriptor f n; S5. Use the filtering method to filter the features of each channel, and use the attention perception mechanism network to fuse the features output from the three-branch network; S6. The fused features are input into a classification network consisting of a fully connected and softmax function to achieve the classification of sleep stages.
2. The sleep stage classification method based on multi-granularity feature fusion according to claim 1 is characterized in that: The preprocessing described in step S2 is to perform signal segmentation, remove noise and artifacts, and remove baseline drift preprocessing on the EEG signal, the EOG signal, and the EMG signal.
3. The sleep stage classification method based on multi-granularity feature fusion according to claim 1 is characterized in that: The time-frequency feature extraction described in step S3 is to divide the EEG, EOG, and EMG signals into multiple sub-segments, and use the time-frequency analysis method Hilbert-Huang transform HHT to transform these data sub-segments. m ( t ) are converted into time-frequency matrix data respectively. The Hilbert-Huang transform (HHT) step includes empirical mode decomposition and Hilbert transform. The specific steps are as follows: (1) Find the signal m ( t ), the upper and lower envelopes are obtained by cubic spline fitting, and the average value is calculated. ; (2) Get the first component , check whether it satisfies the modal component ( IMF )conditions: a. h 1( t ) has a maximum value and a zero-crossing point number that does not differ by more than 1; b. h 1( t )’s upper and lower envelopes always have a mean of 0; If not satisfied, repeat steps (1) and (2) until satisfied. IMF Conditional modal components c 1( t ); (3) Subtract the first modal component from the original signal to obtain the signal ,Will r 1( t ) as the new "original signal", repeat the above steps until r n ( t ) is a monotonic function or has only one extreme point, the original signal is divided into several empirical mode components c i ( t ) and a residual signal, the original signal is expressed as ; (4) For each IMF Quantity c i ( t ) Find its Hilbert transform: , According to the following formula, the corresponding IMF The instantaneous frequency of the component a i ( t ) and instantaneous amplitude w i ( t ), , , in, c i ( t )express m ( t ) IMF Quantity, represents the Hilbert transform of the input signal, d() / dt represents differentiation, and arctan represents the arctan function; The original signal is represented as: 。 4. The sleep stage classification method based on multi-granularity feature fusion according to claim 1 is characterized in that: The global channel feature extraction network described in step S4 first reshapes the input features through the residual convolution module to obtain feature vectors F e and F f , then F e and F f Perform matrix cross multiplication to get the matrix R C ,matrix R C After averaging processing in the pooling layer, a one-dimensional vector representing the global relationship between each channel and all channels is obtained. The Softmax function is used to convert all global relationships into weights for each channel, and the weights are multiplied by the input features to output global channel features.
5. The sleep stage classification method based on multi-granularity feature fusion according to claim 1 is characterized in that: The local feature extraction network described in step S4 is to perform an average pooling operation on the input features and then enter the feature segmentation. The characteristic signals of EEG, EOG and EMG are segmented. They are first divided into three feature vectors through the granularity segmentation function, and then classified through the fully connected layer to generate three feature vectors F1, F2 and F3.
6. The sleep stage classification method based on multi-granularity feature fusion according to claim 1 is characterized in that: The filtering process described in step S5 is: Use the calculation formula of mutual information to calculate the mutual information value for each feature F and target variable Y: , in, p ( f , y ) indicates characteristics f and the target variable y The joint probability distribution of p ( f )and p ( y ) represent the characteristics f and the target variable y The marginal probability distribution of According to the calculated mutual information value, all features are sorted from large to small according to their mutual information values with the target variable, and the first n features with the highest mutual information values are selected based on the sorting results.
7. The sleep stage classification method based on multi-granularity feature fusion according to claim 1 is characterized in that: The process of using the attention perception mechanism to fuse features in step S5 is as follows: (1) The feature signal is input into the attention perception mechanism layer. First, the input data is spliced in the granularity channel direction, and a global average pooling GAP operation is used to retain the number of granularity channels of the input feature. The feature is normalized to obtain the directional information on different granularity features: , Among them, MG represents the number of granularity channels, M×N represents the size of the feature map, Represents the value of each pixel on the feature map; (2) Two linear layers are set to learn the information value weights of features of different granularity. The activation functions of the two linear layers are ReLU function and sigmoid function respectively: , in, W 1 and W 2 represents the weights of the two linear layers, δ (•) and σ(•) represent the ReLU function and the sigmoid function, respectively; (3) Assign attention weights to the original input data and sum them along the granularity channel dimension to obtain the fused output Fusion : , in, Fusion Represents the data of a certain granularity feature after scaling based on the attention weight. weight MG represents the data before the attention weight is applied to a certain granularity feature, MG represents the number of granularity channels, x MG Represents the signal of the MG channel.
8. A sleep stage classification system based on multi-granularity feature fusion implementing the method of any one of claims 1 to 7, characterized in that: include: A sleep data acquisition unit, used to obtain the user's electroencephalogram (EEG) signals, electrooculogram (EOG) signals and electromyography (EMG) signals during sleep; A signal preprocessing unit is used to segment the collected EEG signals, electrooculographic signals and electromyographic signals of the user during sleep, filter out interference noise, and remove baseline drift and interference artifacts; Feature extraction unit, used for extracting time-frequency features of EEG signals, electrooculogram signals, and electromyography signals, extracting high-order sleep features, and extracting local and global features; A feature fusion unit is used to fuse the extracted local and global features; Classification unit, used to implement the classification of sleep stages.
