ST-GCN-based sleep staging method and system

Abstract

The invention provides a sleep staging method and system based on ST-GCN, and the method comprises the following steps: collecting a sleep electroencephalogram signal of a subject, and carrying out the preprocessing of the electroencephalogram signal; based on the preprocessed electroencephalogram signals, an ST-GCN sleep staging network model is constructed; based on the ST-GCN sleep staging network model, sleep staging of the person to be tested is completed. According to the method, more attention is given to valuable space-time information of the electroencephalogram signals, certain interpretability is achieved, and the accuracy of sleep stage division is higher.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a sleep staging method and system based on ST-GCN. Background Art

[0002] During the sleep process, physiological information can be used to divide different sleep time periods into five normal categories: WAKE, REM, N1, N2 and N3 according to international sleep medicine standards.

[0003] In recent years, the method of using computer equipment and programs to monitor human sleep status and automatically classify it has begun to be widely used. Although modern electronic information technology, machine learning theory, biomedical engineering and other aspects have developed rapidly, there is still no internationally recognized standard for automatic sleep staging methods using machine learning theory in scientific research, medical applications, consumer electronics and other fields. The main reasons include that the basic mechanism theory of human sleep is not thorough and complete, the lack of sufficient understanding and trust in the international standards for artificial sleep staging leads to low consistency in sleep state classification, the lack of expert experience of developers of sleep monitoring systems, and abnormal sleep patterns are mixed with normal sleep patterns. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a sleep staging method and system based on ST-GCN, which achieves accurate staging of sleep stages by acquiring the spatiotemporal characteristics of EEG signals.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A sleep staging method based on ST-GCN, comprising the following steps:

[0007] Collecting EEG signals of the subject while sleeping, and preprocessing the EEG signals;

[0008] Based on the preprocessed EEG signals, the ST-GCN sleep staging network model was constructed;

[0009] Based on the ST-GCN sleep staging network model, the sleep staging of the subject is completed.

[0010] Preferably, the EEG signal is preprocessed by combining an eighth-order Butterworth low-pass filter with wavelet denoising.

[0011] Preferably, the method for constructing the ST-GCN sleep staging network model is:

[0012] Based on the preprocessed EEG signals, a sleep stage network is constructed;

[0013] Extracting node features of the sleep stage network based on a convolutional neural network to obtain features of each node; and obtaining feature correlations between nodes based on the features of each node;

[0014] Based on the feature correlation between nodes and ST-GCN spatiotemporal graph convolution, the spatiotemporal features of EEG signals are obtained;

[0015] Based on the fully connected layer, high-level features in the spatiotemporal features are extracted, and the Softmax layer is used to map the high-level features into sleep category probabilities, thereby completing the construction of the ST-GCN sleep staging network model.

[0016] Preferably, the method for obtaining the spatiotemporal characteristics of the EEG signal is:

[0017] Based on the feature correlation between nodes, a functional connection matrix is ​​adaptively constructed for different sleep stages in the sleep stage network;

[0018] Based on the functional connection matrix and the spatiotemporal attention mechanism, obtaining the spatiotemporal information of the EEG signal;

[0019] Based on the ST-GCN spatiotemporal graph convolution, a convolution operation is performed on the spatiotemporal information to obtain the spatiotemporal characteristics of the EEG signal.

[0020] Preferably, the functional connection matrix includes a dynamic matrix and a functional matrix, wherein the dynamic matrix is ​​constructed based on an adaptive module, and the functional matrix of the entire time period is constructed based on a Pearson correlation coefficient.

[0021] Preferably, the spatiotemporal attention mechanism includes spatial attention and temporal attention, wherein the spatial attention formula is as follows:

[0022]

[0023] P' m,n =softmax(P m,n ),

[0024] in, is the input of layer l, C l-1 Indicates the number of channels of each node; V p 、b p , Z1, Z2, and Z3 are learnable parameters, σ(·) is the sigmoid activation function; P represents the spatial attention matrix, P m,n Represents the feature correlation between nodes m and n; softmax is used to normalize the attention matrix P;

[0025] The temporal attention formula is as follows:

[0026] Q=V q·σ(((x l-1 ) T M1)M2(M3(x l-1 )+b q ))

[0027] Q' u,v =softmax(Q u,v ),

[0028] Where V q 、b q , M1, M2 and M3 are learnable parameters, Q u,v represents the correlation strength between two different brain networks u and v, and softmax is used to normalize the attention matrix Q.

[0029] The present invention also provides a sleep staging system based on ST-GCN, which implements the method described, including:

[0030] An EEG signal acquisition module is used to acquire EEG signals of the subject during sleep and pre-process the EEG signals;

[0031] The sleep staging network model building module is used to build the ST-GCN sleep staging network model based on the preprocessed EEG signals;

[0032] The sleep staging module is used to complete the sleep staging of the subject based on the ST-GCN sleep staging network model.

[0033] Preferably, the sleep staging network model building module includes:

[0034] A sleep stage network construction unit, used to construct a sleep stage network based on the preprocessed EEG signals;

[0035] A node feature extraction unit, configured to extract node features of the sleep stage network based on a convolutional neural network to obtain features of each node; and based on the features of each node, obtain feature correlations between nodes;

[0036] The spatiotemporal feature acquisition unit is used to obtain the spatiotemporal features of EEG signals based on the feature correlation between nodes and ST-GCN spatiotemporal graph convolution;

[0037] The sleep staging unit is used to extract high-level features from the spatiotemporal features based on the fully connected layer, and map the high-level features into sleep category probabilities using the Softmax layer to complete the construction of the ST-GCN sleep staging network model.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention combines the constructed dynamic matrix with the functional connection matrix of the entire time period. The combination of the two matrices allows the system to take into account both dynamic changes and the overall time pattern when processing data; spatial graph convolution and temporal convolution are combined to extract and model spatial and temporal characteristics; the spatiotemporal attention mechanism is adopted to pay more attention to valuable spatiotemporal information and has a certain degree of interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 This is a flow chart of a sleep staging method based on ST-GCN according to an embodiment of the present invention;

[0041] Figure 2 This is a flowchart of EEG signal preprocessing according to an embodiment of the present invention;

[0042] Figure 3 The filter frequency response diagram and the filter effect diagram of the embodiment of the present invention are shown in FIG. 1 ; wherein (a) is the filter frequency response diagram; (b) is the filter effect diagram;

[0043] Figure 4 The timing waveform diagrams of the original EEG signal before and after filtering and denoising according to the embodiment of the present invention; wherein (a) is the original EEG signal diagram; (b) is the EEG signal diagram after Butterworth filtering; (c) is the EEG signal diagram after wavelet denoising;

[0044] Figure 5 The embodiment of the present invention is a sleep characteristic matrix diagram;

[0045] Figure 6 This is a flowchart of node feature extraction according to an embodiment of the present invention;

[0046] Figure 7 This is a schematic diagram of a Pearson correlation coefficient connection matrix according to an embodiment of the present invention;

[0047] Figure 8 This is a diagram of sleep staging results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Embodiment 1

[0051] like Figure 1 As shown, a sleep staging method based on ST-GCN includes the following steps:

[0052] S1: Collect the EEG signals of the subjects while they are sleeping and pre-process the EEG signals; in this embodiment, the subjects are made to wear a standard EEG cap and collect EEG signals according to the international standard 10-20 system electrode method, and multiple electrodes are used for synchronous collection. The EEGlab toolbox in Matlab is used to pre-process the EEG signals: 1. Import EEG data and complete the conversion of .edf and other formats; 2. Import electrode positions, continue electrode positioning and complete re-reference; 3. Eliminate problematic EEG signals; 4. Use independent component analysis to decompose and remove blinking and eye movement interference from the signal.

[0053] The present invention uses the traditional ICA method: first, multi-channel EEG signals are collected; then, independent component analysis is performed on the EEG signals to obtain an unmixing matrix and independent components; kurtosis analysis is performed on the independent components to remove independent components of super-Gaussian noise; the mixed matrix is ​​decomposed by clustering and minimum variance methods to remove the independent components of the reference electrode; finally, the signal is restored to obtain the preprocessed EEG signal.

[0054] The original signal can be expressed as a linear combination of source signals, usually in matrix form: X = AS where X is the observed signal, A is the mixing matrix, and S is the independent component.

[0055] Compared with the traditional filtering method, the present invention uses an eighth-order Butterworth low-pass filter combined with wavelet denoising to pre-process the EEG signal in order to reduce the influence of noise generated by physiological movements such as eyeballs, muscles, and heartbeats on the accuracy of sleep staging during the acquisition of the original EEG signal. Figure 2 shown.

[0056] Since the frequency range of sleep-related EEG signals is 0-30Hz, in order to avoid the influence of irrelevant high-frequency signals, an eighth-order Butterworth low-pass filter is selected for filtering with a cutoff frequency of 35Hz. The filter frequency response diagram and filtering effect diagram are shown in Figure 3 (a) Figure 3 (b) shows: By comparison Figure 4 In the figure (a) original EEG signal, (b) EEG signal after Butterworth filtering, and (c) EEG signal after wavelet denoising, it can be seen that the EEG signal becomes much smoother after preprocessing, achieving a good denoising effect.

[0057] S2: Based on the preprocessed EEG signals, the ST-GCN sleep staging network model is constructed;

[0058] A further implementation method is that the method for constructing the ST-GCN sleep staging network model is:

[0059] Based on the preprocessed EEG signals, a sleep stage network is constructed; in this embodiment, the sleep stage network is defined as an undirected graph G = (V, E, A), where V represents a set of nodes, and each node in the network represents an electrode on the brain; |V| is the number of nodes in the sleep stage network; E represents a set of edges, representing the connections between nodes; A represents the adjacency matrix (functional connection matrix) of the sleep stage network G. t represents a time series of 30 seconds (sleep segments), G t Represents the sleep stage network composed of each sleep segment. Figure 5 As shown, G t The 30-second EEG signal sequence S t constitute.

[0060] S21: Extract node features of the sleep stage network based on the convolutional neural network to obtain the features of each node; and based on the features of each node, obtain the feature correlation between nodes; In this embodiment, three convolutional layers are used to extract node features in each time period, such as Figure 6 As shown in the figure, when extracting node features, the input passes through the maximum pooling layer, the discard layer, the convolution layer, and the reshaping layer in sequence.

[0061] S22: Based on the feature correlation between nodes and ST-GCN spatiotemporal graph convolution, the spatiotemporal features of EEG signals are obtained;

[0062] A further implementation method is that the method for obtaining the spatiotemporal characteristics of the EEG signal is:

[0063] S221: Based on the feature correlation between nodes, adaptively construct a functional connection matrix for different sleep stages in the sleep stage network; a further implementation method is that the functional connection matrix includes a dynamic matrix and a functional matrix, wherein the dynamic matrix is ​​constructed based on an adaptive module, and the functional matrix of the entire time period is constructed based on the Pearson correlation coefficient.

[0064] The method of constructing a dynamic matrix based on the adaptive module is:

[0065] Given a non-negative function To represent x m and x n Functional connections between nodes, based on input node features Calculate, where N represents the number of samples, and F d Represents the feature dimension of each sample. g(x m ,x n ) is implemented by a single-layer neural network with a learnable weight vector Its adjacency matrix (dynamic matrix) can be defined as:

[0066]

[0067] ReLU(.) is the activation function to ensure that the value is non-negative, and the softmax operation can be normalized (FC stands for functional connectivity matrix), and updates its weight vector w by minimizing the following loss function:

[0068]

[0069] In this embodiment, the method for constructing the function matrix of the entire time period based on the Pearson correlation coefficient is:

[0070] The Pearson correlation coefficient is used to calculate the connection between channels during the entire sleep period. The Pearson correlation coefficient measures the time domain linear correlation of two EEG signals x(t) and y(t) at zero lag. The calculation method is as follows, where -1≤R x,y ≤1, -1 means there is a perfect linear negative correlation between the two signals, 0 means there is no linear correlation, and 1 means there is a perfect linear positive correlation between the two signals.

[0071]

[0072] The functional connectivity matrix constructed over the entire time period is as follows Figure 7 shown.

[0073] S222: Based on the functional connection matrix and the spatiotemporal attention mechanism, the spatiotemporal information of the EEG signal is obtained; the spatiotemporal information includes spatial information and time information.

[0074] A further implementation method is that the spatiotemporal attention mechanism includes spatial attention and temporal attention. In the spatial dimension, different brain regions have different effects on sleep stages. In order to extract spatial dynamic attention (spatial information), spatial attention is used. The spatial attention formula is as follows:

[0075]

[0076] P' m,n =softmax(P m,n ),

[0077] in, is the input of layer l, C l-1 Indicates the number of channels of each node, T t-1 V represents the time dimension of the lth layer. p , b p ∈R N×N , is a learnable parameter, σ(·) is the sigmoid activation function; P represents the spatial attention matrix, P m,n Represents the feature correlation between nodes m and n; softmax is used to normalize the attention matrix P;

[0078] In the temporal dimension, there are correlations between adjacent sleep segments, and the correlations vary in different situations. Therefore, the temporal attention mechanism is used to capture the dynamic temporal information in the sleep stage network.

[0079] The temporal attention formula is as follows:

[0080] Q=V q ·σ(((x l-1 ) T M1)M2(M3(x l-1 )+b q ))

[0081] Q' u,v =softmax(Q u,v ),

[0082] Where V q , M1∈R N , is a learnable parameter, Q u,v It represents the correlation strength between the brain networks of two adjacent sleep segments u and v, and softmax is used to normalize the attention matrix Q.

[0083] S223: Perform convolution operation on the spatiotemporal information based on the ST-GCN spatiotemporal graph convolution to obtain the spatiotemporal characteristics of the EEG signal.

[0084] Spatiotemporal graph convolution is a combination of spatial graph convolution and standard temporal convolution, which is used to extract spatial and temporal features simultaneously. It extracts spatial features by aggregating the information of neighbor nodes of each sleeping brain network (spatial information) and captures temporal features by exploiting the temporal dependency of adjacent sleep segments (temporal information).

[0085] (1) Spatial Graph Convolution

[0086] Graph convolution based on spectral graph theory is used to extract spatial features in the spatial dimension, and two adjacency matrices (functional connection matrices) are provided for graph convolution for each sleep segment to be identified. Through multi-layer spatial graph convolution operations, the spatial features of nodes can be gradually extracted. The output of each layer contains the information of adjacent nodes, thereby capturing deeper spatial relationships. The Chebyshev expansion of the graph Laplacian operator is then used to reduce the computational complexity. The K-1 order polynomial used by the Chebyshev graph convolution can be defined as:

[0087]

[0088] Among them, g θ represents the convolution kernel, *G x represents the graph convolution operator, θ∈R K represents the Chebyshev coefficient vector, x is the input data (space vector), L = DA is the Laplace matrix, where D∈R N×N is the degree matrix. λ max is the maximum eigenvalue of the Laplace matrix, I N is the identity matrix.

[0089] (2) Temporal Convolution

[0090] In order to capture the sleep transition rules (which sleep experts use to classify the current sleep segment in combination with adjacent sleep segments), CNN is used to perform convolution operations on the time dimension. Specifically, after the graph convolution operation fully extracts the spatial features of each sleep brain network, a standard temporal convolution layer is used to extract the temporal context information of the current sleep segment. When using convolutional neural networks (CNNs) to process data, the temporal convolution layer can effectively extract context information on the time dimension, especially when processing sleep segments. The temporal convolution operation of the lth layer is defined as:

[0091]

[0092] Among them, ReLU represents the activation function, Φ represents the parameters of the convolution kernel, and * represents the standard convolution operation.

[0093] It is necessary to fuse the features from different perspectives, and perform feature fusion using splicing operations:

[0094] w=x FC ||x pC

[0095] where x FC , x pC They represent the adaptive functional connectivity matrix (dynamic matrix) and the features extracted by the Pearson correlation method for the entire time period, respectively, and “∥” represents splicing.

[0096] S23: Based on the fully connected layer, high-level features in the spatiotemporal features are extracted, and the Softmax layer is used to map the high-level features into sleep category probabilities to complete the construction of the ST-GCN sleep staging network model.

[0097] The sleep EEG signal is classified into five stages: Wake, N1, N2, N3, and REM. The classification probability of the model is as follows: The confusion matrix is ​​as follows Figure 8 shown.

[0098] S3: Based on the ST-GCN sleep staging network model, complete the sleep staging of the subject.

[0099] The ST-GCN sleep staging network model is used to classify the sleep stages. The nap sleep parameters before and after different stimulations are shown in Table 1:

[0100] Table 1

[0101]

[0102] Note: The data in the table are from a study of the effects of two stimulation conditions on different sleep stages.

[0103] The above table summarizes the changes in each sleep stage under different stimulation conditions.

[0104] Embodiment 2

[0105] The present invention also provides a sleep staging system based on ST-GCN, which is used to implement a method, including:

[0106] The EEG signal acquisition module is used to collect the EEG signals of the subjects during sleep and pre-process the EEG signals.

[0107] The sleep staging network model building module is used to build the ST-GCN sleep staging network model based on the preprocessed EEG signals. The sleep staging module is used to complete the sleep staging of the subject based on the ST-GCN sleep staging network model.

[0108] In a further embodiment, the sleep staging network model building module includes:

[0109] The sleep stage network construction unit is used to construct a sleep stage network based on the preprocessed EEG signals.

[0110] The node feature extraction unit is used to extract node features of the sleep stage network based on a convolutional neural network to obtain the features of each node; and based on the features of each node, obtain the feature correlation between nodes.

[0111] The spatiotemporal feature acquisition unit is used to obtain the spatiotemporal features of EEG signals based on the feature correlation between nodes and ST-GCN spatiotemporal graph convolution.

[0112] The sleep staging unit is used to extract high-level features from the spatiotemporal features based on the fully connected layer, and use the Softmax layer to map the high-level features into sleep category probabilities to complete the construction of the ST-GCN sleep staging network model.

[0113] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A sleep staging method based on ST-GCN, characterized in that: The following steps are involved: Collecting EEG signals of the subject while sleeping, and preprocessing the EEG signals; Based on the preprocessed EEG signals, the ST-GCN sleep staging network model was constructed; Based on the ST-GCN sleep staging network model, the sleep staging of the subject is completed.

2. The sleep staging method based on ST-GCN according to claim 1, characterized in that: The EEG signal is preprocessed by combining an eighth-order Butterworth low-pass filter with wavelet denoising.

3. The sleep staging method based on ST-GCN according to claim 1, characterized in that: The method for constructing the ST-GCN sleep staging network model is: Based on the preprocessed EEG signals, a sleep stage network is constructed; Extracting node features of the sleep stage network based on a convolutional neural network to obtain features of each node; And based on the features of each node, the feature correlation between nodes is obtained; Based on the feature correlation between nodes and ST-GCN spatiotemporal graph convolution, the spatiotemporal features of EEG signals are obtained; Based on the fully connected layer, high-level features in the spatiotemporal features are extracted, and the Softmax layer is used to map the high-level features into sleep category probabilities, thereby completing the construction of the ST-GCN sleep staging network model.

4. The sleep staging method based on ST-GCN according to claim 3, characterized in that: The method to obtain the spatiotemporal characteristics of EEG signals is: Based on the feature correlation between nodes, a functional connection matrix is ​​adaptively constructed for different sleep stages in the sleep stage network; Based on the functional connection matrix and the spatiotemporal attention mechanism, obtaining the spatiotemporal information of the EEG signal; Based on the ST-GCN spatiotemporal graph convolution, a convolution operation is performed on the spatiotemporal information to obtain the spatiotemporal characteristics of the EEG signal.

5. The sleep staging method based on ST-GCN according to claim 4, characterized in that: The functional connection matrix includes a dynamic matrix and a functional matrix, wherein the dynamic matrix is ​​constructed based on an adaptive module, and the functional matrix of the entire time period is constructed based on a Pearson correlation coefficient.

6. The sleep staging method based on ST-GCN according to claim 4, characterized in that: The spatiotemporal attention mechanism includes spatial attention and temporal attention, wherein the spatial attention formula is as follows: P' m,n =softmax(P m,n ), in, is the input of layer l, C l-1 Indicates the number of channels of each node; V p 、b p , Z1, Z2, and Z3 are learnable parameters, σ(·) is the sigmoid activation function; P represents the spatial attention matrix, P m,n Represents the feature correlation between nodes m and n; softmax is used to normalize the attention matrix P; The temporal attention formula is as follows: Q=V q ·σ(((x l-1 )TM1)M2(M3(x l-1 )+b q )) Q' u,v =softmax(Q u,v ), Where V q 、b q , M1, M2 and M3 are learnable parameters, Q u,v represents the correlation strength between two different brain networks u and v, and softmax is used to normalize the attention matrix Q.

7. A sleep staging system based on ST-GCN, implementing the method according to any one of claims 1 to 6, characterized in that: include: An EEG signal acquisition module is used to acquire EEG signals of the subject during sleep and pre-process the EEG signals; The sleep staging network model building module is used to build the ST-GCN sleep staging network model based on the preprocessed EEG signals; The sleep staging module is used to complete the sleep staging of the subject based on the ST-GCN sleep staging network model.

8. The ST-GCN-based sleep staging system according to claim 7, characterized in that: The sleep staging network model building module includes: A sleep stage network construction unit, used to construct a sleep stage network based on the preprocessed EEG signals; A node feature extraction unit, configured to extract node features of the sleep stage network based on a convolutional neural network to obtain features of each node; and based on the features of each node, obtain feature correlations between nodes; The spatiotemporal feature acquisition unit is used to obtain the spatiotemporal features of EEG signals based on the feature correlation between nodes and ST-GCN spatiotemporal graph convolution; The sleep staging unit is used to extract high-level features from the spatiotemporal features based on the fully connected layer, and map the high-level features into sleep category probabilities using the Softmax layer to complete the construction of the ST-GCN sleep staging network model.

Images