A sleep staging method based on spatiotemporal feature coding and multi-source fusion

Abstract

The present invention provides a sleep staging method based on spatiotemporal feature coding and multi-source fusion, which belongs to the field of big data analysis technology. The method of the present invention utilizes the collected multi-channel and multi-source physiological data, including electroencephalogram, electromyography and electrooculogram signals, to construct a sleep staging neural network model that integrates multi-channel and multi-source physiological data; establishes a graph space encoder for modeling multi-channel spatial features, encoding the interaction and position information between different channels; establishes a multi-signal source fusion module for fusing multi-source signals, and identifies the different contributions of multiple signal sources to staging based on the attention mechanism; establishes a temporal Transformer encoder for modeling time features, and simultaneously encodes local and global time features. The present invention helps to realize the automatic staging of sleep, improves the accuracy of staging, can provide personal help to individuals, provide doctors with convenient tools, and provide heuristic auxiliary guidance for precision medicine.

Description

Technical Field

[0001] The present invention provides a sleep analysis technology based on polysomnography (PSG) electrophysiological data, specifically relates to an automatic sleep staging method based on spatiotemporal feature coding and multi-source fusion, and belongs to the field of big data analysis technology. Background Art

[0002] Advances in communications and computing technologies have enabled widespread application in the collection and analysis of physiological data. Due to its highly flexible, non-invasive, and portable nature, physiological data analysis has become an effective tool for assessing physiological status. Physiological data contain a wealth of potentially valuable information crucial for monitoring vital health. Sleep staging is a key application, as sleep profoundly impacts human health, performance, and well-being. Sleep staging technology can be used to diagnose and prevent a variety of mental and physical health issues. Sleep is divided into several stages, including three non-rapid eye movement (NREM) sleep stages (N1, N2, and N3), REM sleep (R), and wakefulness (W). Correctly identifying each stage is crucial for supporting specific physiological and psychological functions, as well as overall sleep quality. However, manual sleep staging is a demanding, time-consuming, and labor-intensive task for physicians, and the results can be influenced by subjective judgment. Therefore, the development of automated sleep staging technology would greatly assist physicians and patients and possess high practical value.

[0003] Physiological data for sleep staging is typically collected by polysomnography. This data is highly complex. Specifically, it has multiple electrode channels, each strictly corresponding to a specific part of the human body. It originates from multiple physiological signal sources, including electroencephalogram (EEG), electromyography (EMG), and electrooculography (EOG). Furthermore, these signals contain characteristic waveforms, and the type, number, and duration of these waveforms influence sleep staging results. Doctors use a frame as the basic time unit for sleep staging, typically 30 seconds. In recent years, academics and practitioners have developed numerous methods for identifying and analyzing physiological data. Existing technologies focus on analyzing a single channel or a very small number of channels, lacking the ability to utilize spatial information across multiple channels. Furthermore, existing technologies primarily analyze a single physiological signal source, neglecting the synergistic effects of multiple signal sources. Among the few methods that consider multi-channel spatial information or the influence of multiple signal sources, existing technologies typically simply fuse this information through addition or concatenation. This neglects the temporal characteristics of the waveforms unique to polysomnography data and hinders the effective utilization of information from multiple sources of time and space.

[0004] Currently, the identification and analysis of polysomnographic data still face the following challenges: First, polysomnographic data has multiple channels with complex spatial features and interactions between channels, which are difficult to capture. Second, polysomnographic data comes from multiple physiological signal sources, each of which has a unique impact on the staging results. For example, doctors mainly distinguish between N1, N2, and N3 stages based on EEG characteristics, and mainly judge W stage based on EOG characteristics. How to effectively integrate information from multiple sources is a difficult point. Third, doctors judge the staging results based on the characteristic waveforms in a frame of polysomnographic data, including the type, number, location, duration, etc. of these waveforms. How to simultaneously process the global and local temporal features of the characteristic waveforms requires in-depth research. Summary of the Invention

[0005] In response to the shortcomings of the above-mentioned existing technologies, the present invention provides a sleep staging method based on spatiotemporal feature coding and multi-source fusion. Taking into account the multi-channel and multi-source characteristics of data, the spatial and temporal characteristics of polysomnographic data are fully considered. Based on the medical prior knowledge such as waveform characteristics and electrode spatial locations that doctors refer to when manually performing sleep staging, an automatic sleep staging neural network model based on spatiotemporal feature coding and multi-source fusion is established to realize the recognition and analysis of new polysomnographic data.

[0006] The technical solutions provided by the present invention are as follows:

[0007] A sleep staging method based on spatiotemporal feature coding and multi-source fusion uses the collected individual polysomnography data to establish an automatic sleep staging neural network model based on spatiotemporal feature coding and multi-source fusion to achieve the staging and identification of the individual's subsequent sleep stages. It includes the following steps:

[0008] The sleep staging neural network model consists of three modules: 1) graph-space encoder, 2) multi-signal source fusion module, and 3) temporal transformer encoder. The graph-space encoder is used to encode the spatial interaction information in the polysomnography data; the multi-signal source fusion module fuses the information of EEG, EMG, and EOG signals in a learnable way; and the temporal transformer encoder captures local and global temporal features. Each frame of polysomnography data is subjected to sleep staging, and C is set to represent the number of channels of the polysomnography data, and L is set to represent the number of channels of the polysomnography data. e Indicates the number of sampling points in a frame, X e Represents a frame of initial input, i.e. In order to make better use of the characteristic waveform, the polysomnography data of one frame is first divided into multiple segments. p Indicates the number of sampling points in a fragment, X p Represents a fragment, i.e. The input of the graph space encoder and the multi-signal source fusion module is a single segment, while the temporal Transformer encoder takes all segments that have undergone graph space encoding and multi-signal source fusion as input.

[0009] 1) Constructing a graph space encoder for spatial features; including steps 1a) to 1b):

[0010] 1a) Build a multi-layer perceptron encoder for myoelectric and EOG signals:

[0011] Myoelectric and EO signals have only one or two channels, and their spatial information is far less than that of EEG signals, which have relatively more channels. For these signals, a multi-layer perceptron (MLP) consisting of linear layers and residual connections is used to initially encode them. The MLP encoders for myoelectric and EO signals are expressed as Equations 1 and 2, respectively:

[0012] H′ EMG =MLP(H EMG ) (Formula 1)

[0013] H′ EOG =MLP(H EOG ) (Formula 2)

[0014] in, are the encoded myoelectric and oculoscopic signals, Before encoding, they are in a single fragment X p The myoelectric channel and eye electrocardiogram channel signals are selected from C. EMa and C EOG are the number of channels of EMG and EOG signals in the clip, respectively.

[0015] lb) Constructing a sparse graph neural network for EEG signals:

[0016] EEG signals have relatively many channels and are strictly located on the head, so their multi-channel spatial connectivity information is significant. One-dimensional convolution is used to convert EEG signals into graph node embeddings. The conversion process is expressed as Equation 3:

[0017]

[0018] in, is the initial embedding of the graph nodes converted from EEG signals, For a single fragment X p The EEG channel signal selected from C EEG is the number of EEG signal channels in the segment.

[0019] Based on the graph nodes converted from EEG signals, a sparse connection graph is constructed, and the spatial connections between channels are explicitly learned using a multi-head attention mechanism. Compared with a fully connected graph, the sparse connection graph used in this method reduces the number of edges and thus reduces computational complexity. The steps for constructing a sparse connection graph are as follows:

[0020] Random regular graph: Randomly arrange the graph nodes converted from EEG signals, connect the points in the rearranged order, so that each point is connected to a constant number of other points, and construct a random regular graph.

[0021] Virtual connection graph: A constant number of virtual nodes are established. The virtual nodes have learnable node embeddings. The virtual nodes are connected to each real graph node to build a virtual connection graph.

[0022] The random regular graph and the virtual connection graph jointly construct sparse connections on the graph nodes. The multi-head attention mechanism is used to calculate the weights of the edges connecting the nodes. Taking a single edge as an example, the weight calculation is expressed as Equation 4:

[0023] e i：j =(W K h i ) T W Q h j (Formula 4)

[0024] Where i, j = 1, 2, 3, ..., N, N is the total number of segments into which the polysomnographic data is divided, and W K , represents the learnable parameters, represents the weight of the edge connecting node i and node j. The embedding of node i in . The superscript T is the transpose symbol.

[0025] The node embedding of the graph is updated using the message propagation mechanism, as shown in Equation 5:

[0026]

[0027] Where σ is the activation function. b is the bias, is the set of nodes connected to node i. When performing message propagation, the attention weight e of each edge connected to the node calculated by formula 4 is applied j：i . is the updated embedding of node i, and all node embeddings converted from EEG signals together constitute the encoded EEG signal

[0028] 2) Construct a multi-signal source fusion module:

[0029] PSG data includes multiple signal sources such as EEG, EMG, and EOG. This method uses a multi-head attention mechanism to fuse multiple source signals on the channel dimension C. First, multiple signal sources encoded by the graph space encoder are concatenated, as shown in Equation 6:

[0030]

[0031] Among them, H′ EEG , H' EOG , H′ EMG They are EEG, EOG and EMG signals that have passed through the image space encoder. After splicing them and transposing them, we get Then, multi-head attention is used to focus on the contributions of different signal sources. The attention formula is as follows:

[0032]

[0033] Where Q, K, and V correspond to the query, key, and value in the Transformer architecture, respectively. The softmax function maps attention to the interval [0, 1]. This method uses self-attention, and the values ​​of Q, K, and V in Equation 7 are the same and equal to H′. Subsequently, a linear layer is used to fuse the multiple signal sources on the channel dimension C. The output of the multi-signal source fusion module is shown in Equation 8:

[0034] x′ p =Linear(Attn(H′)) (Equation 8)

[0035] in, It is a single segment that has undergone graph space encoding and multi-source signal fusion.

[0036] 3) Construct a temporal Transformer encoder for temporal features

[0037] Physiological signals have diverse local waveforms, which constitute their local temporal features. The number and location of these waveforms constitute their global temporal features. Both local and global temporal features are important for interpreting and classifying physiological signals. The temporal Transformer encoder designed in this method considers both local and global temporal features of the signal. First, all segments that have undergone graph-space encoding and multi-source signal fusion are concatenated as input to the temporal Transformer encoder, as shown in Equation 9.

[0038] z [0] =[x′ p1 , x′ p2 , x′ p3 ,...,x′ pN ] T (Equation 9)

[0039] is the input of the temporal Transformer encoder, which is composed of the segments encoded by 1) and 2). N is the total number of segments into which the polysomnography data is divided, x′ p The subscript numbers in the sequence denote the sequence of the segments. The temporal Transformer encoder uses the Transformer architecture to capture important local temporal features. The encoder consists of L Transformer layers. A single Transformer layer includes a multi-layer perceptron (MLP) consisting of multiple linear layers, layer normalization (LN), and multi-head attention (Attn). The formulas for a single Transformer layer are expressed in Equations 10 and 11.

[0040]

[0041] are the input and intermediate values ​​of the th and th Transformer layers, respectively. To preserve global information, the encoded local temporal features are residually connected with the initial global temporal features. Finally, the connected temporal features are flattened and passed through a linear layer to obtain the sleep stage result for the frame, as shown in Equation 12:

[0042]

[0043] is the automatic staging result of the frame obtained by this method, and its value corresponds to the confidence of the five sleep periods (N1, N2, N3, R, W), z [L] is the output of the last Transformer layer, are the flattened local and global time features.

[0044] 4) Sleep staging neural network model training and solution

[0045] The present invention uses categorical cross entropy (CCE) as the loss function and adopts stochastic gradient descent for training. There are S samples and M categories (in this method, M = 5, corresponding to 5 sleep periods). The categorical cross entropy is as shown in Equation 13:

[0046]

[0047] in, Represents the sth sample The confidence level corresponding to type m, y sm is the label of the sth sample. If the sample belongs to category m, the value is 1; if it does not belong to category m, the value is 0.

[0048] Through the above steps, the physiological state recognition and analysis based on multi-source information fusion is realized, and the sleep staging results are obtained.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention provides a sleep staging method based on spatiotemporal feature coding and multi-source fusion. It utilizes collected multi-channel and multi-source physiological data, including EEG, EMG and EOG, to construct a sleep staging neural network model that integrates multi-channel and multi-source physiological data: a graph space encoder is established for modeling multi-channel spatial features to encode the interactions and position information between different channels; a multi-signal source fusion module is established for fusing multi-source signals to identify the different contributions of multiple signal sources to staging based on a multi-head attention mechanism; a temporal Transformer encoder is established for modeling temporal features to simultaneously encode local and global temporal features. By adopting the technical solution provided by the present invention, it is helpful to realize automatic staging of sleep and improve the accuracy of staging. It can provide personal help to individuals, provide doctors with convenient tools, and provide heuristic auxiliary guidance for precision medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of a sleep staging method based on spatiotemporal feature coding and multi-source fusion according to the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below by way of examples in conjunction with the accompanying drawings, but the scope of the present invention is not limited in any way.

[0053] The present invention provides a sleep staging method based on spatiotemporal feature coding and multi-source fusion. Figure 1 Shown is the process of the method of the present invention, including: using the collected multi-channel and multi-source physiological data (polysomnography data), including EEG, EMG and EOG, to construct a neural network model that integrates multi-channel and multi-source physiological data: establishing a graph space encoder for modeling multi-channel spatial features, encoding the interaction and position information between different channels; establishing a multi-signal source fusion module for fusing multi-source signals, identifying the different contributions of multiple signal sources to staging based on a multi-head attention mechanism; establishing a temporal Transformer encoder for modeling time features, encoding local and global time features at the same time. By adopting the technical solution provided by the present invention, it is helpful to realize the automatic staging of sleep, improve the accuracy of staging, provide personal help to individuals, provide doctors with convenient tools, and provide heuristic auxiliary guidance for precision medicine.

[0054] The following is an example of polysomnographic data collected from multiple individuals. The number of individuals is 26, all of whom are healthy people without sleep disorders. The data was collected by sampling at a rate of 256Hz to obtain polysomnographic data for each individual for one night and has been carefully revised. The average recording time for an individual is 30,765 seconds. Physiological signal records include different sleep stages (such as N1, N2, N3, R, W). The method of the present invention is used to implement sleep analysis based on spatiotemporal feature coding and multi-source fusion, which provides assistance for the identification of various stages of sleep. Specifically, the experimental data is annotated by professional sleep physicians, and sleep stages are divided into one frame every 30 seconds. The signals collected by the polysomnographic monitoring equipment include EEG signals, EOG signals, and EMG signals. The signal acquisition electrodes are installed at the corresponding sites in strict accordance with the "Manual of Interpretation of Sleep and Related Events of the American Academy of Sleep Medicine". Number of EEG signal channels C EEG =6, the electrode sites are F3-M2, F4-M1, C3-M2, C4-M1, O1-M2, O2-M1; the number of electrooculogram signal channels is C EOG =2, the electrode sites are E1-M2 and E2-M2 respectively; the number of myoelectric signal channels is C EMG = 1, and the electrode sites are Chin 1-Chin 2. When collecting signals, the polysomnography equipment used in the experiment filtered EEG and EOG signals using a 0.3Hz low-pass filter and a 35Hz high-pass filter, and processed EMG signals using a 10Hz low-pass filter and a 100Hz high-pass filter. All channels were then normalized to the range [0, 1].

[0055] Polysomnographic data contains a total of C = 9 channels. Each frame belongs to and only belongs to one sleep stage. This method divides each frame into N = 30 segments, each segment is 1 second long.

[0056] The specific implementation of identifying the individual's subsequent physiological state data based on the collected physiological state data of the individual using the method of the present invention includes the following steps:

[0057] 1) Constructing a graph space encoder for spatial features; including steps 1a) to 1b):

[0058] 1a) Build a multi-layer perceptron encoder for myoelectric and EOG signals:

[0059] Myoelectric and EO signals have only one or two channels, and their spatial information is far less than that of EEG signals, which have relatively more channels. For these signals, a multi-layer perceptron (MLP) consisting of linear layers and residual connections is used to initially encode them. The MLP encoders for myoelectric and EO signals are expressed as Equations 1 and 2, respectively:

[0060] H′ EMG =MLP(H EMG) (Formula 1)

[0061] H′ EOG =MLP(H EOG ) (Formula 2)

[0062] in, are the encoded myoelectric and oculoscopic signals, Before encoding, they are in a single fragment X p The myoelectric channel and eye electrocardiogram channel signals are selected from C. EMG and C EOG are the number of channels of EMG and EOG signals in the clip, respectively.

[0063] 1b) Building a sparse graph neural network for EEG signals:

[0064] EEG signals have relatively many channels and are strictly located on the head, so their multi-channel spatial connectivity information is significant. One-dimensional convolution is used to convert EEG signals into graph node embeddings. The conversion process is expressed as Equation 3:

[0065]

[0066] in, is the initial embedding of the graph nodes converted from EEG signals, For a single fragment X p The EEG channel signal selected from C EEG is the number of EEG signal channels in the segment.

[0067] Based on the graph nodes converted from EEG signals, a sparse connection graph is constructed, and the spatial connections between channels are explicitly learned using a multi-head attention mechanism. Compared with a fully connected graph, the sparse connection graph used in this method reduces the number of edges and thus reduces computational complexity. The steps for constructing a sparse connection graph are as follows:

[0068] Random regular graph: Randomly arrange the graph nodes converted from EEG signals, connect the points in the rearranged order, so that each point is connected to a constant number of other points, and construct a random regular graph.

[0069] Virtual connection graph: A constant number of virtual nodes are established. The virtual nodes have learnable node embeddings. The virtual nodes are connected to each real graph node to build a virtual connection graph.

[0070] The random regular graph and the virtual connection graph jointly construct sparse connections on the graph nodes. The multi-head attention mechanism is used to calculate the weights of the edges connecting the nodes. Taking a single edge as an example, the weight calculation is expressed as Equation 4:

[0071] e i：j =(W K h i) T W Q h j (Formula 4)

[0072] Where i, j = 1, 2, 3, ..., N, W K , represents the learnable parameters, represents the weight of the edge connecting node i and node j. It is H EEG The embedding of node i in . The superscript T is the transpose symbol.

[0073] Use the message propagation mechanism and update the node embedding of the graph as shown in Equation 5:

[0074]

[0075] Where σ is the activation function. b is the bias, is the set of nodes connected to node i. h′i is the updated embedding of node i. When propagating messages, the attention weight e of each edge connected to the node calculated by Equation 4 is applied. j：i All node embeddings converted from EEG signals together constitute the encoded EEG signal

[0076] 2) Construct a multi-signal source fusion module:

[0077] PSG data includes multiple signal sources such as EEG, EMG, and EOG. This method uses a multi-head attention mechanism to fuse multiple source signals on the channel dimension C. First, multiple signal sources encoded by the graph space encoder are concatenated, as shown in Equation 6:

[0078]

[0079] Among them, H′ EEG , H′ EOG , H′ EMG They are EEG, EOG and EMG signals that have passed through the image space encoder. After splicing them and transposing them, we get Then, multi-head attention is used to focus on the contributions of different signal sources. The attention formula is as follows:

[0080]

[0081] Where Q, K, and V correspond to the query, key, and value in the Transformer architecture, respectively. The softmax function maps attention to the interval [0, 1]. This method uses self-attention, where Q, K, and V in Equation 7 are identical and equal to H′. Subsequently, a linear layer is used on the channel dimension C to fuse the multiple signal sources. The output of the multi-signal source fusion module is shown in Equation 8:

[0082] x′ p =Linear(Attn(H′)) (Equation 8)

[0083] in, It is a single segment that has undergone graph space encoding and multi-source signal fusion.

[0084] 3) Construct a temporal Transformer encoder for temporal features

[0085] Physiological signals have diverse local waveforms, which constitute their local temporal features. The number and location of these waveforms constitute their global temporal features. Both local and global temporal features are important for interpreting and classifying physiological stages. The temporal Transformer encoder designed in this method considers both local and global temporal features of the signal. First, all segments that have undergone graph-space encoding and multi-source signal fusion are concatenated and fed into the temporal Transformer encoder, as shown in Equation 9.

[0086] z [0] =[x′ p1 , x′ p2 , x′ p3 ,...,x′ pN ] T (Equation 9)

[0087] is the input of the temporal Transformer encoder, which is composed of the segments encoded by 1) and 2). N is the total number of segments into which the polysomnography data is divided, x′ p The subscript numbers in the sequence denote the sequence of the segments. The temporal Transformer encoder uses the Transformer architecture to capture important local temporal features. The encoder consists of L Transformer layers. A single Transformer layer includes a multi-layer perceptron (MLP) consisting of multiple linear layers, layer normalization (LN), and multi-head attention (Attn). The formulas for a single Transformer layer are expressed in Equations 10 and 11.

[0088]

[0089] are the input and intermediate values ​​of the th and th Transformer layers, respectively. To preserve the global temporal features, the encoded local temporal features are residually connected with the initial global temporal features. Finally, the connected temporal features are flattened and passed through a linear layer to obtain the sleep stage result for the frame, as shown in Equation 12:

[0090]

[0091] is the automatic staging result of the frame obtained by this method, and its value corresponds to the confidence of the five sleep periods (N1, N2, N3, R, W), z[ L ] is the output of the last Transformer layer, are the flattened local and global time features.

[0092] 4) Sleep staging neural network model training and solution

[0093] The present invention uses categorical cross entropy (CCE) as the loss function and adopts stochastic gradient descent for training. There are S samples and M categories (in this method, M = 5, corresponding to 5 sleep periods). The categorical cross entropy is as shown in Equation 13:

[0094]

[0095] in, Represents the sth sample The confidence level corresponding to type m, y sm is the label of the sth sample. If the sample belongs to category m, the value is 1; if it does not belong to category m, the value is 0.

[0096] The present invention was compared with three leading deep learning methods in the field of sleep analysis: DeepSleepNet, AttnSleepNet, and CareSleepNet. The results are shown in Tables 1 and 2. Table 1 shows that the proposed method generally outperforms the three leading deep learning methods in terms of precision, recall, and F1-score across all sleep periods. On average, compared to the best performance of the other methods, the proposed method improved precision by 20.8%, recall by 13.3%, and F1-score by 2.4%, demonstrating the superiority of the present invention.

[0097] Table 1. Comparison of the present invention method with other methods in various sleep periods

[0098]

[0099] As can be seen from Table 2, the accuracy of the method of the present invention is improved by 9.2% compared with the best effect of other methods.

[0100] Table 2. Comparison of the method of the present invention with other methods in terms of overall indicators

[0101]

[0102] At the same time, ablation experiments were conducted on the three parts of the graph space encoder, multi-source fusion module and temporal Transformer encoder in the method proposed in the present invention, and the results are shown in Tables 3, 4 and 5. It can be seen from Table 3 that the graph space encoder for spatial features proposed in the present invention (i.e., step 1 above)) is meaningful. Compared with the graph neural network and one-dimensional convolutional neural network without attention mechanism, the accuracy of analyzing sleep periods using the method proposed in the present invention is improved by 4.26%, and Kappa is improved by 9.7%. It can be seen from Table 4 that the multi-source fusion module proposed in the present invention (i.e., step 2 above)) has better effects than other fusion methods. Compared with only linear layers or taking the average of all channels for multi-source fusion, the accuracy of the method proposed in the present invention is improved by 10.2%, and the Kappa value is improved by 14.5%. It can be seen from Table 5 that the temporal Transformer encoder for temporal features proposed in the present invention is effective. Compared with removing the temporal Transformer encoder, the accuracy of the method proposed in the present invention is improved by 13.2%, and Kappa is improved by 17.9%.

[0103] Table 3. Ablation experiment results of graph space encoder

[0104]

[0105] Table 4. Ablation experiment results of multi-source fusion module

[0106]

[0107] Table 5. Ablation experiment results of temporal Transformer encoder

[0108]

[0109] The disclosed embodiments are intended to facilitate a better understanding of the present invention. However, those skilled in the art will appreciate that various substitutions and modifications may be made without departing from the essence of the invention and the appended claims. Therefore, the present invention should not be limited solely to the contents presented in these embodiments. The scope of protection of the present invention should be determined in accordance with the scope defined in the claims.

Claims

1. A sleep staging method based on spatiotemporal feature coding and multi-source fusion, which uses collected individual polysomnography data to establish an automatic sleep staging neural network model based on spatiotemporal feature coding and multi-source fusion to achieve the staging and identification of the individual's subsequent sleep stages; characterized by: The sleep staging neural network model includes three modules: 1) a graph-space encoder, 2) a multi-signal source fusion module, and 3) a temporal Transformer encoder. The graph-space encoder is used to encode spatial interaction information in polysomnographic data. The multi-signal source fusion module fuses information from EEG, EMG, and EOG signals in a learnable manner. The temporal Transformer encoder captures local and global temporal features. Each frame of polysomnographic data is subjected to sleep staging, and C is set to represent the number of channels in the polysomnographic data, and L is set to represent the number of channels in the polysomnographic data. e Indicates the number of sampling points in a frame, X e Represents a frame of initial input, i.e. A frame of polysomnographic data is divided into multiple segments, L p Indicates the number of sampling points in a fragment, X p Represents a fragment, i.e. The input of the graph-space encoder and the multi-signal source fusion module is a single segment, while the temporal transformer encoder takes all segments that have undergone graph-space encoding and multi-signal source fusion as input. The method includes the following steps: (1) Construct a graph space encoder for spatial features and encode the spatial features of EEG, EOG and EMG respectively; (2) Construct a multi-signal source fusion module, which combines the EEG, EOG, and EMG signals encoded in step 1) and fuses them on the channel dimension C using a multi-head attention mechanism; (3) Construct a temporal Transformer encoder for temporal features, concatenate the fused segments and input them into an L-layer temporal Transformer encoder to combine local and global temporal features; (4) The sleep staging neural network model is trained using the classification cross entropy loss function to output the sleep staging results.

2. The sleep staging method based on spatiotemporal feature coding and multi-source fusion according to claim 1, characterized in that: The graph space encoder is constructed in step (1), specifically: 1a) Construct a multi-layer perceptron encoder for EMG and EOG. For EMG and EOG, a multi-layer perceptron (MLP) consisting of linear layers and residual connections is used for initial encoding, satisfying the following requirements: in, and are the encoded myoelectric and oculoscopic signals, and Before encoding, they are in a single fragment X p The myoelectric channel and eye electrooculogram channel signals selected from C EMG and C EOG are the number of channels of myoelectric signal and electrooculographic signal in the segment, L p Indicates the number of sampling points in a fragment; 1b) Construct a sparse graph neural network for EEG signals. Convert EEG into graph node embeddings through one-dimensional convolution. Based on the graph nodes converted from EEG signals, construct a sparse connected graph consisting of a random regular graph and a virtual connection graph. Use a multi-head attention mechanism to calculate the edge weights of the connected nodes, and use a message propagation mechanism to update the graph node embeddings. All node embeddings converted from EEG signals together constitute the encoded EEG signal C EEG is the number of EEG signal channels in the segment.

3. The sleep staging method based on spatiotemporal feature coding and multi-source fusion according to claim 2, characterized in that: In step 1b), the EEG signal is converted into the embedding of the graph node, and the conversion formula is: in, is the initial embedding of the graph nodes converted from EEG signals, For a single fragment X p The EEG channel signal selected from C EEG is the number of channels of EEG signals in the clip; The calculation formula for edge weight is: Where i, j = 1, 2, 3, ..., N, N is the total number of segments into which the polysomnographic data is divided, represents the learnable parameters, represents the weight of the edge connecting node i and node j, yes The embedding of node i; The formula for updating the node embedding of the graph is: Among them, σ is the activation function, b is the bias, is the set of nodes connected to node i, is the updated embedding of node i, and all node embeddings converted from EEG signals together constitute the encoded EEG signal C EEG is the number of EEG signal channels in the segment.

4. The sleep staging method based on spatiotemporal feature coding and multi-source fusion according to claim 2, characterized in that: The construction of the sparse connection graph specifically includes: Random regular graph: Randomly arrange the graph nodes converted from EEG signals, connect the nodes in the rearranged order, so that each node is connected to a constant number of other nodes, and construct a random regular graph; Virtual connection graph: A constant number of virtual nodes are established. The virtual nodes have learnable node embeddings. The virtual nodes are connected to each real graph node to build a virtual connection graph.

5. The sleep staging method based on spatiotemporal feature coding and multi-source fusion according to claim 1, characterized in that: The multi-signal fusion in step (2) specifically includes: The EEG, EOG, and EMG signals encoded by the image space encoder are concatenated and transposed to obtain Calculated through multi-head self-attention mechanism Among them, Q, K, and V correspond to the query, key, and value in the Transformer architecture respectively. The softmax function maps attention to the interval [0, 1]. The values ​​of Q, K, and V are the same and equal to H′; And use linear layer fusion on channel dimension C to get x′ p =Linear(Attn(H′)), It is a single segment that has undergone graph space encoding and multi-source signal fusion.

6. The sleep staging method based on spatiotemporal feature coding and multi-source fusion according to claim 1, characterized in that: The construction of the temporal Transformer encoder in step (3) specifically includes: Splice all segments that have undergone graph space encoding and multi-source signal fusion z [0] =[x′ p1 , x′ p2 , x′ p3 ,...,x′ pN ] T , is the input to the temporal Transformer encoder, N is the total number of segments the polysomnography data is divided into, and x′ p The subscript numbers are the sequence numbers of the fragments; The temporal Transformer encoder consists of L layers of Transformer layers. A single Transformer layer consists of a multi-layer perceptron (MLP) consisting of multiple linear layers connected together, layer normalization (LN), and multi-head attention (Attn). The formula is: and are the input and intermediate values ​​of the,th Transformer layer respectively; The encoded local time features are residually connected with the initial global time features, the connected time features are flattened and passed through a linear layer to obtain the sleep stage result of the frame. The formula is: z [L] is the output of the last Transformer layer, The values ​​correspond to the confidence levels of the five sleep periods N1, N2, N3, R, and W respectively.

7. The method according to claim 1, wherein In step (4), the sleep staging neural network model training uses classification cross entropy as the loss function and adopts the stochastic gradient descent method for training. The classification cross entropy is defined as: Among them, S is the number of samples, M is the number of types, M=5, corresponding to 5 sleep periods, Represents the sth sample The confidence level corresponding to type m, y sm is the label of the sth sample.

Images