An electroencephalogram sleep staging method and device based on spatio-temporal relationship learning and a medium

Abstract

This invention discloses an EEG sleep staging method, device, and medium based on spatiotemporal relationship learning. The method includes: acquiring EEG data; constructing context sequences of adjacent sleep periods based on the EEG data; inputting the constructed sequences into a sleep staging model and outputting staging results. The sleep staging model operates by: generating a spatiotemporal graph by concatenating instances of individual sleep periods with an adaptive brain functional connectivity matrix and spatiotemporal transition matrices of adjacent sleep periods; each adjacent sleep period is directly concatenated with a sample of the sleep period to be tested, and input together with the corresponding spatiotemporal graph into a gated spatiotemporal graph convolutional module to learn the spatiotemporal dependencies between brain channels in adjacent sleep periods, obtaining features for staging. This invention, by combining the construction of the spatiotemporal graph with the design of a gated spatiotemporal graph convolutional network, captures the global spatiotemporal transition relationships of sleep context sequences, demonstrating good reliability and accuracy in EEG sleep staging tasks.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing technology, and in particular to an EEG sleep staging method, device, and medium based on spatiotemporal relationship learning. Background Technology

[0002] Sleep staging refers to dividing a full night's multimodal physiological signal recording (PSG) into 30-second segments, assigning a sleep stage label to each 30-second sleep period, thereby dividing the entire sleep process into different stages. For example, the human sleep process can be divided into five stages: the wakefulness stage (W stage), the three sub-stages N1, N2, and N3 derived from the NREM stage, and the REM stage. By monitoring and analyzing sleep staging, doctors can help determine whether a patient has a sleep disorder and take appropriate treatment measures to promote good sleep and health.

[0003] Human sleep stage classification by experts is subject to the influence of subjective knowledge and variable factors, and is time-consuming and inefficient. To overcome these limitations, increasing research is focused on developing automated sleep staging technologies. Some researchers concentrate on traditional machine learning methods based on time-domain, frequency-domain, and time-frequency-domain features, such as using Support Vector Machines (SVM) and Naive Bayes (NB) to handle frequency features like differential entropy and power spectrum characteristics. However, the classification accuracy of these methods largely depends on feature engineering and feature selection, requiring significant expertise. In recent years, deep learning methods, with their powerful representation learning capabilities, have been widely applied to automated sleep stage classification. For example, Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are commonly used to learn appropriate feature representations from transformed data or directly from raw data, achieving quite good sleep staging performance. Recently, a growing body of research on automated sleep staging has focused on using spatiotemporal graph convolutional networks to explore spatiotemporal dependencies in the sleep process. These methods typically employ graph neural networks (GNNs) to model the topological relationships between different brain channels, and then use spatiotemporal convolutional networks (TCNs) to capture the spatiotemporal transition relationships between adjacent sleep stages. This approach comprehensively explores the connectivity between brain channels and utilizes sleep transition rules from the sleep stage domain, achieving optimal sleep stage classification and recognition rates.

[0004] Existing sleep staging algorithms for capturing the spatiotemporal relationships of sleep stage sequences have limitations in modeling spatial relationships between brain channels and between adjacent sleep stage sequences. Regarding spatial relationship modeling between brain channels, most methods neglect the potential functional connections between physically distant channels and the individual variability in brain channel connectivity. This makes revealing the subtle neural mechanisms between brain channels during sleep a task that still requires improvement. As for spatiotemporal modeling between adjacent sleep stage sequences, many methods tend to use TCNs and GCNs to capture spatiotemporal dependencies. However, this asynchronous capture method may result in insufficient spatiotemporal relationship capture capabilities, leading to limitations in mining the spatiotemporal transition relationships between adjacent sleep stages. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to provide a method, device and medium for EEG sleep staging based on spatiotemporal relationship learning.

[0006] The first technical solution adopted in this invention is:

[0007] A brainwave sleep staging method based on spatiotemporal relationship learning includes the following steps:

[0008] Acquire EEG data;

[0009] Constructing context sequences of adjacent sleep periods based on EEG data;

[0010] The obtained sequence is input into the sleep staging model, and the staging results are output.

[0011] The sleep staging model works as follows: a spatiotemporal map is generated by concatenating an adaptive brain functional connectivity matrix of individual sleep stages with the spatiotemporal transition matrix of adjacent sleep stages; each adjacent sleep stage is directly concatenated with a sample of the sleep stage to be tested, and then correlated with the corresponding spatiotemporal map. Figure 1 In the same input-gated spatiotemporal graph convolution module, the spatiotemporal dependencies between brain channels in adjacent sleep periods are learned to obtain features for staging.

[0012] Furthermore, the acquisition of EEG data includes:

[0013] Raw EEG data is acquired, and temporal dimensionality reduction processing is performed on the acquired EEG data.

[0014] Furthermore, the construction of sleep-adjacent period context sequences based on EEG data includes:

[0015] The time series sample of the sleep period to be tested, after dimensionality reduction, together with the k preceding and k following sleep period samples adjacent to the sample, are used as the time series and input to the sleep staging model.

[0016] In this context, the label for each time series remains the label of the original sleep period sample to be tested; that is, if the original sleep period sample to be tested is x... t The current model input is sequence X. t =(x t-kT ,...,x t ,...,x t+kT ).

[0017] Furthermore, the spatiotemporal graph includes an instance-adaptive brain functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix for adjacent sleep periods;

[0018] The instance adaptive brain functional connectivity matrix of a single sleep period is multiplied with the sample to achieve instance adaptive generation;

[0019] The spatiotemporal transition matrix between adjacent sleep periods is constructed using a multi-graph fusion mechanism, which integrates the brain functional connections between adjacent sleep periods and the sleep period to be tested, to obtain the spatiotemporal transition matrix between the two periods.

[0020] Furthermore, for each sleep period sample x t Its instance adaptive brain functional connectivity matrix is:

[0021]

[0022] X t =(x t-kT ,...,x t ,...,x t+kT )

[0023] In the formula, W σ and W τ There are two learnable parameter matrices; T is the transpose; k is the preset number of sleep period samples;

[0024] The brain functional connectivity matrix for each sleep period is placed on the diagonal of the spatiotemporal map;

[0025] For any adjacent sleep period samples and the sleep period to be tested, their spatiotemporal transition matrix is ​​established and placed in the upper right corner of the spatiotemporal diagram. This represents the dynamic changes and mutual influences of brain channels during sleep and can characterize the spatiotemporal evolution relationship of brain channels in different sleep periods.

[0026] Furthermore, a multi-graph fusion mechanism based on spatial attention and graph attention is employed to integrate the brain functional connectivity matrix A of adjacent sleep time t-kd with the sleep time t to be measured. t-dand A t The corresponding spatiotemporal transition matrix is ​​obtained.

[0027] In the multi-graph fusion mechanism, the spatiotemporal embedding of multiple graphs and the brain functional connectivity matrices from two time periods are jointly input into the spatial attention mechanism and the graph attention mechanism. The spatial attention mechanism is used to calculate the influence of other nodes in each graph on a given node, while the graph attention mechanism is used to calculate the autocorrelation of the same node in different graphs. Finally, a set of weights is output, and matrix A is adjusted based on this set of weights. t-d and A t The weighted summation of different nodes in the matrix yields the corresponding spatiotemporal transition matrix.

[0028] Furthermore, the gated spatiotemporal graph convolution module includes skip-connected gated graph convolution and gated temporal convolution;

[0029] Each adjacent sleep period is directly connected to the sample of the sleep period to be tested to form Z, and is input into the jump-gated graph convolutional network along with the corresponding spatiotemporal graph G. The jump-gated graph convolutional network makes full use of the information of different graph convolutional layers to improve the performance of the graph convolutional network, while capturing the temporal and spatial relationships of adjacent sleep periods to extract richer spatiotemporal features.

[0030] Gated temporal convolution is used to integrate and further extract spatiotemporal information from multiple time points to facilitate the learning of spatiotemporal dependencies between brain channels in adjacent sleep period sequences.

[0031] Furthermore, the sleep staging model also includes a classifier for classifying based on the spatiotemporal features output by the gated spatiotemporal graph convolution module, in order to determine the sleep stage to which each sleep period to be tested belongs.

[0032] The second technical solution adopted in this invention is:

[0033] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement an EEG sleep staging method based on spatiotemporal relationship learning as described above.

[0034] The third technical solution adopted in this invention is:

[0035] A computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement an EEG sleep staging method based on spatiotemporal relationship learning as described above.

[0036] The fourth technical solution adopted in this invention is:

[0037] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned brainwave sleep staging method based on spatiotemporal relationship learning.

[0038] The beneficial effects of this invention are: by combining the construction of a spatiotemporal graph with the design of a gated spatiotemporal graph convolutional network, this invention captures the global spatiotemporal transition relationship of sleep context sequences, and has good reliability and accuracy in EEG sleep staging tasks. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of an EEG sleep staging method based on spatiotemporal relationship learning in an embodiment of the present invention;

[0041] Figure 2 This is an overall architecture diagram of the sleep staging model in this embodiment of the invention;

[0042] Figure 3 This is a schematic diagram of the multi-image fusion mechanism in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram illustrating the construction of the spatiotemporal graph in an embodiment of the present invention;

[0044] Figure 5 This is a structural diagram of the gated spatiotemporal graph convolution module in an embodiment of the present invention. Detailed Implementation

[0045] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0046] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0047] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0048] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0049] Technical explanation:

[0050] Sleep staging: Sleep is generally divided into two main categories: rapid eye movement (REM) and non-rapid eye movement (NREM). According to different international standards, such as AASM and R&K, sleep stages can be further subdivided. AASM is the more widely used international standard in the field of automated sleep staging, dividing the human sleep process into five stages: wakefulness (W stage), three sub-stages (N1, N2, and N3) derived from NREM, and REM sleep. Research on sleep staging can also help us better understand the physiological mechanisms of human sleep, providing a more scientific basis for sleep medicine and health management.

[0051] PSG recordings include physiological indicators such as electroencephalogram (EEG), eye movement (EOG), and electrical muscle activity (EMG).

[0052] Sleep transition rules: A sleep cycle refers to a complete cycle of different sleep stages experienced by humans during sleep. Typically, a sleep cycle lasts approximately 90 to 120 minutes and includes two main stages: rapid eye movement (REM) sleep and non-rapid eye movement (NREM) sleep. In the field of sleep staging, it is believed that the human sleep process follows sleep transition rules. Sleep transition rules refer to the patterns of switching between different sleep stages. Generally, a complete sleep cycle involves multiple alternations between REM and NREM stages. Taking the AASM sleep process as an example, at the beginning of each sleep cycle, people experience three NREM stages (N1, N2, and N3) before entering the first REM stage. Subsequently, REM and NREM stages repeat alternately until wakefulness. This means that in most cases, the labels of adjacent sleep stages in the human body do not undergo highly irregular abrupt changes. Therefore, current sleep staging techniques typically utilize the sleep period to be measured, along with several preceding and following sleep periods, to form a contextual sequence for processing and analysis, thereby improving the accuracy and reliability of sleep staging tasks.

[0053] To address the existing technical problems, this invention proposes an EEG sleep staging scheme based on spatiotemporal relationship learning. It combines the construction of a spatiotemporal graph with the design of a gated spatiotemporal graph convolutional network to capture the global spatiotemporal relationships of sleep context sequences. First, each adjacent sleep stage is directly concatenated with the sample of the sleep stage to be tested. By concatenating the brain functional connectivity map of a single sleep stage and the spatiotemporal transition matrix between adjacent stages, an instance-adaptive spatiotemporal graph is generated. By simultaneously modeling spatiotemporal relationships using graphs, the ability to learn the spatiotemporal transition relationships of different individuals during sleep is improved. Second, the gated spatiotemporal graph convolutional module integrates the spatiotemporal features between multiple sets of adjacent sleep stages to further learn the global spatiotemporal dependencies of adjacent sleep time series, achieving considerable performance on the EEG sleep staging task.

[0054] Example 1

[0055] like Figure 1 As shown, this embodiment provides an EEG sleep staging method based on spatiotemporal relationship learning. It is mainly implemented through the construction of a spatiotemporal map and a gated graph convolutional network. This method can effectively capture the global spatiotemporal transition relationships of different individual brain channels during sleep, exhibiting good reliability and accuracy in EEG sleep staging tasks. The method specifically includes the following steps:

[0056] S1. Acquire EEG data and perform temporal dimensionality reduction on the EEG data.

[0057] In some embodiments, the raw EEG data has excessively high dimensionality for each EEG channel. A pre-trained deep CNN network is used to extract channel-independent temporal features, reducing information redundancy and improving the efficiency of subsequent model training. The raw EEG sleep data records 30 seconds of sleep data per sample at a high sampling rate, resulting in excessively high data dimensionality, which is detrimental to model training and deep feature extraction.

[0058] It should be noted that the above time series dimensionality reduction uses a pre-trained deep CNN network to extract more discriminative channel-independent time series features, but it is not limited to this. Other algorithms such as Convolutional Hidden Markov Model (HMM) and Recurrent Neural Network (RNN) can also be used, as well as statistical features related to time series learning, such as maximum value and average value.

[0059] S2. Construct a context sequence of adjacent sleep periods based on the obtained EEG data.

[0060] As one implementation method, the time-series sample of the sleep period to be tested, after dimensionality reduction, along with the k preceding and k following sleep period samples, is used as a time series and input into the network. The label of each time series remains the label of the original sleep period sample to be tested. That is, if the original sleep period sample to be tested is x... t The current model input is sequence X. t =(x t-kT ,...,x t ,...,x t+kT ).

[0061] S3. Construct a spatiotemporal graph based on the obtained sequence.

[0062] Specifically, the spatiotemporal graph is mainly composed of two block matrices: an instance-adaptive brain pathway functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix between adjacent sleep periods. The brain functional connectivity matrix for each sleep period is located on the diagonal of the spatiotemporal graph, and instance adaptation is achieved by multiplying the adaptively learned matrix with each sample. The spatiotemporal transition matrix between adjacent sleep periods is placed in the upper right corner of the spatiotemporal graph, and is formed by integrating the brain functional connectivity between adjacent sleep periods and the sleep period under test through a graph fusion mechanism.

[0063] S4. Learn spatiotemporal dependencies using gated spatiotemporal graph convolutional networks.

[0064] Specifically, taking time t-kd as an example, adjacent sleep periods x t-kd Compared with the sample of sleep period to be tested x t Directly connected as Z t-kd , and the corresponding spacetime diagram A t-kd A gated spatiotemporal graph convolutional network is used to learn the spatiotemporal dependencies between brain channels in adjacent sleep period sequences.

[0065] S5. Use a classifier to classify spatiotemporal features.

[0066] As an optional implementation, the spatiotemporal features obtained in step S4 are input into a multilayer perceptron for dimensionality reduction. Finally, the softmax function is used to map the processed features to five different sleep stages and output the final sleep stage results, thus completing the five-class classification task of sleep samples under the AASM standard.

[0067] join Figure 2 This embodiment primarily utilizes a sleep staging model, which comprises two main modules: a spatiotemporal graph construction module and a gated spatiotemporal graph convolution module. The adaptive spatiotemporal graph construction module consists of two parts: an instance-adaptive brain pathway functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix for adjacent sleep periods, enabling simultaneous modeling of the spatiotemporal relationships of the sleep period context sequence. The brain pathway functional connectivity of a single sleep period is multiplied with a weight matrix to achieve adaptive instance generation. The construction of the spatiotemporal transition matrix for adjacent sleep periods employs a multi-graph fusion mechanism, integrating the brain functional connectivity between adjacent sleep periods and the sleep period under test to obtain the spatiotemporal transition matrix between the two periods. Each adjacent sleep period is directly concatenated with a sample from the sleep period under test and correlated with the corresponding spatiotemporal... Figure 1 In the same input-gated spatiotemporal graph convolution module, the goal is to further learn the spatiotemporal dependencies between brain channels in adjacent sleep periods.

[0068] The above content will be explained in detail below with reference to the accompanying drawings and specific embodiments.

[0069] (1) Spatiotemporal graph construction module

[0070] The spatiotemporal graph is mainly composed of two types of block matrices: an instance-adaptive brain pathway functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix between adjacent sleep periods.

[0071] a) Instance-adaptive brain functional connectivity matrix

[0072] For electroencephalogram (EEG) signals, the responses of different electrodes and their functional dependencies play a particularly important role in the cognitive generation of EEG-related tasks. However, due to the hidden nature of neural mechanisms, it is very difficult to predefine the functional relationships between different EEG regions. Therefore, this patent employs an instance-based, data-driven approach to adaptively learn the connectivity patterns between brain channels in different samples, aiming to reveal individual-specific and potential brain functional connectivity patterns. For each sleep period sample x t Its instance adaptive brain functional connectivity matrix is:

[0073]

[0074] Xt =(x t-kT ,...,x t ,...,x t+kT )

[0075] Among them, W σ and W τ There are two learnable parameter matrices. The brain functional connectivity matrix for each sleep period is placed on the diagonal of the spatiotemporal graph.

[0076] b) Spatiotemporal transition matrix between adjacent sleep periods

[0077] During human sleep, different brain pathways exhibit various brain activity patterns and connectivity characteristics. These pathways interact not only spatially within the same sleep period but also temporally between different sleep periods. To describe this dynamic influence, a spatiotemporal transition matrix between different sleep periods can be established. For any adjacent sleep period sample and the sleep period to be tested, taking times t-kd and t as examples, their spatiotemporal transition matrix is ​​established. Placed in the upper right corner of the spatiotemporal diagram, it represents the dynamic changes and mutual influences of brain channels during sleep, and can characterize the spatiotemporal evolution of brain channels in different sleep stages.

[0078] See Figure 3 Taking time points t-kd and t as examples, in this embodiment, a multi-graph fusion mechanism based on spatial attention and graph attention is used to integrate the brain functional connectivity matrix A of adjacent sleep time points t-kd and the sleep time t to be tested. t-d and A t The corresponding spatiotemporal transition matrix is ​​obtained. The multi-graph fusion mechanism first consists of matrix A t-d and A t Spatial embedding (SG) and graph embedding (GE) are computed and summed to obtain multi-graph spatial embedding (SGE). In the multi-graph fusion mechanism, the multi-graph spatiotemporal embedding and the brain functional connectivity matrices from two time periods are jointly input into spatial attention and temporal attention mechanisms. The spatial attention mechanism is used to calculate the influence of other nodes in each graph on a given node, while the graph attention mechanism is used to calculate the autocorrelation of the same node in different graphs. Both attention mechanisms are implemented through a multi-head attention mechanism. The multi-graph fusion mechanism ultimately outputs a set of weights, which are used to apply to A. t-d and A t The weighted summation of different nodes in the matrix yields the corresponding spatiotemporal transition matrix. Such a spatiotemporal transition matrix can better characterize the dynamic changes and interactions of brain channels during sleep.

[0079] c) Construction of the spacetime map

[0080] See Figure 4 Using the brain functional connectivity matrix and spatiotemporal transition matrix of each sleep stage as block matrices, the corresponding spatiotemporal graph can be obtained by splicing them together. Taking time points t-kd and t as examples, the brain functional connectivity matrix A t-d and A t The spacetime transition matrix is ​​placed diagonally (matrices from earlier moments are placed in the top left corner). By placing the matrix in the upper right corner and setting the lower left corner to 0, the spatiotemporal graph G corresponding to each time point can be obtained. t-kd .

[0081] (2) Gated spatiotemporal graph convolution module

[0082] Gated graph convolution is used to integrate and capture the spatiotemporal relationships between consecutive sleep stages, aiming to extract spatiotemporal features more relevant to sleep staging tasks. Taking times t-kd and t as an example, each adjacent sleep stage x... t-kd Compared with the sample of sleep period to be tested x t Directly connected as Z t-kd , and the corresponding spacetime diagram A t-kd By using a co-input gated spatiotemporal graph convolutional network, the spatiotemporal dependencies between brain channels in adjacent sleep phase sequences are learned, and spatiotemporal features that are more discriminative for sleep stage tasks are extracted.

[0083] See Figure 5 The gated spatiotemporal graph convolution module consists of skip-connected gated graph convolution and gated temporal convolution. The module uses the GLU function as the activation function, which has a larger number of parameters than conventional activation functions (such as ReLU and sigmoid), thus improving model performance. Furthermore, in the module, the Z-coordinate formed by directly connecting each adjacent sleep period to the sample of the sleep period to be tested is input to the skip-connected gated graph convolution network along with the corresponding spatiotemporal graph G. This skip-connected gated graph convolution network fully utilizes information from different graph convolutional layers, improving the performance of the graph convolutional network while capturing the temporal and spatial relationships between adjacent sleep periods, extracting richer spatiotemporal features. The gated temporal convolution network integrates and further extracts spatiotemporal information from multiple time points to facilitate the learning of spatiotemporal dependencies between brain channels in adjacent sleep period sequences.

[0084] (3) Classifier

[0085] The classifier mainly consists of a multilayer perceptron, whose task is to perform five-class classification on the previously extracted spatiotemporal features to determine the sleep stage to which each sleep period to be tested belongs. By connecting a softmax function after the multilayer perceptron, we can obtain the sleep stage judgment for the EEG sleep sample to be tested.

[0086] In summary, the method of the present invention has at least the following advantages and beneficial effects compared with the prior art:

[0087] (1) This invention proposes a method for EEG sleep staging based on spatiotemporal relationship learning. This method is achieved by constructing a spatiotemporal graph and designing a gated spatiotemporal graph convolutional network. It can effectively capture the spatiotemporal transition relationships of brain channels during sleep and exhibits good reliability and accuracy in EEG sleep staging tasks. In the spatiotemporal graph construction module, the spatiotemporal relationships between adjacent sleep periods and the sleep period to be tested are modeled synchronously using a concatenated block matrix approach. The gated graph convolutional network module is used to learn the direct spatiotemporal influence of each adjacent sleep period on the sleep period to be tested, capturing the global spatiotemporal dependencies of the sleep period context.

[0088] This invention constructs a spatiotemporal graph by splicing brain functional connectivity matrices from different periods and spatiotemporal transition matrices between adjacent sleep periods, thus proposing a more detailed spatiotemporal modeling method. It aims to provide the possibility of synchronously capturing spatiotemporal relationships. Furthermore, through a subsequent gated spatiotemporal graph convolution module, it further enhances the model's ability to learn spatiotemporal dependencies, resulting in higher sleep stage recognition capabilities in sleep stage segmentation tasks.

[0089] (2) This invention proposes a method for constructing spatiotemporal graphs to model the spatiotemporal relationships between adjacent sleep context sequences. This method uses a block matrix by concatenating the brain functional connectivity matrix of a single sleep period and the spatiotemporal transition matrix between adjacent sleep periods, revealing the direct spatiotemporal influence of each adjacent sleep period on the sleep period under test. The brain functional connectivity matrix is ​​generated using an instance-adaptive approach, multiplying a learnable parameter matrix with samples from each sleep period. Furthermore, the generation of the spatiotemporal transition matrix employs a multi-graph fusion mechanism based on spatial attention and graph attention, integrating brain functional connectivity from different periods.

[0090] This invention employs spatiotemporal graphs to model the contextual sequences of sleep periods, representing the close connection between brain topological spatial patterns and temporal patterns in adjacent sleep stages in a graphical manner. The spatiotemporal graph is mainly composed of two parts pieced together as block matrices: an instance-adaptive brain pathway functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix for adjacent sleep periods. The instance-adaptive brain functional network matrix can represent the brain pathway activity patterns of different individuals in different sleep periods. The generation of the spatiotemporal transition matrix utilizes a multi-graph fusion mechanism based on spatial attention and graph attention to integrate the brain functional connectivity between each adjacent sleep period and the sleep period under test, representing the interaction and dynamic changes between brain pathways in different sleep periods.

[0091] (3) This invention proposes a gated spatiotemporal graph convolutional network. The gated spatiotemporal graph convolutional module includes a jump-connected gated graph convolutional network and a gated temporal convolutional network. Each adjacent sleep period is directly connected to the sleep period sample to be tested, and together with the spatiotemporal graph at the corresponding time, it serves as the input to the gated spatiotemporal graph convolutional network, fully learning the spatiotemporal transition relationships between brain channels in adjacent sleep period sequences. The gated temporal convolutional network integrates and further extracts the spatiotemporal transition information between multiple adjacent sleep periods, aiming to learn the global spatiotemporal dependencies between brain channels in adjacent sleep period sequences.

[0092] Compared with other spatiotemporal graph convolutional networks, the gated spatiotemporal graph convolutional module of this invention has the following advantages. First, this module uses the GLU function as the activation function, which has twice the number of parameters compared to other conventional activation functions. By combining gated linear units and nonlinear functions, this module improves the spatiotemporal feature learning ability. Second, residual connections are added between stacked graph convolutional layers. Since the number of nodes in the spatiotemporal graph is relatively small, simply stacking multiple graph convolutional layers can easily lead to oversmoothing problems. However, the residual design of the graph convolutional layers alleviates this limitation by fully utilizing the neighbor information of graph nodes at different orders. This residual design improves the model's spatiotemporal dependency learning ability. Therefore, the gated spatiotemporal graph convolutional module of this invention improves the model's spatiotemporal dependency learning ability on EEG sleep staging tasks by improving the model network design, and aims to achieve better performance.

[0093] Example 2

[0094] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to achieve the following: Figure 1 This illustrates an EEG sleep staging method based on spatiotemporal relationship learning.

[0095] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0096] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory, and accessing data stored in memory to perform various server functions and process data. Optionally, the processor may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor may integrate one or more of the following: Central Processing Unit (CPU) and Modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0097] Since this electronic device is an electronic device corresponding to the EEG sleep staging method based on spatiotemporal relationship learning in this embodiment of the invention, and the principle of solving the problem by this electronic device is similar to that of this method, the implementation of this electronic device can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0098] Example 3

[0099] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to achieve the following: Figure 1 This illustrates an EEG sleep staging method based on spatiotemporal relationship learning.

[0100] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0101] Since this storage medium is the storage medium corresponding to the EEG sleep staging method based on spatiotemporal relationship learning in the embodiments of the present invention, and the principle of the storage medium in solving the problem is similar to that of the method, the implementation of this storage medium can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0102] Example 4

[0103] In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of a spatiotemporal relationship-based EEG sleep staging method according to various exemplary embodiments of the present application described above. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0104] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0105] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0106] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for EEG sleep staging based on spatiotemporal relationship learning, characterized in that, Includes the following steps: Acquire EEG data; Constructing context sequences of adjacent sleep periods based on EEG data; The obtained sequence is input into the sleep staging model, and the staging results are output. The sleep staging model works as follows: by splicing the instances of a single sleep period into an adaptive brain functional connectivity matrix and the spatiotemporal transition matrix of adjacent sleep periods, a spatiotemporal graph is generated; each adjacent sleep period is directly spliced ​​with the sample of the sleep period to be tested, and input together with the corresponding spatiotemporal graph into a gated spatiotemporal graph convolution module to learn the spatiotemporal dependencies between brain channels of adjacent sleep periods and obtain features for staging. For each sleep period, sample x t Its instance adaptive brain functional connectivity matrix is: X t =(x t-d ,...,x t ,...,x t+d ) In the formula, W σ and W τ There are two learnable parameter matrices; The brain functional connectivity matrix for each sleep period is placed on the diagonal of the spatiotemporal map; For any adjacent sleep period samples and the sleep period to be tested, their spatiotemporal transition matrix is ​​established and placed in the upper right corner of the spatiotemporal diagram. This represents the dynamic changes and mutual influences of brain channels during sleep, and can characterize the spatiotemporal evolution relationship of brain channels in different sleep periods. A multi-graph fusion mechanism based on spatial attention and graph attention is employed to integrate the brain functional connectivity matrix A between adjacent sleep times t±i and the sleep time to be measured t. t±i and A t The spatiotemporal transition matrix corresponding to time t±i is obtained. In the multi-graph fusion mechanism, the spatiotemporal embedding of multiple graphs and the brain functional connectivity matrices from two time periods are jointly input into the spatial attention mechanism and the graph attention mechanism. The spatial attention mechanism is used to calculate the influence of other nodes in each graph on a given node, while the graph attention mechanism is used to calculate the autocorrelation of the same node in different graphs. Finally, a set of weights is output, and matrix A is adjusted based on this set of weights. t±i and A t Weighted summation is performed on different nodes to obtain the corresponding spatiotemporal transition matrix.

2. The EEG sleep staging method based on spatiotemporal relationship learning according to claim 1, characterized in that, The acquisition of EEG data includes: Raw EEG data is acquired, and temporal dimensionality reduction processing is performed on the acquired EEG data.

3. The EEG sleep staging method based on spatiotemporal relationship learning according to claim 1, characterized in that, The construction of sleep-adjacent period context sequences based on EEG data includes: The time series sample of the sleep period to be tested, after dimensionality reduction, together with the k preceding and k following sleep period samples adjacent to the sample, are used as the time series and input to the sleep staging model. In this context, the label for each time series remains the label of the original sleep period sample to be tested; that is, if the original sleep period sample to be tested is x... t The current model input is sequence X. t =(x t-d ,...,x t ,...,x t+d ).

4. The EEG sleep staging method based on spatiotemporal relationship learning according to claim 1, characterized in that, The spatiotemporal graph includes an instance-adaptive brain functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix for adjacent sleep periods. The instance adaptive brain functional connectivity matrix of a single sleep period is multiplied with the sample to achieve instance adaptive generation; The spatiotemporal transition matrix between adjacent sleep periods is constructed using a multi-graph fusion mechanism, which integrates the brain functional connections between adjacent sleep periods and the sleep period to be tested, to obtain the spatiotemporal transition matrix between the two periods.

5. The EEG sleep staging method based on spatiotemporal relationship learning according to claim 1, characterized in that, The gated spatiotemporal graph convolution module includes jump-connected gated graph convolution and gated temporal convolution; Each adjacent sleep period is directly connected to the sample of the sleep period to be tested to form Z, and is input into the jump-gated graph convolutional network along with the corresponding spatiotemporal graph G. The jump-gated graph convolutional network makes full use of the information of different graph convolutional layers to improve the performance of the graph convolutional network, while capturing the temporal and spatial relationships of adjacent sleep periods to extract richer spatiotemporal features. Gated temporal convolution is used to integrate and further extract spatiotemporal information from multiple time points to facilitate the learning of spatiotemporal dependencies between brain channels in adjacent sleep period sequences.

6. The EEG sleep staging method based on spatiotemporal relationship learning according to claim 1, characterized in that, The sleep staging model also includes a classifier, which is used to classify based on the spatiotemporal features output by the gated spatiotemporal graph convolution module to determine the sleep stage to which each sleep period to be tested belongs.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method as described in any one of claims 1 to 6.

Images