Electroencephalogram sleep staging method and device based on time-space relation learning and medium

Abstract

The invention discloses an electroencephalogram sleep staging method and device based on time-space relation learning and a medium. The method comprises the following steps: acquiring electroencephalogram data; constructing a context sequence of adjacent sleep periods according to the electroencephalogram data; inputting the sequence obtained by construction into a sleep staging model, and outputting a staging result; wherein the working mode of the sleep staging model is as follows: generating a space-time diagram by splicing an instance adaptive brain function connection matrix of a single sleep period and a space-time transition matrix of an adjacent sleep period; each adjacent sleep period is directly spliced with a to-be-tested sleep period sample, the adjacent sleep periods and the corresponding time-space diagrams are input into a gated time-space diagram convolution module together, the time-space dependency relationship between brain channels of the adjacent sleep periods is learned, and characteristics used for staging are obtained. According to the method, the global space-time transition relation of the sleep context sequence is captured by combining the construction of the space-time diagram and the design of the gated space-time diagram convolutional network, and the method has relatively good reliability and accuracy in electroencephalogram sleep staging tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to an EEG sleep staging method, device and medium based on spatiotemporal relationship learning. Background Art

[0002] Sleep staging refers to dividing the whole night's multimodal physiological signal recording polysomnography (PSG) into 30-second units, and assigning a sleep stage label to each 30-second sleep period of the subject, so as to achieve the purpose of dividing the entire sleep process into different stages. For example, the human sleep process is divided into five stages: the wakefulness period (W period), the three stages of N1, N2 and N3 obtained by subdividing the NREM period, and the REM period. By monitoring and analyzing sleep staging, it can help doctors determine whether patients have sleep disorders and take appropriate treatment measures to promote good sleep and health.

[0003] The manual classification of sleep stages by human sleep experts is affected by the subjective knowledge and variable factors of sleep individuals, and the process is time-consuming and inefficient. In order to overcome these limitations, more and more research is devoted to the development of automatic sleep staging technology. Some researchers focus on using traditional machine learning methods based on time domain, frequency domain, and time-frequency domain features, such as using machine learning methods such as support vector machine (SVM) and naive Bayes (NB) to process frequency features such as differential entropy and power spectrum features. However, the classification accuracy of these methods depends largely on feature engineering and feature selection, which requires a lot of expertise. In recent years, deep learning methods have been widely used to automatically classify sleep stages due to their powerful representation learning capabilities. For example, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are often used to learn appropriate feature representations from transformed data or directly from raw data, and these methods can achieve quite good sleep staging performance. Recently, more and more automatic sleep staging research is devoted to using spatiotemporal graph convolutional networks to explore spatiotemporal dependencies during sleep. These methods usually use graph neural networks (GNNs) to model the topological relationship between brain channels, and then use spatiotemporal convolutional networks (TCNs) to capture the spatiotemporal transition relationship between adjacent sleep stages. This method fully explores the connection relationship between brain channels and utilizes the sleep transition rules in the field of sleep staging, achieving the best sleep stage classification recognition rate.

[0004] Existing sleep staging algorithms that capture the spatiotemporal relationship between sleep period sequences have some limitations in modeling the spatial relationship between brain channels and the spatiotemporal modeling between adjacent sleep period sequences. In terms of modeling the spatial relationship between brain channels, most methods ignore the potential functional connections between channels that are physically far away and the individual differences in brain channel connections. This makes revealing the imperceptible neural mechanisms between brain channels during human sleep a task that still needs to be improved. In terms of spatiotemporal modeling between adjacent sleep period sequences, more methods tend to use TCNs and GCNs components to capture spatiotemporal dependencies. However, this asynchronous capture method may lead to a lack of ability to capture spatiotemporal relationships, resulting in certain limitations in mining the spatiotemporal transition relationship between adjacent sleep periods. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a method, device and medium for EEG sleep staging based on spatiotemporal relationship learning.

[0006] The first technical solution adopted by the present invention is:

[0007] A method for EEG sleep staging based on spatiotemporal relationship learning, comprising the following steps:

[0008] Obtain EEG data;

[0009] Constructing a context sequence of adjacent sleep periods based on EEG data;

[0010] Input the constructed sequence into the sleep staging model and output the staging results;

[0011] The sleep staging model works as follows: a spatiotemporal graph is generated by splicing the instance adaptive brain functional connection matrix of a single sleep period and the spatiotemporal transition matrix of adjacent sleep periods; each adjacent sleep period is directly spliced ​​with the sleep period sample to be tested and connected with the corresponding spatiotemporal graph. Figure 1 In the same-input gated spatiotemporal graph convolution module, the spatiotemporal dependencies between brain channels in adjacent sleep periods are learned to obtain features for stage classification.

[0012] Furthermore, the obtaining of EEG data includes:

[0013] The original EEG data is obtained and time series dimension reduction is performed on the obtained EEG data.

[0014] Furthermore, the step of constructing a sleep adjacent period context sequence according to the EEG data includes:

[0015] The sleep period sample to be tested after time series dimension reduction and the first k and last k sleep period samples adjacent to the sample are taken as a time series as the input of the sleep stage model;

[0016] Among them, the label of each time series is still the label of the original sleep period sample to be tested, that is, if the original sleep period sample to be tested is x t , the model input is now the sequence X t =(x t-kT ,...,x t ,...,x t+kT ).

[0017] Further, the spatiotemporal map includes an example adaptive brain functional connectivity matrix for a single sleep period and a spatiotemporal transition matrix for adjacent sleep periods;

[0018] The instance-adaptive brain functional connectivity matrix of a single sleep period is multiplied by the sample to achieve instance-adaptive generation;

[0019] The construction of the spatiotemporal transition matrix of adjacent sleep periods adopts a multi-graph fusion mechanism, integrating the brain functional connections between adjacent sleep periods and the sleep period to be tested to obtain the spatiotemporal transition matrix between the two periods.

[0020] Furthermore, for each sleep period sample x t , its instance adaptive brain functional connection matrix is:

[0021]

[0022] X t =(x t-kT ,...,x t ,...,x t+kT )

[0023] Where W σ and W τ are two learnable parameter matrices; T is the transpose; k is the number of preset sleep period samples;

[0024] The brain functional connectivity matrix of each sleep period was placed on the diagonal position of the space-time diagram;

[0025] For any adjacent sleep period samples and sleep period samples to be tested, their spatiotemporal transition matrix is ​​established and placed in the upper right corner of the spatiotemporal diagram, which represents the dynamic changes and mutual influence of brain channels during sleep, and can characterize the spatiotemporal evolution relationship of brain channels in different sleep periods.

[0026] Furthermore, a multi-graph fusion mechanism based on spatial attention mechanism and graph attention mechanism is adopted to integrate the brain functional connection matrix A of the adjacent sleep time t-kd and the sleep time t to be measured t-dand A t , and obtain the corresponding space-time transition matrix

[0027] In the multi-graph fusion mechanism, the spatiotemporal embedding of the multi-graphs and the brain functional connectivity matrix of the two periods are input into the spatial attention mechanism and the graph attention mechanism. The spatial attention mechanism is used to calculate the influence of other nodes in each graph on a certain node, and the graph attention mechanism is used to calculate the autocorrelation of the same node in different graphs. Finally, a set of weights is output, and the matrix A is adjusted according to this set of weights. t-d and A t The weighted sum of different nodes in the , thus obtaining the corresponding spatiotemporal transition matrix

[0028] Furthermore, the gated spatiotemporal graph convolution module includes skip-gated graph convolution and gated temporal convolution;

[0029] Each adjacent sleep period is directly connected to the sleep period sample to be tested into Z, and is input into the skip-gated graph convolutional network with the corresponding spatiotemporal graph G. The skip-gated graph convolutional network makes full use of the information of different graph convolutional layers to improve the performance of the graph convolutional network, while capturing the temporal and spatial relationships between adjacent sleep periods to extract richer spatiotemporal features;

[0030] Gated temporal convolutions are used to integrate and further extract spatiotemporal information from multiple moments to facilitate learning of spatiotemporal dependencies between brain channels in sequences of adjacent sleep epochs.

[0031] Furthermore, the sleep staging model also includes a classifier for classifying according to the spatiotemporal features output by the gated spatiotemporal graph convolution module to determine the sleep stage to which each sleep period to be tested belongs.

[0032] The second technical solution adopted by the present invention is:

[0033] An electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, the code set or the instruction set is loaded and executed by the processor to implement an EEG sleep staging method based on spatiotemporal relationship learning as described above.

[0034] The third technical solution adopted by the present invention is:

[0035] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above-mentioned EEG sleep staging method based on spatiotemporal relationship learning.

[0036] The fourth technical solution adopted by the present invention is:

[0037] A computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned EEG sleep staging method based on spatiotemporal relationship learning.

[0038] The beneficial effect of the present invention is that the present invention captures the global spatiotemporal transition relationship of the sleep context sequence by combining the construction of the spatiotemporal graph with the design of the gated spatiotemporal graph convolutional network, and has good reliability and accuracy in the EEG sleep staging task. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 is a flow chart of a method for EEG sleep staging based on spatiotemporal relationship learning in an embodiment of the present invention;

[0041] Figure 2 is an overall architecture diagram of the sleep staging model in an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of a multi-image fusion mechanism in an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of constructing a space-time graph in an embodiment of the present invention;

[0044] Figure 5 is a structural diagram of a gated spatiotemporal graph convolution module in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0046] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0047] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0048] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0049] Technical explanation:

[0050] Sleep staging: The sleep process is usually divided into two categories: rapid eye movement (REM) and non-rapid eye movement (NREM). According to different international standards, such as AASM and R&K, the sleep period can be further divided. AASM is the more widely used international standard in the field of automatic sleep staging, which divides the human sleep process into five stages: wakefulness (W stage), three stages N1, N2 and N3 obtained by subdividing the NREM stage, and REM stage. The study of sleep staging can also help us better understand the physiological mechanism of human sleep and provide a more scientific basis for sleep medicine and health management.

[0051] PSG recording: including physiological indicators such as electroencephalogram (EEG), eye movement (EOG) and muscle electrical activity (EMG).

[0052] Sleep transition rules: A sleep cycle refers to a complete cycle of different sleep stages that humans experience during sleep. Usually, a sleep cycle lasts about 90 minutes to 120 minutes, including two main stages: rapid eye movement sleep (REM stage) and non-rapid eye movement sleep (NREM stage). In the field of sleep staging, it is believed that there are sleep transition rules in the human body's sleep process. Sleep transition rules refer to the switching rules between different sleep stages. Usually, a complete sleep cycle will experience multiple alternations between REM and NREM stages. Taking the sleep process under the AASM standard as an example, at the beginning of each sleep cycle, people will experience three NREM stages N1, N2, and N3, and then enter the first REM stage. Subsequently, the REM and NREM stages will repeat in an alternating form until waking up. This means that in most cases, the labels of adjacent sleep stages of the human body will not undergo extremely irregular mutations. Therefore, current sleep staging technology usually uses the sleep period to be tested and several previous and subsequent sleep periods to form a context sequence for processing and analysis to improve the accuracy and reliability of sleep staging tasks.

[0053] In response to the existing technical problems, the present invention proposes an EEG sleep staging scheme based on spatiotemporal relationship learning, combining the construction of the spatiotemporal graph with the design of the gated spatiotemporal graph convolutional network to capture the global spatiotemporal relationship of the sleep context sequence. First, each adjacent sleep period is directly spliced ​​with the sleep period sample to be tested, and an instance-adaptive spatiotemporal graph is generated by splicing the brain functional connection map of a single sleep period and the spatiotemporal transition matrix between adjacent periods. By using graphs to simultaneously model spatiotemporal relationships, the ability to learn the spatiotemporal transition relationship of sleep in different individuals during the sleep process is improved. Secondly, the gated spatiotemporal graph convolution module integrates the spatiotemporal features between multiple groups of adjacent sleep periods, further learns the global spatiotemporal dependencies of adjacent sleep time series, and achieves considerable performance in the EEG sleep staging task.

[0054] Example 1

[0055] like Figure 1 As shown, this embodiment provides an EEG sleep staging method based on spatiotemporal relationship learning, which is mainly implemented by constructing a spatiotemporal graph and a gated graph convolutional network. This method can effectively capture the global spatiotemporal transition relationship of different individual brain channels during sleep, and has good reliability and accuracy in the EEG sleep staging task. The method specifically includes the following steps:

[0056] S1. Obtain EEG data and perform time series dimension reduction on the EEG data.

[0057] In some embodiments, the dimension of each EEG channel of the original EEG data is too high, and a pre-trained deep CNN network is used to extract channel-independent temporal features to reduce information redundancy and improve the subsequent training efficiency of the model. Each sample of the original EEG sleep data records 30 seconds of sleep data at a high sampling rate, and the data dimension is too high, which is not conducive to model training and deep feature extraction.

[0058] It should be noted that the above-mentioned time series dimensionality reduction uses a pre-trained deep CNN network to extract more discriminative channel-independent time series features, but it is not limited to this. Other algorithms such as convolutional neural hidden Markov model HMM, recurrent neural network RNN, etc. can also be used, and statistical features related to time series learning such as maximum value, average value, etc. can also be used.

[0059] S2. Construct a context sequence of adjacent sleep periods based on the acquired EEG data.

[0060] As an implementation method, the sleep period sample to be tested after time series dimension reduction and the first k and last k sleep period samples adjacent to it are taken as time series and input into the network. The label of each time series is still the label of the original sleep period sample to be tested. That is, if the original sleep period sample to be tested is x t , the model input is now the sequence X t =(x t-kT ,...,x t ,...,x t+kT ).

[0061] S3. Construct a spatiotemporal graph based on the obtained sequence.

[0062] Specifically, the spatiotemporal graph is mainly composed of two block matrices, the instance adaptive brain channel functional connection matrix of a single sleep period and the spatiotemporal transition matrix of adjacent sleep periods. The brain functional connection matrix of each sleep period is located on the diagonal of the spatiotemporal graph, and instance adaptation is achieved by multiplying the adaptively learned matrix with each sample. The spatiotemporal transition matrix of adjacent sleep periods is placed in the upper right corner of the spatiotemporal graph, and is formed by integrating the brain functional connections of adjacent sleep periods with the sleep period to be tested by the graph fusion mechanism.

[0063] S4. Learning spatiotemporal dependencies using gated spatiotemporal graph convolutional networks.

[0064] Specifically, taking time t-kd as an example, the adjacent sleep period x t-kd and the sleep period sample x to be tested t Directly connect to Z t-kd , and the corresponding space-time graph A t-kd They are fed together into a gated spatiotemporal graph convolutional network to learn the spatiotemporal dependencies between brain channels in sequences of adjacent sleep epochs.

[0065] S5. Use a classifier to classify spatiotemporal features.

[0066] As an optional implementation, the spatiotemporal features obtained in step S4 are input into a multi-layer perceptron for dimensionality reduction, and finally the processed features are mapped to five different sleep stages using a softmax function, and the final sleep stage results are output to complete the five-category task of the sleep sample under the AASM standard.

[0067] join Figure 2 , the method of this embodiment is mainly implemented through a sleep staging model, which includes two large modules: a space-time graph construction module and a gated space-time graph convolution module. The adaptive space-time graph construction module consists of two parts, the instance adaptive brain channel functional connection matrix of a single sleep period and the space-time transition matrix of adjacent sleep periods, which realizes the simultaneous modeling of the space-time relationship of the context sequence of sleep periods. The brain channel functional connection of a single sleep period introduces a weight matrix and multiplies it with the sample to achieve instance adaptive generation. The construction of the space-time transition matrix of adjacent sleep periods adopts a multi-graph fusion mechanism, which integrates the brain functional connections of adjacent sleep periods with the sleep period to be tested, and obtains the space-time transition matrix between the two periods. Each adjacent sleep period is directly spliced ​​with the sample of the sleep period to be tested, and is connected with the corresponding space-time Figure 1 In the same input gated spatiotemporal graph convolution module, the purpose of further learning the spatiotemporal dependencies between brain channels in adjacent sleep periods is achieved.

[0068] The above contents are explained in detail below with reference to the accompanying drawings and specific implementation methods.

[0069] (1) Space-time graph construction module

[0070] The spatiotemporal map is mainly composed of two block matrices, the instance adaptive brain channel functional connectivity matrix of a single sleep period and the spatiotemporal transition matrix of adjacent sleep periods.

[0071] a) Example adaptive brain functional connectivity matrix

[0072] For EEG signals, the responses of different electrodes and their functional dependencies play a particularly important role in the cognitive generation of EEG-related tasks. However, due to the hidden nature of neural mechanisms, it is very difficult to define the functional relationships between different EEG areas in advance. Therefore, this patent adopts an instance-based data-driven method to adaptively learn the connection patterns between brain channels of different samples, aiming to reveal the specific and potential brain functional connection patterns of different individuals. For each sleep period sample x t , its instance adaptive brain functional connection matrix is:

[0073]

[0074] Xt =(x t-kT ,...,x t ,...,x t+kT )

[0075] Among them, W σ and W τ are two learnable parameter matrices. The brain functional connection matrix of each sleep period is placed on the diagonal position of the space-time diagram.

[0076] b) Spatiotemporal transition matrix of adjacent sleep periods

[0077] During the sleep process of the human body, different brain channels show a variety of brain activity patterns and connection characteristics. Brain channels not only interact with each other in the spatial dimension within the same sleep period, but also interact with each other in the temporal dimension between different sleep periods. In order to describe this dynamic influence, a spatiotemporal transition matrix between different sleep periods can be established. For any adjacent sleep period sample and the sleep period sample to be tested, here we take the time t-kd and t as examples to establish their spatiotemporal transition matrices Placed in the upper right corner of the space-time diagram, it represents the dynamic changes and mutual influence of brain channels during sleep, and can characterize the space-time evolution relationship of brain channels in different sleep periods.

[0078] See also Figure 3 Taking time t-kd and t as examples, in the method of this embodiment, a multi-graph fusion mechanism based on spatial attention mechanism and graph attention mechanism is adopted to integrate the brain function connection matrix A of the adjacent sleep time t-kd and the sleep time t to be measured t-d and A t , and obtain the corresponding space-time transition matrix The multi-image fusion mechanism first consists of the matrix A t-d and A t Calculate the spatial embedding SG and the graph embedding GE, and add them together to get the multi-graph spatial embedding SGE. In the multi-graph fusion mechanism, we input the multi-graph spatiotemporal embedding and the brain functional connection matrix of the two periods into the spatial attention mechanism and the temporal attention mechanism. The spatial attention mechanism is used to calculate the influence of other nodes in each graph on a certain node, while the graph attention mechanism is used to calculate the autocorrelation of the same node in different graphs. Both of the above attention mechanisms are implemented through the multi-head attention mechanism. The multi-graph fusion mechanism finally outputs a set of weights, which are used to t-d and A t The weighted sum of different nodes in the , thus obtaining the corresponding spatiotemporal transition matrix Such a spatiotemporal transition matrix can better characterize the dynamic changes and mutual influences of brain channels during sleep.

[0079] c) Construction of space-time graph

[0080] See also Figure 4 , the brain functional connection matrix and spatiotemporal transition matrix of each sleep period are used as block matrices, and the corresponding spatiotemporal graph can be obtained by splicing. Taking time t-kd and t as an example, the brain functional connection matrix A t-d and A t Placed in a diagonal position (the matrix of the earlier moment is placed in the upper left corner), the space-time transition matrix Place it in the upper right corner and set the lower left corner to 0 matrix, and you can get the time-space graph G corresponding to each moment t-kd .

[0081] (2) Gated spatiotemporal graph convolution module

[0082] Gated graph convolution is used to integrate and capture the spatiotemporal relationship between the upper and lower sequences of sleep periods, aiming to extract spatiotemporal features that are more relevant to the sleep staging task. Taking time t-kd and t as an example, each adjacent sleep period x t-kd and the sleep period sample x to be tested t Directly connect to Z t-kd , and the corresponding space-time graph A t-kd They are then fed together into a gated spatiotemporal graph convolutional network to learn the spatiotemporal dependencies between brain channels in adjacent sleep epoch sequences and extract spatiotemporal features that are more discriminative for the sleep staging task.

[0083] See also Figure 5 The gated spatiotemporal graph convolution module consists of skip-connected gated graph convolution and gated temporal convolution. The GLU function is used as the activation function in the module. Compared with conventional activation functions (such as ReLU, sigmoid, etc.), the GLU function has a larger number of parameters and can improve the performance of the model. In addition, in the module, each adjacent sleep period is directly connected to the sleep period sample to be tested, and the corresponding spatiotemporal graph G is input into the skip-connected gated graph convolution network. The skip-connected gated graph convolution network makes full use of the information of different graph convolution layers to improve the performance of the graph convolution network, while capturing the temporal and spatial relationships of adjacent sleep periods and extracting richer spatiotemporal features. The gated temporal convolution network integrates and further extracts the spatiotemporal information of multiple moments to promote the learning of the spatiotemporal dependency between brain channels in the sequence of adjacent sleep periods.

[0084] (3) Classifier

[0085] The classifier is mainly composed of a multi-layer perceptron, and its task is to classify the spatiotemporal features extracted previously into five categories to determine the sleep stage to which each sleep period to be tested belongs. By connecting a softmax function after the multi-layer perceptron, we can obtain the judgment of the sleep stage of the EEG sleep sample to be tested.

[0086] In summary, the method of the present invention has at least the following advantages and beneficial effects compared to the prior art:

[0087] (1) The present invention proposes a method for EEG sleep staging based on spatiotemporal relationship learning, which is implemented by constructing a spatiotemporal graph and designing a gated spatiotemporal graph convolutional network. This method can effectively capture the spatiotemporal transition relationship of brain channels during sleep and has good reliability and accuracy in the EEG sleep staging task. In the spatiotemporal graph construction module, the spatiotemporal relationship between adjacent sleep periods and the sleep period to be tested is synchronously modeled by splicing block matrices. The gated graph convolutional network module is used to learn the direct spatiotemporal influence of each adjacent sleep period on the sleep period to be tested, and capture the global spatiotemporal dependency of the sleep period context.

[0088] The present invention constructs a spatiotemporal graph by splicing the brain function connection matrix of each period and the spatiotemporal transition matrix between adjacent sleep periods, and proposes a more detailed spatiotemporal modeling method, aiming to provide the possibility of synchronously capturing spatiotemporal relationships. The subsequent gated spatiotemporal graph convolution module further enhances the model's ability to learn spatiotemporal dependencies, and has a higher ability to recognize sleep stages in the sleep staging task.

[0089] (2) The present invention proposes a method for constructing a spatiotemporal graph for modeling the spatiotemporal relationship of adjacent sleep context sequences. This method concatenates the brain functional connection matrix of a single sleep period and the spatiotemporal transition matrix between adjacent sleep periods into a block matrix, revealing the direct spatiotemporal influence of each adjacent sleep period on the sleep period to be tested. The generation of the brain functional connection matrix adopts an instance-adaptive approach, using a learnable parameter matrix multiplied by each sleep period sample. In addition, the generation of the spatiotemporal transition matrix adopts a multi-graph fusion mechanism based on spatial attention and graph attention, by integrating the brain functional connections of different periods.

[0090] The present invention uses a space-time graph to perform space-time modeling of the context sequence of sleep periods, and uses a graph to characterize the close connection between the topological spatial pattern of the brain and the temporal pattern in adjacent sleep stages. The space-time graph is mainly composed of two parts as block matrices, the instance-adaptive brain channel functional connection matrix of a single sleep period and the space-time transition matrix of adjacent sleep periods. The instance-adaptive brain functional network matrix can characterize the brain channel activity patterns of different individuals in different sleep periods. The generation of the space-time transition matrix utilizes a multi-graph fusion mechanism based on spatial attention and graph attention to integrate the brain functional connections of each adjacent sleep period with the sleep period to be tested, and to show the interaction and dynamic changes between brain channels in different sleep periods.

[0091] (3) The present invention proposes a gated spatiotemporal graph convolutional network. The gated spatiotemporal graph convolutional module includes a skip-connected gated graph convolutional network and a gated temporal convolutional network. Each adjacent sleep period is directly connected to the sleep period sample to be tested, and together with the spatiotemporal graph at the corresponding moment, it serves as the input of the gated spatiotemporal graph convolutional network to fully learn the spatiotemporal transition relationship between brain channels in the sequence of adjacent sleep periods. The gated temporal convolutional network integrates and further extracts the spatiotemporal transition information between multiple adjacent sleep periods, and is expected to learn the global spatiotemporal dependency relationship between brain channels in the sequence of adjacent sleep periods.

[0092] Compared with other spatiotemporal graph convolutional networks, the gated spatiotemporal graph convolution module of the present invention has the following advantages. First, the module uses the GLU function as the activation function. Compared with other conventional activation functions, the number of parameters of the GLU function is twice as much. By combining gated linear units and nonlinear functions, the module improves the ability to learn spatiotemporal features. Secondly, residual connections are added between stacked graph convolutional layers. Due to the small number of nodes in the spatiotemporal graph, simply stacking multiple graph convolutional layers is prone to over-smoothing problems. However, the residual design of the graph convolution layer alleviates this limitation by making full use of the neighbor information of graph nodes of different orders. This residual design improves the model's ability to learn spatiotemporal dependencies. Therefore, the gated spatiotemporal graph convolution module of the present invention improves the model's ability to learn spatiotemporal dependencies in the EEG sleep staging task by improving the model network design, and aims to achieve better performance.

[0093] Example 2

[0094] An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the following Figure 1 A method for EEG sleep staging based on spatiotemporal relationship learning is shown.

[0095] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0096] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0097] Since the electronic device is an electronic device corresponding to the EEG sleep staging method based on spatiotemporal relationship learning in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0098] Example 3

[0099] The embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 A method for EEG sleep staging based on spatiotemporal relationship learning is shown.

[0100] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0101] Since the storage medium is a storage medium corresponding to an EEG sleep staging method based on spatiotemporal relationship learning in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0102] Example 4

[0103] In some possible implementations, various aspects of the method of the embodiment of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the EEG sleep staging method based on spatiotemporal relationship learning according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing various embodiments can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0104] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0106] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for EEG sleep staging based on spatiotemporal relationship learning, characterized in that: The following steps are involved: Obtain EEG data; Constructing a context sequence of adjacent sleep periods based on EEG data; Input the constructed sequence into the sleep staging model and output the staging results; Among them, the working method of the sleep staging model is: by splicing the instance adaptive brain function connection matrix of a single sleep period and the spatiotemporal transition matrix of adjacent sleep periods, a spatiotemporal graph is generated; each adjacent sleep period is directly spliced ​​with the sleep period sample to be tested, and input into the gated spatiotemporal graph convolution module together with the corresponding spatiotemporal graph, the spatiotemporal dependency between brain channels in adjacent sleep periods is learned, and the features used for staging are obtained.

2. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 1, characterized in that: The obtaining of EEG data comprises: The original EEG data is obtained and time series dimension reduction is performed on the obtained EEG data.

3. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 1, characterized in that: The step of constructing a sleep adjacent period context sequence according to the EEG data comprises: The sleep period sample to be tested after time series dimension reduction and the first k and last k sleep period samples adjacent to the sample are taken as a time series as the input of the sleep stage model; Among them, the label of each time series is still the label of the original sleep period sample to be tested, that is, if the original sleep period sample to be tested is x t , the model input is now the sequence X t =(x t-kT ,...,x t ,...,x t+kT ).

4. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 1, characterized in that: The spatiotemporal map includes an example adaptive brain functional connectivity matrix of a single sleep period and a spatiotemporal transition matrix of adjacent sleep periods; The instance-adaptive brain functional connectivity matrix of a single sleep period is multiplied by the sample to achieve instance-adaptive generation; The construction of the spatiotemporal transition matrix of adjacent sleep periods adopts a multi-graph fusion mechanism, integrating the brain functional connections between adjacent sleep periods and the sleep period to be tested to obtain the spatiotemporal transition matrix between the two periods.

5. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 1 or 4, characterized in that: For each sleep epoch sample x t , its instance adaptive brain functional connection matrix is: X t =(x t-kT ,...,x t ,...,x t+kT ) Where W σ and W τ are two learnable parameter matrices; T is the transpose; k is the number of preset sleep period samples; The brain functional connectivity matrix of each sleep period was placed on the diagonal position of the space-time diagram; For any adjacent sleep period samples and sleep period samples to be tested, their spatiotemporal transition matrix is ​​established and placed in the upper right corner of the spatiotemporal diagram, which represents the dynamic changes and mutual influence of brain channels during sleep, and can characterize the spatiotemporal evolution relationship of brain channels in different sleep periods.

6. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 5, characterized in that: A multi-graph fusion mechanism based on spatial attention mechanism and graph attention mechanism is used to integrate the brain functional connection matrix A of adjacent sleep time t-kd and the sleep time t to be measured. t-d and A t , and obtain the corresponding space-time transition matrix In the multi-graph fusion mechanism, the spatiotemporal embedding of the multi-graphs and the brain functional connectivity matrix of the two periods are input into the spatial attention mechanism and the graph attention mechanism. The spatial attention mechanism is used to calculate the influence of other nodes in each graph on a certain node, and the graph attention mechanism is used to calculate the autocorrelation of the same node in different graphs. Finally, a set of weights is output, and the matrix A is adjusted according to this set of weights. t-d and A t The weighted sum of different nodes in the , thus obtaining the corresponding spatiotemporal transition matrix 7. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 1, characterized in that: The gated spatiotemporal graph convolution module includes skip-gated graph convolution and gated temporal convolution; Each adjacent sleep period is directly connected to the sleep period sample to be tested into Z, and input into the skip-gated graph convolutional network with the corresponding spatiotemporal graph G. The skip-gated graph convolutional network makes full use of the information of different graph convolutional layers to improve the performance of the graph convolutional network, while capturing the temporal and spatial relationships between adjacent sleep periods to extract richer spatiotemporal features; the gated temporal convolution is used to integrate and further extract the spatiotemporal information of multiple moments to promote the learning of the spatiotemporal dependencies between brain channels in the sequence of adjacent sleep periods.

8. The method for EEG sleep staging based on spatiotemporal relationship learning according to claim 1, characterized in that: The sleep staging model also includes a classifier for performing classification according to the spatiotemporal features output by the gated spatiotemporal graph convolution module to determine the sleep stage to which each sleep period to be tested belongs.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

Images