Neuroscience inspired sleep staging method and system based on cross-modal long-short-term attention

Abstract

The invention discloses a neuroscience inspired sleep staging method and a neuroscience inspired sleep staging system based on cross-modal long-term and short-term attention. A minority class sample with physiological rationality is generated through a Physio-GAN data enhancement module, and the problem of class imbalance in sleep staging is effectively relieved; a heterogeneous multi-scale feature extraction module inspired by neuroscience is adopted to adaptively extract multi-time scale features of electroencephalogram and electro-oculogram signals through a multi-scale convolution structure of an NMS-EEG sub-module and a fast and slow channel decomposition mechanism of an ODDM-EOG sub-module; a cross-modal long and short time sequence modeling module is utilized, and a bidirectional cross-modal attention mechanism, a bidirectional long and short time memory network and a multi-head self-attention mechanism are combined, so that collaborative modeling of dynamic interaction of multi-modal features and long and short time sequence dependence is realized. According to the method, the defects of a traditional method in the aspects of minority class recognition, multi-scale feature capture and cross-modal fusion are effectively overcome, and the accuracy and practicability of sleep staging results are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical artificial intelligence technology, and in particular to a neuroscience-inspired sleep staging method and system based on cross-modal long and short-term attention. Background Technology

[0002] Automated sleep staging, a key technology in sleep medicine, has long relied primarily on manual visual analysis of polysomnography (PSG) signals by professional physicians. This traditional method has significant limitations: firstly, manual annotation requires substantial time and specialized human resources, resulting in low efficiency; secondly, subjective differences among experts lead to low consistency in annotation results, with studies showing a consistency rate of only about 83% among different experts. This subjectivity and inefficiency severely restrict the large-scale application of sleep staging in practice.

[0003] To overcome the limitations of manual stage segmentation, researchers have explored various automated technical approaches. Early traditional machine learning methods relied on manually designed time-domain and frequency-domain features, combined with classifiers such as support vector machines and random forests to segment sleep stages. However, these methods require deep domain knowledge for feature engineering, and manually designed features have limited generalization ability, making it difficult to adapt to different datasets and individual differences. With the development of deep learning technology, end-to-end models based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Transformers have gradually become mainstream. They can automatically learn feature representations, but still face multiple challenges.

[0004] The most prominent problem with existing deep learning models in practice is the severe imbalance in sample size during sleep stages. In natural sleep, stage N1 typically accounts for only 3-11%, while stage N2 accounts for approximately 40%. This imbalance causes model training to favor the majority class, significantly reducing performance in recognizing the minority class. DeepSleepNet, as a representative model, extracts multi-scale features through a dual-branch convolutional structure, but it only uses single-channel EEG signals and lacks an effective mechanism for handling class imbalance, limiting its performance in distinguishing between easily confused stages such as N1 and REM sleep.

[0005] Capturing features across multiple time scales is another major technical challenge. Sleep signals simultaneously contain short-term transient waveforms (such as spindle waves and K-complex waves) and long-term circadian rhythm features, requiring models to have hierarchical perception capabilities. SeqSleepNet improves temporal modeling by employing a hierarchical recurrent network structure, but it is insufficient in characterizing the time-frequency features of typical sleep events, lacks multi-scale convolutional structures, and struggles to comprehensively capture the dynamic features of the sleep process.

[0006] Insufficient cross-modal information fusion also limits further improvement in model performance. Polysomnography naturally includes multiple complementary signals such as electroencephalography (EEG) and electrooculography (EOG), but existing methods such as SleepViTransformer, while introducing multimodal inputs, only demonstrate cross-modal interaction through feature transfer during the pre-training phase, lacking a dynamic interaction mechanism within the model itself. Furthermore, this model relies on short-time Fourier transform to obtain frequency domain features, failing to directly perceive time-domain waveform morphology, and the Transformer structure has high computational complexity, making it prone to overfitting when samples are limited.

[0007] These technical bottlenecks collectively point to a key need in the current field of sleep staging: a novel technical solution capable of effectively addressing class imbalance, achieving adaptive multi-scale feature extraction, and deeply integrating multimodal information. The shortcomings of existing methods in feature representation, modality fusion, and long-term time-series modeling create both technical space and the necessity for innovation in this invention.

[0008] Definitions of abbreviations and key terms

[0009] EEG: Electroencephalogram.

[0010] EOG: Electrooculogram;

[0011] PSG: Polysomnography;

[0012] Physio-GAN: Physiological Generative Adversarial Network.

[0013] NMS-EEG: Neuro-Inspired Multi-Scale EEG Extractor.

[0014] ODDM-EOG: Ocular Dynamics Decomposition Module.

[0015] CMIA: Cross Modal Interactive Temporal Attention Sequence Module.

[0016] BCMA: Bidirectional cross-modal attention.

[0017] MHSA: Multi-head self-attention;

[0018] MSCNN: Multi-scale Convolutional Neural Networks.

[0019] LCR: Lightweight Channel Rescaler;

[0020] TPP: Temporal pyramid pooling. Summary of the Invention

[0021] This invention addresses the shortcomings of existing technologies by providing a neuroscience-inspired sleep staging method and system based on cross-modal long and short-term attention.

[0022] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0023] A neuroscience-inspired sleep staging method based on cross-modal long and short-term attention includes the following steps:

[0024] S1: Data augmentation: Data augmentation is performed on minority sleep stage samples using Physio-GAN generative adversarial network to generate physiologically plausible EEG and EOG signals.

[0025] S2: Feature extraction, using a neuroscience-inspired heterogeneous multi-scale feature extraction method to extract multi-scale features from EEG and EOG signals respectively;

[0026] S3: Cross-modal interaction, which realizes dynamic interaction between EEG features and EOG features through a bidirectional cross-modal attention mechanism;

[0027] S4: Temporal modeling, combining bidirectional long short-term memory networks and multi-head self-attention mechanisms to achieve long short-term sequence dependency modeling;

[0028] S5: Classification output. Based on the fused features, the sleep stage is classified, and the classification results of the five stages, Wake, N1, N2, N3 and REM, are output.

[0029] Furthermore, the data augmentation in S1 specifically includes: constructing a generative model based on a relative discriminative adversarial network, wherein the generator uses a small convolutional kernel and a multi-layer transposed convolutional structure for EEG signal generation, and a large convolutional kernel structure for EOG signal generation; the discriminator uses spectral normalization and channel attention mechanisms; and the generated signals are constrained by a combination of temporal consistency loss function and frequency consistency loss function to ensure that the generated EEG and EOG signals are consistent with the real samples in terms of temporal waveform and frequency power distribution.

[0030] Furthermore, the EEG feature extraction in S2 employs a multi-scale convolutional neural network, including three parallel convolutional branches that use convolutional kernels of different sizes to capture high-frequency, mid-frequency, and low-frequency EEG features. A lightweight channel scaling mechanism is introduced to assign a learnable scaling factor to each feature channel to achieve adaptive feature adjustment. Finally, the multi-scale features are spliced, pooled, and convolved through feature integration operations to form a structured EEG representation.

[0031] Furthermore, the EOG feature extraction in S2 adopts a fast and slow channel decomposition mechanism. The slow channel models low-frequency slow eye movement features through large convolutional kernels, while the fast channel captures high-frequency fast eye movement features through first-order difference operations and small convolutional kernels. It also combines residual enhancement modules to expand the receptive field and temporal pyramid pooling operations to integrate multi-scale features through adaptive pooling in different time windows.

[0032] Furthermore, the bidirectional cross-modal attention mechanism in S3 is specifically implemented as follows: EEG features and EOG features are linearly mapped to query matrix, key matrix and value matrix respectively; cross-modal attention weights from EEG to EOG are calculated to generate EOG-enhanced EEG features; cross-modal attention weights from EOG to EEG are calculated to generate EEG-enhanced EOG features; and attention features are weighted and injected into the original modal features through residual connections to achieve dynamic interaction between modalities.

[0033] Furthermore, the temporal modeling in S4 includes: performing global temporal modeling on the feature sequence after cross-modal interaction through a bidirectional long short-term memory network to capture global long-term dependencies; modeling the feature correlation at key moments in the sequence through a multi-head self-attention mechanism to focus on local short-term dependent features; and introducing random deactivation processing before the output layer to prevent overfitting and improve model robustness.

[0034] Furthermore, the classification output in S5 is based on a fully connected layer and a Softmax classifier, outputting the probability distribution of five sleep stages; and by optimizing the cross-entropy loss function to train the model, the overall method simulates the hierarchical processing mechanism of the human cerebral cortex, which is suitable for various sleep monitoring needs.

[0035] This invention also discloses a neuroscience-inspired sleep staging system based on cross-modal long and short-term attention, used to implement the above-mentioned neuroscience-inspired sleep staging method, specifically including:

[0036] The data augmentation module is used to augment minority sleep stage samples using the Physio-GAN generative adversarial network. This module includes a generator and a discriminator. The generator uses small convolutional kernels and multi-layer transposed convolutional structures for EEG signal generation and large convolutional kernels for EOG signal generation. The discriminator uses spectral normalization and channel attention mechanisms, combined with temporal consistency loss and frequency consistency loss functions, to ensure that the generated physiological signals are consistent with the real samples in terms of temporal waveform and frequency power distribution.

[0037] The feature extraction module processes the input signal using a neuroscience-inspired heterogeneous multi-scale feature extraction method. This module includes the NMS-EEG submodule and the ODDM-EOG submodule. The NMS-EEG submodule extracts EEG features through a multi-scale convolutional neural network, containing three parallel convolutional branches to capture high-frequency, mid-frequency, and low-frequency EEG features respectively, and integrates a lightweight channel scaling mechanism to achieve adaptive feature adjustment. The ODDM-EOG submodule extracts EOG features through a fast and slow channel decomposition mechanism. The slow channel uses a large convolutional kernel to model low-frequency slow-motion eye movement features, while the fast channel uses first-order difference operations and small convolutional kernels to capture high-frequency fast-motion eye movement features, and integrates multi-scale features by combining temporal pyramid pooling operations.

[0038] The cross-modal interaction module is used to realize the dynamic interaction between EEG features and EOG features through a bidirectional cross-modal attention mechanism. This module linearly maps the EEG and EOG features output by the feature extraction module into query, key and value matrices, respectively, calculates the bidirectional attention weights, and enhances the original modal features through residual connections.

[0039] The temporal modeling module is used to combine bidirectional long short-term memory network and multi-head self-attention mechanism for temporal dependency modeling. This module first captures global long-term dependencies through bidirectional long short-term memory network, then focuses on local short-term key features through multi-head self-attention mechanism, and introduces random deactivation to prevent overfitting.

[0040] The classification output module is used to output the sleep stage classification results based on the fused features. This module generates the probability distribution of five stages: Wake, N1, N2, N3 and REM through a fully connected layer and a Softmax classifier.

[0041] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the above-described neuroscience-inspired sleep staging method.

[0042] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described neuroscience-inspired sleep staging method.

[0043] Compared with the prior art, the advantages of the present invention are as follows:

[0044] 1. This invention innovatively achieves data augmentation for minority sleep stages by introducing a physiological consistency generative adversarial network. This mechanism can generate synthetic samples that are highly consistent with real physiological signals in terms of time-domain waveforms and frequency-domain features, effectively expanding the sample size of the minority class in the training dataset, improving the model's generalization ability under class imbalance conditions, and providing more reliable support for subsequent classification tasks.

[0045] 2. The neural-inspired heterogeneous multi-scale feature extraction module designed in this invention overcomes the limitations of traditional single-scale modeling. By simulating the brain's sleep rhythm mechanism at different time scales, this method can capture feature patterns at different time scales in physiological signals in parallel, focusing on both local transient waveform details and global rhythmic changes. This multi-scale collaborative perception mechanism enables the model to have a more comprehensive feature representation capability, adaptively extracting key features related to sleep stages, laying a solid feature foundation for subsequent classification tasks.

[0046] 3. The bidirectional cross-modal attention mechanism proposed in this invention enables deep interaction of multimodal physiological signals. This method overcomes the shortcomings of traditional splicing or simple fusion methods by establishing a dynamic correlation network between EEG and EOG signals, fully exploring the complementary information between different modalities. This interaction mechanism enables the model to analyze and judge multiple physiological signals in a manner similar to human experts, improving the accuracy and reliability of staging results.

[0047] 4. This invention creatively combines a bidirectional long short-term memory network with a multi-head self-attention mechanism to form a multi-level temporal feature extraction system. This design can capture both the long-term evolution of the sleep process and focus on transient feature changes at key time points, achieving comprehensive modeling of the dynamic sleep process. This temporal processing method is more in line with the essential needs of sleep staging, enabling the model to accurately grasp the transition patterns of sleep stages, thereby outputting more continuous and stable staging results.

[0048] In summary, this invention, through multifaceted technological innovation, effectively solves long-standing technical challenges in the field of sleep staging, achieving breakthroughs in key areas such as feature representation, modal fusion, and temporal modeling. These beneficial effects are not only reflected in improved technical indicators, but more importantly, they create the necessary conditions for the practical application of automatic sleep staging technology, possessing significant scientific value and practical significance. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the structure of the neuroscience-inspired sleep staging model in an embodiment of the present invention;

[0051] Figure 2 These are comparison diagrams of real and generated EEG samples in this embodiment of the invention; (a) is a time-domain waveform diagram of the real and generated samples. (b) is a power spectral density diagram of the real and generated samples.

[0052] Figure 3 These are comparison diagrams of real and generated EOG samples in this embodiment of the invention; (a) is a time-domain waveform diagram of the real and generated samples. (b) is a power spectral density diagram of the real and generated samples.

[0053] Figure 4 This is a comparison chart of model prediction and real label in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention provides a neuroscience-inspired sleep staging method based on cross-modal long and short-term attention, characterized by the following processing steps:

[0056] S1: Data augmentation: Data augmentation is performed on minority sleep stage samples using Physio-GAN generative adversarial network to generate physiologically plausible EEG and EOG signals.

[0057] S2: Feature extraction, using a neuroscience-inspired heterogeneous multi-scale feature extraction method to extract multi-scale features from EEG and EOG signals respectively;

[0058] S3: Cross-modal interaction, which realizes dynamic interaction between EEG features and EOG features through a bidirectional cross-modal attention mechanism;

[0059] S4: Temporal modeling, combining bidirectional long short-term memory networks and multi-head self-attention mechanisms to achieve long short-term sequence dependency modeling;

[0060] S5: Classification output. Based on the fused features, the sleep stage is classified, and the classification results of the five stages, Wake, N1, N2, N3 and REM, are output.

[0061] This invention also provides a neuroscience-inspired sleep staging model based on cross-modal long and short-term attention (ISATSleepNet), which includes the following main modules:

[0062] 1. Physio-GAN minority class sample augmentation module;

[0063] 2. Neuroscience-inspired heterogeneous multiscale feature extraction modules (NMS-EEG and ODDM-EOG).

[0064] 3. Cross-modal long-short time attention sequence modeling module (CMIA).

[0065] I. Overall Model Structure and Signal Flow Description

[0066] The overall structure of ISATSleepNet is as follows: Figure 1 As shown. The model takes multi-channel PSG signals (including EEG and EOG) as input and processes them sequentially through the following steps:

[0067] 1. Data augmentation processing: Generative augmentation of the N1-period signal using Physio-GAN;

[0068] 2. Feature Extraction: NMS-EEG models EEG rhythms using multi-scale convolution and LCR hierarchical modeling. The ODDM-EOG module captures low-frequency slow eye movements and high-frequency fast eye movements by combining fast / slow dual channels with large convolutional residuals and temporal pyramid pooling. This provides feature representations for cross-modal interaction.

[0069] 3. Cross-modal feature interaction and temporal modeling: By simulating the information exchange between multiple sensory pathways in the brain through BCMA, EEG and EOG features are mutually enhanced. At the same time, BiLSTM and multi-head self-attention are combined to simulate the temporal processing of the nervous system from local recursion to global integration, capturing local and long-range dependencies from within the fragment.

[0070] 4. Classification output: The classification results of the sleep stage are output through the fully connected layer.

[0071] II. Physio-GAN Minority Class Augmentation Module

[0072] Physio-GAN comprises a generator and a discriminator. The EEG generator employs a small convolutional kernel and a multi-layer transposed convolutional structure, while the EOG generator uses a large convolutional kernel structure. The discriminator uses spectral normalization and channel attention mechanisms to distinguish between real and generated samples and output a similarity score. Temporal consistency loss and frequency consistency loss are introduced during generator training, and combined to obtain the loss functions for both the generator and discriminator, ensuring that the generated N1-period samples are consistent with real samples in both temporal waveform and frequency power distribution.

[0073] Operation steps: Input real N1-period EEG and EOG samples into Physio-GAN, generate high-quality N1 samples through the trained generator, and merge them with real samples for subsequent model training.

[0074] Technical Results: Physio-GAN combines various regularization methods and time-frequency constraints to ensure that the generated EEG signals are consistent with real samples in both the time and frequency domains. This alleviates the class imbalance problem and improves the model's ability to recognize samples in the N1 stage.

[0075] Compare real and generated EEG and EOG samples in the time and frequency domains, and plot the time-domain waveform and power spectral density respectively. Figure 2 , Figure 3 As shown.

[0076] Depend on Figure 2 and Figure 3 It can be seen that the generated EEG and EOG samples are highly consistent with the real samples in both the time and frequency domains. This indicates that the Physio-GAN model effectively learns the underlying statistical regularities of real physiological signals and possesses the feasibility and reliability for N1-phase data augmentation.

[0077] III. Neuroscience-Inspired Heterogeneous Multiscale Feature Extraction

[0078] Including NMS-EEG and ODDM-EOG modules:

[0079] 1. NMS-EEG module (EEG feature extraction)

[0080] (1) Multi-scale convolution: Three parallel convolution branches are used to capture high-frequency, mid-frequency and low-frequency EEG features using convolution kernels of different sizes.

[0081] (2) Channel scaling: A learnable scaling factor is assigned to each channel through the LCR mechanism to achieve adaptive adjustment of features.

[0082] (3) Feature integration: The results of multi-scale convolution are spliced ​​together, and then subjected to one-dimensional max pooling and two-layer convolution operations to output a structured EEG representation.

[0083] 2. ODDM-EOG module (EOG feature extraction)

[0084] (1) Fast and slow channel decomposition: Eye movement dynamics are modeled through the decomposition mechanism of slow and fast channels. The slow channel uses a large convolutional kernel to model low-frequency slow eye movement features, while the fast channel captures high-frequency fast eye movement features through first-order difference operations and small convolutional kernels.

[0085] (2) Residual enhancement: Introduce a large convolution kernel residual module to expand the receptive field and alleviate gradient vanishing.

[0086] (3) Temporal pyramid pooling: Through adaptive pooling and linear interpolation of different time windows, multi-scale features are integrated and time lengths are aligned to output EOG representation.

[0087] Technical Results: NMS-EEG and ODDM-EOG extract EEG rhythms and eye movement dynamics features through parallel multi-scale modeling, providing structured and physiologically interpretable inputs for subsequent cross-modal attention.

[0088] IV. Cross-modal long and short-term attention sequence modeling module

[0089] It includes two modules: BCMA and long and short time series modeling.

[0090] 1. BCMA module:

[0091] (1) Linear mapping: The features (E) output by NMS-EEG and the features (O) output by ODDM-EOG are linearly mapped to obtain the Q, K and V matrices respectively.

[0092] (2) Bidirectional cross-modal attention calculation: calculate the cross-modal attention features of E→O and O→E respectively, and inject the attention features into the original modal features through residual enhancement.

[0093] (3) Feature splicing: The two enhanced modal features are spliced ​​together in the channel dimension to form a time-aligned feature sequence.

[0094] 2. Long and short time series modeling module:

[0095] (1) Long-term dependency modeling: global long-term dependencies are captured by using BiLSTM to model the features output by BCMA.

[0096] (2) Short-term dependency focus: The correlation of key positions in the sequence is modeled through the MHSA mechanism to highlight the short-term dependency features at key moments. Complementary dependencies are learned in different feature subspaces, thereby enhancing the model's ability to represent the sequence.

[0097] (3) Output processing: After random inactivation and fully connected layers, the sleep staging results are output.

[0098] Technical Results: CMIA, within the same framework, jointly achieves modal interaction and temporal modeling, simulating the brain's interaction and gradual integration mechanisms across multiple pathways. Information sharing between modalities not only maintains a grasp of global rhythms but also enhances sensitivity to local features, thereby effectively improving the classification performance of sleep stages.

[0099] Participants were randomly selected from the SHHS dataset, and their sleep stages were determined. Sleep stage curves were plotted as follows: Figure 4 As shown in the figure, the ISATSleepNet model proposed in this invention demonstrates a high degree of consistency between its predictions for Wake, N1, N2, N3, and REM stages throughout the entire sleep cycle and the expert-annotated ground truth labels, thus validating the effectiveness of the model's feature extraction and temporal modeling.

[0100] In summary, this invention achieves system optimization from the data layer to the time series layer through the collaborative design of Physio-GAN, heterogeneous multi-scale feature extraction, and cross-modal long and short time series modeling. Experimental results show that this invention has high practical application value and promotion potential.

[0101] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of neuroscience-inspired sleep staging methods.

[0102] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0103] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the neuroscience-inspired sleep staging method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by a processor.

[0104] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0106] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A neuroscience-inspired sleep staging method based on cross-modal long and short-term attention, characterized in that, Includes the following steps: S1: Data augmentation: Data augmentation is performed on minority sleep stage samples using Physio-GAN generative adversarial network to generate physiologically plausible EEG and EOG signals. S2: Feature extraction, using a neuroscience-inspired heterogeneous multi-scale feature extraction method to extract multi-scale features from EEG and EOG signals respectively; S3: Cross-modal interaction, which realizes dynamic interaction between EEG features and EOG features through a bidirectional cross-modal attention mechanism; S4: Temporal modeling, combining bidirectional long short-term memory networks and multi-head self-attention mechanisms to achieve long short-term sequence dependency modeling; S5: Classification output. Based on the fused features, the sleep stage is classified, and the classification results of the five stages, Wake, N1, N2, N3 and REM, are output.

2. The method according to claim 1, characterized in that, The data augmentation in S1 specifically includes: constructing a generative model based on a relative discriminant adversarial network, wherein the generator uses a small convolutional kernel and a multi-layer transposed convolutional structure for EEG signal generation, and a large convolutional kernel structure for EOG signal generation; the discriminator uses spectral normalization and channel attention mechanisms; and the generated signals are constrained by a combination of temporal consistency loss function and frequency consistency loss function to ensure that the generated EEG and EOG signals are consistent with the real samples in terms of temporal waveform and frequency power distribution.

3. The method according to claim 1, characterized in that, The EEG feature extraction in S2 uses a multi-scale convolutional neural network, including three parallel convolutional branches, which use convolutional kernels of different sizes to capture high-frequency, mid-frequency and low-frequency EEG features respectively. A lightweight channel scaling mechanism is introduced to assign a learnable scaling factor to each feature channel to achieve adaptive feature adjustment. Finally, multi-scale features are spliced, pooled and convolved through feature integration operations to form a structured EEG representation.

4. The method according to claim 1, characterized in that, The EOG feature extraction in S2 adopts a fast and slow channel decomposition mechanism. The slow channel models low-frequency slow eye movement features through large convolutional kernels, while the fast channel captures high-frequency fast eye movement features through first-order difference operations and small convolutional kernels. It also combines residual enhancement modules to expand the receptive field and temporal pyramid pooling operations to integrate multi-scale features through adaptive pooling in different time windows.

5. The method according to claim 1, characterized in that, The bidirectional cross-modal attention mechanism in S3 is specifically implemented as follows: EEG features and EOG features are linearly mapped to query matrix, key matrix and value matrix respectively; cross-modal attention representation from EEG to EOG is calculated to generate EOG-enhanced EEG features; Calculate the cross-modal attention representation from EOG to EEG to generate EEG-enhanced EOG features; Furthermore, attention features are weighted and injected into the original modal features through residual connections to achieve dynamic interaction between modalities.

6. The method according to claim 1, characterized in that, The temporal modeling in S4 includes: performing global temporal modeling on the feature sequence after cross-modal interaction through a bidirectional long short-term memory network to capture global long-term dependencies; modeling the feature correlation at key moments in the sequence through a multi-head self-attention mechanism to focus on local short-term dependent features; and introducing random deactivation processing before the output layer to prevent overfitting and improve model robustness.

7. The method according to claim 1, characterized in that, The classification output in S5 is based on a fully connected layer and a Softmax classifier, outputting the probability distribution of five sleep stages; and by optimizing the cross-entropy loss function to train the model, the overall method simulates the hierarchical processing mechanism of the human cerebral cortex, which is suitable for various sleep monitoring needs.

8. A neuroscience-inspired sleep staging system based on cross-modal long and short-term attention, used to implement the neuroscience-inspired sleep staging method according to any one of claims 1 to 7, characterized in that, include: The data augmentation module is used to augment minority sleep stage samples using the Physio-GAN generative adversarial network. This module includes a generator and a discriminator. The generator uses small convolutional kernels and multi-layer transposed convolutional structures for EEG signal generation and large convolutional kernels for EOG signal generation. The discriminator uses spectral normalization and channel attention mechanisms, combined with temporal consistency loss and frequency consistency loss functions, to ensure that the generated physiological signals are consistent with the real samples in terms of temporal waveform and frequency power distribution. The feature extraction module processes the input signal using a neuroscience-inspired heterogeneous multi-scale feature extraction method. This module includes the NMS-EEG submodule and the ODDM-EOG submodule. The NMS-EEG submodule extracts EEG features through a multi-scale convolutional neural network, containing three parallel convolutional branches to capture high-frequency, mid-frequency, and low-frequency EEG features respectively, and integrates a lightweight channel scaling mechanism to achieve adaptive feature adjustment. The ODDM-EOG submodule extracts EOG features through a fast and slow channel decomposition mechanism. The slow channel uses a large convolutional kernel to model low-frequency slow-motion eye movement features, while the fast channel uses first-order difference operations and small convolutional kernels to capture high-frequency fast-motion eye movement features, and integrates multi-scale features by combining temporal pyramid pooling operations. The cross-modal interaction module is used to realize the dynamic interaction between EEG features and EOG features through a bidirectional cross-modal attention mechanism. This module linearly maps the EEG and EOG features output by the feature extraction module into query, key and value matrices, respectively, calculates the bidirectional attention weights, and enhances the original modal features through residual connections. The temporal modeling module is used to combine bidirectional long short-term memory network and multi-head self-attention mechanism for temporal dependency modeling. This module first captures global long-term dependencies through bidirectional long short-term memory network, then focuses on local short-term key features through multi-head self-attention mechanism, and introduces random deactivation to prevent overfitting. The classification output module is used to output the sleep stage classification results based on the fused features. This module generates the probability distribution of five stages: Wake, N1, N2, N3 and REM through a fully connected layer and a Softmax classifier.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the neuroscience-inspired sleep staging method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the neuroscience-inspired sleep staging method according to any one of claims 1 to 7.

Images