Sleep staging method and system based on multimode signal fusion and context awareness

Abstract

The invention discloses a sleep staging method and system based on multimode signal fusion and context awareness. The sleep staging method based on multimode signal fusion and context sensing comprises the following steps: acquiring a multimode physiological data set; performing multi-scale feature extraction on the multi-modal physiological data set to obtain a multi-scale feature set; fusing the multi-scale feature set based on a lightweight model to obtain a fused feature set; classifying the fused feature set, and obtaining a sleep staging result based on a classification result; wherein the multi-scale feature extraction of the multi-modal physiological data set comprises the following steps: segmenting the multi-modal physiological data set into a frame sequence set; and performing historical context feature extraction, future context feature extraction simulation and timestamp feature extraction on the frame sequence set to obtain a context feature set. In this way, the staging continuity can be enhanced, and the misjudgment rate caused by burst signal fluctuation is reduced.

Description

Technical Field

[0001] The present application relates to the field of sleep detection, and specifically to a sleep staging method and system based on multimodal signal fusion and context perception. Background Art

[0002] At present, there are many problems in sleep staging technology, such as low efficiency and high subjectivity of manual interpretation, insufficient generalization ability of automated algorithms, high complexity and poor real-time performance of automated calculations, low sleep recognition rate during specific periods, and sensitivity to interference from transient events.

[0003] Traditional sleep staging relies on manual interpretation by professional technicians of polysomnography (PSG). PSG collects physiological signals such as electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG). These technicians interpret sleep stages frame by frame (30 seconds / frame) according to the American Academy of Sleep Medicine (AASM) guidelines. This interpretation is highly subjective. Sleep stages are categorized into five categories: Wake, N1, N2, N3, and REM. Wake is the period of wakefulness; REM is rapid eye movement; N1 sleep is the typical transition from wakefulness to sleep; N2 is characterized by theta waves in the EEG; and N3 sleep is often referred to as "deep sleep" or "slow-wave sleep." Each case takes approximately two hours to interpret based on a single night's data. Large-scale cohort studies (e.g., thousands of cases) can take months, resulting in high labor costs and low efficiency. Furthermore, the consistency of interpretation between different technicians is low (approximately 83%). Repeated interpretations by the same rater have a consistency of only 90%, which is difficult to meet the clinical research demand for high-precision and reproducible results. Manual interpretation is easily influenced by subjective experience and difficult to standardize.

[0004] Existing automated algorithms are often trained on single, central datasets (e.g., DOD, Sleep-EDF), which fail to cover a wide range of age groups, disease states, or device variations, resulting in insufficient generalization. For example, existing automated algorithms are trained on a limited population (e.g., healthy adults) and have poor adaptability to specific populations, such as those with obstructive sleep apnea (OSA) or other sleep disorders, children, or the elderly. Existing automated algorithms are also insufficiently robust to signal variations across devices (e.g., different brands of PSG devices, home wearables), resulting in significant performance degradation in real-world deployments.

[0005] Traditional machine learning methods, based on feature engineering (e.g., Hjorth parameter, spectral power) combined with shallow classifiers (e.g., support vector machines, random forests), require manual feature design and have limited ability to distinguish complex sleep stages (e.g., N1). Existing single-channel deep learning models, such as U-Sleep (Perslev et al., 2021) and Stanford-Stage (Stephansen et al., 2018), use convolutional neural networks (CNNs) or long short-term memory (LSTMs) to extract signal features, but their ability to generalize across datasets is weak.

[0006] Existing multimodal fusion deep learning models combine multiple signal inputs such as EEG, EOG, and EMG, and improve performance through end-to-end training. However, they are relatively complex, have large model sizes (usually >100MB), rely mainly on GPU acceleration, and cannot run in real time on low-power devices (e.g., embedded systems and wearable devices).

[0007] In existing automated sleep staging, stage N1 sleep is a typical transition period from wakefulness to sleep. Its EEG characteristics overlap with the frequency domain of other stages (e.g., wakefulness and stage N2), leading to a high misclassification rate between stage N1 and wakefulness and light sleep. The sensitivity of existing algorithms is generally below 50%. Signal anomalies caused by transient events such as body movement and respiration can easily lead to misclassification. The algorithm is also poorly robust to respiratory events and body movement noise, which can easily cause stage jumps and instability. Summary of the Invention

[0008] One advantage of the present application is that it provides a sleep staging method and system based on multimodal signal fusion and context perception, wherein the sleep staging method based on multimodal signal fusion and context perception can not only perform automated sleep staging, reduce manual participation, improve efficiency (single-night processing <5 seconds), and eliminate subjective bias, but also improve sensitivity to transient events, enhance staging continuity, and reduce the misjudgment rate caused by sudden signal fluctuations.

[0009] Another advantage of the present application is that it provides a sleep staging method and system based on multimodal signal fusion and context perception, wherein the sleep staging method based on multimodal signal fusion and context perception has strong generalization ability and can achieve robust staging across populations and devices.

[0010] Another advantage of the present application is that it provides a sleep staging method and system based on multimodal signal fusion and context perception, wherein the sleep staging method based on multimodal signal fusion and context perception is highly sensitive to key staging difficulties (i.e., stage N1).

[0011] Another advantage of the present application is that it provides a sleep staging method and system based on multi-mode signal fusion and context perception, wherein the sleep staging method based on multi-mode signal fusion and context perception adopts a lightweight model and optimized feature engineering, and supports real-time operation on low-power devices.

[0012] According to one aspect of the present application, a sleep staging method based on multimodal signal fusion and context perception is provided, which includes the steps of: Acquiring a multimodal physiological dataset, wherein the multimodal physiological dataset includes an electroencephalogram (EEG) dataset, an electrooculogram (EOG) dataset, and a mentalis muscle electromyography (EMM) dataset; performing multi-scale feature extraction on the multimodal physiological dataset to obtain a multi-scale feature set; fusing the multi-scale feature set based on a lightweight model to obtain a fused feature set; classifying the fused feature set, and obtaining a sleep staging result based on the classification result; The multi-scale feature extraction of the multimodal physiological dataset includes: Segmenting the multimodal physiological dataset into a set of frame sequences; Performing time domain feature extraction on the frame sequence set to obtain a time domain feature set; Performing frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set; Performing historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction on the frame sequence set to obtain a context feature set.

[0013] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction are performed on the multimodal physiological data set, including the steps of: calculating the statistical feature values of the current frame and the previous N frames through a forward sliding window to obtain a historical context feature set; calculating the statistical feature values of the simulated future frames through a centrally symmetric triangular weighted sliding window to obtain a simulated future context feature set; normalizing the time to obtain normalized time and then obtaining a timestamp feature set, and marking the time period of the current frame based on the normalized time.

[0014] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, the window length of the forward sliding window is 2 minutes, N is 4; the statistical feature value includes the feature mean and standard deviation value.

[0015] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, the centrally symmetric triangular weighted sliding window covers the current frame, historical adjacent frames and historical edge frames, the current frame, the historical adjacent frames and the historical edge frames are continuous, and the time domain statistical features, nonlinear features and Hjorth parameter features of the multimodal physiological data set are extracted, and the centrally symmetric triangular weighted sliding window is used to calculate the statistical feature values of the simulated future frames to obtain the simulated future context features, including the steps of: obtaining the weights of the current frame, the historical adjacent frames and the historical edge frames, the weight value of the current frame is greater than the weight value of the historical adjacent frames, and the weight value of the historical adjacent frames is greater than the weight value of the historical edge frames; calculating the weighted sum of the current frame, the historical adjacent frames and the historical edge frames to obtain the statistical feature values of the simulated future frames and thus obtain a simulated future context feature set.

[0016] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, time domain feature extraction is performed on the frame sequence set to obtain a time domain feature set, including the steps of: performing time domain statistical feature extraction on the frame sequence set; performing nonlinear feature extraction on the frame sequence set; and performing Hjorth parameter feature extraction on the frame sequence set.

[0017] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, the statistical features include standard deviation, interquartile range and root mean square; the nonlinear features include permutation entropy and Petrosian fractal dimension; and the Hjorth parameters include activity and complexity.

[0018] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, frequency domain feature extraction is performed on the frame sequence set to obtain a frequency domain feature set, including the steps of: performing spectrum division on the frame sequence set; performing band division on the frame sequence set; and calculating the absolute power of each spectrum segment and the relative power of each band based on the spectrum division results and the band division results.

[0019] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, obtaining a multimodal physiological dataset includes the steps of: collecting an initial physiological dataset, wherein the initial physiological dataset includes an initial EEG dataset, an initial EOG dataset, and an initial mental muscle EMG dataset; preprocessing the initial physiological dataset to obtain the multimodal physiological dataset; wherein preprocessing the initial physiological dataset to obtain the multimodal physiological dataset includes the steps of: downsampling, bandpass filtering, and power frequency interference suppression of the initial physiological dataset.

[0020] In one embodiment of the sleep staging method based on multimodal signal fusion and context perception described in the present application, the fused feature set is classified, and a sleep staging result is obtained based on the classification result, including the steps of: obtaining a preliminary sleep staging result based on the classification result; calculating the confidence of the preliminary sleep staging result; if the confidence of the preliminary sleep staging result is greater than or equal to a preset value, using the preliminary sleep staging result as the sleep staging result.

[0021] According to another aspect of the present application, the present application proposes a sleep staging system based on multimodal signal fusion and context perception, which includes: A physiological data acquisition unit, configured to acquire a multimodal physiological data set, wherein the multimodal physiological data set includes an electroencephalogram (EEG) data set, an electrooculogram (EOG) data set, and a mental muscle electromyography (EMG) data set; a multi-scale feature extraction unit, configured to perform multi-scale feature extraction on the multi-modal physiological dataset to obtain a multi-scale feature set; A feature fusion unit, configured to fuse the multi-scale feature set based on a lightweight model to obtain a fused feature set; a result output unit, configured to classify the fused feature set and obtain a sleep staging result based on the classification result; Wherein, the multi-scale feature extraction unit includes: a frame sequence set generating unit, configured to segment the multimodal physiological data set into frame sequence sets; A time domain feature extraction unit, configured to extract time domain features from the frame sequence set to obtain a time domain feature set; A frequency domain feature extraction unit, configured to perform frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set; The context feature extraction unit is used to perform historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction on the frame sequence set to obtain a context feature set.

[0022] Further objectives and advantages of the present application will be fully reflected through understanding of the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 The figure illustrates a flowchart of a sleep staging method based on multimodal signal fusion and context perception according to an embodiment of the present application.

[0025] Figure 2 FIG2 illustrates a flowchart of step S110 of a sleep staging method based on multimodal signal fusion and context awareness according to an embodiment of the present application.

[0026] Figure 3 The figure illustrates a flowchart of step S112 of the sleep staging method based on multimodal signal fusion and context awareness according to an embodiment of the present application.

[0027] Figure 4 FIG2 illustrates a flowchart of step S120 of a sleep staging method based on multimodal signal fusion and context awareness according to an embodiment of the present application.

[0028] Figure 5 The flowchart of step S124 of the sleep staging method based on multimodal signal fusion and context awareness according to an embodiment of the present application is shown.

[0029] Figure 6 The figure shows a structural block diagram of a sleep staging system based on multimodal signal fusion and context perception according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0031] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "a" should not be understood as limiting the number. "Multiple" means greater than or equal to two.

[0032] Although ordinal numbers such as "first," "second," and the like will be used to describe various components, these are not intended to limit those components. The terms are used solely to distinguish one component from another. For example, a first component could be referred to as a second component, and similarly, a second component could be referred to as a first component without departing from the teachings of the present disclosure. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0033] The terms used herein are for the purpose of describing various embodiments only and are not intended to be limiting. As used herein, the singular is intended to include the plural, unless the context clearly indicates otherwise. It will also be understood that the terms "including" and / or "having" when used in this specification specify the presence of a stated feature, number, operation, component, element, or combination thereof, and do not preclude the presence or addition of one or more other features, numbers, operations, components, elements, or combinations thereof.

[0034] like Figures 1 to 5 As shown, a sleep staging method based on multimodal signal fusion and context awareness according to an embodiment of the present application is illustrated. The sleep staging method based on multimodal signal fusion and context awareness includes the following steps: S110, acquiring a multimodal physiological dataset, wherein the multimodal physiological dataset includes an electroencephalogram (EEG) dataset, an electrooculogram (EOG) dataset, and a mental muscle electromyography (EMG) dataset; S120, performing multiscale feature extraction on the multimodal physiological dataset to obtain a multiscale feature set; S130, fusing the multiscale feature set based on a lightweight model to obtain a fused feature set; S140, classifying the fused feature set, and obtaining a sleep staging result based on the classification result.

[0035] Specifically, in step S110, a multimodal physiological data set is acquired. Specifically, Figure 2 As shown, step S110 includes step S111, collecting an initial physiological data set, wherein the initial physiological data set includes an initial EEG data set, an initial EOG data set and an initial mental muscle EMG data set; and step S112, preprocessing the initial physiological data set to obtain the multimodal physiological data set.

[0036] In step S111, EEG leads or electrodes are selected as needed to collect the initial EEG dataset; the initial EOG dataset is collected through the EOG electrodes of the left eye and / or the right eye; and the initial chin EMG dataset is collected through the chin electrodes (for example, chin EMG electrodes above and below the chin, and a spare electrode below the chin).

[0037] Furthermore, when selecting EEG leads or electrodes to collect the initial EEG dataset, the initial EEG dataset can be collected using EEG leads such as C4-M1, F4-M1, C4-M1, and O2-M1. The initial EEG dataset can be collected using only one EEG lead, i.e., single-channel EEG acquisition, or multiple EEG leads can be selected, i.e., multi-channel EEG acquisition. Preferably, the initial EEG dataset is collected using EEG lead C4-M1.

[0038] In one example of this application, a European Data Format (EDF) file from a polysomnography (PSG) device is obtained. The EDF file contains EEG signals for leads C4-M1, left eye EOG, and mentalis muscle EMG (sampling rate 256 Hz). In other examples of this application, standard leads such as C3-M2 and Fpz-Cz can be used in place of C4-M1.

[0039] It is worth mentioning that in step S111, if the device used for sleep monitoring (e.g., portable / wearable device, home device) only supports single-channel EEG to collect brain activity signals to obtain the initial EEG data set, it can be processed in a degraded mode, that is, the initial EEG data set is collected only through single-channel EEG, and in subsequent data processing, only the initial EEG data set collected through single-channel EEG is used as the data basis. In this way, the sleep staging method based on multimodal signal fusion and context perception of the present application supports single / multi-channel input and is suitable for clinical PSG instruments and home wearable devices.

[0040] In step S112, the initial physiological dataset is preprocessed to obtain the multimodal physiological dataset. Specifically, in one embodiment of the present application, in step S112, the initial data values are downsampled, bandpass filtered, and power frequency interference suppressed to obtain the multimodal physiological dataset. The multimodal physiological dataset includes an electroencephalogram (EEG) dataset, an electrooculogram (EOG) dataset, and a mentalis muscle electromyography (EMG) dataset.

[0041] Accordingly, if Figure 3 As shown, step S112 includes the steps of: S1121, downsampling the initial physiological data set to obtain a first preprocessed data set; S1122, bandpass filtering the first preprocessed data set to obtain a second preprocessed data set; S1123, suppressing power frequency interference on the second preprocessed data set to obtain the multimodal physiological data set.

[0042] In step S1121, the initial data values are downsampled to obtain a first preprocessed dataset. Specifically, during the downsampling process, the acquired initial physiological dataset is primarily "decimated" to reduce the sampling rate. For example, at least one of the initial EEG dataset, the initial EO dataset, and the initial mentalis electromyography dataset is "decimated" to obtain the first preprocessed dataset, wherein the first preprocessed dataset includes a first preprocessed EEG dataset, a first preprocessed EO dataset, and a first preprocessed mentalis electromyography dataset.

[0043] In one embodiment of the present application, if the original sampling rate is greater than 100 Hz, a finite impulse response (FIR) anti-aliasing filter is used to downsample to 100 Hz. It should be understood that in other embodiments, downsampling can be performed in other ways, such as using an infinite impulse response (IIR) anti-aliasing filter for downsampling, or using multi-stage downsampling, first downsampling through one anti-aliasing filter and then downsampling through another anti-aliasing filter.

[0044] In step S1122, the first preprocessed dataset is bandpass filtered to obtain a second preprocessed dataset. Specifically, the second preprocessed dataset may be bandpass filtered using a bandpass filter on at least one of the first preprocessed EEG dataset, the first preprocessed EO dataset, and the first preprocessed mentalis EMG dataset to obtain the second preprocessed dataset, wherein the second preprocessed dataset includes the second preprocessed EEG dataset, the second preprocessed downsampled EO dataset, and the second preprocessed mentalis EMG dataset.

[0045] A bandpass filter of appropriate type and cutoff frequency may be selected to perform bandpass filtering on at least one of the first preprocessed EEG dataset, the first preprocessed EOG dataset, and the first preprocessed mentalis muscle EMG dataset.

[0046] In one embodiment of the present application, an IIR filter is used to perform a 0.4–30 Hz bandpass filtering on the initial EEG dataset to remove low-frequency drift and high-frequency noise. It should be understood that other bandpass filtering methods can also be used, such as fast Fourier transform frequency domain filtering, or cascading a bandpass (0.4–30 Hz) and power frequency notch filter. To focus on specific sleep events (such as spindles), the frequency band can be adjusted to 12–16 Hz (σ waves), requiring simultaneous optimization of feature combinations. Adaptive filters (such as the LMS algorithm) can be used to dynamically remove motion artifacts, but this requires increased computational resources.

[0047] In step S1123, power frequency interference suppression is performed on the second preprocessed dataset to obtain the multimodal physiological dataset. Specifically, a suitable filter can be used to filter at least one of the second preprocessed EEG dataset, the second preprocessed downsampled EO dataset, and the second preprocessed mentalis EMG dataset to obtain the multimodal physiological dataset.

[0048] In one embodiment of the present application, a 50 / 60 Hz adaptive notch filter is used to eliminate power supply interference and retain high-frequency details of the signal.

[0049] In step S120, multiscale feature extraction is performed on the multimodal physiological dataset to obtain a multiscale feature set. It is worth noting that existing automated staging methods are highly sensitive to signal anomalies / fluctuations caused by transient events (such as body movement and respiratory events). These transient events can easily interfere with model judgment, leading to unstable staging results and frequent, physiologically inconsistent jumps (for example, brief artifacts may cause the model to misclassify deep sleep as wakefulness). This leads to poor continuity and insufficient robustness in staging results.

[0050] Sleep staging is essentially a sequential process. The current sleep stage depends not only on the signal characteristics at the current moment but also on previous and subsequent sleep states. To improve the stability and continuity of staging, the model needs to be able to "perceive" the "contextual" information surrounding the current time point.

[0051] Specifically, if Figure 4 As shown, step S120 includes the following steps: S121, dividing the multimodal physiological data set into a frame sequence set; S122, performing time domain feature extraction on the frame sequence set to obtain a time domain feature set; S123, performing frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set; S124, performing context feature extraction on the frame sequence set to obtain a context feature set.

[0052] In step S121, the multimodal physiological data set is segmented into a frame sequence set. Specifically, the time length of each frame can be designed according to needs. In one embodiment of the present application, the multimodal physiological data set is segmented according to a time length of 30 seconds. The EEG data set is segmented into an EEG frame sequence, the EOG data set is segmented into an EOG frame sequence, and the mental muscle EMG data set is segmented into a mental muscle EMG frame sequence. Accordingly, the frame sequence set includes an EEG frame sequence, an EOG frame sequence, and a mental muscle EMG frame sequence. The EEG frame sequence includes multiple frames of EEG data; the EOG frame sequence includes multiple frames of EOG data; and the mental muscle EMG frame sequence includes multiple frames of mental muscle EMG data.

[0053] In step S122, time domain feature extraction is performed on the multimodal physiological dataset to obtain a time domain feature set. Specifically, the time domain statistical features, nonlinear features, and Hjorth parameter features of the multimodal physiological dataset are extracted. The time domain statistical features include standard deviation, interquartile range, and root mean square; the nonlinear features include permutation entropy (window length = 30 seconds, embedding dimension = 5) and Petrosian fractal dimension; the Hjorth parameters include activity, mobility, and complexity. It should be understood that time domain features also include other types, such as peak-to-peak value, waveform length / line length, number of slope sign changes, mean absolute value, zero crossing rate, and autocorrelation features. In one example of the present application, 14 types of time domain features are extracted.

[0054] Correspondingly, time domain feature extraction is performed on the frame sequence set, that is, time domain feature extraction is performed on the EEG frame sequence, the EGO frame sequence, and the mentalis EMG frame sequence, to obtain EEG time domain features, EGO frame time domain features, and mentalis EMG time domain features. The EEG time domain features include multiple EEG standard deviations, EEG interquartile ranges, EEG root mean squares, EEG permutation entropies, EEG Petrosian fractal dimensions, EEG activity, EEG mobility, and EEG complexity of multiple frames of EEG data. The EGO frame time domain features include multiple EEG standard deviations, EEG interquartile ranges, EEG mean squares, EEG root permutation entropies, EEG Petrosian fractal dimensions, EEG activity, EEG mobility, and EEG complexity of multiple frames of EGO data. The mental muscle electromyography time domain features include multiple mental muscle electromyography standard deviations, mental muscle electromyography interquartile ranges, mental muscle electromyography mean squares, mental muscle electromyography root permutation entropy, mental muscle electromyography Petrosian fractal dimension, mental muscle electromyography activity, EEG mobility, and mental muscle electromyography complexity of multiple frames of mental muscle electromyography data.

[0055] In step S123, frequency domain feature extraction is performed on the frame sequence set to obtain a frequency domain feature set. Specifically, in one embodiment of the present application, the frame sequence set is spectrally segmented, and the Welch method is used to generate a power spectral density (PSD). The segmentation parameters for the success rate spectral density are: Hamming window, window length 500 points (5 seconds), no overlap. The frame sequence set is then segmented into bands: δ waves: 1–4 Hz, θ waves: 4–8 Hz, α waves: 8–12 Hz, σ waves (sleep spindles): 12–16 Hz, and β waves: 16–30 Hz. Based on the spectral segmentation and band segmentation results, the absolute power (unit: μV² / Hz) of each spectral segment, the relative power of each band (the ratio of band power to total power), and the cross-band power ratio (e.g., δ / θ, θ / σ) are calculated. If only a single-channel EEG is available, compensating for signal loss by increasing the frequency domain feature dimension (e.g., wavelet transform) reduces accuracy by approximately 5%. It should be understood that frequency domain features also include other types, such as peak frequency, spectral entropy, etc. In an example of the present application, nine types of frequency domain features are extracted.

[0056] Correspondingly, frequency domain feature extraction is performed on the frame sequence set, that is, frequency domain feature extraction is performed on the EEG frame sequence, the electrooculogram frame sequence and the mental muscle electromyography frame sequence, and EEG frequency domain features, EEG frame frequency domain features and mental muscle electromyography frequency domain features can be obtained. The EEG frequency domain features include the absolute power, relative power and cross-band power ratio of multiple spectral segments of each frame of EEG data. The electrooculogram frame frequency domain features include the absolute power, relative power and cross-band power ratio of multiple spectral segments of each frame of electrooculogram data. The mental muscle electromyography frequency domain features include the absolute power, relative power and cross-band power ratio of multiple spectral segments of each frame of mental muscle electromyography data.

[0057] In the EEG data, the absolute power of the δ wave in stage N3 is the highest, and the relative power of the δ wave is extremely high; the δ / β cross-band power ratio is relatively high, and the (δ+θ) / (β+α) cross-band power is relatively high; the absolute power of the σ wave in stage N2 is relatively high, and the (δ+θ) / (β+α) cross-band power is relatively high; the (δ+θ) / (β+α) cross-band power in stage N1 is relatively high, and the θ / α cross-band power is relatively high; the absolute power of the α wave in the wakeful period is high, the relative power of the α wave is high, the δ / β cross-band power ratio is relatively low, and the (δ+θ) / (β+α) cross-band power is relatively low; the absolute power of the θ wave and β wave in the REM period may be relatively high, the relative power of the θ wave and β wave may be relatively high, the δ / β cross-band power ratio is relatively low, the (δ+θ) / (β+α) cross-band power is relatively low, and the θ / α cross-band power is relatively high.

[0058] The absolute power of EO or mental muscle EMG data reflects the energy of eye movement or mental muscle EMG activity within the corresponding frequency band. The absolute power of each frequency band in REM mental muscle EMG data is extremely low. The relative power of EO or mental muscle EMG data helps to identify which frequency band contributes most to the overall activity. The cross-band power ratio of EO or mental muscle EMG data is primarily used to compare the relative strength of low- and high-frequency activity.

[0059] In step S124, contextual feature extraction is performed on the frame sequence set to obtain a contextual feature set. This application introduces context-aware features (historical sliding window and temporal smoothing) to enhance the continuity of the periodization and reduce the misjudgment rate caused by sudden signal fluctuations. Specifically, contextual sliding window features are calculated based on the current frame timestamp. Historical contextual feature extraction, simulated future contextual feature extraction, and timestamp feature extraction are performed on the frame sequence set to obtain a contextual feature set.

[0060] Accordingly, if Figure 5 As shown, step S124 includes the following steps: S1241, performing historical context feature extraction on the frame sequence set to obtain a historical context feature set; S1242, performing simulated future context feature extraction on the frame sequence set to obtain a simulated future context feature set; S1243, performing timestamp feature extraction on the frame sequence set to obtain a timestamp feature set; and S1244, combining the historical context feature set, the future context feature set, and the timestamp feature set to form the context feature set. In other words, the context feature set includes the historical context feature set, the future context feature set, and the timestamp feature set.

[0061] Specifically, in step S1241, historical context feature extraction is performed on the frame sequence set to obtain a historical context feature set. Specifically, statistical feature values, such as feature mean and feature standard deviation, of the current frame and the previous N frames are calculated using a forward sliding window to obtain historical context features. The statistical feature values include statistical feature values of the time domain feature set and statistical feature values of the frequency domain feature set.

[0062] The window length of the forward sliding window and the value of N are designed according to requirements. In one example of the present application, the window length of the forward sliding window is 2 minutes and N is 4. Accordingly, the current frame and the previous N frames total 5 frames. In this example, the historical context feature set is obtained by calculating the statistical feature values of the 5 frames of multimodal physiological data.

[0063] Below, taking the statistical characteristic value as the characteristic mean, the time domain characteristic as the standard deviation, the frequency characteristic as the absolute power, N as 4, and the frame sequence set including the EEG frame sequence, the electrooculogram frame sequence and the mental muscle electromyography frame sequence as an example, the calculation process of the statistical characteristic values of the current frame and the previous N frames is exemplified.

[0064] First, obtain the EEG time domain standard deviation, EGO frame time domain standard deviation, mentalis EMG time domain standard deviation, the absolute power of each EEG spectrum segment, the absolute power of each EGO spectrum segment, and the absolute power of each mentalis EMG spectrum segment between the current frame and the previous four frames.

[0065] Then, the mean of the EEG time domain standard deviation, the mean of the EGO frame time domain standard deviation, the mean of the mentalis EMG time domain standard deviation, the mean of the absolute power of each EEG spectrum segment, the mean of the absolute power of each EGO spectrum segment, and the mean of the absolute power of each mentalis EMG spectrum segment are calculated between the current frame and the previous four frames to obtain the statistical characteristic values of the current frame and the previous four frames. The statistical characteristic values of other time domain features and other frequency domain features of the current frame and the previous four frames can be calculated by referring to the calculation process of the statistical characteristic values of the time domain standard deviation and absolute power of the current frame and the previous four frames.

[0066] Furthermore, the statistical feature values of other time domain features and the statistical feature values of frequency domain features of the current frame and the previous four frames may be calculated to obtain the historical context feature set.

[0067] In step S1242, simulated future context feature extraction is performed on the frame sequence set to obtain a simulated future context feature set. It is worth mentioning that actual future data cannot be obtained in reality, so the feature values of simulated future frames are calculated by simulation to simulate the impact of future frames.

[0068] Specifically, in one embodiment of the present application, a simulated future context feature set is obtained by calculating the statistical feature values of simulated future frames using a center-symmetric triangular weighted sliding window. In the center-symmetric triangular weighted sliding window application, a selected frame is used as the center frame, and two other sets of frames with the same number of consecutive frames as the center frame are used as symmetric frames. The center frame has the highest weight, and the weights of the symmetric frames decrease as the time distance from the center frame increases.

[0069] The total window length of the centrosymmetric triangular weighted sliding window can be designed as needed. In one embodiment of the present application, the total window length of the centrosymmetric triangular weighted sliding window is 7.5 minutes. The length can be fine-tuned (e.g., 5–10 minutes) to suit different scenarios.

[0070] As mentioned above, the actual future data cannot be truly obtained. This application mainly simulates the statistical characteristic values of future frames through historical data frame calculations. It is worth mentioning that, usually, in applications in centrally symmetric triangular weighted sliding windows, two sets of symmetrical frames are temporally distributed on both sides of the central frame. However, in this application, the current frame is taken as the central frame. Since the real future frame does not exist when the current frame is collected in real time, only the data on one side of the current frame can be obtained, that is, the data of the historical frame on the front side of the current frame, and the data of the future frame on the back side of the current frame cannot be obtained. Based on this, this application selects the two sets of frames on one side of the central frame as symmetrical frames.

[0071] Accordingly, the current frame is taken as the central frame, and the historical adjacent frames and the historical edge frames are taken as symmetrical frames, wherein the current frame, the historical adjacent frames and the historical edge frames are continuous, and the number of frames of the historical adjacent frames and the historical edge frames is the same, and the time distance between the historical adjacent frames and the current frame is less than the time distance between the historical edge frames and the current frame.

[0072] In the process of calculating the statistical feature values of the simulated future frame through a centrally symmetric triangular weighted sliding window to obtain a simulated future context feature set, the weight values of the current frame, the historical adjacent frames and the historical edge frames are first obtained; then the weighted sum of the current frame, the historical adjacent frames and the historical edge frames is calculated to obtain the statistical feature values of the simulated future frame and then obtain the simulated future context feature set.

[0073] In an example of the present application, the central symmetric triangular weighted sliding window covers the (k-14)th frame, the (k-13)th frame, ..., the (k-1)th frame, and the kth frame; wherein the kth frame is the current frame; the (k-5)th frame to the (k-1)th frame are historical adjacent frames; and the (k-14)th frame to the (k-6)th frame are the historical edge frames.

[0074] In the process of calculating the weighted sum of the current frame, the historical adjacent frames and the historical edge frames, the weight value of the current frame is greater than the weight value of the historical adjacent frames, and the weight value of the historical adjacent frames is greater than the weight value of the historical edge frames.

[0075] In one example of the present application, the weight value of the kth frame is equal to 1.0; the weight value of the (k-1)th frame is equal to 13 / 14; the weight value of the (k-2)th frame is equal to 12 / 14; the weight value of the (k-3)th frame is equal to 11 / 14; the weight value of the (k-4)th frame is equal to 10 / 14; the weight value of the (k-5)th frame is equal to 9 / 14; the weight value of the (k-6)th frame is equal to 8 / 14; the weight value of the (k-7)th frame is equal to 7 / 14; the weight value of the (k-8) frame is equal to 6 / 14; the weight value of the (k-9) frame is equal to 5 / 14; the weight value of the (k-10) frame is equal to 4 / 14; the weight value of the (k-11) frame is equal to 3 / 14; the weight value of the (k-12) frame is equal to 2 / 14; the weight value of the (k-13) frame is equal to 1 / 14; the weight value of the (k-14) frame is equal to 0 or greater than 0 and less than 1 / 14, for example, 0.1.

[0076] In another example of the present application, the weight value of the kth frame is equal to 1.0; the weight values of the (k-1)th frame, the (k-2)th frame, the (k-3)th frame, the (k-4)th frame, and the (k-5)th frame are 0.5, that is, the weight values of all historical adjacent frames are 0.5; the weight values of the (k-6)th frame, the (k-7)th frame, the (k-8)th frame, the (k-9)th frame, the (k-10)th frame, the (k-11)th frame, the (k-12)th frame, the (k-13)th frame, and the (k-14)th frame are equal to 0.1, that is, the weight values of all historical edge frames are 0.1.

[0077] The statistical feature values of the current frame, the historical adjacent frames and the historical edge frames include statistical feature values of the time domain feature sets and statistical feature values of the frequency domain feature sets of the current frame, the historical adjacent frames and the historical edge frames, wherein the statistical feature values are weighted sums.

[0078] The following example illustrates the calculation process of the statistical characteristic values of the current frame, the historical adjacent frames, and the historical edge frames, taking the case where the statistical characteristic value is a weighted sum, the time domain characteristic is a standard deviation, the frequency characteristic is absolute power, the kth frame is the current frame, the (k-5)th frame to the (k-1)th frame are historical adjacent frames, and the (k-14)th frame to the (k-6)th frame are the historical edge frames.

[0079] First, obtain the EEG time domain standard deviation, EGO frame time domain standard deviation, mentalis EMG time domain standard deviation, the absolute power of each EEG spectrum segment, the absolute power of each EGO spectrum segment, and the absolute power of each mentalis EMG spectrum segment from the (k-14)th frame to the kth frame.

[0080] Then, the weighted sum of the EEG temporal standard deviation, the weighted sum of the EGO temporal standard deviation, the weighted sum of the mentalis EMG temporal standard deviation, the weighted sum of the absolute power of each EEG spectral segment, the weighted absolute power of each EGO spectral segment, and the weighted sum of the absolute power of each mentalis EMG spectral segment are calculated from the (k-14)th frame to the kth frame to obtain the statistical characteristic values of the temporal standard deviation and absolute power from the (k-14)th frame to the kth frame, that is, to simulate the statistical characteristic values of the temporal standard deviation and absolute power of future frames. The statistical characteristic values of other temporal features and other frequency domain features from the (k-14)th frame to the kth frame can be calculated by referring to the calculation process of the statistical characteristic values of the temporal standard deviation and absolute power from the (k-14)th frame to the kth frame.

[0081] Taking the calculation process of the weighted sum of the EEG time domain standard deviation from the (k-14)th frame to the kth frame as an example, the weighted sum of various time domain features or the weighted sum of frequency domain features from the (k-14)th frame to the kth frame is exemplified. The weighted sum of the EEG time domain standard deviations from the (k-14)th frame to the kth frame is equal to the product of the EEG time domain standard deviation of the (k-14)th frame and the weight value of the (k-14)th frame, plus the product of the EEG time domain standard deviation of the (k-13)th frame and the weight value of the (k-13)th frame, plus the product of the EEG time domain standard deviation of the (k-12)th frame and the weight value of the (k-12)th frame, plus the product of the EEG time domain standard deviation of the (k-11)th frame and the weight value of the (k-11)th frame, plus the product of the EEG time domain standard deviation of the (k-10)th frame and the weight value of the (k-10)th frame, plus the product of the EEG time domain standard deviation of the (k-9)th frame and the weight value of the (k-9)th frame, plus the product of the EEG time domain standard deviation of the (k-8)th frame and the weight value of the (k-8)th frame. The product of the time domain standard deviation of the EEG of the (k-7)th frame and the weight value of the (k-7)th frame is added, the product of the time domain standard deviation of the EEG of the (k-6)th frame and the weight value of the (k-6)th frame is added, the product of the time domain standard deviation of the EEG of the (k-5)th frame and the weight value of the (k-5)th frame is added, the product of the time domain standard deviation of the EEG of the (k-4)th frame and the weight value of the (k-4)th frame is added, the product of the time domain standard deviation of the EEG of the (k-3)th frame and the weight value of the (k-3)th frame is added, the product of the time domain standard deviation of the EEG of the (k-2)th frame and the weight value of the (k-2)th frame is added, the product of the time domain standard deviation of the EEG of the (k-1)th frame and the weight value of the (k-1)th frame is added, and the product of the time domain standard deviation of the EEG of the kth frame and the weight value of the kth frame is added.

[0082] It is worth mentioning that this application improves the continuity of installments based on the historical sliding window (2 minutes) and the proposed future sliding window (7.5 minutes).

[0083] The exponentially weighted moving average (EWMA) can be used instead of the triangular weighted sliding average, and the decay factor (α=0.3) can be adjusted to balance the historical and current weights.

[0084] In step S1243, timestamp features are extracted from the frame sequence set to obtain a timestamp feature set. In one embodiment of the present application, time is normalized to obtain normalized time to obtain timestamp features, and the time period of the current frame is marked based on the normalized time.

[0085] In an example of the present application, the entire night time is normalized to 0 to 1 to mark the time period of the current frame (for example, 0 represents the time of falling asleep, and 1 represents waking up in the morning).

[0086] The data for each night were normalized independently, with the 5% to 95% quantiles of all frames of that night as the reference interval, and Z-score standardization was used to avoid interference from individual differences.

[0087] In step S130, the multi-scale feature set is fused based on the lightweight model to obtain a fused feature set. High-precision staging is achieved by combining single-channel EEG, single-channel EOG, and EMG, reducing hardware dependence. Specifically, the base classifier of the lightweight model uses the LightGBM gradient boosting decision tree with the following hyperparameters: <python> params = { 'num_leaves': 31, # Control model complexity 'max_depth': 5, # prevent overfitting 'learning_rate': 0.05, 'n_estimators': 500, # Number of trees 'min_child_samples': 90, # Minimum number of samples for leaf nodes 'class_weight': 'balanced' # Automatically adjust category weights } This application adopts a lightweight model (LightGBM classifier, with a small model size, for example, the model size is less than 2MB), has high computational efficiency, uses optimized feature engineering, supports real-time operation on low-power devices (Raspberry Pi, wearable devices), and can run on embedded systems. It can to a certain extent solve the problems of high automation computational complexity and poor real-time performance of automatic sleep staging technology.

[0088] This application designs a combination of multimodal features (such as EOG high-frequency power and EEG fractal dimension) and adaptive class weighting for the N1 phase, increasing N1 sensitivity to over 45.4%. Specifically, the lightweight model optimizes class weights and proposes a weight enhancement strategy for the N1 phase: manually increasing the N1 weight by 2.2 times (based on the distribution of training data).

[0089] The lightweight model training strategy utilizes multi-dataset fusion training and cross-validation. Specifically, seven heterogeneous datasets (such as NSRR) were combined, with a total sample size of over 50,000 frames, covering healthy individuals, children, and obstructive sleep apnea (OSA) patients. A 5-fold cross-validation based on time series was performed to avoid data leakage from the same subject. Thus, by fusing multiple heterogeneous datasets (training data covering children, adults, and OSA patients) and combining them with adaptive signal fusion technology, this application achieves robust staging across populations, for example, covering a wide age range of 5-95 years old, and across devices. This approach can, to a certain extent, address the poor generalization of automated sleep staging algorithms. In application testing of this application's sleep staging method based on multimodal signal fusion and contextual awareness, staging accuracy reached 87.5% in healthy individuals and 84.3% in OSA patients, respectively, surpassing most existing algorithms. It is fully capable of realizing clinical diagnosis, providing standardized staging tools for sleep disorders (such as insomnia and sleep apnea), reducing diagnosis time and labor costs, supporting large-scale cohort studies (such as analysis of tens of thousands of sleep data), and promoting the intelligent upgrade of home sleep monitoring equipment (such as real-time staging feedback from smart bracelets).

[0090] In this variant implementation, XGBoost or CatBoost replaces LightGBM, requiring re-optimization of tree depth (max_depth ≤ 7) and learning rate (≤ 0.1). If resources permit, a lightweight CNN (such as MobileNetV3) can be used, but this will sacrifice some interpretability.

[0091] In step S140, the fused feature set is classified, and a sleep staging result is obtained based on the classification result. Specifically, a preliminary sleep staging result is obtained based on the classification result; a confidence level of the preliminary sleep staging result is calculated; and if the confidence level of the preliminary sleep staging result is greater than or equal to a preset value, the preliminary sleep staging result is used as the sleep staging result.

[0092] In one example of the present application, the probability of each sleep stage in each frame is output (Softmax normalization), and when the highest probability is less than 80%, it is marked as "low confidence" for manual review.

[0093] In one embodiment of the present application, the preliminary staging results are post-smoothed using a hidden Markov model (HMM) to generate final labels, and isolated transition frames are corrected (eg, Wake→N1→Wake is corrected to Wake).

[0094] In one embodiment of the present application, the sleep staging results are visualized and outputted, specifically, a hypnogram and a SHAP feature contribution graph are generated (for example, showing the positive contribution of EEG delta wave power to the determination of stage N3).

[0095] In the application test of the sleep staging method based on multimodal signal fusion and context perception of this application, automatic sleep staging was achieved, and the judgment time for a single night was less than 5 seconds, which is about 1,000 times more efficient than manual judgment and can eliminate human bias; benefiting from the use of multi-source data set training and cross-population feature normalization, the accuracy rate in OSA patients was increased to 84.3%, which is an improvement compared with the traditional model (traditional model <75%); benefiting from LightGBM+ feature engineering optimization (model volume <2MB), it can run in real time on Raspberry Pi, reducing hardware costs by 90%, and reducing computing resource consumption compared with traditional deep learning models; benefiting from the targeted design of EOG high-frequency features and category weight optimization, the sensitivity to the N1 stage is significantly improved; benefiting from context sliding window perception and time series smoothing, the misjudgment rate of body movement and respiratory events is reduced by more than 20% (SHHS data set comparison test).

[0096] Feature extraction (step S120) and model inference (steps S130 and S140) can be split into a client-server architecture, where the client completes signal preprocessing and uploads features, and the server returns the classification results.

[0097] like Figure 6 As shown, the present application also proposes a sleep staging system based on multimodal signal fusion and context perception. Specifically, the sleep staging system based on multimodal signal fusion and context perception includes a physiological data acquisition unit 10 for acquiring a multimodal physiological data set, wherein the multimodal physiological data set includes an EEG data set, an EOG data set, and a mental muscle EMG data set; a multiscale feature extraction unit 20 for performing multiscale feature extraction on the multimodal physiological data set to obtain a multiscale feature set; a feature fusion unit 30 for fusing the multiscale feature set based on a lightweight model to obtain a fused feature set; and a result output unit 40 for classifying the fused feature set and obtaining a sleep staging result based on the classification result. The multi-scale feature extraction unit 20 includes a frame sequence set generation unit 21, which is used to divide the multimodal physiological data set into frame sequence sets; a time domain feature extraction unit 22, which is used to perform time domain feature extraction on the frame sequence set to obtain a time domain feature set; a frequency domain feature extraction unit 23, which is used to perform frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set; and a context feature extraction unit 24, which is used to perform historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction on the frame sequence set to obtain a context feature set.

[0098] In summary, the sleep staging method and system based on multimodal signal fusion and contextual awareness are described. The sleep staging method based on multimodal signal fusion and contextual awareness not only automates sleep staging, reduces manual intervention, improves efficiency (processing time per night is less than 5 seconds), and eliminates subjective bias, but also improves sensitivity to transient events, enhances staging continuity, and reduces the misjudgment rate caused by sudden signal fluctuations.

[0099] The above description of the present application and its embodiments is non-limiting. The drawings show only one embodiment of the present application, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the inventive purpose of this application, designs a structure and embodiment similar to this technical solution without creatively designing, they shall fall within the scope of protection of this application.< / python>

Claims

1. A sleep staging method based on multimodal signal fusion and context awareness, characterized in that: Including steps: Acquiring a multimodal physiological dataset, wherein the multimodal physiological dataset includes an electroencephalogram (EEG) dataset, an electrooculogram (EOG) dataset, and a mental muscle electromyography (EMM) dataset; performing multi-scale feature extraction on the multimodal physiological dataset to obtain a multi-scale feature set; fusing the multi-scale feature set based on a lightweight model to obtain a fused feature set; classifying the fused feature set, and obtaining a sleep staging result based on the classification result; The multi-scale feature extraction of the multimodal physiological dataset includes: Segmenting the multimodal physiological dataset into a set of frame sequences; Performing time domain feature extraction on the frame sequence set to obtain a time domain feature set; Performing frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set; Performing historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction on the frame sequence set to obtain a context feature set.

2. The sleep staging method based on multimodal signal fusion and context awareness according to claim 1, characterized in that: Performing historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction on the multimodal physiological dataset includes the following steps: Calculate the statistical feature values of the current frame and the previous N frames through the forward sliding window to obtain the historical context feature set; Calculating the statistical feature values of the simulated future frame through a central symmetric triangular weighted sliding window to obtain a simulated future context feature set; The time is normalized to obtain the normalized time and then the timestamp feature set is obtained, and the time period of the current frame is marked based on the normalized time.

3. The sleep staging method based on multimodal signal fusion and context awareness according to claim 2, characterized in that: The window length of the forward sliding window is 2 minutes, and N is 4; the statistical characteristic value includes a characteristic mean and a standard deviation value.

4. The sleep staging method based on multimodal signal fusion and context awareness according to claim 2, characterized in that: The central symmetric triangular weighted sliding window covers a current frame, a historical adjacent frame, and a historical edge frame, the current frame, the historical adjacent frame, and the historical edge frame are continuous, and the number of frames of the historical adjacent frame and the historical edge frame is the same, the time distance between the historical adjacent frame and the current frame is less than the time distance between the historical edge frame and the current frame, and the central symmetric triangular weighted sliding window is used to calculate the statistical feature value of the simulated future frame to obtain the simulated future context feature, including the steps of: Obtaining weights of the current frame, the historical adjacent frames, and the historical edge frames, wherein the weight value of the current frame is greater than the weight value of the historical adjacent frames, and the weight value of the historical adjacent frames is greater than the weight value of the historical edge frames; A weighted sum of the current frame, the historical adjacent frames, and the historical edge frames is calculated to obtain a statistical feature value of a simulated future frame and thus obtain a simulated future context feature set.

5. The sleep staging method based on multimodal signal fusion and context awareness according to claim 2, characterized in that: Performing time domain feature extraction on the frame sequence set to obtain a time domain feature set includes the steps of: extracting time-domain statistical features of the frame sequence set; extracting nonlinear features of the frame sequence set; Extracting Hjorth parameter features of the frame sequence set.

6. The sleep staging method based on multimodal signal fusion and context awareness according to claim 5, characterized in that: Time domain statistical features include standard deviation, interquartile range and root mean square; nonlinear features include permutation entropy and Petrosian fractal dimension; Hjorth parameter features include activity and complexity.

7. The sleep staging method based on multimodal signal fusion and context awareness according to claim 2, characterized in that: Performing frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set includes the steps of: Performing spectrum division on the frame sequence set; Performing band division on the frame sequence set; The absolute power of each spectrum segment and the relative power of each band are calculated based on the spectrum division results and the band division results.

8. The sleep staging method based on multimodal signal fusion and context awareness according to claim 1, characterized in that: Acquiring a multimodal physiological dataset includes the following steps: Acquiring an initial physiological data set, wherein the initial physiological data set includes an initial EEG data set, an initial EOG data set, and an initial mental muscle EMG data set; The initial physiological data set is preprocessed to obtain the multimodal physiological data set; wherein, the preprocessing of the initial physiological data set to obtain the multimodal physiological data set includes the steps of: downsampling, bandpass filtering and power frequency interference suppression on the initial physiological data set.

9. The sleep staging method based on multimodal signal fusion and context awareness according to claim 1, characterized in that: Classifying the fused feature set and obtaining a sleep staging result based on the classification result includes the following steps: Based on the classification results, preliminary sleep staging results were obtained; Calculate the confidence level of the preliminary sleep staging results; If the confidence level of the preliminary sleep staging result is greater than or equal to a preset value, the preliminary sleep staging result is used as the sleep staging result.

10. A sleep staging system based on multimodal signal fusion and context perception, characterized in that: include: A physiological data acquisition unit, configured to acquire a multimodal physiological data set, wherein the multimodal physiological data set includes an electroencephalogram (EEG) data set, an electrooculogram (EOG) data set, and a mental muscle electromyography (EMG) data set; a multi-scale feature extraction unit, configured to perform multi-scale feature extraction on the multi-modal physiological dataset to obtain a multi-scale feature set; A feature fusion unit, configured to fuse the multi-scale feature set based on a lightweight model to obtain a fused feature set; a result output unit, configured to classify the fused feature set and obtain a sleep staging result based on the classification result; Wherein, the multi-scale feature extraction unit includes: a frame sequence set generating unit, configured to segment the multimodal physiological data set into frame sequence sets; A time domain feature extraction unit, configured to extract time domain features from the frame sequence set to obtain a time domain feature set; A frequency domain feature extraction unit, configured to perform frequency domain feature extraction on the frame sequence set to obtain a frequency domain feature set; The context feature extraction unit is used to perform historical context feature extraction, simulated future context feature extraction, and timestamp feature extraction on the frame sequence set to obtain a context feature set.

Images