A missing modality sleep staging method and system based on latent space distribution alignment

Abstract

This invention relates to the fields of biomedical signal processing, intelligent sleep monitoring, and multimodal machine learning, specifically to a method and system for sleep staging with missing modalities based on latent space distribution alignment. The method includes: acquiring complete EEG, EOG, and EMG data and preprocessing them; encoding single-modal features respectively; mapping each single-modal feature to a unified latent space to generate Gaussian latent distributions for each single modality; constructing a joint latent distribution based on the complete trimodal information; aligning the distributions by minimizing the KL divergence between each single-modal latent distribution and the joint latent distribution to establish a shared latent space; training a classifier based on the joint latent distribution; and, during the testing phase when modalities are missing, using only the latent distributions generated from the observable modalities for classification. This invention maintains high robustness and high accuracy in sleep staging even under modality-missing conditions, making it suitable for applications prone to modality loss, such as clinical monitoring, home sleep detection, and wearable devices.

Description

Technical Field

[0001] This invention relates to the fields of biomedical signal processing, intelligent sleep monitoring, and multimodal machine learning, and particularly to a method and system for sleep staging of missing modalities based on latent space distribution alignment. Background Technology

[0002] Sleep staging is a crucial step in sleep medicine analysis. Its goal is to divide the sleep process into different stages, such as wakefulness, REM sleep, and various non-REM sleep stages, providing a basis for sleep quality assessment and disorder diagnosis. Currently, polysomnography (PSG) is widely used in clinical practice and research, with electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG) being the three most commonly used physiological modalities. Because these three modalities are highly complementary in sleep stage identification, combined sleep staging based on these three modalities has become an important area of ​​development.

[0003] Existing automatic sleep staging technologies are mainly divided into two categories: The first category is end-to-end automatic staging methods based on single-modal EEG. These methods typically employ one-dimensional convolution, recurrent neural networks, or self-attention mechanisms to extract and model features from a single time-series signal. Their advantage lies in their clear structure, but their disadvantage is that relying solely on EEG signals makes it difficult to fully utilize the complementary discriminative information contained in EOG and EMG, thus limiting the accuracy in distinguishing stages such as Rapid Eye Movement (REM) and Light Sleep (N1).

[0004] The second category is automatic sleep staging methods based on complete multimodal fusion. These methods typically collect and preprocess EEG, EOG, and EMG signals simultaneously, then perform multimodal fusion through feature splicing, weighted fusion, or attention mechanisms to improve staging accuracy. For example, Chinese patent CN115040072A discloses a multimodal sleep staging method based on dynamic fusion, which constructs multiple neural network branches to process different types of data information and uses constrained weights for fusion. However, these methods generally implicitly rely on a key assumption: the input modal conditions must be consistent between the training and testing phases, requiring the complete presence of EEG, EOG, and EMG signals. Similarly, Chinese patent CN115500843B discloses a sleep staging method based on zero-shot learning and contrastive learning, which also relies on obtaining complete EEG, EOG, and EMG signals during both the training and testing phases.

[0005] However, in real-world applications, due to sensor detachment, poor contact, artifact contamination, or device channel limitations, partial modalities are easily missing during the testing phase. For example, only EEG may be collected, or both EOG and EMG may be missing simultaneously. In such cases, existing models trained on complete modal inputs will experience significant performance degradation. To address this issue, some studies have attempted simple strategies such as zero-padding, interpolation, or training a separate model for each missing combination. However, these methods struggle to effectively recover discriminative information from missing modalities, resulting in poor model generalization and high maintenance costs.

[0006] In recent years, cross-modal alignment methods based on latent variable modeling have shown potential in cross-modal retrieval and generation. The core idea is to map different modalities to a unified latent space and learn shared semantics between modalities by constraining the consistency of latent variable distributions. However, existing research has not yet applied this idea to address the missing modality problem in sleep staging.

[0007] In summary, how to utilize readily available complete trimodal data during the training phase to learn a shared representation space, enabling the model to stably and accurately generate discriminative latent features and achieve high-quality sleep staging even when dealing with various modality loss scenarios during the testing phase, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The present invention aims to solve the problem that existing multimodal sleep staging methods suffer from severe performance degradation during the testing phase due to the lack of modalities.

[0009] To achieve this objective, the present invention provides the following technical solution: On one hand, the present invention provides a method for sleep staging of missing modalities based on latent space distribution alignment, comprising the following steps: S1: Training Data Construction and Preprocessing Complete sleep monitoring data including three modalities: electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG) is acquired. The complete sleep monitoring data is preprocessed and segmented to obtain multiple training samples, each training sample corresponding to a sleep stage label. S2: Single-modal feature encoding The training samples for each modality were subjected to depth feature extraction using EEG encoder, EOG encoder, and EMG encoder respectively, to obtain unimodal feature representations. , and ; S3: Single-modal Gaussian latent distribution modeling Each single-modal feature representation is mapped to a unified latent space, and the mean vector corresponding to each single mode is output through the latent distribution parameter generation module. Covariance parameter Obtain the Gaussian latent distribution for each single mode:

[0010]

[0011]

[0012] in, N(·) represents the Gaussian distribution function; To share latent variables; These are the latent posterior probability distributions of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the input samples corresponding to the Gaussian latent distributions of the EEG, EOG, and EMG modes, respectively. These are the mean vectors of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the covariance matrix parameters of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. S4: Construction of complete trimodal joint latent distribution Based on the complete information of the three modes EEG, EOG, and EMG, a complete trimodal joint latent distribution is constructed:

[0013] in, This is the mean vector corresponding to the complete three-modal joint latent distribution; The covariance parameter corresponding to the complete three-modal joint latent distribution; It is a complete trimodal joint latent distribution constructed based on complete information from EEG, EOG, and EMG modalities.

[0014] S5: Latent Space Distribution Alignment By minimizing the distribution difference between each unimodal Gaussian latent distribution and the complete trimodal joint latent distribution, each unimodal Gaussian latent distribution is aligned to the complete trimodal joint latent distribution to establish a shared latent space; the distribution difference is measured using at least one of KL divergence, JS divergence, or Wasserstein distance. S6: Classifier Training A sleep staging classifier is trained based on the complete trimodal joint latent distribution; S7: Missing Modal Reasoning in the Testing Phase During the testing phase, when a modality is missing, only the currently observable modality is input into the corresponding encoder to generate the observable modality Gaussian latent distribution, and the sleep staging result is output based on the shared latent space and the sleep staging classifier.

[0015] Furthermore, the distribution difference is expressed using KL divergence, and the alignment loss function is:

[0016] in, The latent space distribution alignment loss is used to measure the distribution difference between each single-modal latent distribution and the complete trimodal joint latent distribution. This is the Kullback-Leibler divergence, used to measure the degree of difference between two probability distributions.

[0017] Furthermore, the complete trimodal joint latent distribution is obtained through a hybrid expert model fusion method: the three unimodal feature representations are input into the attention gating network of the hybrid expert model, three weight coefficients are output, the fused features are calculated, and then the mean vector and covariance parameter of the complete trimodal joint latent distribution are output through a linear layer.

[0018] Furthermore, the total loss function for training the sleep staging classifier is: ;in, Loss due to sleep stage classification; The latent space distribution alignment loss; For distribution regularization; and The weight coefficients are used; the classifier consists of two fully connected layers and a Softmax output layer, with the two fully connected layers containing 128 neurons and 64 neurons respectively.

[0019] Furthermore, the EEG encoder, EOG encoder, and EMG encoder each independently employ a combination of one-dimensional convolutional networks, multi-scale convolutional structures, and Mamba networks.

[0020] Furthermore, the covariance matrix of each single-mode Gaussian latent distribution adopts a diagonal covariance form.

[0021] Furthermore, during the testing phase, if multiple observable modalities exist, the Gaussian latent distributions of each observable modality are fused to obtain the joint latent distribution of the observed modalities, which is then input into the sleep staging classifier. The fusion adopts any one of weighted averaging, attention-based dynamic fusion, or uncertainty-weighted fusion.

[0022] Furthermore, the preprocessing includes: applying a 0.5~30Hz bandpass filter to the EEG signal, a 0.5~10Hz bandpass filter to the EOG signal, and a 10~100Hz bandpass filter to the EMG signal; resampling to 100Hz, with each segment lasting 30 seconds and containing 3000 sampling points, and normalizing using z-score standardization.

[0023] On the other hand, the present invention provides a sleep staging system for missing modalities based on latent space distribution alignment, for performing the above method, including: The data acquisition module is used to acquire EEG, EOG, and EMG signals; The data preprocessing module is used to filter, resample, segment, and standardize the acquired signals. The single-modal coding module uses EEG encoders, EOG encoders, and EMG encoders to extract deep features from each modal signal; The latent distribution modeling module is used to map the features of each modality to a unified latent space and output the Gaussian latent distribution parameters of each modality. A joint latent distribution construction module is used to construct a reference joint latent distribution based on complete trimodal samples during the training phase; A latent space distribution alignment module is used to constrain the consistency between each single-modal latent distribution and the joint latent distribution; The classification training module is used to learn the sleep stage discrimination boundaries in the shared latent space; The missing modality reasoning module is used to generate latent representations and perform classification based on the currently observable modalities during the testing phase; The results output module is used to output the sleep stage recognition results.

[0024] Preferably, the system further includes a model update module for incremental learning or fine-tuning the model using newly collected online data; the missing modality inference module is configured to automatically identify the type and number of input modalities and dynamically select one of the following inference paths: when a complete three-modality is detected, a joint distribution path is used; when two observable modalities are detected, a fusion path is used; when a single modality is detected, the modality distribution path is directly used.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention utilizes complete EEG, EOG and EMG three-modal samples during the training phase to construct a unified shared latent space, which can learn the complementary relationship between modalities and sleep stage discrimination features more fully than the single-modal EEG method, thereby improving the model's ability to represent complex sleep stage boundaries.

[0026] (2) This invention establishes a unified latent semantic mapping relationship between complete modalities and missing modalities by modeling different modalities as Gaussian distributions in a shared latent space and aligning the latent distributions of single or partial modalities to the joint latent distribution of complete trimodalities. Therefore, even when modalities are missing during the testing phase, it is still possible to obtain latent features that are close to the joint representation of complete modalities based on observable modalities, thereby maintaining high sleep staging performance.

[0027] (3) This invention does not rely on complete modal input during the testing phase, and can effectively adapt to real application environments such as sensor detachment, poor contact, partial lead loss, limited equipment resources, and unstable data acquisition in home scenarios, thus having stronger robustness and higher practical value.

[0028] (4) The present invention does not require training an independent classification model for each missing modality combination, nor does it rely on simple zero-padding, interpolation or empirical feature substitution, thus reducing model maintenance costs and system implementation complexity.

[0029] (5) The present invention is compatible with common front-end processing modules of existing single-modal and multi-modal automatic sleep staging systems, and has good scalability and portability. Attached Figure Description

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will now be described in detail with reference to the accompanying drawings, wherein... Figure 1 This is a schematic diagram of the overall framework of the missing modality sleep staging method based on latent space distribution alignment of the present invention. Figure 2 This is a schematic diagram of the training process of the method of the present invention; Figure 3 This is a schematic diagram of the training process of the method of the present invention; Figure 4 This is a schematic diagram of the test reasoning process of the method of the present invention; Figure 5 This is a schematic diagram of the test inference process of the method of the present invention; wherein, (a) is the inference process when one modality is missing, (b) is the inference process when only one modality exists, and (c) is the inference process when the complete modality is input; Figure 6 This is a schematic diagram of the missing modality sleep staging system based on latent space distribution alignment according to the present invention. Detailed Implementation

[0031] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures, and should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0033] Example 1 This embodiment validates the proposed missing modality sleep staging method based on latent space distribution alignment on the publicly available ISRUC-S3 and ISRUC-S1 datasets. Parameter information for the two datasets is shown in Table 1, and the number of epochs for each sleep stage is shown in Table 2.

[0034] Table 1

[0035] Table 2

[0036] S1: Training Data Construction and Preprocessing See Figure 3 Complete sleep monitoring data including EEG, EOG, and EMG modalities were obtained. Continuous sleep recordings were divided into 30-second data segments according to the AASM rule, with each segment corresponding to a sleep stage label. These labels included one of the following: wakefulness (W), light sleep (N1), light sleep (N2), deep sleep (N3), and REM sleep.

[0037] The preprocessing is as follows: EEG, EOG, and EMG signals are filtered, resampled to 100Hz, normalized, and artifact-processed to obtain time-aligned trimodal inputs. Specifically, a 0.5–30Hz bandpass filter is used for EEG signals to remove low-frequency drift and high-frequency noise; a 0.5–10Hz bandpass filter is used for EOG signals to preserve eye-tracking features; and a 10–100Hz bandpass filter is used for EMG signals to preserve electromyographic activity features.

[0038] After resampling, the sampling rate of all modalities was uniformly set to 100Hz, and each 30-second segment contained 3000 sampling points. z-score normalization was used to normalize each modality.

[0039] S2: Single-modal feature encoding See Figure 1 , Figure 3 EEG encoders, EOG encoders, and EMG encoders were constructed to extract deep features from the three modal signals. Each encoder was implemented using a combination of a one-dimensional convolutional network, a multi-scale convolutional structure, and a Mamba network. As a variation, the encoder could also use any one or more combinations of ResNet, Temporal Convolutional Network (TCN), or Transformer encoder, as long as it could extract discriminative intra-modal features.

[0040] Let the three input modes be as follows: , and After passing through their respective encoders, the modal feature representations obtained are as follows: , and These features are not directly used for the final classification, but rather serve as input to the latent space Gaussian distribution parameter generation module.

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047] in, These are the mean vectors of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the covariance matrix parameters of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the linear transformation weight matrices that map the EEG, EOG, and EMG modes to the latent distribution mean vectors, respectively. , respectively, are linear transformation weight matrices that map the EEG, EOG, and EMG modes to the latent distribution log-covariance parameters.

[0048] S3: Single-modal Gaussian latent distribution modeling See Figure 1 , Figure 3 Each modality feature is mapped to a unified latent space, and the latent distribution parameter generation module outputs the corresponding mean vector and covariance parameter, thereby obtaining the latent posterior distribution of each single modality. Specifically, EEG, EOG, and EMG correspond to the following latent distributions:

[0049]

[0050]

[0051] in, N(·) represents the Gaussian distribution function; To share latent variables; These are the latent posterior probability distributions of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the input samples corresponding to the Gaussian latent distributions of the EEG, EOG, and EMG modes, respectively. These are the mean vectors of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the covariance matrix parameters of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively.

[0052] The covariance matrix can be a diagonal covariance matrix, a full covariance matrix, or a covariance matrix formed by combining a low-rank matrix and a diagonal matrix.

[0053] In practical implementation, the covariance parameter can also be in the form of a full covariance matrix or a low-rank decomposition, but for computational efficiency, this embodiment prefers the diagonal covariance form.

[0054] S4: Construction of complete trimodal joint latent distribution See Figure 1 , Figure 3 During the training phase, since all three modalities are fully available, a complete modal joint latent distribution is constructed based on the EEG, EOG, and EMG information:

[0055] in, This is the mean vector corresponding to the complete three-modal joint latent distribution; The covariance parameter corresponding to the complete three-modal joint latent distribution; It is a complete trimodal joint latent distribution constructed based on complete information from EEG, EOG, and EMG modalities.

[0056] The joint latent distribution was obtained through a hybrid expert model (MoE) fusion method.

[0057] Specifically, three unimodal features are input into a MoE-gated network, which outputs three weight coefficients. Then, the fused features are calculated, and the mean and variance of the joint distribution are output through a linear layer. As a variation, simple feature concatenation followed by a linear layer, attention fusion, or gated fusion can also be used to replace the hybrid expert model.

[0058] S5: Latent Space Distribution Alignment See Figure 2 , Figure 6 By utilizing a latent space distribution alignment mechanism, each unimodal latent distribution is brought closer to the complete trimodal joint latent distribution, thereby establishing latent semantic consistency relationships among unimodal, partial, and complete trimodal modes. Alignment constraints are imposed by limiting the KL divergence between unimodal latent distributions and the complete joint latent distribution.

[0059] in, The latent space distribution alignment loss is used to measure the distribution difference between each single-modal latent distribution and the complete trimodal joint latent distribution. This is the Kullback-Leibler divergence, used to measure the degree of difference between two probability distributions.

[0060] In actual training, JS divergence (Jensen-Shannon divergence) or Wasserstein distance can be used instead of KL divergence to enhance stability against modal distribution differences.

[0061] S6: Classifier Training Based on the complete trimodal joint latent distribution, the mean of its corresponding latent variables is... This represents the layer used to train a sleep stage classifier. The classifier consists of two fully connected layers with 128 and 64 neurons respectively, and a softmax output layer. The training objective is a combination of classification loss, latent space distribution alignment loss, and distribution regularization term.

[0062] in, Loss due to sleep stage classification; The latent space distribution alignment loss; For distribution regularization; and These are the weighting coefficients.

[0063] S7: Missing Modal Reasoning in the Testing Phase See Figure 4 , Figure 5 During the testing phase, when one or more modes are missing, only the currently observable modes are input into the corresponding encoder to generate the latent distribution corresponding to the observable modes.

[0064] If only a single-modal input exists during the testing phase, then the single-modal latent distribution can be directly utilized. Classify; If multiple observable modes exist during the testing phase, the latent distributions of these observable modes are fused to obtain the joint latent distribution of the observed modes. Then, the corresponding latent variable representations are input into the trained classifier, and the final sleep stage results are output.

[0065] The following simulates three missing scenarios: (1) Complete input: Input EEG, EOG and EMG at the same time, and the system will directly generate a joint latent distribution and classify it.

[0066] (2) Single-modal missing: For example, if EOG is missing, only EEG and EMG are input into the corresponding encoder to generate two latent distributions. Then, the joint latent distribution of the observed modes is obtained through lightweight fusion and then classified. As a variation, the latent distribution of the mode with the strongest discriminative ability among the observable modes (such as EEG) can be directly used as an approximation of the joint latent distribution of the observations.

[0067] (3) Extreme mode missing: If only EEG single mode exists, the latent distribution of the single mode is directly used for classification.

[0068] During the testing phase, if there are two or more observable modalities, the fusion method is not limited to a simple weighted average. Dynamic fusion based on attention mechanisms or fusion methods based on uncertainty weighting can also be used.

[0069] On the ISRUC-S3 dataset, the method of this invention achieves an overall accuracy of 85.7% with complete trimodal input, drops to 83.2% with missing EOG modality, and maintains an accuracy of 80.5% with only EEG modality present. Compared to the baseline method that directly sets missing modality to zero (84.1% with complete trimodal input, 70.3% with missing EOG, and 68.9% with only EEG), this method significantly reduces the performance degradation in modality-missing scenarios, verifying the robustness of this invention.

[0070] Example 2 This embodiment is basically the same as embodiment 1, except that the encoder is specifically implemented in S2.

[0071] In this embodiment, the EEG encoder uses a one-dimensional residual network (ResNet-18) structure, the EOG encoder uses a two-layer LSTM, and the EMG encoder uses a temporal convolutional network (TCN). Ablation experiments revealed that the performance difference of this method under different encoder structures is less than 2%, indicating that the invention is insensitive to the specific encoder structure and has good versatility.

[0072] Example 3 This embodiment is basically the same as embodiment 1, but the latent distribution of each mode is replaced by a Laplace distribution instead of a Gaussian distribution.

[0073] At this point, the encoder outputs the scale parameter of the Laplace distribution instead of the variance. The distribution alignment loss is expressed as the Laplace form of the KL divergence. Experiments show that when using the Laplace distribution, the model's performance under modal missing conditions is comparable to that of the Gaussian distribution, with an accuracy difference of less than 1%. This further proves that the core of this invention lies in "distribution alignment" rather than a specific distribution form.

[0074] Example 4 See Figure 6 The present invention also provides a sleep staging system for missing modalities based on latent space distribution alignment, comprising: 1. Data Acquisition Module: Used to acquire EEG, EOG, and EMG signals. This module is compatible with various bioelectrical signal acquisition devices, including clinical polysomnography, wearable dry electrode devices, and home portable acquisition terminals.

[0075] 2. Data Preprocessing Module: This module performs filtering, resampling, time segmentation, and normalization. It can adaptively select different filtering parameters and resampling frequencies based on modal characteristics.

[0076] 3. Single-modal coding module: This module extracts features from EEG, EOG, and EMG signals respectively. It supports hot-swapping, meaning different encoder networks can be dynamically loaded for different application scenarios. For example, a lightweight encoder can be used on mobile devices, while a heavy-duty encoder can be used in the cloud.

[0077] 4. Latent distribution modeling module: used to generate Gaussian latent distribution parameters corresponding to each mode.

[0078] 5. Joint Latent Distribution Construction Module: Used to construct a reference joint latent distribution based on complete trimodal samples during the training phase. This module internally employs a hybrid expert model, but allows users to select other fusion strategies such as attention fusion.

[0079] 6. Latent Space Distribution Alignment Module: Used to constrain the consistency between individual modal latent distributions and the joint latent distribution. This module supports multiple alignment loss functions such as KL divergence, JS divergence, and Wasserstein distance, which can be selected through a configuration file.

[0080] 7. Classification Training Module: Used to learn the discrimination boundaries of sleep stages in the shared latent space.

[0081] 8. Missing Modality Inference Module: This module generates latent representations and performs classification based on existing observable modalities during the testing phase. It automatically identifies the type and number of input modalities and dynamically selects the optimal inference path. For example, it uses a joint distribution path when all three modalities are detected, and directly uses the modality distribution path when only a single modality is detected.

[0082] 9. Results Output Module: Used to output sleep stage recognition results and statistical information.

[0083] As a further variation at the system level, the system may also include a model update module, which uses newly collected online data to incrementally learn or fine-tune the model to adapt to changes in data distribution among different populations or under different collection environments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for sleep staging of missing modalities based on latent space distribution alignment, characterized in that, Includes the following steps: S1: Training Data Construction and Preprocessing Complete sleep monitoring data including three modalities: electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG) is acquired. The complete sleep monitoring data is preprocessed and segmented to obtain multiple training samples, each training sample corresponding to a sleep stage label. S2: Single-modal feature encoding The training samples for each modality were subjected to depth feature extraction using EEG encoder, EOG encoder, and EMG encoder respectively, to obtain unimodal feature representations. , and ; S3: Single-modal Gaussian latent distribution modeling Each single-modal feature representation is mapped to a unified latent space, and the mean vector corresponding to each single mode is output through the latent distribution parameter generation module. Covariance parameter Obtain the Gaussian latent distribution for each single mode: in, N(·) represents the Gaussian distribution function; To share latent variables; These are the latent posterior probability distributions of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the input samples corresponding to the Gaussian latent distributions of the EEG, EOG, and EMG modes, respectively. These are the mean vectors of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. These are the covariance matrix parameters of the Gaussian latent distributions corresponding to the EEG, EOG, and EMG modes, respectively. S4: Construction of complete trimodal joint latent distribution Based on the complete information of the three modes EEG, EOG, and EMG, a complete trimodal joint latent distribution is constructed: in, This is the mean vector corresponding to the complete three-modal joint latent distribution; The covariance parameter corresponding to the complete three-modal joint latent distribution; A complete trimodal joint latent distribution constructed based on complete information from EEG, EOG, and EMG modalities; S5: Latent Space Distribution Alignment By minimizing the distribution difference between each unimodal Gaussian latent distribution and the complete trimodal joint latent distribution, each unimodal Gaussian latent distribution is aligned to the complete trimodal joint latent distribution to establish a shared latent space; the distribution difference is measured using at least one of KL divergence, JS divergence, or Wasserstein distance. S6: Classifier Training A sleep staging classifier is trained based on the complete trimodal joint latent distribution; S7: Missing Modal Reasoning in the Testing Phase During the testing phase, when a modality is missing, only the currently observable modality is input into the corresponding encoder to generate the observable modality Gaussian latent distribution, and the sleep staging result is output based on the shared latent space and the sleep staging classifier.

2. The method according to claim 1, characterized in that, The distribution differences are expressed using KL divergence, and the alignment loss function is: in, The latent space distribution alignment loss is used to measure the distribution difference between each single-modal latent distribution and the complete trimodal joint latent distribution. This is the Kullback-Leibler divergence, used to measure the degree of difference between two probability distributions.

3. The method according to claim 1, characterized in that, The complete trimodal joint latent distribution is obtained through a hybrid expert model fusion method: the three unimodal feature representations are input into the attention gating network of the hybrid expert model, three weight coefficients are output, the fused features are calculated, and then the mean vector and covariance parameter of the complete trimodal joint latent distribution are output through a linear layer.

4. The method according to claim 1, characterized in that, The total loss function for training the sleep staging classifier is: in, Loss due to sleep stage classification; The latent space distribution alignment loss; For distribution regularization; and The weight coefficients are used; the classifier consists of two fully connected layers and a Softmax output layer, with the two fully connected layers containing 128 neurons and 64 neurons respectively.

5. The method according to claim 1, characterized in that, The EEG encoder, EOG encoder, and EMG encoder each independently employ a combination of one-dimensional convolutional networks, multi-scale convolutional structures, and Mamba networks.

6. The method according to claim 1, characterized in that, The covariance matrix of each single-mode Gaussian latent distribution adopts the diagonal covariance form.

7. The method according to claim 1, characterized in that, During the testing phase, if multiple observable modalities exist, the Gaussian latent distributions of each observable modality are fused to obtain the joint latent distribution of the observable modalities, which is then input into the sleep staging classifier. The fusion adopts any one of weighted averaging, attention-based dynamic fusion, or uncertainty-weighted fusion.

8. The method according to claim 1, characterized in that, The preprocessing includes: applying a 0.5~30Hz bandpass filter to the EEG signal, a 0.5~10Hz bandpass filter to the EOG signal, and a 10~100Hz bandpass filter to the EMG signal; resampling to 100Hz, with each segment lasting 30 seconds and containing 3000 sampling points, and normalizing using z-score standardization.

9. A sleep staging system for missing modalities based on latent space distribution alignment, used to perform the method according to any one of claims 1 to 8, characterized in that, include: The data acquisition module is used to acquire EEG, EOG, and EMG signals; The data preprocessing module is used to filter, resample, segment, and standardize the acquired signals. The single-modal coding module uses EEG encoders, EOG encoders, and EMG encoders to extract deep features from each modal signal; The latent distribution modeling module is used to map the features of each modality to a unified latent space and output the Gaussian latent distribution parameters of each modality. A joint latent distribution construction module is used to construct a reference joint latent distribution based on complete trimodal samples during the training phase; A latent space distribution alignment module is used to constrain the consistency between each single-modal latent distribution and the joint latent distribution; The classification training module is used to learn the sleep stage discrimination boundaries in the shared latent space; The missing modality reasoning module is used to generate latent representations and perform classification based on the currently observable modalities during the testing phase; The results output module is used to output the sleep stage recognition results.

10. The system according to claim 9, characterized in that, It also includes a model update module, which is used to incrementally learn or fine-tune the model using new data collected online; the missing modality inference module is configured to automatically identify the type and number of input modalities and dynamically select one of the following inference paths: when a complete three-modality is detected, a joint distribution path is used; when two observable modalities are detected, a fusion path is used; when a single modality is detected, the modality distribution path is directly used.

Images