Personalized sleep state monitoring method and system based on multi-modal physiological electrical signals

Abstract

The application discloses a kind of individualized sleep state monitoring method and system based on multi-modal physiological electrical signal.The method comprises: the electroencephalogram, electrooculogram and electromyogram signal in the sleep data collected are preprocessed and feature extraction;Sleep staging model is constructed and the model is pre-trained using multi-source public sleep data set;Low rank adaptation module and modal specific adapter of mode perception are introduced, the main body parameter of pre-training model is frozen, and the pre-acquired sleep data of target user is used to update the adaptation parameter, and the model is individualized fine-tuning;Based on the sleep staging model after fine-tuning, the sleep stage classification prediction of target user is carried out using real-time acquisition sleep data.The present application scheme realizes the efficient fine-tuning of model for individual differences, can effectively improve the accuracy of sleep classification prediction, and is suitable for portable sleep monitoring equipment.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of medical signal processing, artificial intelligence and edge computing technology, and specifically relates to a personalized sleep state monitoring method and system based on multimodal physiological electrical signals. Background Technology

[0002] Sleep quality is closely related to human health, and accurate sleep monitoring is of great significance for the diagnosis of sleep disorders, chronic disease management, and daily health monitoring.

[0003] In recent years, significant progress has been made in sleep staging methods based on deep learning. Patent CN111783534B discloses a deep learning-based sleep staging method that classifies single-channel EEG signals using a convolutional neural network and effectively addresses the data imbalance problem through oversampling and Bayesian optimization. Patent CN116186544A proposes a deep learning-based single-channel EEG sleep staging network that achieves automated EEG signal staging by constructing an MFE loss function. However, these methods only utilize single-channel EEG signals, neglecting the important supplementary role of auxiliary modalities such as electrooculography (EOG) and electromyography (EMG) in sleep state discrimination. They are particularly difficult to distinguish between rapid eye movement (REM) and non-rapid eye movement (N1) stages, as the characteristic rapid eye movements of REM require EOG signals for accurate capture.

[0004] To fully utilize multimodal information, patent CN118749911B proposes a multimodal sleep staging method that generates signal combination strategies by determining the correlation between various physiological signals, achieving low-cost, non-intrusive sleep monitoring. Patent CN116269212A uses a dual-branch convolutional neural network to fuse multimodal PSG signals such as EEG and EEG for sleep stage prediction. Patent CN118592904B discloses a deep learning-based automatic sleep staging system that supports automatic labeling and downstream analysis of EEG and EMG signals. However, while the above methods improve sleep staging performance by introducing multimodal information, they all train models on a unified public dataset and apply them directly to all users, failing to consider the significant differences between individuals in terms of age, gender, underlying diseases, and sleep habits, leading to a decline in the model's generalization performance on specific users.

[0005] To address the aforementioned issues, patent CN120267244B proposes a personalized dynamic sleep staging method. This method achieves real-time accuracy of the generated sleep staging data by maintaining a branch algorithm library containing multiple algorithms, selecting the corresponding basic algorithm based on user category, and switching algorithms based on user feedback. However, this method's personalization is achieved by selecting existing algorithms from a pre-set algorithm library, representing a coarse-grained adaptation through "algorithm selection," and cannot fine-grainedly adjust the model to suit the unique physiological characteristics of each user.

[0006] Furthermore, when adapting the model to user personalization, if a solution is adopted that adjusts all parameters, there are drawbacks such as a huge amount of parameter updates and excessive computational overhead, making it difficult to deploy on portable sleep monitoring devices to achieve the goal of anytime, anywhere sleep monitoring.

[0007] Therefore, the industry urgently needs a sleep monitoring solution that can run efficiently on resource-constrained devices and fine-tune the model according to the sleep characteristics of each user. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention proposes a personalized sleep state monitoring scheme based on multimodal physiological electrical signals. By introducing modality-aware parameter fine-tuning technology, while freezing the main parameters of the model, only a small number of low-rank adaptation parameters are updated to address the characteristic differences of different individuals in each modal signal, so as to achieve high-quality personalized adaptation of the model.

[0009] To achieve the above objectives, the present invention discloses the following technical solution: A personalized sleep state monitoring method based on multimodal physiological electrical signals, the method comprising the following steps: S1. Collect EEG, EEG, and EMG signals during human sleep, and preprocess and extract features from each modality to obtain multimodal sleep signal features; S2. Construct a sleep staging model, which includes an intramodal feature encoder, a cross-modal attention fusion module, a temporal context modeling module, and a classification head. Combined with step S1, the sleep staging model is pre-trained using a public sleep dataset. S3. Freeze the main parameters of the pre-trained sleep staging model, and introduce a modality-aware low-rank adaptation module and a modality-specific adapter for the characteristics of different modal signals. Combined with step S1, use the pre-collected sleep data of the target user to update the adaptation parameters in the low-rank adaptation module and the modality-specific adapter to perform personalized fine-tuning of the model. S4. Based on the fine-tuned sleep stage model, combined with step S1, use real-time collected sleep data to predict the sleep stage classification of the target user.

[0010] Furthermore, in step S1: The preprocessing includes bandpass filtering and Z-score normalization of the raw signals of each mode; The feature extraction includes using wavelet transform to convert the standardized time-domain signal into a time-frequency feature representation.

[0011] Furthermore, in step S2: The time-frequency characteristics of each mode are processed by the encoder and output as corresponding feature sequences; Using EEG features as query vectors and EEG and EMG features as key-value pairs, information related to the current EEG features is adaptively extracted from EEG and EMG features through an attention mechanism. Based on the global representation of the corresponding modality features, the importance weights of each modality are dynamically calculated through a gating network, and then all modality features are weighted and fused. A bidirectional gated cyclic unit is used to perform temporal modeling of the fused features at multiple consecutive time points.

[0012] Furthermore, in step S2: The model was pre-trained using multiple publicly available sleep datasets, and then jointly optimized using supervised contrastive learning loss and cross-entropy loss. The calculation formula is as follows: ; in, The total pre-training loss, For cross-entropy loss, To monitor and compare learning loss, The weight coefficients for supervised contrastive learning loss are P(i), where P(i) is the set of positive samples in the dataset that are of the same class as sample i. Let be the normalized feature vector of sample i, obtained by L2 normalization of the fused features from the previous layer of the classifier. This is the normalized feature vector of positive samples belonging to the same sleep stage category as sample i. This represents the normalized feature vectors of all samples in the current batch except for sample i. and These represent the dot product similarity between the corresponding feature vectors. This refers to the temperature parameter.

[0013] Furthermore, in step S3: The modality-specific adapter adopts a bottleneck structure, and the calculation formula is as follows: ; Where m is the modality index. Let m be the input features of modality m, and LN be the layer normalization operation. Here is the lower projection matrix used for feature dimensionality reduction. It is a non-linear activation function. The upprojection matrix is ​​used for feature dimensionality upscaling. To adapt the output characteristics of the post-modal m.

[0014] Furthermore, in step S3: The low-rank adaptation module assigns different adaptation ranks to different modalities in the pre-collected sleep data. The adaptation rank assigned to the EEG modality is greater than that assigned to the EOS modality and EMG modality. A low-rank adaptation matrix is ​​then introduced into the pre-trained sleep staging model. and The product of the two is used as the incremental correction to the original weight matrix in the model, and the calculation formula is as follows: ; in, Here is the original weight matrix for mode m. From Via A low-rank reduced-dimensional matrix of dimension 1, and , From Via A low-rank, high-dimensional matrix, and , The input dimension of the original weight matrix. The output dimension of the original weight matrix. For the fit rank assigned to mode m, The overall weight matrix for adapting to mode m; During model fine-tuning Keep it unchanged, only update and .

[0015] Furthermore, in step S3: A cross-modal low-rank coupling mechanism is introduced into the low-rank adaptation module to connect the low-rank adaptation matrix. Decomposed into low-rank basis shared by all modes and mode-specific low-rank basis ,in, Used by all modalities to capture common individual characteristics across modalities. Each modality possesses its own unique features, used to capture the individual characteristics specific to that modality.

[0016] Furthermore, in step S3: The model is fine-tuned using a multi-task loss function, calculated as follows: ; in, To fine-tune the total loss, For cross-entropy loss, For intra-individual contrast loss, For timing consistency loss, For regularization loss, , , These are the weighting coefficients for each loss term.

[0017] Furthermore, the method also introduces a weakly supervised domain adaptation mechanism, which uses the subjective sleep information preset by the target user as a weakly supervised signal to cooperate with the model for personalized fine-tuning. The subjective sleep information includes at least one of the following: sleep onset time, wake-up time, number of nighttime awakenings, and subjective sleep quality score. The constraints of the weak supervision signal include: boundary constraints that restrict the prediction of the period before falling asleep and after waking up as a state of wakefulness; a number constraint that restricts the number of times the predicted sleep state transitions to a state of wakefulness to be consistent with the number of times the target user reports wakefulness; and a quality constraint based on the positive correlation between the proportion of deep sleep and the subjective sleep quality score.

[0018] The present invention also discloses a personalized sleep state monitoring system for implementing the above method, the system comprising the following modules: The signal acquisition module is used to acquire multimodal physiological electrical signals during the sleep process of the target user; The feature extraction module is used to preprocess and extract features from the input modal signals to obtain multimodal sleep signal features; The model pre-training module is used to build a sleep staging model that includes an intramodal feature encoder, a cross-modal attention fusion module, a temporal context modeling module, and a classification head, and to pre-train the model using a public sleep dataset; The model fine-tuning module, including a modality-aware low-rank adaptation module and a modality-specific adapter, is used to update the adaptation parameters using pre-collected sleep data from the target user while freezing the main parameters of the pre-trained sleep staging model, thereby performing personalized fine-tuning of the model. The classification and prediction module is used to classify and predict the sleep stages of target users based on a fine-tuned sleep staging model and real-time collected sleep data; and The model fine-tuning module and classification prediction module are executed locally on the edge computing unit of the portable sleep monitoring device.

[0019] Furthermore, the system also includes a weakly supervised domain adaptation module, which is used to perform personalized fine-tuning of the model based on the subjective sleep information preset by the target user as a weakly supervised signal.

[0020] The present invention has at least the following beneficial effects: 1. This invention addresses the different individual manifestations of various modal signals such as EEG, EEG, and EMG by employing modal perception-based differential fine-tuning. This allocates higher degrees of freedom for EEG modal adaptation, achieving more precise personalized adaptation compared to the uniform adaptation strategy used in traditional methods.

[0021] 2. By reasonably setting the adaptation rank and bottleneck dimension, this invention can control the number of fine-tuning parameters to within 5% of the total number of parameters in the pre-trained model. That is, only a small number of model parameters need to be updated to achieve effective personalized adaptation of the sleep monitoring method. This allows model fine-tuning and sleep classification prediction to be completed locally on portable sleep monitoring devices without uploading user data to the cloud. This is suitable for mobile monitoring and can also protect user privacy.

[0022] 3. This invention innovatively utilizes the subjective sleep information filled in by users on mobile applications as a weak supervision signal, eliminating the need for professionally labeled domain adaptation and significantly reducing the usage threshold for personalized adaptation. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the description of the specific embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 The flowchart shows a personalized sleep state monitoring method based on multimodal physiological electrical signals provided in Embodiment 1 of the present invention. Figure 2 This is an architecture diagram of the pre-trained model in Embodiment 1 of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0026] It should be noted that the technical terms used in this invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of this invention. Unless otherwise specified, all instruments and equipment used in the following embodiments of this invention can be purchased from the market or prepared by existing methods. Example 1

[0027] Please see Figure 1 and Figure 2 This embodiment provides a personalized sleep state monitoring method based on multimodal physiological electrical signals, including the following steps: S1. Extract multimodal sleep signal features S101, Signal Acquisition Multimodal physiological electrical signals were collected during human sleep, and the raw data set was as follows: ,in, EEG signals and , EOG signal and , EMG signal and , J is a real matrix. m Let m be the number of channels in mode m, where m is the mode index, and take the value of m. Time indicates brain signals, Time indicates electrooculogram (EOG) signal, Time represents the electromyographic signal, and T is the number of sampling points.

[0028] S102, Signal Preprocessing The raw signals of each mode are first bandpass filtered to remove noise and baseline drift, while retaining frequency components related to the sleep state.

[0029] Z-score normalization is performed on each of the filtered modal signals, and the calculation formula is as follows: ; in, and denoted as mean and standard deviation, respectively, for mode m.

[0030] S103, Time-Frequency Feature Extraction Since wavelet transform can capture the local features of a signal in both time and frequency, it has a good ability to characterize the rhythmic activities of different frequency bands in sleep signals. Therefore, continuous wavelet transform (CWT) can be used to convert time-domain signals into time-frequency feature representations.

[0031] For each modal signal, the magnitudes of its wavelet transform coefficients constitute the time-frequency characteristic representation of the corresponding mode. Where F is the frequency scale and T' is the time dimension.

[0032] Time-frequency characteristics of all modal signals The multimodal sleep signal characteristics that constitute the target user.

[0033] S2. Constructing and Pre-training Sleep Staging Models The sleep staging model includes an intramodal feature encoder, a cross-modal attention fusion module, a temporal context modeling module, and a classification head. Pre-training uses publicly available sleep datasets from multiple sources, and the sleep data in the dataset are preprocessed and feature extracted according to step S1 to obtain the corresponding multimodal sleep signal features.

[0034] S201, Intramodal Feature Coding Time-frequency feature representation using a lightweight encoder Feature extraction The calculation formula is: ; Where Encoder is the encoder. , The time-frequency feature represents the length of the feature sequence output after encoder processing. For feature dimensions.

[0035] The encoder is constructed using depthwise separable convolution, which decomposes standard convolution into two steps: depthwise convolution and pointwise convolution, thereby significantly reducing the number of parameters and computational cost.

[0036] S202, Cross-modal attention fusion Using EEG features as the query vector and EOL and EMG features as key-value pairs, the system adaptively extracts information related to the current EEG features from EOL and EMG features through an attention mechanism. The calculation formula is as follows: ; ; in, Let Q be the feature of modality m, and let Q be the EEG feature. After querying the projection weight matrix The query matrix obtained by linear projection and Modal features Through the corresponding linear projection matrix and The resulting key and value matrices, with Attention as the attention mechanism. These are the features of mode m extracted via the attention mechanism.

[0037] An adaptive gating fusion mechanism is introduced. Based on the global representation of the corresponding modal features, the importance weights of each modality are dynamically calculated through a gating network to achieve weighted fusion of all modal features. The calculation formula is as follows: ; in, The features of the fused multimodal sleep signals, , , The adaptive weights for the corresponding modal features and satisfying .

[0038] Gated fusion weights are calculated using an MLP: ; in, This is a three-dimensional weight vector, representing the overall weights of the three modalities. `softmax` is the normalization function. This is the weight matrix of the gated network, used to map the global representations of the three modalities to fusion weights. The mean represents the weights applied to the feature sequence over its length. Global average pooling is performed on the top layer, and concat is the feature concatenation operation.

[0039] S203, Temporal Context Modeling A bidirectional gated recurrent unit (Bi-GRU) is used to model the temporal context dependencies before and after the model, enabling the model to make more accurate judgments by incorporating adjacent information. The calculation formula is as follows: ; in, Forward GRU, The hidden state of the feedforward GRU at time t is determined by the fused sleep features at the current time. and the hidden state of the previous moment The result is obtained through recursive calculation. For backward GRU, for The hidden state of the backward GRU at time t is determined by the fused sleep features at the current time. and the hidden state at the next moment Recursive calculation yields [;] which represents the feature concatenation operation. This represents the bidirectional contextual sleep feature representation after concatenation at time t.

[0040] S204, Pre-training and Loss Function Multiple publicly available sleep datasets were used for model pre-training. Supervised contrastive learning loss and cross-entropy loss were jointly employed to optimize the model. Supervised contrastive learning loss clusters samples from the same sleep stage in the feature space while separating samples from different classes. Cross-entropy loss directly optimizes classification accuracy. The calculation formula is as follows: ; in, The total pre-training loss, For cross-entropy loss, To monitor and compare learning loss, The weight coefficients for supervised contrastive learning loss are P(i), where P(i) is the set of positive samples in the dataset that are of the same class as sample i. The normalized feature vector of sample i is formed by fusing features from the layer preceding the classifier. After L2 normalization, This is the normalized feature vector of positive samples belonging to the same sleep stage category as sample i. This represents the normalized feature vectors of all samples in the current batch except for sample i. and These represent the dot product similarity between the corresponding feature vectors. This refers to the temperature parameter.

[0041] S3, Model Personalization Fine-tuning S301, Pre-collection of target user data When first using the device, the target user wears it continuously for 3-7 nights. The device automatically records the target user's electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG) signals during sleep and stores them in fixed time intervals.

[0042] The collected sleep data from multiple nights is preprocessed and features are extracted according to step S1 to obtain the corresponding multimodal sleep signal features, which are used as a personalized fine-tuning training dataset.

[0043] The fine-tuning training data is input into the pre-trained sleep staging model for forward inference to obtain the predicted probability distribution of each sleep stage corresponding to each sleep cycle. The maximum probability value is taken as the predicted sleep stage category for that time period, and the maximum probability value is used as the confidence score. When the confidence score is not lower than a preset threshold, the predicted category of that sleep cycle is included as a pseudo-label in the fine-tuning training dataset.

[0044] S302, Modality-aware low-rank adaptation To address the characteristics of different modal signals, a modality-aware low-rank adaptation module and a modality-specific adapter are introduced. The low-rank adaptation module is used to assign different adaptation ranks to different modalities in the fine-tuning training data, and the adaptation rank assigned to the EEG modality is greater than that assigned to the EOS modality and EMG modality.

[0045] A low-rank decomposition matrix is ​​introduced in the linear transform bypass of the encoder and attention layer, with two low-rank fitting matrices. and The product of these terms serves as an incremental correction to the original weight matrix, and is calculated using the following formula: ; in, Here is the original weight matrix for mode m. To reduce the dimensionality of the input to a low-rank matrix corresponding to the fit rank, and , To recover the low-rank matrix from the features by increasing the dimension of the corresponding fit rank, and , The input dimension of the original weight matrix. The output dimension of the original weight matrix. For the fit rank assigned to mode m, and much smaller and For smaller values, incremental correction can be achieved with a small number of parameters by first reducing the dimensionality and then increasing it. The overall weight matrix for adapting to mode m; During model fine-tuning Keep it unchanged, only update and .

[0046] S303, cross-modal low-rank coupling low-rank adaptation matrix Decomposed into low-rank basis shared by all modes and mode-specific low-rank basis The calculation formula is: ; The low-rank base Used by all modalities to capture common individual characteristics across modalities (such as overall sleep depth preference, circadian rhythm characteristics, etc.), the low-rank basis Each modality possesses its own unique features, used to capture modality-specific individual characteristics (such as the intensity of activity in specific frequency bands of scalp EEG, the amplitude characteristics of eye movements, etc.).

[0047] S304, Modal-Specific Adapter Independent lightweight adapter modules are designed for different modalities. These adapters employ a bottleneck structure, enabling them to capture modality-specific adaptation information with minimal parameter input. The calculation formula is as follows: ; in, For the input features of mode m, For compressing features from d dimensions to d r The lower projection matrix of dimension, For use to transfer features from d r The feature dimension is restored to the d-dimensional upprojection matrix. r For the bottleneck dimension, LN is the layer normalization operation. It is a non-linear activation function. To adapt the output characteristics of the post-modal m.

[0048] In the method of this invention, the low-rank fitting matrix , The upper and lower projection matrices are applied to the original weight matrix of the model to modify the weights themselves in a linear incremental manner; while the upper and lower projection matrices are applied to the feature representation output by the encoder.

[0049] S305, Fine-tuning Training and Loss Function Using the fine-tuning training dataset obtained in step S301, the low-rank adapter module parameters and modality-specific adapter parameters introduced in steps S302-S304 are locally fine-tuned in a supervised manner on the edge computing unit of the portable sleep monitoring device.

[0050] The model is fine-tuned using a multi-task loss function, calculated as follows: ; in, To fine-tune the total loss, Cross-entropy loss is used to ensure classification accuracy. Intra-individual contrast loss is used to enhance the feature compactness of similar sleep stage samples from the same target user. This is a temporal consistency loss used to constrain the prediction consistency of adjacent sleep segments to avoid frequent jumps; This is the regularization loss, used to prevent overfitting of the fitting parameters; , , These are the weighting coefficients corresponding to each loss term.

[0051] The temporal consistency loss employs adaptive weights, which are determined based on the sleep stage transition probability. Specifically, the weight is higher when the previous sleep cycle is in a state where stage transitions are less likely to occur, and lower when it is in a state where stage transitions are more likely to occur. The sleep stage transition probability can be obtained from the labeled statistics of the pre-training dataset or by referring to empirical values ​​in sleep medicine literature.

[0052] S4, Weakly Supervised Domain Adaptation When there is a lack of professional sleep stage annotations, i.e. pseudo-labels in step S301, the subjective sleep information preset by the target user (e.g., filled in by the accompanying mobile application after waking up every morning) is used as a weak supervision signal to cooperate with the model for personalized fine-tuning; the subjective sleep information includes at least one of the following: sleep onset time, wake-up time, number of nighttime awakenings, and subjective sleep quality score.

[0053] The aforementioned subjective sleep information is transformed into soft constraints on model predictions. These weak supervision signals include: (1) Sleep-wake boundary constraints: user-reported sleep onset time In the previous period, the model predicted a higher probability of being awake, and the user reported their awakening time. In the subsequent period, the model predicts a higher probability of being in a conscious state; (2) Consistency constraint of the number of awakenings: the number of times the model predicts the transition from sleep to wakefulness. This should be related to the number of times users report waking up at night. Similarity, i.e., minimization ; (3) Sleep quality correlation constraint: Based on the conclusion in sleep medicine research that subjective sleep quality is positively correlated with the proportion of deep sleep, when the user's subjective score is high, the proportion of deep sleep stage in the total sleep time predicted by the model should also be high. The calculation formula is: ; Among them, L weak For soft-constraint total loss, This is the boundary constraint loss, used to constrain the prediction of a waking state during the pre-sleep and post-wake periods. The awakening frequency consistency constraint loss is used to ensure that the predicted sleep-to-wake frequency is consistent with the user-reported awakening frequency. The loss function, which is designed to constrain sleep quality, is based on the positive correlation between the percentage of deep sleep and subjective sleep quality scores.

[0054] S5, Sleep Status Prediction The target user wears the device while sleeping, and the real-time collected sleep data is preprocessed and features are extracted according to step S1 to obtain the corresponding multimodal sleep signal features. These features are then input into the fine-tuned sleep staging model to predict the sleep stage classification of the target user.

[0055] Example 2

[0056] This embodiment provides a personalized sleep state monitoring system based on multimodal physiological electrical signals, including the following modules: The signal acquisition module is used to acquire multimodal physiological electrical signals during the sleep process of the target user; The feature extraction module is used to preprocess and extract features from the input modal signals to obtain multimodal sleep signal features; The model pre-training module is used to build a sleep staging model that includes an intramodal feature encoder, a cross-modal attention fusion module, a temporal context modeling module, and a classification head, and to pre-train the model using a public sleep dataset; The model fine-tuning module, including a modality-aware low-rank adaptation module and a modality-specific adapter, is used to update the adaptation parameters using pre-collected sleep data from the target user while freezing the main parameters of the pre-trained sleep staging model, thereby performing personalized fine-tuning of the model. The weakly supervised domain adaptation module is used to perform personalized fine-tuning of the model based on the subjective sleep information preset by the target user as a weakly supervised signal. The classification and prediction module is used to classify and predict the sleep stages of target users based on a fine-tuned sleep staging model and real-time collected sleep data. The data storage module is used to store the collected sleep data and model-related parameters.

[0057] The model fine-tuning module and classification prediction module are executed locally on the edge computing unit of the portable sleep monitoring device.

[0058] Example 3

[0059] The following description, in conjunction with Examples 1 and 2, details the specific implementation and experimental evaluation results of the method of the present invention in a portable sleep monitoring device, to help those skilled in the art to more deeply understand and apply this technical solution. It should be emphasized that the data sources, model parameters, and index results given in this embodiment are not the only limitations of the present invention, and related details can be flexibly adjusted according to actual needs.

[0060] 1. System hardware configuration and dataset (1) Hardware platform The hardware configuration of the portable sleep monitoring device used in this embodiment is as follows:

[0061] (2) Data source This embodiment uses multiple publicly available sleep datasets for pre-training and performs personalized fine-tuning and verification on self-collected data.

[0062] The pre-training datasets include: the Sleep-EDF Expanded dataset (197 subjects, approximately 400 nights of PSG records), the SHHS dataset (polysomnography of 5793 subjects), and the MASS dataset (full-night PSG records of 200 subjects). These datasets cover subjects of different ages (18-90 years), genders, and ethnicities, totaling approximately 6000 sleep records. All data are labeled with five stages of sleep using the AASM standard, namely wakefulness (W), rapid eye movement (REM), non-rapid eye movement stage 1 (N1), non-rapid eye movement stage 2 (N2), and non-rapid eye movement stage 3 (N3).

[0063] The personalized fine-tuning validation dataset consists of 50 healthy volunteers (25 males and 25 females, aged 22-58 years, with an average age of 35.6 ± 10.2 years). Each volunteer wore a portable sleep monitoring device for 5 consecutive nights to collect individual sleep data. The data from the first 3 nights was used for personalized fine-tuning, and the data from the last 2 nights was used for validation testing. The test data was also manually labeled as a standard.

[0064] (3) Experimental environment The pre-training phase was conducted on a GPU server equipped with an NVIDIA GeForce RTX 4090 (24GB VRAM); the operating system was Linux, and the deep learning framework used was PyTorch 2.0.

[0065] The personalization fine-tuning phase is executed locally on the Rockchip RK3568 processor of the portable sleep monitoring device, using the TensorFlow Lite Micro framework.

[0066] 2. Specific implementation of extracting multimodal sleep signal features (1) Signal acquisition In this embodiment, the acquisition parameters for each modal signal are configured as follows:

[0067] (2) Signal preprocessing In this embodiment, during Z-score normalization, epochs are divided into 30-second units, and each epoch contains T = 256 × 30 = 7680 sampling points. and Calculated by epoch.

[0068] (3) Time-frequency feature extraction In this embodiment, Morlet wavelet transform is used to perform continuous wavelet transform on the standardized signal, with the center frequency range set to 0.5-30Hz. A total of F = 64 frequency scales are extracted to obtain the time-frequency feature representation. The time dimension T' = 240 corresponds to 240 time windows within each epoch (window length is 128ms, step size is 125ms).

[0069] 3. Specific implementation of constructing and pre-training the sleep staging model (1) Intramodal feature encoding The encoder in this embodiment consists of three depthwise separable convolutional layers, each with a kernel size of 3×3 and channel numbers of 32, 64, and 128 respectively. Each convolutional layer is followed by a ReLU activation function and a max pooling operation. The last layer uses global average pooling and maps the feature dimension to 64 dimensions through a linear layer. The feature sequence length N = 240 and the feature dimension d = 64. The number of parameters for each modal encoder is approximately 45K.

[0070] (2) Cross-modal attention fusion The attention mechanism extraction formula used in this embodiment is as follows: ; in, Q, , The query matrix, key matrix, and value matrix are respectively obtained by multiplying the features of the corresponding modality with the projected weight matrix. The final attention output is... Attention weights and value matrix The weighted sum, projection dimension d k = 64, with 4 attention heads.

[0071] (3) Temporal context modeling In this embodiment, the GRU hiding dimension is 64, and the context window is 21 consecutive epochs (10 epochs before and after).

[0072] (4) Pre-training and loss function In this embodiment, when calculating the supervised contrastive learning loss... =0.07, =0.5.

[0073] Pre-training uses the Adam optimizer with an initial learning rate of 1×10⁻⁶. -3 The batch size is 128, the training is conducted for 100 rounds, and the training is stopped early if the batch size does not decrease after 5 rounds on the validation set.

[0074] After pre-training, the total number of parameters in the base model is 2.3M.

[0075] 4. Specific implementation of model personalization and fine-tuning (1) Pre-collection of target user data In this embodiment, the target user wears the device continuously for 5 nights when using it for the first time, with the first 3 nights used for fine-tuning. The device automatically records multimodal signals during the user's sleep and stores them in 30-second epochs.

[0076] In this embodiment, the preset threshold for the confidence score of sleep staging is 0.8.

[0077] (2) Low-rank adaptation of modality perception Considering that EEG signals carry the richest information about sleep states and exhibit the greatest individual variation, this embodiment assigns the highest-fitting rank to EEG modalities. =8, Electrooculogram (EOG) signals play a crucial role in REM phase identification, but individual differences are relatively small, and the assigned adaptive rank... =4, electromyographic signals mainly reflect changes in muscle tone and have minimal individual differences, allocation-fit rank =4.

[0078] For low-rank fitting matrices Gaussian initialization is used. Initialize to a zero matrix to ensure that the model behavior is consistent with the pre-trained model when fine-tuning begins.

[0079] (3) Cross-modal low-rank coupling In this embodiment, the dimension of the shared low-rank basis is 2. Therefore, the dimensions of the specific low-rank basis for each mode are respectively... -2=6、 -2=2、 -2=2.

[0080] (4) Modality-specific adapter In this embodiment, the GELU activation function is used, with a bottleneck dimension d. r = 32, meaning it is first compressed from 64 dimensions to 32 dimensions, and then restored to 64 dimensions.

[0081] (5) Fine-tuning training and loss function Cross-entropy loss: Where c is the sleep stage category index, For real labels, Predict the probability that the current epoch belongs to category c for the model.

[0082] Intra-individual contrast loss: ;in, Let i be the normalized feature vector of sample i. This is the normalized feature vector of positive samples that belong to the same target user and the same sleep stage category as sample i. The normalized feature vector of all samples in the same batch, temperature parameter .

[0083] Temporal consistency loss: ;in, Let KL divergence be the KL divergence. Let be the predicted probability distribution for the nth epoch. This represents the predicted probability distribution for the (n-1)th epoch. The weights are adaptive weights based on the sleep stage transition probability.

[0084] Regularization loss: ;in, This represents the Frobenius norm.

[0085] In this embodiment, the weight coefficients of the multi-task loss function are set as follows: = 0.1, = 0.05, =0.001.

[0086] (6) Statistics on fine-tuning parameters The fine-tuning parameters for each component in this embodiment are as follows: The EEG mode has 2048 low-rank adaptation parameters, the EOG mode and EMG mode each have 1024 low-rank adaptation parameters, the cross-modal shared low-rank basis parameters have 384, the bottleneck adapter parameters for the three modes total 12288, and other parameters are approximately 1500.

[0087] The total number of fine-tuned parameters is approximately 68K, representing only 2.96% of the total parameters in the base model (2.3M). Fine-tuning utilizes the Adam optimizer with an initial learning rate of... With a batch size of 16, training on a portable device takes approximately 20 minutes.

[0088] 5. Specific implementation of weakly supervised domain adaptation When professional sleep stage annotations are lacking, subjective sleep information preset by the target user is used as a weak supervision signal to assist in personalized model fine-tuning. Users answer the following questions via a companion mobile application every morning after waking up: (a) "What time did you fall asleep last night?" -- Users fill in their perceived time of falling asleep; (b) "What time did you wake up this morning?" -- The user enters their final wake-up time; (c) "How many times have you woken up at night?" -- Users fill in the number of times they remember waking up at night (0-10 times); (d) "How would you rate your sleep quality last night?" -- Users select 1-5 (1 indicates very poor, 5 indicates very good).

[0089] 6. Edge device deployment optimization (1) Model quantization: INT8 quantization is adopted after training to compress the parameters from 32-bit floating-point numbers to 8-bit integer representation, and the model size is compressed from 9.2MB to 2.4MB, which significantly reduces the model size and improves the inference speed.

[0090] (2) Operator fusion: The continuous convolution-batch normalization-activation operations are merged into a single operator, which reduces the memory access overhead of the inference process.

[0091] The quantized model runs on Rockchip RK3568, with an inference time of approximately 85ms per epoch, meeting the requirements for real-time monitoring.

[0092] 7. Experimental Results and Performance Comparison (1) Baseline method The following methods were selected as the baseline for comparison: Pre-trained models can be used directly without any fine-tuning. Unified rank low-rank adaptation: All modes adopt the same adaptation rank, r = 6; Full parameter fine-tuning: Unfreeze all parameters for fine-tuning, verified only on the GPU server.

[0093] (2) Experimental results The table below shows the sleep staging performance of each method on a test dataset of 50 volunteers. Higher accuracy and Cohen's Kappa score indicate better method performance.

[0094]

[0095] The table below shows the results of the ablation experiment:

[0096] Experimental results show that the method of the present invention can achieve near-full parameter fine-tuning performance with only 2.96% of the parameters and can be deployed locally on portable devices. Compared with directly applying the pre-trained model, its accuracy is improved by 6.6 percentage points. Compared with uniform rank adaptation, its accuracy is improved by 1.5 percentage points and the K coefficient is higher, which verifies the effectiveness of the modality-aware differential adaptation strategy.

[0097] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A personalized sleep state monitoring method based on multimodal physiological electrical signals, characterized in that, The method includes the following steps: S1. Collect EEG, EEG, and EMG signals during human sleep, and preprocess and extract features from each modality to obtain multimodal sleep signal features; S2. Construct a sleep staging model, which includes an intramodal feature encoder, a cross-modal attention fusion module, a temporal context modeling module, and a classification head. Combined with step S1, the sleep staging model is pre-trained using a public sleep dataset. S3. Freeze the main parameters of the pre-trained sleep staging model, and introduce a modality-aware low-rank adaptation module and a modality-specific adapter for the characteristics of different modal signals. Combined with step S1, use the pre-collected sleep data of the target user to update the adaptation parameters in the low-rank adaptation module and the modality-specific adapter to perform personalized fine-tuning of the model. S4. Based on the fine-tuned sleep stage model, combined with step S1, use real-time collected sleep data to predict the sleep stage classification of the target user.

2. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 1, characterized in that, In step S1: The preprocessing includes bandpass filtering and Z-score normalization of the raw signals of each mode; The feature extraction includes using wavelet transform to convert the standardized time-domain signal into a time-frequency feature representation.

3. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 2, characterized in that, In step S2: The time-frequency characteristics of each mode are processed by the encoder and output as corresponding feature sequences; Using EEG features as query vectors and EEG and EMG features as key-value pairs, information related to the current EEG features is adaptively extracted from EEG and EMG features through an attention mechanism. Based on the global representation of the corresponding modality features, the importance weights of each modality are dynamically calculated through a gating network, and then all modality features are weighted and fused. A bidirectional gated cyclic unit is used to perform temporal modeling of the fused features at multiple consecutive time points.

4. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 3, characterized in that, In step S2: The model was pre-trained using multiple publicly available sleep datasets, and then jointly optimized using supervised contrastive learning loss and cross-entropy loss. The calculation formula is as follows: ； in, The total pre-training loss, For cross-entropy loss, To monitor and compare learning loss, The weight coefficients for supervised contrastive learning loss are P(i), where P(i) is the set of positive samples in the dataset that are of the same class as sample i. Let be the normalized feature vector of sample i, obtained by L2 normalization of the fused features from the previous layer of the classifier. This is the normalized feature vector of positive samples belonging to the same sleep stage category as sample i. This represents the normalized feature vectors of all samples in the current batch except for sample i. and These represent the dot product similarity between the corresponding feature vectors. This refers to the temperature parameter.

5. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 1, characterized in that, In step S3: The modality-specific adapter adopts a bottleneck structure, and the calculation formula is as follows: ； Where m is the modality index. Let m be the input features of modality m, and LN be the layer normalization operation. Here is the lower projection matrix used for feature dimensionality reduction. It is a non-linear activation function. The upprojection matrix is ​​used for feature dimensionality upscaling. To adapt the output characteristics of the post-modal m.

6. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 5, characterized in that, In step S3: The low-rank adaptation module assigns different adaptation ranks to different modalities in the pre-collected sleep data. The adaptation rank assigned to the EEG modality is greater than that assigned to the EOS modality and EMG modality. A low-rank adaptation matrix is ​​then introduced into the pre-trained sleep staging model. and The product of the two is used as the incremental correction to the original weight matrix in the model, and the calculation formula is as follows: ； in, Here is the original weight matrix for mode m. From Via A low-rank reduced-dimensional matrix of dimension 1, and , From Via A low-rank, high-dimensional matrix, and , The input dimension of the original weight matrix. The output dimension of the original weight matrix. For the fit rank assigned to mode m, The overall weight matrix for adapting to mode m; During model fine-tuning Keep it unchanged, only update and .

7. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 6, characterized in that, In step S3: A cross-modal low-rank coupling mechanism is introduced into the low-rank adaptation module to connect the low-rank adaptation matrix. Decomposed into low-rank basis shared by all modes and mode-specific low-rank basis ,in, Used by all modalities to capture common individual characteristics across modalities. Each modality possesses its own unique features, used to capture the individual characteristics specific to that modality.

8. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 7, characterized in that, In step S3: The model is fine-tuned using a multi-task loss function, calculated as follows: ； in, To fine-tune the total loss, For cross-entropy loss, For intra-individual contrast loss, For timing consistency loss, For regularization loss, , , These are the weighting coefficients corresponding to each loss term.

9. The personalized sleep state monitoring method based on multimodal physiological electrical signals according to claim 1, characterized in that, The method also introduces a weakly supervised domain adaptation mechanism, which uses the subjective sleep information preset by the target user as a weakly supervised signal to cooperate with the model for personalized fine-tuning. The subjective sleep information includes at least one of the following: sleep onset time, wake-up time, number of nighttime awakenings, and subjective sleep quality score. The constraints of the weak supervision signal include: boundary constraints that restrict the prediction of the period before falling asleep and after waking up as a state of wakefulness; a number constraint that restricts the number of times the predicted sleep state transitions to a state of wakefulness to be consistent with the number of times the target user reports wakefulness; and a quality constraint based on the positive correlation between the proportion of deep sleep and the subjective sleep quality score.

10. A personalized sleep state monitoring system for implementing the method according to any one of claims 1-9, characterized in that, The system includes the following modules: The signal acquisition module is used to acquire multimodal physiological electrical signals during the sleep process of the target user; The feature extraction module is used to preprocess and extract features from the input modal signals to obtain multimodal sleep signal features; The model pre-training module is used to build a sleep staging model that includes an intramodal feature encoder, a cross-modal attention fusion module, a temporal context modeling module, and a classification head, and to pre-train the model using a public sleep dataset; The model fine-tuning module, including a modality-aware low-rank adaptation module and a modality-specific adapter, is used to update the adaptation parameters using pre-collected sleep data from the target user while freezing the main parameters of the pre-trained sleep staging model, thereby performing personalized fine-tuning of the model. The classification and prediction module is used to classify and predict the sleep stages of target users based on a fine-tuned sleep staging model and real-time collected sleep data. The model fine-tuning module and classification prediction module are executed locally on the edge computing unit of the portable sleep monitoring device.

Images