Intelligent sleep monitoring blanket based on multi-modal physiological signal fusion and optimization method thereof

Abstract

The invention relates to the technical field of intelligent sleep monitoring blankets, in particular to an intelligent sleep monitoring blanket based on multi-modal physiological signal fusion and an optimization method of the intelligent sleep monitoring blanket. The signal acquisition end is used for acquiring multi-modal physiological signal data; the cloud server is used for processing and analyzing the physiological signal data, optimizing a signal acquisition process, performing sleep staging, performing sleep quality evaluation according to a sleep staging result, and providing a sleep quality report and suggestion for a user; the signal acquisition end transmits data to the cloud server end by using a Bluetooth protocol, and optimizes the acquisition process of each sensor according to a specific hierarchical architecture; the cloud server receives various multi-modal physiological signals and performs sleep data fusion, a deep learning algorithm is utilized to establish a sleep staging model to optimize a sleep environment, and an innovative multi-scale time sequence attention network and a multi-model integration framework are combined, so that the sleep staging accuracy is greatly improved, and a reliable basis is provided for sleep quality evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent sleep monitoring blankets, and more specifically, to an intelligent sleep monitoring blanket based on multimodal physiological signal fusion and an optimization method thereof. Background Art

[0002] As people's understanding of the importance of sleep health continues to deepen, sleep monitoring technology has made great progress in recent years. Traditional sleep monitoring mainly relies on polysomnography (PSG), which is highly accurate but complex to operate, expensive, and easily interferes with normal sleep. Therefore, the development of portable, non-invasive and accurate home sleep monitoring equipment has become a research hotspot.

[0003] Currently, there are many home sleep monitoring devices on the market, most of which are based on single or dual physiological signals. Wearable devices mainly use heart rate variability (HRV) data to estimate sleep state. However, such methods often have difficulty in accurately distinguishing between light sleep and deep sleep stages, resulting in incomplete sleep quality assessment.

[0004] On the other hand, some studies have tried to combine multiple physiological signals to improve monitoring accuracy. Some scholars have proposed a sleep apnea screening model that combines surface electromyography and body movement data. Although this method performs well in specific scenarios, it still has the following problems: First, the dimensions of the selected physiological signals are limited, and it is difficult to fully reflect the complex physiological changes during sleep; second, the machine learning algorithm used is relatively simple and fails to fully explore the potential correlation between multimodal data; finally, it lacks the ability to optimize the sleep environment in real time, making it difficult to provide users with personalized sleep improvement solutions.

[0005] In view of the shortcomings of existing technologies, there is an urgent need for an intelligent sleep monitoring blanket that can comprehensively collect multi-dimensional physiological signals and intelligently integrate and analyze them through advanced algorithms. At the same time, the intelligent sleep monitoring blanket based on multi-modal physiological signal fusion should also have the function of dynamically optimizing the sleeping environment to comprehensively improve the user's sleep quality. Summary of the invention

[0006] The intelligent sleep monitoring blanket based on multimodal physiological signal fusion and the optimization method thereof proposed in the present invention are designed to solve the above-mentioned technical problems.

[0007] The present invention provides an intelligent sleep monitoring blanket based on multimodal physiological signal fusion, comprising a signal acquisition end and a cloud service end; the signal acquisition end is used to obtain multimodal physiological signal data; the cloud service end is used to process and analyze the physiological signal data, optimize the signal acquisition process, perform sleep staging, and perform sleep quality assessment based on the sleep staging results, and provide sleep quality reports and suggestions to users; the signal acquisition end comprises a pressure matrix, a heartbeat sensor, a body temperature sensor, a humidity sensor and a microphone array, wherein the pressure matrix is ​​located on the surface of the intelligent sleep monitoring blanket, the heartbeat sensor is attached to the user's wrist, body The temperature sensor is attached to the user's chest, the humidity sensor is attached to the user's nose, and the microphone array is placed next to the ear; the signal acquisition end uses the Bluetooth protocol to transmit data to the cloud service end, and optimizes the acquisition process of each sensor according to a specific hierarchical architecture; the Bluetooth protocol includes three layers, namely, the access layer, the connection layer and the application layer from bottom to top; the standards of the access layer and the connection layer are the Bluetooth 4.0 technical standards, and the standard of the application layer is Bluetooth low energy BLE; the cloud service end receives various multimodal physiological signals and performs sleep data fusion, uses a deep learning algorithm to establish a sleep staging model, and optimizes the sleep environment.

[0008] Specifically, the sleep data fusion includes denoising, feature extraction, screening and fusion processing; the deep learning algorithm includes a multimodal signal fusion algorithm, a multi-scale temporal attention network MS-TAN, the design and training of a sleep staging model, and the design of a multi-model integration framework, and a sleep data set is used to construct and verify the above network; the design and training of the sleep staging model includes a five-layer structure, namely a data preprocessing layer, a first feature extraction layer, a second feature extraction layer, an attention layer, and a fully connected layer; the data preprocessing layer integrates multimodal data into a common data sequence; the first feature extraction layer uses convolutional layers and pooling layers to extract local low-level features; the second feature extraction layer uses LSTM and Flatten layers to extract high-level temporal features; the attention layer uses Scaled The Dot-Product layer is used to calculate the weights of each order of memory in the current hidden state during the activation operation; the fully connected layer uses the ReLU layer, the BN layer, and the Pooling layer to perform the final sleep category classification; the multi-scale temporal attention network is combined with the multi-model integration framework, and multiple time scale capture mechanisms are used together to explore the dynamic relationship between behavioral characteristics and rules at different time scales, improve the accuracy of sleep state discrimination, and enhance the classification performance of multimodal physiological signals; the multi-model integration framework integrates sleep staging models under variable scenarios, and achieves robustness verification and performance optimization of sleep staging results by reasonably balancing the differences between different models.

[0009] Specifically, the combination of the pressure matrix and multiple pressure sensors is used to collect the position distribution of the user on the surface of the smart sleep monitoring blanket; the heartbeat sensor, body temperature sensor and humidity sensor are all wearable sensors, which are connected to the smart sleep monitoring blanket via Bluetooth 4.0; the microphone array is placed next to the ear to collect snoring and breathing sound signals, and is connected to the smart sleep monitoring blanket via Bluetooth 4.0; the cloud service end includes a data fusion unit, a sleep staging unit, and a sleep quality assessment unit, and is also responsible for the dynamic optimization and control of the sleep environment; the data fusion unit is used to receive multimodal physiological signals, and then perform data preprocessing, feature extraction, screening and fusion; and according to the sleep staging As a result, the sleep environment is adaptively controlled based on multimodal physiological signals; the sleep staging unit is used to perform sleep staging on the results of data fusion using a sleep staging model; the sleep quality assessment unit is used to assess the sleep quality of multimodal physiological signals, including multi-dimensional health factor analysis, sleep staging accuracy, sleep staging precision assessment, sleep staging result verification, and sleep staging result presentation; the sleep environment dynamic optimization control realizes personalized sleep environment optimization based on long-term sleep characteristics, the sleep staging results perform feedback control on the sleep environment, the system realizes dynamic optimization of the intelligent sleep environment based on the fuzzy control strategy, and uses a genetic algorithm to optimize the fuzzy predictive control parameters.

[0010] Specifically, the multimodal signal fusion algorithm includes four steps: denoising, feature extraction, screening and fusion processing. Specifically: the denoising module is used to filter out part of the noise; the data normalization module removes the heterogeneity between different modal data, unifies the data format, eliminates noise, and improves data quality. The data normalization module retains effective physiological signal components based on empirical mode decomposition; the feature extraction module uses a multi-scale attention mechanism to identify important features in different modal time series; the multi-scale attention includes an attention unit that captures features of different scales, a network unit that fuses cross-scale information, and a feature extraction layer that combines features, which is conducive to integrating multimodal data; the attention layer is used for self-supervised learning, and a multimodal attention mechanism is proposed. The feature extraction module uses an attention unit to extract time coefficients, calculate the correlation between time contexts, and select elements with differences to generate significant context-aware attention features for the original features of different modalities; the screening module uses a variety of classifiers for preliminary screening, including SVM classifiers, Adaboost classifiers, and One-Class classifiers. The SVM classifier can ultimately screen out 1-2 classifiers while retaining the specificity of multiple classifiers and reducing the inherent differences of the classifiers. The feature fusion module uses improved feature-level fusion and decision-level fusion to supplement the features of physiological signals of different modalities and improve the robustness and separability of features of data of different modalities. At the same time, the fused physiological signal data is integrated into a common data sequence, and the fused features constitute new data features for the sleep staging model.

[0011] The optimization method using the intelligent sleep monitoring blanket comprises:

[0012] (1) The data preprocessing layer integrates multimodal data into a common data sequence;

[0013] (2) The first feature extraction layer uses convolutional layers and pooling layers to extract local low-level features;

[0014] (3) The second feature extraction layer uses LSTM and Flatten layers to extract high-level temporal features;

[0015] (4) The attention layer uses a fully connected weighted attention layer and uses Scaled Dot-Product and Softmax transformations to perform weighted processing, calculate the weights of each order of memory in each hidden state when activated, and finally determine the sleep category;

[0016] (5) Multi-model integration framework, which realizes the integration of multiple sleep staging models and finally outputs the sleep staging results.

[0017] Specifically, the multi-model integration framework adopts a linear combination strategy for decision-level fusion and improves the credibility of sleep staging results by weighted combination of sleep staging models.

[0018] Specifically, the multi-model integration framework uses a linear combination strategy for decision-level fusion. Specifically, there are multiple sleep stage models, and the fusion formula of the linear combination strategy is as follows:

[0019] Y = argmax(Σ(i = 1 to n)W i *P i)

[0020] Among them, Y is the sleep stage category, P i is the confidence of the i-th model in predicting the sleep classification, W i is the weight of the i-th sleep stage model in a certain sleep classification;

[0021] According to the empirical value, the sleep stage model is set to provide the credibility of predicting sleep classification for decision-level fusion, and the least squares method is used to predict W of n sleep stage models. i , the output of the integrated sleep staging results is achieved through linear combination.

[0022] Specifically, based on the fusion results of multimodal physiological data, a multi-scale temporal attention network MS-TAN is used to perform sleep staging, and a sleep staging model is established.

[0023] Specifically, the modeling method of the sleep staging model is as follows: the MS-TAN model consists of 5 layers, including a Scaled Dot-Product layer for evaluating the relationship between input features and hidden states; a fully connected LSTM layer containing multiple hidden units and a gate to control the flow of data; an attention unit using Scaled Dot-Product to calculate the correlation between temporal contexts; an attention layer using a fully connected ReLU layer to increase nonlinearity while reducing computational complexity; an output layer using a probabilistic linear classifier to classify sleep stages.

[0024] Specifically, the sleep staging model adopts a binary classification form to construct multiple sleep data sets of different sizes, and constructs the data sets through random sampling or specific rule-based methods.

[0025] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0026] By integrating pressure matrix, heart rate sensor, body temperature sensor, humidity sensor and microphone array, the present invention realizes comprehensive monitoring of the user's sleep state. Combined with the innovative multi-scale temporal attention network (MS-TAN) and multi-model integration framework, the present invention greatly improves the accuracy of sleep staging and provides a more reliable basis for sleep quality assessment.

[0027] In addition, the present invention also includes a sleep environment dynamic optimization module based on fuzzy control and genetic algorithm, which can adaptively adjust environmental parameters such as room temperature, humidity, and light according to real-time monitoring results to create the best sleeping conditions for users. This function not only fills the gap in the existing technology, but also opens up a new direction for personalized sleep health management.

[0028] In general, this invention significantly improves the accuracy and functionality of home sleep monitoring through multimodal signal fusion, deep learning algorithm optimization and intelligent environment control. It not only provides users with more comprehensive and accurate sleep health information, but also actively optimizes the sleep environment and effectively improves sleep quality. This innovation is expected to play an important role in the early diagnosis of sleep disorders, prevention of chronic diseases, and health management, and has broad application prospects and significant social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a logic block diagram of the signal acquisition end of the present invention.

[0030] Figure 2 It is a logic block diagram of the intelligent sleep monitoring blanket of the present invention.

[0031] Figure 3 It is a logic block diagram of the optimization method of the present invention. DETAILED DESCRIPTION

[0032] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects thereof are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0033] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0034] Please refer to Figure 1-3The present invention relates to an intelligent sleep monitoring blanket based on multimodal physiological signal fusion and an optimization method thereof. The intelligent sleep monitoring blanket includes a signal acquisition end and a cloud service end, collects the user's physiological signal data through a variety of sensors, and uses a deep learning algorithm to process the data, perform sleep staging and sleep quality assessment, and finally provide the user with a sleep quality report and suggestions.

[0035] First, the signal acquisition end of the present invention includes a pressure matrix 1, a heartbeat sensor 2, a body temperature sensor 3, a humidity sensor 4 and a microphone array 5. Among them, the pressure matrix 1 is located on the surface of the smart sleep monitoring blanket, and is used to collect the position distribution information of the user on the blanket. The heartbeat sensor 2 is attached to the user's wrist, the body temperature sensor 3 is attached to the user's chest, and the humidity sensor 4 is attached to the user's nose. These three sensors are all wearable designs and are connected to the main system via Bluetooth 4.0 technology. The microphone array 5 is placed next to the ear to collect snoring and breathing sound signals.

[0036] Preferably, in one embodiment of the present invention, the pressure matrix 1 adopts a flexible capacitive sensor array with a resolution of 32x32 and a sampling frequency of 20Hz. This design can accurately capture the user's body movement information without affecting sleep comfort. The heartbeat sensor 2 adopts photoplethysmography (PPG) technology with a sampling frequency of 100Hz, and can provide continuous heart rate variability data. The accuracy of the body temperature sensor 3 reaches ±0.1℃, and the sampling frequency is 0.1Hz, which is enough to capture subtle changes in body temperature at night. The response time of the humidity sensor 4 is less than 1 second, the accuracy is ±2%RH, and it can promptly reflect the humidity changes during breathing. The microphone array 5 includes 4 omnidirectional microphones with a sampling rate of 16kHz and a signal-to-noise ratio greater than 60dB, which can accurately capture snoring and breathing sound signals.

[0037] Next, the signal acquisition end uses the Bluetooth protocol to transmit the collected data to the cloud service end. The Bluetooth protocol consists of three layers, namely the access layer, the connection layer, and the application layer from bottom to top. Among them, the access layer and the connection layer use the Bluetooth 4.0 technical standard, while the application layer uses the Bluetooth low energy BLE standard. This layered design helps to optimize the acquisition process of each sensor and improve the efficiency and stability of data transmission. For example, by using BLE technology, the system can achieve a data transmission rate of up to 1Mbps while maintaining low power consumption, meeting the real-time transmission requirements of multimodal physiological signals.

[0038] Then, after receiving the multimodal physiological signals, the cloud server first performs sleep data fusion. This process includes four steps: denoising, feature extraction, screening, and fusion processing. In the denoising step, the system uses the wavelet transform method to filter out high-frequency noise. Specifically, the signal is decomposed into 5 layers using the db4 wavelet basis, and then denoised using the soft threshold method. The threshold is set to 5% of the signal energy. This empirical value can effectively remove most of the environmental noise without losing the effective signal.

[0039] In the feature extraction step, the present invention innovatively introduces a multi-scale attention mechanism. The system designs three attention units with different time scales, focusing on time windows of 5 seconds, 30 seconds, and 5 minutes respectively. Each attention unit calculates the attention weight by the following formula:

[0040] A(t)=softmax(W2*tanh(W1*X(t)+b1)+b2)

[0041] Among them, X(t) is the input feature at time t, and W1, W2, b1, and b2 are learnable parameters. This mechanism can effectively capture important features at different time scales, such as instantaneous changes in heart rate (5-second window), breathing patterns (30-second window), and sleep cycle changes (5-minute window).

[0042] In the feature screening stage, the system uses a combination of SVM classifiers, Adaboost classifiers, and One-Class SVM classifiers. Through 10-fold cross-validation, classifiers with a classification accuracy of more than 85% are retained. It is worth mentioning that the One-Class SVM classifier is particularly suitable for detecting abnormal sleep patterns. Its kernel function uses the RBF kernel and the parameter ν is set to 0.1. This empirical value performs well in practice and can effectively identify abnormal events such as sleep apnea.

[0043] Finally, in the feature fusion stage, the present invention adopts the improved Dempster-Shafer evidence theory. Specifically, for each physiological signal S i , we define its basic probability allocation function (BPA) as m i Then, the fused BPA was calculated by the following formula:

[0044]

[0045] Among them, w i is the weight of each signal, and the initial value is set to (n is the number of signal types), which is subsequently optimized by gradient descent. K is the conflict factor, which is used to quantify the degree of contradiction between different evidence sources. This method can effectively handle the uncertainty and conflict of multimodal data and improve the reliability of fusion results.

[0046] Next, the cloud server uses a deep learning algorithm to build a sleep staging model. The model uses a multi-scale temporal attention network (MS-TAN) structure, which consists of five layers: data preprocessing layer, first feature extraction layer, second feature extraction layer, attention layer, and fully connected layer.

[0047] The data preprocessing layer first normalizes the multimodal data to the interval [-1, 1], and then uses the linear interpolation method to align the data with different sampling rates to the same time axis, with the sampling frequency unified to 100 Hz. This step ensures the consistency of subsequent processing.

[0048] The first feature extraction layer uses a 1D convolutional network with 64 convolution kernels of size 3, a step size of 1, and padding set to 'same'. This layer mainly extracts local low-level features, such as the QRS complex features of the electrocardiogram. The second feature extraction layer uses an LSTM network with 128 hidden units and a time step of 30 (corresponding to a 30-second data window). LSTM can effectively capture long-term dependencies and is suitable for extracting high-level temporal features such as sleep cycles.

[0049] The attention layer is the core of MS-TAN and uses the Scaled Dot-Product mechanism. Its mathematical expression is as follows:

[0050]

[0051] Where Q, K, and V represent query, key, and value matrices, respectively. k is the dimension of the key vector. It plays a scaling role and prevents the inner product from being too large, which causes the softmax function gradient to disappear. This mechanism can adaptively focus on important information at different time points, greatly improving the model's ability to recognize complex sleep patterns.

[0052] Finally, the fully connected layer adopts a two-layer structure. The first layer uses the ReLU activation function with 64 nodes. The second layer uses the softmax activation function with 6 nodes, corresponding to 6 sleep stages (awake, REM, N1, N2, N3, N4). Before the fully connected layer, a batch normalization operation is introduced, with the parameters β initialized to 0 and γ initialized to 1, which helps to accelerate model convergence and improve generalization ability.

[0053] In order to further improve the classification performance and robustness, this paper adopts a multi-model integration framework. Specifically, we trained 5 MS-TAN models with different structures or initializations, and then performed decision-level fusion through a linear combination strategy. The fusion formula is as follows:

[0054] Y = argmax(Σ(i = 1 to 5)W i *P i)

[0055] Among them, Y is the final sleep stage category, P i is the confidence of the i-th model predicting the sleep classification, W i is the weight of the i-th model. i The initial value of is set to 0.2 (i.e. 1 / 5), and then optimized by least squares method on the validation set. We found that this ensemble method can improve the overall accuracy of sleep staging by 3%-5%, especially for the difficult-to-distinguish N1 and N2 stages, the improvement is more obvious.

[0056] Preferably, in one embodiment of the present invention, the sleep staging model adopts a binary classification form, that is, the sleep state is first divided into two categories, sleep and wakefulness, and then the sleep state is further subdivided. This hierarchical classification strategy can effectively reduce the complexity of the model and improve the overall classification accuracy. Specifically, we first constructed a balanced data set containing 10,000 samples, of which sleep and wakefulness each accounted for 50%. Then, for the subdivision of sleep states, we adopted a stratified sampling method to ensure that the sample ratio of each sleep stage is close to the distribution in actual sleep (for example, the N2 stage accounts for about 45%-55%, and the REM stage accounts for about 20%-25%).

[0057] In addition, the present invention also includes a sleep quality assessment unit and a sleep environment dynamic optimization control unit. The sleep quality assessment unit is based on multi-dimensional health factor analysis, including sleep efficiency (total sleep time / bed time), sleep latency, the proportion of each sleep stage, the number and duration of night awakenings, etc. For example, for adults, we define poor sleep quality as sleep efficiency less than 85%, sleep latency more than 30 minutes, deep sleep (N3+N4) less than 15%, or more than 5 awakenings at night. These thresholds are determined based on a large amount of clinical research data, but the system will make appropriate adjustments based on individual factors such as the user's age and gender.

[0058] The sleep environment dynamic optimization control unit realizes intelligent adjustment based on fuzzy control strategy. The unit takes environmental factors such as room temperature, humidity, and light as input, and sleep quality score as output, and performs reasoning through the fuzzy rule base. For example, for temperature adjustment, we define five fuzzy sets: Very Cold (<18℃), Cold (18-22℃), Comfortable (22-26℃), Warm (26-30℃), Very Warm (>30℃). The control rule examples are as follows:

[0059] F(Deep_Sleep_Ratio is Low)AND(Temperature is Warm)THEN Decrease_Temperature, where the membership function for "Low" is defined as:

[0060]

[0061] This refined control strategy can dynamically adjust environmental parameters according to the user's real-time sleep state, thereby creating optimal sleeping conditions. In addition, the present invention also uses a genetic algorithm to optimize fuzzy control parameters. Specifically, we set the population size to 50, the crossover probability to 0.8, the mutation probability to 0.1, and the iteration to 100 generations. These parameter settings perform well in practice and can find near-optimal control parameters within a reasonable computing time.

[0062] In summary, the present invention achieves high-precision sleep monitoring and personalized sleep environment optimization through multimodal physiological signal fusion, deep learning algorithm and intelligent control strategy. This method can not only accurately evaluate the user's sleep quality, but also dynamically adjust the sleep environment according to individual characteristics, thereby improving the user's sleep experience in all aspects. Through continuous data collection and algorithm optimization, this system is expected to provide valuable data support for sleep health research and provide a scientific basis for the formulation of personalized sleep improvement programs.

[0063] The present invention proposes an optimization method for an intelligent sleep monitoring blanket based on multimodal physiological signal fusion, which aims to achieve high-precision sleep monitoring and personalized sleep environment optimization through multi-dimensional physiological data collection and deep learning algorithms. The specific implementation steps and core technical points of this method will be described in detail below.

[0064] First, in step S1, the intelligent sleep monitoring blanket of the present invention obtains multimodal physiological signal data through its signal acquisition end. Specifically, the pressure matrix 1 is located on the surface of the blanket and is used to capture the user's body position changes and movement patterns. The heartbeat sensor 2 is attached to the user's wrist to collect heart rate variability data. The body temperature sensor 3 is attached to the user's chest to monitor body temperature fluctuations. The humidity sensor 4 is attached near the user's nose to record humidity changes during breathing. The microphone array 5 is placed next to the ear to collect snoring and breathing sound signals. These sensors transmit data to the system main control unit in real time via Bluetooth 4.0 technology.

[0065] Preferably, in one embodiment of the present invention, the pressure matrix 1 adopts a 32x32 flexible capacitive sensor array with a sampling frequency of 20Hz. This design can accurately capture the user's body movement information without affecting sleep comfort. The heartbeat sensor 2 adopts photoplethysmography (PPG) technology, and the sampling frequency is set to 100Hz to provide continuous heart rate variability data. The accuracy of the body temperature sensor 3 reaches ±0.1℃, and the sampling frequency is 0.1Hz, which is sufficient to capture subtle changes in body temperature at night. The response time of the humidity sensor 4 is less than 1 second, and the accuracy is ±2%RH, which can timely reflect the humidity changes during breathing. The microphone array 5 includes 4 omnidirectional microphones, with a sampling rate of 16kHz and a signal-to-noise ratio greater than 60dB. The setting of these parameters ensures the accurate capture of snoring and breathing sound signals.

[0066] Next, in step S2, the system performs fusion processing on the collected multimodal physiological signal data. This process includes four sub-steps: denoising, feature extraction, screening, and fusion. In the denoising stage, the system uses the wavelet transform method, selects the db4 wavelet basis, and decomposes the signal into 5 layers. By setting a threshold (usually 3%-5% of the signal energy), high-frequency noise is removed while retaining the important features of the signal. This threshold range is based on a large amount of experimental data and can achieve a good balance between denoising effect and signal fidelity.

[0067] Then, in the feature extraction stage, the present invention innovatively introduces a multi-scale attention mechanism. Specifically, the system designs three attention units with different time scales, focusing on time windows of 5 seconds, 30 seconds, and 5 minutes respectively. Each attention unit calculates the attention weight using the following formula:

[0068] A(t)=softmax(W2*tanh(W1*X(t)+b1)+b2)

[0069] Among them, X(t) is the input feature at time t, and W1, W2, b1, and b2 are learnable parameters. This mechanism can effectively capture important features at different time scales, such as instantaneous changes in heart rate (5-second window), breathing patterns (30-second window), and sleep cycle changes (5-minute window).

[0070] In the feature screening stage, the system uses a combination of SVM classifiers, Adaboost classifiers, and One-Class SVM classifiers. Through 10-fold cross-validation, classifiers with a classification accuracy of more than 85% are retained. It is worth mentioning that the One-Class SVM classifier is particularly suitable for detecting abnormal sleep patterns. Its kernel function uses the RBF kernel and the parameter ν is set to 0.1. This empirical value performs well in practice and can effectively identify abnormal events such as sleep apnea.

[0071] Finally, in the feature fusion stage, the present invention adopts the improved Dempster-Shafer evidence theory. Specifically, for each physiological signal S i , we define its basic probability allocation function (BPA) as m i Then, the fused BPA was calculated by the following formula:

[0072]

[0073] Among them, w i is the weight of each signal, and the initial value is set to (n is the number of signal types), which is subsequently optimized by gradient descent. K is the conflict factor, which is used to quantify the degree of contradiction between different evidence sources. This method can effectively handle the uncertainty and conflict of multimodal data and improve the reliability of fusion results.

[0074] In step S3, the system performs sleep stage classification on the fused data based on the multi-scale temporal attention network (MS-TAN). The MS-TAN model consists of five layers: data preprocessing layer, first feature extraction layer, second feature extraction layer, attention layer, and fully connected layer.

[0075] In the data preprocessing layer, the system first standardizes the data of different modes and unifies the value range to the interval [-1,1]. Then, through the linear interpolation method, all data are aligned to the same time axis, and the sampling frequency is unified to 100Hz. This step ensures the consistency of subsequent processing.

[0076] The first feature extraction layer uses a 1D convolutional network with 64 convolution kernels of size 3, a step size of 1, and padding set to 'same'. This layer mainly extracts local low-level features, such as the QRS complex features of the electrocardiogram. The second feature extraction layer uses an LSTM network with 128 hidden units and a time step of 30 (corresponding to a 30-second data window). LSTM can effectively capture long-term dependencies and is suitable for extracting high-level temporal features such as sleep cycles.

[0077] The attention layer is the core of MS-TAN and uses the Scaled Dot-Product mechanism. Its mathematical expression is as follows:

[0078]

[0079] Where Q, K, and V represent query, key, and value matrices, respectively. k is the dimension of the key vector. It plays a scaling role and prevents the inner product from being too large, which causes the softmax function gradient to disappear. This mechanism can adaptively focus on important information at different time points, greatly improving the model's ability to recognize complex sleep patterns.

[0080] Finally, the fully connected layer adopts a two-layer structure. The first layer uses the ReLU activation function with 64 nodes. The second layer uses the softmax activation function with 6 nodes, corresponding to 6 sleep stages (awake, REM, N1, N2, N3, N4). Before the fully connected layer, a batch normalization operation is introduced, with the parameters β initialized to 0 and γ initialized to 1, which helps to accelerate model convergence and improve generalization ability.

[0081] In order to further improve the classification performance and robustness, this paper adopts a multi-model integration framework. Specifically, we trained 5 MS-TAN models with different structures or initializations, and then performed decision-level fusion through a linear combination strategy. The fusion formula is as follows:

[0082] Y = argmax(Σ(i = 1 to 5)W i *P i )

[0083] Among them, Y is the final sleep stage category, P i is the confidence of the i-th model in predicting the sleep classification, W i is the weight of the ith model. The initial value of Wi is set to 0.2 (i.e. 1 / 5), and then optimized by the least squares method on the validation set. We found that this ensemble method can improve the overall accuracy of sleep staging by 3%-5%, especially for the difficult-to-distinguish N1 and N2 stages, the improvement effect is more obvious.

[0084] Preferably, in one embodiment of the present invention, the sleep staging model adopts a binary classification form, that is, the sleep state is first divided into two categories, sleep and wakefulness, and then the sleep state is further subdivided. This hierarchical classification strategy can effectively reduce the complexity of the model and improve the overall classification accuracy. Specifically, we first constructed a balanced data set containing 10,000 samples, of which sleep and wakefulness each accounted for 50%. Then, for the subdivision of sleep states, we adopted a stratified sampling method to ensure that the sample ratio of each sleep stage is close to the distribution in actual sleep (for example, the N2 stage accounts for about 45%-55%, and the REM stage accounts for about 20%-25%).

[0085] In step S4, the system evaluates sleep quality based on the sleep staging results. Evaluation indicators include sleep efficiency (total sleep time / time in bed), sleep latency, proportion of each sleep stage, number and duration of awakenings at night, etc. For example, for adults, we define poor sleep quality as sleep efficiency less than 85%, sleep latency more than 30 minutes, deep sleep (N3+N4) proportion less than 15%, or more than 5 awakenings at night. These thresholds are determined based on a large amount of clinical research data, but the system will make appropriate adjustments based on individual factors such as the user's age and gender.

[0086] Finally, it is worth mentioning that the present invention also includes a sleep environment dynamic optimization control unit. This unit is based on a fuzzy control strategy, taking environmental factors such as room temperature, humidity, and light as input, and sleep quality score as output. For example, for temperature regulation, we define five fuzzy sets: Very Cold (<18°C), Cold (18-22°C), Comfortable (22-26°C), Warm (26-30°C), Very Warm (>30°C). The control rule examples are as follows:

[0087] F(Deep_Sleep_Ratio is Low)AND(Temperature is Warm)THEN Decrease_Temperature, where the membership function for "Low" is defined as:

[0088]

[0089] This refined control strategy can dynamically adjust environmental parameters according to the user's real-time sleep state, thereby creating optimal sleeping conditions. In addition, the present invention also uses a genetic algorithm to optimize fuzzy control parameters. Specifically, we set the population size to 50, the crossover probability to 0.8, the mutation probability to 0.1, and the iteration to 100 generations. These parameter settings perform well in practice and can find near-optimal control parameters within a reasonable computing time.

[0090] In summary, the present invention achieves high-precision sleep monitoring and personalized sleep environment optimization through multimodal physiological signal fusion, deep learning algorithm and intelligent control strategy. This method can not only accurately evaluate the user's sleep quality, but also dynamically adjust the sleep environment according to individual characteristics, thereby improving the user's sleep experience in all aspects. Through continuous data collection and algorithm optimization, this system is expected to provide valuable data support for sleep health research and provide a scientific basis for the formulation of personalized sleep improvement programs.

[0091] In order to verify the superiority of the intelligent sleep monitoring blanket based on multimodal physiological signal fusion and its optimization method of the present invention, we designed a series of experiments to compare the embodiments of the present invention with the prior art solutions. The following is a detailed experimental design, result analysis and discussion.

[0092] Example 1: Intelligent sleep monitoring blanket of the present invention

[0093] In this embodiment, we adopt a complete multimodal physiological signal fusion solution, including pressure matrix, heartbeat sensor, body temperature sensor, humidity sensor and microphone array. The data processing adopts multi-scale temporal attention network (MS-TAN) and multi-model integration framework.

[0094] Comparative Example 1: Single Physiological Signal Monitoring Solution

[0095] This scheme only uses heart rate variability (HRV) data for sleep monitoring and adopts the traditional support vector machine (SVM) algorithm for sleep staging.

[0096] Comparative Example 2: Dual-modal physiological signal monitoring solution

[0097] The scheme combines heart rate variability and breathing signals and uses a long short-term memory network (LSTM) for sleep staging.

[0098] Comparative Example 3: Multimodal Physiological Signal Monitoring without Deep Learning Optimization

[0099] This scheme uses the same sensor configuration as the present invention, but adopts traditional machine learning algorithms (such as random forests) for data processing and sleep staging.

[0100] We selected the following key indicators to evaluate the performance of each solution:

[0101] 1. Sleep staging accuracy: Using polysomnography (PSG) as the gold standard, the consistency between automatic staging results and manual scoring was calculated.

[0102] 2. Sleep efficiency estimation error: Compare the deviation of estimated sleep efficiency with PSG results.

[0103] 3. Sleep latency estimation error: Compare the deviation of the estimated sleep onset time with the PSG results.

[0104] 4. User comfort rating: Through a questionnaire survey, ask users to rate the comfort of the equipment (1-10 points).

[0105] We recruited 50 volunteers (25 males and 25 females, aged 25-60 years old), and each person used each regimen for 7 consecutive nights. PSG monitoring was performed as a control. The experimental results are shown in the following table:

[0106] Sleep stage accuracy 92.5％ 76.3％ 83.7％ 85.2％ Sleep efficiency estimation error ±2.1％ ±7.8％ ±5.3％ ±4.6％ Sleep latency estimation error ±3.5 minutes ±12.6 minutes ±8.9 minutes ±7.2 minutes User comfort rating 8.7 9.2 8.9 8.6

[0107] It can be seen from the experimental results that Example 1 of the present invention is significantly better than other comparative solutions in most key indicators:

[0108] 1. Sleep staging accuracy: Example 1 achieved an accuracy of 92.5%, much higher than other solutions. This is due to the fusion of multimodal signals and advanced deep learning algorithms, which can capture richer and more subtle sleep characteristics.

[0109] 2. Sleep efficiency and sleep latency estimation: Example 1 has the smallest error, which is ±2.1% and ±3.5 minutes respectively. This accuracy is crucial for assessing sleep quality and diagnosing potential sleep problems.

[0110] 3. User comfort: Although the score of Example 1 is slightly lower than that of Comparative Examples 1 and 2, it still remains at a very high level (8.7 points). Considering that this solution contains more sensors, this score actually reflects our success in device integration and miniaturization.

[0111] It is worth noting that although Comparative Example 3 also uses multimodal signals, its performance is significantly lower than that of the embodiment of the present invention due to the lack of advanced deep learning algorithms. This fully demonstrates the importance of the MS-TAN and multi-model integration framework we proposed.

[0112] These results clearly show that the present invention significantly improves the accuracy and comprehensiveness of sleep monitoring while ensuring user comfort through multimodal signal fusion and advanced algorithm processing. In particular, the advantages of the present invention are more obvious in the core indicator of sleep staging. Accurate sleep staging not only helps to more accurately assess sleep quality, but also provides a solid foundation for subsequent sleep disorder diagnosis and personalized recommendations.

[0113] In addition, although the hardware complexity of the present invention is higher than some comparative solutions, we still achieve a longer battery life and higher user comfort through careful system design and optimization. This means that the present invention is not only leading in technical indicators, but also highly competitive in practical applications and user experience.

[0114] In general, these experimental results strongly demonstrate the innovation and advancement of this invention in the field of sleep monitoring. Through the intelligent fusion of multimodal signals and the optimization of deep learning algorithms, our smart sleep monitoring blanket provides users with an unprecedented accurate, comprehensive and comfortable sleep monitoring solution. This not only helps personal health management, but also provides a valuable tool for sleep research and early diagnosis of related diseases.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, replacement, and improvement made within the principle of the present invention should be included in the protection scope of the present invention.

Claims

1. Intelligent sleep monitoring blanket based on multimodal physiological signal fusion, Features: The invention comprises a signal acquisition end and a cloud service end; the signal acquisition end is used to obtain multimodal physiological signal data; the cloud service end is used to process and analyze the physiological signal data, optimize the signal acquisition process, perform sleep staging, and perform sleep quality assessment based on the sleep staging results, and provide sleep quality reports and suggestions to users; the signal acquisition end comprises a pressure matrix, a heartbeat sensor, a body temperature sensor, a humidity sensor and a microphone array, wherein the pressure matrix is ​​located on the surface of the intelligent sleep monitoring blanket, the heartbeat sensor is attached to the user's wrist, the body temperature sensor is attached to the user's chest, the humidity sensor is attached to the user's nose, and the microphone array is placed beside the ear; the signal acquisition end transmits data to the cloud service end using the Bluetooth protocol, and optimizes the acquisition process of each sensor according to a specific hierarchical architecture; the Bluetooth protocol comprises three layers, namely, the access layer, the connection layer and the application layer from bottom to top; wherein the standards of the access layer and the connection layer are the Bluetooth 4.0 technical standards, and the standard of the application layer is the Bluetooth low energy BLE; the cloud service end receives various multimodal physiological signals and performs sleep data fusion, establishes a sleep staging model using a deep learning algorithm, and optimizes the sleep environment.

2. The smart sleep monitoring blanket according to claim 1, Features: The sleep data fusion includes denoising, feature extraction, screening and fusion processing; the deep learning algorithm includes a multimodal signal fusion algorithm, a multi-scale temporal attention network MS-TAN, the design and training of a sleep staging model, and the design of a multi-model integration framework, and a sleep data set is used to build and verify the above network; the design and training of the sleep staging model includes a five-layer structure, namely a data preprocessing layer, a first feature extraction layer, a second feature extraction layer, an attention layer, and a fully connected layer; the data preprocessing layer integrates multimodal data into a common data sequence; the first feature extraction layer uses a convolutional layer and a pooling layer to extract local low-level features; the second feature extraction layer uses LSTM and Flatten layers to extract high-level temporal features; the attention layer uses Scaled The Dot-Product layer is used to calculate the weights of each order of memory in the current hidden state during the activation operation; the fully connected layer uses the ReLU layer, the BN layer, and the Pooling layer to perform the final sleep category classification; the multi-scale temporal attention network is combined with the multi-model integration framework, and multiple time scale capture mechanisms are used together to explore the dynamic relationship between behavioral characteristics and rules at different time scales, improve the accuracy of sleep state discrimination, and enhance the classification performance of multimodal physiological signals; the multi-model integration framework integrates sleep staging models under variable scenarios, and achieves robustness verification and performance optimization of sleep staging results by reasonably balancing the differences between different models.

3. The smart sleep monitoring blanket according to claim 2, Features: The combination of the pressure matrix and multiple pressure sensors is used to collect the position distribution of the user on the surface of the smart sleep monitoring blanket; the heartbeat sensor, body temperature sensor and humidity sensor are all wearable sensors, connected to the smart sleep monitoring blanket via Bluetooth 4.0; the microphone array is placed next to the ear to collect snoring and breathing sound signals, and connected to the smart sleep monitoring blanket via Bluetooth 4.0; the cloud service end includes a data fusion unit, a sleep staging unit, and a sleep quality assessment unit, and is also responsible for dynamic optimization and control of the sleep environment; the data fusion unit is used to receive multimodal physiological signals, and then perform data preprocessing, feature extraction, screening and fusion; and according to the sleep staging results, the sleep environment is adaptively controlled based on the multimodal physiological signals; The sleep staging unit is used to perform sleep staging on the results of data fusion using a sleep staging model; the sleep quality assessment unit is used to assess the sleep quality of multimodal physiological signals, including multi-dimensional health factor analysis, sleep staging accuracy, sleep staging precision assessment, sleep staging result verification, and sleep staging result presentation; the sleep environment dynamic optimization control realizes personalized sleep environment optimization based on long-term sleep characteristics, and the sleep staging results perform feedback control on the sleep environment. The system realizes dynamic optimization of the intelligent sleep environment based on a fuzzy control strategy, and uses a genetic algorithm to optimize fuzzy predictive control parameters.

4. The smart sleep monitoring blanket according to claim 2, Features: The multimodal signal fusion algorithm includes four steps: denoising, feature extraction, screening and fusion processing. Specifically, the denoising module is used to filter out some noise; the data normalization module removes the heterogeneity between different modal data, unifies the data format, eliminates noise, and improves data quality. The data normalization module retains effective physiological signal components according to empirical mode decomposition; The feature extraction module uses a multi-scale attention mechanism to identify important features in different modal time series; the multi-scale attention includes an attention unit that captures features of different scales, a network unit that fuses cross-scale information, and a feature extraction layer that combines features, which is conducive to integrating multimodal data; the attention layer is used for self-supervised learning, and a multimodal attention mechanism is proposed. The feature extraction module uses an attention unit to generate significant context-aware attention features for the original features of different modalities by extracting time coefficients, calculating the correlation between time contexts, and selecting elements with differences; the screening module uses a variety of classifiers for preliminary screening, including SVM classifiers, Adaboost classifiers, and One-Class SVM classifiers, and finally selects 1-2 classifiers while retaining the specificity of multiple classifiers and reducing the inherent differences of classifiers; the feature fusion module uses improved feature level fusion and decision level fusion to supplement the features of physiological signals of different modalities and improve the robustness and separability of features of data of different modalities; at the same time, the fused physiological signal data is integrated into a common data sequence, and the fused features constitute new data features for the sleep staging model.

5. An optimization method for the smart sleep monitoring blanket according to any one of claims 1 to 4, It is characterized in that include: (1) The data preprocessing layer integrates multimodal data into a common data sequence; (2) The first feature extraction layer uses convolutional layers and pooling layers to extract local low-level features; (3) The second feature extraction layer uses LSTM and Flatten layers to extract high-level temporal features; (4) The attention layer uses a fully connected weighted attention layer and uses Scaled Dot-Product and Softmax transformations to perform weighted processing, calculate the weights of each order of memory in each hidden state when activated, and finally determine the sleep category; (5) Multi-model integration framework, which realizes the integration of multiple sleep staging models and finally outputs the sleep staging results.

6. The smart sleep monitoring blanket optimization method according to claim 5, Features: The multi-model integration framework adopts a linear combination strategy for decision-level fusion and improves the credibility of sleep staging results by weighted combination of sleep staging models.

7. The intelligent sleep monitoring blanket optimization method according to claim 6, Features: The multi-model integration framework uses a linear combination strategy for decision-level fusion. Specifically, there are multiple sleep stage models, and the fusion formula of the linear combination strategy is as follows: Y=argmax(Σ(i=1to n)W i *P i) Among them, Y is the sleep stage category, P i is the confidence of the i-th model in predicting the sleep classification, W i is the weight of the i-th sleep stage model in a certain sleep classification; According to the empirical value, the sleep stage model is set to provide the credibility of predicting sleep classification for decision-level fusion, and the least squares method is used to predict W of n sleep stage models. i , the output of the integrated sleep staging results is achieved through linear combination.

8. The smart sleep monitoring blanket optimization method according to claim 5, Features: According to the fusion results of multimodal physiological data, the multi-scale temporal attention network MS-TAN is used to perform sleep staging, and a sleep staging model is established.

9. The smart sleep monitoring blanket optimization method according to claim 8, Features: The modeling method of the sleep staging model is as follows: the MS-TAN model consists of 5 layers, including a Scaled Dot-Product layer for evaluating the relationship between input features and hidden states; a fully connected LSTM layer containing multiple hidden units and a gate to control the flow of data; an attention unit using Scaled Dot-Product to calculate the correlation between temporal contexts; an attention layer using a fully connected ReLU layer to increase nonlinearity while reducing computational complexity; an output layer using a probabilistic linear classifier to classify sleep stages.

10. The smart sleep monitoring blanket optimization method according to claim 9, Features: The sleep staging model adopts a binary classification form to construct multiple sleep data sets of different sizes, and constructs the data sets through random sampling or specific rule-based methods.

Images