A sleep monitoring and adaptive adjustment method fusing multi-modal physiological signals

Abstract

The application discloses a sleep monitoring and adaptive adjustment method fusing multi-modal physiological signals, acquires first electroencephalogram (EEG) feature sequences, first cardiovascular feature sequences, first respiratory feature sequences and first body movement feature sequences of a patient during sleep; inputs the four kinds of feature sequences into respective corresponding one-dimensional convolutional neural networks, time sequence convolutional networks and two bidirectional LSTM networks respectively to obtain respective corresponding extracted feature sequences; utilizes a cross-modal transformer network to perform cross-modal fusion on the extracted feature sequences to obtain fused feature sequences; analyzes and processes the fused feature sequences based on a monitoring network and a personalized baseline to obtain target sleep staging and target pathological risk indexes of the patient, and performs adaptive adjustment on a sleep environment of the patient according to the target sleep staging and the target pathological risk indexes.

Description

Technical Field

[0001] This invention relates to the field of intelligent health monitoring, and relates to, but is not limited to, a method for sleep monitoring and adaptive adjustment that integrates multimodal physiological signals. Background Technology

[0002] With the accelerating pace of life, increasing work pressure, and a rapidly aging population, sleep disorders have become a global public health issue. According to the World Health Organization, approximately 30% of adults worldwide experience some degree of sleep disorder, with obstructive sleep apnea prevalence reaching as high as 24% in middle-aged men. Periodic limb movement disorder and insomnia are also widespread. Long-term untreated sleep disorders not only significantly reduce quality of life but are also closely associated with hypertension, arrhythmia, cognitive decline, and an increased risk of traffic accidents.

[0003] In related technologies, on the one hand, sleep staging monitoring and adaptive adjustment are achieved through devices such as wristbands, watches, chest straps, and smartwatches. However, this method relies on single or limited signal sources such as body movement, EEG, or heart rate, making it difficult to accurately identify complex EEG activity characteristics such as deep sleep and REM sleep, resulting in large staging errors and insufficient reliability of adaptive adjustment. On the other hand, while schemes based on deep learning multimodal monitoring models integrate multi-source signals, they still suffer from drawbacks such as difficulty in generalizing population models to individual differences, poor robustness to pathological populations, and a lack of closed-loop adjustment mechanisms.

[0004] Therefore, how to more accurately monitor patients' sleep in a personalized manner, and how to adaptively adjust the sleep environment based on the monitoring results, has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a sleep monitoring and adaptive adjustment method that integrates multimodal physiological signals, solving the problem that related technologies cannot accurately monitor patients' sleep in a personalized manner, and adaptively adjust the sleep environment based on the monitoring results.

[0006] According to a first aspect of the present invention, a method for sleep monitoring and adaptive regulation integrating multimodal physiological signals is provided. The method is implemented through a sleep monitoring model, which includes a one-dimensional convolutional neural network, a temporal convolutional network, a bidirectional LSTM network, a cross-modal transformer network, and a monitoring network. The method includes: Obtain the patient's first EEG characteristic sequence, first cardiovascular characteristic sequence, first respiratory characteristic sequence, and first body movement characteristic sequence during sleep; The first EEG feature sequence is input into a one-dimensional convolutional neural network to obtain the second EEG feature sequence; The first cardiovascular feature sequence is input into a temporal convolutional network to obtain the second cardiovascular feature sequence; The first respiratory feature sequence and the first body movement feature sequence are respectively input into their respective bidirectional LSTM networks to obtain the hidden layer state sequence; A cross-modal transformer network was used to fuse the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence to obtain a fused feature sequence. The fusion feature sequences are analyzed and processed based on the monitoring network and personalized baseline to obtain the patient's target sleep stage and target pathological risk index. The patient's sleep environment is then adaptively adjusted according to the target sleep stage and target pathological risk index.

[0007] According to a second aspect of the present invention, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect.

[0008] According to the scheme provided in the embodiments of the present invention, a first EEG feature sequence, a first cardiovascular feature sequence, a first respiratory feature sequence, and a first body movement feature sequence of a patient during sleep are obtained; the first EEG feature sequence is input into a one-dimensional convolutional neural network to obtain a second EEG feature sequence; the first cardiovascular feature sequence is input into a temporal convolutional network to obtain a second cardiovascular feature sequence; the first respiratory feature sequence and the first body movement feature sequence are respectively input into their respective bidirectional LSTM networks to obtain hidden state sequences; the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequences are fused across modally using a cross-modal transformer network to obtain a fused feature sequence; the fused feature sequence is analyzed and processed based on a monitoring network and a personalized baseline to obtain the patient's target sleep stage and target pathological risk index, and the patient's sleep environment is adaptively adjusted according to the target sleep stage and target pathological risk index. In this process, multimodal data sequences were collected, improving the accuracy of sleep staging and the detection rate of pathological events, providing a rich information foundation for subsequent analysis. Multiple networks processed the modal data independently, avoiding mutual interference between different modalities while preserving the unique information structure of each modality. A single-layer cross-modal Transformer network was used to fuse the feature sequences of the four modalities. This fusion method effectively utilizes complementary information between modalities and achieves robustness enhancement through a cross-modal attention mechanism in the case of incomplete or noisy multimodal signals, thereby improving the accuracy of sleep staging and pathological event detection. The monitoring network provides recognition capabilities, and personalized baselines introduce individual physiological difference corrections; the combination of these two ensures both basic accuracy and adaptive adjustment. Environmental regulation based on sleep staging and pathological risk indicators can achieve a closed loop from passive monitoring to active intervention, improving sleep quality and reducing health risks. In summary, this method allows for more accurate monitoring of patients' sleep and adaptive adjustment of the sleep environment based on monitoring results. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating a sleep monitoring and adaptive adjustment method that integrates multimodal physiological signals, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention. Based on the examples in the present invention, all other examples obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] In the following description, references are made to “some examples”, which describe a subset of all possible embodiments. However, it is understood that “some examples” may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0012] It should be noted that the terms "first, second, third" used in the examples of this invention are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the examples of this invention described herein can be implemented in an order other than that illustrated or described herein.

[0013] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments of the invention pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0014] Figure 1 This is a flowchart illustrating a sleep monitoring and adaptive adjustment method that integrates multimodal physiological signals according to an embodiment of the present invention. The sleep monitoring and adaptive adjustment method that integrates multimodal physiological signals according to an embodiment of the present invention can be executed by an electronic device, which can be a server.

[0015] like Figure 1 As shown, a sleep monitoring method integrating multimodal physiological signals includes the following steps S101 to S106: S101. Obtain the first EEG characteristic sequence, first cardiovascular characteristic sequence, first respiratory characteristic sequence and first body movement characteristic sequence of the patient during sleep.

[0016] Specifically, while the patient is asleep, a three-channel electroencephalogram (EEG) sequence of Fz, Cz, and Pz can be acquired through a dry electrode headband worn by the user. A photoplethysmography (PPG) sequence is then acquired by a dual-wavelength (red light + infrared) optical sensor, and peripheral capillary oxygen saturation is calculated based on the PPG sequence. ) sequence, then based on PPG sequence and The sequence comprises a cardiovascular signal sequence; a miniature thermal probe on the pillow captures the nasal and oral airflow sequence; a flexible piezoelectric breathing band records the chest and abdominal respiratory movement sequence. Then, a respiratory signal sequence is constructed based on the nasal and oral airflow sequence and the chest and abdominal respiratory movement sequence. A three-axis accelerometer and gyroscope built into the worn wristband sense body movement and sleeping posture, constructing a body movement data sequence. Finally, the first EEG feature sequence (first EEG feature sequence), first cardiovascular feature sequence, first respiratory feature sequence, and first body movement feature sequence are extracted from the cardiovascular signal sequence, respiratory signal sequence, body movement data sequence, and three-channel EEG sequence, respectively.

[0017] The first EEG feature sequence, the first cardiovascular feature sequence, the first respiratory feature sequence, and the first body movement feature sequence can be formed by sorting the feature vectors corresponding to different time periods in chronological order. These different time periods include the current time period and historical time periods, which will be explained in detail later.

[0018] S102. Input the first EEG feature sequence into a one-dimensional convolutional neural network to obtain the second EEG feature sequence.

[0019] Specifically, the first EEG feature sequence is input into the first convolutional layer of a one-dimensional (1D) convolutional neural network for feature extraction, resulting in a processed EEG feature sequence. This processed EEG feature sequence is then input into the second convolutional layer to obtain the second EEG feature sequence. Both the first and second convolutional layers have a kernel size of 5, with the first convolutional layer having 64 channels and the second convolutional layer having 128 channels.

[0020] S103. Input the first cardiovascular feature sequence into the temporal convolutional network to obtain the second cardiovascular feature sequence.

[0021] Specifically, the first cardiovascular feature sequence is input into a temporal convolutional network, which consists of three cascaded convolutional layers with expansion ratios of 1, 2, and 4, respectively. Feature extraction is performed sequentially through the three cascaded convolutional layers to obtain the second cardiovascular feature sequence.

[0022] S104. Input the first respiratory feature sequence and the first body movement feature sequence into their respective bidirectional LSTM networks to obtain the hidden layer state sequence.

[0023] Specifically, two identical bidirectional LSTM networks are proposed, each with 64 hidden units. The first respiratory feature sequence is input into the corresponding bidirectional LSTM network for processing. The non-stationary event sequence is modeled by jointly using the forward and backward states to obtain the corresponding hidden state sequence.

[0024] Simultaneously, the first body motion feature sequence is input into the corresponding bidirectional LSTM network for processing to capture the temporal periodic structure of the body motion event and output the corresponding hidden state sequence.

[0025] S105. Use a cross-modal transformer network to perform cross-modal fusion of the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence to obtain the fused feature sequence.

[0026] Specifically, the second EEG feature sequence, the second cardiovascular feature sequence, and the two hidden state sequences are added element-wise with their respective identity codes. The four feature sequences are then stacked and input into a cross-modal transformer network for cross-modal fusion to obtain the fused feature sequence.

[0027] S106. Based on the monitoring network and personalized baseline, the fused feature sequences are analyzed and processed to obtain the patient's target sleep stage and target pathological risk index, and the patient's sleep environment is adaptively adjusted according to the target sleep stage and target pathological risk index.

[0028] Specifically, the sleep stages used include: wake (representing wakefulness), N1 (representing sleep onset), N2 (representing light sleep), N3 (representing deep sleep), and rapid eye movement (REM) sleep. The stage with the highest probability among all stages is selected as the final target sleep stage. Target pathological risk indicators include the probability of obstructive sleep apnea (OSA), the probability of micro-arousals, the periodic limb movement (PLM) index, and the rate of decrease in blood oxygen saturation. The OSA probability refers to the occurrence of a clinically significant obstructive sleep apnea event; the PLM index refers to the periodic limb movement index; the micro-arousal risk probability reflects the degree of disruption to sleep continuity; and the rate of decrease in blood oxygen saturation is the expected rate of decrease, used to provide early warning of hypoxic events. The personalized baseline is a set of statistical indicators calculated by analyzing a user's undisturbed sleep data from multiple complete nights. This baseline is used to partially correct the output of the sleep monitoring model, making the risk assessment more consistent with the user's individual characteristics, ultimately resulting in the target sleep stage and target pathological risk indicators.

[0029] Furthermore, the timing of intervention is determined based on the target sleep stage, such as prohibiting disturbances during deep sleep and REM sleep, while adjusting the sleep environment during other stages. Environmental control actions are executed by comparing target pathological risk indicators with their respective thresholds. When multiple control actions are simultaneously present, sleep environment regulation is prioritized by decreasing the rate of decrease in blood oxygen saturation, OSA probability, micro-arousal risk probability, and PLM index in that order.

[0030] For example, when the OSA probability is not less than the OSA probability threshold, such as 0.8, the head of the bed is raised by 10° and the right side vibrates once; when the PLM index is not less than the PLM index threshold, such as 25, the calf heating pad is activated, targeting 40°C; when the microarousal risk probability is not less than the microarousal probability threshold, such as 0.7, 45dB pink noise is played; when the rate of decrease in blood oxygen saturation is not greater than the rate of decrease in blood oxygen saturation threshold, such as -1.5% / min, the head of the bed is raised by 15° and the left side is vibrated.

[0031] Among them, the OSA probability threshold is based on the clinical high-risk standard, with a preferred value of 0.8; the PLM index threshold is based on the clinical threshold, with a preferred value of 25; the micro-arousal probability threshold balances sensitivity and false alarm rate, and a 70% probability of triggering intervention can effectively capture most micro-arousal events while avoiding frequent false triggers, so a preferred value of 0.7 is used; the blood oxygen saturation decline rate threshold is derived from the clinical reference value, that is, a blood oxygen saturation decline rate exceeding 1.5% / minute indicates a significant risk of hypoxia.

[0032] In some embodiments of the present invention, S105 can be implemented by the following steps: inputting the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence into the multi-head self-attention layer in the cross-modal transformer network; adding the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence to their respective corresponding identity codes element by element to obtain the added feature sequence; stacking the added feature sequences to obtain the target feature sequence; and obtaining the fused feature sequence based on the target feature sequence.

[0033] Specifically, the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence are each element-wise added to their corresponding identity codes of the same dimension to obtain the summed feature sequence. These summed feature sequences are then stacked to obtain the target feature sequence, or target feature matrix. This target feature sequence is mapped to a query matrix Q, a key matrix K, and a value matrix V through three independent linear projection layers. The query matrix Q, key matrix K, and value matrix V are then divided into multiple parts along the feature dimension, and each part is input to each attention head for weighted summation, resulting in a weighted feature sequence output by each head. Finally, the weighted feature sequences output by each head are concatenated to obtain the final weighted feature sequence.

[0034] Furthermore, the final weighted feature sequence is processed through cascaded layer normalization, feedforward neural network, residual connections, and final layer normalization to obtain the final fused feature sequence. The feedforward neural network consists of two fully connected layers. The first layer expands the dimension from 128 to 256, and after passing through the ReLU activation function, the second layer compresses it back to 128 dimensions.

[0035] The four modal feature sequences are of the same dimension as their respective identity codes. The four identity codes are randomly initialized to form a learnable parameter matrix. This learnable parameter matrix is ​​placed in the input embedding layer of the cross-modal transformer network and updated synchronously with other parameters through backpropagation during training. After training is completed, the four modal feature sequences are added element by element to their respective identity codes.

[0036] In some embodiments of the present invention, S106 can be implemented by the following steps: compressing the dimensionality of the fused feature sequence through a global average pooling module to obtain a compressed feature vector; and using a fully connected layer in the sleep monitoring module to perform stage identification on the compressed feature vector to obtain an initial sleep stage. The compressed feature vector is subjected to risk detection by a cascaded multilayer perceptron in the pathology monitoring module to obtain the current pathology risk index; the fused feature sequence is predicted by a unidirectional LSTM network and a fully connected layer in the pathology monitoring module to obtain the initial blood oxygen saturation decline rate; the initial sleep stage, the current pathology risk index and the initial blood oxygen saturation decline rate are corrected by a personalized baseline to obtain the target sleep stage and the target pathology risk index.

[0037] Specifically, the monitoring network includes a global pooling module, a sleep monitoring module, and a pathology monitoring module. The sleep monitoring module contains a fully connected layer and an activation function. The fused feature sequence is input into the global average pooling module for dimensionality compression, resulting in a compressed feature vector. This compressed feature vector is then input into the sleep monitoring module, processed by the fully connected layer and activation function to obtain the target sleep stage. The pathology monitoring module contains two parallel branches. The first branch contains a cascaded multilayer perceptron, and the second branch contains a unidirectional LSTM network and a fully connected layer. The compressed feature vector is input into the first branch, processed by the cascaded multilayer perceptron to obtain the current pathological risk indicator. The compressed feature vector is then input into the second branch, processed by the unidirectional LSTM network to obtain the hidden state vector. The fully connected layer further processes the hidden state vector to obtain the initial rate of decrease in blood oxygen saturation. The current pathological risk indicators include OSA probability, micro-arousal risk probability, and... The index consists of all pathological risk indicators output by the pathological monitoring module, which are composed of the current pathological risk indicators and the predicted initial rate of decline in blood oxygen saturation. Finally, some pathological risk indicators are corrected through personalized baselines to obtain the final target pathological risk indicator.

[0038] In some embodiments of the present invention, before partially correcting the initial sleep stage, current pathological risk indicators, and initial rate of decline of blood oxygen saturation using a personalized baseline to obtain the target sleep stage and target pathological risk indicators, the following steps are also included: collecting the patient's current complete sleep night, dividing the current complete sleep night into multiple sleep cycles according to a first preset time period; calculating the ratio of low-frequency power to high-frequency power of the mean body energy, respiratory rate, and heart rate variability based on the three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence, and body motion sequence corresponding to each sleep cycle; when the mean body energy, respiratory rate, and ratio all meet their respective threshold conditions, the corresponding sleep cycle is taken as an effective sleep cycle; when the number of effective sleep cycles is not less than the threshold number, the comprehensive signal efficiency corresponding to the current complete sleep night is obtained; when the comprehensive signal efficiency is not less than the threshold efficiency, the current complete sleep night is taken as the target night, and a personalized baseline is constructed based on the target night.

[0039] Specifically, the low-frequency power of heart rate variability is 0.04 ~ 0.15 Hz, and the high-frequency power is 0.15 ~ 0.40 Hz. A complete sleep night can be 7 hours. The patient's current complete sleep night is collected. A sleep cycle can be 90 ~ 110 minutes, so the first preset time period can be 90 ~ 110 minutes. After dividing the current complete sleep night into 90 ~ 110 minutes, multiple sleep cycles are obtained. Each sleep cycle includes a three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence, and body movement sequence collected within that cycle. The mean body movement energy is calculated from the body movement sequence. The number of complete peaks or troughs in the respiratory signal sequence is counted. The counted number is divided by the number of sleep cycles to obtain the respiratory rate. Simultaneously, the LF / HF ratio of heart rate variability (HRV) is calculated in the cardiovascular sequence. When the mean body energy is not greater than the corresponding threshold condition, such as 0.2g, the respiratory rate is stable within the corresponding threshold condition, such as 12~25 breaths / minute, and the LF / HF ratio of heart rate variability (HRV) is not greater than the threshold condition, such as 2, then the complete sleep cycle is judged as an effective sleep period, and then the next sleep cycle is judged.

[0040] Among them, since normal adults have very little body movement during resting sleep, the threshold is set to 0.2g to distinguish between resting and active states; since the resting respiratory rate of normal adults is 12-25 breaths / minute, this range is used to determine stable breathing during non-rapid eye movement sleep; since the parasympathetic nervous system is dominant during resting sleep and the LF / HF ratio is usually less than 2, the threshold is set to 2 to determine that the autonomic nervous system is in a resting state.

[0041] Furthermore, if the number of complete sleep cycles meeting the above conditions in the current complete sleep night is not less than 4, the overall signal efficiency corresponding to the current complete sleep night is obtained. If the overall signal efficiency corresponding to the current complete sleep night is not less than the threshold efficiency, such as 85%, the current complete sleep night is taken as the target night, and a personalized baseline is constructed based on the three-channel EEG sequence, total cardiovascular signal sequence, total respiratory signal sequence, and total dynamic sequence corresponding to the target night. The personalized baseline includes a robust baseline and a minimum blood oxygen saturation. Since the normal number of sleep cycles for adults is 4 to 6, a minimum of 4 is set as the minimum standard. Because engineering experience requires that more than 85% of the data for the entire night is valid to ensure statistical reliability, the threshold is set to 85%. If the number of complete sleep cycles meeting the above conditions in the current complete sleep night is less than 4, or if the overall signal efficiency corresponding to the current complete sleep night is less than the threshold efficiency, the judgment for the next complete sleep night continues.

[0042] In some embodiments of the present invention, obtaining the comprehensive signal efficiency corresponding to the current complete sleep night can be achieved through the following steps: dividing the current complete sleep night into multiple sub-time periods according to a second preset time period; calculating the respective quality scores of the three-channel EEG sequence, cardiovascular signal sequence, and respiratory sequence in each sub-time period; deleting invalid sub-time periods from the multiple sub-time periods according to the quality scores and a preset deletion condition to obtain target sub-time periods; calculating the total quality score of each target sub-time period, and calculating the ratio of the total quality score to the number of target sub-time periods to obtain the comprehensive signal efficiency.

[0043] Specifically, the second preset time period can be 30 seconds. Divide the 7-hour complete sleep night into groups of data with 30 seconds as a group. For the three-channel EEG sequence collected in each group, measure the electrode impedance sequence of each channel through the corresponding measuring device, and then calculate the average electrode impedance of each channel according to the electrode impedance sequence. Perform a fast Fourier transform on the EEG sequence of each channel to convert it into a frequency-domain power spectrum, and calculate the power of the frequency band on the basis of the frequency-domain power spectrum, and then calculate the total power of all frequency bands in the frequency-domain power spectrum. Calculate the ratio of the power of the frequency band to the total power to obtain the power ratio. For each channel, if the average electrode impedance is not greater than 50 kΩ and the rhythm power ratio is not less than 20%, the quality of the channel is qualified. If all three channels are qualified, the quality score corresponding to the three-channel EEG sequence is 1, otherwise it is 0. Because the signal attenuation and power frequency interference are small when the electrode impedance is lower than 50 kΩ, and high-quality electroencephalogram signals can be recorded, the threshold is set to 50 kΩ; because in the electroencephalogram of healthy adults in the closed-eye resting state, the rhythm power ratio is usually greater than 20%, the threshold is set to 20% to confirm the signal validity and physiological authenticity.

[0044] Among them, for the PPG sequence collected in each group, input the PPG sequence into the pulse wave peak detection model to obtain multiple pulse peaks and the confidence level corresponding to each pulse peak. If the average value of the confidence level is not less than 0.9, the quality score corresponding to the PPG sequence is 1, otherwise it is 0; for the sequence collected in each group, for each collected at the sampling time point, calculate the corresponding R value, count the number of R values within the range of 0.3 to 1.0, calculate the ratio of the counted number to the total number. If the ratio is not less than 90%, the quality score is 1, otherwise it is 0. Combine the PPG sequence and The quality scores of the sequences are weighted and summed to obtain the total quality score for the cardiovascular signal sequence. Because engineering experience requires at least 90% of the data points to be valid to ensure statistical reliability, the threshold is set to 90%. Since 0.3–1.0 is the physiologically valid range for R-values, this range is used to determine the validity of R-values.

[0045] For the respiratory sequence, the variance of the airflow signal and the correlation coefficient between the chest and abdominal signals are calculated. If the variance is not less than 0.1 and the correlation coefficient is not less than 0.7, the quality score of the respiratory sequence is 1; otherwise, it is 0. Because the variance of the airflow signal is usually greater than 0.1 during normal breathing, and a value below 0.1 often indicates noise or apnea, the threshold is set to 0.1. Because the correlation coefficient is usually greater than 0.7 during normal synchronous chest and abdominal breathing, and a value below 0.7 suggests asynchronous chest and abdominal movement, the threshold is set to 0.7.

[0046] Furthermore, the preset deletion conditions are as follows: In each 30-second data set, the five modal sequences each correspond to a quality score. When more than two quality scores are both 0, the set is deleted. When three or more consecutive sets need to be deleted, the two adjacent sets within those three sets are also deleted, resulting in the target sub-time period. All quality scores within the target sub-time period are then weighted and summed according to preset weights to obtain the total quality score for each target sub-time period. The sum of the preset weights is 1. After calculating all the total quality scores, the ratio is calculated to the total number of target sub-time periods N to obtain the overall signal effectiveness.

[0047] In some embodiments of the present invention, a personalized baseline can be constructed based on a target night by the following steps: calculating a robust baseline and a minimum blood oxygen saturation based on the cardiovascular signal sequence corresponding to the target sub-time period in the target night; and constructing a personalized baseline using the robust baseline and the minimum blood oxygen saturation.

[0048] Specifically, based on each target sub-time period, in all target sub-time periods... In the sequence of sampling points Robust baseline and determine minimum A robust baseline and a minimum baseline for blood oxygen saturation are used as personalized baselines.

[0049] In some embodiments of the present invention, S101 can be implemented by the following steps: acquiring multiple sets of signal data of the patient during sleep in a third time period; the multiple sets of signal data include historical signal data and current signal data; each set of signal data includes a multimodal data sequence, which includes a corresponding three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence and body movement data sequence; extracting features from each of the multimodal data sequences to obtain a feature vector corresponding to each set; the feature vector corresponding to each set includes an EEG feature vector, a cardiovascular feature vector, a respiratory feature vector and a body movement feature vector; concatenating each set of feature vectors according to the time dimension to obtain a first EEG feature sequence, a first cardiovascular feature sequence, a first respiratory feature sequence and a first body movement feature sequence, respectively.

[0050] Specifically, each set of signal data can be obtained by dividing a preset third time period into sub-time segments of fixed duration, and then obtaining the signal data corresponding to each sub-time segment. For example, a 150-second signal can be divided into five 30-second sub-segments, and the multimodal signal data sequence acquired within the 150 seconds can be divided accordingly. Each 30-second segment contains the acquired three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence, and body motion data sequence. The 150-second time window consists of historical signal data and current signal data, that is, it consists of the current 30 seconds of data and the historical 120 seconds of data. That is, each time data processing is triggered, the data to be processed in the current time is composed of the data acquired in the current 30 seconds, combined with the historical 120 seconds of data. In other words, a sliding window mechanism can be used during data acquisition and processing, with a window length of 150 seconds and a step size of 30 seconds. Every time a new 30-second data is acquired, the window slides forward 30 seconds, continuously acquiring and updating the multimodal signal data sequence to achieve continuous processing of the data stream. A three-channel EEG sequence refers to an electroencephalogram (EEG) signal sequence acquired from three different electrode locations, such as Fz, Cz, and Pz.

[0051] Furthermore, feature extraction is performed on the three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence, and body motion data sequence included in each group to obtain the EEG feature vector, cardiovascular feature vector, respiratory feature vector, and body motion feature vector contained in each group. Then, the EEG feature vectors in the five groups are concatenated in chronological order to obtain the first EEG feature sequence. The cardiovascular feature vectors are concatenated in chronological order to obtain the first cardiovascular feature sequence. The first respiratory feature sequence and the first body motion feature sequence are obtained in the same way.

[0052] Specifically, in determining the EEG feature vectors for each group, the three-channel EEG sequences in each group are processed using Fast Fourier Transform (FFT) to convert the time signal to the frequency domain, and the feature vectors for each channel are calculated in the frequency domain. , , and For each channel, four frequency band power values ​​can be extracted. The three channels are combined into a 12-dimensional EEG feature vector. Then, the EEG feature vectors of all groups are concatenated in chronological order to obtain the first EEG feature sequence.

[0053] The cardiovascular signal sequences in each group consist of PPG sequences and In determining the characteristic sequence of cardiovascular signals, for the PPG sequence, the pulse wave peak corresponding to each heartbeat can be identified using an adaptive dual-threshold peak detection algorithm or the wavelet transform modulus maxima method. Then, based on at least two consecutive detected pulse wave peaks, a pulse interval sequence is constructed by calculating the interval between adjacent pulse wave peaks. According to this pulse interval sequence, the root mean square of the difference between each pair of adjacent pulse intervals is calculated using a sliding window differencing algorithm, yielding the root mean square of successive differences (RMSSD). Then, the standard deviation of the pulse interval sequence is calculated, yielding the standard deviation of normal-to-normal intervals (SDNN). The power spectral density of the pulse interval sequence is estimated using the Welch periodogram method or an autoregressive model spectrum, calculating the ratio (LF / HF) between the low-frequency band power (e.g., 0.04-0.15 Hz) and the high-frequency band power (e.g., 0.15-0.4 Hz). Sequence, extract three dynamic features, including based on Sequence computation Minimum And the calculations in each group Compared to the previous set of calculations The rate of decline between them. This is calculated using RMSSD, SDNN, LF / HF, and [other parameters] for each group. Minimum The descent rate forms the corresponding cardiovascular feature vector for each group. Then, the cardiovascular feature vectors of all groups are concatenated in chronological order to obtain the first cardiovascular feature sequence.

[0054] For each group of motion data sequences, the composite amplitude of triaxial acceleration at each sampling time point is calculated, and a motion event detection threshold is set. The motion event detection threshold is determined by having the subject perform specific actions (such as rolling over, turning the head, and raising the arm) and recording the composite amplitude of triaxial acceleration for each action. The minimum or average of the maximum composite amplitude of triaxial acceleration among all actions is taken as the threshold, for example, 0.3. The composite amplitude of triaxial acceleration corresponding to each sampling time point is compared with 0.3, and sampling time points exceeding the threshold are marked. At least two consecutive sampling points exceeding the threshold are grouped into one motion event, and the total number of motion events in each group is counted as the motion event count. The total number of composite amplitudes of triaxial acceleration in each group is calculated and divided by the total number of sampling time points to obtain the average motion energy. The pitch and roll angles are calculated using three-dimensional acceleration to determine body position. Supine is defined as... and Left lateral decubitus position is defined as... Right lateral decubitus position is defined as... Prone position is defined as and After determining the body position, it is coded: supine = 0, left lateral decubitus = 1, right lateral decubitus = 2. The frequency of each code in each group is counted, and the code with the highest frequency is taken as the dominant conductor position code of that window. Then, the proportion of each body position in all body positions is calculated, and the body position switching entropy is further calculated. The body movement feature vector of each group is constructed by the dominant body position code, the count of body movement events, the average body movement energy, and the body position switching entropy. The body movement feature vectors of all groups are concatenated in chronological order to obtain the first body movement feature sequence.

[0055] In this process, high-pass filtering was applied to the nasal airflow signal at each sampling time point in each group to remove baseline drift. An amplitude threshold was set, and the total time within each group that the airflow was continuously below this threshold was recorded to obtain the airflow interruption duration. For the chest and abdominal respiratory motion signals, a sliding window integration method was used. The absolute value of the chest and abdominal respiratory motion signal at each sampling time point was calculated within each window. The total respiratory effort was obtained by summing the integrals of all windows. The ratio of the total respiratory effort to the total duration of 30 seconds for each group was calculated to obtain the average respiratory effort. Bandpass filtering was applied to the chest and abdominal respiratory motion signals to preserve the respiratory frequency band. Then, cross-correlation analysis was used to calculate the time delay corresponding to the maximum correlation of the respiratory frequency band. The time delay was converted into a phase difference to obtain the... By measuring the duration of airflow interruption, breathing effort, and The respiratory features of each group are constructed, and the respiratory features of all groups are spliced ​​together in chronological order to obtain the first respiratory feature sequence.

[0056] In some embodiments of the present invention, the target pathological risk indicator is obtained by partially correcting the current pathological risk indicator and the initial rate of decline of blood oxygen saturation using a personalized baseline. This can be achieved through the following steps: determining the lowest blood oxygen saturation and the mean blood oxygen saturation from the cardiovascular signal sequence of the current group of signal data; calculating a first difference between the robust baseline of blood oxygen saturation and the mean blood oxygen saturation; matching the first difference with a preset correction library to obtain a first correction amount; and correcting the initial rate of decline of blood oxygen saturation using the first correction amount to obtain the target rate of decline of blood oxygen saturation; calculating the minimum... The second difference between the lowest blood oxygen saturation and the second difference is used to obtain the second correction value. The OSA probability in the current pathological risk indicator is corrected by the second correction value to obtain the target OSA probability. The target pathological risk indicator is constructed based on the target OSA probability, the target blood oxygen saturation decline rate, the micro-arousal risk probability, and the PLM index.

[0057] Specifically, the lowest and mean blood oxygen saturation values ​​are determined from the current set of signal data, i.e., the cardiovascular signal sequence acquired within the current 30 seconds. The difference between a robust baseline and the mean blood oxygen saturation is calculated, and this first difference is matched with a preset correction library to obtain a first correction amount. This first correction amount is then used to adjust the initial rate of decrease in blood oxygen saturation, resulting in a corrected rate of decrease in blood oxygen saturation. For example, if the first difference is within the range of -5% to 0%, the first correction amount is 1.05 times the initial rate of decrease in blood oxygen saturation; if the first difference is within the range of -10% to -5%, the correction amount is 1.10 times the initial rate of decrease in blood oxygen saturation; if the first difference is not greater than -10%, the first correction amount is 1.20 times the initial rate of decrease in blood oxygen saturation; if the first difference is not less than 0%, the first correction amount is 0, i.e., no correction is performed. The corrected rate of decrease in blood oxygen saturation is obtained by multiplying the initial rate of decrease in blood oxygen saturation by the above correction amount.

[0058] At the same time, the smallest The difference between the sum of the oxygen saturation and the lowest blood oxygen saturation is calculated, and the calculated second difference is matched with a preset correction library to obtain a second correction amount. This second correction amount is then used to adjust the OSA probability upwards or downwards to obtain the corrected OSA probability. For example, if the second difference is in the range of -5% to 0%, the second correction amount is +0.05; if the second difference is in the range of -10% to -5%, the second correction amount is +0.10; if the second difference is not greater than -10%, the second correction amount is +0.20; and if the second difference is not less than 0%, the correction amount is 0.

[0059] The pre-defined calibration library is constructed through offline statistical analysis of a large amount of clinical trial data or historical user feedback. Specifically, it collects the deviations between monitored values ​​and baselines for different individuals under various physiological conditions, along with empirical values ​​that achieve the best correction effect under those deviations. These deviations are then divided into continuous intervals and assigned corresponding correction values, forming a lookupable mapping library. In subsequent use, only the current deviation needs to be calculated and matched with an interval in the library to directly obtain the correction value, without the need for online retraining or optimization.

[0060] In embodiments of the present invention, the sleep monitoring model is trained using data from publicly available datasets (Sleep-EDF, SHHS) and clinical cohorts from collaborating hospitals, encompassing approximately 1200 subjects, including 400 healthy subjects, 600 OSA patients, and 200 PLMD patients, aged 18-65 years. Each user is provided with data from one full night without intervention and five nights (allowing data from both interventional and non-interventional states) including EEG, PPG, respiratory flow, etc. Data including body movement was used; the former was used to build a personalized baseline, while the latter, after annotation, was used for model training. In the labeling process, two certified technicians independently completed the sleep stage annotation according to the AASM 2007 standard, using 30-second units for staging. Disagreements were arbitrated by a third expert to ensure consistency. Pathological events were precisely labeled according to clinical definitions; obstructive apnea was defined as a ≥90% drop in airflow lasting more than 10 seconds, occurring consecutively at least four times. These annotation results were used for supervised learning of the model. The sleep monitoring model was trained using training data. The sleep monitoring module and pathology monitoring module output target sleep stages and target pathological risk indicators. Substituting the target sleep stage and actual sleep stage into the cross-entropy loss function yielded the first loss. For pathological risk indicators, losses were calculated separately according to task type: OSA probability and micro-arousal probability used binary cross-entropy loss, while PLM index and blood oxygen saturation decline rate used mean squared error loss. The sum of these four terms yielded the second loss. The first loss and the second loss are weighted and summed according to their respective preset weights to obtain the total loss. The sleep monitoring model is then trained end-to-end using the total loss until the model converges. The sum of the preset weights is 1.

[0061] The optimizer used is AdamW, with an initial learning rate of The weight decay coefficient is To balance convergence speed and overfitting risk, a batch processing strategy was set to include 8 users × 8 epochs per batch (64 samples in total), ensuring that gradient updates reflect both individual differences and group patterns. Training was configured for 50 epochs, with an early stopping mechanism set to patience=8, automatically terminating training when validation set performance no longer improves.

[0062] In some embodiments of the present invention, data collected for 30 seconds is processed until the data collected for the current new night is processed. The data collected for 30 seconds is grouped together, and then all the data for the current new night is judged according to the method of constructing a personalized baseline to determine whether the effective sleep time is not less than 4 hours and the comprehensive signal effectiveness is greater than 85%. The complete night is stored as the target night. If there are 5 target nights in a week, the 5 target nights are used as new samples. Only the attention weights of the cross-modal Transformer, the sleep monitoring module and the pathological monitoring module in the sleep monitoring model are trained until the training conditions are met, and the trained sleep monitoring model is obtained.

[0063] Reference Figure 2 The diagram illustrates a structural schematic of an electronic device according to an embodiment of the present invention. The specific examples of the present invention do not limit the specific implementation of the electronic device.

[0064] like Figure 2 As shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0065] in: The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.

[0066] Communication interface 504 is used to communicate with other electronic devices or servers.

[0067] The processor 502 is used to execute program 510, which can specifically execute the relevant steps in the above-described server-side or user-side method embodiments.

[0068] Specifically, program 510 may include program code that includes computer operation instructions.

[0069] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The smart device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0070] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0071] Specifically, program 510 can be used to cause processor 502 to perform the operations corresponding to the methods described in the above method embodiments.

[0072] The specific implementation of each step in program 510 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0073] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of the present invention can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0074] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0075] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present invention.

[0076] The above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present invention, and the patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A method for sleep monitoring and adaptive regulation that integrates multimodal physiological signals, characterized in that, The method is implemented through a sleep monitoring model, which includes a one-dimensional convolutional neural network, a temporal convolutional network, a bidirectional LSTM network, a cross-modal transformer network, and a monitoring network. The method includes: Obtain the patient's first EEG characteristic sequence, first cardiovascular characteristic sequence, first respiratory characteristic sequence, and first body movement characteristic sequence during sleep; The first EEG feature sequence is input into a one-dimensional convolutional neural network to obtain the second EEG feature sequence; The first cardiovascular feature sequence is input into a temporal convolutional network to obtain the second cardiovascular feature sequence; The first respiratory feature sequence and the first body movement feature sequence are respectively input into their respective bidirectional LSTM networks to obtain the hidden layer state sequence; A cross-modal transformer network was used to fuse the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence to obtain a fused feature sequence. The fusion feature sequences are analyzed and processed based on the monitoring network and personalized baseline to obtain the patient's target sleep stage and target pathological risk index. The patient's sleep environment is then adaptively adjusted according to the target sleep stage and target pathological risk index.

2. The method according to claim 1, characterized in that, The method utilizes a cross-modal transformer network to perform cross-modal fusion of the second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence to obtain a fused feature sequence, including: The second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence are input into the multi-head self-attention layer of the cross-modal transformer network. The second EEG feature sequence, the second cardiovascular feature sequence, and the hidden state sequence are added element by element with their respective identity codes to obtain the added feature sequence. The added feature sequences are stacked to obtain the target feature sequence. The fused feature sequence is then obtained based on the target feature sequence.

3. The method according to claim 1, characterized in that, The monitoring network includes a global pooling module, a sleep monitoring module, and a pathology monitoring module; The analysis and processing of fused feature sequences based on the monitoring network and personalized baseline yields the patient's target sleep stage and target pathological risk indicators, including: The fusion feature sequence is compressed in dimension by using a global average pooling module to obtain a compressed feature vector; The compressed feature vector is segmented and identified by the fully connected layer in the sleep monitoring module to obtain the target sleep stage. The compressed feature vector is subjected to risk detection by a multilayer perceptron cascaded in the pathological monitoring module to obtain the current pathological risk index. The initial rate of decrease in blood oxygen saturation is obtained by predicting the fused feature sequence through a unidirectional LSTM network and a fully connected layer in the pathological monitoring module. By using a personalized baseline, the current pathological risk indicators and the initial rate of decline in blood oxygen saturation are partially corrected to obtain the target pathological risk indicators.

4. The method according to claim 3, characterized in that, Before partially correcting the current pathological risk indicator and the initial rate of decline in blood oxygen saturation using a personalized baseline to obtain the target pathological risk indicator, the method further includes: Collect the patient's current complete sleep night; and divide the current complete sleep night into multiple sleep cycles according to a first preset time period; Based on the three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence and body motion sequence corresponding to each sleep cycle, the ratio of low-frequency power to high-frequency power of mean body motion energy, respiratory rate and heart rate variability was calculated respectively. When the mean body energy, respiratory rate, and ratio all meet their respective threshold conditions, the corresponding sleep cycle is considered an effective sleep cycle. When the number of effective sleep cycles is not less than the threshold, the efficiency of obtaining the comprehensive signal corresponding to the current complete sleep night is determined. When the overall signal efficiency is not less than the threshold efficiency, the current complete sleep night is taken as the target night, and a personalized baseline is constructed based on the target night.

5. The method according to claim 4, characterized in that, The efficiency of obtaining the comprehensive signal corresponding to the current complete sleep night includes: The current complete sleep night is divided into multiple sub-time periods according to the second preset time period; Calculate the quality scores of the three-channel EEG sequence, cardiovascular feature sequence, respiratory feature sequence, and body motion feature sequence for each sub-time period; Based on the quality score and preset deletion conditions, invalid sub-time periods are deleted from multiple sub-time periods to obtain the target sub-time period; Calculate the total quality score for each target sub-time period, and then calculate the ratio of the total quality score to the number of target sub-time periods to obtain the overall signal efficiency. The personalized baseline constructed based on the target night includes: A robust baseline and minimum blood oxygen saturation level are calculated based on the cardiovascular signal sequence corresponding to the target sub-time period during the target night; and a personalized baseline is constructed using the robust baseline and minimum blood oxygen saturation level.

6. The method according to claim 1, characterized in that, The acquisition of the patient's first electroencephalogram (EEG) characteristic sequence, first cardiovascular characteristic sequence, first respiratory characteristic sequence, and first body movement characteristic sequence during sleep includes: Acquire multiple sets of signal data from the patient during sleep in the third time period; the multiple sets of signal data include historical signal data and current signal data; each set of signal data includes a multimodal data sequence, which includes the corresponding three-channel EEG sequence, cardiovascular signal sequence, respiratory signal sequence and body movement data sequence; Feature extraction is performed on each of the multimodal data sequences to obtain the corresponding feature vector for each group; the corresponding feature vector for each group includes EEG feature vector, cardiovascular feature vector, respiratory feature vector and body motion feature vector; Each set of feature vectors is concatenated according to the time dimension to obtain the first EEG feature sequence, the first cardiovascular feature sequence, the first respiratory feature sequence, and the first body movement feature sequence.

7. The method according to claim 6, characterized in that, Current pathological risk indicators include the rate of decline in blood oxygen saturation, the probability of obstructive sleep apnea (OSA), the probability of microarousing, and the periodic limb movement (PLM) index. The method utilizes a personalized baseline to partially correct for current pathological risk indicators and the initial rate of decline in blood oxygen saturation, resulting in target pathological risk indicators, including: Determine the lowest and mean blood oxygen saturation from the cardiovascular signal sequence of the current group of signal data; Calculate the first difference between the robust baseline of blood oxygen saturation and the mean blood oxygen saturation; match the first difference with a preset correction library to obtain the first correction amount; and correct the initial rate of decrease of blood oxygen saturation using the first correction amount to obtain the target rate of decrease of blood oxygen saturation. Calculate The second difference between the lowest blood oxygen saturation and the second difference is used to match the second difference with a preset correction library to obtain a second correction value. The OSA probability in the current pathological risk indicator is then corrected using the second correction value to obtain the target OSA probability. A target pathological risk index was constructed based on the target OSA probability, the target blood oxygen saturation decline rate, the microarousal risk probability, and the PLM index.

Images