Sleep risk early warning method and system for heart failure patients based on multi-modal data fusion

Abstract

The present application relates to the technical field of medical health monitoring, and discloses a sleep risk early warning method and system for heart failure patients based on multi-modal data fusion, which comprises the following steps: collecting and preprocessing electrocardio, respiration and body movement three-channel signals synchronously through a wearable device; extracting time sequence features of each channel by using a multi-scale one-dimensional convolution network and a bidirectional GRU; constructing a cross-modal attention fusion module to dynamically adjust the weights of each channel; performing four-level risk stratification evaluation and establishing a feedback loop through a cascade classifier; and realizing real-time early warning by compressing a cloud model to an edge device through knowledge distillation.

Description

Technical Field

[0001] This invention relates to the field of medical and health monitoring technology, and in particular to a method and system for early warning of sleep risks in heart failure patients based on multimodal data fusion. Background Technology

[0002] Heart failure is a serious and common end-stage manifestation of cardiovascular disease. Patients are highly susceptible to acute exacerbations of heart failure during sleep due to postural changes, autonomic nervous system dysregulation, and fluid redistribution. Clinical statistics show that approximately 40% of acute exacerbations of heart failure occur during nighttime sleep, and a significant proportion of these patients miss the optimal intervention window due to delayed warnings. Therefore, developing a real-time risk warning system tailored to the sleep scenarios of heart failure patients is of significant clinical importance for improving nighttime care efficiency and reducing the mortality and disability rates of acute exacerbations of heart failure.

[0003] Currently, the mainstream technology in sleep monitoring is polysomnography, which comprehensively assesses sleep by collecting signals from multiple channels, including electroencephalography (EEG), electromyography (EMG), electrooculography (EOG), electrocardiography (ECG), and respiration. However, polysomnography devices are bulky, have cumbersome electrode placement, and are only suitable for hospital environments, making them unsuitable for long-term home monitoring of heart failure patients. In wearable sleep monitoring, most existing commercial devices rely on a single physiological parameter for alarm judgment, such as triggering an alarm when blood oxygen saturation falls below a certain fixed threshold or issuing a warning when heart rate exceeds a preset upper limit. While this alarm strategy based on simple threshold rules is easy to implement, it suffers from high false negative rates and frequent false alarms because it fails to comprehensively consider the synergistic changes in multidimensional physiological signals. Clinical practice shows that schemes relying solely on blood oxygen threshold alarms have a sensitivity of only about 69% in detecting nocturnal heart failure exacerbation events, with a false alarm rate as high as 5 to 7 times per night, severely interfering with patients' sleep quality and increasing the workload of nursing staff.

[0004] In the field of deep learning-driven electrocardiogram (ECG) signal analysis, existing research has attempted to achieve automatic classification of arrhythmias by constructing neural network models. For example, Chinese patent CN116361688A discloses a method for constructing a multimodal feature fusion model for automatic ECG rhythm classification. This method employs a dual-channel bottleneck residual network to extract multimodal features of different dimensions from one-dimensional ECG sequences and two-dimensional ECG time-frequency maps, respectively, and enhances the expression of important features through channel domain attention mechanisms and hybrid domain attention mechanisms. This technical solution achieved an average F1 score of 85.1% and an average accuracy of 96.6% in the arrhythmia classification task. However, this scheme has the following shortcomings: First, the signal source is limited to the single ECG mode. Although features are extracted from both the time and frequency domains, it is essentially still based on ECG signals and fails to integrate multi-source heterogeneous physiological signals closely related to heart failure sleep risk, such as respiration and body movement. Second, the scheme targets arrhythmia classification as its objective task, rather than continuous risk warning for heart failure patients' sleep scenarios. It lacks an assessment mechanism and feedback loop to convert classification results into time-series risk levels. Third, the model size of this scheme is large, the computational complexity of the attention mechanism is high, and the computational constraints and real-time requirements of edge device deployment are not considered.

[0005] Furthermore, while some multimodal sleep monitoring methods have emerged in the existing technology, such as the study combining ECG and respiratory airflow for cardiovascular disease screening (SleepCVD-Net) and the multimodal representation learning framework integrating EEG, ECG, and respiratory signals (SleepFM), these methods all employ a centralized cloud processing architecture, with inference latency typically exceeding seconds. This fails to meet the clinical requirement of millisecond-level response to acute exacerbations of heart failure. Simultaneously, these methods lack an adaptive mechanism to dynamically adjust the fusion weights of each modality based on real-time physiological states. In special scenarios involving significant individual differences and drastic fluctuations in physiological signals among heart failure patients, fixed-weight fusion strategies struggle to achieve ideal early warning effects.

[0006] In summary, existing technologies for multimodal fusion risk warning in sleep scenarios for heart failure patients still face several technical bottlenecks, including a single signal source, a lack of adaptive fusion mechanisms, a lack of hierarchical risk assessment and feedback loops, and difficulties in real-time edge deployment. There is an urgent need for an innovative technical solution that can collaboratively utilize ECG, respiration, and body movement signals to achieve real-time risk warning on edge devices. Specifically, this solution needs to address the following four key technical challenges: First, how to achieve high-precision synchronous acquisition and efficient preprocessing of three heterogeneous signals (ECG, respiration, and body movement) with the limited sensor configuration of wearable devices; second, how to design an adaptive multimodal fusion mechanism that can dynamically adjust the contribution weight of each channel according to the patient's real-time physiological state; third, how to construct a hierarchical risk assessment system with closed-loop feedback capabilities to balance warning sensitivity and false alarm rate control; and fourth, how to compress deep learning models to a scale that meets the computing power and power consumption constraints of edge devices while maintaining risk identification accuracy. Summary of the Invention

[0007] To address the technical problems in existing sleep monitoring devices, such as high false negative rates due to reliance on threshold alarms, lack of adaptive mechanisms for multimodal signal fusion, and difficulty in real-time deployment of models on edge devices, this invention provides a method and system for early warning of sleep risks in heart failure patients based on multimodal data fusion.

[0008] The first aspect of this invention provides a method for early warning of sleep risk in heart failure patients based on multimodal data fusion, comprising the following steps:

[0009] Step S1, Multimodal sleep signal acquisition and preprocessing: During the sleep process of heart failure patients, electrocardiogram signals, respiratory signals and body movement signals are synchronously acquired through wearable sensing devices to form a three-channel raw data stream; corresponding filtering and feature enhancement processing are performed on the signals of each channel, and time alignment is performed according to a unified sampling clock to output a synchronized multimodal signal sequence.

[0010] Step S2, Multi-scale temporal feature extraction: Multi-scale one-dimensional convolutional neural network is used to extract time-frequency domain features from ECG signals, and multiple sets of convolutional kernels of different sizes are used to capture short-term heart rate variation features and long-term ST segment morphological features respectively; Bidirectional gated recurrent unit network is used to perform temporal modeling for respiratory signals and body movement signals respectively; The three-channel features are mapped to a feature vector space of a unified dimension, and three sets of normalized feature vectors are output.

[0011] Step S3, cross-modal attention fusion: Construct a cross-modal attention module with ECG feature vector as query vector and respiratory feature vector and body movement feature vector as key-value pairs. Dynamically adjust the contribution of each modality by calculating the attention weight matrix between channels. When ST segment depression is detected in the ECG signal, automatically increase the attention weight of the respiratory channel to enhance the ability to identify acute exacerbations of nocturnal heart failure.

[0012] Step S4, Hierarchical Risk Assessment and Early Warning Decision: A hierarchical risk assessment is performed on the integrated risk characteristics after fusion, dividing the risk level into four levels: safe, attention, early warning, and emergency. Coarse-grained risk screening and fine-grained risk classification are performed sequentially through a cascaded classifier, and personalized threshold correction is performed in combination with the patient's historical baseline data. When the risk level reaches the early warning or emergency level, an alarm is triggered, and the risk assessment results are fed back to the attention weight adjustment module in step S3 to dynamically optimize the subsequent fusion strategy.

[0013] Step S5, Knowledge Distillation Edge Deployment: The cloud-based teacher model is compressed into a lightweight student model through knowledge distillation. During the distillation process, the distribution of intermediate layer features and the soft label distribution of output layer are aligned simultaneously. The distilled student model is then deployed on edge computing devices to achieve real-time risk warning with an inference latency of no more than 200ms.

[0014] The second aspect of the present invention provides a sleep risk warning system for heart failure patients based on multimodal data fusion, including a multimodal signal acquisition and preprocessing module, a multi-scale temporal feature extraction module, a cross-modal attention fusion module, a hierarchical risk assessment and warning decision module, and a knowledge distillation edge deployment module; each module corresponds one-to-one with steps S1 to S5 in the above method, and works together to realize a real-time multimodal risk warning function for sleep scenarios of heart failure patients.

[0015] The beneficial effects of this invention are as follows: First, it constructs a three-channel collaborative sensing architecture of ECG, respiration, and body movement, which significantly improves the coverage of information dimensions compared with traditional single-signal monitoring schemes. It can capture the collaborative abnormal patterns of multiple physiological signals in heart failure sleep risk events, especially the ability to identify complex heart failure exacerbation events where ST segment depression and apnea occur simultaneously. Second, it designs a cross-modal attention fusion mechanism driven by physiological state, which realizes adaptive dynamic adjustment of the contribution of each modality through dynamic gating signals, effectively solving the problem of unstable early warning effect of fixed weight fusion strategy in scenarios with large individual differences in heart failure patients. Third, it establishes a four-level risk stratification assessment system and closed-loop feedback mechanism, which enables the risk assessment results to back-optimize the feature fusion strategy, forming a collaborative closed loop of perception-fusion-assessment-optimization. This closed-loop mechanism enables the system to adaptively switch between high-sensitivity mode and normal mode. Fourth, it achieves high-fidelity compression from large cloud models to lightweight edge models through a dual-channel knowledge distillation strategy, with a model parameter compression ratio of approximately 8:1 and inference latency controlled within 200ms, meeting the needs of real-time clinical alarms. Clinically validated, this invention achieves a 92% sensitivity in predicting nocturnal heart failure exacerbation events, a 23 percentage point improvement over traditional threshold methods, and reduces the false alarm rate to no more than 2 times per night. Attached Figure Description

[0016] Figure 1 This is a flowchart of a sleep risk warning method for heart failure patients based on multimodal data fusion, provided in an embodiment of the present invention.

[0017] Figure 2 This is an architecture diagram of a sleep risk early warning system for heart failure patients based on multimodal data fusion, provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described below are only for explaining the invention and are not intended to limit the scope of protection of the invention.

[0019] Reference Figure 1 As shown in the figure, this invention provides a sleep risk warning method for heart failure patients based on multimodal data fusion. This method is designed for nighttime sleep monitoring of heart failure patients. Through the collaborative perception and deep fusion analysis of ECG, respiration, and body movement signals, it achieves real-time risk warning on edge computing devices. The following describes each step in detail with specific technical details.

[0020] Step S1: Multimodal sleep signal acquisition and preprocessing. In one embodiment of the present invention, a heart failure patient wears a wearable chest strap device containing a multi-sensor array before falling asleep. This device integrates a dry electrode ECG acquisition unit, a piezoelectric film respiratory sensing unit, and a triaxial accelerometer body motion detection unit. The ECG acquisition unit uses a dual-lead dry electrode configuration with an electrode spacing of 8cm to 12cm, a sampling accuracy of 16bit, and an original sampling rate of 500Hz, capable of stably recording the morphological details of the P wave, QRS complex, and T wave in the ECG signal. The respiratory sensing unit is attached to the lower inner edge of the chest strap and indirectly measures respiratory motion by sensing the deformation of the piezoelectric film caused by chest expansion and contraction, with an original sampling rate of 50Hz. The triaxial accelerometer is embedded in the front of the chest strap, with a range of ±4g and an original sampling rate of 100Hz, used to capture body motion information such as patient turning over, limb tremors, and micro-vibrations.

[0021] The three-channel sensors mentioned above share the same clock crystal at the hardware level, thereby ensuring that the raw data stream has a strict time synchronization relationship at the acquisition end. Preferably, to further eliminate sub-millisecond time deviations introduced by differences in data transmission links, this invention performs a fine time alignment operation based on a cross-correlation function at the data aggregation node, improving the time synchronization accuracy of the three-channel signals to within one-tenth of the sampling period.

[0022] In the signal preprocessing stage, this invention designs dedicated filtering and feature enhancement strategies for three different types of physiological signals. For electrocardiogram (ECG) signals, a fourth-order Butterworth bandpass filter with a cutoff frequency of 0.5Hz to 40Hz is first used to eliminate baseline drift and high-frequency electromyographic interference. Subsequently, a notch filter with a center frequency of 50Hz and a bandwidth of 2Hz is used to remove power frequency electromagnetic interference. In one embodiment of this invention, after the above filtering process, the signal-to-noise ratio of the ECG signal is improved by approximately 12dB to 18dB compared to the original signal. Preferably, R-peak detection can also be performed on the filtered ECG signal, using the RR interval sequence as the basic data for heart rate variability analysis.

[0023] For the respiratory signal, a second-order Butterworth low-pass filter with a cutoff frequency of 0.5 Hz is used to remove high-frequency noise components, and then the respiratory amplitude envelope curve is extracted using Hilbert transform. The respiratory amplitude envelope curve can intuitively reflect the temporal change trend of the patient's respiratory depth and is a key feature source for detecting apnea events and Cheyne-Stokes breathing patterns. In one embodiment of the present invention, an apnea event is defined as a signal segment in which the respiratory amplitude envelope drops to less than 30% of the baseline level and lasts for more than 10 seconds.

[0024] For the body motion signal, a high-pass filter with a cutoff frequency of 0.1Hz is used to separate the gravity component, obtaining the pure motion acceleration component. Subsequently, the magnitude of the synthesized acceleration vector is calculated from the triaxial acceleration signal, and motion energy integration is performed within a sliding time window (window length set to 5s, sliding step size set to 1s), thereby transforming the body motion signal from high-sampling-rate instantaneous acceleration data into a low-sampling-rate motion intensity time series. The formula for calculating the motion energy integral value is: ,in, For the first The integral value of kinetic energy at each time point, in units of ; , , The first Each sampling point is at axis, axis, The acceleration value in the axial direction, in units of (1) = 9.8m / s²). This represents the number of sampling points within the sliding window. When the window length is 5 seconds and the sampling rate is 100 Hz... = 500. The integral value of exercise energy can effectively distinguish resting sleep ( < 0.01 ) and active body movement ( ≥ 0.01 This provides a quantitative basis for the subsequent identification of periodic limb movement events.

[0025] The final step in preprocessing is downsampling and unification of the multi-channel signals. Specifically, the ECG signal is downsampled from 500Hz to 250Hz, the respiration signal is upsampled from 50Hz to 250Hz (using cubic spline interpolation), and the body motion intensity sequence is upsampled from 1Hz to 250Hz (using zero-order hold interpolation). In one embodiment of the invention, to avoid spectral aliasing during downsampling, an anti-aliasing low-pass filter with a cutoff frequency of 125Hz is applied to the ECG signal before downsampling. During upsampling, cubic spline interpolation, compared to linear interpolation, better preserves the smooth envelope shape of the respiration signal, avoiding the introduction of jagged artifacts. After the above processing, all three channels are output at a uniform sampling rate of 250Hz, forming a time-aligned synchronized multimodal signal sequence, which serves as input data for subsequent deep learning models. Preferably, the present invention also performs Z-score standardization processing before the data is input into the model, that is, subtracting the mean of the channel in the current sleep record from the signal of each channel and dividing by the standard deviation, so that the input data has a standardized distribution with zero mean and unit variance, which is beneficial to the convergence speed and numerical stability of the deep learning model.

[0026] Step S2: Multi-scale temporal feature extraction. After obtaining the synchronized multimodal signal sequence preprocessed in step S1, this step designs differentiated deep learning feature extraction network structures for electrocardiogram signals, respiratory signals, and body movement signals, respectively, in order to fully explore the physiological and pathological information contained in each modality signal.

[0027] For feature extraction from electrocardiogram (ECG) signals, this invention employs a multi-scale one-dimensional convolutional neural network (Multi-Scale 1D-CNN) architecture. The core design principle of this network is that ECG abnormalities associated with heart failure exhibit significant differences across time scales. For example, arrhythmias typically manifest as fluctuations in the RR interval on the order of tens of milliseconds, while ST segment depression is a slow morphological change lasting hundreds of milliseconds. Traditional networks with a single convolutional kernel size struggle to simultaneously capture these two different time-scale feature patterns.

[0028] Preferably, the multi-scale one-dimensional convolutional neural network of the present invention comprises three sets of parallel convolutional branches, each set of branches configured with convolutional kernels of different sizes to cover different temporal receptive fields. The first branch is configured with a convolutional kernel size of 1×5 (corresponding to a 20ms receptive field) and contains 64 convolutional kernels, mainly responsible for capturing short-term rapid-change features such as QRS complex morphology and R-peak sharpness. The second branch is configured with a convolutional kernel size of 1×15 (corresponding to a 60ms receptive field) and contains 64 convolutional kernels, mainly responsible for capturing medium-time-scale features such as PR interval changes and T-wave symmetry. The third branch is configured with a convolutional kernel size of 1×31 (corresponding to a 124ms receptive field) and contains 64 convolutional kernels, mainly responsible for capturing long-term slowly changing features such as ST segment morphological changes and QT interval prolongation.

[0029] In one embodiment of the present invention, all three sets of convolutional branches use depthwise separable convolution instead of standard convolution operations to significantly reduce the number of parameters and computational complexity. Specifically, the number of parameters in a standard convolution operation is... ( The kernel size is the convolution kernel size. Input the number of channels. (the number of output channels), while depthwise separable convolution decomposes it into channel-wise convolutions (the number of parameters). ) and pointwise convolution (parameter count) In two steps, the total number of parameters is reduced to Taking the third branch as an example, the number of parameters in a standard convolution is 31×1×64 = 1984, while the number of parameters in a depthwise separable convolution is only 31×1 + 1×64 = 95, a reduction of about 95.2%.

[0030] After each of the three convolutional branches passes through a batch normalization layer and a ReLU activation function, their output feature maps are concatenated along the channel dimension to form a multi-scale fused feature map containing 192 channels. This fused feature map is then subjected to channel dimensionality reduction through a 1×1 convolutional layer (containing 128 convolutional kernels) and temporal downsampling through a max pooling layer (pooling window size of 4, stride of 4). Finally, the output dimension of the ECG feature extraction branch is... The electrocardiogram feature matrix, where This represents the number of time steps after downsampling.

[0031] For feature extraction of respiratory signals, this invention employs a Bidirectional Gated Recurrent Unit (GRU) network. GRU was chosen over LSTM because it achieves comparable temporal modeling performance to LSTM with fewer parameters, making it more suitable for lightweight deployment to edge devices. The bidirectional GRU network for respiratory signals is configured as a two-layer stacked structure with a hidden layer dimension of 128 and a dropout rate of 0.3. Input data is segmented into 30-second time steps. Within each time step, statistical features (mean, standard deviation, peak-to-peak value, and respiratory rate) are extracted from the respiratory signal, forming a 4-dimensional feature vector that is input to the GRU network. The forward and backward hidden states of the bidirectional GRU are concatenated and mapped to a 128-dimensional feature space through a linear layer, resulting in an output dimension of [missing information]. The respiratory feature matrix.

[0032] During the extraction of respiratory features, bidirectional GRU networks can effectively capture the preceding gradual process (gradual decrease in respiratory amplitude) and the subsequent recovery process (sudden increase in respiratory amplitude accompanied by fluctuations in blood oxygenation) of apnea events. This bidirectional dependency is particularly important for accurately identifying heart failure-related respiratory abnormalities such as central sleep apnea (CSA) and Cheyne-Stokes respiration (CSR). In one embodiment of the present invention, after bidirectional GRU modeling, the detection sensitivity of apnea events increased from 78% with unidirectional RNNs to 89%, indicating that bidirectional temporal modeling has a significant advantage in capturing the complete evolution pattern of respiratory abnormalities.

[0033] For feature extraction of body motion signals, a bidirectional GRU network is also used for temporal modeling. The network structure configuration is consistent with the respiratory signal processing network (two stacked layers, 128 hidden layer dimensions, and a Dropout rate of 0.3), but the input features are different. Within each 30-second time step, the following features are extracted from the body motion signals: mean integral of motion energy, peak integral of motion energy, zero-crossing rate, and the proportion of motion event duration, forming a 4-dimensional feature vector. The output dimension of the body motion feature matrix is... .

[0034] In one embodiment of the present invention, temporal modeling of body movement signals can effectively identify periodic limb movement (PLM) patterns. PLM is a common movement abnormality during sleep in heart failure patients, typically manifesting as repetitive limb twitches at intervals of 20 to 40 seconds, and is closely related to sleep stages and the risk of cardiovascular events. A bidirectional GRU network, by learning the periodic time interval patterns and movement intensity variations of PLM events, can distinguish PLM from non-pathological body movement events such as spontaneous turning over. Preferably, the present invention also introduces an event detection preprocessing step in the body movement feature extraction stage: when the peak value of the integral movement energy exceeds five times the resting baseline within a 30-second time window, and the duration of the movement is within the range of 0.5 to 10 seconds, the window is marked as a potential movement event window. This marking information is concatenated with the GRU output features as an auxiliary feature, providing event-level prior knowledge of the movement state for the subsequent fusion module, which helps improve the accuracy of distinguishing between PLM events and normal postural adjustments.

[0035] In one embodiment of the present invention, the aforementioned three-channel feature extraction network employs a joint optimization strategy during the training phase. Specifically, the ECG feature extraction branch, the respiration feature extraction branch, and the body motion feature extraction branch share the same loss function for end-to-end training, rather than being trained independently and then combined. This joint optimization strategy enables the branches to adapt to each other during training, resulting in better compatibility and complementarity of the extracted features in the subsequent fusion stage. Specifically, the training process uses the Adam optimizer, with an initial learning rate set to... The training data is decayed to 50% of its original value every 20 training rounds, with a total of 80 rounds and a batch size of 32. The training data is divided into 30-second segments, with each segment corresponding to one training sample. Approximately 960 training samples can be generated from an 8-hour sleep record of a single patient.

[0036] Finally, to ensure that the three-channel feature vectors have a unified feature space distribution, this invention performs feature dimension normalization on the ECG feature matrix, respiratory feature matrix, and body motion feature matrix respectively. Specifically, L2 norm normalization is performed on the 128-dimensional feature vector of each channel, projecting it onto a unit hypersphere to eliminate feature scale bias caused by differences in numerical dimensions between different modes. The normalization operation is calculated using the following formula: ,in, These are the normalized feature vectors; The original feature vector, index Indicates modal index, These correspond to the three channels: electrocardiogram, respiration, and body movement, respectively. for The L2 norm; To prevent the small constant from being divided by zero, the value is taken as... After normalization, all three sets of feature vectors lie on a 128-dimensional unit hypersphere, providing scale-consistent feature inputs for subsequent cross-modal attention fusion.

[0037] Step S3: Cross-modal attention fusion. After obtaining the three sets of normalized feature vectors output in step S2, this step constructs a cross-modal attention fusion module to achieve adaptive weighted fusion of ECG, respiration, and body movement features. Unlike traditional fusion methods that simply splice or sum the signals with fixed weights, the cross-modal attention mechanism of this invention can dynamically adjust the fusion weights according to the amount of information carried by each modality signal and its clinical relevance within the current time window, thereby achieving an adaptive fusion strategy where the modality with higher information quality receives a greater contribution.

[0038] The cross-modal attention module of this invention employs an asymmetric attention architecture that uses electrocardiogram (ECG) features as queries, respiratory features as keys, and body movement features as values. The clinical basis for this design is that, in the context of heart failure sleep risk assessment, ECG signals are the most direct indicator of cardiac function and should naturally be the initiator of the attention query; while respiratory and body movement signals, as carriers of secondary symptoms of heart failure, exhibit dynamic fluctuations in their correlation with ECG signals as the patient's pathological state changes, making them suitable as the objects of the query.

[0039] Preferably, the calculation process for cross-modal attention weights is as follows: First, the ECG feature vector... Through the first linear transformation matrix Mapped to query matrix , respiratory feature vector Through the second linear transformation matrix Mapped to a key matrix , to transform the body motion feature vector Through the third linear transformation matrix Mapped to a value matrix .in, For time steps, The dimension of the key vector. In this embodiment, the dimension is the value vector. Then, perform the scaled dot product attention operation: ,in, For attention fusion output; To query the matrix product of a matrix and the transpose of the key matrix, the result dimension is... ; This is the scaling factor, and its value is... This is used to prevent the Softmax function from entering the gradient saturation region due to excessively large dot product values. This is the normalized exponential function executed along the last dimension.

[0040] In one embodiment of the present invention, to further enhance the expressive power of cross-modal attention fusion, a multi-head attention mechanism is adopted. The 64-dimensional key and value vector space is divided into eight attention heads. Each head independently performs a scaled dot product attention operation, and then the heads are concatenated along the channel dimension. Finally, a linear mapping layer maps the concatenated features to a 128-dimensional comprehensive feature space. The multi-head attention mechanism enables the model to learn cross-modal association patterns simultaneously from multiple different subspaces. For example, some attention heads may focus on the association between the ST segment of the ECG and respiratory depth, while others may focus on the coupling relationship between heart rate variability and body velocity.

[0041] A significant innovation of this invention lies in the introduction of a dynamic gating mechanism based on physiological state. In clinical practice, ST-segment depression in heart failure patients usually indicates worsening myocardial ischemia, often accompanied by respiratory abnormalities related to pulmonary edema (such as orthopnea and paroxysmal nocturnal dyspnea). Therefore, when the ST-segment offset of the electrocardiogram exceeds a preset offset threshold... (In this embodiment) When set to 0.1mV, the dynamic gating mechanism uses a gating signal. Adjust the attention weights for each channel: ,in, This is a gating signal, with a value range of [value range missing]. ; Use the Sigmoid activation function; This is the gating weight vector; The ST segment shift feature vector extracted from the electrocardiogram signal has a dimension of Set to 16; This is the bias term. When the gate signal... When activated, the attentional weight of the respiratory channel is increased to the baseline weight. The attention weight of the body movement channel is reduced by a factor of 1.5 to 2.0 (i.e., 1.5 to 2.0 times) to the baseline weight. The gating mechanism increases the sensitivity to respiratory abnormalities by 0.5 to 0.8 times when ST segment depression occurs. The technical effect of this gating mechanism is that it automatically enhances the sensitivity to respiratory abnormalities during ST segment depression, thereby improving the early identification of the complex event of nocturnal exacerbation of heart failure. Experimental verification showed that after introducing the dynamic gating mechanism, the model's detection sensitivity for acute exacerbation of heart failure combined with apnea increased from 83% to 91%, indicating that this mechanism can effectively uncover the pathological coupling between electrocardiogram and respiration.

[0042] Finally, the multi-head cross-modal attention fusion output is added to the gated and modulated residual features of each channel. After layer normalization and processing by a feedforward neural network (containing two fully connected layers, a hidden layer dimension of 256, and a GELU activation function), the feature dimension is mapped to 64 dimensions through a linear mapping layer, outputting a comprehensive risk feature representation. It is worth noting that this invention employs residual connections to add the features of each channel before fusion to the attention output after fusion. This design, on the one hand, preserves the original feature information of each channel, avoiding potential information loss during fusion; on the other hand, it allows gradients to be directly propagated back to the feature extraction networks of each channel through the residual path, which is beneficial for the collaborative optimization of each module during end-to-end training. In one embodiment of this invention, the hidden layer of the feedforward neural network also introduces a Dropout regularization of 0.2 to prevent overfitting. The total number of parameters in the entire cross-modal attention fusion module is approximately 0.4M, and the computational complexity is approximately 38MFLOPs.

[0043] Step S4: Layered risk assessment and early warning decision-making. After obtaining the comprehensive risk feature representation output by step S3, this step constructs a layered risk assessment and early warning decision-making module, which maps continuous risk features into discrete four-level risk level labels, and dynamically optimizes the front-end fusion strategy through a feedback closed-loop mechanism.

[0044] This invention classifies the nighttime sleep risk of heart failure patients into four levels: Safe (risk score) <0.25), Caution level (0.25 ≤ < 0.50), Warning level (0.50 ≤ < 0.75) and Critical ( ≥ 0.75). The above risk score. The range of values ​​is Its calculation is performed by the following cascaded classifier.

[0045] This invention employs a cascaded classifier architecture to achieve a coarse-to-fine hierarchical risk assessment. This architecture comprises a cascaded coarse-grained screening network and a fine-grained classification network. The coarse-grained screening network is a fully connected network with two hidden layers (32 and 16 dimensions respectively, using ReLU activation function), used to binary classify the 64-dimensional comprehensive risk features into normal states (corresponding to the safety level) and abnormal states (corresponding to the attention level, warning level, or emergency level). The output probability of this binary classifier is the abnormal state probability. Preferably, the decision boundary of the coarse-grained screening network is set to... = 0.3, meaning that when the probability of an anomaly exceeds 0.3, it is judged as an abnormal state and the process is transferred to a fine-grained classification procedure. Using a lower decision boundary can improve the recall rate of potential risk events and avoid false negatives.

[0046] Only samples deemed abnormal by the coarse-grained screening network are further processed into a fine-grained classification network for three-level classification (attention level, warning level, and emergency level). The fine-grained classification network is a fully connected network with three hidden layers (32, 24, and 16 dimensions respectively, using ReLU activation). Its input is a concatenation of 64-dimensional comprehensive risk features and the 16-dimensional features from the last hidden layer of the coarse-grained screening network (80 dimensions in total). The output is the probability distribution of the three risk levels. The final risk score is then calculated. Calculated using the following formula: ,in, This represents the probability of abnormal states output by the coarse-grained screening network. The first output of the fine-grained ranking network The probability of each risk level. These correspond to the attention level, warning level, and emergency level, respectively. These are the risk weighting coefficients corresponding to each level. , , This set of weighting coefficients maps the probabilities of the three levels to a weighted average. Within the corresponding sub-interval of the risk score interval.

[0047] In one embodiment of the present invention, to address the problem of the difficulty in applying a uniform threshold due to significant individual differences among heart failure patients, the present invention introduces a personalized threshold correction mechanism. Specifically, physiological data within the first complete sleep cycle (approximately 90 minutes) after the patient falls asleep are collected as individual baseline data, and the mean of each dimension of the comprehensive risk characteristics within this cycle is used. and standard deviation Constructing personalized normal range intervals When a patient's baseline risk level is high (e.g., a patient with chronic heart failure has a naturally high baseline heart rate), the score thresholds for each risk level are adjusted upwards accordingly; conversely, they are adjusted downwards. Preferably, the threshold adjustment range is controlled within ±20% of the baseline threshold to avoid personalized corrections that result in overly lenient or overly strict risk level settings.

[0048] In terms of early warning decision-making, when the risk level reaches the warning level, the edge device emits a slow orange flashing signal (flashing frequency 1Hz) via an LED indicator, and simultaneously pushes a text alarm message to the nursing terminal at the patient's bedside via Bluetooth Low Energy. When the risk level reaches the emergency level, the edge device triggers a local buzzer to emit a rapid alarm sound (alarm frequency 3kHz, duty cycle 50%, interval 0.5s), and simultaneously pushes an emergency warning message along with the physiological signal waveform data of the most recent 5 minutes to the remote nursing station via Wi-Fi network, so that nursing staff can quickly assess the patient's condition.

[0049] Another key innovation in this step lies in the feedback loop mechanism for the risk assessment results. Specifically, when the risk level reaches the concern level or above for three consecutive time windows (each window is 30 seconds), the system sends the risk assessment results back to the cross-modal attention fusion module in step S3 through the feedback path, thus triggering the activation threshold of the dynamic gating mechanism. The voltage was reduced from the default value of 0.1mV to 0.05mV, thereby enhancing the sensitivity to subtle ST segment changes. After the risk level falls back to the safe level for five consecutive time windows, The system reverts to the default value of 0.1mV. This feedback loop mechanism enables the system to automatically switch to a high-sensitivity monitoring mode when a potential risk trend is detected, and to revert to the normal sensitivity mode after the risk is eliminated, thus balancing early warning sensitivity and false alarm control.

[0050] Step S5: Knowledge Distillation for Edge Deployment. The complete multimodal sleep risk warning model built in steps S2 to S4 is trained and validated on a cloud server. This cloud model (i.e., the teacher model) has approximately 12.8M parameters and approximately 856MFLOPs of floating-point operations, with a single inference latency of approximately 8ms on an NVIDIA V100 GPU. However, when this model is directly deployed on an edge computing device (such as an ARM Cortex-A72 processor with a clock speed of 1.5GHz and 1GB of memory), the inference latency increases to approximately 1200ms, far exceeding the 200ms real-time requirement. Therefore, this step uses knowledge distillation to compress the teacher model into a lightweight student model adapted for edge devices.

[0051] This invention employs a dual-channel knowledge distillation strategy, simultaneously extracting knowledge from both the intermediate and output layers of the teacher model and transferring it to the student model. The goal of intermediate layer feature distillation is to make the intermediate feature distribution of the student model approximate that of the teacher model, thereby inheriting the teacher model's feature extraction capabilities. Specifically, intermediate features from the teacher model in the feature extraction layer (S2) and the fusion layer (S3) are selected as distillation targets. Since the intermediate layer dimensions of the teacher and student models may differ, an adaptation layer (1×1 convolution) is used to map the intermediate features of the student model to the same dimension as the teacher model before calculating the loss. The intermediate layer distillation loss function is: ,in, This represents the characteristic distillation loss of the intermediate layer; In this embodiment, the number of intermediate layers participating in distillation is... ; For the teacher model Feature maps of the intermediate layers; For the student model Feature maps of the intermediate layers; For the first Mapping functions for each adaptation layer; It is the square of the L2 norm.

[0052] The goal of output layer soft label distillation is to make the output probability distribution of the student model approximate the softening probability distribution of the teacher model under high-temperature conditions. The soft labels contain the teacher model's implicit knowledge about the similarity relationships between different risk levels. The output layer distillation loss function is: ,in, For output layer soft label distillation loss; In this embodiment, the distillation temperature coefficient is used. Higher temperatures can smooth out the output probability distribution, thereby exposing more information about inter-class relationships. KL divergence measures the difference between two probability distributions. For the teacher model at temperature The softened output probability distribution is as follows; For student models at temperature The softened output probability distribution is as follows; It is the square of the temperature coefficient, used to compensate for the reduction in gradient amplitude at high temperatures.

[0053] The total training loss of the student model is a weighted combination of the two distillation losses mentioned above and the hard-label cross-entropy loss: ,in, Total training loss; This is the hard-label cross-entropy loss, which is the standard cross-entropy loss between the student model output and the true risk level label; and In this embodiment, the loss weighting coefficient is used. , , The above weighting configuration ensures that soft-label knowledge accounts for a large proportion of the distillation process, while retaining a certain proportion of hard-label supervision to prevent the student model from overfitting the noise of the teacher model.

[0054] In one embodiment of the present invention, the network structure of the student model is designed to be lightweight using the following strategy: In the feature extraction part corresponding to step S2, the multi-scale 1D-CNN three-branch structure of the teacher model is simplified to a single-branch structure (only 1×15 convolutional kernels are retained), and the number of convolutional kernels is reduced from 64 per branch to 32; the hidden layer dimension of the bidirectional GRU is reduced from 128 to 64, and the number of layers is reduced from 2 to 1. In the fusion part corresponding to step S3, the number of heads of the multi-head attention is reduced from 8 to 4, and the key dimension is reduced from 64 to 32. In the classification part corresponding to step S4, the hidden layer dimension of each cascade classifier is halved. After the above lightweight design, the total number of parameters of the student model is approximately 1.6M (compression ratio of approximately 8:1), the floating-point operation volume is approximately 98MFLOPs (compression ratio of approximately 8.7:1), and the single inference latency on the ARM Cortex-A72 processor is approximately 165ms, meeting the real-time requirement of 200ms.

[0055] Preferably, the optimizer used for distillation training is AdamW, and the initial learning rate is set to... The weight decay coefficient is set to The total number of training rounds was set to 100, employing a cosine annealing learning rate scheduling strategy. The training dataset consisted of 412 nights of sleep records from 82 heart failure patients (NYHA class II-IV), with 70% used for training, 15% for validation, and 15% for testing. The student model trained with distillation achieved a weighted F1 score of 0.896 for level 4 risk classification on the test set, only 0.027 lower than the teacher model's 0.923, demonstrating that the dual-channel knowledge distillation strategy can maintain high classification accuracy while significantly compressing the model size.

[0056] Reference Figure 2 As shown, this embodiment of the invention provides a sleep risk warning system for heart failure patients based on multimodal data fusion. The system includes a multimodal signal acquisition and preprocessing module, a multi-scale temporal feature extraction module, a cross-modal attention fusion module, a hierarchical risk assessment and warning decision module, and a knowledge distillation edge deployment module. Each module corresponds one-to-one with steps S1 to S5 in the above method embodiment.

[0057] The multimodal signal acquisition and preprocessing module corresponds to step S1. This module includes a wearable sensor array unit and a signal preprocessing unit. The wearable sensor array unit integrates a dry electrode ECG acquisition unit (dual-lead configuration, 500Hz sampling rate), a piezoelectric film respiration sensing unit (50Hz sampling rate), and a triaxial accelerometer motion detection unit (range ±4g, 100Hz sampling rate). The three sensors share the same clock crystal to ensure time synchronization. The signal preprocessing unit includes an ECG signal preprocessing subunit (performing bandpass filtering, power frequency notch filtering, and R-peak detection), a respiration signal preprocessing subunit (performing low-pass filtering and Hilbert envelope extraction), and a motion signal preprocessing subunit (performing high-pass filtering and motion energy integration). Finally, a synchronized multimodal signal sequence with a uniform sampling rate of 250Hz is output through the time alignment subunit.

[0058] The multi-scale temporal feature extraction module corresponds to step S2. This module includes an ECG feature extraction submodule and a respiratory and body motion feature extraction submodule. The ECG feature extraction submodule uses a multi-scale one-dimensional convolutional neural network with three sets of parallel convolutional branches (kernel sizes of 1×5, 1×15, and 1×31, respectively). The outputs of the three branches are concatenated by the channel dimension and reduced in dimensionality by 1×1 convolution to form a 128-dimensional ECG feature vector. The respiratory and body motion feature extraction submodule uses a bidirectional GRU network (two stacked layers, 128 hidden layer dimension) to perform temporal modeling on the respiratory and body motion signals, respectively, outputting 128-dimensional respiratory and body motion feature vectors. The three sets of feature vectors are projected into a unified 128-dimensional feature space after L2 normalization.

[0059] The cross-modal attention fusion module corresponds to step S3. This module includes a scaled dot product attention unit and a dynamic gating modulation unit. The scaled dot product attention unit uses ECG features as queries, respiratory features as keys, and body movement features as values, achieving adaptive fusion of cross-modal features through an 8-head multi-head attention mechanism. The dynamic gating modulation unit monitors the ST segment offset in the ECG signal. When the offset exceeds a preset threshold, it automatically increases the weight of the respiratory channel and decreases the weight of the body movement channel through a Sigmoid gating signal, thereby enhancing the ability to identify complex events of acute exacerbation of heart failure combined with respiratory abnormalities. The fused features are then processed by layer normalization and a feedforward network and mapped to a 64-dimensional comprehensive risk feature representation.

[0060] The hierarchical risk assessment and early warning decision-making module corresponds to step S4. This module includes a cascaded classifier unit, a personalized threshold correction unit, and an alarm execution unit. The cascaded classifier unit consists of a coarse-grained screening network (for binary classification of normal / abnormal) and a fine-grained grading network (for tri-class classification of concern / warning / emergency levels), both of which jointly output a risk score and a risk level label. The personalized threshold correction unit dynamically adjusts the score thresholds for each risk level based on the patient's physiological baseline data during their first complete sleep cycle. The alarm execution unit triggers corresponding alarm responses according to the risk level: a warning level triggers a slow flashing orange LED and a local Bluetooth alarm push; an emergency level triggers a rapid buzzer alarm and a remote Wi-Fi alarm push. In addition, this module also includes a feedback closed-loop control unit, which, when continuously detecting risks at the concern level or above, sends the assessment results back to the cross-modal attention fusion module, dynamically lowering the gating activation threshold to enhance the high-sensitivity monitoring mode, thus achieving a collaborative closed loop of perception-fusion-assessment-optimization.

[0061] The knowledge distillation edge deployment module corresponds to step S5. This module includes a dual-channel distillation training unit and an edge inference engine unit. The dual-channel distillation training unit is executed on a cloud server, simultaneously optimizing the intermediate layer feature distillation loss and the output layer soft label distillation loss, compressing the teacher model with approximately 12.8M parameters into a student model with approximately 1.6M parameters. The edge inference engine unit deploys the distilled student model on an ARM Cortex-A72 edge computing device, implementing model loading and real-time inference through the ONNXRuntime inference framework. The latency for a single inference is approximately 165ms, meeting the 200ms real-time constraint. Preferably, the edge inference engine also supports INT8 quantization inference mode, further reducing the inference latency to approximately 120ms while only decreasing the classification accuracy by approximately 0.8%.

[0062] In one embodiment of the present invention, the data flow relationship between the above five modules is as follows: The multimodal signal acquisition and preprocessing module continuously outputs a synchronized three-channel signal sequence at a uniform sampling rate of 250Hz. Every 30 seconds of accumulated data forms an analysis time window and is pushed to the multi-scale temporal feature extraction module. After receiving the complete time window data, the multi-scale temporal feature extraction module extracts three sets of 128-dimensional normalized feature vectors through the ECG feature extraction sub-module and the respiratory body motion feature extraction sub-module, respectively. The computational delay of the entire feature extraction process is approximately 45ms. After receiving the three sets of feature vectors, the cross-modal attention fusion module performs cross-modal attention calculation and dynamic gating modulation, and outputs a 64-dimensional comprehensive risk feature representation, with a computational delay of approximately 38ms. After receiving the comprehensive risk features, the hierarchical risk assessment and early warning decision module outputs the risk level through a cascaded classifier, with a computational delay of approximately 12ms. The total computational delay of the above three inference sub-processes is approximately 95ms. Adding the data transmission and preprocessing overhead of approximately 70ms, the end-to-end system delay is approximately 165ms, which meets the 200ms constraint requirement for real-time early warning.

[0063] Preferably, each module employs a pipelined parallel processing strategy on the edge device: when the... When the data from the first time window is processed in the multi-scale temporal feature extraction module, the data from the first time window... The feature vectors of the first time window have entered the cross-modal attention fusion module, while the feature vectors of the second time window have entered the cross-modal attention fusion module. The comprehensive risk characteristics of each time window are being classified and decided in the hierarchical risk assessment module. This pipelined strategy improves the system's effective throughput to a maximum computational latency of only waiting for one module per 30-second time window (approximately 45ms), which is far superior to the cumulative latency of serial execution.

[0064] To verify the effectiveness of the technical solution of this invention, a clinical dataset containing 412 nights of sleep records from 82 heart failure patients was constructed. The patients' NYHA functional classifications were as follows: Class II (28 patients), Class III (36 patients), and Class IV (18 patients), with an age range of 52 to 81 years and a male-to-female ratio of approximately 1.3:1. The recording duration each night ranged from 6 to 9 hours. A total of 187 heart failure exacerbation events were annotated by clinical experts (including 73 cases of ST-segment depression with apnea, 56 cases of acute pulmonary edema, 38 cases of severe arrhythmias, and 20 other cases).

[0065] In terms of experimental setup, a hold-out method was used to divide the dataset into 70% training set, 15% validation set, and 15% test set to ensure that all records of the same patient appear in only one subset to avoid data leakage. Evaluation metrics included alert sensitivity, specificity, weighted-F1 score, and average number of false alarms per night (FP / Night).

[0066] The sleep risk warning system (complete student model) based on multimodal data fusion proposed in this invention achieved the following performance on the test set: warning sensitivity of 92.3%, specificity of 94.7%, weighted F1 score of 0.896, and an average of 1.8 false alarms per night. Compared with the traditional threshold alarm method (based on a fixed threshold rule of SpO2 < 90% or heart rate > 120 bpm), the traditional method has a warning sensitivity of only 69.1% and an average of 5.6 false alarms per night. The present invention improves sensitivity by 23.2 percentage points and reduces the false alarm rate by 67.9%, significantly improving patient sleep quality and nursing response efficiency.

[0067] To further verify the contributions of each technical component, the following ablation experiments were designed: Model A used only a single-channel ECG input (removing respiratory and body movement channels), with a sensitivity of 79.4% and a weighted F1 score of 0.812; Model B used a three-channel input but employed fixed-weight splicing fusion (removing the cross-modal attention mechanism), with a sensitivity of 85.2% and a weighted F1 score of 0.854; Model C used a three-channel input and cross-modal attention fusion but removed the dynamic gating mechanism, with a sensitivity of 88.6% and a weighted F1 score of 0.878; Model D was the complete teacher model, with a sensitivity of 94.1% and a weighted F1 score of 0.923. The ablation experiment results show that multi-channel signal fusion improves sensitivity by approximately 10 percentage points compared to single-channel fusion, cross-modal attention mechanism improves sensitivity by approximately 3 percentage points compared to fixed-weight fusion, and dynamic gating mechanism further improves sensitivity by approximately 4 percentage points. The above results verify the synergistic contribution of the multimodal collaborative perception architecture, cross-modal attention fusion mechanism and dynamic gating mechanism of the present invention to improving the early warning performance of heart failure sleep risk, demonstrating a nonlinear gain effect of 1+1>2.

[0068] Furthermore, this invention also conducted specific verification experiments to verify the effectiveness of the feedback loop mechanism. Comparing the complete system with a control system that removed the feedback loop, on the same test set, the continuous warning accuracy rate of the complete system (i.e., the proportion of effective warnings triggered at least once within 30 minutes before the occurrence of a heart failure deterioration event) was 89.7%, while the control system without the feedback loop only achieved 82.4%, an improvement of 7.3 percentage points. This result indicates that the feedback loop mechanism, by automatically increasing system sensitivity when a potential risk trend is detected, can effectively capture progressively worsening heart failure events, avoiding the problem of missing early, weak signals that may occur under a fixed sensitivity setting.

[0069] Regarding edge deployment performance, the student model of this invention, tested on an ARM Cortex-A72 platform (1.5GHz, 1GB memory), exhibited an inference latency of 165ms, memory usage of approximately 68MB, and power consumption of approximately 0.9W. After enabling INT8 quantization mode, the inference latency decreased to 120ms, memory usage to 42MB, and power consumption to 0.6W, while the classification accuracy only slightly decreased from a weighted F1 score of 0.896 to 0.889. These performance indicators demonstrate that the lightweight student model of this invention can achieve continuous real-time inference throughout the night on low-power edge devices. A single charge (2000mAh lithium battery) can support approximately 10 hours of continuous monitoring, meeting the battery life requirements for overnight sleep monitoring of heart failure patients. In terms of clinical applicability, the wearable chest strap device of this invention weighs approximately 35g. The wearing comfort was assessed using a patient subjective evaluation questionnaire (using the visual analog scale, with a maximum score of 10), resulting in an average comfort score of 7.8, indicating that the device does not significantly interfere with the patient's sleep quality. Furthermore, the present invention demonstrated good stability in a longitudinal test lasting 30 nights, with fluctuations in all performance indicators within 3%, verifying the reliability and robustness of the system in long-term use scenarios, and showing good prospects for clinical application and industrialization value.

[0070] It should be understood that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A sleep risk early warning method for heart failure patients based on multimodal data fusion, characterized in that, Includes the following steps: Step S1, Multimodal sleep signal acquisition and preprocessing: During the sleep process of heart failure patients, electrocardiogram signals, respiratory signals and body movement signals are simultaneously acquired through wearable sensing devices to form a three-channel raw data stream; The ECG signal is sequentially subjected to bandpass filtering and power frequency notch filtering to eliminate baseline drift and electromagnetic interference. The respiratory signal is subjected to low-pass filtering and envelope extraction to obtain the respiratory amplitude envelope curve. The body motion signal is subjected to high-pass filtering and motion energy integration to separate the resting state and active state body motion components. The preprocessed three-channel signals are time-aligned according to a unified sampling clock to output a synchronized multimodal signal sequence. Step S2, Multi-scale Temporal Feature Extraction: The ECG signal output from Step S1 is processed using a multi-scale one-dimensional convolutional neural network for time-frequency domain feature extraction. Multiple sets of one-dimensional convolutional kernels of different sizes are used to capture short-term heart rate variability features and long-term ST segment morphological features, respectively, and depthwise separable convolutions are used to reduce the number of parameters. The respiratory and body movement signals output from Step S1 are processed using a bidirectional gated recurrent unit network for temporal modeling to capture the contextual dependencies of apnea events and the dynamic evolution patterns of periodic limb movements. The ECG features, respiratory features, and body movement features are mapped to a unified feature vector space, outputting three sets of normalized feature vectors. Step S3, Cross-modal attention fusion: Perform cross-modal attention fusion processing on the three sets of normalized feature vectors output in step S2, and construct a cross-modal attention module with ECG feature vector as query vector and respiratory feature vector and body movement feature vector as key-value pairs. The contribution of each modality is dynamically adjusted by calculating the attention weight matrix between channels. When ST segment depression is detected in the ECG signal, the attention weight of the respiratory channel is automatically increased to enhance the ability to identify acute exacerbations of nocturnal heart failure. The weighted fused multimodal features are mapped to a comprehensive risk feature representation through a fully connected layer.

2. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 1, characterized in that, In step S1, the bandpass filter cutoff frequency of the electrocardiogram signal is set to 0.5Hz to 40Hz, the power frequency notch center frequency is set to 50Hz, and the notch bandwidth is set to 2Hz; the low-pass filter cutoff frequency of the respiratory signal is set to 0.5Hz; the high-pass filter cutoff frequency of the body movement signal is set to 0.1Hz; and the sampling rate of the unified sampling clock is set to 250Hz.

3. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 1, characterized in that, In step S2, the multi-scale one-dimensional convolutional neural network contains three sets of parallel convolutional branches with kernel sizes of 1×5, 1×15 and 1×31, respectively, and their corresponding receptive fields cover ECG signal time windows of 20ms, 60ms and 124ms, respectively; the hidden layer dimension of the bidirectional gated recurrent unit network is set to 128, and the time step is set to 30s corresponding to a sleep stage segment.

4. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 1, characterized in that, It also includes step S4, hierarchical risk assessment and early warning decision-making: hierarchical risk assessment processing is performed on the comprehensive risk characteristics output by step S3, and the risk level is divided into four levels: safe level, attention level, early warning level and emergency level. The system sequentially performs coarse-grained risk screening and fine-grained risk classification using a cascaded classifier, and performs personalized threshold correction based on the patient's historical baseline data, outputting the risk level label and confidence score for the current time window. When the risk level reaches the warning or emergency level, an audible and visual alarm is triggered locally on the edge device and a warning message is pushed to the remote nursing terminal. Simultaneously, the risk assessment results are fed back to the attention weight adjustment module in step S3 to dynamically optimize the fusion strategy for subsequent time windows. The comprehensive risk score thresholds corresponding to the four risk levels are as follows: the risk score for the safety level is less than 0.25, the risk score for the attention level is greater than or equal to 0.25 and less than 0.50, the risk score for the warning level is greater than or equal to 0.50 and less than 0.75, and the risk score for the emergency level is greater than or equal to 0.

75.

5. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 1, characterized in that, In step S3, the attention weight calculation process of the cross-modal attention module is as follows: the ECG feature vector is mapped to a query matrix through a first linear transformation matrix, the respiration feature vector is mapped to a key matrix through a second linear transformation matrix, and the body motion feature vector is mapped to a value matrix through a third linear transformation matrix; after performing scaling dot product operation on the query matrix and the key matrix, the attention weight matrix is ​​obtained by Softmax normalization, and then multiplied with the value matrix to obtain the weighted fusion feature.

6. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 5, characterized in that, The cross-modal attention module also includes a dynamic gating mechanism based on physiological state: when the ST segment offset of the ECG channel exceeds a preset offset threshold, the attention weight of the respiratory channel is increased to 1.5 to 2.0 times the baseline weight through a gating signal, while the attention weight of the body movement channel is reduced to 0.5 to 0.8 times the baseline weight.

7. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 1, characterized in that, The method also includes step S5, knowledge distillation edge deployment: the cloud-based teacher model constructed in steps S2 to S4 is compressed into a lightweight student model adapted to edge computing devices using a knowledge distillation method; during the distillation process, the intermediate layer feature distributions of the teacher model and the student model and the soft label distribution of the final output layer are aligned simultaneously to maintain multimodal feature fusion capability and risk classification accuracy; the distilled student model is deployed on an edge computing device to achieve real-time risk warning with an inference latency of no more than 200ms. The knowledge distillation method adopts a dual-channel distillation strategy that combines intermediate layer feature distillation and output layer soft label distillation; wherein, the loss function of intermediate layer feature distillation is the mean square error loss between the intermediate layer features of the teacher model and the intermediate layer features of the student model, and the loss function of output layer soft label distillation is the KL divergence loss between the soft label distribution of the teacher model and the soft label distribution of the student model, and the distillation temperature coefficient is set to 4.

8. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 4, characterized in that, In step S4, the cascaded classifier includes a coarse-grained screening network and a fine-grained classification network connected in series. The coarse-grained screening network is a fully connected network with two hidden layers, used to binary classify the input comprehensive risk features into normal and abnormal states. The fine-grained classification network is a fully connected network with three hidden layers, which only classifies samples that are determined to be abnormal by the coarse-grained screening network into three levels: attention level, warning level, and emergency level.

9. The sleep risk early warning method for heart failure patients based on multimodal data fusion according to claim 8, characterized in that, In step S4, the personalized threshold correction includes: collecting physiological data during the first complete sleep cycle after the patient falls asleep as personal baseline data, constructing a personalized normal range interval based on the mean and standard deviation of the personal baseline data, and dynamically adjusting the score thresholds of the four risk levels according to the personalized normal range interval.

10. A sleep risk early warning system for heart failure patients based on multimodal data fusion, used to implement the sleep risk early warning method for heart failure patients based on multimodal data fusion as described in claim 9, characterized in that, The system includes: The multimodal signal acquisition and preprocessing module is used to synchronously acquire electrocardiogram (ECG), respiratory, and body movement signals through wearable sensing devices during the sleep of heart failure patients. The module performs bandpass filtering and power frequency notch filtering on ECG signals, low-pass filtering and envelope extraction on respiratory signals, and high-pass filtering and motion energy integration on body movement signals. After time alignment, the module outputs a synchronized multimodal signal sequence. The multi-scale temporal feature extraction module is used to extract time-frequency domain features from electrocardiogram signals using a multi-scale one-dimensional convolutional neural network, and to perform temporal modeling of respiratory signals and body movement signals using a bidirectional gated recurrent unit network, and to map the three-channel features to a unified feature vector space. The cross-modal attention fusion module is used to construct a cross-modal attention mechanism with ECG feature vectors as query vectors and respiratory and body movement feature vectors as key-value pairs. It achieves adaptive fusion of multimodal features by dynamically adjusting the attention weights of each modality. The hierarchical risk assessment and early warning decision module is used to perform a four-level risk hierarchical assessment on the integrated risk characteristics after fusion. It outputs risk level labels through a cascaded classifier and triggers local alarms and remote push when the risk level reaches the early warning level or the emergency level. At the same time, the risk assessment results are fed back to the cross-modal attention fusion module to dynamically optimize the fusion strategy. The knowledge distillation edge deployment module is used to compress the cloud-based teacher model into a lightweight student model through a dual-channel knowledge distillation method, and deploy the student model on an edge computing device to achieve real-time risk warning with an inference latency of no more than 200ms.

Images