Flexible wearable heart function monitoring method, system and device based on multiple modes

Abstract

The invention discloses a multi-mode-based flexible wearable heart function monitoring method, system and device, and relates to the technical field of health monitoring, and the method comprises the steps: outputting a multi-mode signal through a flexible patch sensing module when the flexible patch sensing module is arranged in a detection area; wherein the flexible patch sensing module comprises an electrocardio sensor, a pulse wave sensor, a heart sound sensor, a strain sensor and a microfluidic biosensor. According to the invention, high-precision synchronous signal acquisition is realized through flexible patch integrated multi-modal sensing, heart multi-source signal features are adaptively extracted in combination with a multi-scale convolutional neural model, and timing sequence relevance is captured by using a bidirectional long-short-term memory network, so that the accuracy and reliability of heart disease diagnosis are remarkably improved, and the accuracy and reliability of heart disease diagnosis are improved. Meanwhile, non-invasive and continuous monitoring and early pathological screening are supported.

Description

Technical Field

[0001] The present invention relates to the field of health monitoring technology, and in particular to a multimodal-based flexible wearable cardiac function monitoring method, system and device. Background Art

[0002] In the clinical diagnosis of cardiovascular diseases, cardiac electrophysiological signals (ECG), mechanical vibration signals (PCG) and blood oxygen saturation (SpO2) are core indicators for assessing cardiac function. Existing technologies mainly collect these signals through devices such as Holter monitors, stethoscopes and oximeters. However, a single modal signal can only reflect the local characteristics of cardiac activity and is difficult to meet the needs of comprehensive analysis of complex pathological mechanisms.

[0003] Current cardiac monitoring technology faces significant bottlenecks: on the one hand, traditional methods rely on single-modal signals (such as analyzing only ECG or PCG), ignoring the dynamic coupling between electrical, mechanical, and oxygen signals. For example, they cannot synchronously capture the temporal correlation between the ECG R wave and the S1 component of the heart sound. On the other hand, deep learning-based methods have inherent flaws. For example, fixed-scale convolution kernels cannot adapt to the multi-scale characteristics of the high-frequency transient components of the heart sound signal and the low-frequency rhythm of the ECG signal. The RNN architecture is prone to gradient vanishing when modeling long sequences, and unidirectional time series modeling makes it difficult to analyze the reverse physiological conduction mechanism of the cardiac cycle. In addition, multimodal fusion often uses feature splicing or weighted averaging, resulting in the deep interactions between signals (such as the collaborative judgment of SpO2 on myocardial ischemia) not being effectively explored.

[0004] Existing technologies suffer from four core flaws: (1) Single-signal analysis dissects the spatiotemporal correlations of cardiac electromechanical activity; (2) Fixed convolution kernels limit the ability to extract multi-scale features; (3) RNN architectures are inadequate for modeling long sequences and ignore reverse temporal dependencies; and (4) Multimodal feature fusion is simplistic and cannot achieve deep interaction of physiological correlations. These issues result in insufficient sensitivity and specificity for early diagnosis of heart disease, especially for latent lesions such as coronary heart disease and arrhythmias, resulting in a high rate of missed detection. An innovative monitoring method that integrates multimodal signals, adaptive feature extraction, and bidirectional temporal modeling is urgently needed. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a flexible wearable cardiac function monitoring method, system and device based on multimodality, aiming to solve the above-mentioned problems recorded in the prior art.

[0006] A first aspect of the present invention is to provide a multimodal, flexible, wearable cardiac function monitoring method, the method comprising: After a preset flexible patch sensing module is attached and placed on the corresponding area of ​​the patient's heart, the multimodal signal is detected by the flexible patch sensing module; wherein the flexible patch sensing module includes an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor, and a microfluidic biosensor; Acquiring an electrocardiogram signal, a pulse wave signal, a heart sound signal, a strain signal, and a biological signal, and performing denoising, filtering, and normalization processing on the electrocardiogram signal, the pulse wave signal, the heart sound signal, the strain signal, and the biological signal; Extract features from the preprocessed ECG signal, pulse wave signal, heart sound signal, strain signal, and biological signal using a multi-scale convolutional neural model to obtain corresponding feature signals, perform feature fusion, and output a fused signal; According to the fusion signal, the heart disease is predicted and the disease type is classified through a bidirectional long short-term memory network, and evaluation data corresponding to the patient's heart function is output according to the classification result.

[0007] According to one aspect of the above technical solution, the steps of extracting features from the preprocessed electrocardiogram signal, pulse wave signal, heart sound signal, strain signal, and biological signal using a multi-scale convolutional neural model, obtaining corresponding feature signals, performing feature fusion, and outputting the fused signal include: Extract features of the electrocardiogram (ECG) signal, pulse wave signal, heart sound signal, strain signal, and bio-signal at different time scales using a multi-scale convolutional neural model to obtain ECG feature signals, pulse wave feature signals, heart sound feature signals, strain feature signals, and bio-marker feature signals, respectively; At the end of the multi-scale convolutional neural model, feature fusion is performed on the electrocardiogram feature signal, pulse wave feature signal, heart sound feature signal, strain feature signal and biomarker feature signal to integrate and superimpose all extracted feature signals to obtain a fused signal.

[0008] According to one aspect of the above technical solution, when extracting features from the electrocardiogram signal using a multi-scale convolutional neural model, electrocardiogram feature signals from the electrocardiogram signal are extracted using three parallel convolutional layers; Wherein, the parallel convolutional layer includes: The first convolutional layer is used to capture local heartbeat mutation characteristics based on the electrocardiogram signal; A second convolutional layer is used to expand the receptive field based on the local heartbeat mutation feature; A third convolutional layer is configured to downsample to extract macro features based on the local heartbeat mutation features to obtain the electrocardiogram characteristic signal; Each of the parallel convolutional layers is followed by a normalization layer, a batch normalization layer, an activation layer, and an attention layer.

[0009] According to one aspect of the above technical solution, when extracting features from the heart sound signal using a multi-scale convolutional neural model, a depthwise separable convolutional layer is used to extract the heart sound feature signal from the heart sound signal; The depth-wise separable convolutional layer includes: a first depthwise separable convolutional layer, configured to perform main frequency band analysis based on the heart sound signal; The second depthwise separable convolutional layer is used to perform harmonic analysis based on the main frequency band of the heart sound signal to capture the rise of heart murmurs; The third depth-wise separable convolutional layer is used for broadband pattern analysis to receive all abnormal sounds in the heart sound signal; Each layer of the depthwise separable convolutional layer is followed by frequency domain windowing and absolute value activation.

[0010] According to one aspect of the above technical solution, at the end of the multi-scale convolutional neural model, the step of performing feature fusion on the electrocardiographic characteristic signal, the pulse wave characteristic signal, the heart sound characteristic signal, the strain characteristic signal, and the biomarker characteristic signal to integrate and superimpose all the extracted characteristic signals to obtain a fused signal includes at least: Performing maximum pooling on the ECG time domain features of the ECG feature signal, and taking key points at each preset time; Linearly interpolate the heart sound domain features of the heart sound characteristic signal to adjust the time series length, stretch the auscultation recording to the same duration as the time series length, downsample the blood oxygen protection signal to 1 / 8 the frequency of the electrocardiogram characteristic signal and scale it to a range of 90-100%; The ECG time domain features are used as the query, the cardiac sound domain features are used as the key, and the blood oxygen saturation is used as the value. The attention weight matrix is ​​calculated to achieve weighted fusion. The original characteristic signal of each signal is added to the fused characteristic signal to achieve residual superposition.

[0011] According to one aspect of the above technical solution, the steps of predicting heart disease and classifying the disease type using a bidirectional long short-term memory network based on the fusion signal, and outputting evaluation data corresponding to the patient's heart function according to the classification results include: Based on the fusion signal, the temporal features output by the bidirectional long short-term memory network are compressed by attention pooling through the fully connected layer of the multi-scale convolutional neural model to achieve prediction of heart disease and classification of disease types; According to the classification results, the evaluation data corresponding to the patient's cardiac function is output.

[0012] According to one aspect of the above technical solution, the method further includes: Segmentally processing the heart sound signal and the electrocardiogram signal, obtaining corresponding heart sound characteristic signals and electrocardiogram characteristic signals and performing preprocessing; Blood pressure prediction is performed using a pre-trained blood pressure prediction model based on the heart sound characteristic signal and the electrocardiogram characteristic signal.

[0013] A second aspect of the present invention is to provide a multimodal flexible wearable cardiac function monitoring system, which is applied to the method described in the above technical solution, and the system includes: A signal detection module is configured to detect multimodal signals via a pre-set flexible patch sensing module after the module is attached to a corresponding area of ​​the patient's heart; wherein the flexible patch sensing module includes an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor, and a microfluidic biosensor; a signal processing module, configured to obtain an electrocardiogram signal, a pulse wave signal, a heart sound signal, a strain signal, and a biological signal, and to perform denoising, filtering, and normalization on the electrocardiogram signal, the pulse wave signal, the heart sound signal, the strain signal, and the biological signal; A signal fusion module is used to extract features from the pre-processed electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal through a multi-scale convolutional neural model, obtain corresponding feature signals, perform feature fusion, and output a fused signal; The diagnosis and classification module is used to predict heart diseases and classify disease types based on the fusion signal through a bidirectional long short-term memory network, and output evaluation data corresponding to the patient's heart function according to the classification results.

[0014] A third aspect of the present invention is to provide a multimodal flexible wearable cardiac function monitoring device, which is applied to the method described in the above technical solution, and the device comprises: Flexible patches; An electrocardiogram sensor, comprising a first electrocardiogram electrode and a second electrocardiogram electrode, which are disposed on the same surface of the flexible patch and are spaced apart from each other; A heart sound sensor is provided on the flexible patch, and the heart sound sensor is provided between the first electrocardiogram electrode and the second electrocardiogram electrode.

[0015] According to one aspect of the above technical solution, the device further includes: A pulse wave sensor is provided on the flexible patch and is used to detect changes in weak light signals to output a pulse wave signal; A strain sensor is provided on the flexible patch and monitors minute chest deformation based on the linear relationship between the resistance change rate and the strain signal; and microfluidic biosensors that integrate microfluidic channels with immunosensors to detect biomarkers.

[0016] Compared with the prior art, the multimodal flexible wearable cardiac function monitoring method, system and device shown in the present invention have the following beneficial effects: This invention comprehensively breaks through the technical bottleneck of traditional cardiac monitoring by integrating multimodal sensing and deep learning. The flexible patch synchronously collects five-dimensional signals of electrocardiogram, pulse wave, heart sound, myocardial deformation and biomarkers to realize holographic fusion analysis of cardiac electrophysiological activity, hemodynamics, mechanical vibration, tissue mechanics and biochemical indicators; dynamically extracts different frequency domain features through adaptive multi-scale convolutional neural network to overcome the perception limitations of fixed convolution kernel; based on the deep fusion of temporal correlation characteristics of bidirectional LSTM network, the collaborative modeling of pulse wave conduction time, heart sound components and myocardial strain data is used to significantly improve the ability to identify pathological conditions; innovatively introduces microfluidic biosensors to realize cross-validation of sweat biomarkers and physical signals, providing early warning for hidden lesions. This technology greatly improves the sensitivity and specificity of heart disease diagnosis, supports non-invasive, continuous dynamic monitoring, and builds a closed-loop solution for cardiovascular health management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which: Figure 1 Schematic diagram of a flow chart of a multimodal flexible wearable cardiac function monitoring method according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a multimodal flexible wearable cardiac function monitoring system in one embodiment of the present invention. DETAILED DESCRIPTION

[0018] To make the objectives, features, and advantages of the present invention more readily apparent, the following detailed description of specific embodiments of the present invention is provided in conjunction with the accompanying drawings. The accompanying drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0019] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly attached to the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] Example 1 See also Figure 1 The first embodiment of the present invention provides a multimodal flexible wearable cardiac function monitoring method, the method comprising steps S10 to S40: Step S10 : After the preset flexible patch sensor module is attached and arranged on the corresponding area of ​​the patient's heart, the multimodal signal is detected by the flexible patch sensor module.

[0022] The flexible patch sensing module includes an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor, and a microfluidic biosensor. This embodiment implements the above method by adopting a multimodal flexible wearable cardiac function monitoring device, which includes a flexible patch sensing module, which includes a flexible patch, an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor and a microfluidic biosensor.

[0023] The ECG sensor includes a first ECG electrode and a second ECG electrode, which are arranged on the same surface of the flexible patch and are spaced apart. The heart sound sensor is arranged on the flexible patch and located between the first ECG electrode and the second ECG electrode.

[0024] More specifically, the electrocardiogram (RCG) sensor utilizes flexible dry electrode materials, preferably graphene electrodes, leveraging their excellent conductivity and flexibility to measure cardiac electrical signals on the patient's skin surface, capturing cardiac depolarization and repolarization processes. This provides key evidence for diagnosing conditions such as arrhythmias and myocardial ischemia. The pulse wave sensor, also known as the photoplethysmography (PPG) sensor, comprises a green LED and a photodiode. Based on the principles of light reflection and absorption, it captures pulse wave signals, measures heart rate and blood oxygen saturation, and estimates blood pressure using pulse wave transit time (PTT). The heart sound sensor is a MEMS piezoresistive heart sound sensor, designed to capture mechanical vibration signals generated by the opening and closing of heart valves and myocardial contraction and relaxation. By analyzing the components of the heart sounds, it aids in the diagnosis of valvular disease, heart failure, and other conditions. The strain sensor utilizes flexible nanomaterials, such as a carbon nanotube / PDMS composite, to create a strain gauge. This strain gauge adheres to the patient's chest skin, monitoring minute chest deformation during heartbeat and further assessing cardiac contractility and pumping function. Microfluidic biosensors integrate microfluidic channels and biorecognition elements, such as immunosensors, to detect biomarkers such as cardiac troponin I (cTnI) and brain natriuretic peptide (BNP) through skin micropermeation to achieve early warning of heart disease.

[0025] When using the multimodal flexible wearable cardiac function monitoring device, the flexible patch is adhered to the corresponding area of ​​the patient's heart, and then the multimodal signal is output according to the sensor integrated in the above-mentioned flexible patch, including detecting the patient's corresponding electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal, so that multiple signals can be obtained simultaneously through the flexible wearable cardiac function monitoring device, which is particularly suitable for long-term dynamic monitoring of bedridden patients, reducing the inconvenience of use for patients.

[0026] Step S20 , acquiring an electrocardiogram signal, a pulse wave signal, a heart sound signal, a strain signal, and a biological signal, and performing denoising, filtering, and normalization processing on the electrocardiogram signal, the pulse wave signal, the heart sound signal, the strain signal, and the biological signal.

[0027] In this embodiment, after the multimodal signal is detected by the flexible patch sensing module, in order to facilitate subsequent disease prediction and type classification, this embodiment will also perform denoising, filtering and normalization on the electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal.

[0028] Among them, the electrocardiogram (ECG) sensor, through electrodes in contact with the skin, records the potential difference generated by cardiac electrical activity, forming an electrocardiogram (ECG). By analyzing waveform features such as the P wave, QRS complex, and T wave, it can identify abnormalities such as arrhythmias and myocardial infarction. The pulse wave sensor uses a green LED to illuminate the skin. Since hemoglobin's absorption of green light changes with changes in vascular volume, the photodiode receives the reflected light signal and converts it into an electrical signal. An algorithm then extracts heart rate, blood oxygen saturation, and pulse wave characteristic parameters. The heart sound sensor converts the mechanical vibrations of the heart into electrical signals, capturing the first and second heart sounds S1 and S2, as well as additional heart sounds such as S3 and S4. Time-frequency analysis is then used to identify abnormal heart sound patterns. The strain sensor changes its resistance with the deformation of the chest muscles and skin. This signal is converted into a voltage signal using a Wheatstone bridge circuit, reflecting the mechanical changes during cardiac contraction and assisting in the assessment of cardiac function. Microfluidic biosensors utilize the reaction between skin microexudates and biorecognition elements, such as antibody-antigen specific binding. Through electrochemical or optical detection, the signal is quantified to predict the risk of myocardial injury.

[0029] Specifically, the above signals are subjected to denoising, filtering and normalization processing, including synchronously transmitting the electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal to a microprocessor, such as synchronously transmitting it to a low-power ARM chip, and then aligning the above electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal based on time series to construct a multimodal data set.

[0030] Step S30, extracting features from the pre-processed ECG signal, pulse wave signal, heart sound signal, strain signal and biological signal through a multi-scale convolutional neural model, obtaining corresponding feature signals, performing feature fusion, and outputting a fused signal.

[0031] In this embodiment, after preprocessing the above-mentioned ECG signals, pulse wave signals, heart sound signals, strain signals and biological signals and constructing a multimodal data set, the above-mentioned ECG signals, pulse wave signals, heart sound signals, strain signals and biological signals are respectively subjected to feature extraction through a multi-scale convolutional neural model to obtain corresponding feature signals and perform feature fusion to obtain a fused signal.

[0032] The steps of extracting features from the pre-processed electrocardiogram signal, pulse wave signal, heart sound signal, strain signal, and biological signal using a multi-scale convolutional neural model, obtaining corresponding feature signals, performing feature fusion, and outputting fused signals include: Extract features of the electrocardiogram (ECG) signal, pulse wave signal, heart sound signal, strain signal, and bio-signal at different time scales using a multi-scale convolutional neural model to obtain ECG feature signals, pulse wave feature signals, heart sound feature signals, strain feature signals, and bio-marker feature signals, respectively; At the end of the multi-scale convolutional neural model, feature fusion is performed on the electrocardiogram feature signal, pulse wave feature signal, heart sound feature signal, strain feature signal and biomarker feature signal to integrate and superimpose all extracted feature signals to obtain a fused signal.

[0033] When the electrocardiogram signal is subjected to feature extraction using a multi-scale convolutional neural model, electrocardiogram feature signals in the electrocardiogram signal are extracted using three parallel convolutional layers; Wherein, the parallel convolutional layer includes: The first convolutional layer is used to capture local heartbeat mutation characteristics based on the electrocardiogram signal; A second convolutional layer is used to expand the receptive field based on the local heartbeat mutation feature; A third convolutional layer is configured to downsample to extract macro features based on the local heartbeat mutation features to obtain the electrocardiogram characteristic signal; Each of the parallel convolutional layers is followed by a normalization layer, a batch normalization layer, an activation layer, and an attention layer.

[0034] Wherein, when extracting features of the heart sound signal through the multi-scale convolutional neural model, a depthwise separable convolutional layer is used to extract the heart sound feature signal in the heart sound signal; The depth-wise separable convolutional layer includes: a first depthwise separable convolutional layer, configured to perform main frequency band analysis based on the heart sound signal; The second depthwise separable convolutional layer is used to perform harmonic analysis based on the main frequency band of the heart sound signal to capture the rise of heart murmurs; The third depth-wise separable convolutional layer is used for broadband pattern analysis to receive all abnormal sounds in the heart sound signal; Each layer of the depthwise separable convolutional layer is followed by frequency domain windowing and absolute value activation.

[0035] Among them, at the end of the multi-scale convolutional neural model, the step of performing feature fusion on the electrocardiogram characteristic signal, the pulse wave characteristic signal, the heart sound characteristic signal, the strain characteristic signal and the biomarker characteristic signal to integrate and superimpose all the extracted characteristic signals to obtain a fused signal includes at least: Performing maximum pooling on the ECG time domain features of the ECG feature signal, and taking key points at each preset time; Linearly interpolate the heart sound domain features of the heart sound characteristic signal to adjust the time series length, stretch the auscultation recording to the same duration as the time series length, downsample the blood oxygen protection signal to 1 / 8 the frequency of the electrocardiogram characteristic signal and scale it to a range of 90-100%; The ECG time domain features are used as the query, the cardiac sound domain features are used as the key, and the blood oxygen saturation is used as the value. The attention weight matrix is ​​calculated to achieve weighted fusion. The original characteristic signal of each signal is added to the fused characteristic signal to achieve residual superposition.

[0036] Specifically, feature extraction is performed on the above-mentioned signals, that is, the deep-level features implicit in the above-mentioned signals are extracted based on the above-mentioned signals, so as to better utilize the above-mentioned signals for subsequent disease prediction and classification. For electrocardiographic signals, this includes calculating parameters such as the RR interval, ST segment deviation, and QT interval, and extracting the corresponding electrocardiographic feature signals. For pulse wave signals, this includes identifying the pulse wave conduction velocity and waveform characteristics, and extracting the corresponding pulse wave feature signals. For heart sound signals, this includes identifying the frequency, intensity, and duration of the heart sound components, and extracting the corresponding heart sound feature signals. For strain signals, this includes calculating the amplitude and frequency of chest deformation, and extracting the corresponding strain feature signals. And for biological signals, this includes comparing biomarkers with normal thresholds, judging their abnormal concentrations, and extracting the corresponding biological feature signals.

[0037] This embodiment performs feature fusion on the above-mentioned electrocardiographic characteristic signal, pulse wave characteristic signal, heart sound characteristic signal, strain characteristic signal and biometric characteristic signal to obtain a fused signal.

[0038] Step S40: predicting heart disease and classifying disease types through a bidirectional long short-term memory network based on the fusion signal, and outputting evaluation data corresponding to the patient's heart function according to the classification results.

[0039] In this embodiment, based on the above-mentioned fusion signal, a bidirectional long short-term memory network is used to predict heart disease and classify the disease type, that is, to determine whether heart disease exists and classify the disease type of heart disease, so that evaluation data corresponding to the patient's heart function can be output according to the classification results.

[0040] Specifically, a bidirectional long short-term memory network is used to train a model based on a large amount of clinical data to achieve functions such as arrhythmia classification such as atrial fibrillation, ventricular premature beats, myocardial ischemia warning, and heart failure risk assessment.

[0041] More specifically, the output assessment data can be transmitted via Bluetooth or low-power wide-area networks to smart terminals or cloud platforms. Smart terminals can be used by patients or medical staff. On the patient side, corresponding data such as heart rate, blood pressure, heart sound waveforms, and biomarker concentrations can be displayed in real time, and a risk assessment report can be provided. On the medical side, remote testing can be performed, and AI can be combined to output diagnostic recommendations to assist in clinical decision-making.

[0042] In summary, compared with the prior art, the multimodal flexible wearable cardiac function monitoring method shown in this embodiment has the following beneficial effects: This embodiment comprehensively breaks through the technical bottleneck of traditional cardiac monitoring by integrating multimodal sensing and deep learning. The flexible patch synchronously collects five-dimensional signals of electrocardiogram, pulse wave, heart sound, myocardial deformation and biomarkers to achieve holographic fusion analysis of cardiac electrophysiological activity, hemodynamics, mechanical vibration, tissue mechanics and biochemical indicators; dynamically extracts different frequency domain features through adaptive multi-scale convolutional neural network to overcome the perception limitations of fixed convolution kernel; based on the deep fusion of temporal correlation characteristics of bidirectional LSTM network, the collaborative modeling of pulse wave conduction time, heart sound components and myocardial strain data is used to significantly improve the ability to identify pathological conditions; innovatively introduces microfluidic biosensors to achieve cross-validation of sweat biomarkers and physical signals, providing early warning for hidden lesions. This technology greatly improves the sensitivity and specificity of heart disease diagnosis, supports non-invasive, continuous dynamic monitoring, and builds a closed-loop solution for cardiovascular health management.

[0043] Example 2 The second embodiment of the present invention provides a multimodal flexible wearable cardiac function monitoring method. The method shown in this embodiment is basically similar to the method shown in the first embodiment, except that: In this embodiment, the steps of predicting heart disease and classifying the disease type using a bidirectional long short-term memory network based on the fusion signal, and outputting assessment data corresponding to the patient's heart function according to the classification results include: Based on the fusion signal, the temporal features output by the bidirectional long short-term memory network are compressed by attention pooling through the fully connected layer of the multi-scale convolutional neural model to achieve prediction of heart disease and classification of disease types; According to the classification results, the evaluation data corresponding to the patient's cardiac function is output.

[0044] Specifically, the temporal feature matrix (dimension: time steps T × feature dimension D) output by a bidirectional long short-term memory (Bi-LSTM) network is input into a fully connected layer, where it undergoes a nonlinear transformation: h'_t = ReLU(W_fc·h_t+b_fc), where W_fc is a weight matrix. This maps the original features into a 128-dimensional high-resolution space, enhancing the representation of pathological features such as ST segment deviation and splitting of heart sounds. Attention weights are then calculated for the feature vector h'_t at each time step, and the weighted, aggregated fixed-dimensional vector is used as the classification input: h_att = Σ(α_t·h'_t). The compressed feature vector h_att is then input into a fully connected classifier, which outputs disease probability predictions and clinical assessment data such as NYHA cardiac function class.

[0045] Additionally, the method further comprises: Segmentally processing the heart sound signal and the electrocardiogram signal, obtaining corresponding heart sound characteristic signals and electrocardiogram characteristic signals and performing preprocessing; Blood pressure prediction is performed using a pre-trained blood pressure prediction model based on the heart sound characteristic signal and the electrocardiogram characteristic signal.

[0046] Among them, the blood pressure prediction model is a cross-modal regression network based on LSTM-Transformer.

[0047] Specifically, when segmenting electrocardiogram (ECG) signals, the R wave peak is used as the anchor point, and a window of [R-100ms, R+400ms] is captured to cover the complete QRS-T cycle, from which the QRS duration and ST segment slope are extracted. When segmenting cardiac sound signals (PCG), the ECG R wave is synchronized and locked, and the S1 / S2 components are divided into [systole, diastole] to extract the S1 amplitude and S2-S1 interval.

[0048] This embodiment performs attention-enhanced classification and ECG / PCG fusion blood pressure prediction. While retaining the advantages of multimodal signals, it not only improves the sensitivity and specificity of disease diagnosis, but also adds a non-invasive continuous blood pressure monitoring function, providing a closed-loop management solution for cardiovascular diseases.

[0049] Example 3 See also Figure 2 A third embodiment of the present invention provides a multimodal flexible wearable cardiac function monitoring system, which is applied to the method described in any of the above embodiments, and the system includes: The signal detection module 10 is configured to detect multimodal signals via a pre-set flexible patch sensor module after the module is attached to the corresponding area of ​​the patient's heart. The flexible patch sensor module includes an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor, and a microfluidic biosensor.

[0050] The signal processing module 20 is used to obtain electrocardiographic signals, pulse wave signals, heart sound signals, strain signals and biological signals, and perform denoising, filtering and normalization on the electrocardiographic signals, pulse wave signals, heart sound signals, strain signals and biological signals.

[0051] The signal fusion module 30 is used to extract features of the pre-processed electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal through a multi-scale convolutional neural model, obtain corresponding feature signals, perform feature fusion, and output a fused signal.

[0052] The diagnosis and classification module 40 is used to predict heart diseases and classify disease types based on the fusion signal through a bidirectional long short-term memory network, and output evaluation data corresponding to the patient's heart function according to the classification results.

[0053] In summary, compared with the prior art, the multimodal flexible wearable cardiac function monitoring system shown in this embodiment has the following beneficial effects: This embodiment comprehensively breaks through the technical bottleneck of traditional cardiac monitoring by integrating multimodal sensing and deep learning. The flexible patch synchronously collects five-dimensional signals of electrocardiogram, pulse wave, heart sound, myocardial deformation and biomarkers to achieve holographic fusion analysis of cardiac electrophysiological activity, hemodynamics, mechanical vibration, tissue mechanics and biochemical indicators; dynamically extracts different frequency domain features through adaptive multi-scale convolutional neural network to overcome the perception limitations of fixed convolution kernel; based on the deep fusion of temporal correlation characteristics of bidirectional LSTM network, the collaborative modeling of pulse wave conduction time, heart sound components and myocardial strain data is used to significantly improve the ability to identify pathological conditions; innovatively introduces microfluidic biosensors to achieve cross-validation of sweat biomarkers and physical signals, providing early warning for hidden lesions. This technology greatly improves the sensitivity and specificity of heart disease diagnosis, supports non-invasive, continuous dynamic monitoring, and builds a closed-loop solution for cardiovascular health management.

[0054] Example 4 A fourth embodiment of the present invention provides a multimodal, flexible, wearable cardiac function monitoring device, applicable to the method described in any of the above embodiments, the device comprising: Flexible patches; An electrocardiogram sensor, comprising a first electrocardiogram electrode and a second electrocardiogram electrode, which are disposed on the same surface of the flexible patch and are spaced apart from each other; A heart sound sensor is provided on the flexible patch, and the heart sound sensor is provided between the first electrocardiogram electrode and the second electrocardiogram electrode.

[0055] Wherein, the device further includes: A pulse wave sensor is provided on the flexible patch and is used to detect changes in weak light signals to output a pulse wave signal; A strain sensor is provided on the flexible patch and monitors minute chest deformation based on the linear relationship between the resistance change rate and the strain signal; and microfluidic biosensors that integrate microfluidic channels with immunosensors to detect biomarkers.

[0056] In this embodiment, the flexible patch synchronously collects five-dimensional signals of electrocardiogram, pulse wave, heart sound, myocardial deformation, and biomarkers, realizing holographic fusion analysis of cardiac electrophysiological activity, hemodynamics, mechanical vibration, tissue mechanics, and biochemical indicators. It dynamically extracts different frequency domain features through an adaptive multi-scale convolutional neural network, overcoming the perception limitations of fixed convolution kernels. Based on the deep fusion of temporal correlation characteristics of a bidirectional LSTM network, it utilizes collaborative modeling of pulse wave conduction time, heart sound components, and myocardial strain data to significantly improve the ability to identify pathological conditions. The innovative introduction of microfluidic biosensors enables cross-validation of sweat biomarkers and physical signals, providing early warning for hidden lesions. This technology significantly improves the sensitivity and specificity of heart disease diagnosis, supports non-invasive, continuous dynamic monitoring, and builds a closed-loop solution for cardiovascular health management.

[0057] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or for use in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0059] More specific examples (a non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0060] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0061] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0062] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A multimodal flexible wearable cardiac function monitoring method, characterized in that: The method comprises: After a preset flexible patch sensing module is attached and placed on the corresponding area of ​​the patient's heart, the multimodal signal is detected by the flexible patch sensing module; wherein the flexible patch sensing module includes an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor, and a microfluidic biosensor; Acquiring an electrocardiogram signal, a pulse wave signal, a heart sound signal, a strain signal, and a biological signal, and performing denoising, filtering, and normalization processing on the electrocardiogram signal, the pulse wave signal, the heart sound signal, the strain signal, and the biological signal; Extract features from the preprocessed ECG signal, pulse wave signal, heart sound signal, strain signal, and biological signal using a multi-scale convolutional neural model to obtain corresponding feature signals, perform feature fusion, and output a fused signal; According to the fusion signal, the heart disease is predicted and the disease type is classified through a bidirectional long short-term memory network, and evaluation data corresponding to the patient's heart function is output according to the classification result.

2. The multimodal wearable cardiac function monitoring method according to claim 1, characterized in that: The steps of extracting features from the pre-processed electrocardiogram signal, pulse wave signal, heart sound signal, strain signal, and biological signal using a multi-scale convolutional neural model, obtaining corresponding feature signals, performing feature fusion, and outputting fused signals include: Extract features of the electrocardiogram (ECG) signal, pulse wave signal, heart sound signal, strain signal, and bio-signal at different time scales using a multi-scale convolutional neural model to obtain ECG feature signals, pulse wave feature signals, heart sound feature signals, strain feature signals, and bio-marker feature signals, respectively; At the end of the multi-scale convolutional neural model, feature fusion is performed on the electrocardiogram feature signal, pulse wave feature signal, heart sound feature signal, strain feature signal and biomarker feature signal to integrate and superimpose all extracted feature signals to obtain a fused signal.

3. The multimodal wearable cardiac function monitoring method according to claim 2, characterized in that: When extracting features from the electrocardiogram signal using a multi-scale convolutional neural model, extracting electrocardiogram feature signals from the electrocardiogram signal using three parallel convolutional layers; Wherein, the parallel convolutional layer includes: The first convolutional layer is used to capture local heartbeat mutation characteristics based on the electrocardiogram signal; A second convolutional layer is used to expand the receptive field based on the local heartbeat mutation feature; A third convolutional layer is configured to downsample to extract macro features based on the local heartbeat mutation features to obtain the electrocardiogram characteristic signal; Each of the parallel convolutional layers is followed by a normalization layer, a batch normalization layer, an activation layer, and an attention layer.

4. The multimodal wearable cardiac function monitoring method according to claim 2, characterized in that: When extracting features of the heart sound signal through a multi-scale convolutional neural model, a depthwise separable convolutional layer is used to extract a heart sound feature signal from the heart sound signal; The depth-wise separable convolutional layer includes: a first depthwise separable convolutional layer, configured to perform main frequency band analysis based on the heart sound signal; The second depthwise separable convolutional layer is used to perform harmonic analysis based on the main frequency band of the heart sound signal to capture the rise of heart murmurs; The third depth-wise separable convolutional layer is used for broadband pattern analysis to receive all abnormal sounds in the heart sound signal; Each layer of the depthwise separable convolutional layer is followed by frequency domain windowing and absolute value activation.

5. The multimodal wearable cardiac function monitoring method according to claim 2, characterized in that: At the end of the multi-scale convolutional neural model, the step of performing feature fusion on the electrocardiographic characteristic signal, the pulse wave characteristic signal, the heart sound characteristic signal, the strain characteristic signal, and the biomarker characteristic signal to integrate and superimpose all the extracted characteristic signals to obtain a fused signal includes at least: Performing maximum pooling on the ECG time domain features of the ECG feature signal, and taking key points at each preset time; Linearly interpolate the heart sound domain features of the heart sound characteristic signal to adjust the time series length, stretch the auscultation recording to the same duration as the time series length, downsample the blood oxygen protection signal to 1 / 8 the frequency of the electrocardiogram characteristic signal and scale it to a range of 90-100%; The ECG time domain features are used as the query, the cardiac sound domain features are used as the key, and the blood oxygen saturation is used as the value. The attention weight matrix is ​​calculated to achieve weighted fusion. The original characteristic signal of each signal is added to the fused characteristic signal to achieve residual superposition.

6. The multimodal wearable cardiac function monitoring method according to claim 1, characterized in that: The steps of predicting heart disease and classifying the disease type using a bidirectional long short-term memory network based on the fusion signal, and outputting evaluation data corresponding to the patient's heart function according to the classification results include: Based on the fusion signal, the temporal features output by the bidirectional long short-term memory network are compressed by attention pooling through the fully connected layer of the multi-scale convolutional neural model to achieve prediction of heart disease and classification of disease types; According to the classification results, the evaluation data corresponding to the patient's cardiac function is output.

7. The multimodal, wearable, flexible cardiac function monitoring method according to any one of claims 1 to 6, characterized in that: The method further comprises: Segmentally processing the heart sound signal and the electrocardiogram signal, obtaining corresponding heart sound characteristic signals and electrocardiogram characteristic signals and performing preprocessing; Blood pressure prediction is performed using a pre-trained blood pressure prediction model based on the heart sound characteristic signal and the electrocardiogram characteristic signal.

8. A multimodal flexible wearable cardiac function monitoring system, characterized in that: The method according to any one of claims 1 to 7, wherein the system comprises: A signal detection module is configured to detect multimodal signals via a pre-set flexible patch sensing module after the module is attached to a corresponding area of ​​the patient's heart; wherein the flexible patch sensing module includes an electrocardiogram sensor, a pulse wave sensor, a heart sound sensor, a strain sensor, and a microfluidic biosensor; a signal processing module, configured to obtain an electrocardiogram signal, a pulse wave signal, a heart sound signal, a strain signal, and a biological signal, and to perform denoising, filtering, and normalization on the electrocardiogram signal, the pulse wave signal, the heart sound signal, the strain signal, and the biological signal; A signal fusion module is used to extract features from the pre-processed electrocardiogram signal, pulse wave signal, heart sound signal, strain signal and biological signal through a multi-scale convolutional neural model, obtain corresponding feature signals, perform feature fusion, and output a fused signal; The diagnosis and classification module is used to predict heart diseases and classify disease types based on the fusion signal through a bidirectional long short-term memory network, and output evaluation data corresponding to the patient's heart function according to the classification results.

9. A multimodal flexible wearable cardiac function monitoring device, characterized in that: The method according to any one of claims 1 to 7, wherein the device comprises: Flexible patches; An electrocardiogram sensor, comprising a first electrocardiogram electrode and a second electrocardiogram electrode, which are disposed on the same surface of the flexible patch and are spaced apart from each other; A heart sound sensor is provided on the flexible patch, and the heart sound sensor is provided between the first electrocardiogram electrode and the second electrocardiogram electrode.

10. The multimodal flexible wearable cardiac function monitoring device according to claim 9, characterized in that: The device further comprises: A pulse wave sensor is provided on the flexible patch and is used to detect changes in weak light signals to output a pulse wave signal; A strain sensor is provided on the flexible patch and monitors minute chest deformation based on the linear relationship between the resistance change rate and the strain signal; and microfluidic biosensors that integrate microfluidic channels with immunosensors to detect biomarkers.

Images