Multi-modal physiological data fusion analysis method and device and intelligent wearable equipment

Abstract

The invention is suitable for the technical field of intelligent wearable equipment, and provides a multi-modal physiological data fusion analysis method and device and intelligent wearable equipment. According to the multi-modal physiological data fusion analysis method, the various asynchronous original body data collected by the intelligent wearable device are subjected to preprocessing corresponding to the data types, and basic optimization of various data quality is ensured. The method comprises the steps of preprocessing various data, then performing space-time alignment fusion on the preprocessed various data, generating a space-time synchronous multi-dimensional fusion data vector, effectively overcoming the problem of time sequence misalignment caused by sampling frequency difference of different sensors, and reducing time sequence correlation errors caused by a traditional simple interpolation method. On the basis, a plurality of data features are extracted and fused to obtain a fused feature vector, so that the integrity and consistency of features dependent on subsequent analysis are guaranteed. And data features of different dimensions in the fusion feature vector are further independently derived, so that the state of each physiological index can be analyzed in a targeted manner.

Description

Technical Field

[0001] This application belongs to the field of intelligent wearable device technology, and particularly relates to a multimodal physiological data fusion analysis method, device and intelligent wearable device. Background Technology

[0002] With the integration of IoT and AI technologies, wearable devices such as smartwatches are increasingly being applied to health monitoring. These devices integrate multiple sensors to collect various body data in real time, such as heart rate, blood oxygen saturation, posture, and blood pressure. Compared to traditional medical devices, they offer advantages such as continuous monitoring and non-invasive convenience, making them an effective tool for managing chronic diseases like hypertension and arrhythmia.

[0003] Because different sensors have different sampling frequencies—for example, acceleration data is sampled at 100Hz while blood pressure data is sampled at 1Hz—related techniques typically involve simple interpolation of asynchronous data. This can easily lead to significant temporal correlation errors between data from different dimensions, affecting the analysis results. In other words, the multimodal data fusion accuracy of smart wearable devices is insufficient in these techniques, resulting in low accuracy in data analysis results. Summary of the Invention

[0004] This application provides a method, apparatus, and smart wearable device for multimodal physiological data fusion analysis, which can solve the problem of insufficient multimodal data fusion accuracy in related technologies, resulting in low accuracy of data analysis results.

[0005] In a first aspect, embodiments of this application provide a method for multimodal physiological data fusion analysis. The method is applied to a multimodal physiological data fusion analysis system, which includes a smart wearable device. The method includes:

[0006] Acquire asynchronous raw body data collected by various sensors in smart wearable devices;

[0007] Based on the data type of each type of raw body data, a corresponding preprocessing method is used to preprocess each type of raw body data to obtain multiple processed data;

[0008] Spatiotemporal alignment and fusion are performed on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector;

[0009] Extracting multiple data features from multi-dimensional fused data vectors;

[0010] Multiple data features are fused to obtain a fused feature vector;

[0011] The data features of different dimensions in the fused feature vector are independently derived, and the corresponding derivation results are obtained respectively.

[0012] A comprehensive analysis of the derivation results is conducted to obtain a data analysis report.

[0013] Secondly, embodiments of this application provide a multimodal physiological data fusion and analysis device. The device is applied to a multimodal physiological data fusion and analysis system, which includes a smart wearable device. The device includes:

[0014] The acquisition module is used to acquire asynchronous raw body data collected by various sensors in the smart wearable device;

[0015] The preprocessing module is used to preprocess each type of raw body data according to its data type, using the corresponding preprocessing method to obtain multiple processed data.

[0016] The spatiotemporal alignment and fusion module is used to perform spatiotemporal alignment and fusion on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector.

[0017] The extraction module is used to extract various data features from multi-dimensional fused data vectors;

[0018] The feature fusion module is used to fuse multiple data features to obtain a fused feature vector.

[0019] The derivation module is used to independently derive the data features of different dimensions in the fused feature vector and obtain the corresponding derivation results.

[0020] The analysis module is used to comprehensively analyze the derivation results and obtain a data analysis report.

[0021] Thirdly, embodiments of this application provide an intelligent wearable device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described multimodal physiological data fusion analysis method.

[0022] Fourthly, embodiments of this application provide a multimodal physiological data fusion analysis system, which includes an intelligent wearable device for performing the steps of the above-described multimodal physiological data fusion analysis method.

[0023] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described multimodal physiological data fusion analysis method.

[0024] Sixthly, embodiments of this application provide a computer program product that, when run on a smart wearable device, causes the smart wearable device to execute the aforementioned multimodal physiological data fusion analysis method.

[0025] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment ensures basic optimization of the quality of various types of data by preprocessing various asynchronous raw body data collected by smart wearable devices according to their data types. Then, the preprocessed data are spatiotemporally aligned and fused to generate a spatiotemporally synchronized multi-dimensional fused data vector, effectively overcoming the timing inaccuracy problem caused by the difference in sampling frequency of different sensors (such as photoelectric sensors, blood pressure sensors, accelerometers, etc.), and significantly reducing the timing correlation error caused by traditional simple interpolation methods. On this basis, multiple data features are extracted and fused to obtain a fused feature vector, ensuring the integrity and consistency of the features on which subsequent analysis depends. Furthermore, the data features of different dimensions in the fused feature vector are independently derived, which can be used to analyze the state of various physiological indicators in a targeted manner. Finally, a comprehensive analysis of the independent derivation results is performed to obtain a data analysis report. Because this report is based on more accurate fused data and more reliable feature analysis, its accuracy and reliability of the comprehensive assessment of the user's health status (such as blood pressure, heart rate, etc.) are significantly improved. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the implementation process of a multimodal physiological data fusion and analysis method provided in an embodiment of this application;

[0028] Figure 2 This is a schematic diagram of the structure of a multimodal physiological data fusion and analysis device provided in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of the structure of the smart wearable device provided in the embodiments of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0031] It should be noted that the terms "comprising," "including," and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application, are intended to cover non-exclusive inclusion. For example, a process, method, terminal, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Terms such as "first" and "second" in the claims, specification, and accompanying drawings of this application, as well as relational terms, are used merely to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such immediate relationship or order between these entities / operations / objects.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] With the integration of IoT and AI technologies, wearable devices such as smartwatches are increasingly being applied to health monitoring. These devices integrate multiple sensors to collect various body data in real time, such as heart rate, blood oxygen saturation, posture, and blood pressure. Compared to traditional medical devices, they offer advantages such as continuous monitoring and non-invasive convenience, making them an effective tool for managing chronic diseases like hypertension and arrhythmia.

[0034] Because different sensors have different sampling frequencies—for example, acceleration data is sampled at 100Hz while blood pressure data is sampled at 1Hz—related techniques typically involve simple interpolation of asynchronous data. This can easily lead to significant temporal correlation errors between data from different dimensions, affecting the analysis results. In other words, the multimodal data fusion accuracy of smart wearable devices is insufficient in these techniques, resulting in low accuracy in data analysis results.

[0035] In view of this, this application embodiment preprocesses various asynchronous raw body data collected by smart wearable devices according to their respective data types, ensuring basic optimization of the quality of various types of data. Then, the preprocessed data are spatiotemporally aligned and fused to generate a spatiotemporally synchronized multi-dimensional fused data vector. This effectively overcomes the timing inaccuracy caused by differences in sampling frequencies between different sensors (such as photoelectric sensors, blood pressure sensors, and accelerometers), significantly reducing the timing correlation errors caused by traditional simple interpolation methods. Based on this, multiple data features are extracted and fused to obtain a fused feature vector, ensuring the integrity and consistency of the features relied upon for subsequent analysis. Furthermore, the data features of different dimensions in the fused feature vector are independently derived, allowing for targeted analysis of the state of various physiological indicators. Finally, a comprehensive analysis of the independently derived results yields a data analysis report. Because this report is based on more accurate fused data and more reliable feature analysis, its accuracy and reliability regarding the user's health status (such as a comprehensive assessment of indicators like blood pressure and heart rate) are significantly improved.

[0036] To illustrate the technical solution of this application, specific embodiments are described below.

[0037] Figure 1 The illustration shows a schematic diagram of the implementation process of a multimodal physiological data fusion analysis method provided in an embodiment of this application. This method can be applied to smart wearable devices in a multimodal physiological data fusion analysis system.

[0038] Specifically, the above-mentioned multimodal physiological data fusion analysis method may include the following steps S101 to S107.

[0039] Step S101: Obtain asynchronous raw body data collected by each sensor in the smart wearable device.

[0040] Asynchronous raw body data refers to physiological signal data collected by different sensors at different sampling frequencies. For example, acceleration data is collected at a frequency of 100Hz, and blood pressure data is collected at a frequency of 1Hz. The data timestamps are not strictly aligned. Asynchronous raw body data can include timestamps and sensor identifiers.

[0041] In the embodiments of this application, the smart wearable device can collect physiological signals in real time through the multimodal sensors built into the smart wearable device, such as an accelerometer recording movement posture at a frequency of 100Hz and a blood pressure sensor recording systolic / diastolic blood pressure at a frequency of 1Hz, so as to obtain multi-dimensional physiological information and provide a raw data basis for subsequent processing.

[0042] Step S102: Based on the data type of each type of raw body data, preprocess each type of raw body data using a corresponding preprocessing method to obtain multiple processed data.

[0043] In the embodiments of this application, the smart wearable device can perform targeted processing for each data type. For example, it can apply a Kalman filter to acceleration data to eliminate motion interference, use linear interpolation to fill in missing values ​​for blood pressure data, and normalize all data to eliminate dimensional differences, thereby improving data quality and reducing the impact of noise and dimensional inconsistencies on subsequent analysis.

[0044] Step S103: Perform spatiotemporal alignment and fusion on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector.

[0045] In the embodiments of this application, the smart wearable device can use the highest frequency data (such as 100Hz acceleration data) as a reference, and align the low frequency data (such as 1Hz blood pressure data) to a unified time axis through timestamp matching and interpolation algorithms. At the same time, it can eliminate positional differences through spatial registration (such as sensor coordinate system transformation), thereby eliminating the timing error between asynchronous data and constructing a spatiotemporally consistent multimodal dataset.

[0046] Step S104: Extract various data features from the multi-dimensional fused data vector.

[0047] In the embodiments of this application, the smart wearable device can extract PPG signal features, motion features and blood pressure features from the fused data vector to generate a feature matrix, thereby compressing the data dimension and retaining key health-related information.

[0048] Step S105: Perform feature fusion on the various data features to obtain a fused feature vector.

[0049] In the embodiments of this application, smart wearable devices may use weighted fusion, deep learning models (such as autoencoders) or other methods to map multi-dimensional features to low-dimensional fused feature vectors in order to integrate multimodal information and improve feature representation capabilities.

[0050] Step S106: Independently derive the data features of different dimensions in the fused feature vector to obtain the corresponding derivation results.

[0051] In the embodiments of this application, the smart wearable device can apply dedicated algorithms to different dimensions of the fused feature vector, such as using support vector machine (SVM) to deduce the risk of arrhythmia and using long short-term memory network (LSTM) to predict blood pressure trends, so as to perform refined analysis for different health indicators.

[0052] Step S107: Perform a comprehensive analysis on the derivation results to obtain a data analysis report.

[0053] In the embodiments of this application, the smart wearable device can input each independent derivation result (such as the probability of arrhythmia and the predicted value of blood pressure) into a decision tree or rule engine, and can also combine the user's historical health records to generate a data analysis report containing abnormal warnings and intervention suggestions, thereby providing interpretable health management decision support.

[0054] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment ensures basic optimization of the quality of various types of data by preprocessing various asynchronous raw body data collected by smart wearable devices according to their data types. Then, the preprocessed data are spatiotemporally aligned and fused to generate a spatiotemporally synchronized multi-dimensional fused data vector, effectively overcoming the timing inaccuracy problem caused by the difference in sampling frequency of different sensors (such as photoelectric sensors, blood pressure sensors, accelerometers, etc.), and significantly reducing the timing correlation error caused by traditional simple interpolation methods. On this basis, multiple data features are extracted and fused to obtain a fused feature vector, ensuring the integrity and consistency of the features on which subsequent analysis depends. Furthermore, the data features of different dimensions in the fused feature vector are independently derived, which can be used to analyze the state of various physiological indicators in a targeted manner. Finally, a comprehensive analysis of the independent derivation results is performed to obtain a data analysis report. Because this report is based on more accurate fused data and more reliable feature analysis, its accuracy and reliability of the comprehensive assessment of the user's health status (such as blood pressure, heart rate, etc.) are significantly improved.

[0055] In some specific implementations, the step of preprocessing each type of raw body data according to its data type to obtain multiple processed data may include steps S401 to S403.

[0056] The asynchronous raw body data includes: PPG signal, acceleration data, and blood pressure data.

[0057] PPG signal stands for photoplethysmography signal, which is a physiological signal obtained by detecting changes in blood volume in skin tissue using photoelectric sensors. It is commonly used for monitoring heart rate and blood oxygen saturation.

[0058] Acceleration data is triaxial motion acceleration data collected by accelerometer sensors, which can reflect the wearer's motion posture and activity intensity.

[0059] Blood pressure data, measured by barometric or optical sensors, consists of systolic and diastolic pressure values, which characterize the stress state of the cardiovascular system.

[0060] Step S401: The PPG signal is denoised using a bandpass filter to obtain the first processed data.

[0061] In the embodiments of this application, the smart wearable device can acquire the raw PPG signal from the photoelectric sensor, design a bandpass filter of 0.5Hz to 10Hz, retain the heart rate-related frequency components (typically 0.8Hz to 4Hz corresponding to a heart rate range of 48-240 bpm), filter out baseline drift (<0.5Hz) and electromyographic interference (>10Hz), and obtain the first processed data after noise reduction. This can effectively eliminate motion artifacts and power frequency interference, and improve the accuracy of heart rate and blood oxygen calculation.

[0062] Step S402: The acceleration data is processed using a Kalman filter to eliminate motion interference, resulting in second processed data.

[0063] In the embodiments of this application, the smart wearable device can acquire raw acceleration data from a triaxial accelerometer, establish a motion interference model, predict the device's motion trajectory and correct the sensor measurements through a Kalman filter, eliminate signal distortion caused by device shaking, and obtain smoothed second-processed data, thereby suppressing the influence of motion artifacts on heart rate variability analysis and improving data reliability in sports scenarios.

[0064] Step S403: Perform sliding window normalization on the blood pressure data to obtain the third processed data.

[0065] In the embodiments of this application, the smart wearable device can acquire the raw blood pressure data sequence from the barometric pressure sensor, set a 60-second sliding window, calculate the mean and standard deviation of the blood pressure values ​​within the window, normalize each data point, and generate third-processed data with uniform dimensions in order to eliminate individual blood pressure baseline differences, enhance the comparability between data from different users, and provide standardized input for subsequent cross-modal fusion.

[0066] This application's implementation significantly improves the quality of multimodal physiological data by employing targeted preprocessing techniques for PPG signals, acceleration data, and blood pressure data. Specifically, bandpass filtering effectively removes noise interference from PPG signals, Kalman filtering suppresses motion artifacts in acceleration data, and sliding window normalization eliminates dimensional differences in blood pressure data. These steps collectively construct a high-quality data foundation, providing a reliable guarantee for subsequent spatiotemporal alignment fusion and feature analysis, ultimately improving the accuracy and robustness of data analysis in chronic disease management scenarios.

[0067] In some specific implementations, the step of performing spatiotemporal alignment and fusion on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector may include steps S501 to S503.

[0068] Step S501: Using the highest sampling frequency data in the processed data as the reference clock, perform linear interpolation resampling on the low-frequency processed data in the processed data.

[0069] The reference clock is a unified time reference constructed from the timestamp sequence of the highest sampling frequency data, used to synchronize the time axis of other low-frequency data.

[0070] In embodiments of this application, the smart wearable device can identify the signal with the highest sampling frequency in the processed data (such as 100Hz acceleration data), use its timestamp sequence as a reference clock, and perform linear interpolation between adjacent sampling points on low-frequency data (such as 1Hz blood pressure data) to generate a resampling sequence aligned with the reference clock. For example, 99 estimated values ​​are inserted into the blood pressure data at intervals of one second to make its time resolution consistent with the acceleration data, so as to eliminate timing errors between asynchronous data and provide a basis for time alignment for subsequent fusion.

[0071] Step S502: Align the acceleration coordinate system with the PPG optical plane using a quaternion rotation matrix to eliminate spatial offset.

[0072] Among them, the quaternion rotation matrix is ​​a three-dimensional coordinate system transformation matrix constructed based on quaternion algebra, which is used to eliminate spatial attitude differences between different sensors.

[0073] In the embodiments of this application, the smart wearable device can use accelerometer data to calculate the device's attitude quaternion, construct a rotation matrix from the acceleration coordinate system to the PPG sensor's optical plane, and project the three-axis acceleration components onto the same spatial reference frame as the PPG signal. For example, when the device is worn on the wrist, quaternion correction eliminates the acceleration direction deviation caused by wrist rotation, ensuring that the X / Y / Z axes are strictly aligned with the PPG transmission-reception direction. This eliminates the impact of spatial attitude differences on data correlation analysis and improves the accuracy of data fusion in motion scenarios.

[0074] Step S503: The processed data after spatiotemporal synchronization is superimposed according to the time point vector to generate the multi-dimensional fused data vector.

[0075] In the embodiments of this application, the smart wearable device can concatenate resampled and spatially aligned multimodal data (such as PPG waveforms, acceleration components, and blood pressure values) into a high-dimensional vector at the same time points. For example, at time t0, a vector [PPG0, AccX0, AccY0, AccZ0, BP0] is generated, forming a time-space-modal three-dimensional aligned data structure. This can convert heterogeneous data into a unified representation, providing structured input for subsequent feature extraction and fusion analysis.

[0076] This application's implementation achieves strict spatiotemporal alignment of multimodal data through linear interpolation resampling and quaternion rotation matrices, significantly improving the accuracy of data fusion. Specifically, reference clock synchronization eliminates timing errors, spatial coordinate system alignment resolves correlation deviations caused by pose, and vector superposition constructs a structured fusion dataset. Together, these measures reduce the temporal correlation errors and spatial registration deviations in asynchronous multimodal data fusion, providing a higher-quality data foundation for subsequent feature fusion and health analysis, ultimately improving the accuracy and reliability of data analysis in chronic disease monitoring scenarios.

[0077] In some specific implementations, the above-mentioned feature fusion of multiple data features to obtain a fused feature vector may further include steps S601 to S605.

[0078] The data features include: PPG signal features, motion features, and blood pressure features.

[0079] PG signal features are physiological parameters extracted from photoplethysmography signals, which may include time-domain / frequency-domain features such as heart rate (HR), blood oxygen saturation (SpO2), and pulse wave conduction time (PTT).

[0080] Motion features are motion pattern parameters extracted from acceleration data, which may include statistical features such as gait cycle, activity intensity, and frequency of postural changes.

[0081] Blood pressure characteristics are pressure change parameters extracted from blood pressure data, which may include time-series features such as mean systolic blood pressure (SBP), mean diastolic blood pressure (DBP), and diurnal rhythm fluctuation amplitude.

[0082] Step S601: Combine PPG signal features, motion features, and blood pressure features according to time series to construct a joint feature matrix.

[0083] In the embodiments of this application, the smart wearable device can align PPG signal features (such as HR = 72 bpm), motion features (such as gait cycle = 1.2 s), and blood pressure features (such as SBP = 120 mmHg) by timestamps, and concatenate them into a matrix by column to form an N×3 matrix (N is the number of time points), thereby integrating multi-dimensional physiological information and providing structured input for subsequent fusion analysis.

[0084] Step S602: Perform PCA transformation on the joint feature matrix and perform principal component analysis to reduce dimensionality, thereby obtaining low-dimensional feature vectors.

[0085] In the embodiments of this application, the smart wearable device can perform zero-mean standardization on the joint feature matrix, calculate the covariance matrix and its eigenvalue decomposition, select the principal components with a cumulative variance contribution rate >95% (such as the first two components), project the original 3D data into a 2D space, and generate low-dimensional feature vectors in order to eliminate feature redundancy, reduce computational complexity, and at the same time retain the main physiological change patterns.

[0086] Step S603: Obtain user profile data.

[0087] In the embodiments of this application, smart wearable devices can extract structured information such as age (e.g., 45 years old), gender (male), and medical history (grade 2 hypertension) from health record databases or user questionnaires, and encode it into vector form, thereby providing basic data for personalized feature optimization.

[0088] Step S604: Based on the user profile data, optimize the low-dimensional feature vector through linear discriminant analysis to obtain the optimized feature vector.

[0089] In the embodiments of this application, the smart wearable device can use health tags (such as whether arrhythmia has occurred) in the user profile data as supervision information to calculate the inter-class scatter matrix and intra-class scatter matrix, solve for the optimal projection direction, map the 2D vector obtained by PCA to the 1D discriminant space, and generate an optimized feature vector, thereby enhancing the ability of the features to express the user's specific physiological patterns.

[0090] Step S605: Generate the fused feature vector based on the optimized feature vector.

[0091] In the embodiments of this application, the smart wearable device can associate the optimized feature vector with timestamps and user IDs to construct a comprehensive feature representation containing spatiotemporal-user-modal information, which can then be used as input for a subsequent health analysis model.

[0092] This application's implementation significantly improves the personalization and accuracy of physiological data analysis through multimodal feature fusion and user profile-driven optimization. Specifically, the joint feature matrix integrates multi-dimensional information such as cardiovascular, exercise, and blood pressure; PCA dimensionality reduction removes redundant noise; and user profile-guided linear discriminant analysis enhances the adaptability of features to individual physiological patterns. Together, they construct a fusion feature vector that is both universal and personalized, effectively supporting early warning and precise management of chronic diseases.

[0093] In some more specific embodiments of this application, the step of independently deriving the data features of different dimensions in the fused feature vector to obtain the corresponding derivation results may specifically include steps S701 to S704.

[0094] Step S701: The fused feature vector is split into a motion-related feature subset, a blood pressure-related feature subset, and a PPG-related feature subset.

[0095] In the embodiments of this application, the smart wearable device can classify the fused feature vector (such as a vector containing heart rate, gait cycle, and systolic blood pressure) according to the feature type label, and extract motion-related features (such as gait cycle = 1.2s), blood pressure-related features (such as systolic blood pressure = 120mmHg), and PPG-related features (such as heart rate = 72bpm) to form independent subsets, so as to improve the targeting of abnormality detection by subsequent modular analysis according to physiological systems.

[0096] Step S702: Based on the user profile data, calculate the personalized warning threshold for each feature subset.

[0097] In the embodiments of this application, the smart wearable device can generate dynamic thresholds for each feature subset (such as exercise intensity threshold = 5 MET, blood pressure fluctuation threshold = 20 mmHg) based on user profile data (such as a 45-year-old male hypertensive patient) using statistical models (such as age-corrected percentile method) or machine learning models (such as random forest regression), thereby establishing an individualized health benchmark.

[0098] Step S703: Based on the personalized warning threshold, execute the corresponding medical rule engine on the exercise-related feature subset, the blood pressure-related feature subset, and the PPG-related feature subset respectively to obtain the corresponding abnormal markers.

[0099] In the embodiments of this application, the smart wearable device can input each feature subset into the corresponding rule engine. For example, it can apply the rule "gait cycle variation coefficient > 30% indicates fall risk" to the motion feature subset and the rule "nocturnal systolic blood pressure drop rate < 10% indicates non-dipper hypertension" to the blood pressure feature subset, and output binary abnormality markers (0 = normal, 1 = abnormal), thereby realizing automated abnormality recognition based on medical knowledge.

[0100] Step S704: Generate the corresponding derivation result based on the anomaly marker.

[0101] In the embodiments of this application, the smart wearable device can aggregate the abnormal markers of each subset (such as exercise marker = 1, blood pressure marker = 0, PPG marker = 1), and combine them with a medical knowledge graph to generate structured derivation results, thereby transforming the underlying data anomalies into clinically interpretable health advice.

[0102] This application's implementation significantly improves the accuracy and clinical applicability of multimodal physiological data analysis through modular feature decomposition and personalized threshold calculation. Specifically, independent analysis of exercise / blood pressure / PPG feature subsets can accurately pinpoint the source of anomalies, user profile-driven dynamic thresholds avoid false positives and false negatives, and a medical rule engine ensures the medical interpretability of the conclusions. Together, they construct a complete link from data to clinical decision-making, enabling earlier detection of early signs of chronic diseases such as hypertension and arrhythmia, providing a reliable basis for personalized health management, and ultimately enhancing the medical value of smart wearable devices.

[0103] In some more specific embodiments of this application, the step of calculating the personalized warning threshold for each feature subset based on the user profile data may specifically include steps S801 to S804.

[0104] Step S801: Extract age values ​​and medical history lists from the user profile data.

[0105] In the embodiments of this application, the smart wearable device can parse the user's health record database or questionnaire data, extract the age field (e.g., 45 years old) and medical history entries (e.g., "5-year history of hypertension, taking amlodipine"), and form structured data tuples, thereby providing basic input parameters for personalized threshold calculation.

[0106] Step S802: Calculate the motion threshold based on the medical history list.

[0107] In the embodiments of this application, the smart wearable device can traverse the disease entries in the medical history list, match them with a predefined exercise contraindication rule base (such as "the maximum heart rate threshold corresponding to the history of heart disease = 120 - age"), calculate the upper limit of the allowable range of the exercise-related feature subset (such as the maximum safe gait cycle variation coefficient = 25%), thereby establishing a sports risk protection boundary based on medical history.

[0108] Step S803: Calculate the blood pressure threshold based on the age value.

[0109] In the embodiments of this application, the smart wearable device can use an age correction formula (such as upper limit of systolic blood pressure = 90 + age × 0.6, upper limit of diastolic blood pressure = 60 + age × 0.4) to calculate the normal range of blood pressure. For example, the systolic blood pressure threshold for a 45-year-old user is 90 + 45 × 0.6 = 117 mmHg, thereby establishing a blood pressure assessment standard that conforms to the physiological decay law.

[0110] Step S804: Calculate the PPG threshold based on the age value.

[0111] In the embodiments of this application, the smart wearable device can apply an age-related physiological model (such as upper limit of heart rate = 220 - age, lower limit of blood oxygen saturation = 95% - 0.1 × age) to generate PPG characteristic thresholds, for example, the upper limit of heart rate for a 45-year-old user = 220 - 45 = 175 bpm, thereby establishing a pulse wave assessment benchmark that adapts to age changes.

[0112] This application's implementation significantly improves the accuracy and clinical applicability of multimodal physiological data analysis by dynamically calculating personalized early warning thresholds. Specifically, history-driven exercise thresholds avoid over-recommending exercise to patients with contraindications, and age-corrected blood pressure / PPG thresholds better align with physiological decline patterns. This application's implementation effectively solves the false alarm and missed alarm problems caused by traditional static thresholds, enabling more accurate identification of early signs of chronic diseases such as hypertension and arrhythmia.

[0113] In some specific embodiments of this application, the multimodal physiological data fusion analysis system further includes a cloud server, and after the derivation results are comprehensively analyzed to obtain a data analysis report, it may also include steps S901 to S904.

[0114] Step S901: Upload the data analysis report to the cloud server in real time.

[0115] In the embodiments of this application, smart wearable devices can encrypt and transmit locally generated structured reports (including health scores, anomaly markers, and recommended measures) to a cloud-based distributed storage system via the HTTPS protocol. For example, a JSON-formatted report can be uploaded every 5 minutes, and AES-256 encryption is used to ensure data security, thereby establishing a cloud backup of the user's health record and providing a data foundation for long-term trend analysis.

[0116] Step S902: Call the pre-trained time-series health model in the cloud server to perform trend analysis on N consecutive reports.

[0117] In the embodiments of this application, the smart wearable device can read the user's historical reports from the past 72 hours (N=864 reports) from a cloud database and input them into a pre-trained LSTM time series model. The model outputs 12 trend indicators, including the rate of blood pressure rise and the decay trend of heart rate variability, such as detecting an abnormal trend of "systolic blood pressure rising by 0.8 mmHg per hour in the past 24 hours." This can reveal progressive health risks that are difficult to detect in a single report.

[0118] Step S903: When a cross-reporting anomaly pattern is detected, a collaborative early warning instruction is generated.

[0119] In the embodiments of this application, when the trend index output by the time series model exceeds the dynamic threshold (e.g., blood pressure rise rate > 0.5 mmHg / h), the smart wearable device can trigger the abnormal pattern detection engine. The engine matches a predefined rule base (e.g., "sustained rise in blood pressure + abnormal PPG waveform = atrial fibrillation precursor") and generates an early warning instruction that includes the abnormality type (arrhythmia risk), urgency level (high), and treatment suggestion (seek immediate medical attention), thereby achieving early risk warning and avoiding the occurrence of acute events.

[0120] Step S904: Select the push terminal combination according to the warning instruction level, adapt the push format according to the terminal type, and push the warning information to the corresponding terminal.

[0121] In the embodiments of this application, the smart wearable device can parse the urgency level (such as high priority) in the warning command, select the push terminal combination (such as user mobile phone, family member tablet, community hospital terminal), call the terminal adaptation engine to convert the unified warning content into different formats (such as mobile phone receiving short text + vibration reminder, hospital terminal receiving detailed report + handling flowchart), and realize multi-channel synchronous push through cloud messaging service (such as Firebase Cloud Messaging) to ensure that key information reaches relevant parties in a timely manner and improve emergency response efficiency.

[0122] This application's implementation significantly enhances the risk prediction capabilities and emergency response levels of health monitoring systems through cloud-based collaborative time-series analysis and multi-terminal early warning mechanisms. Specifically, real-time report uploads construct complete health records, time-series models reveal progressive risks, abnormal pattern detection enables early warnings, and terminal-adaptive push notifications ensure information delivery. Together, these form a complete closed loop from data collection and analysis to emergency response, enabling earlier detection of acute events such as hypertensive crises and malignant arrhythmias, providing key technical support for chronic disease management and emergency scenarios.

[0123] This application provides a multimodal physiological data fusion and analysis system, which includes an intelligent wearable device. The multimodal physiological data fusion and analysis system is used for:

[0124] Acquire asynchronous raw body data collected by each sensor in the smart wearable device;

[0125] Based on the data type of each type of raw body data, each type of raw body data is preprocessed using a corresponding preprocessing method to obtain multiple processed data;

[0126] Each type of processed data is spatiotemporally aligned and fused to generate a spatiotemporally synchronized multi-dimensional fused data vector.

[0127] Multiple data features are extracted from the multi-dimensional fused data vector;

[0128] The various data features are fused to obtain a fused feature vector;

[0129] The data features of different dimensions in the fused feature vector are independently derived, and the corresponding derivation results are obtained respectively.

[0130] A comprehensive analysis of the derivation results yields a data analysis report.

[0131] Figure 2 This illustration shows a schematic diagram of a multimodal physiological data fusion analysis device according to an embodiment of this application. The device is applied to a multimodal physiological data fusion analysis system, which includes a smart wearable device. The multimodal physiological data fusion analysis device 2 can be configured on the smart wearable device. Specifically, the multimodal physiological data fusion analysis device 2 may include:

[0132] The acquisition module 201 is used to acquire asynchronous raw body data collected by each sensor in the smart wearable device;

[0133] Preprocessing module 202 is used to preprocess each type of raw body data according to the data type of each type of raw body data using a corresponding preprocessing method to obtain multiple processed data;

[0134] The spatiotemporal alignment and fusion module 203 is used to perform spatiotemporal alignment and fusion on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector.

[0135] Extraction module 204 is used to extract multiple data features from the multi-dimensional fused data vector;

[0136] Feature fusion module 205 is used to fuse multiple data features to obtain a fused feature vector;

[0137] The derivation module 206 is used to independently derive the data features of different dimensions in the fused feature vector and obtain the corresponding derivation results respectively;

[0138] Analysis module 207 is used to perform comprehensive analysis on the derivation results and obtain a data analysis report.

[0139] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment ensures basic optimization of the quality of various types of data by preprocessing various asynchronous raw body data collected by smart wearable devices according to their data types. Then, the preprocessed data are spatiotemporally aligned and fused to generate a spatiotemporally synchronized multi-dimensional fused data vector, effectively overcoming the timing inaccuracy problem caused by the difference in sampling frequency of different sensors (such as photoelectric sensors, blood pressure sensors, accelerometers, etc.), and significantly reducing the timing correlation error caused by traditional simple interpolation methods. On this basis, multiple data features are extracted and fused to obtain a fused feature vector, ensuring the integrity and consistency of the features on which subsequent analysis depends. Furthermore, the data features of different dimensions in the fused feature vector are independently derived, which can be used to analyze the state of various physiological indicators in a targeted manner. Finally, a comprehensive analysis of the independent derivation results is performed to obtain a data analysis report. Because this report is based on more accurate fused data and more reliable feature analysis, its accuracy and reliability of the comprehensive assessment of the user's health status (such as blood pressure, heart rate, etc.) are significantly improved.

[0140] In some embodiments of this application, the asynchronous raw body data includes: PPG signal, acceleration data, and blood pressure data. The preprocessing module 202 is further configured to: perform noise reduction processing on the PPG signal using a bandpass filter to obtain first processed data; perform motion interference elimination on the acceleration data using a Kalman filter to obtain second processed data; and perform sliding window normalization processing on the blood pressure data to obtain third processed data.

[0141] In some embodiments of this application, the spatiotemporal alignment and fusion module 203 is further configured to: use the highest sampling frequency data in the processed data as a reference clock to perform linear interpolation resampling on the low-frequency processed data in the processed data; align the acceleration coordinate system with the PPG optical plane through a quaternion rotation matrix to eliminate spatial offset; and superimpose the spatiotemporally synchronized processed data according to time point vectors to generate the multi-dimensional fused data vector.

[0142] In some embodiments of this application, the data features include: PPG signal features, motion features, and blood pressure features. The feature fusion module 205 is further configured to: combine the PPG signal features, motion features, and blood pressure features according to a time series to construct a joint feature matrix; perform PCA transformation on the joint feature matrix and perform principal component analysis to reduce dimensionality, obtaining a low-dimensional feature vector; acquire user profile data; optimize the low-dimensional feature vector based on the user profile data through linear discriminant analysis to obtain an optimized feature vector; and generate the fused feature vector based on the optimized feature vector.

[0143] In some embodiments of this application, the derivation module 206 is further configured to: split the fused feature vector into a subset of motion-related features, a subset of blood pressure-related features, and a subset of PPG-related features; calculate a personalized warning threshold for each feature subset based on the user profile data; execute the corresponding medical rule engine on the subset of motion-related features, the subset of blood pressure-related features, and the subset of PPG-related features according to the personalized warning threshold to obtain corresponding anomaly markers; and generate the corresponding derivation result based on the anomaly markers.

[0144] In some embodiments of this application, the derivation module 206 is further configured to: extract age values ​​and medical history lists from the user profile data; calculate exercise thresholds based on the medical history lists; calculate blood pressure thresholds based on the age values; and calculate PPG thresholds based on the age values.

[0145] In some embodiments of this application, the multimodal physiological data fusion analysis system further includes a cloud server, and the analysis module 207 is further configured to: upload the data analysis report to the cloud server in real time; call the pre-trained time-series health model in the cloud server to perform trend analysis on N consecutive reports; generate a collaborative early warning instruction when an abnormal pattern across reports is detected; select a combination of push terminals according to the level of the early warning instruction, and adapt the push format according to the terminal type to push the early warning information to the corresponding terminals.

[0146] like Figure 3 The diagram shown is a schematic of a smart wearable device provided in an embodiment of this application. The smart wearable device 3 may include: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301, such as a multimodal physiological data fusion analysis program. When the processor 301 executes the computer program 303, it implements the steps in the various multimodal physiological data fusion analysis method embodiments described above, for example... Figure 1 Steps S101 to S107 are shown.

[0147] A computer program can be divided into one or more modules / units. One or more modules / units are stored in memory 302 and executed by processor 301 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a smart wearable device.

[0148] Smart wearable devices may include, but are not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3This is merely an example of a smart wearable device and does not constitute a limitation on smart wearable devices. It may include more or fewer components than shown, or combine certain components, or different components. For example, a smart wearable device may also include input / output devices, network access devices, buses, etc.

[0149] The processor 301 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0150] The memory 302 can be an internal storage unit of the smart wearable device, such as a hard drive or RAM. The memory 302 can also be an external storage device of the smart wearable device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 302 can include both internal and external storage units. The memory 302 is used to store computer programs and other programs and data required by the smart wearable device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0151] It should be noted that, for the sake of convenience and brevity, the structure of the above-mentioned smart wearable device can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0153] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described multimodal physiological data fusion analysis method.

[0154] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps in the above-mentioned multimodal physiological data fusion analysis method.

[0155] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0156] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0157] In the embodiments provided in this application, it should be understood that the disclosed devices / smart wearable devices and methods can be implemented in other ways. For example, the device / smart wearable device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0161] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for multimodal physiological data fusion and analysis, characterized in that, The method is applied to a multimodal physiological data fusion and analysis system, which includes a smart wearable device, and the method includes: Acquire asynchronous raw body data collected by each sensor in the smart wearable device; Based on the data type of each type of raw body data, each type of raw body data is preprocessed using a corresponding preprocessing method to obtain multiple processed data; Each type of processed data is spatiotemporally aligned and fused to generate a spatiotemporally synchronized multi-dimensional fused data vector. Multiple data features are extracted from the multi-dimensional fused data vector; The various data features are fused to obtain a fused feature vector; The data features of different dimensions in the fused feature vector are independently derived, and the corresponding derivation results are obtained respectively. A comprehensive analysis of the derivation results yields a data analysis report.

2. The multimodal physiological data fusion and analysis method as described in claim 1, characterized in that, The asynchronous raw body data includes: PPG signals, acceleration data, and blood pressure data. Based on the data type of each type of raw body data, a corresponding preprocessing method is applied to each type of raw body data to obtain various processed data, including: The PPG signal is denoised using a bandpass filter to obtain the first processed data; The acceleration data is processed using a Kalman filter to eliminate motion interference, resulting in second processed data. The blood pressure data is normalized using a sliding window to obtain the third processed data.

3. The multimodal physiological data fusion and analysis method as described in claim 1, characterized in that, The step of performing spatiotemporal alignment and fusion on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector includes: Using the highest sampling frequency data in the processed data as a reference clock, the low frequency processed data in the processed data is linearly interpolated and resampled. Spatial offset is eliminated by aligning the acceleration coordinate system with the PPG optical plane using a quaternion rotation matrix. The processed data after spatiotemporal synchronization is superimposed according to time point vectors to generate the multi-dimensional fused data vector.

4. The multimodal physiological data fusion and analysis method as described in claim 1, characterized in that, The data features include: PPG signal features, motion features, and blood pressure features. The process of fusing multiple data features to obtain a fused feature vector includes: The PPG signal features, motion features, and blood pressure features are combined according to time series to construct a joint feature matrix; Perform PCA transformation on the joint feature matrix, then perform principal component analysis to reduce dimensionality and obtain low-dimensional feature vectors. Obtain user profile data; Based on the user profile data, the low-dimensional feature vector is optimized through linear discriminant analysis to obtain the optimized feature vector; The fused feature vector is generated based on the optimized feature vector.

5. The multimodal physiological data fusion and analysis method as described in claim 4, characterized in that, The step of independently deriving data features of different dimensions in the fused feature vector to obtain corresponding derivation results includes: The fused feature vector is split into a subset of motion-related features, a subset of blood pressure-related features, and a subset of PPG-related features; Based on the user profile data, the personalized early warning threshold for each feature subset is calculated respectively; Based on the personalized early warning threshold, the corresponding medical rule engine is executed on the exercise-related feature subset, the blood pressure-related feature subset, and the PPG-related feature subset to obtain the corresponding abnormal markers; The corresponding derivation result is generated based on the anomaly marker.

6. The multimodal physiological data fusion and analysis method as described in claim 5, characterized in that, The process of calculating personalized early warning thresholds for each feature subset based on the user profile data includes: Extract age values ​​and medical history lists from the user profile data; Calculate the movement threshold based on the medical history list; Calculate the blood pressure threshold based on the stated age value; The PPG threshold is calculated based on the stated age value.

7. The multimodal physiological data fusion and analysis method as described in claim 1, characterized in that, The multimodal physiological data fusion and analysis system also includes a cloud server. After comprehensively analyzing the derivation results and obtaining a data analysis report, the method further includes: The data analysis report is uploaded to the cloud server in real time. Call the pre-trained time-series health model in the cloud server to perform trend analysis on N consecutive reports; When a cross-reporting anomaly pattern is detected, a collaborative early warning instruction is generated; Based on the warning instruction level, select the push terminal combination, adapt the push format according to the terminal type, and push the warning information to the corresponding terminals.

8. A multimodal physiological data fusion and analysis device, characterized in that, The device is applied to a multimodal physiological data fusion and analysis system, which includes a smart wearable device. The device comprises: The acquisition module is used to acquire asynchronous raw body data collected by various sensors in the smart wearable device; The preprocessing module is used to preprocess each type of raw body data according to its data type, using a corresponding preprocessing method to obtain multiple processed data. The spatiotemporal alignment and fusion module is used to perform spatiotemporal alignment and fusion on each type of processed data to generate a spatiotemporally synchronized multi-dimensional fused data vector. The extraction module is used to extract various data features from the multi-dimensional fused data vector; The feature fusion module is used to fuse multiple data features to obtain a fused feature vector. The derivation module is used to independently derive the data features of different dimensions in the fused feature vector and obtain the corresponding derivation results respectively; The analysis module is used to perform comprehensive analysis on the derivation results and obtain a data analysis report.

9. A smart wearable device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multimodal physiological data fusion analysis method as described in any one of claims 1 to 7.

10. A multimodal physiological data fusion and analysis system, characterized in that, The multimodal physiological data fusion analysis system includes a smart wearable device, which is used to perform the steps of the multimodal physiological data fusion analysis method as described in any one of claims 1 to 7.

Images