A health early warning method and system based on multi-modal data and a storage medium

Abstract

The present application belongs to the field of intelligent health monitoring and digital technology of traditional Chinese medicine, and particularly relates to a health early warning method and system based on multi-modal data and a storage medium. The method comprises the following steps: obtaining pulse feature data collected by a wearable device, emotion feature data collected by a voice interaction device, and behavior and posture feature data collected by an environment or body posture sensor; inputting the pulse feature, emotion feature and behavior and posture feature into a traditional Chinese medicine fusion model; based on a preset dynamic weight distribution strategy, combining the historical physical characteristics and real-time state of the user, calculating the fusion weight coefficient of each feature data in the current application scenario; generating a comprehensive health risk assessment result according to the fusion weight coefficient; and when the assessment result meets a preset early warning condition, generating an early warning signal of the corresponding level. The core advantage of the present application is that the dialectical thinking of traditional Chinese medicine "treating pulse according to syndrome and treating syndrome according to pulse" is converted into a calculable dynamic weight algorithm, solving the technical problems of high misjudgment rate of single data source in traditional health monitoring and inability to make decisions when subjective feelings and objective signs conflict, and realizing the leap from "passive alarm" to "active prevention of disease".

Description

Technical Field

[0001] This invention relates to the field of intelligent health monitoring technology, specifically to a health early warning method, system, and storage medium based on multimodal data. Background Technology

[0002] With the increasing aging of the population, the demand for health monitoring and safety protection for elderly people living alone is growing. Existing health monitoring solutions are usually based on analysis and judgment based on a single data source: such as physiological parameter monitoring based on wearable devices (heart rate, blood oxygen, blood pressure, etc.), semantic analysis based on voice interaction devices, or behavioral posture analysis based on sensor detection.

[0003] However, the existing technology has the following shortcomings: (1) The reliability of analysis based on a single data source is limited. Although physiological parameter monitoring is objective and true, an abnormality in a single indicator does not necessarily represent a health risk; although speech and semantic analysis can capture subjective feelings, it may have a tendency to conceal the good news and not the bad; although behavioral posture analysis can reflect the activity state, it may produce misjudgments due to sensor drift or improper wearing position.

[0004] (2) Multi-source data fusion schemes lack dynamic adjustment mechanisms. In existing technologies, even when multi-source data fusion is used, fixed or static weights are often employed, which cannot adaptively adjust the confidence levels of each data source according to the user's real-time status. For example, when a user verbally expresses "no discomfort" but physiological indicators are significantly abnormal, the static weight scheme cannot automatically increase the weight of physiological indicators to capture hidden risks.

[0005] (3) Lack of deep integration with the TCM syndrome differentiation system. Most existing health early warning programs are driven by Western medicine indicators and fail to use the TCM constitution identification results as a dynamic regulating factor for health risk assessment, ignoring the different health risk characteristics of users with different constitution types.

[0006] Therefore, there is an urgent need for a TCM early warning method that can integrate multimodal data and achieve dynamic weight adjustment. Summary of the Invention

[0007] This invention aims to solve the technical problems in existing technologies, such as the limited reliability of single data source analysis, the lack of dynamic adjustment mechanism for multi-source data fusion, and the lack of deep integration with the TCM syndrome differentiation system.

[0008] To achieve the above objectives, this invention provides a health early warning method based on multimodal data, comprising the following steps: acquiring pulse feature data collected by a wearable device; acquiring emotional feature data collected by a voice interaction device; acquiring behavioral posture feature data collected by an environmental or body posture sensor; inputting the pulse features, emotional features, and behavioral posture features into a traditional Chinese medicine fusion model; calculating the fusion weight coefficient of each feature data in the current application scenario based on a preset dynamic weight allocation strategy, combined with the user's historical physical characteristics and real-time status; generating a comprehensive health risk assessment result based on the fusion weight coefficient; and generating an early warning signal when the assessment result meets preset early warning conditions.

[0009] Furthermore, the dynamic weight allocation strategy adaptively adjusts based on the user's real-time status. When the pulse characteristic data shows a significant deviation but the emotional characteristic data is normal, the weight coefficient of the emotional characteristic is reduced, while the weight coefficient of the pulse characteristic is increased to capture the risk of hidden physiological abnormalities. When the emotional characteristic data is abnormal but the pulse characteristic data is normal, behavioral posture characteristic data is used to assist in the judgment, increasing the weight coefficients of both emotional and behavioral posture characteristics to identify the risk of postural symptoms. When the behavioral posture characteristic data shows prolonged stillness and decreased activity in the emotional characteristic data, the behavioral posture characteristic is assigned the highest judgment weight to capture the risk of coma.

[0010] Furthermore, the pulse feature data includes time-domain feature parameters, frequency-domain feature parameters, and waveform morphology feature parameters of the PPG pulse wave signal. The emotional feature data includes speech semantic features and voiceprint features. The behavioral posture feature data includes acceleration data, body posture data, and ambient temperature data.

[0011] Compared with the prior art, the present invention has the following beneficial effects: (1) For the first time, the TCM diagnostic thinking (discarding the syndrome and following the pulse, discarding the pulse and following the syndrome) is transformed into the dynamic weight allocation strategy of the algorithm, realizing the deep integration of TCM theory and artificial intelligence.

[0012] (2) The multimodal data fusion scheme based on dynamic weights solves the technical problems of unreliability of single data sources and lack of adaptive adjustment in multi-source fusion.

[0013] (3) By using the results of TCM constitution identification as a weight adjustment factor, accurate early warning that is "different for each person" is achieved, overcoming the problem of underreporting and false reporting of users with different constitutions by the traditional general threshold. Attached Figure Description

[0014] Figure 1 This is an overall flowchart of the health early warning method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the structure of a health early warning system provided in an embodiment of the present invention. Detailed Implementation

[0015] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Example

[0016] This embodiment provides a health early warning method based on multimodal data.

[0017] Step S1: The user's PPG pulse wave signal is collected by the PPG optical sensor built into the smart ring. The TCM pulse characteristic parameters are extracted from it, including time domain characteristic parameters (main wave height, ascending limb slope, descending limb slope and dicrotic wave position), frequency domain characteristic parameters (low frequency power 0.04-0.15Hz, high frequency power 0.15-0.4Hz and low frequency / high frequency ratio) and waveform morphology characteristic parameters (pulse wave rise time, fall time and peak-to-trough ratio).

[0018] Step S2: Collect daily voice interaction data between the user and the device through the built-in microphone of the smart speaker, and extract emotional feature data, including voice semantic features (the user's main complaint and emotional tendency) and voiceprint features (speech rate changes, pitch fluctuations and breath characteristics).

[0019] Step S3: Collect behavioral posture feature data, including acceleration data, body posture data and ambient temperature data, through the accelerometer and / or environmental sensor built into the smart ring.

[0020] Step S4: Input the pulse characteristics, emotional characteristics, and behavioral posture characteristics into the TCM fusion model; the model calculates the fusion weight coefficient of each feature data in the current application scenario based on a preset dynamic weight allocation strategy and in combination with the user's historical physical characteristics and real-time status.

[0021] Scenario 1: Prioritizing Pulse Diagnosis Over Evidence – Intercepting Hidden Risks. When a user is at rest, the emotional characteristics data collected by the voice interaction device indicate "no discomfort," but the pulse characteristics data (such as PPG waveform) collected by the wearable device show a persistent and severe deviation (e.g., abnormally high systolic blood pressure waveform). In this case, the system determines that the user may have a hidden risk of hypertension, automatically reducing the weighting coefficient of emotional characteristics and increasing the weighting coefficient of pulse characteristics, generating a Level 1 warning signal. This scenario reflects the TCM diagnostic thinking of "prioritizing pulse diagnosis over evidence"—when subjective statements contradict objective pulse findings, the pulse findings are given priority.

[0022] Scenario 2: Prioritizing Symptoms over Pulse Diagnosis – Postural Risk Identification. When a user complains of dizziness and nausea (abnormal emotional and mental characteristics data), but their pulse characteristics are within the normal fluctuation range, the system uses behavioral and postural characteristics data to assist in the judgment. If head movement or rapid changes in posture are detected, the weighting coefficients of emotional and behavioral / postural characteristics are automatically increased, classifying it as postural vertigo and generating a secondary warning signal. This scenario reflects the TCM diagnostic thinking of "prioritizing symptoms over pulse diagnosis"—when the pulse is normal but clinical symptoms are clear, the syndrome and behavioral / postural characteristics are given priority.

[0023] Scenario 3: Escalating Criticality – Coma Risk Detection: When behavioral and postural feature data shows the user is in a prolonged state of stillness, and the activity of the voice interaction device drops sharply (e.g., no response or slurred speech), regardless of whether the pulse feature data is abnormal, the system will increase the weight coefficient of the behavioral and postural features to the highest judgment weight, and combine it with historical TCM constitution identification results (e.g., phlegm-dampness constitution), directly determining a high-risk coma risk, generating a level-three alarm, and preparing to trigger SOS. This scenario employs an "escalating criticality" strategy—when both behavioral and voice data point to a critical state, the behavioral data receives the highest judgment weight.

[0024] Step S5: Generate a comprehensive health risk assessment result based on the fusion weighting coefficient.

[0025] Step S6: When the evaluation result meets the preset warning conditions, a warning signal is generated; the warning signal includes: Level 1 warning: pushed to the user's App; Level 2 warning: pushed to the child / guardian's terminal simultaneously; Level 3 warning: triggers SOS and links the emergency door opening and 120 notification system.

[0026] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the inventive concept of the present invention and the description and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A health early warning method based on multimodal data, characterized in that, Includes the following steps: Acquire pulse feature data collected by wearable devices; Acquire emotional characteristic data collected by voice interaction devices; Acquire behavioral posture feature data collected by environmental or body posture sensors; The pulse characteristics, emotional characteristics, and behavioral posture characteristics are input into the TCM fusion model; Based on a preset dynamic weight allocation strategy, and combined with the user's historical physical characteristics and real-time status, the fusion weight coefficient of each feature data in the current application scenario is calculated. A comprehensive health risk assessment result is generated based on the aforementioned fusion weighting coefficients; When the evaluation results meet the preset early warning conditions, an early warning signal is generated.

2. The method according to claim 1, characterized in that, The dynamic weight allocation strategy adaptively adjusts based on the user's real-time status.

3. The method according to claim 2, characterized in that, When the pulse feature data shows a significant deviation but the emotional feature data is normal, the weight coefficient of the emotional feature is reduced and the weight coefficient of the pulse feature is increased.

4. The method according to claim 2, characterized in that, When the emotional characteristic data is abnormal but the pulse characteristic data is normal, behavioral posture characteristic data is invoked to assist in the judgment, thereby increasing the weight coefficients of emotional and behavioral posture characteristics.

5. The method according to claim 2, characterized in that, When the behavioral posture feature data shows a prolonged period of stillness and a decrease in the activity level of the emotional feature data, the behavioral posture feature is given the highest judgment weight, or the weight coefficient of the behavioral posture feature is increased to above a preset threshold.

6. The method according to claim 1, characterized in that, The pulse feature data includes time-domain feature parameters, frequency-domain feature parameters, and waveform morphology feature parameters of the PPG pulse wave signal.

7. The method according to claim 1, characterized in that, The emotional feature data includes speech semantic features and voiceprint features.

8. The method according to claim 1, characterized in that, The behavioral posture feature data includes acceleration data, body posture data, and ambient temperature data.

9. A health early warning system based on multimodal data, characterized in that, include: Wearable devices used to collect users' pulse characteristics data; Voice interaction devices are used to collect users' emotional and psychological characteristics data; Environmental or body posture sensors are used to collect data on the user's behavioral and posture characteristics. The processor is connected to the wearable device, the voice interaction device, and the sensor, and executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 8.

Images