A blood relationship graph-based multi-modal data intelligent acquisition, storage and analysis system and method

Abstract

This invention discloses a multimodal data intelligent acquisition, storage, and analysis system and method based on kinship graphs, addressing the problems of scattered family health data, unutilized soft data, difficulty in identifying genetic risks, and data silos hindering collaboration. The invention constructs a structured family knowledge base containing family member nodes, kinship edges, and associated multi-source data. It collects physiological indicators, speech patterns, facial images, and psychological characteristic scale data through intelligent hardware, extracts features in real time, and automatically aggregates them to corresponding member nodes based on kinship. A multi-source fusion analysis engine is constructed to achieve health indicator screening, genetic risk identification, emotional state monitoring, and personality trait analysis, generating intergenerational health trend prompts and individual status reminders. The system collaborates with a digital avatar engine to drive real-time evolution of the avatar, forming a complete closed loop of acquisition → storage → analysis → application, providing technical support for family health management, genetic risk early warning, and individual status perception.

Description

Technical Field

[0001] This invention relates to the fields of digital health management, biometric recognition, data mining and artificial intelligence, and in particular to a family health data intelligent management system and method that uses smart hardware devices to collect multimodal data, performs automatic aggregation and fusion analysis based on kinship maps, and is driven in collaboration with a digital avatar engine. Background Technology

[0002] With the widespread adoption of smart hardware devices and the deepening of health management concepts, more and more families are using smart bracelets, blood pressure monitors, blood glucose meters, and other devices to monitor the health of family members. At the same time, voice-interactive devices and psychological testing tools are also gradually entering family life. However, existing technologies have the following technical shortcomings: First, family health data is scattered, making unified management and cross-referencing analysis impossible. Family members' health data is dispersed across different devices and accounts, making it impossible to create a unified family health record. When users want to understand the health trends of an elder or analyze family genetic traits, they can only manually browse through records on various devices, which is inefficient and prone to omissions.

[0003] Second, soft data such as voice tone and psychological characteristics are not being effectively utilized. Existing health management software only focuses on hard physiological indicators such as blood pressure and blood sugar, ignoring soft data such as speech rate, tone variation, and psychological test results. These data also contain rich health information; for example, the degree of voice tremor can reflect anxiety, and changes in speech rate can indicate the risk of cognitive decline, but current technology lacks the ability to collect and analyze this soft data.

[0004] Third, health alerts lack family genetic background information and cannot identify intergenerational risks. Current health alerts are based solely on individual historical data and cannot utilize family genetic information. When a member's blood pressure trend is highly similar to that of their grandfather who already has hypertension, the existing system cannot identify this risk, resulting in a lack of early warning.

[0005] Fourth, personal status reminders are disconnected from family connections and lack emotional support. Current psychological test results are only provided to users once, lacking continuous monitoring and family connection. When a user is detected as being in a low mood, there is no way to connect with family relationships and provide emotional support (such as "suggesting more communication with family members").

[0006] Fifth, the collected data is disconnected from the family knowledge base and digital avatars, failing to form a closed loop. In existing technologies, health data, voice data, and psychological data exist in isolation, unable to be integrated with the family knowledge base, let alone drive the evolution of digital avatars, thus limiting the long-term value of the data.

[0007] In summary, constructing a family health data intelligent management system that can uniformly collect multimodal data, automatically aggregate data based on blood relations, integrate and analyze health trends, and co-evolve with digital avatars has become a pressing technical challenge in this field. Summary of the Invention

[0008] The purpose of this invention is to provide a multimodal data intelligent acquisition, storage and analysis system and method based on kinship mapping, so as to solve the technical problems of scattered family health data, unutilized soft data, difficulty in identifying genetic risks, and lack of collaboration among data silos in the prior art.

[0009] II. Technical Solution In a first aspect, embodiments of the present invention provide a method for intelligent acquisition, storage, and analysis of multimodal data based on kinship maps, comprising the following steps:

[0010] A structured family knowledge base is constructed, comprising family member nodes, blood relationship edges, and related multi-source data. This knowledge base achieves data fusion by constructing an association graph of family member nodes, blood relationship edges, health indicator nodes, psychological characteristic nodes, and event nodes, providing a foundation for subsequent data collection and analysis.

[0011] Step S2: Smart hardware device data collection and identity binding.

[0012] Multimodal data of target members is collected through smart hardware devices, including smart bracelets, smart blood pressure monitors, smart blood glucose meters, smart body fat scales, smart speakers, psychological testing terminals, and smartphone cameras and microphones. Upon first use, the devices bind family member identities via facial recognition or voiceprint recognition, and subsequently collected data is automatically associated with that member.

[0013] Step S3: Real-time multimodal feature extraction.

[0014] Real-time feature extraction of the collected multimodal data: 3.1 Physiological indicator feature extraction: Systolic blood pressure and diastolic blood pressure were extracted from the sphygmomanometer; fasting blood glucose and postprandial blood glucose were extracted from the blood glucose meter; resting heart rate and exercise heart rate were extracted from the heart rate monitoring device; weight, body fat percentage, and BMI were extracted from the body fat scale; baseline values ​​and coefficients of variation for each indicator were calculated.

[0015] 3.2 Speech voice feature extraction: The fundamental frequency extraction algorithm is used to calculate the mean fundamental frequency and jitter of speech, and extract speech rate (syllables / second), pitch (mean fundamental frequency), tremor (fundamental frequency jitter coefficient), pause frequency (silence duration ratio), and volume (mean energy).

[0016] 3.3 Facial Image Feature Extraction: The activation intensity of facial action units was extracted using a facial action coding system to identify the probability distribution of seven basic emotions (joy, sadness, anger, fear, surprise, disgust, and calmness).

[0017] 3.4 Data Extraction from Psychological Characteristic Scales: Extract score vectors for types 1-9 from the Enneagram test results; extract scores for four dimensions (extroversion / introversion, sensing / intuition, etc.) from the MBTI test results; extract scores for five dimensions (openness, conscientiousness, extraversion, agreeableness, and neuroticism) from the Big Five personality test results; extract the total score from the Self-Rating Depression Scale (SDS); extract the total score from the Self-Rating Anxiety Scale (SAS); and extract the total score from the Life Satisfaction Scale (SWLS).

[0018] Step S4: Automatic aggregation and intergenerational association based on blood relations.

[0019] The collected data and its feature vectors are associated with corresponding member nodes, establishing kinship links between data and members. Immediate family members (parents, children, spouse) automatically subscribe to health summary data, while collateral relatives set visibility ranges based on kinship distance. Cross-generational data is automatically linked, establishing genetic similarity edges between grandparents' health indicators and grandchildren's indicators for subsequent genetic risk analysis. Data from multiple devices belonging to the same member is automatically merged to form a complete personal health record.

[0020] Step S5: Joint analysis using multi-source fusion analysis engine.

[0021] Build a multi-source fusion analysis engine to perform joint analysis on the collected data: 5.1 Health Indicator Screening: Real-time indicators are compared with personal and family baselines to identify abnormal fluctuations. For example, blood pressure exceeding personal baseline by 20% or family baseline by 15% is marked as abnormal.

[0022] 5.2 Genetic Risk Identification: Genetic similarity is calculated based on intergenerational blood relations. Cosine similarity or Euclidean distance is used to calculate the similarity of health indicators between the target member and the relatives with the disease. When the similarity exceeds the threshold, a genetic risk warning is generated.

[0023] 5.3 Emotional State Monitoring: Integrating speech and facial expression features, the system identifies emotional states and monitors trends in real time. An emotional alert is triggered when a person experiences prolonged periods of low mood or increased voice tremor.

[0024] 5.4 Personality Trait Analysis: Based on psychological trait scale data, personality trait vectors are constructed and compared with family personality maps to identify the personality positioning of members in the family and analyze the intergenerational personality inheritance trend.

[0025] 5.5 Multi-indicator joint analysis: Combine abnormalities from multiple dimensions for joint judgment. For example, high blood pressure with increased voice tremor can identify "emotional hypertension"; high blood sugar with a family history of diabetes can strengthen the early warning of diabetes risk.

[0026] Step S6: Generate health trend tips and personalized status reminders.

[0027] Based on the results of the multi-source fusion analysis, the following prompt message is generated: 6.1 Health Trend Tips: Personal alert: "Your blood pressure is high today (systolic pressure 142). Rest and monitoring are recommended." Family similarity indicator: "Your blood sugar trend is highly similar to your father's (similarity 0.85). Your father has a history of diabetes; we recommend monitoring this." Genetic risk warning: "Three family members have a history of hypertension, and your blood pressure is trending upward. Regular monitoring is recommended." A cross-generational warning sign: "The grandson's blood pressure is high, similar to his grandfather's when he was young, indicating a family history of hypertension." 6.2 Personal Status Reminder: Stress level assessment: "Recent stress score is high; relaxation exercises are recommended." Mood fluctuation monitoring: "Your mood has been low for the past week; it is recommended that you communicate more with your family." Social activity analysis: "Social interactions have decreased in the past month; however, individuals are more likely to proactively contact relatives and friends." Cognitive trend: "Speech rate decreased by 12% compared to the same period last year; cognitive assessment is recommended."

[0028] Collected data, feature vectors, and analysis results are stored in the family knowledge base, updating the family health map and genetic trait map. Abnormal events automatically create map nodes, associating them with time, members, indicators, and alert information. Indicator trend charts are linked to timelines, supporting visual backtracking.

[0029] Step S8: Collaborate with the family digital clone engine.

[0030] Data synchronization and context interaction are achieved through the collaborative interface module and the digital clone engine described in Patent 4: Real-time data collection is synchronized to the digital avatar engine for incremental learning of the avatar model. Injecting health trend alerts, genetic risk alerts, and personality status reminders into the clone's context enables the clone to proactively remind users in natural language. The results of the multi-source fusion analysis will be stored in the family knowledge base to update the family health map and genetic trait map. Receive data returned by the clone engine, such as new knowledge and events learned by the clone, and update the knowledge base. Secondly, embodiments of the present invention provide a multimodal data intelligent acquisition, storage, and analysis system based on kinship maps, comprising: The family knowledge base management module is used to build and maintain a structured family knowledge base that includes family member nodes, blood relationship edges, and related multi-source data; The smart hardware interface module is used to connect to smart hardware devices and receive multimodal data in real time. The feature extraction module is used to extract feature vectors from multimodal data in real time. The lineage aggregation module is used to automatically associate collected data and its feature vectors with corresponding member nodes based on lineage relationships; A multi-source fusion analysis engine is used to jointly analyze collected data and generate health trend alerts, genetic risk alerts, and individual status reminders. The collaborative interface module is used for data synchronization and context interaction with the family digital clone engine; The privacy protection module is used to implement biometric-derived key encryption, kinship-based differentiated access control, and federated learning-based distributed analysis.

[0031] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0032] III. Beneficial Technical Effects Compared with the prior art, the present invention has the following beneficial technical effects: 1. Achieve integrated collection and aggregation of family health data: Automatically aggregate data scattered across devices such as smart bracelets, blood pressure monitors, blood glucose meters, and smart speakers into a family knowledge base to form a unified family health record, thus solving the problem of data silos.

[0033] 2. For the first time, soft data such as speech patterns and psychological characteristic scales are incorporated into family health management: By extracting features such as speech rate, tone, and tremor, and combining them with the results of psychological characteristic scales, real-time monitoring of emotional state, cognitive trends, and stress levels is achieved, filling the gap in the utilization of soft data.

[0034] 3. Intelligent identification of genetic risks based on blood relations: By utilizing blood relations in the family knowledge base, the similarity of health indicators with relatives who have already contracted the disease can be identified, enabling cross-generational genetic risk warning and providing a scientific basis for family health management.

[0035] 4. Multi-source fusion joint analysis capability: Combine physiological indicators, voice features and psychological features for joint analysis to identify complex health risks (such as emotional hypertension) and improve the accuracy of analysis.

[0036] 5. Deep collaboration with the digital avatar engine: Real-time data collection drives avatar evolution, and analysis results are injected into the avatar context, enabling the avatar to proactively remind users in natural language, forming a complete closed loop of collection → storage → analysis → application.

[0037] 6. Real-time perception and reminders of individual status: Based on psychological characteristic scales and voice analysis, it assesses stress level, mood fluctuations, social activity, and cognitive trends in real time, and provides personalized status reminders and emotional support.

[0038] 7. Financial-grade privacy protection: Employs biometric-derived key encryption, kinship-based differentiated access control, and federated learning-based distributed analysis to ensure the security and privacy of sensitive health data.

[0039] 8. Activate dormant family health data: Transform previously scattered and dormant health data into analyzable, predictable, and inheritable family health assets, thereby enhancing data value. Attached Figure Description

[0040] Figure 1 The system architecture diagram provided for embodiments of the present invention; Figure 2 This is a flowchart of multi-source data acquisition and feature extraction provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the automatic collection and intergenerational association of blood relations provided in an embodiment of the present invention; Figure 4 This is a diagram of the multi-source fusion analysis engine architecture provided in an embodiment of the present invention; Figure 5 A flowchart for generating health trend prompts and personalized status reminders provided in this embodiment of the invention; Figure 6 This is a schematic diagram illustrating the collaborative interaction with the digital clone engine provided in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0042] Example 1: Family Knowledge Base Construction and Hardware Device Binding This embodiment uses the Zhang family, which has four generations living under one roof, as an example to illustrate the process of building a family knowledge base and binding it with smart hardware devices.

[0043] 1.1 Initialization of Family Knowledge Base Users enter family member information, including: Great-grandfather: Zhang Deshan (1925-1998), deceased, with a history of hypertension. Grandfather: Zhang Jianguo (1955-), alive, with a history of hypertension. Father: Zhang Ming (1985-), still alive Son: Zhang Xiaobao (2015-), still alive Establish bloodline relationships: Great-grandfather → Grandfather (father and son), Grandfather → Father (father and son), Father → Son (father and son).

[0044] 1.2 Smart Hardware Device Binding Family members bind their respective smart devices: When the device is used for the first time, the system guides the user to complete facial or voiceprint collection and establish biometric anchor points. All subsequent collected data is automatically associated with the corresponding member nodes.

[0045] Example 2: Real-time Acquisition and Feature Extraction of Multi-Source Data This embodiment demonstrates the entire process of the system acquiring multi-source data in real time and extracting features.

[0046] 2.1 Physiological Indicator Collection and Feature Extraction My grandfather used a smart blood pressure monitor to measure his blood pressure; the system collects the data in real time. Systolic blood pressure: 142 mmHg Diastolic blood pressure: 88 mmHg Heart rate: 78 beats / min Feature extraction: Compared to personal baseline (personal baseline: systolic blood pressure 135, diastolic blood pressure 85) → Systolic blood pressure exceeds baseline by 5%. Comparison with family baseline (family history of hypertension) → Mark genetic risk Calculate the coefficient of variation: 3% higher than the average of last week. Father's smart bracelet syncs in real time: Resting heart rate: 72 beats / min Sleep duration: 6.2 hours Steps: 8500 Feature extraction: Sleep below the recommended 7 hours → Mark as insufficient sleep normal heart rate range

[0047] The smart speaker in the living room collects voice recordings in real time during daily family conversations, and performs speaker separation and feature extraction: Grandfather said, "The weather is nice today, but I feel a little tired." Feature extraction: Speech rate: 3.2 syllables / second (personal baseline 3.5 → decreased by 8.6%) Tone: 155 Hz (personal baseline 165 → down 6%) Tremor level: 0.18 (personal baseline 0.12 → increase of 50%) Pause frequency: 0.22 (individual baseline 0.15 → up 47%) Overall assessment: Decreased speech rate, lowered tone, increased trembling, and more pauses → indicate fatigue, possibly suggesting physical discomfort.

[0048] The father said, "Okay, let's eat together tonight." Feature extraction: All indicators are normal, and the mood is stable.

[0049] 2.3 Facial Image Acquisition and Feature Extraction During a family video call, the smartphone camera captures facial images: Grandfather's facial expressions during the phone call: Facial motion units: AU4 (frowning) intensity 0.6, AU15 (corner of mouth pulled down) intensity 0.5 Probability of facial expression: Sadness 0.65, Calm 0.25, Others 0.10 Feature extraction: Label the sadness emotion and cross-validate it with speech features (decreased speech rate, increased tremor).

[0050] 2.4 Collection and feature extraction of psychological characteristic scales The father completed the psychological characteristic scale test pushed by the system: Enneagram test results: Type 1 (Perfectionist) score 8.2, Type 6 (Loyalist) score 7.5. MBTI Test Result: ISTJ (Introverted-Sensing-Thinking-Judging) The Big Five personality traits: Conscientiousness 0.85, Openness 0.45, Extraversion 0.35, Agreeableness 0.70, Neuroticism 0.40 Self-Rating Depression Scale (SDS) score: 32 (normal range) Self-Rating Anxiety Scale (SAS) score: 28 (normal range) Life Satisfaction Scale (SWLS) score: 28 (Satisfied) Feature extraction: Conscientious personality, introverted tendency, high conscientiousness, and healthy mental state.

[0051] Example 3: Automatic aggregation and intergenerational association based on kinship This example demonstrates how multi-source data can be automatically aggregated and cross-generationally associated based on kinship.

[0052] 3.1 Automatic Data Collection The system will automatically associate all collected data with the corresponding member nodes: Grandfather node association: Physiological indicators: Blood pressure 142 / 88, heart rate 78 Voice characteristics: decreased speech rate, increased tremor Facial features: Sad expression Comprehensive analysis: Fatigue state, possible physical discomfort Parent node association: Physiological indicators: heart rate 72, sleep 6.2 hours, steps 8500 Psychological characteristics: ISTJ type, conscientious personality, mental health Overall condition: Sleep deprivation Child node association: Physiological indicators: Heart rate 85 (normal for children) Overall condition: Healthy

[0053] The system automatically pushes health summaries based on blood relations: The father automatically received a push notification: "Your father's blood pressure today is 142 / 88, slightly higher than baseline; his voice characteristics indicate fatigue, we suggest you pay attention." The son's (minor) guardian (father) received a notification: "Zhang Xiaobao's health data is normal today."

[0054] The system automatically establishes intergenerational bloodline association edges: Great-grandfather (history of hypertension) → Grandfather (hypertension trend) → In progress of genetic similarity calculation Grandfather (hypertension trend) → Father (normal blood pressure) → Genetic risk markers Grandfather (hypertension trend) → Son (minor) → Establish early warning boundary The system calculates genetic similarity: Similarity in blood pressure trends between grandfather and great-grandfather: 0.82 (above the threshold of 0.75) Similarity in blood pressure trends between father and grandfather: 0.45 (below the threshold) Generate genetic risk labels: Grandfather: High risk of hereditary hypertension Father: Moderate risk of inheriting hypertension Son: The risk of hereditary hypertension remains to be observed.

[0055] This example demonstrates how a multi-source fusion analysis engine can combine data from multiple dimensions for comprehensive analysis.

[0056] 4.1 Health indicator screening The system compares various indicators in real time:

[0057] The system calculates genetic similarity based on intergenerational kinship: Comparison of blood pressure trends between grandfather and great-grandfather: - Great-grandfather's medical history: Blood pressure rose at age 45, and he was diagnosed with hypertension at age 55. - Grandfather's current age: 65, blood pressure is already high. - Trend similarity: 0.82 The system generates the following message: "Your blood pressure trend is highly similar to your father's. Your father has a history of hypertension; close monitoring is recommended."

[0058] A scale integrating speech tone, facial expressions, and psychological characteristics: grandfather: - Speech characteristics: decreased speech rate, increased tremor, lower pitch - Facial expression: Probability of sadness 0.65 - Recent psychological characteristics scale: Not measured Overall assessment: Mood state score 4.2 / 10 (low), marked as "depressed, fatigued". The system prompts: "You've been feeling down and fatigued lately. We recommend getting plenty of rest and spending more time with your family."

[0059] Constructing a family personality map based on the results of psychological trait scales: Family personality traits: Strong sense of conscientiousness, good mental health. The message reads: "You and your father share a similar tendency toward conscientiousness, a trait that is passed down within the family and contributes to the continuation of a sense of family responsibility."

[0060] The system performs joint judgment on anomalies from multiple dimensions: Scene: Grandfather's blood pressure is high (142 / 88) + Increased tremor in his voice (+50%) + Sad facial expression Joint analysis: - Excluding simple physiological hypertension - Likelihood of being identified as "emotional hypertension" - The system generates a notification: "Today's high blood pressure may be related to your emotional state. We suggest you relax, rest for 30 minutes, and then retest." Scenario: Father sleep-deprived (6.2 hours) + normal heart rate + normal step count + normal SDS / SAS Joint analysis: - Identified as "simple sleep deprivation" - A prompt will be generated: "You have been sleep-deprived recently. We recommend going to bed half an hour earlier to ensure 7 hours of sleep." ---

[0061] This example demonstrates various prompts and reminders generated by the system.

[0062] 5.1 Generation of Health Trend Alerts Personal anomaly alert: "Your blood pressure today is 142 / 88, which is higher than your baseline (135 / 85). We recommend resting for 15 minutes and then retesting. If it remains high, please consult a doctor." Family similarity hints: "Your blood pressure trend is highly similar to your father's (similarity 0.82), and your father has a history of hypertension. Based on family health data, hypertension has a genetic predisposition in your family, and it is recommended that you monitor your blood pressure quarterly." Genetic risk warning: "Family health analysis shows that three of your immediate family members have a history of hypertension (great-grandfather, grandfather, and uncle), with a genetic risk score of 0.75. It is recommended that you have a cardiovascular health check-up every year starting at age 35." Cross-generational warning: "Grandfather's blood pressure is high today (142 / 88), and Father needs to monitor his own blood pressure trend. Although the grandson has not shown any abnormalities, it is recommended to cultivate healthy living habits from an early age to reduce the risk of hereditary transmission."

[0063] Stress level assessment: "Recent stress score: 6.8 / 10 (slightly above average), mainly from work. Recommendation: Do 10 minutes of deep breathing exercises every day, and schedule leisure activities on weekends." Mood fluctuation monitoring: "Your mood has fluctuated significantly over the past week, with a 15% increase in speech rate and a 22% increase in tone. Recommendation: Communicate more with family members, keep an emotional diary, and seek professional support if necessary." Social activity analysis: "Social interactions have decreased by 35% in the past 30 days, with a drop in voice interactions. Recommendation: Take the initiative to contact relatives and friends and attend family gatherings." Cognitive state trends: "Speech rate decreased by 8% compared to the same period last year, and vocabulary richness decreased by 5%. Tip: Engage in mental activities such as Sudoku and reading to maintain cognitive activity."

[0064] The system generates a comprehensive health report daily: [Zhang Family Health Daily Report - March 12, 2026] Member Health Overview Grandfather: High blood pressure (142 / 88), signs of voice fatigue, and low mood. Father: Sleep deprivation (6.2 hours), mental health is good. Son: Health Key Focus Grandfather's health advice: Abnormal blood pressure and mood; it is recommended to rest and retest. Family risk warning: The risk of hypertension being inherited persists. Personality status reminder: Father is under moderate stress; relaxation is recommended. Trend Analysis Grandfather's blood pressure: 3% higher than last week Father's sleep: less than 7 hours for 3 consecutive days Family health score: 82 / 100 Health advice Grandfather: Relax, check your blood pressure again in 30 minutes. Father: Go to bed half an hour earlier tonight. Family: Arrange a family walk on the weekend.

[0065] This embodiment demonstrates how the data collected by this patent can collaborate with the digital clone engine of Patent 4 to drive clone evolution and achieve natural interaction.

[0066] 6.1 Data Synchronization and Clone Learning The system will collect data in real time and synchronize it to the digital clone engine: Synchronization data packets: { "member": "grandfather", "time": "2026-03-22 10:30", "vital_signs": {"bp_sys": 142, "bp_dia": 88, "hr": 78}, "speech_features": {"speech_rate": 3.2, "pitch": 155, "jitter":0.18}, "emotion": "sadness", "health_alert": "High blood pressure, fatigued state" } After the digital clone engine receives the data: Use the data as incremental learning samples to update the adapter parameters of the grandfather clone. I learned something new: When my grandfather's blood pressure was high, he tended to speak in a low, slow tone. Update the knowledge graph: Add this record to the "Grandfather's Health Events" node.

[0067] That evening, the father interacted with his grandfather's digital alter ego: Father: "Grandpa, how are you feeling today?" Clone (generated based on new learning data): (Gentle but slightly somber tone) I'm a little tired today, and my blood pressure is a bit higher than usual. But don't worry, the doctor said to just take my medication on time. How was your work today? Father: "It's fine, just a bit busy." Clone: ​​"(Concerned tone) No matter how busy you are, you must remember to rest. I've noticed you haven't been getting enough sleep lately, so go to bed early tonight."

[0068] The clone can sense the context and provide personalized suggestions: Father: "What are your plans for the weekend?" Clone: ​​"(Flipping through the knowledge base) Your dad's blood pressure has been a bit high lately, and he's not in a good mood either. How about we take him to the park this weekend? He loves going to the botanical garden; he used to go there all the time." Father: "Good idea." Clone: ​​"(In a warm tone) Then I'll help you check the recent flowering season at the botanical garden, as well as the route. Remember to bring his blood pressure medication."

[0069] Through repeated interactions, the grandfather's alter ego learned the new mode of expression: When healthy: Speaks in a light and cheerful tone, at a normal pace, and frequently uses "hehe" and "haha". When fatigued: The tone is low and the speech is slow; phrases like "sigh" and "a little tired" are frequently used. After evolving, the clone can automatically adjust its tone and content based on its current health status, achieving "clone interaction with health status awareness".

[0070] 6.5 Quantitative Evaluation of Synergistic Effects

[0071] This embodiment demonstrates the system's privacy protection mechanism.

[0072] 7.1 Biometric Derivative Key Encryption Grandfather's health data encryption process: A 256-bit encryption key is derived from the grandfather's voiceprint feature using a secure hash function. The key is derived in a TEE (Trusted Execution Environment) and is never stored. Health data is stored after being encrypted with this key. Decryption requires real-time voiceprint collection to generate a new key. Even if the database is leaked, it cannot be decrypted without biometric signatures.

[0073] Differences in views accessing the same health data across different blood relations:

[0074] Health data analysis is performed on a local device; only anonymized features are uploaded. Grandfather's blood pressure data was used to calculate trend characteristics locally. Only upload the desensitized feature "blood pressure trend rising by 3%". Raw blood pressure reading (142 / 88) will not be uploaded. Even with secure multi-party computation, it is impossible to infer individual data when aggregating multi-member data.

[0075] Users can request the deletion of specific health data: Deletion requests require multimodal liveness verification (face + voiceprint). Delete the original data from the database. Remove relevant event nodes from the knowledge graph Reverse fine-tuning of the affected digital clone adapter Ensure the model no longer mentions the deleted health events. The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent acquisition, storage, and analysis of multimodal data based on kinship maps, characterized in that, include: A structured family knowledge base is constructed and maintained, containing family member nodes, blood relationship edges, and associated multi-source data. Multimodal data of target members is collected through intelligent hardware devices, including physiological indicator data, speech pattern data, facial image data, and psychological characteristic scale data. Feature vectors of the multimodal data are extracted in real time, including physiological feature vectors, speech prosody feature vectors, facial expression feature vectors, and psychological feature vectors. Based on blood relationships, the collected data and its feature vectors are automatically aggregated and associated with corresponding member nodes, establishing cross-generational blood relationship edges. A multi-source fusion analysis engine is constructed to jointly analyze the aggregated data, generating health trend alerts, genetic risk alerts, and personality status reminders. The analysis results are stored in the family knowledge base and work in conjunction with the family digital clone engine to drive the clone's real-time evolution.

2. The method according to claim 1, characterized in that, The smart hardware devices include smart bracelets, smart blood pressure monitors, smart blood glucose meters, smart body fat scales, smart speakers, psychological testing terminals, and smartphone cameras and microphones; when the devices are used for the first time, family members' identities are bound through facial recognition or voiceprint recognition.

3. The method according to claim 1, characterized in that, The physiological indicators include blood pressure, heart rate, blood sugar, blood lipids, and body mass index; the speech and vocal data include speech rate, pitch, tremor, pause frequency, and volume; the facial image data includes facial motion unit activation intensity and expression category probability; and the psychological characteristic scale data includes one or more of the following: Enneagram, MBTI dimensions, Big Five personality dimensions, Self-Rating Depression Scale, Self-Rating Anxiety Scale, and Life Satisfaction Scale.

4. The method according to claim 1, characterized in that, The automatic data collection based on blood relations includes: associating collected data with corresponding member nodes and establishing blood relation edges between data and members; automatic subscription of health summary data for immediate family members and setting visibility range for collateral family members according to kinship distance; automatic cross-generational data association, establishing genetic similarity edges between the health indicators of grandparents and grandchildren; and multi-device data fusion, automatically merging data from multiple devices of the same member to form a complete health record.

5. The method according to claim 1, characterized in that, The multi-source fusion analysis engine includes a health indicator screening submodule, a genetic risk identification submodule, an emotional state monitoring submodule, a personality trait analysis submodule, and a multi-indicator joint analysis submodule. The health indicator screening submodule is used to compare real-time indicators with personal and family baselines to identify abnormal fluctuations. The genetic risk identification submodule is used to calculate genetic similarity based on intergenerational blood relations and identify health trends similar to those of relatives who have already developed the disease. The emotional state monitoring submodule is used to integrate voice tone features and facial expression features to identify emotional states and monitor trends in real time. The personality trait analysis submodule is used to construct a personality trait vector based on psychological trait scale data and compare it with a family personality map. The multi-indicator joint analysis submodule is used to jointly judge anomalies in multiple dimensions and identify complex health risks.

6. The method according to claim 1, characterized in that, The health trend alerts include: personal abnormality alerts, triggered when indicators exceed personal baselines; family similarity alerts, triggered when indicators are highly similar to those of relatives with the same illness; genetic risk alerts, triggered when multiple relatives have the same medical history and their indicators show similar trends; and intergenerational warning alerts, triggered when grandchildren's indicators are similar to those of their grandparents.

7. The method according to claim 1, characterized in that, The personalized status reminder information includes: stress level assessment, calculated based on a psychological characteristic scale and voice tremor; emotion fluctuation monitoring, based on speech rate changes and facial expression changes; social activity analysis, calculated based on voice interaction frequency; and cognitive state trends, based on speech rate changes and vocabulary richness analysis.

8. The method according to claim 1, characterized in that, The collaboration with the family digital avatar engine includes: synchronizing real-time collected data to the digital avatar engine for incremental learning of the avatar model; injecting health trend alerts, genetic risk alerts, and personality status reminders into the avatar context, enabling the avatar to proactively remind the user in natural language; and storing multi-source fusion analysis results in the family knowledge base to update the family health map and genetic characteristic map.

9. A multimodal data intelligent acquisition, storage, and analysis system based on kinship maps, characterized in that, include: The family knowledge base management module is used to build and maintain a structured family knowledge base that includes family member nodes, blood relationship edges, and related multi-source data; The intelligent hardware interface module is used to connect to intelligent hardware devices and receive multimodal data in real time; the feature extraction module is used to extract feature vectors from multimodal data in real time; and the lineage aggregation module is used to automatically associate the collected data and its feature vectors with the corresponding member nodes based on lineage relationships. A multi-source fusion analysis engine is used to jointly analyze collected data and generate health trend alerts, genetic risk alerts, and individual status reminders. The Collaboration Interface module is used for data synchronization and context interaction with the Family Digital Avatar Engine.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images