Elderly health management system and method based on family audio and video

Abstract

The present invention discloses an elderly health management system and method based on family audio and video, and relates to the field of health management technology. The elderly health management system based on family audio and video includes: a family audio and video interaction module, a health data collection module, a health data analysis module, a health record management module, a health management suggestion module and a data encryption and privacy protection module. The present invention scientifically organizes the interaction arrangements between relatives and the elderly through an audio and video interaction schedule, and automatically uses a preset message template or an AI simulation template to replace it when the relatives fail to complete the interaction in time, ensuring that the elderly receive timely emotional care and avoid loneliness due to lack of interaction; innovatively proposes a solution combining family audio and video with health management, which makes up for the lack of emotional care, the limitations of single-modal analysis and the static generation of suggestions in traditional health management systems.

Description

Technical Field

[0001] The present invention relates to the technical field of health management, and in particular to a health management system and method for the elderly based on family audio and video. Background Art

[0002] As society ages, health management for the elderly has become a focus of social attention. Current health management systems for the elderly rely on health monitoring devices to collect physiological data, such as heart rate, blood pressure, and body temperature, and generate health management recommendations based on fixed rules. Current health management methods for the elderly have the following problems:

[0003] Lack of emotional care and interactive support: Existing health management systems focus more on the collection and analysis of physiological data, ignoring the mental health and family needs of the elderly. When relatives fail to communicate with the elderly in a timely manner due to work or other reasons, the elderly are prone to loneliness, which affects their overall health.

[0004] Single health data analysis: Traditional systems usually only process single-modal data (such as physiological data), and have limited ability to evaluate behavioral activity data and psychological status. Single data analysis models are difficult to capture the multi-dimensional characteristics of the health status of the elderly, and there is a problem of missed health risks.

[0005] Health management recommendations lack dynamic adjustment: Current health management recommendations are usually generated based on fixed rules and lack the ability to dynamically respond to the individual health status and historical data of the elderly, making it difficult to provide accurate personalized management.

[0006] Insufficient privacy and security protection: The privacy and security management of elderly health data has not received sufficient attention, especially during the process of data transmission, storage and sharing, there is a risk of sensitive information leakage and abuse. Summary of the invention

[0007] To solve the above problems, the present invention proposes a method and system for elderly health management based on family audio and video. Through innovative family interaction management mechanism, multimodal health data analysis technology, dynamic health management suggestion generation and perfect data privacy protection mechanism, the present invention realizes comprehensive assessment, dynamic management and emotional support of the health status of the elderly, while ensuring data security and privacy, filling the gaps in existing technologies.

[0008] The elderly health management method based on family audio and video includes the following steps:

[0009] Step S1: obtaining the opinions of the relatives who are in contact with the target elderly by asking them, and formulating an audio and video interaction schedule based on the opinions; regularly reminding the relatives to conduct audio and video interaction with the elderly as planned according to the audio and video interaction schedule; the audio and video interaction schedule includes real-time audio and video interaction arrangements and preset reminder contents;

[0010] Step S2: If the relative does not conduct audio and video interaction within the specified time, it is regarded as an abnormal interaction, and the preset audio and video template is used to automatically conduct audio and video interaction, and the voice and video data during the interaction are recorded to generate template audio and video information; the generated template audio and video information is sent to the corresponding relative to remind him to supplement the actual audio and video interaction; the abnormal interaction is counted, and the audio and video interaction schedule is optimized;

[0011] Step S3: During the audio and video interaction process, health information data of the elderly is collected; the health information data includes physiological data obtained by the smart health monitoring device and behavioral activity data obtained by the camera and microphone;

[0012] Step S4: inputting the health information data into the elderly health data analysis model for calculation to obtain real-time health analysis information; the real-time health analysis information includes physiological indicators, behavioral indicators, psychological indicators or abnormal reminders;

[0013] Step S5: storing the health information data and real-time health analysis information to generate the elderly's health records, and performing statistical analysis based on the daily, weekly and monthly data of the health records to generate health management suggestions;

[0014] Step S6: Data encryption and privacy protection, through encrypted transmission and storage of health information data and audio and video records, setting access permissions for health records, and privacy protection through data anonymization and access record auditing.

[0015] As a preferred technical solution of the present invention, the automatic audio and video interaction using a preset audio and video template includes:

[0016] Obtaining the interaction index of the relatives through the audio and video interaction plan table, and randomly selecting the type of preset audio and video templates according to the weighted interaction index, wherein the types of audio and video templates include preset message templates and AI simulation templates;

[0017] Construction and use of preset message templates: obtain the daily needs of the target elderly and the care content provided by relatives, and pre-record voice or video clips for the elderly, the video clips include daily greetings, holiday blessings or health reminders; in the event of abnormal interaction, select the corresponding content from the preset message template according to the time information of the day and send it to the elderly;

[0018] Construction and use of AI simulation templates: Set audio and video interaction scenarios under different scenarios, and construct audio and video samples for each relative of the target elderly. The audio and video samples include audio and video construction parameters and relative information; the audio and video construction parameters are obtained through family audio and video interaction or pre-collection, including voice parameters, portrait parameters and habit parameters; use audio and video samples in combination with preset audio and video interaction scenarios to generate AI simulation templates; the AI ​​simulation template generates audio and video to interact with the elderly by simulating the voice, portrait and behavioral habits of relatives;

[0019] The interaction records using preset message templates or AI simulation templates are stored as template audio and video information for subsequent statistical analysis and optimization.

[0020] As a preferred technical solution of the present invention, the types of preset audio and video templates selected randomly by weight according to the interaction index include:

[0021] Obtaining the interaction index: Calculate the interaction index based on the historical records of audio and video interactions between relatives and the elderly. I , the calculation formula of the interaction index is: I = w 1 F + w 2 D + w 3 C ,in F Indicates the frequency of audio and video interaction between relatives and the elderly. D Indicates the duration of audio and video interaction between relatives and the elderly. C Indicates the richness of interactive content or the coverage of interactive scenarios; w 1 、w 2 、w 3 is the weight coefficient of the corresponding indicator, and satisfies w 1 +w 2 +w 3 =1 ;

[0022] Weighted random selection method: Calculate the selection probability of the preset message template and the AI ​​simulation template. Suppose the selection probability of the preset message template is , the selection probability of AI simulation template is ,but and Satisfy the following formula:

[0023] , ;in is the interaction index, It is a preset balance factor used to adjust the selection tendency of the two templates;

[0024] Weighted random selection of template types: Generate a random number K in the interval [0,1) and calculate K and and The difference is calculated by comparing the absolute values ​​of the difference calculation results, and the template type corresponding to the selection probability with a smaller absolute value is selected for audio and video interaction.

[0025] As a preferred technical solution of the present invention, the method of collecting statistics on abnormal interactions and optimizing the audio and video interaction schedule includes:

[0026] Abnormal interaction statistics: record the relevant information of each abnormal interaction, including the time of occurrence, corresponding relatives, abnormal type and the audio and video template used; the abnormal types include no interaction and supplementary actual audio and video interaction; for abnormal records without supplementary actual audio and video interaction, count the number of abnormal interactions of each relative N e and abnormality rate R , the abnormal rate R The calculation formula is: R = N e / N t ,in N t is the total number of planned interactions in the same time period;

[0027] Supplementary interaction statistics: distinguish the records of actual audio and video interactions that have been supplemented, and count the time intervals between supplementation completions T c and replenishment rate C r , the replenishment rate C r The calculation formula is: C r =N c / N e ,in N c The number of actual audio and video interactions that have been supplemented;

[0028] Optimize the audio and video interaction schedule: adjust the interaction reminder time according to the statistical abnormal time distribution to avoid abnormal high-incidence periods; R For relatives whose number exceeds the set threshold, increase the frequency of interactive reminders or optimize the reminder method; T cFor relatives whose abnormality rate exceeds the set threshold, the reminder content is improved, and the improvement includes adding detailed explanations or adding emergency reminders; dynamically adjusting the selection ratio of audio and video template types, giving priority to using AI simulation templates for relatives with high abnormality rates, and reducing plan dependence.

[0029] As a preferred technical solution of the present invention, the elderly health data analysis model includes the following levels:

[0030] Physiological data processing layer: Use the numerical submodule based on convolutional neural network to process the physiological data collected by the intelligent health monitoring equipment; extract the characteristics of the data through one-dimensional convolution operation, and combine the multi-layer fully connected network for classification and regression calculation to generate physiological indicator characteristics; the physiological indicator characteristics include but are not limited to heart rate variability, blood pressure volatility, and abnormal temperature detection values;

[0031] Behavior activity data processing layer: Use the audio-visual submodule based on the long short-term memory network to process the behavior activity data collected by the camera and microphone; extract the speech speed, intonation and volume features of the voice data, analyze the action sequence features of the video data, and identify the behavior indicator features in combination with the time series model; the behavior indicator features include but are not limited to activity volume, gait analysis, and voice emotion tendency; the physiological and behavioral indicator features are output in the form of embedded vectors;

[0032] Multimodal fusion layer: The embedding vectors output by the physiological data processing layer and the behavioral activity data processing layer are processed comprehensively through the multimodal fusion mechanism; the contribution of data from different sources is weighted and adjusted using the attention mechanism to generate a comprehensive health status embedding vector; the preliminary comprehensive analysis results are output as input to the psychological state assessment layer and the anomaly detection layer;

[0033] Psychological state assessment layer: Use the psychological submodule based on multi-layer perceptron to perform nonlinear mapping on the comprehensive health state embedding vector to infer the psychological indicator characteristics; the psychological indicator characteristics include emotional state and cognitive state;

[0034] Abnormal detection layer: Use the abnormal detection submodule based on the autoencoder to judge the reconstruction error of physiological index characteristics, behavioral index characteristics and psychological index characteristics; if the reconstruction error exceeds the preset threshold, an abnormal reminder is output to indicate health risks or abnormal behaviors;

[0035] The comprehensive output layer organizes the output results of each layer into real-time health analysis information and outputs it.

[0036] As a preferred technical solution of the present invention, the construction of the elderly health data analysis model includes:

[0037] Acquisition and division of data sets: obtain data sets containing physiological data, behavioral activity data, and psychological state labels from target elderly health monitoring devices, cameras, microphones, and interaction records; divide the data sets into training sets, validation sets, and test sets; preprocess the data, including missing value filling, outlier removal, and normalization;

[0038] Training of numerical submodule: Use a pre-trained model based on a convolutional neural network to train a subset of physiological data; use the cross-entropy loss function to optimize model parameters, and use the learning rate scheduler to dynamically adjust the learning rate; evaluate model performance through a validation set, with indicators including mean square error and classification accuracy; and obtain the final numerical submodule;

[0039] Training of the audio-visual submodule: Use a pre-trained model based on the long short-term memory network to train the activity data subset; use the cross entropy loss function in the classification task and the mean square error in the regression task to optimize the model; evaluate the model performance through the validation set; and obtain the final audio-visual submodule;

[0040] Multimodal fusion: The output embedding vectors of the numerical submodule and the audio-visual submodule are multimodally fused through the attention mechanism. The fusion process dynamically adjusts the contribution values ​​of different modalities according to the weight of the data source to generate a comprehensive health status embedding vector. The parameters of multimodal fusion are optimized through the validation set to evaluate the accuracy of the fused model in assessing the psychological state.

[0041] Construction of the anomaly detection submodule: Use the autoencoder to train the anomaly detection submodule on the comprehensive health status embedding vector; obtain the final anomaly detection submodule;

[0042] Model performance evaluation: Evaluate the performance of each submodule on the test set, with indicators including classification accuracy, mean square error, F1 score, and accuracy of anomaly detection; adjust model parameters and complete the final model deployment.

[0043] As a preferred technical solution of the present invention, generating health management suggestions includes:

[0044] Acquisition of health data analysis results: Acquisition of real-time health analysis information from the elderly health data analysis model; extraction of daily, weekly and monthly statistical analysis results based on historical data stored in health records;

[0045] Health status assessment: Evaluate the health status of the elderly based on the trend comparison of real-time health analysis information and historical data; comprehensively score the physiological status, behavioral activities and psychological status;

[0046] Health management advice generation: Generate targeted health management advice based on the results of health status assessment and abnormal reminders; the advice includes but is not limited to dietary advice, exercise plans, psychological care advice and medical advice;

[0047] Dynamic adjustment of recommendations: Dynamically adjust health management recommendations based on the daily changes in the elderly’s health information data and implementation feedback; regularly update recommendations to adapt to changes in the elderly’s health status.

[0048] As a preferred technical solution of the present invention, the data encryption and privacy protection includes:

[0049] Data encryption transmission and storage: Use symmetric encryption algorithms to perform end-to-end encryption transmission of health information data and audio and video records; use asymmetric encryption algorithms to encrypt and protect health records and interaction records stored in the cloud and locally;

[0050] Access rights management: Set hierarchical access rights for health records and audio and video data, authorize different relatives and medical institutions to access specific ranges of data; use multi-factor authentication to ensure access security;

[0051] Data anonymization: During statistical analysis and data sharing, health information data is anonymized to remove personal identity information associated with the elderly; voice and portrait data in audio and video recordings are blurred or desensitized to ensure privacy security;

[0052] Access records and audits: Record detailed logs for each data access, modification, and use operation; audit logs regularly to detect and prevent unauthorized access.

[0053] The elderly health management system based on family audio and video includes the following modules:

[0054] Family audio and video interaction module: used to obtain relatives’ opinions and formulate an audio and video interaction schedule; remind relatives to conduct audio and video interactions with the elderly as planned, and automatically generate audio and video interaction content based on preset message templates or AI simulation templates when relatives fail to interact as planned;

[0055] Health data collection module: collects physiological data through smart health monitoring devices; collects behavioral activity data through cameras and microphones;

[0056] Health data analysis module: used to analyze the collected health information data and generate real-time health analysis information, including: physiological data processing unit: feature extraction and classification of physiological data based on convolutional neural network; behavioral data processing unit: feature analysis of behavioral activity data based on long short-term memory network; multimodal fusion unit: use attention mechanism to fuse and analyze physiological and behavioral data to generate comprehensive health status embedding vector; psychological state assessment unit: psychological assessment of health status embedding vector based on multi-layer perceptron; abnormality detection unit: detect abnormalities based on autoencoder and output abnormality reminder;

[0057] Health record management module: stores health information data and real-time health analysis information to form the health records of the elderly; generates health statistics based on daily, weekly and monthly data;

[0058] Health management advice module: Generates health management advice based on health records and real-time analysis information, including diet advice, exercise plans, psychological care advice and medical reminders;

[0059] Data encryption and privacy protection module: used for data encryption and privacy protection.

[0060] The present invention has the following advantages:

[0061] The present invention scientifically organizes the interaction arrangements between relatives and the elderly through an audio and video interaction plan, and automatically uses a preset message template or an AI simulation template to replace it when the relatives fail to complete the interaction in time, ensuring that the elderly receive timely emotional care and avoid loneliness due to lack of interaction; it innovatively proposes a solution that combines family audio and video with health management, which makes up for the lack of emotional care, the limitations of single-modal analysis and the static generation of suggestions in traditional health management systems.

[0062] The present invention processes physiological data and behavioral activity data through different network models respectively, and combines them with multimodal fusion technology to comprehensively evaluate the health status of the elderly; the analysis results cover physiological indicators, behavioral indicators and psychological indicators, and are combined with the anomaly detection layer to output abnormal reminders, effectively improving the comprehensiveness and accuracy of health assessments.

[0063] The present invention combines emotional care with health monitoring and uses AI to simulate the voices, portraits and behavioral habits of relatives to achieve a more realistic interactive experience; it solves the emotional void problem caused by relatives not interacting on time and effectively enhances the mental health and emotional satisfaction of the elderly.

[0064] The present invention generates dynamically adjusted health management suggestions based on real-time health analysis information and historical health records, covering diet, exercise, psychological care and medical reminders; the health management suggestions can be dynamically optimized according to changes in health status to provide personalized solutions that better meet the needs of the elderly.

[0065] The present invention adopts end-to-end encryption technology to encrypt, transmit and store health data and audio and video records; through hierarchical access rights, data anonymization processing and access record auditing, the privacy and usage security of the elderly's health data are fully protected, ensuring the legality and compliance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings in the following description are only schematic diagrams of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

[0067] Figure 1 This is a schematic diagram of the structure of the elderly health management system based on family audio and video adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] Embodiment 1, a method for managing the health of the elderly based on family audio and video, comprises the following steps:

[0070] Step S1: obtaining the opinions of the relatives who are in contact with the target elderly person by asking them, and formulating an audio and video interaction schedule based on the opinions; the steps specifically include the following:

[0071] Information collection: Interviews or surveys with relatives who have contact with the target elderly to collect the following opinions: relatives' available interaction time periods (such as weekday evenings or weekends); relatives' preferences for interaction frequency (such as once a day, several times a week, etc.); the care that relatives hope to convey during the interaction (such as health reminders, daily greetings, holiday wishes, etc.); relatives' special needs (such as mainly through voice interaction, whether interaction reminders are needed, etc.).

[0072] Preliminary design of interaction schedule: Based on the opinions collected from relatives and combined with the daily routines and health needs of the target elderly, a preliminary audio and video interaction schedule is developed;

[0073] The schedule includes: real-time audio and video interaction arrangements: specify the time and frequency of real-time interaction between each relative and the elderly; preset reminder content: configure specific reminder content for each relative, including interaction prompts, preset audio and video template types, etc.; special matters: interaction arrangements designed for special dates (such as the elderly’s birthdays and holidays).

[0074] Confirmation and optimization of the schedule: Send the preliminary audio and video interaction schedule to all relatives for confirmation; adjust the interaction time, frequency or reminder content based on the relatives' feedback to ensure that the schedule is more executable and flexible; combine the schedule with the health management goals of the target elderly, and give priority to interactive content related to the elderly's health and mental state (such as increasing dietary advice interactions for patients with hypertension).

[0075] Schedule deployment and reminders: The final confirmed audio and video interaction schedule is deployed to the system; the system automatically reminds relatives to conduct audio and video interactions with the elderly on time according to the schedule; and provides multiple reminder methods such as timed reminders, emergency reminders, and personalized interaction reminders to ensure the effective implementation of the schedule.

[0076] Step S2: If the relative does not conduct audio and video interaction within the specified time, it is regarded as an abnormal interaction, and the preset audio and video template is used to automatically conduct audio and video interaction, and the voice and video data during the interaction are recorded to generate template audio and video information; the generated template audio and video information is sent to the corresponding relative to remind him to supplement the actual audio and video interaction; the abnormal interaction is counted, and the audio and video interaction schedule is optimized;

[0077] The automatic audio and video interaction using the preset audio and video template includes:

[0078] Obtaining the interaction index of the relatives through the audio and video interaction plan table, and randomly selecting the type of preset audio and video templates according to the weighted interaction index, wherein the types of audio and video templates include preset message templates and AI simulation templates;

[0079] Construction and use of preset message templates: obtain the daily needs of the target elderly and the care content provided by relatives, and pre-record voice or video clips for the elderly, the video clips include daily greetings, holiday blessings or health reminders; in the event of abnormal interaction, select the corresponding content from the preset message template according to the time information of the day and send it to the elderly;

[0080] Construction and use of AI simulation templates: Set audio and video interaction scenarios under different scenarios, and construct audio and video samples for each relative of the target elderly. The audio and video samples include audio and video construction parameters and relative information; the audio and video construction parameters are obtained through family audio and video interaction or pre-collection, including voice parameters, portrait parameters and habit parameters; use audio and video samples in combination with preset audio and video interaction scenarios to generate AI simulation templates; the AI ​​simulation template generates audio and video to interact with the elderly by simulating the voice, portrait and behavioral habits of relatives;

[0081] The interaction records using preset message templates or AI simulation templates are stored as template audio and video information for subsequent statistical analysis and optimization.

[0082] The types of preset audio and video templates randomly selected by weight according to the interaction index include:

[0083] Obtaining the interaction index: Calculate the interaction index based on the historical records of audio and video interactions between relatives and the elderly. I , the calculation formula of the interaction index is: I = w 1 F + w 2 D + w 3 C ,in F Indicates the frequency of audio and video interaction between relatives and the elderly. D Indicates the duration of audio and video interaction between relatives and the elderly. C Indicates the richness of interactive content or the coverage of interactive scenarios; w 1 、w 2 、w 3 is the weight coefficient of the corresponding indicator, and satisfies w 1 +w 2 +w 3 =1 ;

[0084] Weighted random selection method: Calculate the selection probability of the preset message template and the AI ​​simulation template. Suppose the selection probability of the preset message template is , the selection probability of AI simulation template is ,but and Satisfy the following formula:

[0085] , ;in is the interaction index, It is a preset balance factor used to adjust the selection tendency of the two templates;

[0086] Weighted random selection of template types: Generate a random number K in the interval [0,1) and calculate K and and The difference is calculated by comparing the absolute values ​​of the difference calculation results, and the template type corresponding to the selection probability with a smaller absolute value is selected for audio and video interaction.

[0087] The counting of abnormal interactions and optimization of the audio and video interaction schedule includes:

[0088] Abnormal interaction statistics: record the relevant information of each abnormal interaction, including the time of occurrence, corresponding relatives, abnormal type and the audio and video template used; the abnormal types include no interaction and supplementary actual audio and video interaction; for abnormal records without supplementary actual audio and video interaction, count the number of abnormal interactions of each relative N e and abnormality rate R , the abnormal rate R The calculation formula is: R = N e / N t ,in N t is the total number of planned interactions in the same time period;

[0089] Supplementary interaction statistics: distinguish the records of actual audio and video interactions that have been supplemented, and count the time intervals between supplementation completions T c and replenishment rate C r , the replenishment rate C r The calculation formula is: C r =N c / N e ,in N c The number of actual audio and video interactions that have been supplemented;

[0090] Optimize the audio and video interaction schedule: adjust the interaction reminder time according to the statistical abnormal time distribution to avoid abnormal high-incidence periods; R For relatives whose number exceeds the set threshold, increase the frequency of interactive reminders or optimize the reminder method; T cFor relatives whose abnormality rate exceeds the set threshold, the reminder content is improved, and the improvement includes adding detailed explanations or adding emergency reminders; dynamically adjusting the selection ratio of audio and video template types, giving priority to using AI simulation templates for relatives with high abnormality rates, and reducing plan dependence.

[0091] Step S3: During the audio and video interaction process, collect the health information data of the elderly; the step specifically includes the following contents:

[0092] Initialize the health data collection environment: Configure the health data collection function on the audio and video interactive device, including connecting smart health monitoring devices (such as blood pressure monitors, heart rate monitors, thermometers) and multimedia sensor devices (cameras, microphones); ensure that the equipment operates normally and the accuracy and stability of the collected data meet the requirements.

[0093] Physiological data collection: real-time collection of the elderly's physiological data through smart health monitoring equipment, including: heart rate and heart rate variability; blood pressure and its volatility; blood oxygen saturation; body temperature and its changing trend; the device is set to collect timed collection or dynamically trigger the collection mode, and record a variety of physiological data according to interactive scenarios (such as after exercise or at rest).

[0094] Behavioral activity data collection:

[0095] The camera is used to obtain the behavioral activity data of the elderly, including: facial expression characteristics (such as smiling, frowning, and expressionless); posture and movement characteristics (such as sitting, standing, and range of motion); gait and activity volume (such as number of steps, exercise time);

[0096] The microphone is used to collect the elderly’s voice data, including: changes in speech speed, intonation and voice tone; volume and voice stability; and whether there are any abnormal sounds (such as coughing, wheezing).

[0097] Synchronous collection of multimodal data: synchronize timestamps of physiological data and behavioral activity data to ensure the consistency of multimodal data; record the collected data stream in real time during audio and video interaction, and store it in local cache or cloud storage for subsequent analysis.

[0098] Preprocessing and quality control of collected data: preprocessing of collected health information data, including: data denoising and outlier processing; sampling and sorting of dynamic data according to set time intervals; ensuring the accuracy and validity of collected data through data quality inspection algorithms, and eliminating errors or anomalies in the collection process.

[0099] Dynamic feedback and display of collected data: Some collection results (such as heart rate and blood pressure trends) are displayed in real time on the audio and video interactive interface, so that relatives or the elderly can intuitively understand the health status; when abnormal data appears (such as high heart rate, voice interruption), the system automatically triggers prompts or alarms to remind relatives to pay attention to the health of the elderly.

[0100] Step S4: inputting the health information data into the elderly health data analysis model for calculation to obtain real-time health analysis information; the real-time health analysis information includes physiological indicators, behavioral indicators, psychological indicators or abnormal reminders;

[0101] The elderly health data analysis model includes the following levels:

[0102] Physiological data processing layer: Use the numerical submodule based on convolutional neural network to process the physiological data collected by the intelligent health monitoring equipment; extract the characteristics of the data through one-dimensional convolution operation, and combine the multi-layer fully connected network for classification and regression calculation to generate physiological indicator characteristics; the physiological indicator characteristics include but are not limited to heart rate variability, blood pressure volatility, and abnormal temperature detection values;

[0103] Behavior activity data processing layer: Use the audio-visual submodule based on the long short-term memory network to process the behavior activity data collected by the camera and microphone; extract the speech speed, intonation and volume features of the voice data, analyze the action sequence features of the video data, and identify the behavior indicator features in combination with the time series model; the behavior indicator features include but are not limited to activity volume, gait analysis, and voice emotion tendency; the physiological and behavioral indicator features are output in the form of embedded vectors;

[0104] Multimodal fusion layer: The embedding vectors output by the physiological data processing layer and the behavioral activity data processing layer are processed comprehensively through the multimodal fusion mechanism; the contribution of data from different sources is weighted and adjusted using the attention mechanism to generate a comprehensive health status embedding vector; the preliminary comprehensive analysis results are output as input to the psychological state assessment layer and the anomaly detection layer;

[0105] Psychological state assessment layer: Use the psychological submodule based on multi-layer perceptron to perform nonlinear mapping on the comprehensive health state embedding vector to infer the psychological indicator characteristics; the psychological indicator characteristics include emotional state and cognitive state;

[0106] Abnormal detection layer: Use the abnormal detection submodule based on the autoencoder to judge the reconstruction error of physiological index characteristics, behavioral index characteristics and psychological index characteristics; if the reconstruction error exceeds the preset threshold, an abnormal reminder is output to indicate health risks or abnormal behaviors;

[0107] The comprehensive output layer organizes the output results of each layer into real-time health analysis information and outputs it.

[0108] The construction of the elderly health data analysis model includes:

[0109] Acquisition and division of data sets: obtain data sets containing physiological data, behavioral activity data, and psychological state labels from target elderly health monitoring devices, cameras, microphones, and interaction records; divide the data sets into training sets, validation sets, and test sets; preprocess the data, including missing value filling, outlier removal, and normalization;

[0110] Training of numerical submodule: Use a pre-trained model based on a convolutional neural network to train a subset of physiological data; use the cross-entropy loss function to optimize model parameters, and use the learning rate scheduler to dynamically adjust the learning rate; evaluate model performance through a validation set, with indicators including mean square error and classification accuracy; and obtain the final numerical submodule;

[0111] Training of the audio-visual submodule: Use a pre-trained model based on the long short-term memory network to train the activity data subset; use the cross entropy loss function in the classification task and the mean square error in the regression task to optimize the model; evaluate the model performance through the validation set; and obtain the final audio-visual submodule;

[0112] Multimodal fusion: The output embedding vectors of the numerical submodule and the audio-visual submodule are multimodally fused through the attention mechanism. The fusion process dynamically adjusts the contribution values ​​of different modalities according to the weight of the data source to generate a comprehensive health status embedding vector. The parameters of multimodal fusion are optimized through the validation set to evaluate the accuracy of the fused model in assessing the psychological state.

[0113] Construction of the anomaly detection submodule: Use the autoencoder to train the anomaly detection submodule on the comprehensive health status embedding vector; obtain the final anomaly detection submodule;

[0114] Model performance evaluation: Evaluate the performance of each submodule on the test set, with indicators including classification accuracy, mean square error, F1 score, and accuracy of anomaly detection; adjust model parameters and complete the final model deployment.

[0115] Step S5: storing the health information data and real-time health analysis information to generate the elderly's health records, and performing statistical analysis based on the daily, weekly and monthly data of the health records to generate health management suggestions;

[0116] Generating health management suggestions includes:

[0117] Acquisition of health data analysis results: Acquisition of real-time health analysis information from the elderly health data analysis model; extraction of daily, weekly and monthly statistical analysis results based on historical data stored in health records;

[0118] Health status assessment: Evaluate the health status of the elderly based on the trend comparison of real-time health analysis information and historical data; comprehensively score the physiological status, behavioral activities and psychological status;

[0119] Health management advice generation: Generate targeted health management advice based on the results of health status assessment and abnormal reminders; the advice includes but is not limited to dietary advice, exercise plans, psychological care advice and medical advice;

[0120] Dynamic adjustment of recommendations: Dynamically adjust health management recommendations based on the daily changes in the elderly’s health information data and implementation feedback; regularly update recommendations to adapt to changes in the elderly’s health status.

[0121] Step S6: Data encryption and privacy protection, through encrypted transmission and storage of health information data and audio and video records, setting access permissions for health records, and privacy protection through data anonymization and access record auditing.

[0122] The data encryption and privacy protection include:

[0123] Data encryption transmission and storage: Use symmetric encryption algorithms to perform end-to-end encryption transmission of health information data and audio and video records; use asymmetric encryption algorithms to encrypt and protect health records and interaction records stored in the cloud and locally;

[0124] Access rights management: Set hierarchical access rights for health records and audio and video data, authorize different relatives and medical institutions to access specific ranges of data; use multi-factor authentication to ensure access security;

[0125] Data anonymization: During statistical analysis and data sharing, health information data is anonymized to remove personal identity information associated with the elderly; voice and portrait data in audio and video recordings are blurred or desensitized to ensure privacy security;

[0126] Access records and audits: Record detailed logs for each data access, modification, and use operation; audit logs regularly to detect and prevent unauthorized access.

[0127] Example 2, Elderly health management system based on family audio and video, see Figure 1 As shown, the following steps are included:

[0128] Family audio and video interaction module: used to obtain relatives’ opinions and formulate an audio and video interaction schedule; remind relatives to conduct audio and video interactions with the elderly as planned, and automatically generate audio and video interaction content based on preset message templates or AI simulation templates when relatives fail to interact as planned;

[0129] Health data collection module: collects physiological data through smart health monitoring devices; collects behavioral activity data through cameras and microphones;

[0130] Health data analysis module: used to analyze the collected health information data and generate real-time health analysis information, including: physiological data processing unit: feature extraction and classification of physiological data based on convolutional neural network; behavioral data processing unit: feature analysis of behavioral activity data based on long short-term memory network; multimodal fusion unit: use attention mechanism to fuse and analyze physiological and behavioral data to generate comprehensive health status embedding vector; psychological state assessment unit: psychological assessment of health status embedding vector based on multi-layer perceptron; abnormality detection unit: detect abnormalities based on autoencoder and output abnormality reminder;

[0131] Health record management module: stores health information data and real-time health analysis information to form the health records of the elderly; generates health statistics based on daily, weekly and monthly data;

[0132] Health management advice module: Generates health management advice based on health records and real-time analysis information, including diet advice, exercise plans, psychological care advice and medical reminders;

[0133] Data encryption and privacy protection module: used for data encryption and privacy protection.

[0134] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for managing the health of the elderly based on audio and video of family affection, characterized in that: The following steps are involved: Step S1: obtaining the opinions of the relatives who are in contact with the target elderly by asking them, and formulating an audio and video interaction schedule based on the opinions; regularly reminding the relatives to conduct audio and video interaction with the elderly as planned according to the audio and video interaction schedule; the audio and video interaction schedule includes real-time audio and video interaction arrangements and preset reminder contents; Step S2: If the relative does not conduct audio and video interaction within the specified time, it is regarded as an abnormal interaction, and the preset audio and video template is used to automatically conduct audio and video interaction, and the voice and video data during the interaction are recorded to generate template audio and video information; Send the generated template audio and video information to the corresponding relatives to remind them to supplement the actual audio and video interaction; Collect statistics on abnormal interactions and optimize the audio and video interaction schedule; The automatic audio and video interaction using the preset audio and video template includes: Obtaining the interaction index of the relatives through the audio and video interaction plan table, and randomly selecting the type of preset audio and video templates according to the weighted interaction index, wherein the types of audio and video templates include preset message templates and AI simulation templates; Construction and use of preset message templates: obtain the daily needs of the target elderly and the care content provided by relatives, and pre-record voice or video clips for the elderly, the video clips include daily greetings, holiday blessings or health reminders; in the event of abnormal interaction, select the corresponding content from the preset message template according to the time information of the day and send it to the elderly; Construction and use of AI simulation templates: Set audio and video interaction scenarios under different scenarios, and construct audio and video samples for each relative of the target elderly. The audio and video samples include audio and video construction parameters and relative information; the audio and video construction parameters are obtained through family audio and video interaction or pre-collection, including voice parameters, portrait parameters and habit parameters; use audio and video samples in combination with preset audio and video interaction scenarios to generate AI simulation templates; the AI ​​simulation template generates audio and video to interact with the elderly by simulating the voice, portrait and behavioral habits of relatives; The interaction records using the preset message template or AI simulation template are stored as template audio and video information for subsequent statistical analysis and optimization; The types of preset audio and video templates randomly selected by weight according to the interaction index include: Obtaining the interaction index: Calculate the interaction index based on the historical records of audio and video interactions between relatives and the elderly. I , the calculation formula of the interaction index is: I=w 1 F+w 2 D+w 3 C ,in F Indicates the frequency of audio and video interaction between relatives and the elderly. D Indicates the duration of audio and video interaction between relatives and the elderly. C Indicates the richness of interactive content or the coverage of interactive scenarios; w 1 、w 2 、w 3 is the weight coefficient of the corresponding indicator, and satisfies w 1 +w 2 +w 3 =1 ; Weighted random selection method: Calculate the selection probability of the preset message template and the AI ​​simulation template. Suppose the selection probability of the preset message template is , the selection probability of AI simulation template is ,but and Satisfy the following formula: , ;in is the interaction index, It is a preset balance factor used to adjust the selection tendency of the two templates; Weighted random selection of template types: Generate a random number K in the interval [0,1) and calculate K and and The difference is calculated by comparing the absolute values ​​of the difference calculation results, and the template type corresponding to the selection probability with the smaller absolute value is selected for audio and video interaction; Step S3: During the audio and video interaction process, health information data of the elderly is collected; the health information data includes physiological data obtained by the smart health monitoring device and behavioral activity data obtained by the camera and microphone; Step S4: inputting the health information data into the elderly health data analysis model for calculation to obtain real-time health analysis information; the real-time health analysis information includes physiological indicators, behavioral indicators, psychological indicators or abnormal reminders; Step S5: storing the health information data and real-time health analysis information to generate the elderly's health records, and performing statistical analysis based on the daily, weekly and monthly data of the health records to generate health management suggestions; Step S6: Data encryption and privacy protection, through encrypted transmission and storage of health information data and audio and video records, setting access permissions for health records, and privacy protection through data anonymization and access record auditing.

2. The elderly health management method based on family audio and video according to claim 1 is characterized in that: The counting of abnormal interactions and optimization of the audio and video interaction schedule includes: Abnormal interaction statistics: record the relevant information of each abnormal interaction, including the time of occurrence, corresponding relatives, abnormal type and the audio and video template used; the abnormal types include no interaction and supplementary actual audio and video interaction; for abnormal records without supplementary actual audio and video interaction, count the number of abnormal interactions of each relative N e and abnormality rate R , the abnormal rate R The calculation formula is: R=N e / N t ,in N t is the total number of planned interactions in the same time period; Supplementary interaction statistics: distinguish the records of actual audio and video interactions that have been supplemented, and count the time intervals between supplementation completions T c and replenishment rate C r , the replenishment rate C r The calculation formula is: C r =N c / N e ,in N c The number of actual audio and video interactions that have been supplemented; Optimize the audio and video interaction schedule: adjust the interaction reminder time according to the statistical abnormal time distribution to avoid abnormal high-incidence periods; R For relatives whose number exceeds the set threshold, increase the frequency of interactive reminders or optimize the reminder method; T c For relatives whose abnormality rate exceeds the set threshold, the reminder content is improved, and the improvement includes adding detailed explanations or adding emergency reminders; dynamically adjusting the selection ratio of audio and video template types, giving priority to using AI simulation templates for relatives with high abnormality rates, and reducing plan dependence.

3. The elderly health management method based on family audio and video according to claim 1 is characterized in that: The elderly health data analysis model includes the following levels: Physiological data processing layer: Use the numerical submodule based on convolutional neural network to process the physiological data collected by the intelligent health monitoring equipment; extract the characteristics of the data through one-dimensional convolution operation, and combine the multi-layer fully connected network for classification and regression calculation to generate physiological indicator characteristics; the physiological indicator characteristics include but are not limited to heart rate variability, blood pressure volatility, and abnormal temperature detection values; Behavior activity data processing layer: Use the audio-visual submodule based on the long short-term memory network to process the behavior activity data collected by the camera and microphone; extract the speech speed, intonation and volume features of the voice data, analyze the action sequence features of the video data, and identify the behavior indicator features in combination with the time series model; the behavior indicator features include but are not limited to activity volume, gait analysis, and voice emotion tendency; The physiological and behavioral indicator features are output in the form of embedded vectors; Multimodal fusion layer: The embedded vectors output by the physiological data processing layer and the behavioral activity data processing layer are comprehensively processed through a multimodal fusion mechanism; Use the attention mechanism to weight the contributions of data from different sources and generate a comprehensive health status embedding vector; Output preliminary comprehensive analysis results as input to the psychological state assessment layer and anomaly detection layer; Psychological state assessment layer: Use the psychological submodule based on multi-layer perceptron to perform nonlinear mapping on the comprehensive health state embedding vector to infer the psychological indicator characteristics; the psychological indicator characteristics include emotional state and cognitive state; Abnormal detection layer: Use the abnormal detection submodule based on the autoencoder to judge the reconstruction error of physiological index characteristics, behavioral index characteristics and psychological index characteristics; if the reconstruction error exceeds the preset threshold, an abnormal reminder is output to indicate health risks or abnormal behaviors; The comprehensive output layer organizes the output results of each layer into real-time health analysis information and outputs it.

4. The elderly health management method based on family audio and video according to claim 3 is characterized in that: The construction of the elderly health data analysis model includes: Acquisition and division of data sets: obtain data sets containing physiological data, behavioral activity data, and psychological state labels from target elderly health monitoring devices, cameras, microphones, and interaction records; divide the data sets into training sets, validation sets, and test sets; preprocess the data, including missing value filling, outlier removal, and normalization; Training of numerical submodule: Use a pre-trained model based on a convolutional neural network to train a subset of physiological data; use the cross-entropy loss function to optimize model parameters, and use the learning rate scheduler to dynamically adjust the learning rate; evaluate model performance through a validation set, with indicators including mean square error and classification accuracy; and obtain the final numerical submodule; Training of the audio-visual submodule: Use a pre-trained model based on the long short-term memory network to train the activity data subset; use the cross entropy loss function in the classification task and the mean square error in the regression task to optimize the model; evaluate the model performance through the validation set; and obtain the final audio-visual submodule; Multimodal fusion: The output embedding vectors of the numerical submodule and the audio-visual submodule are multimodally fused through the attention mechanism. The fusion process dynamically adjusts the contribution values ​​of different modalities according to the weight of the data source to generate a comprehensive health status embedding vector. The parameters of multimodal fusion are optimized through the validation set to evaluate the accuracy of the fused model in assessing the psychological state. Construction of the anomaly detection submodule: Use the autoencoder to train the anomaly detection submodule on the comprehensive health status embedding vector; obtain the final anomaly detection submodule; Model performance evaluation: Evaluate the performance of each submodule on the test set, with indicators including classification accuracy, mean square error, F1 score, and accuracy of anomaly detection; adjust model parameters and complete the final model deployment.

5. The elderly health management method based on family audio and video according to claim 1 is characterized in that: Generating health management suggestions includes: Acquisition of health data analysis results: Acquisition of real-time health analysis information from the elderly health data analysis model; extraction of daily, weekly and monthly statistical analysis results based on historical data stored in health records; Health status assessment: Evaluate the health status of the elderly based on the trend comparison of real-time health analysis information and historical data; comprehensively score the physiological status, behavioral activities and psychological status; Health management advice generation: Generate targeted health management advice based on the results of health status assessment and abnormal reminders; the advice includes but is not limited to dietary advice, exercise plans, psychological care advice and medical advice; Dynamic adjustment of recommendations: Dynamically adjust health management recommendations based on the daily changes in the elderly’s health information data and implementation feedback; regularly update recommendations to adapt to changes in the elderly’s health status.

6. The elderly health management method based on family audio and video according to claim 1 is characterized in that: The data encryption and privacy protection include: Data encryption transmission and storage: Use symmetric encryption algorithms to perform end-to-end encryption transmission of health information data and audio and video records; use asymmetric encryption algorithms to encrypt and protect health records and interaction records stored in the cloud and locally; Access rights management: Set hierarchical access rights for health records and audio and video data, authorize different relatives and medical institutions to access specific ranges of data; use multi-factor authentication to ensure access security; Data anonymization: During statistical analysis and data sharing, health information data is anonymized to remove personal identity information associated with the elderly; voice and portrait data in audio and video recordings are blurred or desensitized to ensure privacy security; Access records and audits: Record detailed logs for each data access, modification, and use operation; audit logs regularly to detect and prevent unauthorized access.

7. Elderly health management system based on family audio and video, characterized by: The system applies any one of the above-mentioned elderly health management methods based on family audio and video in claims 1 to 6, and includes the following modules: Family audio and video interaction module: used to obtain relatives’ opinions and formulate an audio and video interaction schedule; remind relatives to conduct audio and video interactions with the elderly as planned, and automatically generate audio and video interaction content based on preset message templates or AI simulation templates when relatives fail to interact as planned; Health data collection module: collects physiological data through intelligent health monitoring equipment; Collect behavioral activity data through cameras and microphones; Health data analysis module: used to analyze the collected health information data and generate real-time health analysis information, including: physiological data processing unit: feature extraction and classification of physiological data based on convolutional neural network; behavioral data processing unit: feature analysis of behavioral activity data based on long short-term memory network; multimodal fusion unit: use attention mechanism to fuse and analyze physiological and behavioral data to generate comprehensive health status embedding vector; psychological state assessment unit: psychological assessment of health status embedding vector based on multi-layer perceptron; abnormality detection unit: detect abnormalities based on autoencoder and output abnormality reminder; Health record management module: stores health information data and real-time health analysis information to form the health records of the elderly; generates health statistics based on daily, weekly and monthly data; Health management advice module: Generates health management advice based on health records and real-time analysis information, including diet advice, exercise plans, psychological care advice and medical reminders; Data encryption and privacy protection module: used for data encryption and privacy protection.

Images