Question and answer type old people health management method and device

Abstract

The invention discloses a question and answer type old people health management method and device. The method comprises the following steps: acquiring video data of old people in a preset time period; behavior description features, state description features and emotion description features corresponding to the video data are input into a trained language model, the health state of the old people is determined, the behavior features are features used for describing behaviors of the old people in the preset time period, and the state description features are features used for describing emotion of the old people in the preset time period. The state description features are text description features corresponding to physiological conditions of the old people in the preset time period, and the emotion description features are features obtained by analyzing expressions of the old people in the preset time period; and generating health management suggestions of the old people based on the health state and the rehabilitation training scheme of the old people. According to the invention, the health conditions of the old people are monitored and analyzed in different dimensions, so that complex and diversified health management requirements of the old people are better met.

Description

Technical Field

[0001] This application relates to the field of intelligent healthcare, and particularly to a question-and-answer-based elderly health management method and its device. Background Art

[0002] With the development of the global social economy and the improvement of medical standards, the life expectancy of the population has been extended, and the proportion of the elderly population has been continuously increasing. The health and care needs of the elderly have become the focus of society. Although hospitals can provide inpatient care, facing the huge elderly population, resources are relatively scarce. Moreover, the elderly often suffer from multiple chronic and mental diseases, and family care is difficult to meet the long-term treatment and monitoring needs.

[0003] In this context, information technology has given rise to the health management of the elderly. However, traditional health management systems have many deficiencies: First, they adopt a standardized service model and cannot accurately meet the personalized needs of the elderly due to differences in physical fitness, past medical history, lifestyle, and health needs. Second, they adopt a passive response strategy and only intervene when the elderly actively seek help. They can neither actively detect potential problems nor provide the humanistic care of active communication and interaction with the elderly. Third, data collection is limited to physiological data such as heart rate and blood pressure, and it is difficult to comprehensively and accurately depict the overall health of the elderly. Therefore, it is urgent to innovate and optimize the elderly health management system to better meet the complex and diverse health management needs of the elderly. Summary of the Invention

[0004] The embodiments of this application provide a question-and-answer-based elderly health management method and its device, which are used to solve at least the above-mentioned technical problems.

[0005] The embodiments of this application provide a question-and-answer-based elderly health management method, including: obtaining the video data of the elderly within a preset time period, where the video data includes image data and audio data; inputting the behavior description features, state description features, and emotion description features corresponding to the video data into a trained language model to determine the health status of the elderly, where the behavior description features are the features used to describe the behavior of the elderly within the preset time period, the state description features are the text description features corresponding to the physiological conditions of the elderly within the preset time period, and the emotion description features are the features obtained by analyzing the emotions of the elderly within the preset time period; generating health management suggestions for the elderly based on the health status of the elderly and the rehabilitation training plan.

[0006] Optionally, based on the health status of the elderly and the rehabilitation training plan, health management suggestions for the elderly are generated, including: after sending the health status of the elderly to the medical personnel matched with the elderly, receiving the health management suggestions provided by the medical personnel, where the health management suggestions are the suggestions given by the medical personnel based on the health status and rehabilitation training plan of the elderly.

[0007] Optionally, after determining the health status of the elderly, it further includes: in the case where it is determined that the health status of the elderly is abnormal, sending the health status of the elderly to the family terminal of the elderly.

[0008] Optionally, the behavioral description feature is a feature obtained through a trained behavior recognition model, and the trained behavior recognition model can extract the behavioral description feature by using the changes in the joint point data of the elderly within the preset time period.

[0009] Optionally, the state description feature is obtained through the following operations:

[0010] Obtain the physiological data of the elderly collected by the data acquisition device;

[0011] Input the physiological data of the elderly into a trained text model to obtain the state description feature.

[0012] Optionally, the emotion description feature is obtained through the following steps: after performing frame division processing on the video data, obtain image data and audio data with timestamps; use the image emotion recognition result and audio emotion recognition result obtained at each timestamp to determine the emotion description feature representing the preset time period.

[0013] Optionally, input the behavioral description feature, state description feature, and emotion description feature corresponding to the video data into the trained language model to determine the health status of the elderly, including: input the behavioral description feature, the state description feature, and the emotion description feature into the feature fusion module to obtain the fusion feature representing the elderly within the preset time period; send the fusion feature to the trained language model to determine the health status of the elderly.

[0014] The embodiment of the present application further provides a question-and-answer type health management device, and the device includes: a processor; and a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the above method.

[0015] The embodiment of the present application further provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed, the above method is implemented.

[0016] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects:

[0017] According to the Q&A-based elderly health management method of the exemplary embodiments of the present application, by integrating behavior description features, state description features, and emotion description features, it breaks through the limitation of traditional health assessment that only relies on physiological indicators. It can not only master the physical condition of the elderly, but also pay attention to the impact of daily behavior habits and emotional changes on health, realizing a more comprehensive and accurate assessment of the health status of the elderly. For example, it can analyze the change of joint point data through a behavior training model to timely detect the decline of the elderly's daily activity ability; use an emotion prediction model to insight into the long-term anxiety and loneliness emotions of the elderly and prevent health hazards caused by psychological problems in advance. In addition, the Q&A-based elderly health management method of the present application fully considers the differences of each elderly person in terms of living habits, underlying diseases, physical functions, etc. Based on the health status determined by multi-dimensional data and combined with a rehabilitation training plan, it provides personalized health management suggestions for the elderly. Whether it is a pension institution formulating a dedicated diet and exercise plan, or a community health service center giving targeted disease prevention and rehabilitation guidance, it can better meet individual needs. Further, the method can collect real-time health data of the elderly by means of video data and physiological data collection devices. Once an abnormal health status is determined, the information can be quickly sent to the family terminal for timely measures to be taken. In a pension institution, the elderly with abnormal health can be quickly screened out and intervened, and children in the family can also understand the health of the elderly through a mobile application at any time, greatly improving the timeliness of health management. In addition, the method can also send the health status of the elderly to the matching medical staff, who can give health management suggestions based on the health status and rehabilitation training plan, effectively integrating professional medical resources. The community health service center and the community hospital are linked to realize the rapid referral and follow-up service of the elderly with abnormal health, so that the elderly can receive more professional and systematic medical protection. Finally, the entire health management process automatically completes data collection, analysis, and generation of health management suggestions through a variety of training models and intelligent devices, greatly reducing labor costs and time costs. At the same time, the organic integration of multi-dimensional data and the powerful data analysis ability of the language model can quickly obtain health assessment results, significantly improving the efficiency and quality of elderly health management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0019] Figure 1 is a schematic diagram showing a Q&A-based elderly health management system according to an embodiment of the present application.

[0020] Figure 2 is a schematic flowchart showing a question-and-answer based elderly health management method according to an embodiment of the present application.

[0021] Figure 3 is a schematic diagram showing the acquisition of health status according to an exemplary embodiment of the present application.

[0022] Figure 4 is a block diagram showing a question-and-answer based elderly health management device according to an exemplary embodiment of the present application. Detailed Description of the Embodiments

[0023] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0024] The following will describe in detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0025] Figure 1 is a schematic diagram showing a question-and-answer based elderly health management system according to an embodiment of the present application. The question-and-answer based elderly health management system at least includes a user terminal, a medical terminal, a data processing device (e.g., a central processing unit), and various data acquisition devices. These devices / terminals have connection functions and can communicate with each other.

[0026] The data acquisition device may include various devices for acquiring user data. As Figure 1 shown, the data acquisition device may include multiple sensors, for example, the first sensor to the Nth sensor, where N is greater than 2. These sensors may include various wearable devices or implantable devices for collecting physiological data of the elderly, such as heart rate, blood pressure, body temperature, blood glucose, blood oxygen saturation, etc. Different sensors can monitor different physiological indicators, providing multi-dimensional health information for the system. These sensors can be fixed at key parts such as the wrist, ankle, knee, and waist, facilitating the elderly to wear them naturally during daily activities or rehabilitation training without imposing too much restraint on their movements.

[0027] The data acquisition device may also include an image capture device, such as a camera or a monitor, for capturing the behavior images of the elderly. Through image analysis, information such as the activity status, posture, and facial expression of the elderly can be understood, which helps to judge their physical condition and emotional state.

[0028] The data acquisition device may further include an audio picker, which can collect the voice information of the elderly, such as speaking voice, coughing sound, breathing sound, etc. These audio data can be used to analyze the language ability, emotional changes and possible respiratory health problems of the elderly, etc.

[0029] In addition to the data acquisition device mentioned above, the question-and-answer elderly health management system can also obtain various data from user terminals and medical terminals. Specifically, the medical terminal refers to the terminal of the medical resources matched with the elderly, including but not limited to the terminals of medical staff and hospital servers. The medical terminal can provide professional medical data such as the medical records, diagnosis results, treatment plans, etc. of the elderly. In addition, the medical terminal can also provide further medical decisions and interventions according to the physical status of the elderly.

[0030] The user terminal can refer to the terminal used by the elderly and also the terminal used by the family members of the elderly. In an embodiment, the user terminal can refer to the terminals of the elderly and their authorized family members. In this way, the question-and-answer elderly health management system can utilize the user terminal to provide some personal information. For example, the age, gender, illness situation, exercise habits, work type, etc. of the elderly. In addition, the question-and-answer elderly health management system can utilize the user terminal to provide some subjective information, such as the daily feelings of the elderly, changes in living habits, etc., and at the same time will also receive the health care suggestions given by the system.

[0031] As Figure 1 shown, the question-and-answer elderly health management system may include a data processing device. The data processing device can be the server (processor) of the question-and-answer elderly health management application. The data processing device can receive various data from the data acquisition device, and process and analyze these data.

[0032] As an example, the data processor includes data preprocessing, behavior recognition algorithm, emotion prediction algorithm, rehabilitation training evaluation algorithm, large language model. Specifically, the data preprocessing can preliminarily process the data from various sensors, image capture devices and audio pickers, including data cleaning (removing noise, outliers, etc.), format conversion, data integration, etc., to ensure the quality and usability of the data, and provide accurate input for subsequent algorithm analysis.

[0033] The behavior recognition algorithm refers to analyzing the behavior patterns of the elderly, such as walking gait, sitting and standing postures, frequency and duration of daily activities, etc., so as to judge the physical activity ability of the elderly and whether there are abnormal behaviors, such as falling, sedentary behavior, etc. The emotion prediction algorithm refers to combining data such as sound information collected by the audio picker and facial expressions obtained by the image capture device, and using the emotion prediction algorithm to analyze the emotional state of the elderly, such as whether they are happy, anxious, depressed, etc. This is very important for paying attention to the mental health of the elderly, because the emotional state will affect physical health and quality of life. The rehabilitation training evaluation algorithm can evaluate the effect of rehabilitation training according to the motion data collected by the sensor and other relevant information, such as the improvement of joint range of motion, the enhancement of muscle strength, etc., and provide a basis for adjusting the rehabilitation training plan. The large language model can, based on the results of various data and algorithms, make a more comprehensive and accurate prediction and analysis of the health status of the elderly, such as predicting the risk of disease occurrence, health trends, etc., and provide strong support for formulating personalized health care suggestions.

[0034] Figure 2 is a schematic flowchart showing a question-and-answer type elderly health management method according to an embodiment of the present application. The method includes step S210, step S220, and step S230.

[0035] In step S210, video data of the elderly within a preset time period is obtained, where the video data includes image data and audio data.

[0036] According to an exemplary embodiment of the present application, the preset time period can be a time set by the elderly or medical staff, for example, one hour. In addition, in order to more accurately reflect the situation of the elderly, the preset time period can be selected as the time period when the elderly perform exercise rehabilitation training. The method can perform the above operations every once in a while, and the execution frequency can be determined according to the actual situation of the elderly. For example, the method can obtain the video data of the elderly from 9:00 am to 10:00 am every day.

[0037] In step S220, the behavior description features, state description features, and emotion description features corresponding to the video data are input into the trained language model to determine the health status of the elderly, where the behavior description features are features used to describe the behavior of the elderly within the preset time period, the state description features are text description features corresponding to the physiological conditions of the elderly within the preset time period, and the emotion description features are features obtained by analyzing the emotions of the elderly within the preset time period.

[0038] For better description, the following will refer to Figure 3 Describe step S220 in detail. As Figure 3As shown, the method can obtain video data within a preset time period. For example, within the preset time period, through a depth camera installed in a specific area, the video data contains the joint point data sequence of the elderly. As an example, in an exemplary embodiment of the present application, the elderly can also wear wearable devices (such as bracelets with sensors, clothing, etc.) to obtain the joint point data of the elderly. These devices can accurately track the three-dimensional coordinate positions of each joint point of the human body (such as the shoulder joint, elbow joint, wrist joint, hip joint, knee joint, ankle joint, etc.) in space, continuously record the position information of the joint points at different times at a relatively high sampling frequency, and form a joint point data sequence.

[0039] Then, the method uses the trained behavior recognition model to process the joint point data sequence to obtain behavior description features. During the training process, the method can use a large amount of joint point data containing different behavior labels as training samples. For example, collect the joint point data of various common behaviors of the elderly, such as normal walking, sitting down, standing up, falling, etc., and manually label the corresponding behavior categories. Use architectures such as recurrent neural networks (RNN), long short-term memory networks (LSTM), or convolutional neural networks (CNN) in deep learning to train these sample data. The model establishes a mapping relationship between the behavior pattern and the joint point data by learning features such as the change pattern, movement trajectory, speed, and acceleration of the joint point data under different behaviors.

[0040] After the model is trained, the joint point data of the elderly collected within the preset time period is input into the behavior recognition model. The model will analyze the changes in the joint point data during this period and extract behavior description features. As an example, the behavior recognition model can output the joint point movement trajectory features, joint angle change features, and / or movement speed and acceleration features of the elderly. Among them, the joint point movement trajectory feature is to calculate the movement trajectory of each joint point in space. For example, the trajectory curve of the knee joint during walking can reflect information such as the size of the walking step and the step frequency. The joint angle change feature is to analyze the change in the angle between joints. For example, the change in the elbow joint angle when the arm is raised can judge the movement amplitude and posture of the arm. The movement speed and acceleration feature is to calculate the speed and acceleration of the joint point at different times to understand the speed and intensity changes of the action. For example, the speed of the getting-up action can reflect the flexibility of the elderly's body.

[0041] According to an exemplary embodiment, the emotion description feature is obtained through the following steps: input the video data into the trained emotion prediction model to obtain the emotion description feature of the emotional change of the elderly within the preset time period.

[0042] Specifically, the method can perform data preprocessing on the image data and speech data in the video data respectively. In implementation, the method can perform frame-by-frame processing on the video images to obtain image data and audio data with timestamps. For the image data, the method can use image enhancement algorithms to improve the clarity and contrast of the images and remove noise interference to ensure the accuracy of subsequent facial expression analysis. For the speech data, a noise reduction algorithm is adopted to remove ambient noise and audio interference, and the audio is normalized to have a unified volume and sampling rate, laying a foundation for subsequent speech analysis.

[0043] Subsequently, the method can use a convolutional neural network (CNN) model based on deep learning to perform emotion recognition on the frame images corresponding to each timestamp. By learning a large number of facial expression images with different emotion labels, this model can accurately identify emotion states such as happy, anxious, and depressed, thereby obtaining the image emotion recognition results corresponding to each timestamp. For example, when a facial expression with an upturned mouth and squinted eyes is detected, it is determined as a happy emotion; while a frowning brow and wandering eyes may correspond to an anxious emotion.

[0044] According to an exemplary embodiment of the present application, the method can convert the audio data with each timestamp and after preprocessing into corresponding text data, and then the text data can be converted into a text vector and an emotion vector representing the emotion tendency of the text data, and use the text vector and the emotion vector to generate an emotion recognition feature vector

[0045] Specifically, the method can use advanced automatic speech recognition (ASR) technology to convert the audio data with each timestamp and after preprocessing into corresponding text data. Automatic speech recognition systems are usually based on deep learning models, such as the Transformer-Transducer model based on the Transformer architecture or the model improved from the recurrent neural network (RNN) with the long short-term memory network (LSTM) combined with the connectionist temporal classification (CTC) loss function. These models are trained on a large number of speech data sets to learn the mapping relationship between the acoustic features in the audio signal and the text. When the input audio data is provided, the model can, according to the learned patterns, segment the continuous audio stream into individual speech units and identify the corresponding text content, thereby realizing the conversion from audio to text.

[0046] Subsequently, the method can convert the text data into text vectors. Specifically, the method can adopt word embedding techniques in natural language processing (NLP), such as Word2Vec or GloVe. Taking Word2Vec as an example, it constructs a language model and trains it on a large text corpus. During the training process, the model learns the semantic relationships of each word in the context and finally maps each word to a vector with a fixed dimension, that is, a word vector. For a piece of text data, by combining the word vectors corresponding to each word in a certain way, such as a simple average pooling operation, a text vector representing this piece of text can be obtained. This text vector can reflect the semantic features of the text to a certain extent, and the numerical value of each dimension in the vector contains the semantic information of the text.

[0047] After obtaining the text vector, the method can use sentiment analysis algorithms to determine the sentiment vector representing the emotional tendency of the text data. It can be based on pre-trained sentiment classification models, such as sentiment classifiers based on convolutional neural networks (CNNs) or recurrent neural networks (RNNs). These models are trained on a dataset containing a large number of texts with sentiment labels (such as positive, negative, neutral), learning the emotional features in the texts. When the input text data is provided, the model judges the emotional tendency of the text according to the learned feature patterns and outputs a vector representing the emotional tendency. For example, for a text with a positive emotion, the vector may have a higher value in the positive emotion dimension and lower values in the negative and neutral dimensions; the opposite is true for a text with a negative emotion.

[0048] Finally, the method can fuse the obtained text vector and sentiment vector to generate an emotion recognition feature vector. As an example, the method can concatenate the text vector and the sentiment vector to obtain the emotion recognition feature vector.

[0049] The method can obtain the final emotion recognition result corresponding to each time stamp according to the image emotion recognition result and the speech emotion recognition result. In implementation, the method can assign different weights to the image emotion recognition result and the speech emotion recognition result. For example, if facial expressions account for a relatively large proportion in emotion expression, the weight of the image emotion recognition result can be appropriately increased; if the speech content is more critical for emotion judgment, the weight of the speech emotion recognition result can be increased. By means of weighted summation and other methods, the final emotion recognition result corresponding to each time stamp is obtained.

[0050] Finally, the method can obtain the emotion description features of the emotional changes of the elderly within the preset time period according to the final emotion recognition result corresponding to each time stamp.

[0051] As an example, the method can summarize and organize the final emotion recognition results for each timestamp, integrating emotion categories (such as happy, anxious, depressive, etc.) and the corresponding time information into a data sequence. By observing this data sequence, one can intuitively see the emotional states of the elderly at different moments.

[0052] Next, the method can calculate relevant indicators of emotional changes to construct emotional description features. For example, count the frequency of each emotional state occurring within a preset time period. An emotion with a higher frequency can reflect the dominant emotion of the elderly during this period. At the same time, calculate the number of emotion conversions, such as the number of conversions from happy to anxious, which can reflect the emotional stability of the elderly. If the emotion conversions are frequent, it indicates that their emotional stability is poor and there may be psychological fluctuations.

[0053] Finally, analyze the duration of the emotion. For each emotional state, record the time of its first appearance and the last disappearance, so as to calculate the duration of this emotion. Being in a certain negative emotion (such as depression) for a long time may imply that the elderly have relatively serious psychological problems.

[0054] In addition, the intensity change of the emotion can also be calculated. If the emotion recognition result not only includes the emotion category but also quantitative information on the emotion intensity (for example, from mild anxiety to severe anxiety), then the rising and falling trends of the emotion intensity within a preset time period, as well as the highest and lowest intensities reached, can be analyzed.

[0055] Finally, combine these calculated indicators, such as emotion frequency, number of conversions, duration, intensity change, etc., into a feature vector according to certain rules. This vector is the emotional description feature of the emotional changes of the elderly within the preset time period obtained.

[0056] According to an exemplary embodiment of the present application, the state description feature is obtained through the following operations: obtaining the physiological data of the elderly collected by the data acquisition device; inputting the physiological data of the elderly into a trained text model to obtain the state description feature. Specifically, in order to convert physiological data into meaningful state description features, a text model needs to be trained first. The training data comes from a wide range of sources, covering the physiological data of a large number of different elderly people and the corresponding professional medical description texts. These medical description texts are written by professional doctors or health experts based on the physiological data, and detail the health state meanings represented by each physiological data. For example, a too high heart rate may indicate an excessive heart load, and insufficient sleep duration may affect physical recovery, etc.

[0057] During the training process, natural language processing (NLP) techniques in deep learning are used, such as pre-trained language models based on the Transformer architecture, such as variants of GPT-3 or BERT. These models are capable of learning the complex mapping relationships between physiological data and medical description texts. Through repeated learning of a large amount of training data, the model gradually masters how to extract key information from the given physiological data and generate accurate text descriptions.

[0058] As Figure 3 shown, input the behavior description features, state description features, and emotion description features corresponding to the video data into the trained language model to determine the health status of the elderly, including: input the behavior description features, the state description features, and the emotion description features into the feature fusion module to obtain the fusion features representing the elderly within the preset time period; send the fusion features to the trained language model to determine the health status of the elderly.

[0059] As an example, the behavior description features, state description features, and emotion description features may differ in format, dimension, and numerical range. For example, the behavior description features may be represented as a numerical sequence of joint point movements, the state description features may be vectors after text transformation, and the emotion description features may be probability vectors based on facial expression analysis. Therefore, before fusion, these features need to be preprocessed. For numerical features, normalization or standardization methods are used to map them to the same numerical range to ensure relatively balanced weights of different features in subsequent calculations.

[0060] Then, the method can input the three preprocessed features into the feature fusion module. Common fusion methods include concatenation fusion, that is, concatenating the behavior description feature vector, state description feature vector, and emotion description feature vector in sequence from start to end to form a longer fusion feature vector. In addition, weighted fusion can also be used, and corresponding weights are assigned to each feature according to the importance of different features for judging the health status of the elderly.

[0061] According to the exemplary embodiments of the present application, before inputting the fusion features into the language model, the language model has been trained on a large amount of data related to the health of the elderly. These data include the behavior, physiological, and emotion features of different elderly people, as well as the corresponding accurate health status labels, which are determined by professional medical staff based on comprehensive health examinations and diagnoses. During the training process, the language model learns the complex mapping relationship between features and health status. As an example, a language model using the Transformer architecture is used, and its self-attention mechanism can capture the long-range dependencies between different parts of the fusion features. Through multiple layers of encoding and decoding operations, the model parameters are continuously optimized to accurately predict the health status.

[0062] Send the obtained fused features to the pre-trained language model. The language model deeply analyzes the fused features and predicts the health status of the elderly within a preset time period based on the patterns and knowledge learned during its training. The health status may be presented in various forms, such as classification results like healthy, sub-healthy, increased risk of having a certain chronic disease, or a continuous health score reflecting the relative level of the elderly's health condition. For example, the model outputs "The elderly is currently in a sub-healthy state. Due to the recent decrease in activity level, low mood, and slightly elevated blood pressure, it is recommended to increase appropriate exercise, maintain a good mental state, and regularly monitor blood pressure", providing clear directions and suggestions for subsequent health management.

[0063] In step S230, generate health management suggestions for the elderly based on the health status of the elderly and the rehabilitation training plan.

[0064] As an example, based on the existing rehabilitation training plan of the elderly, adjust the rehabilitation training plan according to the determined health status of the elderly. As an example, if the health status shows that the elderly has abnormal behavior and becomes anxious, it indicates that the current rehabilitation training may be too intense or inappropriate in method, causing an additional burden on the body and affecting the mental state. Based on this, the health management suggestion can be to reduce the intensity of the rehabilitation training.

[0065] Optionally, generate health management suggestions for the elderly based on the health status of the elderly and the rehabilitation training plan, including: after sending the health status of the elderly to the medical staff matched with the elderly, receive the health management suggestions provided by the medical staff, and the health management suggestions are the suggestions given by the medical staff based on the health status and rehabilitation training plan of the elderly.

[0066] Optionally, after determining the health status of the elderly, it further includes: in the case where it is determined that the health status of the elderly is abnormal, send the health status of the elderly to the family terminal of the elderly. As an example, the method can convey the generated health management suggestions to the elderly and their families in various ways, such as paper reports, mobile application push notifications, regular health lectures, etc. Ensure that the elderly and their families understand the content of the suggestions and can actively cooperate with the implementation. At the same time, establish a health management tracking mechanism, regularly collect the health data of the elderly, and evaluate the implementation effect of the suggestions. According to the changes in the health status of the elderly, timely adjust the health management suggestions to form a dynamic and continuous health management process, truly safeguarding the health of the elderly.

[0067] The Q&A-based elderly health management method according to the exemplary embodiments of the present application breaks the limitation of traditional health assessment that only relies on physiological indicators by integrating behavior description features, state description features, and emotion description features. It can not only understand the physical condition of the elderly, but also pay attention to the impact of daily behavior habits and emotional changes on health, achieving a more comprehensive and accurate assessment of the health status of the elderly. For example, it can analyze the changes in joint point data through a behavior training model to timely detect the decline in the daily activity ability of the elderly; use an emotion prediction model to insight into the long-term anxiety and loneliness of the elderly and prevent health risks caused by psychological problems in advance. In addition, the Q&A-based elderly health management method of the present application fully considers the differences of each elderly person in terms of living habits, underlying diseases, physical functions, etc. Based on the health status determined by multi-dimensional data and combined with a rehabilitation training plan, it provides personalized health management suggestions for the elderly. Whether it is a pension institution formulating a dedicated diet and exercise plan, or a community health service center giving targeted disease prevention and rehabilitation guidance, it can better meet individual needs. Further, the method can use video data and physiological data collection devices to realize real-time collection of the health data of the elderly. Once an abnormal health status is determined, the information can be quickly sent to the family terminal for timely measures to be taken. In a pension institution, the elderly with abnormal health can be quickly screened out and intervened, and children in the family can also understand the health of the elderly through a mobile application at any time, greatly improving the timeliness of health management. In addition, the method can also send the health status of the elderly to the matched medical staff, who can give health management suggestions based on the health status and rehabilitation training plan, effectively integrating professional medical resources. The community health service center and the community hospital are linked to realize the rapid referral and follow-up service of the elderly with abnormal health, enabling the elderly to receive more professional and systematic medical protection. Finally, the entire health management process automatically completes data collection, analysis, and generation of health management suggestions through a variety of training models and intelligent devices, greatly reducing labor costs and time costs. At the same time, the organic integration of multi-dimensional data and the powerful data analysis ability of the language model can quickly obtain health assessment results, significantly improving the efficiency and quality of elderly health management.

[0068] According to an exemplary embodiment of the present application, the question-and-answer elderly health management system can identify that the activity level of a certain elderly person has recently decreased, actively push exercise incentive information, and provide suitable exercise suggestions. Specifically, the question-and-answer elderly health management system collects a series of data through a variety of data collection devices arranged in the surrounding environment and daily activity areas of a certain elderly person, such as smart bracelets, smart insoles, and environmental monitoring devices. Through complex algorithm analysis, the system determines that the activity level of this elderly person has decreased significantly. For example, the elderly person used to have an average of 6,000 steps per day, but the average number of steps per day has dropped to 3,000 in the past two weeks, and the activity time has also been significantly shortened. Further analysis reveals that the elderly person has been sitting for longer periods, and it is very likely that there have been changes in their physical or mental state.

[0069] Based on this, the system takes prompt action. It pushes exercise incentive information to the elderly person's smart terminal: "Uncle, you have always been a symbol of vitality. Is there anything uncomfortable recently?" In addition, the system customizes exclusive exercise suggestions for him by combining information such as the elderly person's age, physical condition, and past medical history. Considering that the elderly person has poor knees, the exclusive exercise suggestions can recommend low-intensity exercises, such as taking a 20-30 minute slow walk on a flat community road at around 10 am every day, with a moderate stride and a speed that allows for easy conversation. Simple indoor stretching exercises, such as standing forward bend and arm stretching, can also be done from 3 to 4 pm, repeating each action 10-15 times to help relax muscles and increase joint flexibility.

[0070] According to an exemplary embodiment of the present application, when the question-and-answer elderly health management system monitors that the blood pressure value of a certain elderly person exceeds the normal range through a bound smart blood pressure monitor or a blood pressure monitor embedded in a wearable device, such as when the systolic blood pressure is higher than 140 mmHg multiple times or the diastolic blood pressure is higher than 90 mmHg, it will quickly activate the reminder mechanism. The question-and-answer elderly management system will send a reminder in the form of voice broadcast on the elderly person's smart terminal: "Your blood pressure is abnormal. Please pay attention in time." At the same time, detailed pop-up messages and text messages are sent to the elderly person's and their family members' mobile phones, including the specific current blood pressure value, a description of the abnormal situation, and a suggestion to go to a nearby medical institution for treatment as soon as possible. In addition, the question-and-answer elderly health management system also sends the abnormal situation of this elderly person to the medical terminal matched with the elderly person. Moreover, the large prediction model of the question-and-answer elderly health management system can give suggestions regarding diet based on the physical condition of the elderly person. For example, the question-and-answer elderly health management system can give the suggestion "It is recommended that you reduce your salt intake and eat more potassium-rich foods."

[0071] Figure 4 Block diagram of a question-and-answer elderly health management device showing an exemplary embodiment of the present application. Refer toFigure 4 At the hardware level, the device includes a processor, an internal bus, and a computer-readable storage medium. Among them, the computer-readable storage medium includes a volatile memory and a non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory and then runs it. Of course, in addition to the software implementation, this application does not exclude other implementation methods, such as logical devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, but can also be hardware or logical devices.

[0072] Specifically, the processor performs the following operations: obtaining video data of the elderly within a preset time period, where the video data includes image data and audio data; inputting the behavior description features, state description features, and emotion description features corresponding to the video data into a trained language model to determine the health status of the elderly, where the behavior description features are features used to describe the behavior of the elderly within the preset time period, the state description features are text description features corresponding to the physiological conditions of the elderly within the preset time period, and the emotion description features are features obtained by analyzing the emotions of the elderly within the preset time period; generating a health management recommendation for the elderly based on the health status of the elderly and the rehabilitation training plan.

[0073] Optionally, the processor further performs the following operations: after sending the health status of the elderly to the medical staff matched with the elderly, receiving the health management recommendation provided by the medical staff, where the health management recommendation is the recommendation given by the medical staff based on the health status of the elderly and the rehabilitation training plan.

[0074] Optionally, the processor further performs the following operations: in the case where it is determined that the health status of the elderly is abnormal, sending the health status of the elderly to the family terminal of the elderly.

[0075] Optionally, the behavior description features are features obtained by a trained behavior recognition model, and the trained behavior recognition model can extract the behavior description features by using the changes in the joint point data of the elderly within the preset time period.

[0076] Optionally, the state description features are obtained through the following operations: obtaining the physiological data of the elderly collected by a data collection device; inputting the physiological data of the elderly into a trained text model to obtain the state description features.

[0077] Optionally, the emotional description feature is obtained through the following steps: input the video data into a trained emotion prediction model to obtain the emotional description feature of the emotional changes of the elderly within the preset time period.

[0078] Optionally, the processor performs the following steps: input the behavior description feature, the state description feature, and the emotional description feature into a feature fusion module to obtain a fusion feature representing the elderly within the preset time period; send the fusion feature to a trained language model to determine the health status of the elderly.

[0079] In summary, the question-and-answer type elderly health management device according to the exemplary embodiment of the present application breaks the limitation of traditional health assessment that only relies on physiological indicators by integrating behavior description features, state description features, and emotional description features. It can not only master the physical condition of the elderly, but also pay attention to the impact of daily behavior habits and emotional changes on health, realizing a more comprehensive and accurate assessment of the health status of the elderly. For example, it can analyze the changes in joint point data through a behavior training model to timely detect the decline in the daily activity ability of the elderly; use an emotion prediction model to insight into the long-term anxiety and loneliness emotions of the elderly and prevent health hazards caused by psychological problems in advance. In addition, the question-and-answer type elderly health management method of the present application fully considers the differences of each elderly person in terms of living habits, underlying diseases, physical functions, etc. Based on the health status determined by multi-dimensional data and combined with a rehabilitation training plan, it provides personalized health management suggestions for the elderly. Whether it is a pension institution formulating a dedicated diet and exercise plan, or a community health service center giving targeted disease prevention and rehabilitation guidance, it can better meet individual needs. Further, the method can use video data and physiological data collection devices to realize the real-time collection of the health data of the elderly. Once it is determined that the health status is abnormal, the information can be quickly sent to the family terminal for timely measures to be taken. In a pension institution, the elderly with abnormal health can be quickly screened out and intervened, and children in the family can also understand the health of the elderly through a mobile application at any time, greatly improving the timeliness of health management. In addition, the method can also send the health status of the elderly to the matching medical staff, who can give health management suggestions based on the health status and the rehabilitation training plan, effectively integrating professional medical resources. The community health service center and the community hospital are linked to realize the rapid referral and follow-up service of the elderly with abnormal health, enabling the elderly to receive more professional and systematic medical protection. Finally, the entire health management process automatically completes data collection, analysis, and generation of health management suggestions through various training models and intelligent devices, greatly reducing labor costs and time costs. At the same time, the organic integration of multi-dimensional data and the powerful data analysis ability of the language model can quickly obtain health assessment results, significantly improving the efficiency and quality of elderly health management.

[0080] It should be noted that the execution entity of each step of the method provided in Embodiment 1 can be the same device, or the method can also be executed by different devices. For example, the execution entity of steps 21 and 22 can be Device 1, and the execution entity of step 23 can be Device 2; or, the execution entity of step 21 can be Device 1, and the execution entities of steps 22 and 23 can be Device 2; and so on.

[0081] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0083] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the specified functions in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0085] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0086] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0087] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0089] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0090] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A question-and-answer type elderly health management method, characterized in that: include: Acquire video data of the elderly within a preset time period, wherein the video data includes image data and audio data; Inputting the behavior description features, state description features and emotion description features corresponding to the video data into the trained language model to determine the health status of the elderly, wherein the behavior description features are features used to describe the behavior of the elderly within the preset time period, the state description features are text description features corresponding to the physiological conditions of the elderly within the preset time period, and the emotion description features are features obtained by analyzing the emotions of the elderly within the preset time period; Based on the health status and rehabilitation training program of the elderly, health management suggestions for the elderly are generated.

2. The method according to claim 1, characterized in that Based on the health status and rehabilitation training program of the elderly, health management suggestions for the elderly are generated, including: After the health status of the elderly is sent to the medical staff matched with the elderly, the health management suggestions provided by the medical staff are received. The health management suggestions are suggestions given by the medical staff based on the health status and rehabilitation training plan of the elderly.

3. The method according to claim 1, characterized in that After determining the health status of the elderly, the method further includes: When it is determined that the health status of the elderly is abnormal, the health status of the elderly is sent to a terminal of a family member of the elderly.

4. The method according to claim 1, characterized in that The behavior description feature is a feature obtained through a trained behavior recognition model, and the trained behavior recognition model can extract the behavior description feature by utilizing the changes in the joint point data of the elderly within the preset time period.

5. The method according to claim 1, characterized in that The state description feature is obtained by the following operations: Acquiring physiological data of the elderly collected by a data collection device; The physiological data of the elderly is input into a trained text model to obtain the state description features.

6. The method according to claim 1, characterized in that The emotion description feature is obtained by the following steps: Processing the video data by framing to obtain image data and audio data with timestamps; The emotion description features representing the preset time period are determined by using the image emotion recognition results and the audio emotion recognition results obtained at each timestamp.

7. The method according to claim 1, characterized in that the behavior description features, state description features and emotion description features corresponding to the video data are input into a trained language model to determine the health status of the elderly, comprising: Inputting the behavior description feature, the state description feature and the emotion description feature into a feature fusion module to obtain a fusion feature representing the elderly person in the preset time period; The fused features are sent to a trained language model to determine the health status of the elderly.

8. A question-and-answer type elderly health management device, characterized in that: include: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the question-and-answer type elderly health management method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed, the question-and-answer type elderly health management method described in any one of claims 1 to 7 is implemented.

Images