State monitoring and early warning method and system for elderly people living alone

Abstract

The invention belongs to the technical field of state monitoring of elderly people living alone, and particularly relates to a state monitoring and early warning method and system for elderly people living alone. According to the specific technical scheme, the method comprises the following steps of: acquiring multi-source behavior state data when the elderly living alone uses different mobile phone applications in different seasons, different weathers and different postures by utilizing a plurality of sensors built in the smart phone, and acquiring corresponding physiological state data; establishing a personalized physiological state prediction model according to the multi-source behavior state data and the physiological state data; acquiring normal physiological status characteristics of the elderly living alone when using different mobile phone applications in different seasons, different weathers and different postures; acquiring daily mobile phone use record data of the elderly people living alone in different seasons, and generating a daily mobile phone use work and rest table of the elderly people living alone in different seasons; the physiological state of the elderly living alone is monitored by using the work and rest table according to the mobile phone in different seasons. Multi-dimensional data such as mobile phone use habits, environmental perception and physiological states of the elderly living alone are deeply integrated, real-time behavior states and physiological states of the elderly living alone are accurately analyzed, abnormity is timely detected, corresponding rescue processes are rapidly started, and the personalized state monitoring and early warning method for the elderly living alone is provided.

Description

Technical Field

[0001] The present invention belongs to the technical field of status monitoring for elderly people living alone, and specifically relates to a status monitoring and early warning method and system for elderly people living alone. Background Technology

[0002] The aging of the population is becoming more and more serious, and the number of elderly people living alone is increasing. The safety of the elderly living alone has become a social problem that needs to be solved urgently. In real life, when an elderly person living alone has an accident or sudden illness at home, because no one knows to send him to the hospital in time, it is likely to delay the best rescue time. Therefore, it is very important to provide a method for real-time monitoring and early warning of the behavior status of elderly people living alone.

[0003] At present, the monitoring methods for elderly people living alone are mostly scattered and single. It is necessary to install cameras in multiple places at home and monitor various sensors such as sound, gas usage, water usage, and electricity usage to analyze the behavior status of elderly people living alone. It is difficult to meet all-round and accurate needs. In addition, the deployment and maintenance costs of multiple sensors are high, resulting in high monitoring costs. With the popularity of smartphones among the elderly, existing smartphones generally have built-in modules such as cameras, microphones, acceleration sensors, light sensors, and distance sensors. At the same time, the elderly's work and rest time is generally regular, and the elderly's mobile phone usage habits contain their own health and living status information. The current status monitoring methods for elderly people living alone have not fully utilized the functions of smartphones themselves to mine these key data sources. How to make full use of these built-in modules of smartphones to monitor the behavior status of elderly people living alone, and combine them with their mobile phone usage habits to analyze their health status, this will be a hot topic for future research. SUMMARY OF THE INVENTION

[0004] To solve the above technical problems, the present invention provides a method and system for monitoring and warning the status of elderly people living alone, which deeply integrates the mobile phone usage habits of elderly people living alone with multi-source heterogeneous data obtained by multiple sensors built into smart phones, so as to achieve more efficient and low-cost monitoring and warning of the status of elderly people living alone.

[0005] To achieve the above-mentioned invention purpose, the technical scheme adopted by the present invention is: a method for monitoring and warning the status of elderly people living alone, using multiple sensors built into a smart phone to obtain multi-source behavioral status data of elderly people living alone when they use different mobile phone applications in different seasons, different weathers and different postures, and obtain corresponding physiological status data, and establish a personalized physiological status prediction model based on the multi-source behavioral status data and physiological status data; obtain the normal physiological status characteristics of elderly people living alone when they use different mobile phone applications in different seasons, different weathers and different postures; the real-time monitoring status of the monitoring system includes two modes: using mobile phone status monitoring and not using mobile phone status monitoring; Using the mobile phone status monitoring mode: The monitoring system obtains the behavior status data monitored by multiple sensors built into the smart phone in real time and regularly, and obtains the current weather data and the type of mobile phone application used, and uses the physiological state prediction model of the corresponding season to obtain the predicted physiological state characteristics, and compares the predicted physiological state characteristics with the normal physiological state characteristics to determine whether the elderly living alone are in an abnormal physiological state when using the mobile phone, and promptly issues an early warning for abnormal physiological states; Unused mobile phone status monitoring mode: The monitoring system periodically starts voice prompts to inquire about the physiological status of the elderly living alone. If the elderly living alone respond normally, the system waits for the next voice prompt; if the elderly living alone do not respond normally, the monitoring system activates the built-in camera and microphone of the smartphone, and collects the video data and sound data of the elderly living alone through the camera and microphone respectively in real time, and collects the video data of the camera installed indoors at the same time, analyzes the real-time posture and facial emotions of the elderly living alone based on the video data, analyzes the sound status of the elderly living alone based on the sound data, and judges whether the elderly living alone are in an abnormal state based on their real-time posture, facial emotions, and sound status, and issues timely warnings for abnormal states.

[0006] Preferably, the daily mobile phone usage record data of the elderly living alone in different seasons is obtained to generate a mobile phone usage schedule for the elderly living alone in different seasons; the evening rest period and lunch break period of the elderly living alone in different seasons are added to the mobile phone usage schedule, and the mobile phone usage schedule in different seasons includes four attribute states: evening rest period, lunch break period, mobile phone use period, and mobile phone non-use period; monitoring is performed according to the following method and steps: S1, the monitoring system determines whether the elderly person living alone is at home based on the mobile phone positioning results. If not, the monitoring system enters a suspended monitoring state; if yes, the monitoring system enters a real-time monitoring state and enters step S2; S2. Determine whether the current time is a mobile phone usage time period according to the mobile phone usage schedule in different seasons. If the current time is a mobile phone usage time period, determine whether the elderly living alone are using their mobile phones normally. If so, enter the mobile phone usage monitoring mode; if not, enter the non-mobile phone usage monitoring mode; if the current time is other attribute states, enter step S3; S3, further determine the current attribute status, if it is the period of no mobile phone use, then enter the no mobile phone use monitoring mode; if it is the lunch break period or the evening break period, then access the monitoring results of the third-party sleep monitoring APP, and issue a timely warning for abnormal physiological status.

[0007] Preferably, the process of generating the mobile phone usage schedule is as follows: T11. Obtain the daily mobile phone usage record data of the elderly living alone in different seasons, wherein the mobile phone usage record data includes the unlocking / locking time of each mobile phone use, the duration of each mobile phone use, the type of mobile phone application used and the corresponding duration;​ T12. Group the mobile phone usage record data according to different seasons, organize the daily mobile phone usage record data into the first time series data in units of days; for each day, organize the hourly mobile phone usage record data into the second time series data in units of hours; construct the mobile phone usage record time series data set of different seasons; T13. Use the time series data of different seasons as model input, use the mobile phone usage records of different seasons as model output, and build and train the long short-term memory network model; T14. Use the long short-term memory network model to predict the time series data of different seasons and generate a daily mobile phone usage schedule for elderly people living alone in different seasons.

[0008] Preferably, the physiological state prediction model construction process is as follows: T21. Use multiple sensors built into smartphones to obtain multi-source behavioral state data of elderly people living alone when they use different mobile applications in different seasons, weather conditions and postures, and extract corresponding behavioral state features from the multi-source behavioral state data; T22. Use wearable devices to obtain the physiological state data corresponding to the use of different mobile phone applications by elderly people living alone in different seasons, weather conditions and postures, and extract physiological state characteristics from the physiological state data, wherein the physiological state characteristics include heart rate change rate, average heart rate, blood pressure change rate and average blood pressure; T23, taking the multi-source behavioral state characteristics when using different mobile phone applications in different seasons, different weather and different postures as input, and taking the corresponding physiological state characteristics as output, the physiological state prediction model framework adopts a fusion neural network based on the attention mechanism, and trains the physiological state prediction model according to different seasons and different postures; T24. Based on big data analysis, obtain the normal physiological state characteristics of elderly people living alone when using different mobile phone applications in different seasons, weather and postures.

[0009] Preferably, step T21 includes the following steps: Use the acceleration sensor to obtain the acceleration data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract the acceleration features from the acceleration data. The acceleration features include the peak value, mean value and standard deviation of the acceleration; Use pressure sensors to obtain pressure data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract hand-held force features from the pressure data. The hand-held force features include the peak value, mean and standard deviation of the hand-held force; Use distance sensors to obtain facial distance data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract facial distance features from the distance data. Facial distance features include the peak value, mean value and standard deviation of the distance between the face and the mobile phone; Use gyroscope sensors to obtain the tilt angle data of elderly people living alone when they use different mobile phone applications in different seasons, weather and postures, and extract tilt angle features from the tilt angle data. The tilt angle features include the angle change rate of the gyroscope.

[0010] Preferably, the different postures include standing, walking, jogging, lying, squatting, sitting and running of the elderly living alone; The abnormal physiological state warning includes: the monitoring system starts a voice prompt to ask the elderly living alone whether their physiological state is normal. If the elderly living alone answer that it is normal, the warning ends; if the elderly living alone do not respond or answer abnormally, the nearby rescue station and relatives will be contacted by phone, and the physiological state data monitored in real time by the monitoring system will be sent to the rescue station and relatives.

[0011] Preferably, if the elderly person living alone does not respond normally in the non-mobile phone monitoring mode, then: Use the human body key point detection method to analyze the relative position, angle and movement trajectory of the key points of the parts to determine the real-time posture of the elderly living alone; use the facial expression recognition algorithm to extract the local and overall features of the face to determine the facial emotions of the elderly living alone; The built-in microphone of the smartphone captures the sound signals in the room, and the monitoring system identifies abnormal sounds. When abnormal sounds are detected, the monitoring system controls the camera installed in the room to automatically focus on the video data in the direction of the sound source, and conducts key analysis on the obtained video data.

[0012] Preferably, the head, neck, palm, shoulder, elbow, waist, buttocks, knees and soles of the human body are represented by simple sketches of different shapes respectively, and each key part is identified through the captured image. According to the connection relationship between various parts of the body, each part is connected by simple lines to finally form a human body simple sketch, and the formed human body simple sketch is compared and analyzed with the existing posture simple sketches, so as to identify the state of the human body at this time; Local facial features include the contour points and internal feature points of the eyes, eyebrows, mouth, and nose. The texture features of the facial skin are extracted by calculating the texture information of the local facial area; the high-level semantic features of the entire facial image are extracted through the multi-layer convolution and pooling operations of the convolutional neural network to form a feature vector, thereby analyzing the overall pattern of facial expressions.

[0013] ​Correspondingly, the present invention also discloses a monitoring and early warning system for the state of elderly people living alone, the monitoring system comprises a mobile phone use schedule establishment module, a multi-source behavior state data acquisition module, a physiological state analysis module, an abnormal state analysis module, and an early warning module; Mobile phone usage schedule establishment module, which builds a mobile phone usage schedule based on the mobile phone usage records of elderly people living alone in different seasons; The multi-source behavior state acquisition module uses multiple sensors built into smartphones to obtain multi-source behavior state data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extracts corresponding behavior state features from the multi-source behavior state data, and transmits the behavior state features to the physiological state analysis module; The physiological state analysis module uses the physiological state prediction model to obtain the predicted physiological state characteristics based on the real-time multi-source behavioral state characteristics, combined with the current weather data and the type of mobile phone application used, and compares the predicted physiological state characteristics with the normal physiological state characteristics to determine whether the elderly living alone are in an abnormal physiological state when using the mobile phone, and transmit the abnormal physiological state signal to the early warning module; The abnormal state analysis module uses the camera and microphone to obtain video data and sound data, analyzes the real-time posture and facial emotions of the elderly living alone based on the video data, analyzes the voice status of the elderly living alone based on the sound data, determines whether the elderly living alone are in an abnormal state based on their real-time posture, facial emotions, and voice status, and transmits the abnormal state signal to the early warning module; The early warning module starts a voice prompt to ask whether the elderly living alone are in a normal physiological state. If the elderly living alone answer normally, the early warning ends; if the elderly living alone do not respond or answer abnormally, the nearby rescue station and relatives will be contacted by phone, and the physiological state data monitored in real time by the monitoring system will be sent to the rescue station and relatives.

[0014] Preferably, the monitoring system also includes a cloud and an internal storage module. The cloud is used to store all the analysis and processing data of each module, so as to facilitate the use of the data stored in the cloud to update and optimize the physiological state prediction model and the mobile phone use schedule; the internal storage module is used to store all the analysis and processing data of each module in the current season.

[0015] The present invention has the following beneficial effects: ​1. The present invention uses multiple sensors built into smart phones to obtain multi-source behavioral state data of elderly people living alone when they use different mobile phone applications in different seasons, different weather and different postures, and obtains corresponding physiological state data. A personalized physiological state prediction model is established based on the multi-source behavioral state data and physiological state data. It not only fully considers the differences in behavioral state and physiological state of elderly people living alone in different seasons, different postures and when using different mobile phone applications, but also integrates and analyzes the monitoring data of multiple types of sensors built into the mobile phone to establish a personalized and accurate physiological state prediction model, which can effectively improve the accuracy of abnormal state recognition of elderly people living alone and reduce monitoring costs.

[0016] 2. The present invention obtains the daily mobile phone usage record data of the elderly living alone in different seasons, generates a daily mobile phone usage schedule for the elderly living alone in different seasons, combines the personalized physiological state prediction model constructed by multi-source data sources, deeply integrates the multi-dimensional data such as the mobile phone usage habits, environmental perception, and physiological state of the elderly living alone, accurately analyzes the real-time behavioral state and physiological state of the elderly living alone, detects abnormalities in time and quickly initiates the corresponding rescue process, and provides a personalized state monitoring and early warning method for the elderly living alone. Brief Description of the Figures

[0017] Figure 1 This is a flowchart of the method for monitoring and early warning the status of elderly people living alone according to the present invention. Specific implementation method

[0018] The following will combine the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0019] If Figure 1 As shown, the present invention discloses a method for monitoring and warning the status of elderly people living alone. It uses multiple sensors built into a smart phone to obtain multi-source behavioral status data of elderly people living alone when they use different mobile phone applications in different seasons, different weathers and different postures, and obtains corresponding physiological status data. A personalized physiological status prediction model is established based on the multi-source behavioral status data and physiological status data; the normal physiological status characteristics of elderly people living alone when they use different mobile phone applications in different seasons, different weathers and different postures are obtained; the real-time monitoring status of the monitoring system includes two modes: using the mobile phone status monitoring and not using the mobile phone status monitoring.

[0020] Using the mobile phone status monitoring mode: The monitoring system obtains the behavior status data monitored by multiple sensors built into the smartphone in real time and regularly, and obtains the current weather data and the type of mobile phone application used, and uses the physiological state prediction model of the corresponding season to obtain the predicted physiological state characteristics, and compares the predicted physiological state characteristics with the normal physiological state characteristics to determine whether the elderly living alone are in an abnormal physiological state when using the mobile phone, and promptly issues an early warning for abnormal physiological states. The sensors include acceleration sensors, pressure sensors, distance sensors, and gyroscope sensors.

[0021] Unused mobile phone status monitoring mode: The monitoring system periodically starts voice prompts to inquire about the physiological status of the elderly living alone. If the elderly living alone respond normally, the system waits for the next voice prompt; if the elderly living alone do not respond normally, the monitoring system activates the built-in camera and microphone of the smartphone, and collects the video data and sound data of the elderly living alone through the built-in camera and microphone of the smartphone respectively in real time, and collects the video data of the camera installed indoors at the same time, analyzes the real-time posture and facial emotions of the elderly living alone based on the video data, analyzes the sound status of the elderly living alone based on the sound data, and judges whether the elderly living alone are in an abnormal state based on their real-time posture, facial emotions, and sound status, and issues timely warnings for abnormal states.

[0022] Furthermore, if the elderly person living alone does not respond normally in the non-mobile phone monitoring mode, then: The relative position, angle and movement trajectory of the key points of the human body parts are analyzed by using the key point detection method to determine the real-time posture of the elderly living alone. For example, the head, neck, palm, shoulder, elbow, waist, buttocks, knees and soles of the human body are represented by simple sketches of different shapes. The key parts are identified through the captured images. According to the connection relationship of the various parts of the body, the various parts are connected by simple lines to finally form a human body sketch. The formed human body sketch is compared and analyzed with the existing posture sketches to identify the current state of the human body.

[0023] The facial expression recognition algorithm is used to extract local and overall facial features to determine the facial emotions of elderly people living alone. Local facial features include contour points and internal feature points of eyes, eyebrows, mouth, nose and other parts. The texture features of the facial skin are extracted by calculating the texture information of the local facial area. The high-level semantic features of the entire facial image are extracted through the multi-layer convolution and pooling operations of the convolutional neural network to form a feature vector, thereby analyzing the overall pattern of facial expressions.

[0024] The built-in microphone of the smartphone captures the sound signals in the room, and the monitoring system identifies abnormal sounds. When abnormal sounds are detected, the monitoring system controls the camera installed in the room to automatically focus on the video data in the direction of the sound source, and conducts key analysis on the obtained video data. Abnormal sounds include the sound of heavy objects falling, painful groans, violent coughs, and severe asthma. Combined with real-time posture, facial expressions and environmental sounds, the physiological state of the elderly living alone at this time is comprehensively analyzed.

[0025] Furthermore, the abnormal physiological state warning includes: the monitoring system starts a voice prompt to ask the elderly living alone whether their physiological state is normal. If the elderly living alone answer that it is normal, the warning ends; if the elderly living alone do not respond or answer abnormally, the nearby rescue station and relatives will be contacted by phone, and the physiological state data monitored in real time by the monitoring system will be sent to the rescue station and relatives.

[0026] Furthermore, the daily mobile phone usage record data of the elderly living alone in different seasons is obtained to generate a mobile phone usage schedule for the elderly living alone in different seasons; the evening rest period and lunch break period of the elderly living alone in different seasons are added to the mobile phone usage schedule, and the mobile phone usage schedule in different seasons includes four attribute states: evening rest period, lunch break period, mobile phone use period, and mobile phone non-use period; the different postures include the standing state, slow walking state, fast walking state, lying state, squatting state, sitting state, and running state of the elderly living alone.

[0027] The monitoring method of the present invention is carried out according to the following steps: S1, the monitoring system determines whether the elderly person living alone is at home based on the mobile phone positioning results. If not, the monitoring system enters a suspended monitoring state; if yes, the monitoring system enters a real-time monitoring state and enters step S2; S2. Determine whether the current time is a mobile phone usage time period according to the mobile phone usage schedule in different seasons. If the current time is a mobile phone usage time period, determine whether the elderly living alone are using their mobile phones normally. If so, enter the mobile phone usage monitoring mode; if not, enter the non-mobile phone usage monitoring mode; if the current time is other attribute states, enter step S3; S3, further determine the current attribute status, if it is the period of no mobile phone use, then enter the no mobile phone use monitoring mode; if it is the lunch break period or the evening break period, then access the monitoring results of the third-party sleep monitoring APP, and issue a timely warning for abnormal physiological status.

[0028] It is not difficult to see that the monitoring method disclosed in the present invention is mainly aimed at elderly people living alone at home.

[0029] It should be noted that before the monitoring system is officially used, it is necessary to obtain the mobile phone usage record data, multi-source behavior status data, and physiological status data of the elderly living alone in different seasons, that is, at least 1 year of data is needed to train the long-term and short-term memory network model and the fusion neural network based on the attention mechanism. This data acquisition cycle is relatively long, so the model can be trained with data from a shorter period of time, such as 1 week, 10 days, or 15 days. However, the model prediction results at this time cannot be well adapted to the elderly living alone. However, as the elderly living alone use the monitoring system for a longer time and obtain more relevant data, the long-term and short-term memory network model and the fusion neural network based on the attention mechanism are constantly updated and optimized, which can accurately and comprehensively provide personalized monitoring and early warning methods for the elderly living alone.

[0030] Furthermore, the process of generating the mobile phone usage schedule is as follows: T11. Obtain the daily mobile phone usage record data of the elderly living alone in different seasons. The mobile phone usage record data includes the unlocking / locking time of each mobile phone use, the duration of each mobile phone use, the type of mobile phone application used and the corresponding duration.

[0031] T12. Group the mobile phone usage record data according to different seasons, organize the daily mobile phone usage record data into the first time series data in units of days; for each day, organize the hourly mobile phone usage record data into the second time series data in units of hours; construct the mobile phone usage record time series data set of different seasons.

[0032] T13. Use the time series data of different seasons as model input, use the mobile phone usage records of different seasons as model output, and build and train the long short-term memory network model.

[0033] T14. Use the long short-term memory network model to predict the time series data of different seasons and generate a daily mobile phone usage schedule for elderly people living alone in different seasons.

[0034] Furthermore, the physiological state prediction model construction process is as follows: T21. Use multiple sensors built into smartphones to obtain multi-source behavioral state data of elderly people living alone when they use different mobile applications in different seasons, weather conditions and postures, and extract corresponding behavioral state features from the multi-source behavioral state data; specifically: Use the acceleration sensor to obtain the acceleration data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract the acceleration features from the acceleration data. The acceleration features include the peak value, mean value and standard deviation of the acceleration; Use pressure sensors to obtain pressure data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract hand-held force features from the pressure data. The hand-held force features include the peak value, mean and standard deviation of the hand-held force; Use distance sensors to obtain facial distance data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract facial distance features from the distance data. Facial distance features include the peak value, mean value and standard deviation of the distance between the face and the mobile phone; Use gyroscope sensors to obtain the tilt angle data of elderly people living alone when they use different mobile phone applications in different seasons, weather and postures, and extract tilt angle features from the tilt angle data. The tilt angle features include the angle change rate of the gyroscope.

[0035] T22. Use wearable devices to obtain the physiological state data corresponding to the use of different mobile phone applications by elderly people living alone in different seasons, weather conditions and postures, and extract physiological state characteristics from the physiological state data, the physiological state characteristics including heart rate change rate, average heart rate, blood pressure change rate and average blood pressure. The wearable device can be a smart bracelet or badge with a built-in heart rate sensor and blood pressure sensor to collect physiological state data and transmit it to the physiological state analysis module via Bluetooth mode or wireless communication.

[0036] T23. The multi-source behavioral state characteristics when using different mobile phone applications in different seasons, different weather and different postures are used as input, and the corresponding physiological state characteristics are used as output. The physiological state prediction model framework adopts a fusion neural network based on the attention mechanism, and trains the physiological state prediction model according to different postures in different seasons to obtain the physiological state prediction model in different posture modes in different seasons. The fusion neural network based on the attention mechanism can use different weights for different types of behavioral state characteristics, so that the physiological state model can deeply understand the comprehensive state of different elderly people living alone when using different mobile phone applications in different seasons and different postures, making up for the defects of single data source analysis, and at the same time distinguishing the differences between different elderly people living alone due to different seasons, different postures, and different mobile phone applications, and providing personalized monitoring methods.

[0037] T24. Based on the historical data of elderly people living alone, obtain the normal physiological state characteristics of elderly people living alone when using different mobile phone applications in different seasons, weather and postures based on big data analysis, and store these normal physiological state characteristics in the internal storage module for easy analysis and processing.

[0038] Corresponding to the above monitoring method, the present invention also discloses a monitoring and early warning system for the status of elderly people living alone, the monitoring system includes a mobile phone use schedule establishment module, a multi-source behavior status data acquisition module, a physiological status analysis module, an abnormal status analysis module, an early warning module, a cloud, and an internal storage module;​ Build a module for mobile phone usage schedule, obtain the daily mobile phone usage record data of elderly people living alone in different seasons, use the time series data of different seasons as model input, use the mobile phone usage records of different seasons as model output, build and train the long short-term memory network model, and use the long short-term memory network model to generate a daily mobile phone usage schedule for elderly people living alone in different seasons; The multi-source behavior state acquisition module uses multiple sensors built into smartphones to obtain multi-source behavior state data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extracts corresponding behavior state features from the multi-source behavior state data, and transmits the behavior state features to the physiological state analysis module; The physiological state analysis module takes the multi-source behavioral state characteristics when using different mobile phone applications in different seasons, different weather and different postures as input, and the corresponding physiological state characteristics as output. It uses the fusion neural network based on the attention mechanism as the network framework to establish and train the physiological state prediction model; according to the real-time multi-source behavioral state characteristics, combined with the current weather data and the type of mobile phone application used, the physiological state prediction model of the corresponding season is used to obtain the predicted physiological state characteristics, and the predicted physiological state characteristics are compared with the normal physiological state characteristics to determine whether the elderly living alone are in an abnormal physiological state when using mobile phones, and the abnormal physiological state signal is transmitted to the early warning module; The abnormal state analysis module analyzes the real-time posture and facial emotions of the elderly living alone based on the video data, analyzes the voice status of the elderly living alone based on the voice data, determines whether the elderly living alone are in an abnormal state based on their real-time posture, facial emotions, and voice status, and transmits the abnormal state signal to the early warning module; The early warning module starts a voice prompt to ask whether the elderly living alone are in a normal physiological state. If the elderly living alone answer normally, the early warning ends; if the elderly living alone do not respond or answer abnormally, the nearby rescue station and relatives will be contacted by phone, and the physiological state data monitored in real time by the monitoring system will be sent to the rescue station and relatives; The cloud is used to store all the analysis and processing data of each module, so as to update and optimize the physiological state prediction model and long-term and short-term memory network model using the data stored in the cloud; The internal storage module is used to store all analysis and processing data of each module in the current season.

[0039] Smartphones have multiple sensors built in, including distance sensors, sound sensors, acceleration sensors, gyroscope sensors, etc. The present invention uses distance sensors to monitor the distance between the face and the phone to reflect the normal habits of elderly people living alone when using the phone; uses the acceleration changes monitored by the acceleration sensor to reflect the activity range, activity amount, and whether there are sudden accidents (such as the phone suddenly falling, which may be caused by the elderly living alone falling down accidentally, falling into a coma, etc.); uses the ambient sound monitored by the sound sensor to determine whether the elderly are in an abnormal state; uses the pressure monitored by the pressure sensor to reflect the strength of the elderly living alone holding the phone and the strength of the fingers clicking the screen; uses the tilt angle monitored by the gyroscope sensor to reflect the hand shaking of the elderly living alone when using the phone.

[0040] The state monitoring method of the present invention uses the data monitored by these sensors to comprehensively analyze the behavior state of the elderly living alone when using mobile phones, combined with the physiological state at this time. Since the behavior state and physiological state of the elderly living alone may be greatly different in different seasons, different postures, and when using different mobile phone applications, a personalized physiological state prediction model in different posture modes in different seasons is established; since most elderly people living alone have a certain regular daily routine, the mobile phone usage of the elderly living alone in different seasons is obtained, and a personalized mobile phone usage schedule is constructed. The living state of the elderly living alone is monitored according to the mobile phone usage schedule.

[0041] The embodiments described above are only descriptions of the preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations, modifications, and substitutions made by ordinary technicians in this field to the technical solutions of the present invention should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for monitoring and warning the status of elderly people living alone, characterized by: Use multiple sensors built into smartphones to obtain multi-source behavioral state data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and obtain corresponding physiological state data to establish a personalized physiological state prediction model; obtain the normal physiological state characteristics of elderly people living alone in different seasons; monitor the physiological state of elderly people living alone based on their daily mobile phone usage habits: When elderly people living alone use mobile phones, they can obtain behavioral status data monitored by multiple sensors built into the smartphone in real time and regularly, and obtain current weather data and the type of mobile phone applications used. They can use the physiological state prediction model to obtain predicted physiological state characteristics, compare the predicted physiological state characteristics with normal physiological state characteristics, and determine whether they are abnormal. When the elderly living alone are not using their mobile phones, voice prompts are used to monitor their physiological state. If any abnormal situation occurs, the camera and microphone are used to obtain video data and sound data, and the real-time posture and facial emotions of the elderly living alone are analyzed based on the video data. The voice state of the elderly living alone is analyzed based on the sound data to determine whether there is an abnormality.

2. The method for monitoring and early warning of the status of elderly people living alone according to claim 1 is characterized by: Obtain the daily mobile phone usage record data of the elderly living alone in different seasons, and generate a mobile phone usage schedule for the elderly living alone in different seasons; add the evening rest period and lunch break period of the elderly living alone in different seasons to the mobile phone usage schedule, so the mobile phone usage schedule in different seasons includes four attribute states: evening rest period, lunch break period, mobile phone use period, and mobile phone non-use period; monitor according to the following method and steps: S1. The monitoring system determines whether the elderly person living alone is at home based on the mobile phone positioning results. If not, the monitoring system enters a suspended monitoring state; if yes, the monitoring system enters a real-time monitoring state and enters step S2; S2. Determine whether the current time is a mobile phone usage time period according to the mobile phone usage schedule of different seasons. If the current time is a mobile phone usage time period, determine whether the elderly living alone are using their mobile phones normally. If so, enter the mobile phone usage monitoring mode; if not, enter the non-mobile phone usage monitoring mode; if the current time is other attribute states, enter step S3; S3, further determining the attribute status at the current moment, if it is a period of time when the mobile phone is not used, entering the non-use mobile phone monitoring mode; If it is during lunch break or evening break, the monitoring results of a third-party sleep monitoring app will be connected to issue timely warnings for abnormal physiological conditions.

3. The method for monitoring and early warning of the status of elderly people living alone according to claim 2 is characterized by: The process of generating the mobile phone usage schedule is as follows: T11. Obtaining the daily mobile phone usage record data of the elderly living alone in different seasons, wherein the mobile phone usage record data includes the unlocking / locking time of each mobile phone use, the duration of each mobile phone use, the type of mobile phone application used and the corresponding duration; T12. Group the mobile phone usage record data according to different seasons, organize the daily mobile phone usage record data into the first time series data in units of days; for each day, organize the hourly mobile phone usage record data into the second time series data in units of hours; construct a mobile phone usage record time series data set for different seasons; T13. Use the time series data of different seasons as model input and the mobile phone usage records of different seasons as model output to build and train the long short-term memory network model. T14. Use the long short-term memory network model to predict the time series data of different seasons and generate a daily mobile phone usage schedule for elderly people living alone in different seasons.

4. The method for monitoring and early warning of the status of elderly people living alone according to claim 1 is characterized by: The physiological state prediction model construction process is as follows: T21. Use multiple sensors built into smartphones to obtain multi-source behavioral state data of elderly people living alone when they use different mobile applications in different seasons, weather conditions and postures, and extract corresponding behavioral state features from the multi-source behavioral state data; T22. Use wearable devices to obtain physiological state data corresponding to the use of different mobile phone applications by elderly people living alone in different seasons, weather conditions and postures, and extract physiological state features from the physiological state data, wherein the physiological state features include heart rate change rate, average heart rate, blood pressure change rate and average blood pressure; T23. The multi-source behavioral state characteristics when using different mobile phone applications in different seasons, different weather and different postures are used as input, and the corresponding physiological state characteristics are used as output. The physiological state prediction model framework adopts a fusion neural network based on the attention mechanism to train the physiological state prediction model according to different seasons and different postures. T24. Based on big data analysis, obtain the normal physiological state characteristics of elderly people living alone when they use different mobile phone applications in different seasons, weather and postures.

5. The method for monitoring and early warning of the status of elderly people living alone according to claim 4 is characterized by: The step T21 comprises the following steps: The acceleration sensor is used to obtain the acceleration data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and the acceleration features are extracted from the acceleration data. The acceleration features include the peak value, mean value and standard deviation of the acceleration. Use pressure sensors to obtain pressure data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extract hand-held force features from the pressure data. The hand-held force features include the peak value, mean and standard deviation of the hand-held force. The distance sensor is used to obtain the facial distance data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures. The facial distance features are extracted from the distance data. The facial distance features include the peak value, mean value and standard deviation of the distance between the face and the mobile phone. The gyroscope sensor is used to obtain the tilt angle data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures. The tilt angle features are extracted from the tilt angle data. The tilt angle features include the angle change rate of the gyroscope.

6. The method for monitoring and early warning of the status of elderly people living alone according to claim 4 is characterized by: The different postures include standing, walking, jogging, lying, squatting, sitting, and running for the elderly living alone; The abnormal physiological state warning includes: the monitoring system starts a voice prompt to ask whether the physiological state of the elderly living alone is normal. If the elderly living alone answer that it is normal, the warning ends; If the elderly person living alone does not respond or gives an abnormal answer, the system will contact the nearby rescue station and relatives by phone, and send the physiological status data monitored in real time by the monitoring system to the rescue station and relatives.

7. The method for monitoring and early warning of the status of elderly people living alone according to claim 1 is characterized by: If the elderly person living alone does not respond normally in the non-mobile phone monitoring mode, then: The human body key point detection method is used to analyze the relative position, angle and motion trajectory of the key points of the body parts to determine the real-time posture of the elderly living alone; the facial expression recognition algorithm is used to extract the local and overall features of the face to determine the facial emotions of the elderly living alone; The monitoring system captures sound signals in the room through the built-in microphone of the smartphone and identifies abnormal sounds. When abnormal sounds are detected, the monitoring system controls the camera installed in the room to automatically focus the video data in the direction of the sound source and conducts focused analysis on the obtained video data.

8. The method for monitoring and early warning the status of elderly people living alone according to claim 7 is characterized by: The head, neck, palm, shoulder, elbow, waist, buttocks, knees and soles of the human body are represented by simple sketches of different shapes, and each key part is identified through the captured image. According to the connection relationship between the various parts of the body, each part is connected by simple lines, and finally a human body sketch is formed. The formed human body sketch is compared and analyzed with the existing posture sketches, so as to identify the state of the human body at this time; Local facial features include the contour points and internal feature points of the eyes, eyebrows, mouth, and nose. The texture features of the facial skin are extracted by calculating the texture information of the local facial area. The high-level semantic features of the entire facial image are extracted through the multi-layer convolution and pooling operations of the convolutional neural network to form a feature vector, thereby analyzing the overall pattern of facial expressions.

9. The monitoring and early warning system for elderly people living alone is characterized by: The monitoring system includes a mobile phone usage schedule establishment module, a multi-source behavior status data acquisition module, a physiological status analysis module, an abnormal status analysis module, and an early warning module; A module for establishing a mobile phone usage schedule, which builds a mobile phone usage schedule based on the mobile phone usage records of elderly people living alone in different seasons; The multi-source behavior state acquisition module uses multiple sensors built into the smartphone to obtain multi-source behavior state data of elderly people living alone when they use different mobile phone applications in different seasons, weather conditions and postures, and extracts corresponding behavior state features from the multi-source behavior state data, and transmits the behavior state features to the physiological state analysis module; The physiological state analysis module uses the physiological state prediction model to obtain predicted physiological state characteristics based on real-time multi-source behavioral state characteristics, combined with current weather data and the type of mobile phone applications used, and compares the predicted physiological state characteristics with normal physiological state characteristics to determine whether the elderly living alone are in an abnormal physiological state when using a mobile phone, and transmits the abnormal physiological state signal to the early warning module; The abnormal state analysis module uses a camera and a microphone to obtain video data and sound data, analyzes the real-time posture and facial emotions of the elderly living alone based on the video data, analyzes the voice status of the elderly living alone based on the sound data, determines whether the elderly living alone is in an abnormal state based on their real-time posture, facial emotions, and voice status, and transmits the abnormal state signal to the early warning module; The early warning module starts a voice prompt to ask whether the physiological state of the elderly living alone is normal. If the elderly person living alone answers that it is normal, the early warning ends; if the elderly person living alone does not respond or answers abnormally, the nearby rescue station and relatives will be contacted by phone, and the physiological state data monitored in real time by the monitoring system will be sent to the rescue station and relatives.

10. The monitoring and early warning system for the elderly living alone according to claim 9 is characterized by: It also includes cloud and internal storage modules. The cloud is used to store all analysis and processing data of each module, so as to facilitate the use of data stored in the cloud to update and optimize the physiological state prediction model and mobile phone usage schedule; the internal storage module is used to store all analysis and processing data of each module in the current season.

Images