An intelligent elderly health monitoring method, device, terminal and storage medium

Abstract

The present invention discloses an intelligent elderly health monitoring method, device, terminal and storage medium. The method obtains activity video data corresponding to a target user, and determines pose information corresponding to the target user according to the activity video data; obtains physiological data corresponding to the target user, and determines motion state information corresponding to the target user according to the physiological data; and determines a physiological safety coefficient of the target user according to the pose information and the motion state information. The present invention comprehensively determines the physiological safety coefficient of the target user through the pose information and the motion state information of the target user, which can improve the accuracy of iterative detection. It solves the problem in the prior art that the fall detection method only uses sensor data, the data type is single, and false judgment is prone to occur.

Description

Technical Field

[0001] The present invention relates to the field of data analysis, and in particular to an intelligent elderly health monitoring method, device, terminal and storage medium. Background Art

[0002] With the development of medical technology, health care services have emerged to help the elderly or people with limited mobility to manage their daily lives with peace of mind. However, with the advancement of an aging society, the proportion of elderly and weak patients who are unable to move due to their advanced age is increasing, so it is necessary to improve the manpower or system that can care for them. Due to insufficient manpower, it is actually impossible to care for the increasing number of elderly and weak patients 24 hours a day, so intelligent care systems are being actively studied. In particular, fall rescue services for elderly people living alone or those who have lived alone for a long time are attracting attention to deal with indecent accidents caused by falls. In particular, people with limited mobility or the elderly and weak are prone to fall accidents. In order to solve this problem, many fall detection methods have been developed. The current mainstream fall detection method is based on sensor technology. The sensor obtains the user's movement conditions such as acceleration or vibration in different directions to determine whether the user has fallen. Since this fall detection method uses a single type of data, it is easy to misjudge.

[0003] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0004] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, an intelligent elderly health monitoring method, device, terminal and storage medium are provided, aiming to solve the problem that the fall detection method in the prior art only uses sensor data, the data type is single, and misjudgment is prone to occur.

[0005] The technical solution adopted by the present invention to solve the problem is as follows:

[0006] In a first aspect, an embodiment of the present invention provides an intelligent elderly health monitoring method, wherein the method comprises:

[0007] Acquire activity video data corresponding to a target user, and determine posture information corresponding to the target user according to the activity video data;

[0008] Acquire physiological data corresponding to the target user, and determine motion state information corresponding to the target user according to the physiological data;

[0009] A physiological safety factor of the target user is determined according to the posture information and the motion state information.

[0010] In one embodiment, the activity video data includes a plurality of frame image data, and obtaining the activity video data corresponding to the target user and determining the pose information corresponding to the target user according to the activity video data includes:

[0011] Obtaining a plurality of frames of the image data through a preset camera device;

[0012] Respectively determining target objects in a plurality of frames of the image data, where the target object is the target user captured in a plurality of frames of the image data;

[0013] Labeling the target objects in a plurality of frames of the image data to obtain a set of recognition points corresponding to each of the plurality of frames of the image data, where the number of recognition points included in each set of recognition points is equal, and the recognition points included in different sets of recognition points have a one-to-one correspondence, and a plurality of the recognition points with a corresponding relationship are used to reflect the position information of the same body part of the target user;

[0014] Determining the pose information according to the set of recognition points corresponding to each of the plurality of frames of the image data.

[0015] In one embodiment, the determining the pose information according to the set of recognition points corresponding to each of the plurality of frames of the image data includes:

[0016] Determining movement data corresponding to each of the plurality of body parts according to the set of recognition points corresponding to each of the plurality of frames of the image data;

[0017] Determining the pose information according to the movement data corresponding to each of the plurality of body parts.

[0018] In one embodiment, the physiological data is several types of physiological data, and determining the exercise state information corresponding to the target user according to the physiological data includes:

[0019] Obtaining the standard data ranges corresponding to each of the several types of physiological data;

[0020] If any one type of the physiological data is outside the corresponding standard data range, determining the exercise state information as a warning state.

[0021] In one embodiment, the determining the physiological safety factor of the target user according to the pose information and the exercise state information includes:

[0022] Obtaining the coefficient ratios corresponding to the pose information and the exercise state information respectively;

[0023] Determine the physiological safety factor according to the attitude information, the motion state information, and the coefficient ratios corresponding to the attitude information and the motion state information respectively.

[0024] In one embodiment, the method further includes:

[0025] If only the motion state information is in a warning state, generate a body warning message;

[0026] Send the body warning message to the monitoring terminal corresponding to the target user.

[0027] In one embodiment, the method further includes:

[0028] Perform face recognition operations on several frames of the image data respectively;

[0029] Judge whether there is a strange user according to the results of the face recognition operations corresponding to several frames of the image data, where the strange user is a user whose face information has not been pre-entered;

[0030] When there is the strange user, issue an alarm message.

[0031] In a second aspect, an embodiment of the present invention further provides an intelligent elderly health monitoring device, where the device includes:

[0032] An attitude monitoring module, configured to obtain the activity video data corresponding to the target user, and determine the attitude information corresponding to the target user according to the activity video data;

[0033] A physiological monitoring module, configured to obtain the physiological data corresponding to the target user, and determine the motion state information corresponding to the target user according to the physiological data;

[0034] A state determination module, configured to determine the physiological safety factor of the target user according to the attitude information and the motion state information.

[0035] In a third aspect, an embodiment of the present invention further provides a terminal, where the terminal includes a memory and one or more processors; the memory stores one or more programs; the programs include instructions for executing the intelligent elderly health monitoring method as described in any one of the above; the processor is configured to execute the programs.

[0036] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which multiple instructions are stored, where the instructions are suitable for being loaded and executed by a processor to implement the steps of the intelligent elderly health monitoring method as described in any one of the above.

[0037] Advantages of the present invention: In the embodiments of the present invention, activity video data corresponding to a target user is obtained, and pose information corresponding to the target user is determined according to the activity video data; physiological data corresponding to the target user is obtained, and a motion state information corresponding to the target user is determined according to the physiological data; according to the pose information and the motion state information, a physiological safety coefficient of the target user is determined. The present invention comprehensively determines the physiological safety coefficient of the target user through the pose information and the motion state information of the target user, which can improve the accuracy of iterative detection. It solves the problem in the prior art that the fall detection method only applies sensor data, and the data type is single, which is prone to misjudgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 It is a flowchart of an intelligent elderly health monitoring method provided by an embodiment of the present invention.

[0040] Figure 2 It is a flowchart of a face recognition operation provided by an embodiment of the present invention.

[0041] Figure 3 It is an internal module diagram of an intelligent elderly health monitoring device provided by an embodiment of the present invention.

[0042] Figure 4 It is an intelligent block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The present invention discloses an intelligent elderly health monitoring method, device, terminal and storage medium. To make the purpose, technical solution and effect of the present invention clearer and more definite, the following will further describe the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] Those skilled in the art of the present technology can understand that unless specifically stated, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention means

[0045] There are the described features, integers, steps, operations, elements and / or components, but the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof is not excluded. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or wireless coupling. The phrase "and / or" used here includes all or any unit and all combinations of one or more related listed items.

[0046] Those skilled in the art of this technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used here have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention pertains. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0047] With the development of medical technology, there have emerged healthcare services that target the elderly or physically handicapped walkers, etc., to help them live their daily lives with peace of mind. However, with the advancement of an aging society, the proportion of the elderly, the weak, and the sick who are physically handicapped due to old age is increasing, so there is a need to improve the manpower or system capable of caring for and managing them. Due to a shortage of manpower, it is virtually impossible to continuously care for the increasing number of the elderly, the weak, and the sick for 24 hours. Therefore, intelligent care systems are being actively studied. In particular, fall rescue services that target the elderly living alone or the elderly who have lived alone for a long time, etc., to deal with undignified accidents caused by falls are receiving much attention. In particular, people with limited mobility or the elderly and the weak are prone to fall accidents. To solve such problems, many fall detection methods have been developed. The current mainstream fall detection methods are implemented based on sensor technology. By obtaining the motion conditions of the user, such as acceleration or vibration in different directions, through sensors, it is determined whether the user has fallen. Since the data types applied in this fall detection method are single, the situation of misjudgment is likely to occur.

[0048] In view of the above defects of the prior art, the present invention provides an intelligent elderly health monitoring method. The method obtains the activity video data corresponding to the target user, and determines the posture information corresponding to the target user according to the activity video data; obtains the physiological data corresponding to the target user, and determines the motion state information corresponding to the target user according to the physiological data; determines the physiological safety factor of the target user according to the posture information and the motion state information. The present invention comprehensively determines the physiological safety factor of the target user through the posture information and the motion state information of the target user, which can improve the accuracy of iterative detection. It solves the problem that the fall detection method in the prior art only uses sensor data, the data type is single, and misjudgment is likely to occur.

[0049] As Figure 1 shown, the method includes:

[0050] Step S100, obtain the activity video data corresponding to the target user, and determine the posture information corresponding to the target user according to the activity video data.

[0051] Specifically, in this embodiment, one or more camera devices are pre-set for the activity range of the target user, and the target user is monitored by the camera devices to obtain the activity video data corresponding to the target user. Among them, the activity video data can reflect the current behavior of the target user. Therefore, by analyzing the activity video data, the current posture information of the target user can be determined.

[0052] In one implementation, the camera device can be a robot. The robot can not only be used to determine the posture information of the target user, but also record the changes in the indoor environment for further analysis of the personal safety of the target user. In addition, the robot also interacts with the target user for medical care, providing certain auxiliary interventions for the target user, such as cognitive training, music therapy, etc., preventing the decline of the cognitive ability of the elderly, relieving the mental pressure of the target user, and reducing the loneliness of the target user.

[0053] In one implementation, the activity video data includes several frame image data, and the step S100 specifically includes the following steps:

[0054] Step S101, obtain several frames of the image data through a preset camera device;

[0055] Step S102, respectively determine the target objects in several frames of the image data, where the target object is the target user captured in several frames of the image data;

[0056] Step S103: Label the target object in several frames of the image data to obtain a set of recognition points corresponding to each of the several frames of the image data. The number of recognition points included in each set of recognition points is equal, and there is a one-to-one correspondence between the recognition points included in different sets of recognition points. The several recognition points with a corresponding relationship are used to reflect the position information of the same body part of the target user;

[0057] Step S104: Determine the pose information according to the sets of recognition points corresponding to the several frames of the image data.

[0058] Specifically, the activity video data in this embodiment includes multiple frames of image data. For each frame of image data, first determine the target object in this frame of image data, that is, the target user captured, and label the target object, so as to obtain a set of recognition points corresponding to this frame of image data. The set of recognition points contains multiple recognition points, and each recognition point corresponds to a different body part of the target user. Each recognition point can reflect the position information of a body part of the target user at the time point corresponding to this frame of image data. The number of recognition points in the sets of recognition points corresponding to different frames of image data is equal, that is, in this embodiment, each time of labeling is performed on the same several body parts. Therefore, there is a one-to-one correspondence between the recognition points in different sets of recognition points, and the recognition points with a corresponding relationship correspond to the same body part. Therefore, based on the sets of recognition points corresponding to the several frames of the image data, the position change of each body part of the target user can be determined, and then the pose information of the target user can be obtained.

[0059] In one implementation, the imaging device can automatically change its angle as the area where the target user is located changes. Specifically, after obtaining a number of frames of image data, these image data are used as input data for the camera tracking model. Among them, the camera tracking model uses the Gaussian mixture background modeling algorithm to establish a Gaussian distribution for each pixel point, so as to be able to handle the situation of multi-modal background distribution, and then determine appropriate parameters and frame rate update rates to solve the problem of large-area false detection caused by sudden changes in illumination. In addition, the camera tracking model also uses the video optical flow method to seek the relatively changing optical flow field in the invariant optical flow field, so as to determine that the changed part is the part where the moving target is located (on the premise that it is assumed that the brightness information in the scene does not change, and then the running vectors of pixel points between consecutive image frames are calculated, that is, the optical flow field. For the background area, since the change is relatively small, a relatively uniform optical flow vector field is generally generated, while for the moving object, its optical flow vector field is not very regular, and the moving target can be detected based on this). In addition, the camera tracking model also combines the adjacent frame difference method (temporal difference method), and uses the difference between two adjacent frames or several frames to obtain the difference image as the detection result. Finally, the camera tracking model outputs the part where the target user is located in each frame of image data. Then, based on the part where the target user is located corresponding to each frame of image data, pose recognition is performed to extract the pose information of the target user.

[0060] In one implementation, the step S104 specifically includes the following steps:

[0061] Step S1041: Determine the movement data corresponding to each of the body parts according to the set of recognition points corresponding to each of the several frames of the image data;

[0062] Step S1042: Determine the pose information according to the movement data corresponding to each of the body parts.

[0063] Specifically, the recognition points with corresponding relationships in each set of recognition points actually reflect the position information of the same body part on the target user at different time points. Therefore, according to each set of recognition points, the movement data corresponding to each body part can be determined, and then based on the movement data corresponding to each body part, motion analysis of the target user is performed to obtain the pose information corresponding to the target user.

[0064] In one implementation, each set of recognition points is input into a pre-trained deep learning model, and the deep learning model classifies and recognizes the semantics of human pose expressions through the input sets of recognition points. The advantage of using a deep learning model for pose recognition is that it can solve technical difficulties such as pose displacement scale transformation, pose size scale transformation, recognition point noise and recognition point missing, and video region segmentation of human pose expressions.

[0065] In one implementation, the deep learning model employs an efficient pose recognition algorithm, which can specifically perform human detection box and bone point extraction, and then splice bone features according to time information, and send the bone features into a convolutional neural network and an artificial neural network to obtain an action recognition result.

[0066] In one implementation, several pose types can be pre-personalized for the target user, such as smoking pose, falling pose, stroke pose, and so on. When the pose information of the target user is detected as any one of the several pose types, a warning process is performed.

[0067] As Figure 1 shown, the method further includes the following steps:

[0068] Step S200: Obtain the physiological data corresponding to the target user, and determine the motion state information corresponding to the target user according to the physiological data.

[0069] Specifically, since the body of the target user will present different physiological states when performing different behavioral actions, this embodiment also needs to obtain the physiological data of the target user, perform motion analysis on the target user through the physiological data, and obtain the motion state information of the target user. For example, when the target user has a falling behavior, the body's acceleration is very fast, so the heart rate of the target user fluctuates greatly. Therefore, by combining the heart rate change of the target user, it is possible to more accurately determine whether the target user has a falling behavior.

[0070] In one implementation, the physiological data is several types of physiological data, and the determining the motion state information corresponding to the target user according to the physiological data specifically includes the following steps:

[0071] Step S201: Obtain the standard data ranges corresponding to several types of the physiological data respectively;

[0072] Step S202: If any one of the several types of physiological data is outside the corresponding standard data range, determine that the motion state information is a warning state.

[0073] Specifically, since it is difficult to accurately determine the exercise state of the target user by using a single type of physiological data, this embodiment needs to collect multiple types of physiological data of the target user. For each type of physiological data, obtain the preset standard data range corresponding to this type of physiological data, and compare this type of physiological data with its corresponding standard data range. If this type of physiological data is within its corresponding standard data range, it indicates that this type of physiological data is normal, and it is determined that the exercise state information of the target user is in a normal state, and the target user has not had a fall behavior; if this type of physiological data is outside its corresponding standard data range, it indicates that this type of physiological data is abnormal, and it is determined that the exercise state information of the target user is in a warning state, and there is a high probability that the target user has already had a fall behavior.

[0074] In one implementation, several types of the physiological data can be obtained through a smart wearable device, such as a smart bracelet or a smart watch. Through the smart wearable device, it is possible to monitor the physiological data of the target user for a long time, such as physiological parameters such as electrocardiogram, pulse wave, body temperature, blood pressure, and blood sugar, and prevent the occurrence of diseases in a timely manner.

[0075] In one implementation, a sensor is provided on the smart wearable device. The sensor can sense motion conditions such as acceleration or vibration in different directions through a capacitive accelerometer. The motion state sensors of three-dimensional rhythm are divided into three-axis and six-axis. The three-axis ones generally record data when swinging the arm, while the six-axis ones will improve the data recording and accuracy of exercise through walking, running, cycling, and climbing stairs. Then, according to the data of the three dimensions captured in real time by the three-axis acceleration, through processes such as filtering and peak-valley detection, using various algorithms and scientific and rigorous logical operations, these data are finally transformed into readable numbers on the APP side of the smart wearable device, such as the number of steps, distance, calorie consumption value, etc., and presented.

[0076] In one implementation, considering that a smart watch has basic functional modules such as electrocardiogram, non-invasive blood pressure, body temperature, blood oxygen saturation and other physiological parameter monitoring and processing, medical information extraction and data transmission, and its product forms include hardware, software, and a combination of hardware and software, and has characteristics such as miniaturization, portability, low power consumption, and interactivity, this embodiment can use a smart watch as the smart wearable device. Among them, the smart watch is equipped with an electrocardiogram monitoring module, a blood pressure monitoring module, a blood oxygen saturation monitoring module, and a body temperature monitoring module.

[0077] Among them, the electrocardiogram monitoring module: uses a standard signal generator, and through a supporting test port, outputs a standard signal to the input port of the device under test. After the device under test collects the signal, it outputs test results such as an electrocardiogram waveform, and compares and calculates with the standard signal parameters to complete the calibration of electrocardiogram-related parameters such as the internal calibration voltage and heart rate in the device under test.

[0078] Among them, blood pressure monitoring module: There are three ways to monitor blood pressure in smart watches or bracelets on the market: photoelectric sensors, photoelectric sensors combined with electrocardiogram sensors, and oscillometric boost measurement technology. Taking into account the maturity of existing technologies, costs, and regulatory approval, photoelectric sensors are more popular. The principle is to collect the pulse waveform at the wrist through a photoelectric sensor, and analyze the characteristic parameters such as the rising slope and band time of the pulse wave to estimate the blood pressure value through a specific algorithm. For example, when blood pressure rises, the slope of the rising band of the ejection period will increase. Therefore, the blood pressure monitoring module will be implemented by a pulse blood pressure sensor chip combined with a specific chip and algorithm.

[0079] Among them, blood oxygen saturation monitoring module: the dial of the smart watch can be embedded with a heart rate meter and a blood oximeter at the same time. The heart rate meter and the blood oximeter use the same structure in a time-sharing multiplexing manner, adopt a reflective type, and are set on the back of the watch dial in contact with the skin. The structure is composed of two LED lights and a photosensitive element in contact with the skin. The LED light outputs a fixed wavelength light beam to illuminate the skin capillaries, and then the light reflected back by the skin, tissue, and blood is collected by the photosensitive element, and the light signal representing the blood oxygen saturation value is converted into an electrical signal, thereby completing blood oxygen monitoring. For example, the smart watch includes a photosensitive sensor and two green wavelength light-emitting LEDs, and the heart rate data of the target user is measured by the photosensitive sensor and the two light-emitting LEDs. Specifically, since the blood in the blood vessels of the target user's arm will change in density when pulsating, thereby causing a change in transmittance, when the light-emitting LED emits a green wavelength light wave, the photosensitive sensor can receive the reflected light from the target user's arm skin and sense the change in light field intensity, thereby calculating the target user's heart rate based on the change in light field intensity. By continuously measuring the heart rate of the target user, the average heart rate of the target user can be calculated, and the maximum heart rate of the target user can be recorded.

[0080] Among them, the body temperature monitoring module: The smart watch body temperature monitoring module uses an NTC temperature sensor. The sensor has high detection accuracy, with an accuracy of 0.1°C in the temperature range of 0°C to 70°C. The sensor has good thermistor consistency, fast temperature data acquisition, and a sensing speed of only 2s. During the measurement process, the NTC body temperature probe obtains real data, and there is no need to study algorithms to improve measurement accuracy, and body temperature can be measured directly from the wrist.

[0081] In one implementation, a natural language processing algorithm is embedded in the smartwatch, so the smartwatch can recognize the target user's voice to complete operations such as dialing a family number, and the robot can be linked to complete basic instructions issued by the target user, such as closing the door, drawing the curtains, etc. In order to ensure the recognition accuracy and precision of the instructions, the target user can complete the recording and verification of the instructions in the company of a guardian.

[0082] In one implementation, several types of the physiological data can be sent to a big data analysis platform and provided for relevant personnel to refer to, so as to provide a convenient and reassuring home experience for the target user.

[0083] In one implementation, when the fluctuation of any one of several types of the physiological data exceeds a preset range, it indicates that there is a sharp change in the physiological data. Then, an alarm signal is generated based on the physiological data whose fluctuation exceeds the preset range and sent to the mobile phone of the guardian corresponding to the target user. Moreover, the main control chip will start an emergency call program and directly automatically dial a preset emergency call number. If the call is not connected, the smart watch will automatically call in a loop until it is connected. After the call is connected, the smart watch will broadcast the emergency call content through automatic voice, and send the health data and location information of the target user to the relative's mobile phone APP. The GPS positioning module extracts the longitude and latitude positioning information of the location where the target user is located and sends it to the main control module, and at the same time sends the fall alarm information and longitude and latitude positioning information to the guardian through the remote communication module. Moreover, there is a one-key SOS function on the smart watch of the target user. Usually, when the smart watch is in the on state, long press this button for 2 to 3 seconds, and the smart watch will emit an alarm sound and automatically call a pre-set emergency contact until someone answers the call for help. The guardian can view the location of the target user through the mobile phone APP, so as to rush to the incident site at the fastest speed for rescue.

[0084] In one implementation, before analyzing the several types of the physiological data, first use the fast-RNN neural network to train a signal filtering model based on a deep learning algorithm, use a stacked denoising autoencoder to denoise and repair the signal, and combine it with a traditional Kalman filter model. The combination has high filtering efficiency and high model robustness.

[0085] As Figure 1 shown, the method further includes the following steps:

[0086] Step S300: Determine the physiological safety factor of the target user according to the attitude information and the motion state information.

[0087] Specifically, since the posture information can reflect the position changes of the target user's current body parts, and the motion state information can reflect the physiological state of the target user's current body, when the target user has a falling behavior, both the posture information and the motion state information have specific characteristics respectively. Therefore, in this embodiment, the posture information and the motion state information are combined to comprehensively determine whether the user has a falling behavior, and then the physiological safety coefficient of the target user is determined to improve the accuracy of the judgment. It can be understood that when the target user has a falling behavior or other abnormal behaviors, the value of the physiological safety coefficient is relatively low, and when the target user performs normal behaviors, the value of the physiological safety coefficient is relatively high.

[0088] In one implementation, the step S300 specifically includes the following steps:

[0089] Step S301, obtain the coefficient ratios corresponding to the posture information and the motion state information respectively;

[0090] Step S302, determine the physiological safety coefficient according to the posture information, the motion state information, and the coefficient ratios corresponding to the posture information and the motion state information respectively.

[0091] Specifically, since the posture information can more intuitively and accurately determine whether the target user has a falling behavior, the reliability of the posture information is higher than that of the motion state information. Therefore, when calculating the physiological safety coefficient of the target user, different coefficient ratios can be configured for the posture information and the motion state information respectively. The data with a higher coefficient ratio has a greater impact on the calculation result of the physiological safety coefficient. Finally, according to the coefficient ratios corresponding to the posture information and the motion state information and their respective corresponding values, the physiological safety coefficient of the target user can be comprehensively calculated. For example, assuming that the full score of the physiological safety coefficient is 100 points, the coefficient ratio of the posture information can be set to 70%, and the coefficient ratio of the motion state information can be set to 30%. If the motion state information consists of two types of data, namely the heart rate value and the blood pressure value, the coefficient ratio of the heart rate value can be set to 15%, and the coefficient ratio of the blood pressure value can be set to 15%. In this embodiment, by combining the analysis of the posture information and the motion state information, the physical safety monitoring of the target user can be comprehensively completed. In one implementation, a high-computing power server can also be used to achieve core functions such as accident warning and accident alarm for the target user with low latency, so that the guardians and service platforms corresponding to the target user can learn about the health status of the target user in a timely manner.

[0092] In one implementation, the method further includes the following steps:

[0093] Step S10, generate a body warning information if only the motion state information is in a warning state;

[0094] Step S11: Send the body warning information to the monitoring terminal corresponding to the target user.

[0095] Specifically, if the posture information of the target user is a normal posture and the motion state information is a warning state, it means that the target user has not fallen, but the physical condition is abnormal. Therefore, the system automatically generates the body warning information corresponding to the target user. The body warning information may include the abnormal physiological parameters of the target user. To ensure that the target user can receive timely assistance, the body warning information can be sent to the monitoring terminal corresponding to the target user. For example, the monitoring terminal can be the mobile phone of the target user's guardian.

[0096] In one implementation, the method further includes the following steps:

[0097] Step S20: Perform face recognition operations on several frames of the image data respectively;

[0098] Step S21: Determine whether there is a strange user according to the results of the face recognition operations corresponding to several frames of the image data respectively. The strange user is a user whose face information has not been pre-entered;

[0099] Step S22: When there is the strange user, issue an alarm message.

[0100] Specifically, to further ensure the personal safety of the target user, for each frame of image data obtained by the camera device, in this embodiment, face recognition operations are performed on it to determine whether there is a strange user in the environment where the target user is located. When there is a strange user, it means that the current personal safety of the target user is threatened. Therefore, an alarm message is immediately issued so that the target user can quickly get support.

[0101] In one implementation, the face recognition operation is implemented based on a face recognition algorithm. Several frames of the image data are respectively input into the face recognition algorithm, and the face recognition results are output through the face recognition algorithm. Specifically, the face recognition algorithm in this embodiment is based on the facial features of people. For the input face image or video stream, it first determines whether there is a face. If there is a face, further information such as the position, size of each face and the positions of the main facial organs are given. And based on this information, the identity features contained in each face are further extracted and compared with the known faces to identify the identity of each face. The robot captures and recognizes the faces of the people in the area. Using the AI recognition system, it enters the face database for comparison. If suspicious or unknown people are found in the home, it captures and uploads the data to the data management center, and transmits it to the relative's mobile phone APP in real time, and records the behavior trajectory and behavior analysis of the suspicious or unknown people (as Figure 2 shown).

[0102] In another implementation, the face recognition operation is implemented based on Base-attention deconstruction and a convolutional neural network. Specifically, several frames of the image data are dimensionally reduced and features are extracted through Base-attention deconstruction and a convolutional neural network. At the same time, algorithms such as the cosine distance formula and the Euler distance formula are used to comprehensively calculate the feature encoding. In addition, in this embodiment, a high-quality server is used for GPU-accelerated calculation to ensure that the face recognition results are obtained in a timely manner and the transmission has low latency.

[0103] In one implementation, an environmental sensor group can be pre-set in the environment where the target user is located. Among them, the environmental sensor group can collect several types of home environmental data, such as environmental temperature, environmental humidity, and so on. When any one of several types of home environmental data exceeds a preset warning value, a preset indicator light on the robot gives a prompt and a buzzer alarms; when the environmental temperature and humidity drop below the warning value, the alarm automatically turns off. For example, the environmental sensor group includes a smoke detector sensor. The smoke detector sensor is used to detect the smoke in the surrounding environment of the target user, and can monitor the gas methane in real time, as well as the smoke from a fire or the concentration of smoking around the target user. If it exceeds the standard, the buzzer alarms and an indicator light acts to remind you to solve the current problem. After the problem is solved, these display and indication indicators automatically turn off.

[0104] The technical effects of the present invention can be referred to the following scenarios:

[0105] Scenario 1: When the target user is alone at home, the robot can answer some questions raised by the target user and have a pleasant conversation with the target user.

[0106] Scenario 2: By setting the corresponding reminder time, the robot makes a sound similar to an alarm clock to remind the target user of the time to take medicine or other matters.

[0107] Scenario 3: When there is a toxic gas or smoke in the home, an alarm is issued to remind the target user whether there is a gas leak or the stove has been forgotten to be turned off.

[0108] Scenario 4: When the target user is accidentally injured at home, the robot recognizes the posture of the target user and combines the physiological parameters detected by the smart watch to determine whether the target user needs to call the police.

[0109] Scenario 5: When the target user is happy, such as when it is recognized that the target user is doing exercise or is bored at home, soothing music can be played.

[0110] Scenario 6: When the target user goes out and leaves home, through watch positioning, if the target user gets lost, the robot can call the children to report, and the target user can remind the elderly of the way home.

[0111] Scenario 7: When a stranger appears at home, the target user can accurately identify the stranger and report to family members.

[0112] Scenario 8: Detect and collect the physiological parameters of the target user, such as blood oxygen, blood pressure, respiratory rate, sleep and wake-up time, etc., and send the generated health report of the target user to the children, with the help of the health management center server for the target user.

[0113] The advantages of the present invention are as follows: Compared with using a single data type for gesture recognition, the present invention comprehensively combines physiological data and visual images collected by intelligent wearable devices to perform gesture recognition. The intelligent wearable device can determine the daily movement posture changes of the target user's body through the signal data output by the built-in acceleration sensor and gyroscope, including the acceleration direction and angular velocity direction of the x, y, and z axes. Using physiological data as auxiliary data can improve the fault tolerance rate of visual machine learning in the image recognition process.

[0114] Based on the above embodiments, the present invention also provides an intelligent elderly health monitoring device, as Figure 3 shown. The device includes:

[0115] A gesture monitoring module 01, configured to obtain the activity video data corresponding to the target user, and determine the gesture information corresponding to the target user according to the activity video data;

[0116] A physiological monitoring module 02, configured to obtain the physiological data corresponding to the target user, and determine the exercise state information corresponding to the target user according to the physiological data;

[0117] A state determination module 03, configured to determine the physiological safety coefficient of the target user according to the gesture information and the exercise state information.

[0118] In one implementation, the device uses a self-designed PCB board, which only meets the product functions of its own design, avoiding the waste of energy caused by purchasing others' hardware.

[0119] Based on the above embodiments, the present invention also provides a terminal, and its principle block diagram can be as Figure 4 shown. The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes the intelligent elderly health monitoring method. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0120] Those skilled in the art can understand that Figure 4 the principle block diagram shown in [[ID=]],

[0121] merely represents the block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminals to which the solution of the present invention is applied. The specific terminals may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database or other medium used in the embodiments provided by the present invention may include non-volatile and / or volatile memories. Non-volatile memories may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memories may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0123] In summary, the present invention discloses an intelligent elderly health monitoring method, device, terminal and storage medium. The method obtains the activity video data corresponding to the target user, and determines the posture information corresponding to the target user according to the activity video data; obtains the physiological data corresponding to the target user, and determines the motion state information corresponding to the target user according to the physiological data; and determines the physiological safety coefficient of the target user according to the posture information and the motion state information. The present invention comprehensively determines the physiological safety coefficient of the target user through the posture information and the motion state information of the target user, which can improve the accuracy of iterative detection. It solves the problem that in the prior art, the fall detection method only uses sensor data, and the data type is single, which is prone to misjudgment.

[0124] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. An intelligent elderly health monitoring method, characterized in that, The method includes: Obtaining activity video data corresponding to a target user, and determining pose information corresponding to the target user according to the activity video data; Obtaining physiological data corresponding to the target user, and determining motion state information corresponding to the target user according to the physiological data; Determining a physiological safety coefficient of the target user according to the pose information and the motion state information, including: obtaining coefficient ratios corresponding to the pose information and the motion state information respectively; determining the physiological safety coefficient according to the pose information, the motion state information, and the coefficient ratios corresponding to the pose information and the motion state information respectively.

2. The intelligent elderly health monitoring method according to claim 1, characterized in that, The activity video data includes a plurality of frame image data; obtaining activity video data corresponding to a target user, and determining pose information corresponding to the target user according to the activity video data, includes: Obtaining a plurality of frames of the image data through a preset camera device; Respectively determining target objects in a plurality of frames of the image data, where the target object is the target user captured in a plurality of frames of the image data; Labeling the target objects in a plurality of frames of the image data to obtain a set of recognition points corresponding to each of the plurality of frames of the image data, where the number of recognition points included in each set of recognition points is equal, and the recognition points included in different sets of recognition points have a one-to-one correspondence, and the plurality of recognition points with a corresponding relationship are used to reflect the position information of the same body part of the target user; Determining the pose information according to the set of recognition points corresponding to each of the plurality of frames of the image data.

3. The intelligent elderly health monitoring method according to claim 2, characterized in that, Determining the pose information according to the set of recognition points corresponding to each of the plurality of frames of the image data, includes: Determining movement data corresponding to each of the plurality of body parts according to the set of recognition points corresponding to each of the plurality of frames of the image data; Determining the pose information according to the movement data corresponding to each of the plurality of body parts.

4. The intelligent elderly health monitoring method according to claim 1, characterized in that, The physiological data is several types of physiological data; Determining the motion state information corresponding to the target user according to the physiological data, includes: Obtaining standard data ranges corresponding to several types of the physiological data respectively; When any one type of the physiological data is outside the corresponding standard data range, determining the motion state information as a warning state.

5. The intelligent elderly health monitoring method according to claim 1, characterized in that, The method further includes: When only the motion state information is in a warning state, generating a body warning information; Sending the body warning information to a monitoring terminal corresponding to the target user.

6. The intelligent elderly health monitoring method according to claim 2, characterized in that, The method further includes: Performing face recognition operations on each of the plurality of frames of the image data; Judging whether there is a strange user according to the results of the face recognition operations corresponding to each of the plurality of frames of the image data, where the strange user is a user whose face information has not been pre-entered; When there is the strange user, issuing an alarm information.

7. An intelligent elderly health monitoring device, characterized in that, The device includes: A pose monitoring module, configured to obtain activity video data corresponding to a target user, and determine pose information corresponding to the target user according to the activity video data; A physiological monitoring module, configured to obtain physiological data corresponding to the target user, and determine motion state information corresponding to the target user according to the physiological data; A state determination module, configured to determine a physiological safety coefficient of the target user according to the posture information and the motion state information, including: obtaining coefficient ratios corresponding to the posture information and the motion state information respectively; determining the physiological safety coefficient according to the posture information, the motion state information, and the coefficient ratios corresponding to the posture information and the motion state information respectively.

8. A terminal, characterized in that, The terminal includes a memory and one or more processors; the memory stores one or more programs; the programs include instructions for executing the intelligent elderly health monitoring method according to any one of claims 1-6; the processor is configured to execute the programs.

9. A computer-readable storage medium, on which multiple instructions are stored, characterized in that, The instructions are adapted to be loaded and executed by a processor to implement the steps of the intelligent elderly health monitoring method according to any one of claims 1-6 above.

Images