Abnormal behavior detection system and method for monitoring elderly people living alone

Through the skeleton behavior recognition system of the image acquisition device and the overall processor, the problem of insufficient abnormal behavior detection in the monitoring system of elderly people living alone is solved, timely detection and privacy protection of various abnormal behaviors is realized, and effective health for elderly people living alone is provided. guardianship.

CN115116133BActive Publication Date: 2024-11-26PENG CHENG LAB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210667931.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-14
Publication Date
2024-11-26
Estimated Expiration
2042-06-14

AI Technical Summary

Technical Problem

The existing monitoring system for elderly people living alone has problems such as single identification and insufficient detection of abnormal behaviors other than falling. It is difficult to detect sudden dangers in time based on wearable sensors and position track methods, and there is privacy exposure to video surveillance. and identify inaccurate issues.

Method used

An abnormal behavior detection system including an image acquisition device, a total processor and a remote guardian terminal was designed. A skeleton behavior recognition algorithm was used to locate, track, skeleton extraction, behavior recognition and abnormal judgment of elderly people living alone, and collect images through multiple cameras. Data, the pose estimation model and behavior recognition model are used to realize the detection and alarm of multiple abnormal behaviors, and the behavioral restoration of privacy protection is carried out through 3D models.

Benefits of technology

It realizes timely detection and alarm for various abnormal behaviors other than falling, improves the ability to detect sudden dangers in a timely manner, ensures privacy protection, and provides early warning and guardianship support for health risks for elderly people living alone.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115116133B_ABST
    Figure CN115116133B_ABST
Patent Text Reader

Abstract

The present invention discloses an abnormal behavior detection system and method for monitoring elderly people living alone, the abnormal behavior detection system for monitoring elderly people living alone comprises: an image acquisition device, a main processor and a remote guardian terminal; the image acquisition device is used to collect image data of daily life of elderly people living alone in an indoor environment, and send the image data to the main processor; the main processor is used to locate, track, extract skeletons, identify behaviors and judge abnormalities of the human body based on the behavior recognition algorithm of the skeleton, and detect various abnormal behaviors in the home environment of elderly people living alone; the remote guardian terminal is used to restore the facts of human behavior in a real environment based on the skeleton sequence data of the human body, and to give an alarm according to the abnormal behavior. Under the premise of privacy protection, the present invention can timely monitor the abnormal behavior of elderly people living alone, and timely give an alarm for the detected abnormal behavior, so that the elderly living alone can get timely assistance when abnormal behavior occurs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to an abnormal behavior detection system and method for monitoring elderly people living alone. Background Art

[0002] In recent years, with the aging of the population structure and the deepening of empty nests, elderly people living alone have become a large group. Elderly people living alone refer to the elderly group whose children are not around and who are widowed. They often rely on home-based elderly care institutions or community elderly care. If they fall, have a heart attack or other sudden abnormal behaviors, they may lose their precious lives because no one notices and there is no timely rescue. In addition, many elderly people living alone conceal and delay their abnormal behaviors such as heart discomfort, back pain, and headaches for fear of burdening their children, which has a great impact on the timely prevention, detection, and treatment of chronic diseases in the elderly. Therefore, real-time intelligent identification of abnormal behaviors of elderly people living alone in daily life is becoming a problem that needs to be solved urgently in the elderly care process of elderly people living alone.

[0003] However, the existing monitoring systems for elderly people living alone on the market mainly focus on the elderly's vital signs (such as heart rate, blood pressure, etc.). In terms of abnormal behavior, only a few products focus on the sudden abnormal situation of elderly people falling, and there is a lack of effective analysis of other abnormal behaviors other than falling, such as back pain, coughing, headache, vomiting, staggering, abnormal wandering, etc. The reason is that the existing abnormal behavior detection methods for elderly people living alone have the following shortcomings: the wearable sensor-based method has a single type of behavior recognition due to the limited sensor types, and the elderly are prone to forget to wear them; the location trajectory-based method is heavily dependent on the user's historical data, and it is difficult to detect and alarm sudden and dangerous abnormal behaviors in time. The video surveillance-based method has the problems of indoor monitoring privacy exposure, manual extraction of behavioral features, lack of time series information, and few and inaccurate recognition types.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide an abnormal behavior detection system and method for monitoring elderly people living alone, aiming to solve the problem that the existing system for monitoring elderly people living alone has a single abnormal behavior recognition function and is insufficient in detecting various abnormal behaviors other than falls.

[0006] To achieve the above object, the present invention provides an abnormal behavior detection system for monitoring elderly people living alone, the abnormal behavior detection system for monitoring elderly people living alone comprising:

[0007] An image acquisition device, a main processor and a remote guardian terminal; the main processor is connected to the image acquisition device and the remote guardian terminal respectively;

[0008] The image acquisition device is used to collect image data of the daily life of an elderly person living alone in an indoor environment, and send the image data to the main processor;

[0009] The main processor is used to locate, track, extract skeletons, identify behaviors and judge abnormalities of human bodies based on the skeleton-based behavior recognition algorithm, and detect various abnormal behaviors of elderly people living alone in their home environment;

[0010] The remote guardian terminal is used to restore the facts of human behavior in a real environment based on the skeleton sequence data of the human body, and to issue an alarm based on abnormal behavior.

[0011] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the image acquisition device comprises: a plurality of cameras and a data storage processor;

[0012] The plurality of cameras are respectively arranged above a plurality of activity areas where the elderly living alone move, and are used to capture image data of the daily life of the elderly living alone in an indoor environment from multiple angles;

[0013] The data storage processor is used to store image data captured by multiple cameras.

[0014] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the main processor includes: an image reading module, a recognition and tracking module, a skeleton sequence extraction module, a behavior recognition module and an abnormality determination module;

[0015] The image reading module, the recognition and tracking module, the skeleton sequence extraction module, the behavior recognition module and the abnormality determination module are connected in sequence;

[0016] The image reading module is used to receive the image data captured by the multiple cameras, obtain the video image within the target activity space area, and send the read video image to the target recognition and tracking module;

[0017] The recognition and tracking module is used to detect human targets according to video images to obtain human target areas, and use target tracking algorithms to track the same target in multiple frames of images to obtain a target image stream of the elderly living alone, and send the target image stream of the elderly living alone to the skeleton sequence extraction module;

[0018] The skeleton sequence extraction module is used to use a posture estimation model to obtain the current posture data of the elderly living alone according to the target image stream, wherein the posture data is multiple skeleton key point data of the human body, and the skeleton key points within a period of time are combined to form skeleton sequence data, and the skeleton sequence data is sent to the behavior recognition module;

[0019] The behavior recognition module is used to input the skeleton sequence data into the trained behavior recognition model for recognition, extract the human body's behavior characteristics in time and space, obtain the behavior classification result, and send the behavior classification result to the abnormality determination module;

[0020] The anomaly determination module is used to dynamically update the frequency and duration data of various current behaviors at each moment according to the behavior classification results, and to set threshold triggers according to multi-dimensional data, formulate anomaly determination strategies, and realize the identification of multiple types of abnormal behaviors.

[0021] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the remote guardian terminal includes: a monitoring display module and an abnormal alarm module;

[0022] The monitoring display module is used to realize the fact restoration of human behavior in a real environment through the 3D model by driving the 3D model based on the skeleton sequence data of the human body;

[0023] The abnormal alarm module is used to obtain abnormal behavior determination information and generate notification, emergency notification and alarm information according to the abnormal behavior level.

[0024] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the plurality of cameras are automatic tracking cameras.

[0025] Optionally, the abnormal behavior detection system for monitoring elderly people living alone, wherein the step of acquiring video images within the target activity space area includes:

[0026] The image reading module automatically selects the camera in the activity space area where the elderly person living alone is located according to the foreground human body detection result of the camera, and obtains the video image of the camera in the activity space area where the elderly person living alone is located.

[0027] Optionally, the abnormal behavior detection system for monitoring elderly people living alone, wherein the human target detection is performed based on the video image to obtain the human target area, and the target tracking algorithm is used to track the same target in multiple frames of images to obtain the target image stream of the elderly living alone, includes:

[0028] The recognition and tracking module constructs a network model, and extracts distribution features of human target objects in video images based on the network model;

[0029] The recognition and tracking module detects and generates the positions of all human targets and target object frames in the video image;

[0030] The recognition and tracking module predicts the current position of each target object frame using a Kalman filter according to the spatial continuity of the human object target movement, and then associates the target object frame to the existing human target number through the Hungarian algorithm. If there is no corresponding target, a new human target number is created. When a new human target number is generated, a face recognition algorithm is called to identify the person. When it is detected that the acquired target video stream is the target elderly person living alone, the target video stream of the current human target number is acquired.

[0031] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the identification and tracking module is also used to compensate and correct the human target frame output by the network model using Kalman filtering when detection discontinuity appears in the target detection result, and to infer the human body area position using the Kalman filtering algorithm, and to add the human body area position to the target detection result.

[0032] Optionally, the abnormal behavior detection system for monitoring elderly people living alone, wherein the step of combining skeleton key points within a period of time to form skeleton sequence data includes:

[0033] The skeleton sequence extraction module extracts a frame of video from the target image stream;

[0034] The skeleton sequence extraction module extracts the next frame of video at intervals of a preset number of video frames and sends it into a queue;

[0035] The skeleton sequence extraction module checks the queue length and packages it into skeleton sequence data when the queue length reaches a set number.

[0036] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the behavior classification results include coughing, staggering, falling, headache, covering the heart, back pain, neck pain, vomiting, standing, sitting, standing, walking, sitting and lying down.

[0037] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the abnormal behavior includes: sudden abnormal behavior, abnormal health behavior and abnormal status behavior.

[0038] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the judgment dimensions of the abnormal behavior include: action category, duration and frequency of occurrence.

[0039] Optionally, in the abnormal behavior detection system for monitoring elderly people living alone, the method of driving the 3D model through skeleton data includes:

[0040] The monitoring and display module obtains real-time skeleton sequence data of the elderly living alone;

[0041] The monitoring display module binds the virtual 3D model and performs alignment and calibration of the skeleton sequence data;

[0042] The monitoring display module uses the skeleton sequence data to drive and update the position and posture of the 3D model in real time.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides an abnormal behavior detection method for monitoring elderly people living alone, and the abnormal behavior detection method for monitoring elderly people living alone comprises the following steps:

[0044] The image acquisition device acquires image data of the daily life of an elderly person living alone in an indoor environment, and sends the image data to the main processor;

[0045] The main processor performs positioning, tracking, skeleton extraction, behavior recognition and abnormality determination on the human body based on the skeleton behavior recognition algorithm, and detects various abnormal behaviors of elderly people living alone in their home environment;

[0046] The remote guardian terminal realizes the factual restoration of human behavior in a real environment based on the skeleton sequence data of the human body, and issues an alarm according to abnormal behavior.

[0047] In the present invention, the abnormal behavior detection system for monitoring elderly people living alone includes: an image acquisition device, a main processor and a remote guardian terminal; the main processor is connected to the image acquisition device and the remote guardian terminal respectively; the image acquisition device is used to collect image data of the daily life of elderly people living alone in an indoor environment, and send the image data to the main processor; the main processor is used to locate, track, extract skeletons, identify behaviors and judge abnormalities of the human body based on the skeleton behavior recognition algorithm, and detect various abnormal behaviors in the home environment of elderly people living alone; the remote guardian terminal is used to restore the facts of human behavior in a real environment based on the skeleton sequence data of the human body, and to give an alarm according to the abnormal behavior. Under the premise of privacy protection, the present invention can timely monitor the abnormal behavior of elderly people living alone, and timely give an alarm for the detected abnormal behavior, so that the elderly living alone can get timely assistance when abnormal behavior occurs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the principle of a preferred embodiment of the abnormal behavior detection system for monitoring elderly people living alone of the present invention;

[0049] Figure 2It is a schematic diagram of the distribution of cameras in the activity space of the elderly living alone in a preferred embodiment of the abnormal behavior detection system for monitoring the elderly living alone of the present invention;

[0050] Figure 3 It is a schematic diagram of a behavior recognition framework based on skeleton sequences in a preferred embodiment of the abnormal behavior detection system for monitoring elderly people living alone of the present invention;

[0051] Figure 4 It is a schematic diagram of the skeleton sequence extraction part in a preferred embodiment of the abnormal behavior detection system for monitoring elderly people living alone of the present invention;

[0052] Figure 5 It is a schematic diagram of a 3D model driven by a skeleton sequence in a preferred embodiment of the abnormal behavior detection system for monitoring elderly people living alone of the present invention;

[0053] Figure 6 This is a schematic diagram of a monitoring display interface in a preferred embodiment of the abnormal behavior detection system for monitoring elderly people living alone of the present invention;

[0054] Figure 7 It is a flow chart of a preferred embodiment of the abnormal behavior detection method for monitoring elderly people living alone of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] In order to overcome the problem that the existing monitoring systems for elderly people living alone have a single abnormal behavior recognition system and insufficient detection of various abnormal behaviors other than falls, the present invention provides an abnormal behavior detection system for monitoring elderly people living alone, which can not only promptly detect other dangerous sudden abnormal behaviors other than falls, such as vomiting, coughing, etc., but also analyze abnormal behaviors that constitute long-term health risks, such as headaches, back pain, staggering, heart discomfort, abnormal wandering, etc. At the same time, in view of the problem of visual privacy exposure, visual privacy protection technology is adopted to achieve safe detection, analysis and alarm of various abnormal behaviors of elderly people living alone indoors.

[0057] The abnormal behavior detection system for monitoring elderly people living alone described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the abnormal behavior detection system for monitoring elderly people living alone includes: an image acquisition device 10, a main processor 20 and a remote guardian terminal 30; the main processor 20 is connected to the image acquisition device 10 and the remote guardian terminal 30 respectively.

[0058] The image acquisition device 10 is used to acquire image data of the daily life of an elderly person living alone in an indoor environment, and send the image data to the main processor 20 .

[0059] The main processor 20 is used to locate, track, extract skeletons, identify behaviors and judge abnormalities of the human body based on the skeleton-based behavior recognition algorithm, and detect various abnormal behaviors of elderly people living alone in their home environment.

[0060] The remote guardian terminal 30 is used to restore the actual human behavior in a real environment based on the skeleton sequence data of the human body, and to issue an alarm based on abnormal behavior.

[0061] Specifically, the image acquisition device 10 includes: a plurality of cameras 11 (for example Figure 1 The plurality of cameras 11 are respectively arranged above a plurality of activity areas where the elderly living alone are active, and are used to capture image data of the daily life of the elderly living alone in an indoor environment from multiple angles, such as Figure 2 As shown, a camera is set at a corner of the activity space 1, two cameras are set at two corners of the activity space 2, and a camera is set at a corner of the activity space 3, and the multiple cameras 11 are automatic tracking cameras. The multiple cameras 11 are placed above multiple activity places of the elderly living alone (such as living rooms, kitchens, bedrooms, etc.) to capture the most comprehensive daily life video images of the elderly living alone in the environment; the data storage processor 12 is used to store image data captured by the multiple cameras 11.

[0062] Specifically, the main processor 20 includes: an image reading module 21, a recognition and tracking module 22, a skeleton sequence extraction module 23, a behavior recognition module 24 and an abnormality judgment module 25; the image reading module 21, the recognition and tracking module 22, the skeleton sequence extraction module 23, the behavior recognition module 24 and the abnormality judgment module 25 are connected in sequence.

[0063] Among them, the image reading module 21 is used to receive image data captured by multiple cameras 11 in the home environment of the elderly living alone, obtain video images within the target activity space area, and send the read video images to the target recognition and tracking module 22; the image reading module 21 automatically selects the camera in the activity space area where the elderly living alone is located according to the camera's foreground human detection results (used to determine which camera range the elderly living alone is mainly active in), and obtains the video images of the cameras in the activity space area where the elderly living alone is located.

[0064] The recognition and tracking module 22 is used to detect human targets according to the video image to obtain the human target area, and use the target tracking algorithm to track the same target in multiple frames of images to obtain the target image stream of the elderly living alone, and send the target image stream of the elderly living alone to the skeleton sequence extraction module 23; further, Figure 3 As shown, the recognition and tracking module 22 constructs a network model (such as YOLO_Tiny), and extracts the distribution characteristics of human target objects in the video image based on the network model; detects the positions of all human targets and target object frames in the generated video image; for each target object frame, according to the spatial continuity of the human object target movement, that is, the position of the human object target in the next frame is located near the position of the same object target in the previous frame, the Kalman filter is used to predict the current position, and then the target object frame is associated with the existing human target number through the Hungarian algorithm. If there is no corresponding target, a new human target number is created, thereby realizing the tracking of each human target.

[0065] Considering that elderly people living alone spend most of their time at home alone, in order to improve the speed of the network model and reduce unnecessary computing overhead, the number of target detections of the network model used is limited to two people or less; at the same time, in order to ensure that the target image stream obtained is a valid video stream of the monitored elderly people living alone, when a new human target number is generated, the face recognition algorithm is called to perform character identification. When it is detected that the acquired target video stream is the target elderly person living alone, the target video stream of the current human target number is obtained and sent to the skeleton sequence extraction module 23 for processing.

[0066] Furthermore, given that the target detection results may be discontinuous, especially when a person is performing some relatively fast actions, such as falling, some frames may not detect the human body, which has a great impact on the accuracy of subsequent behavior recognition based on continuous skeleton sequences. In view of this situation, the present invention introduces Kalman filtering to compensate and correct the human target frame output by the target detection network. When the human body area is not detected, the Kalman filtering algorithm is used for reasoning to estimate the position of the human body area and add it to the target detection result.

[0067] Among them, Figure 3 As shown, the skeleton sequence extraction module 23 is used to adopt a posture estimation model to obtain the current posture data of the elderly living alone according to the target image stream. The posture data is multiple skeletal key point data of the human body, and the skeletal key points within a period of time are combined to form skeleton sequence data, and the skeleton sequence data is sent to the behavior recognition module 24.

[0068] For the posture estimation model, the present invention adopts the FastPose network model of AlphaPose to obtain the skeletal key point information of the elderly living alone at each moment. Since a variety of posture estimation models can achieve this goal, they can also be used as equivalent replacements.

[0069] like Figure 4 As shown, the acquired skeleton sequence data reflects the behavioral posture information of the elderly living alone over a period of time. Since the subsequent recognition model based on the skeleton sequence needs to extract features in time and space, the reasoning speed may not be able to achieve real-time detection. The present invention finds that the elderly living alone move slowly, the posture of the current video frame is very similar to that of the previous frames, and the posture information is relatively redundant. If the traditional frame-by-frame skeleton sequence extraction method is used, it is easy to cause problems such as limited extraction speed and delayed subsequent behavior recognition results. The model reasoning speed can be increased by a regular video frame extraction method.

[0070] Therefore, in the present invention, the present invention also discloses a method for extracting skeletons: (1) extracting a frame of video from the target image stream; (2) extracting the next frame after a certain number of video frames, and sending it to the queue. The present invention uses an interval of 3 frames, but in practice, a suitable interval of video frames can be selected according to the final effect; (3) checking the queue length, and packaging it to form skeleton sequence data after reaching a certain number, and sending it to the behavior recognition module 24. The present invention uses 30 frames as the extraction queue length to achieve the best effect. In practice, the queue length can also be dynamically adjusted to adapt to most of the behavior action types in the model to achieve the best result.

[0071] The behavior recognition module 24 is used to input the skeleton sequence data into the trained behavior recognition model for recognition, extract the human behavior characteristics in time and space, obtain the behavior classification results, and send the behavior classification results to the abnormality determination module 25. Compared with the traditional method, it has the characteristics of light weight, fast speed, strong pertinence and high accuracy.

[0072] like Figure 3 As shown, the behavior recognition model is based on a behavior recognition framework based on skeleton sequences. The target video stream is obtained through the recognition and tracking module 22, the skeleton sequence is obtained through the skeleton sequence extraction module 23, and the spatiotemporal graph of the skeleton sequence is constructed. Then, the multi-layer spatiotemporal graph convolution ST-GCN is used to gradually generate higher-level feature graphs on the graph, and finally the standard SoftMax classifier is used to classify it into corresponding action categories.

[0073] In the present invention, the trained behavior recognition model is a ST-GCN model containing 14 behavior categories generated by training on a self-made data set, including coughing, staggering, falling, headache, covering the heart, backache, neck pain, vomiting, standing, sitting, standing, walking, sitting, and lying. Among them, in order to reduce the misjudgment of some relatively static behaviors of elderly people living alone, some daily state behaviors such as standing, sitting, and lying are specially added.

[0074] It should be noted that the behavior recognition model in the present invention only selects some representative behavior categories for abnormality judgment. In other embodiments, the trained behavior recognition model can be other behavior recognition models based on skeleton sequences, and can at least recognize one or more behaviors including falling, vomiting, coughing, staggering, walking, headache, back pain, covering the heart, etc.

[0075] The abnormality determination module 25 is used to dynamically update the frequency and duration data of various current behaviors at each moment according to the behavior classification results, and to set threshold triggers according to multi-dimensional data, formulate abnormality determination strategies, and realize the identification of multiple types of abnormal behaviors.

[0076] Furthermore, the abnormal behavior includes: sudden abnormal behavior, health abnormal behavior and status abnormal behavior;

[0077] Among them, sudden abnormal behavior judgment: some sudden abnormal behaviors can directly reflect the safety and health information of elderly people living alone, such as falling, vomiting, etc.; the investigation found that falling is one of the important reasons for the high mortality rate of elderly people living alone. When the elderly fall, if they cannot be rescued in time, it will bring very serious life-threatening dangers; and patients with angina pectoris or coronary heart disease have vomiting as the first manifestation. In cerebrovascular diseases, jet-like vomiting caused by increased intracranial pressure is a disease that endangers the life and health of the elderly. Abnormal behaviors like this are characterized by sudden occurrence and serious consequences. Therefore, when such actions occur, there is a certain danger. When this type of action occurs, it is judged as abnormal. When it exceeds a certain time threshold, this type of abnormal behavior needs to be set to the highest level.

[0078] Among them, the judgment of abnormal health behavior: there is a type of behavior that has no obvious impact when it occurs in a small amount for a short time, but if it is judged in combination with the frequency, it may reflect the long-term health risks of the elderly living alone. For example, long-term and frequent support of the waist can reflect the elderly’s waist discomfort, multiple hands covering the chest may have heart problems, and frequent unsteady steps reflect certain chronic diseases, etc. As the elderly’s physical functions weaken with age, early detection of such abnormal behaviors can timely discover the health risks of the elderly living alone, and at the same time provide more accurate auxiliary diagnostic information for the elderly to seek medical treatment. Therefore, it is necessary to make judgments based on the frequency, and when a certain frequency is reached, the abnormal level is upgraded.

[0079] Among them, the determination of abnormal behavior: There is another type of behavior that is caused by daily behaviors that last too long and the frequency exceeds the normal range. For the special group of elderly people living alone, it indirectly indicates that the elderly living alone are in an abnormal state to a certain extent. For example, the behavior of elderly people living alone wandering indoors for a long time often reflects the abnormal health status of the elderly, such as memory impairment, emotional instability, or Alzheimer's disease, etc.; in addition, the elderly have been drowsy for a long time, which also shows that they may suffer from certain chronic diseases such as cerebral arteriosclerosis. It is necessary to combine the duration and frequency of each behavior to determine the abnormal level with a comprehensive threshold. When both exceed the threshold, the abnormal level is upgraded.

[0080] It should be noted that in the present invention, the data of three dimensions, namely, action category, duration, and frequency of occurrence, are mainly used to calibrate three abnormal behaviors, including sudden abnormality, state abnormality, and health abnormality, and determine two different levels of abnormality. In other embodiments, according to the actual behavior category, data of at least one dimension can be used to calibrate the characteristics of the abnormal behavior, and the abnormal level of the abnormal behavior can be determined using at least one level of rules.

[0081] The present invention uses a remote guardian terminal 30 to monitor the daily activities of elderly people living alone. However, due to the frequent occurrence of monitoring leakage problems in recent years, the present invention adopts a privacy protection monitoring method based on skeleton sequences. By receiving skeleton sequences, human behavior is visually restored and remote monitoring is performed. Different from traditional methods, this method hides the facial identity information, living environment details, etc. of the monitored elderly, and monitors and alarms under the premise of visual privacy protection.

[0082] Specifically, the remote guardian terminal 30 includes: a monitoring display module 31 and an abnormal alarm module 32 .

[0083] The monitoring display module 31 is used to drive the 3D model based on the skeleton sequence data of the human body, and realize the factual restoration of human behavior in the real environment through the 3D model, so as to facilitate the guardian to check and confirm the real-time condition of the elderly.

[0084] Furthermore, in the present invention, a method of driving a 3D model with human skeleton data is disclosed, and the steps mainly include: (1) obtaining real-time skeleton sequence data of an elderly person living alone; (2) binding a virtual 3D model, and aligning and calibrating the skeleton data; (3) using the skeleton data to drive the update of the position and posture of the 3D model in real time.

[0085] Reference Figure 5 and Figure 6,From the implementation effect, it can be seen that this method can ,better restore human behavior, hide unnecessary privacy information, and ,provide real-time monitoring when only data with relatively weak privacy such as the ,human skeleton are obtained.

[0086] The abnormal alarm module 32 is used to obtain abnormal behavior determination information, and generate notification, emergency notification and alarm information according to the abnormal behavior level, and notify relevant departments and personnel.

[0087] The present invention provides an abnormal behavior detection system for elderly people living alone. Under the premise of visual privacy protection, it can detect a variety of abnormal behaviors including falls and send out alarms in time, which is convenient for guardianship agencies, medical care, and children to find and rescue in time, and prevent and treat health risks, thus providing strong support for the guardianship of elderly people living alone.

[0088] The present invention can not only detect dangerous sudden abnormal behaviors, such as falling, vomiting, etc., but also analyze abnormal behaviors that constitute long-term health risks, such as headaches, back pain, staggering, coughing, heart discomfort, abnormal wandering, etc. At the same time, under the premise of privacy protection, it provides a means of remote monitoring of elderly people living alone, providing effective support for the safety and health protection of elderly people living alone.

[0089] Furthermore, based on the above-mentioned abnormal behavior detection system for monitoring elderly people living alone, the present invention also provides an abnormal behavior detection method for monitoring elderly people living alone. The abnormal behavior detection method for monitoring elderly people living alone described in the preferred embodiment of the present invention is as follows: Figure 7 As shown, the abnormal behavior detection method for monitoring elderly people living alone includes the following steps:

[0090] Step S10, the image acquisition device acquires image data of the daily life of the elderly living alone in an indoor environment, and sends the image data to the main processor;

[0091] Step S20, the main processor performs positioning, tracking, skeleton extraction, behavior recognition and abnormality determination on the human body based on the skeleton behavior recognition algorithm, and detects various abnormal behaviors of the elderly living alone in the home environment;

[0092] Step S30: The remote guardian terminal restores the actual human behavior in a real environment based on the human skeleton sequence data, and issues an alarm based on abnormal behavior.

[0093] In summary, the present invention provides an abnormal behavior detection system and method for monitoring elderly people living alone, and the abnormal behavior detection system for monitoring elderly people living alone includes: an image acquisition device, a main processor and a remote guardian terminal; the main processor is connected to the image acquisition device and the remote guardian terminal respectively; the image acquisition device is used to collect image data of the daily life of elderly people living alone in an indoor environment, and send the image data to the main processor; the main processor is used to locate, track, extract skeletons, identify behaviors and judge abnormalities of the human body based on the skeleton behavior recognition algorithm, and detect various abnormal behaviors in the home environment of elderly people living alone; the remote guardian terminal is used to restore the facts of human behavior in a real environment based on the skeleton sequence data of the human body, and to give an alarm according to the abnormal behavior. Under the premise of privacy protection, the present invention can timely monitor the abnormal behavior of elderly people living alone, and timely give an alarm for the detected abnormal behavior, so that the elderly living alone can get timely assistance when abnormal behavior occurs.

[0094] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0095] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0096] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An abnormal behavior detection system for monitoring elderly people living alone, characterized in that: The abnormal behavior detection system for monitoring elderly people living alone comprises: An image acquisition device, a main processor and a remote guardian terminal; the main processor is connected to the image acquisition device and the remote guardian terminal respectively; The image acquisition device is used to collect image data of the daily life of an elderly person living alone in an indoor environment, and send the image data to the main processor; The main processor is used to locate, track, extract skeletons, identify behaviors and judge abnormalities of human bodies based on the skeleton-based behavior recognition algorithm, and detect various abnormal behaviors of elderly people living alone in their home environment; The remote guardian terminal is used to restore the human body behavior in a real environment based on the human skeleton sequence data, and to issue an alarm based on abnormal behavior; The image acquisition device includes: a plurality of cameras and a data storage processor; The plurality of cameras are respectively arranged above a plurality of activity areas where the elderly living alone move, and are used to capture image data of the daily life of the elderly living alone in an indoor environment from multiple angles; The data storage processor is used to store the image data captured by the multiple cameras; The main processor includes: an image reading module, a recognition and tracking module, a skeleton sequence extraction module, a behavior recognition module and an abnormality determination module; The image reading module, the recognition and tracking module, the skeleton sequence extraction module, the behavior recognition module and the abnormality determination module are connected in sequence; The image reading module is used to receive the image data captured by the multiple cameras, obtain the video image within the target activity space area, and send the read video image to the target recognition and tracking module; The recognition and tracking module is used to detect human targets according to video images to obtain human target areas, and use target tracking algorithms to track the same target in multiple frames of images to obtain a target image stream of the elderly living alone, and send the target image stream of the elderly living alone to the skeleton sequence extraction module; The skeleton sequence extraction module is used to use a posture estimation model to obtain the current posture data of the elderly living alone according to the target image stream, wherein the posture data is multiple skeleton key point data of the human body, and the skeleton key points within a period of time are combined to form skeleton sequence data, and the skeleton sequence data is sent to the behavior recognition module; The behavior recognition module is used to input the skeleton sequence data into the trained behavior recognition model for recognition, extract the human body's behavior characteristics in time and space, obtain the behavior classification result, and send the behavior classification result to the abnormality determination module; The abnormality determination module is used to dynamically update the frequency and duration data of various behaviors at each moment according to the behavior classification results, and to set threshold triggers according to multi-dimensional data, formulate abnormality determination strategies, and realize the identification of multiple types of abnormal behaviors; Characterizing the abnormal behavior according to the frequency, the duration data, and the action category; The abnormal behavior is judged according to a preset grade rule to obtain the abnormal behavior grade.

2. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1 is characterized in that: The remote guardian terminal includes: a monitoring display module and an abnormal alarm module; The monitoring display module is used to realize the fact restoration of human behavior in a real environment through the 3D model by driving the 3D model based on the skeleton sequence data of the human body; The abnormal alarm module is used to obtain abnormal behavior determination information and generate notification, emergency notification and alarm information according to the abnormal behavior level.

3. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1 is characterized in that: The plurality of cameras are automatic tracking cameras.

4. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1 is characterized in that: The method of obtaining a video image within the target activity space area includes: The image reading module automatically selects the camera in the activity space area where the elderly person living alone is located according to the foreground human body detection result of the camera, and obtains the video image of the camera in the activity space area where the elderly person living alone is located.

5. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1 is characterized in that: The method of performing human target detection according to the video image to obtain a human target area, and using a target tracking algorithm to track the same target in multiple frames of images to obtain a target image stream of the elderly living alone includes: The recognition and tracking module constructs a network model, and extracts distribution features of human target objects in video images based on the network model; The recognition and tracking module detects and generates the positions of all human targets and target object frames in the video image; The recognition and tracking module predicts the current position of each target object frame using a Kalman filter according to the spatial continuity of the human object target movement, and then associates the target object frame to the existing human target number through the Hungarian algorithm. If there is no corresponding target, a new human target number is created. When a new human target number is generated, a face recognition algorithm is called to identify the person. When it is detected that the acquired target video stream is the target elderly person living alone, the target video stream of the current human target number is acquired.

6. The abnormal behavior detection system for monitoring elderly people living alone according to claim 5 is characterized in that: The recognition and tracking module is also used to compensate and correct the human target frame output by the network model using Kalman filtering when detection discontinuity occurs in the target detection result, use the Kalman filtering algorithm to infer the human area position, and add the human area position to the target detection result.

7. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1 is characterized in that: The skeleton key points within a period of time are combined to form skeleton sequence data, including: The skeleton sequence extraction module extracts a frame of video from the target image stream; The skeleton sequence extraction module extracts the next frame of video at intervals of a preset number of video frames and sends it into a queue; The skeleton sequence extraction module checks the queue length, and packages the queue data to form skeleton sequence data when the queue length reaches a set number.

8. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1 is characterized in that: The behavioral classification results include coughing, stumbling, falling, headache, holding heart, back pain, neck pain, vomiting, standing, sitting, standing, walking, sitting and lying down.

9. The abnormal behavior detection system for monitoring elderly people living alone according to claim 1, characterized in that: The abnormal behaviors include: sudden abnormal behaviors, health abnormal behaviors and status abnormal behaviors.

10. The abnormal behavior detection system for monitoring elderly people living alone according to claim 9, characterized in that: The judgment dimensions of the abnormal behavior include: action category, duration and frequency of occurrence.

11. The abnormal behavior detection system for monitoring elderly people living alone according to claim 2, characterized in that: The method of driving the 3D model through skeleton data includes: The monitoring and display module obtains real-time skeleton sequence data of the elderly living alone; The monitoring display module binds the virtual 3D model and performs alignment and calibration of the skeleton sequence data; The monitoring display module uses the skeleton sequence data to drive and update the position and posture of the 3D model in real time.

12. An abnormal behavior detection method for monitoring elderly people living alone based on the abnormal behavior detection system for monitoring elderly people living alone according to any one of claims 1 to 11, characterized in that: The abnormal behavior detection method for monitoring elderly people living alone includes: The image acquisition device acquires image data of the daily life of an elderly person living alone in an indoor environment, and sends the image data to the main processor; The main processor performs positioning, tracking, skeleton extraction, behavior recognition and abnormality determination on the human body based on the skeleton behavior recognition algorithm, and detects various abnormal behaviors of elderly people living alone in their home environment; The remote guardian terminal realizes the factual restoration of human behavior in a real environment based on the skeleton sequence data of the human body, and issues an alarm according to abnormal behavior.