A fall detection system for elderly persons that is based on head speed and silhouette variations

Abstract

The invention relates to the development of a fall detection system for the elderly population or adults or children. Our system relies exclusively on surveillance cameras for fluid and discreet daily monitoring. Our approach combines the human silhouette and head speed for fall detection. The head is detected from the mask and key points of the silhouette extracted from pixels using a pre-trained neural network. Head speed is calculated by detecting the centroid of the head and by measuring the distance travelled by that point during activity. This advanced method ensures accurate and efficient classification of falls and can be used to detect head injuries caused by high-speed ground impact.

Description

[0001] Titled

[0002] A Fall Detection System for the Elderly Based on Head Speed ​​and Silhouette Variations

[0003] Scope of the invention

[0004] The invention relates to the field of signaling and alarm systems, particularly alarms that respond to specific abnormal or undesired conditions. It focuses specifically on alarms designed to ensure the safety of individuals, especially those that react to inactivity, such as that observed in the elderly. This field includes devices aimed at detecting situations where a lack of activity might indicate a need for immediate assistance, thereby contributing to improved safety and well-being of the individuals being monitored.

[0005] Context of the invention

[0006] Public health faces a major challenge with the increase in falls among older adults, leading to recurrent injuries and hospitalizations. Older adults are particularly vulnerable to falls due to their generally declining health. Vision problems and balance issues, common in this population, contribute to the rise in falls.

[0007] Head injuries are among the most serious consequences of falls. These injuries can have significant neurological implications, potentially leading to conditions such as dementia and Alzheimer's disease. Furthermore, falls are associated with high mortality rates among older adults. In addition to physical injuries, falls can also have psychological consequences. One of the most significant effects is a decrease in self-confidence due to the fear of falling again. This fear can disrupt a person's daily routine and potentially lead to depression, resulting in a decline in overall quality of life.

[0008] Given the significant impact falls have on the lives of older adults, it is clear that they require an effective fall detection system. Early detection is crucial as it considerably reduces the harmful effects of falls. Considering the challenges this age group faces in using modern technology, cameras appear to be an excellent solution for fall detection. Furthermore, this offers the advantage of monitoring multiple individuals simultaneously without requiring body-worn sensors that need frequent adjustments. Therefore, we propose a solution based on Deep Learning, using automatic human silhouette and head detection for more accurate classification.Given that head injuries from high-speed ground impact are a major concern for this population, our solution incorporates this information to provide caregivers with more precise details about the accident, enabling them to make informed decisions. The state-of-the-art documentation includes EP patent 2763116, entitled "Fall detection system and method for detecting a fall of a monitored person." This system uses an infrared source to irradiate the monitored person with a pattern of infrared dots, detectable by a depth-infrared sensor, to generate an image of the infrared pattern. An image processing unit is used to generate at least one identification point located in the head region, based on the infrared dot pattern, and then adds this point to the image received from the infrared sensor.A fall detection module is then used to detect falls by comparing the vertical positions of identification points between two images taken at different time intervals. The system also adjusts the minimum threshold value based on the distance between the monitored person's head and the infrared depth sensor, thus ensuring accurate fall detection.

[0009] Our fall detection system is based on an innovative approach using semantic segmentation to extract the human silhouette from video images (Figure 1). Unlike patent EP 2763116, which uses an infrared dot pattern to detect falls, our system uses cameras exclusively to capture and analyze silhouette variations and head velocity. We employ a convolutional sequential learning model (ConvLSTM+CNN) to understand the temporal changes in the human silhouette, enabling accurate activity classification. By integrating the head velocity detected from the movement of the head's centroid, we effectively distinguish falls from other prone positions, particularly in cases of high velocity, which likely indicates a fall with a risk of head injury.

[0010] To detect the head, our system extracts key points from the body and uses the Mask R-CNN model pre-trained on COCO to segment the silhouette and identify the head's position (Figure 2). We then calculate the head velocity using the distance traveled by the head's centroid between the start and end of the activity, corrected for the actual head size estimated from the average human head height (Figure 4). Unlike the patent, which uses infrared point measurements, our method uses computer vision techniques and OpenCV-based tracking algorithms to accurately track head movement across video images, thus calculating the head velocity (Figure 5).

[0011] In summary, our system represents an advance over existing methods by using a fully visual approach for fall detection, combining semantic segmentation of the human silhouette and head velocity detection for accurate and rapid detection of critical incidents.

[0012] Description

[0013] Our system is based on classifying variations in the human silhouette extracted at the pixel level using semantic segmentation. These variations provide essential information about the nature of the activity (Figure 1). To effectively leverage this information, we implemented convolutional sequential learning (ConvLSTM+CNN), enabling us to understand the temporal changes in the human silhouette for accurate activity classification. The silhouette-based classification result is combined with head velocity to determine the user's situation. A high head velocity indicates a fall and a possible head injury. High velocity also makes it easy to distinguish between a fall and a prone position, as both can exhibit similar silhouette variations.However, a fall is characterized by high speed.

[0014] The human head is detected in our system based on the silhouette and key points of the person's body (Figure 1). We then extract the head's centroid at the beginning and end of the activity to measure the distance traveled by the head and calculate its velocity. One of the main challenges in detecting the velocity of objects from cameras is finding a reference point to convert the distance traveled by the object in pixels into a real-world distance. To solve this problem, we use the height of the head as a reference, since we have the average height of the human head and its height in pixels.

[0015] Our system begins by detecting the human silhouette and identifying key body points using the Mask R-CNN model pre-trained on the COCO dataset (Figure 2). The head is then extracted using this information. Since the Mask R-CNN cannot directly extract the mask from the head, information provided by the location of the nose and the positions of the left and right shoulders or ears is used to perform this operation.

[0016] After extracting the head from key points on the body and silhouette (Figure 3), the next step is to extract the head's Bounding Box. This information is used to track the head and determine the final position of its centroid. To do this, the 'TrackerMedianFlowCreateQ' function from the OpenCV library is used. This function takes the Bounding Box, or region of interest, of the extracted head as input and tracks the head's movements through the video frame sequence. The process begins by initializing a head tracker in the first frame, and then the Median Flow algorithm uses the reference image of the extracted head to estimate the head's position in subsequent frames.

[0017] After extracting the head from key points on the body and silhouette (Figure 3), the next step is to extract the head's Bounding Box. This information is used to track the head and determine the final position of its centroid. To do this, the 'TrackerMedianFlowCreateQ' function from the OpenCV library is used. This function takes the Bounding Box, or region of interest, of the extracted head as input and tracks the head's movements through the video frame sequence. The process begins by initializing a head tracker in the first frame, and then the Median Flow algorithm uses the reference image of the extracted head to estimate the head's position in subsequent frames. The distance d in pixels between the initial position (x t , ) and the final position (x n , y nThe centroid of the head can be calculated using the Euclidean distance formula. The duration of the activity can be determined by calculating the difference between the beginning is the end of activity t n .

[0018] To convert the distance in pixels to a real distance, it is necessary to use a conversion scale (5) based on a known measurement in the image. In the case where the head mask is known, the head height (h) réei ) can be used as a measure to convert the distance in pixels traversed by the head into real distance (Equation 1).

[0019] The average head height for men is 25.5 cm and for women 23.8 cm, measured from the chin to the top of the head (Figure 4). In our study, we chose 25.5 cm because the head mask is always larger than the human head, with a few extra pixels at the edges.

[0020] To calculate the velocity (v in cm / s) of the head, simply use the actual distance d traveled by the centroid from position i to n (Equation 2). v = d_real_t n ~ t_i (2)

[0021] Figure 5 shows an example of extracting the distance traveled by the head. To convert this information into an actual distance, the actual height of the head, which is 25.5 cm, can be used, along with the size of the head in pixels, which is 47. After applying Equation 1, the actual distance is calculated to be 158.19 cm. Since we have information about the duration of the activity (0.61 s), Equation 2 can be used to calculate the speed. Therefore, the speed of the head in this case is 259.32 cm / s

[0022] Finally, this head velocity information can be used to inform caregivers about a fall accident in combination with silhouette variation classification results, using convolutional sequence learning (ConvLSTM) and feature extraction with a convolutional neural network (CNN). Head velocity can also provide information about the severity of the fall and any head injuries caused by a high-speed ground impact.

Claims

Demands Claim 1: A system for detecting a fall in a human subject, comprising: • Capturing a video sequence of the human subject; • Extracting a human silhouette from each image of the video sequence; • Analysis of temporal changes in the human silhouette using a deep learning model to classify the activity as a fall or an activity without a fall; • Tracking the head of the human subject in the video sequence; • Calculating the speed of the head; and • The determination that a fall has occurred based on a combination of the classification result and the calculated head velocity. Claim 2: The method according to claim 1, wherein the deep learning model is a convolutional short- and long-term memory network (ConvLSTM) combined with a convolutional neural network (CNN). Claim 3: The method according to claim 1,2, wherein the speed of the head is calculated as a function of the distance traveled by the center of the head over a predetermined period. Claim 4: The method according to claim 1,2,3, wherein the human silhouette is extracted using a semantic segmentation technique. Claim 5: The method according to claim 1, 2, 3, 4 further comprising the step of generating an alert upon detection of a fall. Claim 6: The method according to claim 1, 2, 3, 4, 6 wherein the video sequence is captured using a surveillance camera.

Images