Early warning system for falling of old people from building based on multi-mode artificial intelligence model

Abstract

The invention discloses a building falling early warning system for old people based on a multi-modal artificial intelligence model. The building falling early warning system comprises a data acquisition module, a multi-modal fusion module, an artificial intelligence analysis module, an early warning module and a background management module. The data acquisition module is used for acquiring multi-modal data such as video image data, wearable equipment data and environment data of old people; the multi-modal fusion module carries out fusion processing on the data to generate a multi-modal feature vector; the artificial intelligence analysis module analyzes the feature vectors based on a multi-modal artificial intelligence model, and outputs the probability and confidence of the falling risk; when the early warning condition is met, the early warning module is started and sends early warning information to related personnel; the background management module carries out all-around management on the system. According to the invention, various modal data are fused, the artificial intelligence technology is utilized to realize early warning and timely warning of the falling risk of the old people, related personnel can be timely reminded to take measures, the occurrence probability of falling accidents is reduced, the life safety of the old people is guaranteed, and the system is suitable for various old people institutions and residential places of the old people, and has important safety protection value.

Description

[0001] manual Technical Field

[0002] The present invention relates to the technical field of elderly safety monitoring technology, and in particular to an elderly falling warning system based on a multimodal artificial intelligence model. Background Art

[0003] With the increasing elderly population, elderly safety issues are a growing concern, with falls from buildings being a particularly serious safety incident. Elderly people may accidentally approach windows or balcony edges due to declining physical function, decreased balance, cognitive impairment, or emotional distress, putting them at risk of falling. Traditional preventive measures rely primarily on physical safeguards (such as guardrails) and manual supervision. However, physical safeguards can be vulnerable to holes or accidental movement, while manual supervision struggles to provide full-time, comprehensive monitoring, making it prone to oversight.

[0004] In recent years, video surveillance technology has been widely used in the field of safety and security. However, simple video surveillance systems require real-time human monitoring and are unable to automatically and promptly detect elderly people approaching dangerous areas and issue warnings. Some automatic detection methods based on single video analysis are also under investigation, but they are susceptible to factors such as ambient lighting changes and obstructions, leading to false positives and missed alerts. Furthermore, video data alone cannot fully and accurately determine key information such as the elderly person's posture, movement intent, and distance from dangerous areas. Therefore, a more intelligent and efficient elderly fall warning system that can integrate multimodal data is needed to improve the accuracy and timeliness of fall risk detection and provide more reliable protection for the elderly's life safety. Summary of the Invention

[0005] The purpose of the present invention is to provide an elderly fall warning system based on a multimodal artificial intelligence model, which integrates data from multiple modalities and uses artificial intelligence technology to achieve early and timely warning of the risk of elderly people falling from buildings.

[0006] The elderly falling warning system of the present invention includes the following modules:

[0007] Data acquisition module: used to collect multiple modal data of the elderly, including video image data, wearable device data (such as location data, posture data, heart rate data and other physiological and motion data) and environmental data (such as indoor window and balcony status data, weather data, etc.).

[0008] Video image data can be collected through cameras installed in nursing homes or elderly activity areas (such as rooms, living rooms, corridors, etc.) to observe the elderly's position, movements, posture and other information; wearable device data is monitored and transmitted in real time by devices worn by the elderly to capture the elderly's real-time position, posture changes (such as body tilt angle), heart rate and other physiological and motion characteristics; environmental data is obtained through corresponding sensors (such as window switch sensors, balcony door sensors, weather sensors, etc.), which helps to assist in determining whether there are risk factors for falling in the elderly's environment, such as whether the windows are open, whether the weather is bad (such as strong winds, heavy rain, etc.), etc.

[0009] Multimodal Fusion Module: This module fuses the collected data from different modalities. First, preprocessing the data, such as cleaning, normalization, and time synchronization, is performed to eliminate discrepancies and noise. Then, using deep learning and other techniques, it organically integrates the position, motion, and posture features of the video images; the position, posture, and physiological characteristics of the wearable device; and environmental data such as window and balcony status and weather information. This generates a multimodal feature vector that comprehensively reflects the risk of elderly people falling from buildings.

[0010] Artificial Intelligence Analysis Module: This module builds a fall risk identification algorithm based on a multimodal AI model. Using a fused multimodal feature vector as input, the model is trained on a large amount of labeled and unlabeled elderly behavioral data (both normal and those with fall risk). This allows the model to accurately identify movement patterns (e.g., leaning forward, climbing, and other combinations of features) that indicate a fall risk when the elderly approach dangerous areas like windows or balcony edges. The model then outputs the probability and confidence level of a fall risk.

[0011] Early Warning Module: When the AI ​​analysis module determines that the probability of a fall has reached a preset warning threshold and the confidence level meets the preset requirements, the early warning module quickly activates. Warning methods include local audio and visual alerts to alert staff within the nursing home and those in the same area. Simultaneously, an early warning message is sent online to a designated guardian, family member, or emergency rescue agency. This information may include the elderly person's identity, the time and location of the fall, and relevant on-site images or video clips and wearable device data screenshots, allowing rescuers to quickly obtain accurate information and take appropriate measures.

[0012] Backend Management Module: This module is used to configure, manage, monitor, and maintain the entire system. It enables functions such as setting parameters for data acquisition equipment (such as camera shooting parameters and wearable device data collection frequency), adjusting multimodal fusion strategies, optimizing and updating AI models, and setting warning thresholds. Furthermore, it can query, compile, and analyze historical fall risk data and warning records, providing data support for the safety management of nursing homes and the care arrangements for the elderly.

[0013] The beneficial effects of the present invention are:

[0014] It can integrate multiple modal data on the elderly and environmental data, overcome the shortcomings of single modality detection, improve the accuracy and reliability of fall risk detection, and effectively reduce the occurrence of false alarms and missed alarms.

[0015] With the help of advanced artificial intelligence models, early warning of the risk of elderly people falling from buildings can be achieved, which helps to detect and take measures in time before the danger occurs, reduce the probability of falling accidents, and protect the lives of the elderly.

[0016] A timely and comprehensive early warning mechanism can ensure that relevant personnel can obtain accurate information in the shortest time possible, quickly carry out rescue and response work, and minimize the possible harm caused by falling from a building.

[0017] The system has good scalability and maintainability, and can be flexibly configured and optimized and upgraded according to different scenarios and needs, adapting to the safety protection requirements of various nursing homes and living environments. DETAILED DESCRIPTION

[0018] The following is a detailed description of the elderly falling warning system of the present invention.

[0019] The elderly falling warning system of the present invention is mainly composed of a data acquisition module, a multimodal fusion module, an artificial intelligence analysis module, an early warning module and a background management module.

[0020] In the data acquisition module, cameras are strategically installed in the nursing home's rooms, living rooms, hallways, and other areas, ensuring comprehensive coverage of key locations such as windows and balconies. These cameras capture real-time video data of the elderly, clearly recording their position, movements, and posture changes. Meanwhile, wearable devices worn by the elderly incorporate multiple sensors, including positioning chips, posture sensors, and heart rate sensors. These devices monitor the elderly's location coordinates (using indoor positioning technologies such as Wi-Fi, Bluetooth, or ultra-wideband), body posture (such as body tilt angle and limb position), heart rate, and other physiological and motion data in real time. These data are then transmitted to the system via wireless networks (such as Bluetooth or Wi-Fi). Furthermore, window and balcony door sensors are installed on the corresponding windows and doors to monitor their open and closed status. Weather sensors are installed outdoors to collect weather data (such as wind speed and rainfall). This environmental data is collected via wired or wireless communication to the data acquisition module's central processing unit.

[0021] After receiving various data types, the multimodal fusion module first performs preprocessing operations on the data. For example, video image data is subjected to noise removal, resizing, and frame rate unification; wearable device data is subjected to data format conversion, unit unification, and missing value filling; and environmental data is subjected to data smoothing. Then, using feature extraction techniques from deep learning algorithms, the module extracts the elderly person's positional features (distance from windows and balconies), motion features (such as movement trajectory and whether climbing is occurring), and posture features (such as whether the body is leaning forward and balance status) from the video images. This data is combined with the position, posture, and physiological data collected by the wearable device, as well as the window and balcony status and weather data monitored by environmental sensors. These data are then fused into a multimodal feature vector using a specific fusion strategy (such as feature fusion based on deep neural networks). This feature vector comprehensively reflects the elderly person's current fall risk and is then fed into the artificial intelligence analysis module.

[0022] The AI ​​analysis module pre-builds and trains a fall risk identification model based on a multimodal AI model. During the model training phase, a large amount of multimodal data, including scenarios of normal elderly behavior and scenarios with fall risk, is collected and annotated. The annotated dataset is divided into a training set, a validation set, and a test set. The model is trained using the training set, and the model parameters are adjusted through an optimization algorithm. This allows the model to learn the characteristic patterns of various modal data when fall risk occurs and the correlations between them, thereby accurately distinguishing normal behavior from fall risk behavior. The model is verified and tuned on the validation set to determine the optimal model structure and hyperparameter settings. During actual operation, the fused multimodal feature vector is input into the trained model. After calculation and analysis, the model outputs the probability of the elderly person's fall risk occurring at the current moment and the corresponding confidence level.

[0023] When the probability of a fall risk output by the AI ​​analysis module reaches a preset warning threshold (e.g., 80%) and the confidence level meets a preset requirement (e.g., 70%), the warning module immediately activates. A local audio and visual warning device emits a strong audio and visual signal in the vicinity of the elderly person and at the nursing home's monitoring center, alerting on-site personnel and monitoring staff to the risk of a fall. Simultaneously, the warning module transmits detailed warning information via network communications (e.g., 4G, 5G, or wired networks) to designated management personnel (e.g., nursing staff, security officers), family members, and emergency response agencies. This information includes the elderly person's identity (e.g., name, room number), the specific time and location of the fall risk (accurate down to the window or balcony within the room), on-site video footage (which can be transmitted in real time for personnel to review the individual's specific situation), screenshots of the wearable device's position and posture data, and environmental data (e.g., whether windows are open, weather conditions, etc.). This allows personnel to quickly identify the elderly person's dangerous situation and respond promptly, such as caregivers immediately responding to the scene to stop the elderly person's dangerous behavior or rescue personnel rushing to the scene to provide protection.

[0024] The backend management module provides system administrators with a comprehensive interface for managing data acquisition devices, such as cameras, wearable devices, and environmental sensors. This allows administrators to configure parameters for camera capture parameters (resolution, frame rate, etc.), set the data collection frequency and upload cycle for wearable devices, and configure the sensitivity of window and balcony sensors. Furthermore, the algorithms and strategies used in multimodal fusion can be adjusted and optimized to meet the needs of fall risk monitoring in different nursing home environments and the behavioral characteristics of the elderly. The system also supports further training and optimization of AI models, such as incrementally training the models with new behavioral data, to continuously improve their recognition accuracy and generalization capabilities. Different warning thresholds and confidence levels can be set in the backend management module, allowing flexible adjustment of warning sensitivity based on actual needs and experience. The system also automatically records all warning events and related data. Administrators can easily query historical warning records through the backend management module and conduct statistical analysis on the frequency, time distribution, and location distribution of fall risk. This provides strong data support for the safety deployment of nursing homes and the care and monitoring of elderly residents, helping to formulate more scientific and reasonable safety protection strategies and measures.

[0025] To sum up, the elderly fall warning system based on the multimodal artificial intelligence model of the present invention can achieve early warning and timely warning of the risk of elderly people falling from buildings by integrating multiple modal elderly data and environmental data, and using advanced artificial intelligence technology for analysis and identification. It is of great significance to improve the safety level of the elderly in nursing homes and prevent falling accidents. It has broad application prospects and can be widely used in various types of nursing homes, senior apartments, and homes with elderly people.

Claims

1. An elderly falling warning system based on a multimodal artificial intelligence model, characterized by: include: Data acquisition module, used to collect video image data, wearable device data and environmental data of the elderly; The multimodal fusion module is used to fuse the collected data of different modalities and generate a multimodal feature vector; The artificial intelligence analysis module builds a falling risk identification algorithm based on a multimodal artificial intelligence model. It uses the fused multimodal feature vector as input and outputs the probability and confidence level of the falling risk. The early warning module activates the early warning mechanism and sends early warning information to relevant personnel when the probability of falling risk reaches the preset early warning threshold and the confidence level reaches the preset requirements; The backend management module is used to configure, manage and monitor the system.

2. The elderly falling warning system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The data acquisition module includes: a camera installed in the elderly activity area of ​​the nursing home, used to capture video image data of the elderly; a wearable device worn by the elderly, used to monitor the elderly's position data, posture data, heart rate data and other physiological and motion data in real time; environmental sensors, including window switch sensors, balcony door sensors and weather sensors, which are used to monitor the opening and closing status of windows and balconies and weather conditions respectively, and transmit the monitoring data to the central processing unit of the data acquisition module.

3. The elderly falling warning system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The method for the multimodal fusion module to perform data fusion processing includes: preprocessing various types of data, such as data cleaning, normalization and time synchronization; extracting the elderly's position features, movement features and posture features from video image data; extracting position features, posture features and physiological features from wearable device data; extracting window and balcony status features and weather features from environmental data; and using a deep learning algorithm to fuse the features of the above different modalities to generate a multimodal feature vector.

4. The elderly falling warning system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The multimodal artificial intelligence model of the artificial intelligence analysis module adopts a deep neural network architecture and is trained with a large amount of labeled and unlabeled elderly behavior data. It can accurately identify the movement patterns of the elderly approaching dangerous areas such as windows or balcony edges and where there is a risk of falling.

5. The elderly falling warning system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The warning modes of the warning module include local sound and light warning and network warning. The local sound and light warning is used to remind the staff and on-site personnel in the nursing home. The network warning sends warning information containing the identity information of the elderly, the time and location of the risk of falling from a building, as well as on-site images or video clips and wearable device data screenshots to the preset management personnel, family members or emergency rescue agencies through the communication network.

6. The elderly falling warning system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The background management module has the following functions: parameter setting of data acquisition equipment; adjustment of multimodal fusion algorithms and strategies; optimization and updating of artificial intelligence models; setting of warning thresholds and confidence requirements; query, statistics and analysis of historical falling risk data and warning records.