Old people dysphagia alarm system based on multi-mode artificial intelligence model

Abstract

The invention discloses an elder dysphagia alarm system based on a multi-modal artificial intelligence model. The elder dysphagia alarm system comprises a data acquisition module, a multi-modal fusion module, an artificial intelligence analysis module, an alarm module and a background management system. The data acquisition module acquires video image data, wearable equipment data and environment data of the elderly; the multi-modal fusion module carries out fusion processing on the collected data of different modals to generate a multi-modal feature vector; the artificial intelligence analysis module analyzes the feature vectors based on a multi-modal artificial intelligence model, and outputs the probability and confidence of dysphagia of the old; when the elder is dysphagia and the confidence reaches a preset threshold, the alarm module starts to give an alarm; and the background management system configures, manages and monitors the system. The dysphagia condition of the old people is efficiently and accurately detected and timely alarmed by fusing various modal data and utilizing a deep learning technology, a powerful guarantee is provided for the safety and health of the old people, and the dysphagia condition detection method is suitable for the scenes of old people institutions, community old people service centers, families of the old people living alone and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of elderly health monitoring, and in particular to an elderly choking alarm system based on a multimodal artificial intelligence model. Background Art

[0002] Choking is a common accident among the elderly, especially those with swallowing disorders, cognitive impairments, or physical decline. The risk of choking is higher, and once choking occurs, if timely treatment is not provided within a short period of time, it will seriously endanger the life of the elderly. Traditional choking monitoring methods mainly rely on manual care or simple physiological monitoring equipment, which have shortcomings such as high labor costs, discontinuous monitoring, and inability to provide real-time warnings. In recent years, with the development of artificial intelligence technology, research on choking monitoring using single-modality data (such as video images or wearable device data) has emerged. However, single-modality data is easily affected by environmental factors, equipment performance, etc., resulting in low monitoring accuracy and prone to false alarms and missed alarms. Therefore, there is an urgent need for a more intelligent and efficient elderly choking alarm system that can integrate multiple modal data to improve the accuracy and timeliness of choking detection and provide more reliable protection for the life safety of the elderly. Summary of the Invention

[0003] The purpose of the present invention is to provide an elderly choking alarm system based on a multimodal artificial intelligence model, which integrates data from multiple modalities and uses artificial intelligence technology to achieve early warning and accurate alarm of elderly choking situations.

[0004] The elderly choking alarm system of the present invention includes the following modules:

[0005] Data acquisition module: used to collect multiple modal data of the elderly, including video image data, wearable device data (such as neck acceleration data, swallowing movement data and other physiological and motion data) and environmental data (such as indoor noise data, etc.). Video image data can be collected through cameras installed in nursing homes, elderly activity areas, restaurants and other places, mainly used to observe the elderly's eating movements, expressions and facial features; wearable device data is monitored and transmitted in real time by the device worn by the elderly, and is used to capture subtle movement changes and physiological signals of the elderly's neck during the swallowing process; environmental data is obtained through corresponding environmental sensors, which helps to assist in determining whether the eating environment may affect the elderly's swallowing safety.

[0006] Multimodal Fusion Module: This module fuses the collected data from different modalities. First, preprocessing the data, such as cleaning, normalization, and time synchronization, is performed to eliminate discrepancies and noise. Then, using deep learning and other techniques, it organically integrates information such as eating movements and facial expressions from video images, neck movements and physiological characteristics from wearable devices, and noise from environmental data. This generates a multimodal feature vector that comprehensively reflects the elderly person's eating habits and choking risk.

[0007] Artificial Intelligence Analysis Module: Builds a choking recognition algorithm based on a multimodal artificial intelligence model. Using the fused multimodal feature vector as input, the model is trained and learned from a large amount of labeled and unlabeled eating data (including normal eating data and choking data). This allows the model to accurately distinguish between normal eating and choking, and outputs the probability and confidence level of choking. Alarm Module: When the artificial intelligence analysis module determines that an elderly person is choking and the confidence level reaches a preset threshold, the alarm module promptly activates the alarm mechanism. Alarm methods include local sound and light alarms to alert staff within the nursing home or people in the same area. At the same time, an alarm message is sent via the network to a preset guardian, family member, or emergency rescue agency. The alarm message may include the elderly person's identity information, the time and location of the choking, and related on-site images or video clips and wearable device data screenshots, so that rescue personnel can quickly obtain accurate information and take appropriate measures.

[0008] Backend Management System: This system is used to configure, manage, and monitor the system. It enables parameter settings for data acquisition devices, adjustment of multimodal fusion strategies, training and optimization of AI models, setting alarm thresholds, and querying and compiling alarm records. The backend management system also provides a user interface, allowing managers and family members to easily monitor the patient's eating status and system operation at any time.

[0009] The beneficial effects of the present invention are:

[0010] By integrating multiple modal information such as video images, wearable devices and environmental data, the limitations of single modal data such as susceptibility to interference and incomplete information are overcome, and the accuracy and reliability of elderly choking detection are improved.

[0011] The multimodal artificial intelligence model based on deep learning can automatically learn the normal eating patterns and choking characteristics of the elderly. It has strong adaptability and generalization capabilities, and can effectively meet the choking detection needs of the elderly in different scenarios and individual differences.

[0012] The timely alarm mechanism can ensure that when an elderly person chokes, relevant personnel can be quickly notified and take rescue actions, providing strong protection for the elderly person's life safety.

[0013] The modular design of the system facilitates expansion and optimization according to actual needs, such as adding new data acquisition modalities, improving fusion algorithms, or upgrading artificial intelligence models, and has good scalability and maintainability. DETAILED DESCRIPTION

[0014] The elderly choking alarm system of the present invention is described in detail below.

[0015] The elderly choking alarm system of the present invention mainly includes a data acquisition module, a multimodal fusion module, an artificial intelligence analysis module, an alarm module and a background management system.

[0016] In the data acquisition module, cameras are installed in the dining room, rooms and other areas where the elderly eat in the nursing home to capture all-round video image data of the elderly while eating, including the elderly's eating movements, changes in facial expressions, etc.; wearable devices are worn by the elderly, such as accelerometers and swallowing monitors worn on the neck, which monitor in real time the acceleration changes of the elderly's neck during swallowing, the frequency and amplitude of swallowing movements and other physiological and motion data, and transmit the data to the system via Bluetooth or wireless network; environmental sensors are distributed in different locations in the room to collect environmental noise data, such as the sound of tableware colliding and the sound of the elderly talking. These data are collected to the central processing unit of the data acquisition module through wired or wireless communication.

[0017] After receiving the various data transmitted by the data acquisition module, the multimodal fusion module first preprocesses the data, including data cleaning, normalization, and time synchronization, to eliminate differences and noise interference between the data. Then, a multimodal feature extraction method based on deep learning is used to extract the characteristics of the elderly's eating movements and facial expression changes from the video images, neck movement characteristics and physiological characteristics from the wearable device data, and noise characteristics from the environmental data. Next, the features of these different modalities are fused through fusion strategies (such as attention mechanisms or feature splicing) to generate a multimodal feature vector that can comprehensively reflect the elderly's eating status and choking risk, and then input it into the artificial intelligence analysis module.

[0018] The AI ​​analysis module builds a multimodal AI model based on a deep neural network. During the model training phase, a large amount of multimodal data, including both normal eating and choking scenarios of the elderly, is collected and annotated. The annotated data is then divided into training, validation, and test sets. The model is trained using the training set, and its parameters are adjusted using an optimization algorithm to enable it to learn the associations between the different modal data and the representation of choking characteristics. The model is evaluated and tuned on the validation set to determine the optimal model structure and hyperparameters. Finally, the trained model is tested on the test set to verify its performance metrics, such as accuracy and recall, for detecting choking in the elderly. When the system is running, the fused multimodal feature vector is input into the trained model, which then calculates and outputs the probability and confidence level of choking. If the confidence level reaches a preset alarm threshold, for example, 85%, the patient is deemed to have choked, triggering the alarm module.

[0019] Upon receiving the alarm command, the alarm module immediately activates the local audio and visual alarm system, emitting dazzling lights and loud sounds in the corresponding area of ​​the nursing home to alert on-site staff and other personnel. Simultaneously, the alarm information is sent via the network to a pre-determined guardian, family member, or emergency rescue agency. This alarm information includes the individual's name, room number, the specific time and location of the choking incident, as well as on-site images or video clips captured by the camera and screenshots of neck movement data recorded by the wearable device, allowing relevant personnel to quickly understand the situation and respond. For example, family members can receive an alarm notification via a mobile app, view the individual's real-time status and on-site video, and promptly communicate with nursing home staff and arrange for rescue. Rescue agencies can then use the alarm information to quickly locate the individual and dispatch an ambulance.

[0020] The backend management system provides comprehensive management capabilities for system administrators. Administrators can use the backend management system to configure parameters for data acquisition devices such as cameras, wearable devices, and environmental sensors. These parameters include adjusting the camera's shooting angle, resolution, and frame rate, and setting the wearable device's data collection frequency and transmission interval. Furthermore, the multimodal fusion module's fusion algorithms and strategies can be adjusted to suit the data characteristics and detection requirements of different scenarios. Furthermore, the backend management system supports the training and optimization of artificial intelligence models. Administrators can import new training data, update model parameters, and improve detection performance. Alarm thresholds can also be set within the backend management system, allowing for flexible adjustment of alarm sensitivity based on actual needs and false alarm rates. The system also records all alarm events and system operation logs. Administrators can query and compile alarm records through the backend management system, analyzing the frequency, time distribution, and location of choking incidents, providing data support for the nursing home's food management and care arrangements. Family members and authorized personnel can also use the backend management system's user interface to view the patient's eating status, including video images, physiological data, and eating patterns, to understand their diet and health status in the nursing home.

[0021] To sum up, the elderly choking alarm system based on the multimodal artificial intelligence model of the present invention integrates multiple modal data and uses advanced deep learning technology for intelligent analysis and judgment, thereby achieving efficient and accurate detection and timely alarm of elderly choking situations, providing strong protection for the safety and health of the elderly, and has broad application prospects in scenarios such as nursing homes, community elderly care service centers, and families of elderly people living alone.

Claims

1. A choking alarm system for the elderly based on a multimodal artificial intelligence model, characterized in that: include: Data acquisition module, used to collect video image data, wearable device data and environmental data of the elderly; The multimodal fusion module is used to fuse the collected data of different modalities and generate a multimodal feature vector; The artificial intelligence analysis module constructs a choking recognition algorithm based on a multimodal artificial intelligence model, uses the fused multimodal feature vector as input, and outputs the probability and confidence level of the elderly choking; the alarm module activates the alarm mechanism and sends an alarm message to relevant personnel when the artificial intelligence analysis module determines that the elderly chokes and the confidence level reaches a preset threshold; the backend management system is used to configure, manage, and monitor the system.

2. The elderly choking alarm system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The data acquisition module includes: a camera installed in the elderly's eating area, used to capture video image data of the elderly while eating; a wearable device worn by the elderly, used to monitor in real time the elderly's neck acceleration data, swallowing movement data and other physiological and motion data during swallowing; and environmental sensors distributed in different locations indoors, used to collect environmental noise data.

3. The elderly choking alarm system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The multimodal fusion module adopts multimodal feature extraction and fusion technology based on deep learning, including the following steps: preprocessing data of different modalities, including data cleaning, normalization and time synchronization; extracting eating action features and expression change features from video image data, extracting neck movement features and physiological features from wearable device data, and extracting noise features from environmental data; and using fusion strategies such as attention mechanism or feature splicing to fuse features of different modalities to generate a multimodal feature vector.

4. The elderly choking alarm system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The multimodal artificial intelligence model of the artificial intelligence analysis module adopts a deep neural network architecture to process the spatiotemporal information of video images and the temporal characteristics of wearable device and environmental data.

5. The elderly choking alarm system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The alarm module's alarm modes include local sound and light alarms and network alarms. The local sound and light alarms are used to alert on-site personnel, and the network alarms send alarm information to preset guardians, family members or emergency rescue agencies through the network. The alarm information includes the elderly person's identity information, the time and location of choking, as well as on-site images or video clips and wearable device data screenshots.

6. The elderly choking alarm system based on a multimodal artificial intelligence model according to claim 1 is characterized in that: The background management system has the following functions: parameter setting of data acquisition equipment; adjustment of multimodal fusion strategy; training and optimization of artificial intelligence models; setting of alarm thresholds; query and statistics of alarm records.