A multi-modal perception-based cruise ship intelligent medical care and aging monitoring system

Abstract

The application discloses a kind of based on multimodal perception's cruise ship wisdom medical care old-age monitoring system, including multimodal perception module, for through sensor collection multimodal perception information, multimodal perception information includes the environment in cruise ship, user's physiological state information;Processing module is used to receive the information transmitted by multimodal perception module in real time, process information, and analyze the state of user according to the information after processing;If abnormal state appears, trigger alarm, and send early warning signal to shore base cloud.The beneficial effects of the present application: the present application is provided with multimodal perception module, processing module, multi-terminal interaction module, realizes the all-round health monitoring and medical care service support for cruise ship elderly passengers, solves the problem that elderly passengers are difficult to operate and inconvenient to interact, realizes convenient help, and improves the comfort and safety of elderly passengers on sea travel.

Description

Technical Field

[0001] This invention relates to the technical field of smart healthcare and elderly care, specifically to a cruise ship smart healthcare and elderly care monitoring system based on multimodal perception. Background Technology

[0002] With the aging population and the rapid expansion of the senior tourism market, cruise ships, with their comfort, convenience, and one-stop service, have become an important choice for leisure travel for the elderly. However, as a mobile vessel at sea, cruise ships have unique characteristics such as enclosed spaces, limited medical resources, satellite communication delays, susceptibility to turbulence, and complex cabin layouts, making it difficult to guarantee the health monitoring and medical care services for elderly passengers.

[0003] Therefore, we designed a smart healthcare and aging-friendly monitoring system for cruise ships based on multimodal perception to solve the above problems. Summary of the Invention

[0004] Purpose of the invention: To provide a smart healthcare and aging-friendly monitoring system for cruise ships based on multimodal perception, so as to solve the above-mentioned problems existing in the prior art.

[0005] Technical Solution: A smart elderly care and monitoring system for cruise ships based on multimodal perception includes a multimodal perception module for collecting multimodal perception information through sensors, including environmental and physiological status information of users within the cruise ship; a processing module for receiving information transmitted from the multimodal perception module in real time, processing the information, and analyzing the user's status based on the processed information; triggering an alarm and sending a warning signal to the shore-based cloud if an abnormal state occurs; and a multi-terminal interaction module, connecting the cruise ship to the shore-based cloud via a 5G private network and low-orbit satellite, with a medical and elderly care monitoring platform set up on the shore-based cloud; upon receiving the warning signal from the processing module, the medical and elderly care monitoring platform on the shore-based cloud activates an emergency response mechanism, classifying and prioritizing the warning signal through the medical and elderly care monitoring platform; the system realizes the collection, fusion analysis, abnormal warning, age-friendly interaction, and full-process medical and elderly care linkage of multimodal data of elderly cruise ship passengers' physiology, behavior, and environment.

[0006] Preferably, the multimodal sensing module specifically includes: A wearable wristband worn by the user collects the user's physiological status information in real time, including sleep status, heart rate, blood pressure, blood oxygen saturation, body temperature, and activity level. Environmental monitoring devices installed inside cruise ship cabins are used to monitor air quality, temperature and humidity, light intensity, noise levels, and the degree of hull sway.

[0007] Preferably, the processing module specifically includes a data preprocessing unit, a feature extraction unit, a multimodal fusion unit, an anomaly detection unit, and a local decision-making unit for information processing; The data preprocessing unit includes time synchronization, noise filtering, normalization, and missing value completion for the information collected by the multimodal perception module. The feature extraction unit is used to extract standardized feature vectors of physiology, behavior, and environment; The multimodal fusion unit automatically adjusts the weights of each modality feature according to the scene and outputs a comprehensive feature vector; The anomaly identification unit includes comparing the input comprehensive feature vector with a pre-trained health and behavior dataset of elderly cruise passengers, constructing an anomaly state determination model, and outputting anomaly confidence. The local decision-making unit is used to perform real-time status assessment based on the anomaly confidence level to determine whether there is an abnormal situation for the user; if an anomaly is detected, the processing module will immediately generate an early warning signal and transmit the relevant information to the shore-based cloud.

[0008] Preferably, the medical and elderly care monitoring platform includes a health data center, a telemedicine unit, an emergency command system, and a family and server terminal; The health data center is used to store users’ health records, physiological status information, behavior records and environmental monitoring data to form a complete user health profile. The telemedicine unit is used to connect to a shore-based tertiary hospital to support video consultations, image transmission, and medication guidance. The emergency command system enables early warning classification, location positioning, medical staff dispatch, and emergency plan recommendation. The system allows family members to view the user's health status and early warning information through a mobile application, and also enables them to communicate in real time with the onshore cloud-based medical and elderly care monitoring platform.

[0009] Preferably, the expression for the comprehensive feature vector is: , Among them, F f The comprehensive feature vector is defined as follows: F1 is the physiological feature vector, F2 is the behavioral feature vector, F3 is the environmental feature vector, and ω1, ω2, and ω3 are adaptive weights; and ω1+ω2+ω3=1.

[0010] Preferably, the early warning classification is set with three levels, including: Level 1 Response: 0.90 ≤ Anomaly Confidence ≤ 1.00, triggering an emergency warning. Response measures include automatic unlocking of the cabin door, emergency dispatch of the nearest crew member, preparation of the ICU at the medical center, sending of the pre-screening and triage report from the shore-based hospital, and helicopter rescue assessment. Secondary response: 0.70 ≤ Abnormal confidence level < 0.90, triggering a high-risk warning. Response measures include voice confirmation of passenger status, crew directional inspections, automatic environmental adjustment, and real-time push to the family member APP; Tertiary response: 0.50 ≤ Abnormal confidence level < 0.70, triggering a concern warning. Response measures include pushing health reminders via the APP, voice exercise suggestions, and booking a health assessment for the next day.

[0011] Advantages of the present invention: By providing a multi-modal perception module, a processing module, and a multi-terminal interaction module, the present invention realizes all-round health monitoring and medical care service support for elderly passengers on the cruise ship, solves the problems of difficult operation and inconvenient interaction for elderly passengers, enables convenient assistance, and improves the comfort and sense of security of elderly passengers during sea travel. Brief Description of the Drawings

[0012] Figure 1 is a schematic structural diagram of the present invention; Figure 2 is a schematic flowchart of the processing module in the present invention. Detailed Embodiments

[0013] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention. Specific Embodiment

[0014] As shown in the Figure 1-2 accompanying drawings, in this embodiment, a cruise ship intelligent medical care and aging-friendly monitoring system based on multi-modal perception includes a multi-modal perception module for collecting multi-modal perception information through sensors. The multi-modal perception information includes the environment inside the cruise ship and the physiological state information of the user; a processing module for receiving the information transmitted by the multi-modal perception module in real time, processing the information, analyzing the status of the user based on the processed information; triggering an alarm if an abnormal state occurs and sending a warning signal to the shore-based cloud; a multi-terminal interaction module. The cruise ship is connected to the shore-based cloud through a 5G private network and a low-earth orbit satellite. The shore-based cloud is provided with a medical care monitoring platform; upon receiving the warning signal from the processing module, the medical care monitoring platform of the shore-based cloud starts an emergency response mechanism, classifies and evaluates the priority of the warning signal through the medical care monitoring platform; the system realizes the collection, fusion analysis, abnormal warning, aging-friendly interaction, and full-process medical care linkage of multi-modal data of the physiological, behavioral, and environmental data of elderly passengers on the cruise ship.

[0015] During operation, the multimodal sensing module collects real-time physiological status information of users and the environment within the cruise ship, transmitting the collected data to the processing module in real time. The processing module analyzes the information to determine if any abnormalities exist. If an abnormality is detected, the processing module generates an early warning signal and transmits the relevant information to the onshore cloud-based medical and elderly care monitoring platform. Upon receiving the early warning signal, the platform activates the emergency response mechanism, classifying and assessing the signal according to its type and priority. Subsequently, the system connects to a top-tier onshore hospital via a telemedicine unit, providing video consultations, image transmission, and medication guidance services. Simultaneously, the emergency command system formulates corresponding response measures based on the warning level, including dispatching medical personnel, location tracking, and recommending emergency plans. Family members and service providers can view the user's health status and early warning information in real time through a mobile application and maintain communication with the onshore cloud to ensure the comprehensive protection of the health and safety of elderly passengers on the cruise ship.

[0016] The multimodal sensing module specifically includes: A wearable wristband worn by the user collects the user's physiological status information in real time, including sleep status, heart rate, blood pressure, blood oxygen saturation, body temperature, and activity level. Environmental monitoring devices installed inside cruise ship cabins are used to monitor air quality, temperature and humidity, light intensity, noise levels, and the degree of hull sway.

[0017] In practical applications, the system of the present invention continuously collects users' physiological status information and environmental data inside the cruise ship cabin through the wearable wristband and environmental monitoring device in the multimodal sensing module.

[0018] The wearable wristband uses lightweight materials and a skin-friendly strap to improve user comfort during extended wear. It is also waterproof and dustproof, suitable for various activities on a cruise ship. The wristband integrates a heart rate sensor, blood oxygen sensor, blood pressure sensor, body temperature sensor, body motion sensor, and ECG acquisition module, and features a large SOS emergency call button.

[0019] The environmental monitoring device is deployed in a distributed manner to cover key areas inside the cabin, such as the passenger bed and toilet. It can capture data such as air quality and temperature and humidity changes in real time and transmit this information to the processing module for comprehensive analysis.

[0020] The processing module specifically includes a data preprocessing unit, a feature extraction unit, a multimodal fusion unit, an anomaly detection unit, and a local decision-making unit for information processing; The data preprocessing unit includes time synchronization, noise filtering, normalization, and missing value completion for the information collected by the multimodal perception module. The feature extraction unit is used to extract standardized feature vectors of physiology, behavior, and environment; The multimodal fusion unit automatically adjusts the weights of each modality feature according to the scene and outputs a comprehensive feature vector; The anomaly identification unit includes comparing the input comprehensive feature vector with a pre-trained health and behavior dataset of elderly cruise passengers, constructing an anomaly state determination model, and outputting anomaly confidence. The local decision-making unit is used to perform real-time status assessment based on the anomaly confidence level to determine whether there is an abnormal situation for the user; if an anomaly is detected, the processing module will immediately generate an early warning signal and transmit the relevant information to the shore-based cloud.

[0021] After receiving data from the multimodal perception module, the processing module performs time synchronization, noise filtering, normalization, and feature extraction to obtain a comprehensive feature vector. The anomaly detection unit analyzes the comprehensive feature vector using a pre-trained data model to determine whether there are potential health risks or environmental anomalies. If an anomaly is detected, the local decision-making unit generates a corresponding early warning signal based on the anomaly confidence level and quickly transmits the relevant information to the onshore cloud-based medical and elderly care monitoring platform.

[0022] This invention is based on an LSTM+Transformer hybrid network architecture, which trains a dataset of health and behavior data of elderly cruise ship passengers, and can identify anomalies in multiple scenarios and output anomaly confidence scores.

[0023] The medical and elderly care monitoring platform includes a health data center, a telemedicine unit, an emergency command system, and a family and service terminal. The health data center is used to store users’ health records, physiological status information, behavior records and environmental monitoring data to form a complete user health profile. The telemedicine unit is used to connect to a shore-based tertiary hospital to support video consultations, image transmission, and medication guidance. The emergency command system enables early warning classification, location positioning, medical staff dispatch, and emergency plan recommendation. The system allows family members to view the user's health status and early warning information through a mobile application, and also enables them to communicate in real time with the onshore cloud-based medical and elderly care monitoring platform.

[0024] Preferably, the expression for the comprehensive feature vector is: , Among them, F fThe comprehensive feature vector consists of F1 (physiological feature vector), F2 (behavioral feature vector), and F3 (environmental feature vector), with ω1, ω2, and ω3 as adaptive weights, satisfying ω1 + ω2 + ω3 = 1. In scenarios involving falls and fainting, ω2 = 0.5-0.6; in scenarios involving chronic disease monitoring, ω1 = 0.5-0.6; and in scenarios involving abnormal environments, ω3 = 0.4-0.5.

[0025] Preferably, the early warning classification is set with three levels, including: Level 1 Response: 0.90 ≤ Anomaly Confidence ≤ 1.00, triggering an emergency warning. Response measures include automatic unlocking of the cabin door, emergency dispatch of the nearest crew member, preparation of the ICU at the medical center, sending of the pre-screening and triage report from the shore-based hospital, and helicopter rescue assessment. Level 2 Response: 0.70 ≤ Anomaly Confidence < 0.90, triggering a high-risk warning. Response measures include voice confirmation of passenger status, crew-oriented patrols, automatic environmental adjustment, and real-time push notifications to family members via the app. Level 3 response: 0.50 ≤ anomaly confidence level < 0.70, triggering a warning. Response measures include APP health reminder push, voice exercise suggestions, and appointment for a health assessment the next day.

[0026] Upon receiving an early warning signal, the medical and elderly care monitoring platform activates its emergency response mechanism. It combines user health records and real-time data stored in the health data center to conduct a graded assessment of the warning. Based on the different levels of warnings, the system will trigger corresponding response measures.

[0027] In a Level 1 response, the system will automatically unlock the cabin door for rapid entry by medical personnel, dispatch the nearest crew member to assess the situation, and prepare ICU resources at the medical center. The shore-based hospital will receive a triage report, providing an assessment basis for potential helicopter rescue. In a Level 2 response, the system will confirm the passenger's status via voice, arrange for crew members to conduct directional checks, and automatically adjust the cabin's temperature, humidity, and light intensity based on environmental data, while also pushing real-time information to family members' apps. In a Level 3 response, the system primarily uses a mobile application to send health reminders and exercise suggestions to users, schedule health assessments for the following day, and continuously monitor the user's potential risks.

[0028] Meanwhile, through family members and the service terminal, the family members of elderly passengers can use the mobile application to keep track of the user's health status and early warning information at any time, and maintain real-time communication with the shore-based cloud.

[0029] Working principle explanation: During operation, the multimodal sensing module on the cruise ship runs continuously, with wearable wristbands and environmental monitoring devices constantly collecting data and transmitting this information to the processing module in real time. The processing module, through the collaborative work of its internal multiple units, performs data preprocessing, feature extraction, fusion analysis, and anomaly identification. When a potential risk is detected, the system sends an early warning signal to the shore-based cloud platform and transmits it to the shore-based medical and elderly care monitoring platform.

[0030] Upon receiving an early warning signal, the medical and elderly care monitoring platform immediately activates its emergency response mechanism. The health data center provides the platform with a comprehensive health profile of each user, including historical health records, real-time physiological status, and environmental monitoring data. The telemedicine unit then develops corresponding response plans based on the warning level, including the allocation of medical resources, location tracking, and emergency plan recommendations.

[0031] The preferred embodiments have been shown and described, but should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A smart healthcare and aging-friendly monitoring system for cruise ships based on multimodal perception, characterized in that: include The multimodal sensing module is used to collect multimodal sensing information through sensors. The multimodal sensing information includes the environment inside the cruise ship and the physiological state information of the users. The processing module is used to receive information transmitted by the multimodal sensing module in real time, process the information, and perform status analysis on the user based on the processed information; if an abnormal state occurs, an alarm is triggered and a warning signal is sent to the shore-based cloud. The multi-terminal interaction module connects the cruise ship to the shore-based cloud via a 5G private network and low-orbit satellite. The shore-based cloud is equipped with a medical and elderly care monitoring platform. Upon receiving an early warning signal from the processing module, the shore-based cloud medical and elderly care monitoring platform activates an emergency response mechanism, classifying and prioritizing the early warning signal. The system enables the collection, fusion, and analysis of multimodal data on the physiological, behavioral, and environmental aspects of elderly cruise ship passengers, as well as abnormality warnings, age-friendly interactions, and integrated medical and elderly care throughout the entire process.

2. The cruise ship smart healthcare and aging-friendly monitoring system based on multimodal perception according to claim 1, characterized in that: The multimodal sensing module specifically includes: A wearable wristband worn by the user collects the user's physiological status information in real time, including sleep status, heart rate, blood pressure, blood oxygen saturation, body temperature, and activity level. Environmental monitoring devices installed inside cruise ship cabins are used to monitor air quality, temperature and humidity, light intensity, noise levels, and the degree of hull sway.

3. The cruise ship smart healthcare and aging-friendly monitoring system based on multimodal perception according to claim 2, characterized in that: The processing module specifically includes a data preprocessing unit, a feature extraction unit, a multimodal fusion unit, an anomaly detection unit, and a local decision-making unit for information processing; The data preprocessing unit includes time synchronization, noise filtering, normalization, and missing value completion for the information collected by the multimodal perception module. The feature extraction unit is used to extract standardized feature vectors of physiology, behavior, and environment; The multimodal fusion unit automatically adjusts the weights of each modality feature according to the scene and outputs a comprehensive feature vector; The anomaly identification unit includes comparing the input comprehensive feature vector with a pre-trained health and behavior dataset of elderly cruise passengers, constructing an anomaly state determination model, and outputting anomaly confidence. The local decision-making unit is used to perform real-time status assessment based on the anomaly confidence level to determine whether there is an abnormal situation for the user; if an anomaly is detected, the processing module will immediately generate an early warning signal and transmit the relevant information to the shore-based cloud.

4. The cruise ship smart healthcare and aging-friendly monitoring system based on multimodal perception according to claim 3, characterized in that: The medical and elderly care monitoring platform includes a health data center, a telemedicine unit, an emergency command system, and a family and service terminal. The health data center is used to store users’ health records, physiological status information, behavior records and environmental monitoring data to form a complete user health profile. The telemedicine unit is used to connect to a shore-based tertiary hospital to support video consultations, image transmission, and medication guidance. The emergency command system enables early warning classification, location positioning, medical staff dispatch, and emergency plan recommendation. The system allows family members to view the user's health status and early warning information through a mobile application, and also enables them to communicate in real time with the onshore cloud-based medical and elderly care monitoring platform.

5. The cruise ship smart healthcare and aging-friendly monitoring system based on multimodal perception according to claim 4, characterized in that: The expression for the comprehensive feature vector is: ， Among them, F f The comprehensive feature vector is defined as follows: F1 is the physiological feature vector, F2 is the behavioral feature vector, F3 is the environmental feature vector, and ω1, ω2, and ω3 are adaptive weights; and ω1+ω2+ω3=1.

6. The cruise ship smart healthcare and aging-friendly monitoring system based on multimodal perception according to claim 5, characterized in that: The early warning system is divided into three levels, including: Level 1 Response: 0.90 ≤ Anomaly Confidence ≤ 1.00, triggering an emergency warning. Response measures include automatic unlocking of the cabin door, emergency dispatch of the nearest crew member, preparation of the ICU at the medical center, sending of the pre-screening and triage report from the shore-based hospital, and helicopter rescue assessment. Level 2 Response: 0.70 ≤ Anomaly Confidence < 0.90, triggering a high-risk warning. Response measures include voice confirmation of passenger status, crew-oriented patrols, automatic environmental adjustment, and real-time push notifications to family members via the app. Level 3 response: 0.50 ≤ anomaly confidence level < 0.70, triggering a warning. Response measures include APP health reminder push, voice exercise suggestions, and appointment for a health assessment the next day.

Images