Multi-modal data fusion old people sleep monitoring system and control method thereof

Abstract

The invention discloses a multi-modal data fusion old people sleep monitoring system and a control method thereof. The multi-modal data fusion old people sleep monitoring system comprises a basic function module, an auxiliary function module, a control function module and a super-system function module, the basic function module comprises a physiological monitoring unit and an environment sensing unit; the auxiliary function module comprises an intervention execution unit and an information feedback unit; the control function module comprises a data processing center and a communication relay unit; the hyper-system function module comprises an energy supply unit and an external resource unit. According to the multi-modal data fusion old people sleep monitoring system and the control method thereof, the data processing center runs an AI algorithm, time alignment and abnormal value filtering are carried out on physiological and environmental data, then time domain and frequency domain features are mined, an REM period, a light sleep period and a deep sleep period are accurately identified by means of a sleep stage classification model, and the sleep monitoring accuracy is improved. And the abnormal event detection model captures apnea and body movement abnormity, and performs multi-dimensional data fusion and intelligent analysis.

Description

Technical Field

[0001] This invention relates to the field of health management and monitoring technology, and in particular to a multimodal data fusion sleep monitoring system for the elderly and its control method. Background Technology

[0002] With the aging population, health management for the elderly is becoming increasingly crucial. Sleep quality, as an important indicator of health, is of great significance for the physical recovery and disease prevention of the elderly. Traditional sleep monitoring methods, such as wearable devices (wristbands, chest straps, etc.), cause discomfort for the elderly, affecting their sleep experience. Furthermore, data collection is easily interrupted due to resistance or forgetting to wear the devices, making it impossible to continuously and comprehensively monitor sleep status.

[0003] While non-wearable monitoring devices have seen development, they still suffer from the following problems: 1. Low data fusion: They simply summarize physiological and environmental data without exploring the deeper connections between the two, making it difficult to accurately identify the causes of sleep disorders (such as the causal relationship between fluctuations in environmental temperature and humidity and sleep apnea); 2. Single and passive intervention methods: They often only respond after abnormalities occur, without dynamically adjusting to different sleep stages. Strong light and noise interventions during deep sleep can easily wake the elderly and disrupt sleep continuity; 3. Poor system coordination: Energy management, external medical systems, and smart home systems are not deeply integrated with sleep monitoring, leading to high risks of energy waste or power outages during critical monitoring stages. Sleep abnormality data cannot be promptly pushed to the medical end, and there is no way to coordinate adjustments to the home environment. Therefore, we propose a multimodal data fusion-based sleep monitoring system for the elderly and its control method. Summary of the Invention

[0004] The main objective of this invention is to provide a multimodal data fusion sleep monitoring system for the elderly and its control method, which can effectively solve the problems in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A multimodal data fusion sleep monitoring system for the elderly includes basic functional modules, auxiliary functional modules, control functional modules, and super-system functional modules;

[0007] The basic functional modules include a physiological monitoring unit and an environmental sensing unit;

[0008] The auxiliary function module includes an intervention execution unit and an information feedback unit;

[0009] The control function module includes a data processing center and a communication relay unit;

[0010] The supersystem functional modules include an energy supply unit and an external resource unit.

[0011] As a further improvement to the above solution, the physiological monitoring unit includes millimeter-wave radar and nanobiosensors for collecting heart rate, respiration, and body movement physiological data of the elderly during sleep.

[0012] The environmental sensing unit includes a temperature and humidity sensor and an air quality detection sensor, used to monitor the temperature, humidity, and air quality parameters of the sleep environment.

[0013] As a further improvement to the above solution, the intervention execution unit includes a bed angle adjustment motor, a temperature-controlled fiber pillow, and a fresh air system. The bed angle adjustment motor is used to adjust the bed angle, the temperature-controlled fiber pillow is used to adjust the temperature of the sleeping area, and the fresh air system is used to optimize the air quality of the sleeping environment.

[0014] The information feedback unit includes a TFT display screen and a mobile terminal. The TFT display screen is used to locally display monitoring data and sleep stage recognition results, and the mobile terminal is used to remotely receive sleep analysis reports.

[0015] As a further improvement to the above solution, the data processing center runs an AI algorithm to fuse and process the multimodal data collected by the physiological monitoring unit and the environmental perception unit to identify sleep stages and abnormal sleep events.

[0016] The communication relay unit adopts a 4G / WiFi dual-mode communication method to realize data transmission and command interaction between the data processing center and other functional modules and external devices.

[0017] As a further improvement to the above solution, the energy supply unit includes a high-precision sensor monitoring the mattress and a temperature control system;

[0018] The external resource unit includes an oxygen concentration monitoring module for the air component and a 4G network, which are used to supplement environmental data monitoring dimensions and ensure remote communication of the system.

[0019] As a further improvement to the above solution, the bed angle adjustment motor, temperature-controlled fiber pillow, and fresh air system of the intervention execution unit automatically adjust the bed angle, sleep area temperature, and ambient air quality based on the sleep stage identification results and abnormal event analysis results output by the data processing center. The adjustment strategy is pre-stored in the data processing center and supports personalized settings according to the user's sleep habits.

[0020] As a further improvement to the above scheme, the AI ​​algorithm running in the data processing center includes multimodal data feature extraction, sleep stage classification, and abnormal event detection model. The feature extraction model performs time-domain and frequency-domain feature mining on physiological and environmental data. The sleep stage classification model identifies REM sleep, light sleep, and deep sleep based on features. The abnormal event detection model identifies sleep apnea and abnormal body movement events.

[0021] As a further improvement to the above solution, the energy supply unit can dynamically adjust the power supply strategy according to the sleep stage identified by the data processing center. During deep sleep, the power supply to non-critical modules is reduced to ensure the stable operation of the physiological monitoring unit and the data processing center. During light sleep and wakefulness, the power supply to all modules is restored to replenish the power of the energy components.

[0022] A control method for a multimodal data fusion-based sleep monitoring system for the elderly includes the following steps:

[0023] Step 1: The physiological monitoring unit and environmental sensing unit continuously collect sleep physiological data and environmental data of the elderly, and transmit them to the data processing center through the communication relay unit;

[0024] Step 2: The data processing center runs AI algorithms to fuse and process multimodal data, identify sleep stages, detect abnormal sleep events, and generate analysis results;

[0025] Step 3: The analysis results are displayed through the information feedback unit. If abnormal sleep events or uncomfortable sleep environment are detected, the data processing center sends instructions to the intervention execution unit to adjust the bed angle, temperature control fiber temperature, and fresh air system operating parameters.

[0026] Step 4: The energy supply unit dynamically adjusts the power supply strategy based on the sleep stage identified by the data processing center to ensure stable and energy-efficient system operation. The external resource unit supplements environmental data in real time and ensures communication, working together to improve sleep monitoring and intervention.

[0027] As a further improvement to the above scheme, when the data processing center in step two integrates and processes multimodal data, it first performs time alignment and outlier filtering preprocessing on physiological data and environmental data, and then extracts multidimensional features through feature fusion network, inputs them into sleep stage classification and abnormal event detection models, and outputs recognition results.

[0028] In step three, after receiving the instruction, the intervention execution unit performs adjustment actions according to the preset priority, prioritizing respiratory-related interventions, and then adjusting the bed angle and temperature. The adjustment process is fed back to the information feedback unit in real time to record the intervention effect.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. In this invention, heart rate, respiration, and body movement data are collected by millimeter-wave radar and nano-biosensors in the physiological monitoring unit, and environmental parameters are captured by temperature and humidity sensors and air quality detection sensors in the environmental perception unit. The data is transmitted to the data processing center via the communication relay unit. The data processing center runs AI algorithms to first align the physiological and environmental data in time and filter out outliers, and then mine time-domain and frequency-domain features. With the help of the sleep stage classification model, REM sleep, light sleep, and deep sleep are accurately identified, and the abnormal event detection model captures apnea and abnormal body movement. Multi-dimensional data fusion and intelligent analysis break through the limitations of traditional isolated monitoring data, clearly present the relationship between sleep state and environment and physiology, provide an accurate basis for sleep health assessment, and make the causes of sleep disorders such as breathing difficulties caused by hot and stuffy environments nowhere to hide.

[0031] 2. In this invention, when the data processing center identifies sleep abnormalities such as sleep apnea or environmental discomfort such as excessive temperature and humidity, the bed angle adjustment motor, temperature-controlled fiber pillow, and fresh air system in the intervention execution unit automatically respond according to pre-stored and personalized settings. Priority is given to respiratory-related interventions such as increased ventilation by the fresh air system during sleep apnea, and then the bed angle and sleep area temperature are adjusted. The adjustment process is fed back to the information feedback unit in real time. This dynamic and layered intervention is in line with the sleep cycle characteristics of the elderly. It accurately maintains a comfortable environment during deep sleep and flexibly optimizes sleep conditions during light sleep, avoiding the blindness and interference of traditional interventions, effectively improving sleep quality and reducing sleep interruptions.

[0032] 3. In this invention, the energy supply unit dynamically adjusts the power supply according to the sleep stage. During deep sleep, the power consumption of non-critical modules is reduced to ensure the stability of the core monitoring and processing modules. During light sleep and wakefulness, the power supply is restored and energy is replenished. The external resource unit expands the environmental data dimension through oxygen concentration monitoring. The 4G network ensures remote communication and links with community medical care and smart homes. Intelligent energy scheduling avoids waste and power outage risks. External collaboration allows abnormal sleep data to reach the medical end in a timely manner and adjusts the home environment in sync, forming a complete closed loop of "monitoring-analysis-intervention-energy adaptation-external collaboration". This extends from sleep monitoring to health management, providing comprehensive protection for the sleep health of the elderly. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the overall structure of the multimodal data fusion sleep monitoring system for the elderly according to the present invention;

[0035] Figure 2 This is a flowchart of the control method for the multimodal data fusion sleep monitoring system for the elderly according to the present invention. Detailed Implementation

[0036] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0037] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0038] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0039] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0040] Example

[0041] Multimodal data fusion sleep monitoring system for the elderly, such as Figure 1 As shown, it includes basic function modules, auxiliary function modules, control function modules, and supersystem function modules;

[0042] The basic functional modules include a physiological monitoring unit and an environmental sensing unit;

[0043] The auxiliary function module includes an intervention execution unit and an information feedback unit;

[0044] The control function module includes a data processing center and a communication relay unit;

[0045] The supersystem functional modules include an energy supply unit and an external resource unit.

[0046] In this embodiment, the physiological monitoring unit includes millimeter-wave radar and nanobiosensors for collecting heart rate, respiration, and body movement physiological data of the elderly during sleep; the environmental sensing unit includes temperature and humidity sensors and air quality sensors for monitoring temperature, humidity, and air quality parameters of the sleep environment; the intervention execution unit includes a bed angle adjustment motor, a temperature-controlled fiber pillow, and a fresh air system, wherein the bed angle adjustment motor is used to adjust the bed angle, the temperature-controlled fiber pillow is used to adjust the temperature of the sleep area, and the fresh air system is used to optimize the air quality of the sleep environment; the information feedback unit includes a TFT display screen and a mobile terminal, wherein the TFT display screen is used to locally display monitoring data and sleep stage identification results, and the mobile terminal is used to remotely receive sleep analysis reports; the data processing center runs an AI algorithm to fuse and process the multimodal data collected by the physiological monitoring unit and the environmental sensing unit to identify sleep stages and abnormal sleep events; the communication relay unit adopts a 4G / WiFi dual-mode communication method to realize data transmission and command interaction between the data processing center and other functional modules and external devices; the energy supply unit includes a high-precision transmission... The system includes a sensor monitoring mattress and temperature control system; the external resource unit includes an oxygen concentration monitoring module for the air component and a 4G network to supplement environmental data monitoring dimensions and ensure remote communication of the system; the intervention execution unit, consisting of a bed angle adjustment motor, a temperature-controlled fiber pillow, and a fresh air system, automatically adjusts the bed angle, sleep area temperature, and ambient air quality based on the sleep stage identification results and abnormal event analysis results output by the data processing center. The adjustment strategy is pre-stored in the data processing center and supports personalized settings according to the user's sleep habits; the AI ​​algorithm running in the data processing center includes multimodal data feature extraction, sleep stage classification, and an abnormal event detection model. The feature extraction model performs time-domain and frequency-domain feature mining on physiological and environmental data, the sleep stage classification model identifies REM sleep, light sleep, and deep sleep based on features, and the abnormal event detection model identifies apnea and abnormal body movement events; the energy supply unit can dynamically adjust the power supply strategy according to the sleep stage identified by the data processing center. During deep sleep, the power supply to non-critical modules is reduced to ensure the stable operation of the physiological monitoring unit and the data processing center. During light sleep and wakefulness, the power supply to all modules is restored to replenish the energy components.

[0047] In this embodiment, during use, the physiological monitoring unit utilizes millimeter-wave radar and nano-biosensors to continuously capture the elderly person's heart rate, respiration, and body movement physiological data during sleep. The environmental perception unit monitors the temperature, humidity, and air quality parameters of the sleep environment in real time through temperature and humidity sensors and air quality detection sensors. Both types of data are stably transmitted to the data processing center via a communication relay unit using 4G / WiFi dual-mode communication. The data processing center runs AI algorithms to process the received multimodal data. First, time alignment is performed to match physiological and environmental data in the time dimension. Outlier filtering is performed to remove interfering data. Then, a feature fusion network is used to mine the time and frequency domain features of physiological and environmental data. Based on these features, a sleep stage classification model identifies REM sleep, light sleep, and deep sleep, while an abnormal event detection model identifies events such as apnea and abnormal body movement, generating sleep analysis results. The information feedback unit's TFT display screen locally displays the monitoring data and sleep stage identification results, and a mobile terminal remotely receives the sleep analysis report, allowing users to... If the caregiver is aware of the sleep state, and the data processing center detects abnormal sleep events such as sleep apnea or uncomfortable sleep environments such as excessive temperature and humidity, it sends instructions to the intervention execution unit. The intervention execution unit's bed angle adjustment motor, temperature-controlled fiber pillow, and fresh air system automatically adjust the bed angle, sleep area temperature, and ambient air quality according to the adjustment strategy stored in the data processing center. During adjustment, priority is given to respiratory-related interventions, such as linking the fresh air system to increase ventilation during sleep apnea, before adjusting the bed angle and temperature. The adjustment process is fed back to the information feedback unit in real time to record the intervention effect. The energy supply unit dynamically adjusts the power supply strategy according to the sleep stage identified by the data processing center. During deep sleep, the power supply to non-critical modules is reduced to ensure the stable operation of the physiological monitoring unit and the data processing center. During light sleep and wakefulness, the power supply to all modules is restored to replenish the power of the energy components. The external resource unit supplements the environmental data monitoring dimension through the oxygen concentration monitoring module of the air component and uses the 4G network to ensure remote communication of the system, collaboratively improving the sleep monitoring and intervention process.

[0048] In this embodiment, as Figure 2 As shown, the control method of the multimodal data fusion sleep monitoring system for the elderly includes the following steps:

[0049] Step 1: The physiological monitoring unit and environmental sensing unit continuously collect sleep physiological data and environmental data of the elderly, and transmit them to the data processing center through the communication relay unit;

[0050] Step 2: The data processing center runs AI algorithms to fuse and process multimodal data, identify sleep stages, detect abnormal sleep events, and generate analysis results;

[0051] Step 3: The analysis results are displayed through the information feedback unit. If abnormal sleep events or uncomfortable sleep environment are detected, the data processing center sends instructions to the intervention execution unit to adjust the bed angle, temperature control fiber temperature, and fresh air system operating parameters.

[0052] Step 4: The energy supply unit dynamically adjusts the power supply strategy based on the sleep stage identified by the data processing center to ensure stable and energy-efficient system operation. The external resource unit supplements environmental data in real time and ensures communication, working together to improve sleep monitoring and intervention.

[0053] In this embodiment, when the data processing center integrates and processes multimodal data in step two, it first performs time alignment and outlier filtering preprocessing on physiological data and environmental data, and then extracts multidimensional features through a feature fusion network, inputs them into the sleep stage classification and abnormal event detection model, and outputs the recognition results; after receiving the instruction in step three, the intervention execution unit performs adjustment actions according to the preset priority, giving priority to ensuring respiratory-related interventions, and then performing bed angle and temperature adjustments. The adjustment process is fed back to the information feedback unit in real time, and the intervention effect is recorded.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal data fusion sleep monitoring system for the elderly, characterized in that, It includes basic function modules, auxiliary function modules, control function modules, and supersystem function modules; The basic functional modules include a physiological monitoring unit and an environmental sensing unit; The auxiliary function module includes an intervention execution unit and an information feedback unit; The control function module includes a data processing center and a communication relay unit; The supersystem functional modules include an energy supply unit and an external resource unit.

2. The multimodal data fusion sleep monitoring system for the elderly according to claim 1, characterized in that, The physiological monitoring unit includes millimeter-wave radar and nanobiosensors, used to collect physiological data on heart rate, respiration, and body movement of elderly people during sleep. The environmental sensing unit includes a temperature and humidity sensor and an air quality detection sensor, used to monitor the temperature, humidity, and air quality parameters of the sleep environment.

3. The multimodal data fusion sleep monitoring system for the elderly according to claim 1, characterized in that, The intervention execution unit includes a bed angle adjustment motor, a temperature-controlled fiber pillow, and a fresh air system. The bed angle adjustment motor is used to adjust the bed angle, the temperature-controlled fiber pillow is used to adjust the temperature of the sleeping area, and the fresh air system is used to optimize the air quality of the sleeping environment. The information feedback unit includes a TFT display screen and a mobile terminal. The TFT display screen is used to locally display monitoring data and sleep stage recognition results, and the mobile terminal is used to remotely receive sleep analysis reports.

4. The multimodal data fusion sleep monitoring system for the elderly according to claim 1, characterized in that, The data processing center runs an AI algorithm to fuse and process multimodal data collected by the physiological monitoring unit and the environmental perception unit to identify sleep stages and abnormal sleep events. The communication relay unit adopts a 4G / WiFi dual-mode communication method to realize data transmission and command interaction between the data processing center and other functional modules and external devices.

5. The multimodal data fusion sleep monitoring system for the elderly according to claim 1, characterized in that, The energy supply unit includes high-precision sensors that monitor the mattress and a temperature control system; The external resource unit includes an oxygen concentration monitoring module for the air component and a 4G network, which are used to supplement environmental data monitoring dimensions and ensure remote communication of the system.

6. The multimodal data fusion sleep monitoring system for the elderly according to claim 3, characterized in that, The intervention execution unit, including the bed angle adjustment motor, temperature-controlled fiber pillow, and fresh air system, automatically adjusts the bed angle, sleep area temperature, and ambient air quality based on the sleep stage identification results and abnormal event analysis results output by the data processing center. The adjustment strategy is pre-stored in the data processing center and supports personalized settings according to the user's sleep habits.

7. The multimodal data fusion sleep monitoring system for the elderly according to claim 4, characterized in that, The AI ​​algorithm running in the data processing center includes multimodal data feature extraction, sleep stage classification, and abnormal event detection models. The feature extraction model performs time-domain and frequency-domain feature mining on physiological and environmental data. The sleep stage classification model identifies REM sleep, light sleep, and deep sleep based on features. The abnormal event detection model identifies sleep apnea and abnormal body movement events.

8. The multimodal data fusion sleep monitoring system for the elderly according to claim 5, characterized in that, The energy supply unit can dynamically adjust the power supply strategy according to the sleep stage identified by the data processing center. During deep sleep, the power supply to non-critical modules is reduced to ensure the stable operation of the physiological monitoring unit and the data processing center. During light sleep and wakefulness, the power supply to all modules is restored to replenish the power of the energy components.

9. The control method of the multimodal data fusion sleep monitoring system for the elderly according to any one of claims 1-8, characterized in that, Includes the following steps, Step 1: The physiological monitoring unit and environmental sensing unit continuously collect sleep physiological data and environmental data of the elderly, and transmit them to the data processing center through the communication relay unit; Step 2: The data processing center runs AI algorithms to fuse and process multimodal data, identify sleep stages, detect abnormal sleep events, and generate analysis results; Step 3: The analysis results are displayed through the information feedback unit. If abnormal sleep events or uncomfortable sleep environment are detected, the data processing center sends instructions to the intervention execution unit to adjust the bed angle, temperature control fiber temperature, and fresh air system operating parameters. Step 4: The energy supply unit dynamically adjusts the power supply strategy based on the sleep stage identified by the data processing center to ensure stable and energy-efficient system operation. The external resource unit supplements environmental data in real time and ensures communication, working together to improve sleep monitoring and intervention.

10. The control method for the multimodal data fusion sleep monitoring system for the elderly according to claim 9, characterized in that, In step two, when the data processing center integrates and processes multimodal data, it first performs time alignment and outlier filtering preprocessing on physiological and environmental data. Then, it extracts multidimensional features through a feature fusion network, inputs them into the sleep stage classification and abnormal event detection model, and outputs the recognition results. In step three, after receiving the instruction, the intervention execution unit performs adjustment actions according to the preset priority, prioritizing respiratory-related interventions, and then adjusting the bed angle and temperature. The adjustment process is fed back to the information feedback unit in real time to record the intervention effect.

Images