Multimodal sensing intelligent monitoring and management methods and systems for nursing home parks

Abstract

This invention relates to the field of nursing home management technology, specifically to a multimodal sensing-based intelligent monitoring and management method and system for nursing homes. It collects real-time multimodal time-series data of elderly residents through IoT nodes, automatically identifies and labels personality traits, activity paths, and biorhythm characteristics, and constructs dynamic personalized digital profiles. Based on these profiles and nursing resources, a multi-objective optimization model is built to generate globally optimal personalized management decision-making schemes. During execution, data is monitored in real time, triggering rescheduling when rhythm deviations or task conflicts occur. At shift handover, a personalized nursing task list is extracted, and highly relevant tasks are highlighted and pushed to the incoming caregiver. Feedback on execution results is collected to iteratively update profiles and model parameters. This system can dynamically construct personalized digital profiles based on multimodal sensing data, and through multi-objective optimization and real-time feedback, achieve adaptive matching and continuous optimization of nursing home management systems to individual differences among elderly residents.

Description

Technical Field

[0001] This invention relates to the field of nursing home park management technology, and in particular to a multimodal sensing-based intelligent monitoring and management method and system for nursing home parks. Background Technology

[0002] Existing multimodal sensing-based smart monitoring and management methods and systems for nursing home parks, through the deployment of IoT sensing nodes such as infrared sensors, wearable devices, and cameras, can collect real-time data on the location trajectory, vital signs, and activity status of the elderly. At the safety protection level, they achieve basic functions such as immediate fall detection, automatic boundary crossing alarms, and nighttime bed-leaving monitoring, effectively reducing the risk of sudden accidents for the elderly and improving the park's emergency response speed. Simultaneously, through the fusion analysis and visualization of multi-source data, the system helps managers grasp the overall activity status and health dynamics of the elderly in real time, reducing the burden of manual inspections for caregivers to a certain extent. This provides fundamental support for the transformation of nursing homes from traditional labor-intensive management to data-driven refined operations, realizing the digitalization and real-time monitoring of nursing home park safety and improving the basic guarantee capabilities of elderly care services. While existing smart monitoring and management methods and systems for nursing homes based on multimodal perception can collect location, physiological, and activity data of the elderly, the data utilization is superficial, mostly limited to threshold alarms or single-event monitoring. They fail to automatically mine and construct personalized digital profiles reflecting individual personality traits, activity paths, and biorhythms from massive amounts of time-series data. Furthermore, management decisions still rely on rigid, uniform scheduling systems, lacking in-depth modeling of individual differences and global resource optimization. During execution, they cannot detect deviations between actual conditions and plans in real time and trigger dynamic rescheduling. In addition, information gaps exist in caregiver handover processes, resulting in the inability to accurately identify, dynamically adapt, and continuously meet the personalized needs of the elderly. Consequently, existing nursing home management systems cannot adaptively match and continuously optimize based on individual differences among the elderly. Summary of the Invention

[0003] The purpose of this invention is to provide a smart monitoring and management method and system for nursing home parks based on multimodal perception. It can dynamically construct personalized digital profiles based on multimodal perception data, and achieve adaptive matching and continuous optimization of nursing home management systems to individual differences of the elderly through multi-objective optimization and real-time feedback.

[0004] To achieve the above objectives, this invention provides a multimodal sensing-based intelligent monitoring and management method for nursing home campuses, comprising the following steps: By deploying multiple IoT sensing nodes in the nursing home campus, multimodal time-series data of the elderly are collected in real time. Based on the multimodal time-series data, the feature extraction algorithm automatically identifies and labels an individual's personality tendencies, high-frequency activity paths, and basic biorhythm characteristics, constructing a personalized digital profile containing dynamic tags. Based on the personalized digital profiles of all elderly people and real-time nursing resource data, a multi-objective optimization model is constructed with "maximizing the satisfaction of the elderly people's personalized needs" as the first objective function and "minimizing the scheduling conflict of nursing resources" as the second objective function. Solve the multi-objective optimization model to generate a globally optimal personalized management decision-making scheme; During the execution of the personalized management decision-making scheme, newly generated multimodal data is monitored in real time. When a deviation is detected between the preset personalized biorhythm and the actual execution time, or when a time conflict is detected between nursing tasks, a rescheduling mechanism is triggered to dynamically adjust the remaining unexecuted decision-making schemes. Based on the identity information of the caregiver taking over the shift and the list of elderly people under their care, the corresponding "personalized care task list" is extracted from the personalized management decision-making scheme. In the personalized care task list, tasks that are strongly related to the elderly’s personalized biorhythms are visually distinguished by highlighting or prioritizing them, and are pushed to the caregiver on the next shift via mobile terminal. The system collects elderly satisfaction feedback data after the actual nursing tasks are performed, or quantifies the performance effect through sensing devices. The personalized digital profile is iteratively corrected and its labels are updated based on the feedback data, while the parameters of the multi-objective optimization model are also corrected.

[0005] Specifically, the method of collecting multimodal time-series data of the elderly in real time through multiple IoT sensing nodes deployed in the nursing home campus includes: The location movement trajectory and indoor status data of the elderly are collected by infrared sensors, door magnetic sensors and millimeter-wave radar deployed indoors; Collect elderly people’s heart rate, body movement and sleep cycle data through smart bracelets or smart badges; The data collected by cameras deployed in public areas will be used to collect data on the social participation and emotional expression of the elderly. The data collected by these cameras will only be used for behavioral feature analysis and will not involve specific facial recognition in order to protect privacy.

[0006] Specifically, the step of automatically identifying and labeling an individual's personality tendencies based on the multimodal time-series data using a feature extraction algorithm includes: Based on the frequency and timing of elderly people actively triggering external interactions in time series data, as well as the distribution of elderly people's stay time in different functional areas; The personality traits of the elderly were labeled using a classification model as one or more of the following: "active and social," "quiet and introverted," or "regular and stable."

[0007] Specifically, the automatic identification and labeling of an individual's high-frequency activity paths and basic circadian rhythm characteristics based on the multimodal time-series data through a feature extraction algorithm includes: Cluster analysis was performed on location trajectory data collected continuously for no less than 7 days, and movement trajectories that appeared more than a preset threshold within a fixed time period each day were extracted as high-frequency activity paths; Periodic analysis was performed on physiological and in-room status data collected for at least 7 consecutive days, and sleep onset time, wake-up time, and pre-meal activity patterns were extracted as basic circadian rhythm characteristics.

[0008] The construction of the multi-objective optimization model with "maximizing the satisfaction of the elderly's personalized needs" as the first objective function specifically includes: Maximizing the satisfaction of the elderly’s personalized needs is achieved by minimizing the sum of the absolute values ​​of the deviations between the planned execution time and the expected execution time recorded in the personalized digital profile; The expected execution time is determined based on the elderly person's basic biorhythm characteristics, and different deviation weighting coefficients are set for different types of care tasks.

[0009] The construction of the multi-objective optimization model with "minimizing nursing resource scheduling conflicts" as the second objective function specifically includes: Minimizing nursing resource scheduling conflicts is achieved by constraining the spatial path overlap and time occupancy rate of the same caregiver performing nursing tasks within the same time unit. When the care tasks of two or more elderly people overlap in time, and the distance between the task execution locations exceeds the reach of the caregiver within a unit of time, it is determined to be a resource scheduling conflict, and the assignment scheme of such task combination will be excluded during the model solution process.

[0010] The step of solving the multi-objective optimization model to generate a globally optimal personalized management decision-making scheme specifically includes: The multi-objective optimization model is solved using a non-dominated sorting genetic algorithm with an elitist strategy, generating a set of Pareto optimal solutions. From the Pareto optimal solution set, an optimal solution is selected as the final personalized management decision scheme based on preset weight preferences or park management rules.

[0011] The rescheduling mechanism is triggered when a deviation between the preset personalized biorhythm and the actual execution time is detected, or when a timing conflict is detected between nursing tasks. Specifically, this includes: A first tolerance threshold is preset. When it is sensed that the elderly person’s actual wake-up time is later than the expected wake-up time recorded in the personalized digital file and exceeds the first tolerance threshold, the morning care task that was originally scheduled to be performed immediately after wake-up will be postponed for the corresponding time. A second tolerance threshold is preset. When changes in the actual condition of multiple elderly people are detected, causing tasks that were originally non-conflicting to overlap on the timeline, the nursing tasks in the overlapping period are partially rearranged to prioritize the timely execution of tasks that are strongly related to biological rhythms.

[0012] Among them, tasks in the personalized care task list that are strongly related to the elderly's personalized biorhythms are visually distinguished by highlighting or prioritizing them, specifically including: Match each task item in the personalized care task list with the corresponding elderly person's personalized digital profile; When a task falls under the category of tasks marked as rigid rhythmic requirements in the archive, add a background highlight mark or pin it to the top of the task list on the mobile terminal.

[0013] The multimodal perception-based smart monitoring and management system for nursing homes is used to implement the multimodal perception-based smart monitoring and management method for nursing homes. It includes a multimodal data acquisition module, a personalized digital profile construction module, a multi-objective optimization decision-making module, a dynamic rescheduling module, a shift handover information push module, and a feedback learning module. The multimodal data acquisition module is connected to the personalized digital profile construction module, the personalized digital profile construction module is connected to the multi-objective optimization decision-making module, the multi-objective optimization decision-making module is connected to the dynamic rescheduling module, the dynamic rescheduling module is connected to the multimodal data acquisition module, the shift handover information push module is connected to the multi-objective optimization decision-making module, and the feedback learning module is connected to both the personalized digital profile construction module and the multi-objective optimization decision-making module. The multimodal data acquisition module is deployed at multiple IoT sensing nodes in the nursing home park to collect multimodal time-series data of the elderly in real time. The personalized digital profile construction module is used to receive the multimodal time series data, automatically identify and label an individual's personality tendencies, high-frequency activity paths and basic biorhythm characteristics through feature extraction algorithms, and construct a personalized digital profile containing dynamic tags. The multi-objective optimization decision module is used to receive the personalized digital files of all elderly people and real-time nursing resource data, construct a multi-objective optimization model with maximizing the satisfaction of the elderly people's personalized needs as the first objective function and minimizing the scheduling conflict of nursing resources as the second objective function, and solve the model to generate the globally optimal personalized management decision scheme. The dynamic rescheduling module is used to receive newly generated multimodal data in real time during the execution of the personalized management decision-making scheme. When a deviation between the preset personalized biorhythm and the actual execution time is detected, or when a time conflict is detected between nursing tasks, the rescheduling mechanism is triggered to dynamically adjust the remaining unexecuted decision-making schemes. The handover information push module is used to respond to the handover instructions of the caregiver. Based on the identity information of the caregiver taking over and the list of elderly people under his / her care, it extracts the corresponding personalized care task list from the personalized management decision scheme, and then visually distinguishes the tasks in the list that are strongly related to the elderly people's personalized biorhythms by highlighting or prioritizing them, and pushes them to the caregiver taking over through the mobile terminal. The feedback learning module is used to collect elderly satisfaction feedback data after the actual nursing tasks are performed or to quantify the performance through sensing devices. Based on the feedback data, the personalized digital profile is iteratively corrected and the labels are updated, while the parameters of the multi-objective optimization model are also corrected.

[0014] The multimodal sensing-based intelligent monitoring and management method and system for nursing homes of this invention, taking an actual operating nursing home as an example, at 6:00 AM, the system automatically monitors the sleep cycles of all the elderly residents through millimeter-wave radar and smart bracelet data deployed in their rooms. The system finds that Grandpa A in room 305 actually woke up at 6:53 AM, 23 minutes later than the expected wake-up time of 6:30 AM recorded in his personalized digital profile. Based on this deviation, the system automatically triggers a rescheduling mechanism, postponing the morning blood pressure measurement and medication reminder tasks originally scheduled for 7:00 AM for Grandpa A to 7:25 AM. At the same time, considering the work path of Caregiver A, who is responsible for this area, the system automatically allocates Caregiver A's originally planned free window period of 7:15-7:30 AM to perform Grandpa A's morning care tasks. The entire day in the nursing home can be intelligently scheduled through the system, respecting the living habits of each elderly person and making efficient use of the energy of each caregiver, realizing a fundamental shift from "people adapting to the system" to "the system adapting to people". Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0016] Figure 1 This is a flowchart of the overall multimodal perception-based smart monitoring and management method for nursing home parks according to the present invention.

[0017] Figure 2 This is a flowchart of the multimodal data acquisition process of the present invention.

[0018] Figure 3 This is a flowchart of the multi-objective optimization and solution process of the present invention.

[0019] Figure 4 This is a schematic diagram of the structure of the multimodal sensing smart monitoring and management system for nursing home parks according to the present invention.

[0020] In the diagram: 1-Multimodal data acquisition module, 2-Personalized digital profile construction module, 3-Multi-objective optimization decision-making module, 4-Dynamic rescheduling module, 5-Shift handover information push module, 6-Feedback learning module; Detailed Implementation

[0021] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0022] In the description of this invention, it should be understood that "a plurality of" means two or more, unless otherwise explicitly specified.

[0023] Please see Figures 1 to 3 This invention provides a multimodal sensing-based intelligent monitoring and management method for nursing home parks, comprising the following steps: S1: Collect multimodal time-series data of the elderly in real time through multiple IoT sensing nodes deployed in the nursing home park; S2: Based on the multimodal time series data, the individual's personality tendencies, high-frequency activity paths, and basic biorhythm characteristics are automatically identified and labeled through feature extraction algorithms to construct a personalized digital profile containing dynamic tags; S3: Based on the personalized digital profiles of all elderly people and real-time nursing resource data, a multi-objective optimization model is constructed with "maximizing the satisfaction of the elderly people's personalized needs" as the first objective function and "minimizing the scheduling conflict of nursing resources" as the second objective function. S4: Solve the multi-objective optimization model to generate a globally optimal personalized management decision-making scheme; S5: During the execution of the personalized management decision-making scheme, newly generated multimodal data is monitored in real time. When a deviation is detected between the preset personalized biorhythm and the actual execution time, or when a time conflict is detected between nursing tasks, a rescheduling mechanism is triggered to dynamically adjust the remaining unexecuted decision-making schemes. S6: Based on the identity information of the caregiver taking over the shift and the list of elderly people under their care, extract the corresponding "personalized nursing task list" from the personalized management decision-making scheme; S7: Tasks in the personalized care task list that are strongly related to the elderly’s personalized biorhythms are visually distinguished by highlighting or prioritizing them, and are pushed to the caregiver on the next shift via a mobile terminal. S8: Collect elderly satisfaction feedback data after the actual nursing task is performed or quantify the performance effect through sensing devices. The personalized digital profile is iteratively corrected and the labels are updated based on the feedback data, and the parameters of the multi-objective optimization model are corrected at the same time.

[0024] In this embodiment, the decision-making scheme includes, but is not limited to, a personalized daily routine task list and a corresponding nursing resource scheduling plan. Taking a real-world nursing home as an example, at 6:00 AM, the system automatically monitors the sleep cycles of all residents using millimeter-wave radar and smart bracelet data deployed in their rooms. The system detects that Grandpa A in room 305 actually woke up at 6:53 AM, 23 minutes later than his expected wake-up time of 6:30 AM recorded in his personalized digital profile. Based on this discrepancy, the system automatically triggers a re-enactment. The scheduling mechanism postponed the morning blood pressure measurement and medication reminder for Grandpa A, originally scheduled for 7:00 AM, to 7:25 AM. Simultaneously, considering the work path of caregiver Xiao A, who is responsible for the area, the system automatically allocated Xiao A's planned free time from 7:15 AM to 7:30 AM to perform Grandpa A's morning care tasks. This allows for intelligent scheduling throughout the entire facility, respecting the living habits of every elderly resident and efficiently utilizing the energy of every caregiver, achieving a fundamental shift from "people adapting to the system" to "the system adapting to people."

[0025] Furthermore, the real-time collection of multimodal time-series data of the elderly through multiple IoT sensing nodes deployed in the nursing home campus specifically includes: The location movement trajectory and indoor status data of the elderly are collected by infrared sensors, door magnetic sensors and millimeter-wave radar deployed indoors; Collect elderly people’s heart rate, body movement and sleep cycle data through smart bracelets or smart badges; The data collected by cameras deployed in public areas will be used to collect data on the social participation and emotional expression of the elderly. The data collected by these cameras will only be used for behavioral feature analysis and will not involve specific facial recognition in order to protect privacy.

[0026] In this embodiment, a door magnetic sensor is installed above the door frame of each room in the park to record the frequency and time of the elderly entering and leaving the room; infrared sensors and millimeter-wave radar are deployed in the corners of the room ceiling to sense the intensity of the elderly’s activities, movement trajectory and whether they are in bed or out of bed; and the smart bracelet worn by the elderly uploads physiological data such as heart rate, blood oxygen and body movement in real time.

[0027] In the corridors and dining area of ​​the public activity area, depth cameras or ordinary cameras with privacy masking algorithms are deployed. They do not collect specific facial images, but instead use skeletal point recognition technology to count the duration of elderly people's participation in group activities, the frequency of their interaction with others, and analyze their emotional state through gait. For example, the system found that Grandma Zhang in room 103 sat quietly alone in the corner of the activity room for three consecutive afternoons with zero interaction with others. This data change will be recorded as a basis for subsequent personality tendency analysis and health status warning.

[0028] Furthermore, the automatic identification and labeling of an individual's personality tendencies based on the multimodal time-series data using a feature extraction algorithm specifically includes: Based on the frequency and timing of elderly people actively triggering external interactions in time series data, as well as the distribution of elderly people's stay time in different functional areas; The personality traits of the elderly were labeled using a classification model as one or more of the following: "active and social," "quiet and introverted," or "regular and stable."

[0029] In this embodiment, the system mines data from the past 30 days. For Grandpa B in room 201, the data analysis shows that he is always active in the chess and card room from 9 to 11 am every day and often chats with people in the garden from 3 to 5 pm. On average, he initiates more than 20 interactions with others every day. The system marks him as "active social type".

[0030] For Grandma C in room 305, data shows that she leaves her room only 3 times a day on average, mostly in the early morning and evening when there are fewer people, and spends up to 20 hours a day in her room. She almost never participates in group activities. The system labels her as "quiet and introverted". These personality labels are not static. If Grandma C starts to appear frequently in the activity room for a period of time, the system will dynamically update her label weights and may even trigger a health assessment process to explore the reasons for the change in her behavior patterns.

[0031] Furthermore, the automatic identification and labeling of an individual's high-frequency activity paths and basic circadian rhythm characteristics based on the multimodal time-series data through feature extraction algorithms specifically includes: Cluster analysis was performed on location trajectory data collected continuously for no less than 7 days, and movement trajectories that appeared more than a preset threshold within a fixed time period each day were extracted as high-frequency activity paths; Periodic analysis was performed on physiological and in-room status data collected for at least 7 consecutive days, and sleep onset time, wake-up time, and pre-meal activity patterns were extracted as basic circadian rhythm characteristics.

[0032] In this embodiment, the system performs cluster analysis on the location trajectory of each elderly person for at least 7 days. For example, for Grandpa Z in room 402, the system found that his trajectory points densely appeared on the path from his room to the dining room every morning from 7:10 to 7:20; and his trajectory frequently appeared on the path from his room to the rehabilitation room every afternoon from 2:00 to 2:30. The system extracted these as "high-frequency activity paths". At the same time, the system analyzed the body movement data of Grandpa Z's smart bracelet and the infrared sensor data of the room, and found that his average sleep time over the past 7 days was 21:35 (standard deviation 15 minutes), and his average wake-up time was 6:45 (standard deviation 10 minutes). He also had a fixed habit of using the toilet and washing up before breakfast (7:00-7:30). After processing these data, the system generated accurate "basic biorhythm characteristics" for Grandpa Z, recording key nodes such as his expected wake-up time, meal time, and nap time.

[0033] Furthermore, the construction of the multi-objective optimization model with "maximizing the satisfaction of the elderly's personalized needs" as the first objective function specifically includes: Maximizing the satisfaction of the elderly’s personalized needs is achieved by minimizing the sum of the absolute values ​​of the deviations between the planned execution time and the expected execution time recorded in the personalized digital profile; The expected execution time is determined based on the elderly person's basic biorhythm characteristics, and different deviation weighting coefficients are set for different types of care tasks.

[0034] In this embodiment, during model construction, the system quantifies "personalized needs satisfaction" into a calculable mathematical index. Assuming the three elderly individuals' desired breakfast times are 7:00, 7:30, and 8:00 respectively, and the system's meal delivery schedule is set for 7:10, 7:35, and 7:55 respectively, the absolute deviations are 10 minutes, 5 minutes, and 5 minutes respectively, totaling 20 minutes. One of the system's optimization goals is to minimize the sum of these deviations across all care tasks for all elderly individuals. Simultaneously, the model assigns different weights to different tasks: for example, for medical tasks like insulin injections that must be performed at strict times, the deviation weight coefficient is set to 10; while for daily care tasks like room cleaning, the deviation weight coefficient is set to 1. This means that the system would rather delay the cleaning task by 10 minutes to ensure timely insulin injections, reflecting the clinical rationality of the decision.

[0035] Furthermore, the construction of the multi-objective optimization model with "minimizing nursing resource scheduling conflicts" as the second objective function specifically includes: Minimizing nursing resource scheduling conflicts is achieved by constraining the spatial path overlap and time occupancy rate of the same caregiver performing nursing tasks within the same time unit. When the care tasks of two or more elderly people overlap in time, and the distance between the task execution locations exceeds the reach of the caregiver within a unit of time, it is determined to be a resource scheduling conflict, and the assignment scheme of such task combination will be excluded during the model solution process.

[0036] In this embodiment, the system simulates the movement of caregivers. Suppose that caregiver A needs to measure the blood pressure of an elderly person in room 301 and assist an elderly person in room 302 to use the toilet at 8:00. The execution times of these two tasks overlap and the rooms are adjacent. The movement distance is only 1 minute, which does not constitute a conflict. Caregiver A can complete them in turn. However, if the system attempts to arrange for caregiver A to measure the blood pressure of an elderly person in room 301 in the East Building at 8:00 and at the same time conduct rehabilitation training for an elderly person in room 201 in the West Building (8 minutes' walk) at 8:05, this will constitute a resource scheduling conflict because the movement time exceeds the task interval. During the model solution process, such an unrealistic assignment scheme will be automatically excluded by the constraints to ensure that the generated schedule is feasible in both physical space and time.

[0037] Furthermore, solving the multi-objective optimization model to generate a globally optimal personalized management decision-making scheme specifically includes: The multi-objective optimization model is solved using a non-dominated sorting genetic algorithm with an elitist strategy, generating a set of Pareto optimal solutions. From the Pareto optimal solution set, an optimal solution is selected as the final personalized management decision scheme based on preset weight preferences or park management rules.

[0038] In this embodiment, facing the mutually constraining objectives of "maximizing demand satisfaction" and "minimizing resource conflict," the system does not employ a simple weighted summation but instead uses the NSGA-II algorithm. This algorithm generates a series of Pareto optimal solutions with varying advantages and disadvantages. For example, solution set A may achieve extremely high demand satisfaction (total deviation of only 50 minutes), but with a higher resource conflict index; solution set B has very few conflicts, but slightly lower demand satisfaction (total deviation of 65 minutes). Park managers can select the solution that best suits their current management strategy from the Pareto front as the final implementation plan based on current operational priorities (for example, during the service quality evaluation week at the beginning of the month, solutions with higher demand satisfaction can be prioritized; while during holidays when caregivers are in short supply, solutions with less resource conflict can be selected). This achieves both flexibility and scientific decision-making.

[0039] Furthermore, when a deviation between the preset personalized biorhythm and the actual execution time is detected, or when a timing conflict is detected between nursing tasks, a rescheduling mechanism is triggered, specifically including: A first tolerance threshold is preset. When it is sensed that the elderly person’s actual wake-up time is later than the expected wake-up time recorded in the personalized digital file and exceeds the first tolerance threshold, the morning care task that was originally scheduled to be performed immediately after wake-up will be postponed for the corresponding time. A second tolerance threshold is preset. When changes in the actual condition of multiple elderly people are detected, causing tasks that were originally non-conflicting to overlap on the timeline, the nursing tasks in the overlapping period are partially rearranged to prioritize the timely execution of tasks that are strongly related to biological rhythms.

[0040] In this embodiment, the system is configured with dynamic adjustment rules. The first tolerance threshold is set at 15 minutes. When the system detects that Grandpa C in room 501 wakes up 20 minutes later than the record, the system determines that the threshold has been exceeded and automatically triggers rescheduling: the "assisting in getting up" and "fasting blood glucose measurement" tasks, which were originally scheduled to be performed within 15 minutes of waking up, are shifted back by 20 minutes, and the task list on the caregiver's terminal is updated. The second tolerance threshold is used to handle sudden task backlog. For example, at 10:00 a.m., three elderly people press the call bell almost simultaneously, causing the rehabilitation tasks that caregiver Xiao Z had arranged to overlap instantly. After the system detects this timing conflict, it will immediately perform a local rescheduling. It will prioritize the execution of the task of one of the elderly people who is strongly related to their biological rhythm (such as the elderly person's record showing that they have a fixed medication need at 10:15 a.m. every day), while slightly delaying the non-urgent needs of the other two elderly people (such as TV adjustment), and push the adjusted new path to Xiao Z.

[0041] Furthermore, tasks in the personalized care task list that are strongly correlated with the elderly person's personalized biorhythms are visually distinguished by highlighting or prioritizing them, specifically including: Match each task item in the personalized care task list with the corresponding elderly person's personalized digital profile; When a task falls under the category of tasks marked as rigid rhythmic requirements in the archive, add a background highlight mark or pin it to the top of the task list on the mobile terminal.

[0042] In this embodiment, at 7:30 AM, the early shift caregiver, Xiao L, opens her work mobile app. The system automatically pushes a personalized care task list for the eight elderly people she is responsible for. In the list, the task of "assisting with morning medication" for Grandpa D in room 602 is highlighted in orange and placed at the top of the list. Xiao L clicks to view the notes, and the system prompts: "Grandpa D's medication habit is extremely punctual (7:45±5 minutes). The execution time has been automatically optimized to 7:45 based on his wake-up time today (7:20). Please prioritize timely execution." In the middle of the list, the task of "room ventilation" for Grandma S in room 605 is not highlighted. This visual distinction allows caregivers to identify at a glance which are rigid needs that must be completed "on time" and which are flexible tasks that can be adjusted flexibly, effectively avoiding the omission of key information due to information overload or verbal communication during shift handover.

[0043] Please see Figure 4 A multimodal perception-based intelligent monitoring and management system for nursing home parks is provided to implement the aforementioned multimodal perception-based intelligent monitoring and management method for nursing home parks. The system includes a multimodal data acquisition module 1, a personalized digital profile construction module 2, a multi-objective optimization decision-making module 3, a dynamic rescheduling module 4, a shift handover information push module 5, and a feedback learning module 6. The multimodal data acquisition module 1 is connected to the personalized digital profile construction module 2, the personalized digital profile construction module 2 is connected to the multi-objective optimization decision-making module 3, the multi-objective optimization decision-making module 3 is connected to the dynamic rescheduling module 4, the dynamic rescheduling module 4 is connected to the multimodal data acquisition module 1, the shift handover information push module 5 is connected to the multi-objective optimization decision-making module 3, and the feedback learning module 6 is connected to both the personalized digital profile construction module 2 and the multi-objective optimization decision-making module 3. The multimodal data acquisition module 1 is deployed at multiple IoT sensing nodes in the nursing home park to collect multimodal time-series data of the elderly in real time. The personalized digital profile construction module 2 is used to receive the multimodal time series data, automatically identify and label an individual's personality tendencies, high-frequency activity paths and basic biorhythm characteristics through feature extraction algorithms, and construct a personalized digital profile containing dynamic tags. The multi-objective optimization decision module 3 is used to receive the personalized digital files of all elderly people and real-time nursing resource data, construct a multi-objective optimization model with maximizing the satisfaction of the elderly people's personalized needs as the first objective function and minimizing the scheduling conflict of nursing resources as the second objective function, and solve the model to generate the globally optimal personalized management decision scheme. The dynamic rescheduling module 4 is used to receive newly generated multimodal data in real time during the execution of the personalized management decision scheme. When a deviation between the preset personalized biorhythm and the actual execution time is detected, or when a time conflict is detected between nursing tasks, the rescheduling mechanism is triggered to dynamically adjust the remaining unexecuted decision schemes. The handover information push module 5 is used to respond to the caregiver handover instruction, extract the corresponding personalized care task list from the personalized management decision scheme according to the identity information of the incoming caregiver and the list of elderly people under their care, and visually distinguish the tasks in the list that are strongly related to the elderly people's personalized biorhythms by highlighting or prioritizing them, and then push them to the incoming caregiver through the mobile terminal. The feedback learning module 6 is used to collect elderly satisfaction feedback data after the actual nursing task is performed or to quantify the performance effect through sensing devices. Based on the feedback data, the personalized digital profile is iteratively corrected and the labels are updated, while the parameters of the multi-objective optimization model are also corrected.

[0044] In this embodiment, the modules work together to form a complete intelligent closed loop. The multimodal data acquisition module 1 continuously senses all dynamics within the park and feeds the raw data stream to the personalized digital archive construction module 2 in real time.

[0045] The personalized digital profile building module 2 extracts each elderly person's behavioral patterns and habit preferences from massive amounts of data through an algorithm model, generating personalized tags rich in semantics.

[0046] The multi-objective optimization decision module 3 receives the "personalized profiles" of all elderly people and real-time changing nursing resources (such as the number of caregivers on duty, their skills, and their locations), and uses complex optimization algorithms to automatically generate the globally optimal shift schedule and task list for the next day every day.

[0047] During execution, the dynamic rescheduling module 4 constantly monitors the real-time data stream. Once a deviation is detected between the plan and the actual situation, an emergency plan is immediately activated to fine-tune subsequent tasks.

[0048] The shift handover information push module 5 accurately pushes the refined and enhanced key tasks to the caregivers at the time of shift handover.

[0049] Finally, the feedback learning module 6, as the "memory and evolution center" of the system, collects satisfaction evaluations and effect data after each task execution, continuously reflects on and optimizes the observation model and decision-making strategy, making the entire system smarter and more in line with the actual needs of the park with the use of the system, and truly realizing the closed-loop management of the entire process of perception, decision-making, execution, feedback and optimization.

[0050] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A multimodal sensing-based intelligent monitoring and management method for nursing home parks, characterized in that, Includes the following steps, By deploying multiple IoT sensing nodes in the nursing home campus, multimodal time-series data of the elderly are collected in real time. Based on the multimodal time-series data, the feature extraction algorithm automatically identifies and labels an individual's personality tendencies, high-frequency activity paths, and basic biorhythm characteristics, constructing a personalized digital profile containing dynamic tags. Based on the personalized digital profiles of all elderly people and real-time nursing resource data, a multi-objective optimization model is constructed with "maximizing the satisfaction of the elderly people's personalized needs" as the first objective function and "minimizing the scheduling conflict of nursing resources" as the second objective function. Solve the multi-objective optimization model to generate a globally optimal personalized management decision-making scheme; During the execution of the personalized management decision-making scheme, newly generated multimodal data is monitored in real time. When a deviation is detected between the preset personalized biorhythm and the actual execution time, or when a time conflict is detected between nursing tasks, a rescheduling mechanism is triggered to dynamically adjust the remaining unexecuted decision-making schemes. Based on the identity information of the caregiver taking over the shift and the list of elderly people under their care, the corresponding "personalized care task list" is extracted from the personalized management decision-making scheme. In the personalized care task list, tasks that are strongly related to the elderly’s personalized biorhythms are visually distinguished by highlighting or prioritizing them, and are pushed to the caregiver on the next shift via mobile terminal. The system collects elderly satisfaction feedback data after the actual nursing tasks are performed, or quantifies the performance effect through sensing devices. The personalized digital profile is iteratively corrected and its labels are updated based on the feedback data, while the parameters of the multi-objective optimization model are also corrected.

2. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, The method involves collecting multimodal time-series data of the elderly in real time through multiple IoT sensing nodes deployed in the nursing home campus, specifically including: The location movement trajectory and indoor status data of the elderly are collected by infrared sensors, door magnetic sensors and millimeter-wave radar deployed indoors; Collect elderly people’s heart rate, body movement and sleep cycle data through smart bracelets or smart badges; The data collected by cameras deployed in public areas will be used to collect data on the social participation and emotional expression of the elderly. The data collected by these cameras will only be used for behavioral feature analysis and will not involve specific facial recognition in order to protect privacy.

3. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, The automatic identification and labeling of individual personality tendencies based on the multimodal time-series data using a feature extraction algorithm specifically includes: Based on the frequency and timing of elderly people actively triggering external interactions in time series data, as well as the distribution of elderly people's stay time in different functional areas; The personality traits of the elderly were labeled using a classification model as one or more of the following: "active and social," "quiet and introverted," or "regular and stable." 4. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, The automatic identification and labeling of individuals' high-frequency activity paths and basic circadian rhythm characteristics based on the multimodal time-series data through feature extraction algorithms specifically includes: Cluster analysis was performed on location trajectory data collected continuously for no less than 7 days, and movement trajectories that appeared more than a preset threshold within a fixed time period each day were extracted as high-frequency activity paths; Periodic analysis was performed on physiological and in-room status data collected for at least 7 consecutive days, and sleep onset time, wake-up time, and pre-meal activity patterns were extracted as basic circadian rhythm characteristics.

5. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, The construction of the multi-objective optimization model with "maximizing the satisfaction of the elderly's personalized needs" as the first objective function specifically includes: Maximizing the satisfaction of the elderly’s personalized needs is achieved by minimizing the sum of the absolute values ​​of the deviations between the planned execution time and the expected execution time recorded in the personalized digital profile; The expected execution time is determined based on the elderly person's basic biorhythm characteristics, and different deviation weighting coefficients are set for different types of care tasks.

6. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, The construction of the multi-objective optimization model with "minimizing nursing resource scheduling conflicts" as the second objective function specifically includes: Minimizing nursing resource scheduling conflicts is achieved by constraining the spatial path overlap and time occupancy rate of the same caregiver performing nursing tasks within the same time unit. When the care tasks of two or more elderly people overlap in time, and the distance between the task execution locations exceeds the reach of the caregiver within a unit of time, it is determined to be a resource scheduling conflict, and the assignment scheme of such task combination will be excluded during the model solution process.

7. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, Solving the multi-objective optimization model to generate a globally optimal personalized management decision-making scheme specifically includes: The multi-objective optimization model is solved using a non-dominated sorting genetic algorithm with an elitist strategy, generating a set of Pareto optimal solutions. From the Pareto optimal solution set, an optimal solution is selected as the final personalized management decision scheme based on preset weight preferences or park management rules.

8. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, When a deviation between the preset personalized biorhythm and the actual execution time is detected, or when a timing conflict is detected between nursing tasks, a rescheduling mechanism is triggered, specifically including: A first tolerance threshold is preset. When it is sensed that the elderly person’s actual wake-up time is later than the expected wake-up time recorded in the personalized digital file and exceeds the first tolerance threshold, the morning care task that was originally scheduled to be performed immediately after wake-up will be postponed for the corresponding time. A second tolerance threshold is preset. When changes in the actual condition of multiple elderly people are detected, causing tasks that were originally non-conflicting to overlap on the timeline, the nursing tasks in the overlapping period are partially rearranged to prioritize the timely execution of tasks that are strongly related to biological rhythms.

9. The multimodal sensing-based intelligent monitoring and management method for nursing home parks according to claim 1, characterized in that, Tasks strongly correlated with the elderly person's individual biorhythms in the personalized care task list are visually distinguished by highlighting or prioritizing them, specifically including: Match each task item in the personalized care task list with the corresponding elderly person's personalized digital profile; When a task falls under the category of tasks marked as rigid rhythmic requirements in the archive, add a background highlight mark or pin it to the top of the task list on the mobile terminal.

10. A multimodal sensing-based intelligent monitoring and management system for nursing home parks, used to implement the multimodal sensing-based intelligent monitoring and management method for nursing home parks as described in claim 1, characterized in that, It includes a multimodal data acquisition module, a personalized digital archive construction module, a multi-objective optimization decision-making module, a dynamic rescheduling module, a shift handover information push module, and a feedback learning module. The multimodal data acquisition module is connected to the personalized digital archive construction module, the personalized digital archive construction module is connected to the multi-objective optimization decision-making module, the multi-objective optimization decision-making module is connected to the dynamic rescheduling module, the dynamic rescheduling module is connected to the multimodal data acquisition module, the shift handover information push module is connected to the multi-objective optimization decision-making module, and the feedback learning module is connected to both the personalized digital archive construction module and the multi-objective optimization decision-making module. The multimodal data acquisition module is deployed at multiple IoT sensing nodes in the nursing home park to collect multimodal time-series data of the elderly in real time. The personalized digital profile construction module is used to receive the multimodal time series data, automatically identify and label an individual's personality tendencies, high-frequency activity paths and basic biorhythm characteristics through feature extraction algorithms, and construct a personalized digital profile containing dynamic tags. The multi-objective optimization decision module is used to receive the personalized digital files of all elderly people and real-time nursing resource data, construct a multi-objective optimization model with maximizing the satisfaction of the elderly people's personalized needs as the first objective function and minimizing the scheduling conflict of nursing resources as the second objective function, and solve the model to generate the globally optimal personalized management decision scheme. The dynamic rescheduling module is used to receive newly generated multimodal data in real time during the execution of the personalized management decision-making scheme. When a deviation between the preset personalized biorhythm and the actual execution time is detected, or when a time conflict is detected between nursing tasks, the rescheduling mechanism is triggered to dynamically adjust the remaining unexecuted decision-making schemes. The handover information push module is used to respond to the handover instructions of the caregiver. Based on the identity information of the caregiver taking over and the list of elderly people under his / her care, it extracts the corresponding personalized care task list from the personalized management decision scheme, and then visually distinguishes the tasks in the list that are strongly related to the elderly people's personalized biorhythms by highlighting or prioritizing them, and pushes them to the caregiver taking over through the mobile terminal. The feedback learning module is used to collect elderly satisfaction feedback data after the actual nursing tasks are performed or to quantify the performance through sensing devices. Based on the feedback data, the personalized digital profile is iteratively corrected and the labels are updated, while the parameters of the multi-objective optimization model are also corrected.

Images