The invention relates to a method and system for monitoring a person in a room

Abstract

The invention relates to a method for monitoring a person in a room, in which method: in a first step (1), a model of the room is created; in a second step (2), a system of sensors is arranged in the room; in a third step (3), the movements of the person in the room are detected by the sensor system; in a fourth step (4), the detected movements of the person are classified into at least a first class and a second class by means of a movement algorithm; and in a fifth step (5), the movements assigned to a first class are categorized.

Description

[0001] The invention relates to a method and a system for monitoring a person in a room.

[0002] As the world's population ages, Alzheimer's disease is becoming increasingly prevalent. Because Alzheimer's disease leads to cognitive decline, people who suffer from a deterioration in memory, speech, and other cognitive abilities due to the death of nerve cells in the brain require constant care for the rest of their lives. An appropriate therapeutic environment can help them make decisions regarding the performance of their daily activities and guide residents in coping with their everyday lives.

[0003] However, applying the same therapeutic environment design to different people with Alzheimer's disease ("PWADs") is insufficient, as the health status and associated problems vary from PWAD to PWAD. Each resident has a different identity, background, habits, and symptoms and therefore requires a specifically adapted environment to function well. For this reason, it is necessary to find a way to adapt new or existing environments to the specific needs of an individual PWAD.

[0004] Furthermore, it is necessary to monitor PWADs as closely as possible to ensure their best possible care. However, for ethical reasons, the use of conventional surveillance cameras is not feasible, as continuous video surveillance and the associated invasion of privacy through the recording of personal data cannot be expected of anyone.

[0005] State-of-the-art systems use infrared sensors to monitor Alzheimer's patients. The data obtained by the infrared sensors is then divided into movement categories and displayed.

[0006] A particular disadvantage is that the evaluation of the recorded movements is largely left to the user. Furthermore, conventional methods make it difficult to distinguish between individual people in a room.

[0007] It is therefore an object of the invention to provide a method and a system for monitoring a person in a room, in which continuous monitoring is possible, whereby the specific situation of the monitored person can be quickly and easily recognized.

[0008] According to the invention, a method for monitoring a person in a room is provided, wherein in a first step a model of the room is created, in a second step a system of sensors is arranged in the room, in a third step the movements of the person in the room are detected by the sensor system, in a fourth step the detected movements of the person are classified into at least a first class and a second class using a movement algorithm and in a fifth step the movements assigned to a first class are categorized using a categorization algorithm.

[0009] Generally, the various steps of the invention are performed sequentially, e.g., the second step is performed after the first step, the third step is performed after the second step, the fourth step is performed after the third step, and the fifth step is performed after the fourth step. However, some steps may overlap, e.g., the third step and the fourth step. Classification is already performed while the sensor system continues to collect motion data.

[0010] The method according to the invention uses sensors, e.g. in shadow observation, to continuously collect data on the movements of a person in a room in order to be able to react quickly if the person gets lost, for example. On the other hand, the movement data can be used to identify the person's points of self-disorientation in their room and to determine the architectural barriers that prevent them from carrying out their daily activities independently without the help of their carers. This supports the person's ability to orientate themselves when carrying out their daily tasks, e.g. in their room in a long-term health center, and consequently improves their quality of life. However, the present invention can also be used in other contexts, e.g. in elderly care, other areas of healthcare or for optimizing company buildings or schools.The space in which the person is monitored can be any environment, e.g., a part of a house, a room, an entire apartment or house, or an outdoor area. In particular, the space can be a personalized therapeutic environment in a healthcare center.

[0011] The various steps are preferably carried out with the aid of a computer and are therefore preferably computer-implemented.

[0012] The movements detected by the sensor system are assigned, for example, the attributes "object", "time", "space" and "duration", which describe the exact movement.

[0013] In the first step, a model of the room can be created using a Building Information Modeling (BIM) system. One of the following three methods can be used for this:

[0014] (1) Laser scanning or photogrammetry, primarily known as "scan-to-BIM" for highly detailed models. For most everyday facility management operations, the details provided by these models are not required;

[0015] (2) Automated conversion of 2D plans to 3D BIM; the goal is to enable smoother and more flexible workflows that eliminate the need to manually recreate BIM models; and (3) Translation of 3D geometric models to BIM: Translation of the 3D models created in suitable software into BIM software for BIM modeling. The model of the space is preferably stored in a digital format on a computer, especially a control unit. The model of the space is, for example, a digital 3D model.

[0016] In the second step of the present invention, before the

[0017] Arrangement of the sensors initially prefers a step to

[0018] Finding the optimal placement of the sensors in the room is carried out. First, the room is analyzed, and based on this, an appropriate selection and arrangement of sensors is proposed. For this purpose, an appropriate algorithm can be used, for example. The placement of the sensors can be a key factor in monitoring the movement and activity of PWADs to avoid the collection of misleading information. A preferred procedure, which should be carried out in the second step before arranging the sensors in the room, is the following, which allows the user to have an optimal solution displayed on a screen after just a few analysis steps. These analysis features are preferred.

[0019] (1) The size of the room;

[0020] (2) the sensor functions;

[0021] (3) the neighboring zones that must be covered by two different sensor combinations to distinguish the target location according to the sensor activation; and

[0022] (4) the determination of a sensor number for each zone, which depends on the sensor's field of view and the zone size.

[0023] Following this analysis, the sensors of the sensor system can then be optimally arranged according to the analysis to record the person's movements in the room. In the fourth step, the person's recorded movements are divided into at least two classes. Preferably, the movements are classified into the classes "completed" and "not completed." This allows for a coarse filtering of the movements, meaning that not all recorded movements need to be categorized in the next step, but only specific movements, e.g., movements classified as "not completed." In the case of incomplete movements, the monitored person started a movement but did not complete it correctly or to its full extent. These movements are categorized in detail in the next step to obtain a good overview of the person's problems. The movement algorithm preferably includes movement data from correctly performed movements.The actually recorded movements are compared with the stored movement data and, if a deviation is detected, classified as "not completed." If no (relevant) deviation is detected, the recorded movement is classified as "completed."

[0024] Movements classified as "completed," on the other hand, do not need to be further categorized, but are output, for example, together with the attributes "object," "time," "space," and "duration," which describe the exact movement.

[0025] In the fifth step, the person's movements, which were initially classified as "not completed," are classified into different categories to enable a quick and processed output of the recorded movements. For example, it is possible to determine human behavior at different levels of granularity. Movements are divided into categories based, for example, on the distance traveled and the length of time spent in a location. The terms "action," "activity," and "habit" are often used in activity recognition communities to describe physical behavior. The term "action" refers to a simple behavior of a person for a short period of time, such as opening a door.The term "activities" refers to complex behaviors that involve a series of actions and / or interlocking or overlapping actions over an extended period of time and that can be performed by one or more people, for example, two people preparing meals. A "habit" refers to one or more activities that are repeated regularly and in a consistent manner. Therefore, to define a set of activities as a habit, there should be little or no variation in the way such activities are performed, for example, taking medication with a glass of water every morning after breakfast.

[0026] Furthermore, the behavior of PWADs is characterized by recurrent incoherences in their actions, especially when performing complex activities that require cognitive skills. For example, the incoherent behavior of PWADs can be classified into six error categories: (1) initiation (failing to start a task); (2) organization (performing some steps of an activity inappropriately); (3) realization (skipping the main task and completing another one); (4) sequencing (disorganizing the steps of the activity); (5) judgment (performing a task in an unsafe manner); and (6) completion (failing to complete a task). These categories aim to cover all types of common errors characteristic of PWAD at each stage of degeneration.Preferably, in the fifth step, the movement data are divided into the six error categories "initiation", "organization", "realization", "sequence", "judgment" and "completion" using a categorization algorithm.

[0027] The movement algorithm and / or the categorization algorithm can, for example, be generated using person-specific data. In this case, the movement algorithm or the categorization algorithm is individually and differently developed for each observed person. The person-specific activity patterns, which are divided into "action", "activity", and "habit", are used as a basis for the movement algorithm or the categorization algorithm. The person-specific activity patterns of a person are preferably used in the respective movement algorithm in order to be able to classify the movements into one of two classes. The person-specific activity patterns of a person are preferably used in the respective categorization algorithm in order to be able to classify the movements into one of six error categories. In this case, it is therefore intended that movement data from a person is first collected, e.g.using a sensor system, and this movement data is classified into categories, e.g., "action," "activity," and "habit." This classified movement data is then used to create the movement algorithm and / or the categorization algorithm. For example, the classified movement data is used as a reference, which is compared with the movement data recorded during monitoring in order to detect deviations and thus errors.

[0028] Machine learning approaches and algorithms can be used to perform classification in the fourth step and categorization in the fifth step. Such approaches are generally more compatible with the four main phases of activity (object, time, space, and duration) and anomaly detection than classical mathematics and statistical techniques in extracting knowledge and discovering, learning, and inferring activities from data. These algorithms include (1) rule-based reasoning methods and the provision of adaptive learning mechanisms to make them more personalized, and (2) machine learning techniques to learn the activities / patterns of PWADs and detect unusual behavior. It is advantageous that such pattern recognition algorithms provide a holistic view of PWADs' activities and routines.They analyze PWADs' data at different temporal granularities to develop automated procedures for macro- and micro-assessments. Furthermore, the micro-assessment framework focuses on identifying underlying activity patterns that can be used to infer PWADs' well-being and categorize them according to the six aforementioned biases.

[0029] The sensor system preferably comprises at least one 3D infrared sensor, e.g., a 3D infrared camera. These sensors detect the thermal radiation (infrared radiation) that changes temporally and spatially due to the movement of people within a defined angular range, allowing the precise determination of the area in which and at what time a person is located. A particularly advantageous feature here is that no image of the person is taken, thus avoiding any ethical concerns.

[0030] Furthermore, the sensor system preferably comprises at least one wearable sensor. Wearable sensors are among the most useful sensor types for monitoring the movements of PWADs. For example, such a wearable sensor can comprise an accelerometer and / or a GPS sensor. Smartwatches, wristbands, and / or smart steps are preferably used as wearable sensors. These sensors measure acceleration changes, which can be used for movement detection. Due to the typically small size of wearable sensors, they are easy to attach to a belt, wristband, watch, etc., to monitor the activity of the PWAD and provide corresponding movement data.

[0031] Particularly preferably, the sensor system comprises at least one wearable sensor which is designed to detect at least one physiological characteristic of the person wearing the sensor. This sensor can, for example, be designed to detect the blood pressure, heart rate and / or body temperature of the wearer. These detected values ​​can, in particular, be used to distinguish one person from another based on these values. Furthermore, these values ​​provide a good overview of the person's state of health. In a preferred embodiment, the sensor system comprises at least one environmental sensor, for example an RFID sensor (Radio Frequency Identification), which can be used to locate misplaced objects, a switch sensor and / or a pressure sensor. Switch sensors are sensors based, for example, on magnetic properties and are attached to doors to monitor whether the door is open or closed.These sensors can help ensure the safety and well-being of PWADs.

[0032] The sensors are connected to each other or to a control element by wire or wirelessly in order to exchange data and, in particular, to send the measured values ​​from the sensors to the control element.

[0033] To utilize a modified model of the room, it is preferably provided that, in a sixth step, a suggestion for modifying the room is output based on the classified movement data. In this embodiment, the movement data is used, for example, to create a modified model of the room based on the model of the room and to use it as a template for modifying the room. For example, furniture can be moved to another location in the room to improve the movement of the respective PWAD.

[0034] For this purpose, the movement data is fed into the spatial modeling system, e.g., the BIM system. The movement data is processed in the spatial modeling system or BIM system to create a modified model of the room. By combining the model of the room and the movement data, the architectural obstacles in the room can be identified and a proposal can then be issued before modifying the room. The proposal can, for example, be a modified model of the room or include one. This enables the room to be optimized with regard to the people living there.

[0035] In intelligent therapeutic settings, the vast majority of recognition systems focus primarily on logical or temporal constraints, overlooking the limitations of spatial aspects. Nevertheless, these aspects can significantly influence recognition. A spatial constraint is the spatial configuration of an object in relation to its surroundings and other objects. For example, since each object has a size that represents the spatial area in 2D / 3D, the area of ​​a first object (dining table) should not overlap the area of ​​a second object (wardrobe) during a step (opening the wardrobe door) of an activity. Based on this premise, constraints can be set regarding the objects involved in an activity, thus improving the model of the space.

[0036] As a next step after the fourth or, if applicable, fifth step, the inventive method can be repeated, e.g., starting with the second or third step of the inventive method, to create an improvement cycle. This involves recording the movement of the PWAD in the new, modified environment, and based on this, creating a new, modified model and, ultimately, a new, modified room. This contributes to further improving the environment in which the PWAD lives.

[0037] To simplify the method, it is preferably provided that the data acquired in the third step is sent to a control unit, where the movement data is classified using the movement algorithm and then, if necessary, categorized using the categorization algorithm (fourth and fifth steps). A control unit is provided that collects and classifies the PWAD data in the third step. The control unit is further preferably configured to execute the sixth step and / or output the movement data.

[0038] Preferably, a method is provided in which a neural network is used in the fourth step and / or the fifth step to classify or categorize the recorded movement data. Furthermore, the neural network is preferably part of the control unit. Neural networks are powerful at learning, e.g., the behavior of the respective PWAD, and can be used to reliably classify or categorize the movement data accordingly.

[0039] Furthermore, it is preferred that the classified or categorized movement data be output, for example, in the form of a report, on an output device. The output device is preferably part of a control unit or connected to a control unit. The output device is, for example, a screen. Alternatively and / or additionally, the output device can comprise a printer to print a report containing the movement data. The output data can be used, for example, by a treating physician and / or a nurse to gain insight into the person's condition or movements.

[0040] The invention further relates to a system for monitoring a person in a room, in particular for carrying out a method according to the invention, comprising a control unit, a system of sensors arranged in a room, which are designed to detect the movements of the person in the room and to send the movement data to the control unit, wherein the control unit is designed to receive the movement data detected by the sensor system and to classify the movement data into at least a first class and a second class using a movement algorithm, wherein the movements assigned to a first class are then categorized using a categorization algorithm.

[0041] It is preferably provided that the sensor system comprises at least one 3D infrared sensor.

[0042] Furthermore, it is preferably provided that the control unit comprises an output unit.

[0043] In addition, the system may include the components mentioned above.

[0044] The invention will be explained in more detail below with reference to an embodiment shown schematically in the drawing. In this drawing, Fig. 1 shows a schematic representation of the method according to the invention and Fig. 2 shows a

[0045] Part of the inventive method in detail.

[0046] Fig. 1 shows a method according to the invention for monitoring a person in a room. In a first step 1, a model of the room is created, e.g. by recording the dimensions and inventory of an existing room or by creating a new room using a software program. The model of the room is stored digitally on a control unit, for example. In a second step 2, a system of sensors is arranged in the room, e.g. 3D infrared sensors and / or switch sensors. In addition, a person living in the room is preferably equipped with one or more wearable sensors which are designed, for example, to determine the person's blood pressure or heart rate. The sensors are each connected to a control unit. In a third step 3 of the method, the movements of the person in the room are recorded by the sensor system. In order to differentiate between individual people, e.g.the data collected by the wearable sensors are used. Subsequently, in a fourth step 4, the movement data collected by the sensor system is classified into at least two classes using a movement algorithm.

[0047] Based on the classification carried out in the fourth step 4, a categorization of the movement data is carried out in the fifth step 5.

[0048] The fourth step 4 and the fifth step 5 can, for example, take place in a control unit connected to the sensor system to receive the recorded movement data. The control unit is connected, for example, to an output unit on which the classified movement data is output.

[0049] Additionally, in a sixth step, a suggestion for modifying the room can be output based on the classified movement data. For example, a modified model of the room can be output. Using this suggestion or the modified model, the room can be adjusted to match the suggestion or the modified model. The process can then be restarted, e.g., in the second or third step, to measure the influence of the modified room on the behavior and movement of the person living in the room.

[0050] An embodiment of steps 4 and 5 is explained in detail in Fig. 2. In step 3, the movement data is recorded, and in step 4, the movement data is divided into a first class 7 and a second class 8. In this embodiment, movements that were not completed are classified into the first class 7, and movements that were completed are classified into the second class 8. The movements assigned to the first class 7 are then classified or categorized into one of the six error categories at 9. The movements are then assigned the respective values ​​of the attributes "object," "duration," "time," and "location" at 10, with the values ​​being taken from the recorded movement data.

[0051] The categorized movement data is output at 11 and presented in a report 12 and / or fed to a spatial modeling system, e.g., a BIM system, at 13. The report 12 can be used, for example, to assess the health of the monitored person. The data from the spatial modeling system can then be processed by a planning algorithm 14 to adapt the room model based on the movement data.

[0052] The movement data classified as "completed" in step 4 are assigned the respective values ​​of the attributes "object," "duration," "time," and "location" at 15, with the values ​​being taken from the recorded movement data. The movement data are then output at 16 and presented in a report 17.

[0053] It's worth mentioning that the method of monitoring a person in a room to generate a proposal for modifying the space can also be applied to entire buildings with a variety of uses. For example, in a school building, one or more people could be monitored, and the resulting data could be used to generate one or more proposals for modifying the school's rooms. Other types of buildings include hospitals, government buildings, or train stations, to name just a few possible applications.

Claims

Patent claims 1. Method for monitoring a person in a room in order to create a proposal for modifying the room, wherein in a first step (1) a model of the room is created, in a second step (2) a system of sensors is arranged in the room, in a third step (3) the movements of the person in the room are recorded by the sensor system, in a fourth step (4) the recorded movements of the person are classified into at least a first class and a second class using a movement algorithm and in a fifth step (5) the movements classified in the first class are categorized using a categorization algorithm and in a sixth step a proposal for modifying the room is created and output based on the categorized movement data.

2. Method according to claim 1, characterized in that the method is restarted after the sixth step in a modified room with the second step (2) or the third step (3) in order to measure the influences of the modified room on the behavior and movement of the person in the room.

3. Method according to claim 1 or 2, characterized in that the sensor system comprises at least one 3D infrared sensor.

4. Method according to one of claims 1 to 3, characterized in that the data acquired in the third step are a control unit, where the movement data are classified into at least the first class and the second class in the fourth step (4) using the movement algorithm and then in the fifth step (5) the movements classified in the first class are categorized using the categorization algorithm.

5. Method according to one of claims 1 to 4, characterized in that in the fourth step a neural network is used to classify the recorded movement data.

6. Method according to one of claims 1 to 5, characterized in that the classified and / or categorized movements are output on an output device.

7. System for monitoring a person in a room, for carrying out a method according to one of claims 1 to 6, comprising a control unit, a system of sensors arranged in a room, which are designed to detect the movements of the person in the room and to send the movement data to the control unit, wherein the control unit is designed to receive the movement data detected by the sensor system and to classify the movement data into at least a first class and a second class using a movement algorithm, wherein the movements assigned to a first class are then categorized using a categorization algorithm, wherein the control unit comprises an output unit which is designed to is configured to output the suggestion for modifying the room based on the categorized movement data.

8. System according to claim 7, characterized in that the sensor system comprises at least one 3D infrared sensor.

Images