Life support system and life support method

Abstract

To improve an accuracy of an alert for notifying an abnormality of a target person when watching the target person in a living space.SOLUTION: A life support system 1 comprises: a moving body 2 which moves to a vicinity of a resident 11, calculates a feature amount 110 of a physical activity of the resident 11 and a living behavior of the resident 11 from a measurement result including position information of the resident 11 obtained from a measurement sensor provided in the moving body, and transmits a calculation result to a server 3; and the server 3 which obtains a change from a feature amount 110 of a past physical activity to a feature amount 110 of a current physical activity for each received living behavior of the resident 11, and transmits an alert to an information terminal 5 according to the change.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a life assistance system and a life assistance method. [Background technology]

[0002] By measuring the physical activity and health status of elderly people receiving care during their daily lives, appropriate interventions such as rehabilitation can be implemented at the appropriate time, for example, when they become unwell and have difficulty moving their legs. However, elderly people may feel burdened or uncomfortable when wearing wearable sensors on their bodies to directly measure their activity. Therefore, a method has been proposed for detecting health conditions while reducing the burden on the subject by placing a device that measures physical activity in the living space where the subject is present.

[0003] A common method is to install cameras or sensors on the ceiling or walls of the room where the subject lives to capture images and distance images, and capture the movements of the subject as they are active indoors. However, the method of installing cameras and sensors indoors has the following problems. Measurements can only be taken of people who are in the room where the camera is installed and within the camera's field of view. - From a privacy standpoint, there is a reluctance to install cameras in private living spaces that are not public spaces. Measuring physical activity throughout the home requires installing cameras and sensors in each room, which is costly.

[0004] Therefore, a method has been proposed in which a camera is mounted on a mobile object such as a robot and made movable, allowing the camera to capture human physical activity regardless of location. The mobile object uses a non-contact sensor to detect the presence of a person, predicts the person's next action, estimates the range of that action, and identifies obstacles. The mobile object then sets a travel route to avoid the recognized obstacles, thereby avoiding collisions with people or walls.

[0005] Patent Document 1 describes a mobile training support device in which a running body having a driving means is provided with a detection means for detecting the posture of a user undergoing training such as walking. Patent document 2 describes a robotic measuring instrument that is equipped with a three-dimensional spatial information sensor on its running section that takes images using optical means, and that continuously measures the movement of the entire body of the person being measured, including the limbs, by estimating the correspondence between the collected three-dimensional spatial coordinates and each part of the person being measured from the two-dimensional images through computational processing. Patent Document 3 describes an autonomous robot that has an imaging unit that captures images of its surroundings and maintains a natural sense of distance from an object in accordance with the size of the captured object. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-73630 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-80671 [Patent Document 3] International Publication No. 2018 / 047802 Summary of the Invention [Problem to be solved by the invention]

[0007] Family members and care managers who care for elderly people and other measurement subjects often do not want to constantly monitor the elderly person's daily life, but rather would like to be alerted only at appropriate times (when their health condition deteriorates or is likely to deteriorate) so that they can take appropriate interventions such as hospital visits.In other words, there is a need for a well-balanced monitoring service that reduces the burden of caregiving by minimizing alerts under normal circumstances, but supports intervention by issuing alerts when necessary.

[0008] Furthermore, unlike a medical checkup at a hospital, when measuring subjects in their living space, the subjects are engaged in various daily activities such as sleeping, eating, etc. Therefore, there are situations where it may be inappropriate to immediately issue an alert indicating a decline in the subject's physical ability just because the subject's walking speed has decreased compared to yesterday. For example, when comparing walking speeds at the same time at 10:00 a.m., if today's walking speed during a break is slower than yesterday's walking speed while cleaning, this is simply because daily activities were different, and in this case an alert should not be issued.

[0009] The present invention has been made in consideration of the above circumstances, and its main objective is to improve the accuracy of alerts that notify of abnormalities in a subject when monitoring the subject in a living space. [Means for solving the problem]

[0010] In order to solve the above problems, the life assistance system of the present invention has the following features. The present invention provides a mobile object that moves to the vicinity of a resident, calculates a feature of the physical activity of the resident and the living behavior of the resident from a measurement result including location information of the resident obtained from a measurement sensor provided in the mobile object, and transmits the calculation result to a server; The server compares past physical activity and current physical activity for each received living behavior of the resident with each other for the same living behavior to determine a change from a feature amount of the past physical activity to a feature amount of the current physical activity, and transmits an alert to an information terminal in response to the change. And, The moving body is a distance sensor that uses electromagnetic waves or sound waves emitted from the moving object to determine a distance to the resident and a moving speed of the resident along the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity; an image sensor that determines the moving speed of the resident perpendicular to the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity based on a time change in the position of the resident reflected in an image of the resident; and It is characterized by: Other means will be described later. [Effects of the Invention]

[0011] According to the present invention, when a subject is being monitored in a living space, the accuracy of alerts that notify the subject of abnormalities can be improved. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram of a life assistance system according to a first embodiment. [Figure 2] FIG. 2 is a plan view of the house of FIG. 1 according to the first embodiment. [Figure 3] FIG. 2 is a diagram showing a display screen of the information terminal according to the first embodiment. [Figure 4] 1 is a schematic diagram of a moving body of a life assistance system according to a first embodiment. [Figure 5] 1 is a block diagram of a life assistance system according to a first embodiment. [Figure 6] 10 is an example of a flowchart illustrating processing of a moving body according to the first embodiment. [Figure 7] FIG. 2 is an explanatory diagram illustrating an example of feature amounts related to the first embodiment. [Figure 8] 8 is an example of a flowchart showing a process of the server following the process of FIG. 7 according to the first embodiment. [Figure 9] FIG. 10 is a plan view of a house according to a second embodiment. [Figure 10] FIG. 10 is a plan view of the same house as FIG. 9 according to the second embodiment, but with a different charging station location. [Figure 11] 10 is an example of a flowchart of a moving body according to the second embodiment. [Figure 12] FIG. 10 is a schematic diagram of a life assistance system according to a third embodiment. [Figure 13] 10 is a schedule graph showing the daily activities of residents according to the third embodiment. [Figure 14] FIG. 10 is a block diagram illustrating an example of a life assistance system according to a third embodiment. [Figure 15] 10 is an example of a flowchart of a life assistance system according to a third embodiment. [Figure 16] 16 is an example of a flowchart of the server following the process of FIG. 15 according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings. [Example]

[0014] 1 is a schematic diagram of a life assistance system 1 according to a first embodiment. In the first embodiment, an example will be described in which a resident 11 is monitored while a mobile object 2 is moving. The life assistance system 1 includes a mobile object 2 and a server 3 . The mobile object 2 moves close enough to the resident 11 so that it can observe the resident 11, and calculates the physical activity characteristics 110 (Figure 7) of the resident 11 and the daily activities of the resident 11 from the measurement results including the location information of the resident 11 obtained from the measurement sensors (image sensor 21, distance sensor 22) equipped on the mobile object 2, and transmits the calculation results to the server 3. For each received daily living activity of the resident 11, the server 3 calculates the change from the feature 110 of past physical activity to the feature 110 of current physical activity, and transmits an alert and the feature 110 of physical activity to the information terminal 5 in accordance with the change. The daily activities include, for example, sleeping, going out, resting, eating, and doing laundry, as will be described later in Fig. 13. The physical activity feature 110 includes, for example, the amount of activity, energy (energy consumed), sharpness of movement, time (time required), and number of times for the resident 11, as will be described later in the radar chart 52 of Fig. 3.

[0015] The mobile object 2 and the server 3 are connected wirelessly or by wire so as to be able to communicate with each other. 1, the server 3 is installed outside the residence 10. On the other hand, as long as the server 3 can communicate with the mobile object 2, the installation location of the server 3 may be within the operating company of the life support service, the operating company of the cloud service, or within the residence 10. The mobile object 2 is installed inside a house 10 where a resident 11 resides, and can move freely and autonomously within the house 10 by driving drive wheels 23. Autonomous movement means that the mobile object 2 moves along a movement route calculated autonomously by a computer such as the server 3 or the mobile object 2, rather than being directly commanded by a human such as the resident 11 using operation signals such as moving forward or turning left.

[0016] The moving object 2 is equipped with an image sensor 21 and a distance sensor 22. The image sensor 21 acquires a visible light image of the resident 11. The distance sensor 22 can measure the distance 221 between the mobile object 2 and the resident 11 by emitting radio waves 220 such as microwaves or millimeter waves from the mobile object 2 to the outside. The distance sensor 22 may emit electromagnetic waves (radio waves 220, laser light, infrared light, etc.) or sound waves (ultrasound waves) to measure the distance to the target object. This allows the resident 11 to have his or her physical activity measured without having to wear anything.

[0017] The information terminal 5 receives an alert about the physical activity of the resident 11 and the feature amount 110 of the physical activity from the server 3, and displays them to the user of the information terminal 5 as follows. The resident (e.g., elderly people, children living alone to attend school, preschoolers, elementary, junior high, and high school students, etc.) Family members who look after Resident 11 (Resident Family Members 5A). Resident Family Members 5A are, for example, Resident 11's adult children who live with or separately from Resident 11. Resident 11's Care Manager Persons related to the nursing care facility that has a contract with Resident 11

[0018] Therefore, the information terminal 5 is configured as a device that can receive the following information in the server 3, for example. A smartphone that communicates with Server 3 and displays the internal information of Server 3 Devices such as tablets and PCs Image display devices such as televisions and monitors - Monitoring system device for watching over residents -System equipment for managing medical and nursing care information

[0019] This allows the user of the information terminal 5 to detect changes or abnormalities in the physical activity or daily behavior of the resident 11. Furthermore, by sending alert information to the information terminal 5 of a system that manages medical and nursing care information, the physical information of the resident 11 and related health information can be shared with a wide range of stakeholders, such as medical professionals and nursing care workers. In this case, in addition to detecting the amount of change and issuing an alert, necessary information from the physical activity feature amount 110 transmitted by the mobile object 2 may be transmitted to the information terminal 5 or another system.

[0020] FIG. 2 is a plan view of the house 10 of FIG. The map data 301 of the house 10 shown in this floor plan may be registered in advance in the mobile body 2, or may be created by measuring the shape of the interior of the house 10 using the image sensor 21 and distance sensor 22 while the mobile body 2 travels within the house. This map data 301 is used not only to set a route for the moving object 2 so that it does not collide with walls or furniture, but also to identify the behavior of the resident 11 from his / her location within the house 10, such as when the resident 11 is eating in the kitchen.

[0021] FIG. 3 is a diagram showing the display screen of the information terminal 5. As shown in FIG. The information terminal 5 receives and displays information stored in the server 3, thereby informing the resident's family 5A and the like of the state of the resident 11. The display screen of the information terminal 5 displays time series data 51 that displays characteristic quantities measured within the house 10, such as the activity level of the resident 11, in a time series graph, and a radar chart 52 for comparing the balance between multiple characteristic quantities measured from the resident 11. Since the server 3 stores information that the mobile body 2 has transmitted up to now, the server 3 may display the results of a comparison between past information and current information on the information terminal 5, such as time series data 51.

[0022] On the other hand, the server 3 may display, as detailed data at a certain time, a total of seven (heptagonal) feature amounts such as the amount of activity (maximum) and energy (total) as shown in a radar chart 52 in Fig. 3 on the information terminal 5. For this purpose, the mobile object 2 calculates, for example, the following information and transmits the calculation results to the server 3: Room shape in house 10 Position of the moving object 2 within the house 10 Distance 221 between moving object 2 and resident 11 Position and posture of occupant 11 Skeletal model of Resident 11 Physical activity features 110 extracted from the skeletal model of the resident 11 (details in Figure 7) include walking speed, limb movement speed, movement speed of specific points on the body, and the angle of limbs relative to the trunk. A physical activity index (such as movement sharpness and activity level) calculated using a specific formula from the physical activity characteristics 110 of the resident 11 Resident 11's location Resident 11's daily activities (time spent sleeping, going out, etc., and number of meals, etc.; see Figure 13 for details)

[0023] FIG. 4 is a schematic diagram of a moving body of the life assistance system 1. As shown in FIG. The moving body 2 is equipped with an image sensor 21, a distance sensor 22, drive wheels 23, and a control board 24. 4, the distance sensor 22 is mounted on the front surface of the main body of the moving body 2 and is configured to measure the distance ahead. However, the position of the distance sensor 22 is not limited to the front surface, and it may be mounted on the top surface or in all directions, or multiple distance sensors may be mounted. 4, the image sensor 21 is mounted on the top surface of the main body of the moving object 2. However, the position of the image sensor 21 is not limited to the top surface, and it may be mounted on a side surface, or multiple image sensors may be mounted. In addition, the angle of view that can be photographed is arbitrary, but the wider the angle, the wider the range of image information that can be acquired with one image sensor 21.

[0024] It is desirable to use a moving object 2 that is designed for use within a house 10 as described below. - The size of a typical robot vacuum cleaner (long diameter φ250-350mm) that can move freely around the house -Weight limit is 5kg No noise generation Works wirelessly Wheels and legs that do not damage the floor of a typical home On the other hand, outdoor mobile bodies weighing more than 30 kg and mobile bodies for buildings, offices, and facilities with a diameter of more than 500 mm are not suitable for tracking occupants 11 who move frequently, but these mobile bodies may also be used.

[0025] Furthermore, the moving body 2 may be equipped with sensors other than the image sensor 21 and the distance sensor 22, and may have at least one of the following functions: cleaning function, air purification function, pet function, security function, sound recognition function, and monitoring function. For example, the moving object 2 may be provided with functions other than movement and sensing, such as a household robot such as the following (Operation 1) and (Operation 2). (Operation 1) The mobile object 2 is operated as a robot vacuum cleaner. A robot vacuum cleaner is equipped with a fan, brushes, etc. inside the main body, and can collect and clean dust accumulated on the floor. Furthermore, if the robot vacuum cleaner is equipped with a dust sensor that can detect fine particles, it can detect the amount of dust and clean with an appropriate airflow.

[0026] (Operation 2) The moving object 2 is operated as a pet robot. The appearance of the pet robot is made to resemble a pet that people like, such as a small dog. If the pet robot is equipped with an audio sensor to acquire sound information, it can move in the direction of the sound in response to a voice calling the pet. In this way, if a robot vacuum cleaner or pet robot has the configuration of the moving body 2, physical activity can be measured as needed while the robot vacuum cleaner or pet robot is normally operated according to its original functions and roles.

[0027] Furthermore, if the mobile object 2 has other functions such as a robot vacuum cleaner or a pet robot, it can measure physical activity as needed while performing its original functions of cleaning the house 10 and being loved by the resident 11. Therefore, the mobile object 2 can obtain physical activity and related health information without interfering with the resident 11's life or invading it more than necessary. Furthermore, small mobile objects 2 such as robot vacuum cleaners and pet robots are much smaller than the camera-equipped mobile objects that have been common in the past, so they can be moved freely within the home 10, and there is no need to prepare a large space for measuring physical activity.

[0028] Generally, from the perspective of protecting privacy, it is often not desirable to have devices that collect images and sounds, such as cameras and microphones, permanently installed in the house 10 where the resident 11 lives. Therefore, by measuring the physical activity of the resident using a mobile object 2 equipped with an image sensor 21 and a distance sensor 22, sensors such as the image sensor 21 approach only when necessary, so that the physical activity of the resident 11 can be measured without causing discomfort to the resident 11, and the health condition of the resident 11 can be understood from the physical activity of the resident 11.

[0029] Furthermore, by using the mobile object 2, which is used as a home appliance in the house 10, to watch over the resident 11, the following effects can be obtained. There is no need to prepare a separate environment to measure the behavior of subjects, such as a large public facility hall or a senior care facility where elderly people receive mobility training. A general-purpose mobile robot measuring instrument would have difficulty moving over steps and other obstacles within the house 10. On the other hand, the mobile body 2, which is designed in advance to move within the house 10, can move smoothly within the house 10 and easily maintain an appropriate distance to measure (photograph) the entire body of the resident 11. By using the mobile object 2 that has been coexisting in the living space for some time, the psychological burden of being monitored by the resident 11 can be reduced.

[0030] FIG. 5 is a block diagram of the life assistance system 1. As shown in FIG. The mobile object 2 is equipped with an image sensor 21, a distance sensor 22, and drive wheels 23, and detects the state of the occupant 11 using the image sensor 21 and the distance sensor 22 while moving on the drive wheels 23. The control board 24 in the moving body 2 includes a control unit 41, a communication unit 42, and a storage unit 43.

[0031] The control unit 41 uses information from the image sensor 21 and the distance sensor 22 to perform the following control. The drive wheels 23 are driven to move the moving body 2. The control unit 41 estimates the shape of the rooms in the house 10 and the position of the mobile object 2 itself within the house 10. Then, the control unit 41 stores the results of detecting the shape of the rooms in the house 10 in the memory unit 43 as map data 301 in FIG. 2. If it is not the first time the moving object 2 has moved into the room, the map data 301 learned up to that point is retrieved from the storage unit 43 and used to move the moving object 2 in accordance with the shape of the room. The feature quantity expressing the physical activity of the resident 11 and the daily living behavior of the resident 11, as explained in the radar chart 52 of FIG. 3, are calculated, and the results are transmitted to the server 3 via the communication unit 42.

[0032] The server 3 includes a control unit 31 , a communication unit 32 , and a storage unit 33 . The communication unit 32 can exchange information by communicating with the mobile object 2 or with an external device (e.g., information terminal 5) outside the life assistance system 1. The communication unit 32 receives information transmitted from the mobile object 2, and the control unit 31 adds time information to the information received via the communication unit 32 and stores the information in the memory unit 33. The control unit 31 retrieves the information stored in the memory unit 33, processes it together with the information received from the mobile body 2 via the communication unit 32, and outputs the processing result (e.g., time series data 51 in Figure 3) to the outside (e.g., information terminal 5) via the communication unit 32.

[0033] The mobile object 2 and the server 3 are each configured as a computer having a CPU, RAM, ROM, HDD, communication I / F, input / output I / F, and media I / F. The communication I / F is connected to an external communication device. The input / output I / F is connected to an input / output device. The media I / F reads and writes data from a recording medium. Furthermore, the CPU controls each processing unit by executing a program (also called an application or its abbreviation, "app") loaded into RAM. This program can also be distributed via a communication line or recorded on a recording medium such as a CD-ROM and distributed.

[0034] FIG. 6 is an example of a flowchart showing the processing of the moving object 2. The mobile object 2 checks whether the memory unit 43 contains map data 301 of the interior of the house 10 shown in Fig. 2 (S111). If the map data 301 does not exist, the control unit 41 creates map data 301 of the interior of the house while the mobile object 2 travels inside the house 10 (S113) and stores it in the memory unit 43. In the process of generating the map data 301, the shapes of the walls, furniture, etc. inside the house 10 are measured using the image sensor 21 and distance sensor 22. The moving object 2 travels freely within the house 10 using the map data 301 created in S113 or the map data 301 previously stored in the storage unit 43 (S112).

[0035] Next, the mobile object 2 determines whether or not it is in a state where it can start measurement, such as when it detects the resident 11 (S114). If the resident 11 is in a state that the resident 11 dislikes, such as when he or she is changing clothes, the mobile object 2 may receive a voice or gesture notification from the resident 11, whose physical activity feature 110 it is measuring, that measurement is not possible, and may stop measuring the physical activity feature 110 (No in S114), and return to S112. In addition, if multiple residents 11 (such as family members living with the elderly person or caregivers caring for the elderly person) are detected in S114, the detected resident 11 may be identified from a facial image or the like to determine whether or not the resident 11 is a person to be monitored, and measurement may be started if a person to be monitored is detected (Yes in S114).

[0036] In this way, when it becomes possible to start measurement (Yes in S114), the moving object 2 acquires the distance 221 between the resident 11 and the moving object 2 and the moving object 2's own position (S121). Here, the method of detecting the resident 11 may be to recognize the shape of a person using the image sensor 21, or to detect an object that does not fit the shape of the room using the distance sensor 22. If the distance 221 in S121 is shorter than the specified value, the moving body 2 is too close to the resident 11 (Yes in S122), so the control unit 41 drives the drive wheels 23 to move the moving body 2 so that the distance 221 becomes longer than the specified value (S126).

[0037] Here, the specified value in S126 is the maximum distance at which the image sensor 21 can acquire the feature amount of the occupant 11, which will be described later, and if the distance 221 is shorter than the specified value, the feature amount cannot be acquired. Since the specified value differs depending on the installation position of the image sensor 21, the angle of view that can be captured, and the feature amount to be extracted, it is necessary to determine the specified value in advance according to the conditions of the image sensor 21 and the feature amount. In Figure 4, a wide-angle image sensor 21 is installed on the moving body 2, and can acquire an image of the resident 11 in front of it. Also, since the physical activity feature 110 is calculated using the speed of the limbs as described below, a specified distance at which the image sensor 21 can capture an image of the entire body of the resident 11, including the tips of the limbs, is set and stored in advance in the memory unit 43. As a result, the mobile object 2 moves to a position where the entire body of the resident 11 to be measured is within the imaging range of the image sensor 21, and then captures an image of the resident 11 from that position.

[0038] When the moving object 2 moves to a position where the distance 221 is greater than the specified value (No in S122 or after execution of S126), the moving object 2 acquires the following information (S123). Image sensor 21 captures images of resident 11 The distance sensor 22 acquires the distance 221 between the resident 11 and the moving object 2. · Obtain the current position of moving object 2 using map data 301

[0039] Thereafter, the control unit 41 of the moving object 2 calculates the feature amount of the resident 11 as explained with reference to the radar chart 52 in FIG. 3 from the image, distance, and self-position acquired in S123 (S124). 7 is an explanatory diagram showing an example of the feature amount calculated in S 124. The control unit 41 calculates the feature amount 110 of the physical activity of the skeletal model or the like extracted from the captured image of the resident 11. The mobile object 2 transmits the feature calculated in S124 to the server 3 via the communication unit 42 (S125). By transmitting the feature extracted (compressed) from the image, the amount of data communicated between the mobile object 2 and the server 3 can be reduced compared to when the image itself acquired in S123 is directly transmitted to the server 3.

[0040] FIG. 8 is an example of a flowchart showing the process of the server 3 subsequent to the process of FIG. The server 3 receives the information on the feature amount of S125 from the mobile object 2 via the communication unit 32 (S201), and the control unit 31 adds time information to the received feature amount and stores it in the storage unit 33. The control unit 31 also compares the latest feature amount with past feature amounts already stored in the storage unit 33, and calculates the amount of change from the past feature amount (S202). In this way, since the server 3 performs the data storage process of S201 and the data comparison process of S202 instead of the mobile object 2, there is no need to provide high processing performance or large data storage capacity within the mobile object 2. Therefore, by minimizing the performance of the mobile object 2 compared to when all processing is performed by the mobile object 2, the mobile object 2 can be made smaller and less expensive.

[0041] If the amount of change calculated in S202 is greater than the specified value (Yes in S203), it is determined that the amount of change from the past is large, and alert information is sent to the external information terminal 5 via the communication unit 32 (S204). The specified value in S203 is set to a value that is a certain amount lower than the past feature amount, as shown in the following example. - When the walking speed, which is one of the currently acquired features, is more than 20% (= the specified value) lower than the walking speed measured several months ago. If the physical activity index calculated from walking speed and limb movement speed is lower than the specified value The alert information in S204 is, for example, an alert that the physical activity of the resident 11 has decreased, or an alert that something abnormal has occurred in the resident 11. [Example]

[0042] FIG. 9 is a plan view of the house 10. In the second embodiment, the mobile object 2 is fixed and not moved, and is used like a surveillance camera. The following description focuses on the differences from the first embodiment. In the second embodiment, an example in which the mobile object 2 is a robot vacuum cleaner will be described. The mobile object 2 moves autonomously when necessary, and when not moving, waits and charges at the location of the charging station 12 inside the house 10.

[0043] In the plan view of the house 10, the horizontal direction of the plane in FIG. 9 is the X axis, the vertical direction of the plane is the Y axis, and the height direction from the ceiling to the floor is the Z axis. The charging station 12 is installed at a distance equal to or greater than a specified value in a direction substantially perpendicular to the resident movement path 111 along which the resident 11 moves (Y-axis direction). Here, the resident movement trajectory 111 is the movement trajectory of an area in which the resident 11 moves in one direction (the X-axis direction in Figure 9), such as near an opening or closing door such as the front door of the house 10 or in a corridor within a room. In a large space, the resident 11 moves freely and not in a specific direction, so the direction of movement is not fixed, but in a corridor or an opening / closing door, the direction of movement is fixed to one direction: the direction of the corridor or the direction of passing through the opening / closing door. Therefore, it is assumed that the resident 11 moves along the resident movement trajectory 111.

[0044] This arrangement of the moving object 2 allows the radio waves 220 to reach the resident movement trajectory 111 and measure the entire body of the resident 11. Therefore, the moving object 2 waiting at the charging station 12 continues to transmit the radio waves 220 toward the resident movement trajectory 111 (in the Y-axis direction). The moving object 2 then measures physical activity data from the resident 11 using the images captured by the image sensor 21 mounted on the moving object 2 and the radio waves 220 emitted from the distance sensor 22.

[0045] The movement of the occupant 11 in the X-axis direction and the Z-axis direction along the occupant movement trajectory 111 is measured from the images captured by the image sensor 21. The movement of the resident 11 in a direction perpendicular to the resident movement trajectory 111 (Y-axis direction) is determined from the change in the distance between the moving object 2 and the resident 11 detected by the radio waves 220 of the distance sensor 22. The control board 24 can measure the movement speed of the occupant 11 with high accuracy by dividing the movement distance of the occupant 11 in each direction (X, Y, Z) by the movement time.

[0046] Fig. 10 is a plan view of the same house 10 as Fig. 9. The position of the charging station 12 is different between Fig. 9 and Fig. 10. The charging station 12 is installed on an extension line of the resident movement path 111 along which the resident 11 moves (in the X-axis direction) at a certain distance from the moving object 2. The movements of the occupant 11 in the direction perpendicular to the occupant movement trajectory 111 (Y-axis direction) and in the Z-axis direction are measured from the images captured by the image sensor 21. The movement of the resident 11 in the X-axis direction along the resident movement trajectory 111 is determined from the change in the distance between the moving object 2 and the resident 11 detected by the radio waves 220 of the distance sensor 22 . The control board 24 can measure the movement speed of the resident 11 with high accuracy by dividing the movement distance in each direction (X, Y, Z) of the resident 11 by the movement time. Also, because there is a certain distance between the moving object 2 and the resident movement trajectory 111, it is possible to measure the entire body of the resident 11 and measure physical activity without the moving object 2 moving away from the resident 11 to create a distance.

[0047] As explained above in Figures 9 and 10, the image sensor 21 determines the movement speed of the resident 11 perpendicular to the direction of irradiation of the radio waves 220 as a feature 110 of physical activity based on the change over time in the position of the resident 11 shown in the image captured of the resident 11. The distance sensor 22 uses radio waves 220 emitted from the mobile object 2 to determine the distance to the resident 11 and the moving speed of the resident 11 along the direction of the radio waves 220 as the feature amount 110 of physical activity.

[0048] FIG. 11 is an example of a flowchart for the moving object 2. If the moving object 2 is not moving autonomously, it is charged at the charging station 12 (S131). If the moving object 2, which is stationary at the charging station 12, receives a measurement instruction or detects that the resident 11 is moving along the resident movement trajectory 111, it determines that measurement should be started (Yes in S114B). In this case, the moving object 2 acquires an image of the resident 11 with the image sensor 21 and acquires the distance to the resident 11 with the distance sensor 22 (S123B). Note that since the moving object 2 is not moving, there are no errors such as blurring of the captured image, and the physical activity of the resident 11 can be measured more easily than in the first embodiment.

[0049] Here, the measurement instruction may be any method, such as the following examples. (Method 1) When it is estimated from the resident's daily life that the resident 11 moves along the resident movement trajectory 111, a measurement instruction is received from the server 3 at the time of movement. For example, when the charging station 12 is installed in a position where the hallway leading to the entrance is visible as shown in FIGS. 9 and 10, the movement time of the resident 11 is the time when the resident returns home or leaves the house 10. (Method 2) If it is expected that the resident 11 will soon return home to the house 10 based on location data from the smartphone he or she has, a measurement instruction is received from the server 3 at the expected time of return. (Method 3) Receive measurement instructions from a door opening / closing sensor or another sensor installed at the entrance.

[0050] Then, the control board 24 of the moving object 2 calculates the feature amount 110 of the physical activity of the resident 11 from the information acquired in S123B (S124), and transmits the information of the calculation result to the server 3 (S125). Meanwhile, since the position of the charging station 12 is known, the process of the moving object 2 acquiring its own position can be omitted in S123B, unlike S123 in Fig. 6 . As described above, in the second embodiment, the mobile object 2 measures the physical activity of the resident 11 without moving while the mobile object 2 is installed at the charging station 12. This makes it possible to measure the physical activity and health condition of the resident 11 in the same way as in the first embodiment, and also makes it easier to measure the physical activity. [Example]

[0051] FIG. 12 is a schematic diagram of a life assistance system 1 according to a third embodiment. In the third embodiment, a non-contact sensor 6 and a home appliance 7 that are installed in a house 10 and calculate a feature amount 110 of physical activity are added to the configuration of the first or second embodiment. Note that it is not necessary to have both the non-contact sensor 6 and the home appliance 7; either one of them is sufficient. The non-contact sensor 6 and the home appliance 7 can each be connected to the server 3.

[0052] The non-contact sensor 6 detects the position and posture of the resident 11 by emitting radio waves 60 such as millimeter waves or microwaves to the resident 11, and transmits the detection results to the server 3. The non-contact sensor 6 is installed, for example, on a wall or ceiling surface inside the house 10, and therefore can measure physical activity more precisely than when measured by the mobile object 2. The number of non-contact sensors 6 is not limited to one as illustrated in FIG. 12, and multiple non-contact sensors 6 may be provided inside the house 10. Instead of the non-contact sensor 6, the following sensor may be used. - A human presence sensor that uses infrared light to detect whether or not a person is present Door opening / closing sensor that detects the opening and closing of the door -Vibration sensor to detect vibrations

[0053] Furthermore, by installing the non-contact sensor 6 at a higher position on the wall of the house 10 or on the ceiling, it is possible to obtain physical information of the resident 11 without the view being obstructed by furniture or other installed objects. On the other hand, if the resident 11 is located in a blind spot from the installation position of the non-contact sensor 6, it may not be possible to obtain physical information of the resident 11. For this reason, by having the mobile object 2 obtain physical information of the resident 11 in a blind spot, it is possible to obtain more physical information of the resident 11 by both the mobile object 2 and the non-contact sensor 6.

[0054] The home appliance 7 can indirectly acquire physical activity of the resident 11 by transmitting operation information of the home appliance when the resident 11 uses (operates) the home appliance to the server 3. Note that although a washing machine is shown in FIG. 12, the type and number of home appliances are arbitrary. For example, instead of a washing machine, a home appliance 7 such as a refrigerator, a conventional vacuum cleaner held by the resident 11, a robot vacuum cleaner, or a cooking appliance may be used. Furthermore, the operation information of the home appliance 7 may include, for example, whether the door of the refrigerator is open or closed and the temperature inside the refrigerator, the washing mode and washing time of the washing machine, and the cooking menu and temperature inside the cooking appliance.

[0055] FIG. 13 is a schedule graph showing the daily activities of the resident 11. The vertical axis of the schedule graph 112 represents the day of the week (Monday to Sunday), and the horizontal axis represents the daily activities of the resident 11 from morning to night on that day. Daily activities are classified into, for example, sleeping, going out, resting, eating, choosing, etc., and the resident 11 performs one of the daily activities at each time. The server 3 estimates daily living activities at each time, such as the schedule graph 112, based on information acquired from at least one of the mobile object 2 and the home appliance 7. For example, the server 3 estimates the current daily living activity (eating) of the resident 11 from information acquired from the home appliance 7 (e.g., that the resident 11 is currently operating a cooking appliance).

[0056] From such estimated daily living activities at each time, the server 3 obtains what daily living activities are performed every day for each day of the week, and creates a schedule graph 112. The schedule graph 112 is transmitted from the server 3 to the information terminal 5 and is viewed by the resident family 5A, so that the resident family 5A can understand the lifestyle habits of the resident 11 without directly monitoring the resident 11.

[0057] The server 3 identifies the living activity as exemplified below. - "Button operation information" and "door opening / closing information" are acquired from the washing machine, and combined with hot water supply, bathing status of the resident11 is estimated. "Door opening / closing information" is acquired from the refrigerator, and "button operation information" and "door opening / closing information" are acquired from the cooking appliances, and by combining both, the meals of resident 11 are estimated. Obtain "operation" and "movement information" from the vacuum cleaner to estimate the cleaning status of the resident 11.

[0058] FIG. 14 is an example of a block diagram of the life assistance system 1. As shown in FIG. The life assistance system 1 of the third embodiment includes a moving object 2, a server 3, a non-contact sensor 6, and a home appliance 7. The configurations of the moving object 2 and the server 3 are the same as those of the first embodiment (FIG. 5).

[0059] The non-contact sensor 6 includes a detection unit 61 , a control unit 62 , and a communication unit 63 . The detection unit 61 acquires physical information of the resident 11 by irradiating the resident 11 with radio waves 60. Alternatively, the detection unit 61 acquires physical information of the resident 11 by irradiating the resident 11 with light (laser) from LiDAR (Light Detection And Ranging) instead of the radio waves 60 and performing a 3D scan. The physical information of the resident 11 is, for example, activity data of the resident obtained from the position and posture of the resident as point cloud data (distance data) and the time change of the point cloud data (speed of the point cloud, etc.). The control unit 62 extracts features indicating the physical activity of the resident 11 from the results acquired by the detection unit 61, and transmits the extracted feature results to the server 3 via the communication unit 63. The features include the position, movement speed, posture, and movement speed of the limbs of the resident, as well as various numerical values ​​of the physical activity calculated from the information, similar to the information acquired by the moving object 2 in the first embodiment.

[0060] The home appliance 7 includes a detection unit 71, a control unit 72, and a communication unit 73. The detection unit 71 acquires resident information 70 such as operation information when the resident 11 operates the home appliance and environmental information of the house 10. The environmental information of the house 10 is, for example, information surrounding the home appliance 7, such as the temperature, humidity, and air pressure around and inside the home appliance 7, the water pressure of the tap water, and the voltage of the grid power. Similar to the control unit 62, the control unit 72 extracts features indicating the physical activity of the resident 11 from the results obtained by the detection unit 71 and transmits the extracted feature results to the server 3 via the communication unit 73. The features transmitted here include operation information of the home appliances, and therefore, information such as when and what home appliance was operated, when which home appliance was operated, under what conditions, and in what mode can be obtained.

[0061] FIG. 15 is an example of a flowchart of the life assistance system 1 (the moving object 2, the non-contact sensor 6, and the home appliance 7) according to the third embodiment. When the control is started, the mobile object 2 moves freely within the house (S112) in the same manner as in the first embodiment (FIG. 6). Meanwhile, the non-contact sensor 6 and the home appliance 7 enter a standby state to wait for an operation (S141).

[0062] When the non-contact sensor 6 detects the resident 11, or when the resident 11 operates the home appliance 7 (Yes in S142), the non-contact sensor 6 acquires information about the resident 11, and the home appliance 7 acquires operation information (S143). If the non-contact sensor 6 detects the resident 11, the position of the resident 11 is known, and if the home appliance 7 is operated, the position of the resident 11 is known from the position of the home appliance 7, and therefore, detection information of the resident 11 is transmitted to the mobile object 2 based on that position (S143B). In the transmission process of S143B, the information may be transmitted directly from the non-contact sensor 6 or the home appliance 7 to the mobile object 2, or the information may be transmitted via the server 3.

[0063] The moving object 2 travels to the location of the resident 11 notified in S143B (S112B) and starts measuring the physical information of the resident 11 (Yes in S114). The subsequent processes (S123 to S125) executed by the moving object 2 are the same as those in the first embodiment (FIG. 6). Furthermore, the non-contact sensor 6 and the home appliance 7 calculate the necessary features from the physical information of the resident 11 acquired in S143 (S144, similar to S124 in Figure 6), and transmit the calculation results to the server 3 (S145, similar to S125 in Figure 6).

[0064] FIG. 16 is an example of a flowchart of the server 3 following the process of FIG. The server 3 receives the information notified in S125 from the moving object 2 (S201), and receives the information notified in S145 from the non-contact sensor 6 and the home appliance 7 (S201B). Examples of the received information are shown below. Resident 11's presence information - Resident 11 activity data (mainly received from non-contact sensor 6) Resident 11's posture The position of the resident 11 (center of gravity, position of hands and feet). In the case of the home appliance 7, whether or not the resident 11 is present at the location of the home appliance. Resident 11 speed (walking speed, limb movement speed, center of gravity movement speed) - Value calculated from the collection of information listed above Resident behavior detection (where they are, what they're doing)

[0065] The server 3 synthesizes (combines) the information of S201 and the information of S201B to calculate the feature amount of the resident 11 as exemplified below (S201C). Energy consumption of Resident 11 (= walking speed + limb movement speed) - Resident 11's behavioral sharpness (obtained by frequency analysis of walking speed) - Daily activities of the resident 11 (eating, resting, etc.). Furthermore, if the daily activities of the resident 11 are specific, by combining image information from the mobile object 2, it is possible to measure the physical activity of the resident 11 during a certain daily activity that would be difficult to obtain using only the mobile object 2. For example, it is possible to measure the posture of the resident 11 while cooking, or the walking speed of the resident 11 when going back and forth between the bedroom and the toilet at night. - Resident 11's activity levels and movement speeds for each lifestyle - Activity levels and movement speeds for each location of the resident (entrance, kitchen, living room, bedroom, hallway, and toilet)

[0066] When combining the information from S201 and the information from S201B, if the information does not overlap, one of the pieces of information is adopted, and if the information overlaps, the more accurate piece of information is adopted. For example, if location information for a resident 11 at the same time is received from both the mobile object 2 and the non-contact sensor 6, the information source closest to the resident 11 should be adopted.

[0067] The server 3 saves the current information synthesized in S201C and calculates the amount of change in the feature of the resident 11 by comparing the current information with past information (S202). In the comparison process of S202, the accuracy of the comparison is improved by comparing physical activities of the same daily activities (for example, physical activity during current laundry and physical activity during past laundry). Examples of the change or amount of change compared in S202 are as follows, and health information of the resident 11 can be obtained with higher precision. Changes in walking speed and posture Changes in walking speed, walking posture, and physical activity index in specific daily activities Changes in daily activities at the same time on the current day and past days. For example, if a person had a meal at 7:00 AM on a past day, but was sleeping at 7:00 AM on the current day, the difference between eating and sleeping will be recognized as an abnormality (poor health) for Resident 11.

[0068] The server 3 determines whether the amount of change calculated in S202 is greater than a specified value (predetermined threshold) (S203). Alternatively, the server 3 may determine in S203 whether a change was detected in S202. If S203 is Yes (amount of change > predetermined threshold, or there is a change), the server 3 notifies the resident family 5A of a possible abnormality in the resident 11 by issuing an alert to the information terminal 5 (S204). The alert is information indicating a change, such as, for example, that a certain behavior is usually performed at this time of day on this day, but that behavior was not detected today.

[0069] Furthermore, the server 3 may instruct the mobile object 2 to check the location of the abnormality, such as where the resident 11 has fallen, and the mobile object 2 may notify the resident 11 with a voice message such as "Are you OK?". This allows the mobile object 2 to receive a detailed voice response from the resident 11, such as "I think I sprained my ankle," and notify the resident's family 5A of this as a more detailed alert.

[0070] The life support system 1 of Examples 1 to 3 described above includes an image sensor 21 that acquires images, a distance sensor 22 that acquires the distance to an object, a mobile body 2 that estimates its own position indoors, and a server 3 that can communicate with the mobile body 2. The mobile object 2 then processes the physical activity of the resident 11 acquired by the mounted sensor to obtain physical activity feature quantities 110, which are then transmitted to the server 3, and the server 3 stores the physical activity feature quantities 110.

[0071] Furthermore, the server 3 detects the feature 110 of the physical activity of the resident 11 to be measured, and if there is a large change from the previously saved feature 110 of the physical activity, it notifies the information terminal 5 of this fact as an alert. This alert can improve accuracy by being the result of comparing the feature 110 of the same daily activity between the past and present. This allows the physical activity and health status of the resident 11 in the house 10 to be measured, and if any changes occur in that status, the resident 11 and those involved can be notified of the change, thereby encouraging appropriate intervention and maintaining and improving the health status of the resident 11.

[0072] It should be noted that the present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, remove, or replace part of the configuration of each embodiment with other configurations. Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partly or entirely realized in hardware, for example, by designing them as integrated circuits. Furthermore, each of the above-mentioned configurations and functions may be realized by software, with a processor interpreting and executing a program that realizes each function.

[0073] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a storage medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).The cloud can also be used. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. Furthermore, the communication means connecting the devices is not limited to wireless LAN, but may be changed to wired LAN or other communication means. [Explanation of symbols]

[0074] 1. Life support system 2. Mobile 3 Server 5. Information terminals 5A Resident family 6. Non-contact sensors 7 Home appliances 10. Housing 11 Resident 12 Charging Stations 21 Image sensor (measurement sensor) 22 Distance sensor (measurement sensor) 23 Drive wheels 24 Control board 31 Control Unit 32 Communications Department 33 Storage section 41 Control Unit 42 Communications Department 43 Storage section 51 Time Series Data 52 Radar Chart 60 Radio Waves 61 Detection unit 62 Control Unit 63 Communications Department 70 Resident information 71 Detection unit 72 Control Unit 73 Communications Department

Claims

1. a mobile object that moves to the vicinity of a resident, calculates a feature of the physical activity of the resident and the living behavior of the resident from a measurement result including location information of the resident obtained from a measurement sensor provided in the mobile object, and transmits the calculation result to a server; the server compares, for each received living behavior of the resident, past physical activities of the same living behavior with the current physical activity to determine a change from a feature amount of the past physical activity to a feature amount of the current physical activity, and transmits an alert to an information terminal in response to the change; The moving body is a distance sensor that uses electromagnetic waves or sound waves emitted from the moving object to determine a distance to the resident and a moving speed of the resident along the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity; and an image sensor that determines, as a feature of the physical activity, a moving speed of the resident perpendicular to the direction of irradiation of the electromagnetic waves or sound waves, based on a time change in the position of the resident shown in an image of the resident. Assisted living system.

2. The moving body moves to a position where the whole body of the resident to be measured is within the imaging range of the image sensor, and then the resident is photographed from that position. The life assistance system according to claim 1 .

3. When the mobile object receives a notification by voice or gesture from the resident whose physical activity feature is being measured that measurement is not possible, the mobile object stops measuring the physical activity feature. The life assistance system according to claim 1 .

4. The mobile object calculates, as a feature of the physical activity of the resident, at least one of a set of the presence information of the resident, the position of the resident, the posture of the resident, the movement speed of the center of gravity of the resident, the movement speed of the limbs of the resident, and the activity amount of the resident, and a value calculated from the set. The life assistance system according to claim 1 .

5. The information terminal is a terminal used by at least one of the resident, the resident's family, the resident's care manager, and a person involved in a nursing care facility contracted with the resident, and is characterized in that it receives and displays, in addition to an alert, the characteristic amount of the physical activity of the resident. The life assistance system according to claim 1 .

6. The mobile object further has at least one function selected from the group consisting of a cleaning function, an air cleaning function, a petting function, a security function, a sound recognition function, and a monitoring function. The life assistance system according to claim 1 .

7. The server receives the feature amount of the physical activity calculated from at least one of a non-contact sensor installed in the home and a home appliance installed in the home. The life assistance system according to claim 1 .

8. The life assistance system includes a mobile object and a server, The moving body is a distance sensor that uses electromagnetic waves or sound waves emitted from the moving object to determine a distance to the resident and a moving speed of the resident along the direction of irradiation of the electromagnetic waves or sound waves as a feature of physical activity; an image sensor that determines a moving speed of the resident perpendicular to the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity based on a time change in the position of the resident shown in an image of the resident; and The mobile object moves to the vicinity of the resident, calculates a feature of the physical activity of the resident and a living behavior of the resident from a measurement result including location information of the resident obtained from the measurement sensor provided in the mobile object, and transmits the calculation result to the server; The server compares the past physical activity of the resident with the current physical activity of the same living activity for each received living activity of the resident, to determine a change from a feature amount of the past physical activity to a feature amount of the current physical activity, and transmits an alert to an information terminal in response to the change. Life support methods.

Images