Life Support System

Abstract

To provide, as one of watching services, a life support system that is concerned with a resident so that the resident can return to a healthy state according to measures against a frail resident and a behavior of a resident in a frail state.SOLUTION: A life support system of the present invention comprises: a movable body 2 that autonomously moves inside a home; and a behavior support device 1 that controls the movable body according to information on a resident's behavior or information on an environment inside the home recognized on the basis of, sensor information acquired by a sensor 21 included in the movable body or sensors 3 inside the home. Especially, the behavior support device includes: a behavior recognition unit that recognizes information on the resident's current behavior as current behavior information on the basis of the sensor information, and sequentially stores the recognized behavior information as time-series behavior information; and a movable body instruction unit that analyzes that the current behavior information has changed with respect to the time-series behavior information, and when an amount of change of the behavior information is equal to or more than a threshold, instructs the movable body to perform an operation to urge the resident to change the behavior.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a life assistance system. [Background technology]

[0002] Elderly people move from a healthy state to a state requiring nursing care due to aging. In modern medicine, the intermediate state between healthy and nursing care is called "frailty," and it is said that by providing appropriate interventions to frail elderly people, they can return to a healthy state.

[0003] Currently, various sensor devices are installed in homes as part of home security and monitoring services, and can detect the behavior of people in the home. For example, Patent Document 1 describes that the devices function as a person estimation unit 730 that estimates a person captured in video data 610 (see FIG. 3), a behavior and item extraction unit 740 that extracts the behavior of the estimated person 320 (see FIG. 2) and items related to the behavior from the video data 610, and a behavior data generation unit 750 that generates behavior data 620 (see FIG. 1) based on the extracted information (see paragraph 0028), and discloses that "behavior data" that associates "people" with "people's behavior" is generated. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-185184 Summary of the Invention [Problem to be solved by the invention]

[0005] The above prior art can detect the behavior of people in the home, but it merely discloses providing information about the behavior to contacts linked to the "behavioral data," and cannot be expected to restore a frail state to a healthy state.

[0006] The object of the present invention is to provide a life support system that, as one of the monitoring services, engages with residents to promote the health of residents who are not in a frail state and to help frail residents recover to a healthy state based on their behavior. [Means for solving the problem]

[0007] In order to solve the above problems, the life support system of the present invention is a and has a sensor A moving object and By detecting behavioral information of the resident and environmental information of the home based on sensor information acquired from sensors in the home, sensors possessed by the mobile object, and sensors in the home, a behavior support device for controlling the moving object; a behavior recognition unit that recognizes current behavior information of a resident as current behavior information based on sensor information acquired from sensors of the mobile body and sensors in the home, and sequentially stores the recognized behavior information as time-series behavior information; and a mobile body instruction unit that analyzes the time-series behavior information to determine that the current behavior information has changed, and instructs the mobile body to perform an action that encourages a behavior change of the resident when an amount of change in the behavior information is equal to or greater than a threshold. It was designed to be equipped with the following. [Effects of the Invention]

[0008] According to the present invention, it is possible to contribute to measures against frailty in residents and to help frail residents recover to a healthy state, and to provide a useful monitoring service for residents. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a configuration diagram of a life assistance system according to an embodiment. [Figure 2] 1 is a diagram illustrating a configuration of hardware on which an action support device of a life support system according to an embodiment operates. [Figure 3] FIG. 2 is a flowchart of processing of the first embodiment periodically performed by the action support device. [Figure 4] FIG. 10 is a flowchart of processing according to a second embodiment that is periodically performed by the action support device. [Figure 5] FIG. 11 is a flowchart of processing of a third embodiment periodically performed by the action support device. [Figure 6] FIG. 10 is a flowchart of processing according to a fourth embodiment that is periodically performed by the action support device. [Figure 7] FIG. 11 is a flowchart of processing according to a fifth embodiment, which is periodically performed by the action support device. [Figure 8] FIG. 20 is a flowchart of processing according to a sixth embodiment that is periodically performed by the action support device. [Figure 9] FIG. 13 is a flowchart of processing according to a seventh embodiment, which is periodically performed by the action support device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a configuration diagram of a life assistance system and an action assistance device according to an embodiment.

[0011] The life support system is composed of a mobile object 2 that moves around the house, such as a robot vacuum cleaner, a robot air purifier, or a pet robot, multiple home sensors 3 such as cameras and motion sensors that detect environmental information such as temperature, humidity, and airflow rate in the house and detect the behavior of the resident, home appliances such as a washing machine 4, a refrigerator 5, and an air conditioner 6, and an action support device 1, all connected via an in-house LAN 7. The action support device 1 also communicates via the Internet 8 with a resident terminal 9a such as the resident's smartphone and a watcher terminal 9b of the watcher, such as a family member or caregiver.

[0012] Details will be described later, but the behavioral support device 1 controls the mobile object 2 according to the behavioral information of the resident or the environmental information within the home recognized based on sensor information acquired by a sensor 21 possessed by the mobile object 2 that moves autonomously within the home or an in-home sensor 3 within the home.

[0013] Next, the configuration of the moving object 2 and the action support device 1 will be described with reference to FIG. The mobile object 2 is configured to be able to move autonomously using map information within the house and a self-position detection means, and moves to a specified location based on instructions from the action support device 1, and further detects the resident using the installed sensor 21. The mobile object 2 can also come close to the resident and communicate with them.

[0014] More specifically, the mobile object 2 is equipped with sensors 21 such as a camera, a floor surface distance measuring sensor, a distance measuring sensor, a millimeter wave radar, and a thermograph, and a control unit 23 controls a driving means 22 such as a motor based on map information 25 to autonomously travel within the house. During this travel, the mobile object 2 avoids surrounding obstacles detected by the sensors 21.

[0015] The sensor 21 of the mobile object 2 not only detects obstacles in the surrounding area, but also enables the identification of the resident, tracking the resident's movements, turning toward the resident, detecting the resident's behavior, etc., based on the camera images of the sensor 21. Furthermore, a sound sensor, a temperature sensor, a humidity sensor, an illuminance sensor, a microphone, etc. may be installed as the sensor 21 to detect environmental information within the home.

[0016] The mobile object 2 is also equipped with a housing heater 24 that raises the surface temperature of the housing, allowing it to adjust the temperature when the resident comes into contact with it and notify the resident that their body temperature is high. It may also be equipped with a speaker or light-emitting unit (not shown) that outputs a message and serves as a human interface with the resident.

[0017] The mobile object 2 notifies the behavioral support device 1 of sensor information such as camera images and temperature distribution images detected by the sensor 21 in order to detect resident behavioral information and environmental information, and moves to a specified location within the house in response to movement instructions from the behavioral support device 1.

[0018] The behavior support device 1 is composed of a behavior recognition unit 11 that recognizes the behavior of the resident and obtains behavior information, a time-series behavior information memory unit 12 that stores the behavior information of the resident recognized by the behavior recognition unit 11 in chronological order, a mobile body instruction unit 13 that analyzes the state of the resident from the current behavior information of the resident recognized by the behavior recognition unit 11 and the time-series behavior information in the time-series behavior information memory unit 12 and instructs the mobile body 2 to perform an action or support action that encourages the resident to change their behavior, and a monitoring information notification unit 14 that notifies the terminal of the person watching over the resident of information related to the resident's behavior information.

[0019] In detail, the person identification unit 111 of the behavior recognition unit 11 identifies the resident (obtains person identification information) by recognizing the face image of the resident captured by the sensor 21 (camera) of the moving object 2. The identification of the resident is not limited to face authentication, and may be performed by biometric authentication such as iris authentication using the face image.

[0020] The human behavior detection unit 112 of the behavior recognition unit 11 estimates the behavior of each resident identified by the person identification unit 111 based on the sensor information from the sensors 21 of the mobile object 2 and the sensor information from the in-home sensors 3, and acquires behavior information for the specific resident. In detail, the behavior information is obtained by recognizing human movements from changes in the position of the resident's body parts, such as the head, hands, arms, legs, and fingers, as well as changes in the direction and posture of the body, and estimating actions such as "getting up," "lying down," and "sitting" from combinations of the recognized human movements. Then, the human behavior detection unit 112 identifies the bedroom, kitchen, and day / night based on the acquisition location and acquisition time of the sensor information, and estimates actions such as "getting up," "going to bed," "sitting at the dinner table," and "relaxing," and uses these as behavior information.

[0021] Furthermore, the human behavior detection unit 112 acquires environmental information within the home based on the sensor information from the sensor 21 of the mobile object 2 and the sensor information from the in-home sensor 3, and makes it possible to refer to the environmental information when analyzing the behavior of residents. More specifically, the operating states of the washing machine 4 and the air conditioner 6 are acquired as environmental information.

[0022] Furthermore, the human behavior detection unit 112 acquires the amount of activity, walking speed, moving speed, standing speed, and fall state of the resident from the recognized human movements, and sets these as behavior information.

[0023] The time-series behavior information storage unit 12 is a storage unit that stores, in time series, the behavior information of the resident recognized by the behavior recognition unit 11 for each resident (person identification information) identified by the person identification unit 111. The behavior recognition unit 11 periodically obtains the behavior information of the resident, and the recognition time and the behavior information are sequentially recorded in the time-series behavior information storage unit 12. In addition, when the behavior information obtained by the behavior recognition unit 11 changes, the recognition time and the behavior information may be recorded in the time-series behavior information storage unit 12.

[0024] In the following description, the behavior information in the time-series behavior information storage unit 12 is referred to as time-series behavior information, and the most recent behavior information recognized by the behavior recognition unit 11 is referred to as current behavior information.

[0025] The behavior estimation unit 131 of the mobile object instruction unit 13 compares the time-series behavior information and current behavior information for each resident to analyze the state of the resident. The behavior estimation unit 131 also analyzes the state of the resident by adding environmental information acquired based on the sensor information of the sensor 21 of the mobile object 2 and the sensor information of the in-house sensor 3.

[0026] The instruction information generating unit 132 generates instruction information for an action or a support action that prompts the resident of the moving object 2 to change their behavior, according to the analysis result of the behavior estimating unit 131.

[0027] The instruction information notification unit 133 notifies the moving object 2 of the instruction information generated by the instruction information generation unit 132 via the home LAN 7 .

[0028] The monitoring information notification unit 14 notifies the resident terminal 9a or the watcher terminal 9b via the Internet 8 of information related to the analysis results of the behavior estimation unit 131 or information about the resident obtained by instructing the mobile body 2.

[0029] The detailed operation of the moving object instruction unit 13 will be explained later with reference to a flow chart.

[0030] FIG. 2 is a diagram showing the configuration of hardware on which the action support device 1 of FIG. 1 operates. The action support apparatus 1 is configured as a computer 900 having a CPU 901 , a RAM 902 , a ROM 903 , a HDD 904 , a communication I / F 905 , an input / output I / F 906 , and a media I / F 907 . The communication I / F 905 is connected to an external communication device 915. The input / output I / F 906 is connected to an input / output device 916. The media I / F 907 reads and writes data from a recording medium 917. Furthermore, the CPU 901 controls each processing unit by executing a program (also called an application or an app for short) loaded into the RAM 902. This program can also be distributed via a communication line or recorded on a recording medium 917 such as a CD-ROM and distributed.

[0031] In the action support device 1, the CPU 901 executes a program to realize the functions of the action recognition unit 11, the moving object instruction unit 13, and the watching information notification unit 14, and the time-series action information storage unit 12 is configured in the HDD 904.

[0032] Hereinafter, the operation of the action support device 1 of the embodiment will be described in detail.

[0033] Example 1 The time-series behavior information storage unit 12 stores behavior information such as activity amount, walking speed, moving speed, standing up speed, and falls as time-series behavior information for each resident. Therefore, by comparing the time-series behavior information with the current behavior information, the behavior estimation unit 131 can analyze whether there is a lack of exercise or an increase in exercise, such as whether the amount of time has increased or decreased by XX% compared to XX months ago, or whether the number of times has increased or decreased by XX% compared to XX months ago.

[0034] In this embodiment, the action support device 1 determines that the resident is experiencing insufficient exercise or increased exercise if the amount of change in the action information is equal to or greater than a threshold. Depending on the analysis result of the resident's insufficient exercise or increased exercise, the action support device 1 generates instruction information for the mobile object 2 to say to the resident, "You should do some exercise," as an action to encourage the resident to change their behavior to resolve the lack of exercise, and notifies the mobile object 2 of the instruction information. Alternatively, the action support device 1 may notify the mobile object 2 of the instruction information so that the mobile object 2 itself performs an action related to exercise to encourage the resident to change their behavior.

[0035] At this time, the action support device 1 notifies the resident terminal 9a or the watcher terminal 9b of the change in the resident's action information and the need for action modification as monitoring information. Furthermore, the action support device 1 may notify the resident terminal 9a or the watcher terminal 9b of the result of the action modification caused by the movement of the moving object 2 as monitoring information.

[0036] FIG. 3 is a flowchart of the processing periodically performed by the action support device 1 of this embodiment.

[0037] In step S31, the behavior recognition unit 11 acquires sensor information from the moving object 2 and the in-home sensor 3.

[0038] In step S32, the person identification unit 111 recognizes the face image in the sensor information to obtain person identification information of the resident, and identifies the resident.

[0039] In step S33, the human behavior detection unit 112 estimates the behavior of each resident identified by the person identification information based on the sensor information of the moving object 2 and the in-home sensor 3, and acquires behavior information (current behavior information).

[0040] In step S34, the time-series behavior information storage unit 12 accumulates the behavior information acquired in step S33 as time-series behavior information.

[0041] In step S35, the behavior estimation unit 131 compares the time-series behavior information and the current behavior information for each resident, and analyzes the amount of change in the behavior information of the resident.

[0042] In step S36, the instruction information generation unit 132 determines whether the amount of change in the behavior information between the time-series behavior information and the current behavior information is equal to or greater than a threshold, and if it is equal to or greater than the threshold (Yes in S36), the analysis result indicates that the resident is either not exercising enough or is exercising more, and the process proceeds to step S37. If the amount of change in the behavior information is smaller than the threshold (No in S36), the process ends.

[0043] In step S37, the instruction information generation unit 132 sets, in the instruction information for the moving body 2, a verbal action (for example, saying "You should do some exercise") that encourages behavioral changes to resolve the lack of exercise, depending on the analysis results of the resident's lack of exercise or increase in exercise.

[0044] In step S38, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in step S37. At this time, the moving object 2 may be instructed to move to the location of the resident so as to move to the vicinity of the resident.

[0045] In step S39, the watching information notifying unit 14 notifies the resident terminal 9a or the watcher terminal 9b of the amount of change in the resident's behavioral information and the fact that behavioral modification is necessary as watching information, and ends the process.

[0046] According to the processing of the action support device 1, the life support system encourages the resident to maintain a certain amount of exercise, thereby helping the resident to maintain his or her health.

[0047] Example 2 The time-series behavior information storage unit 12 stores time-series behavior information for each resident, so it is possible to recognize the resident's daily rhythm (for example, every Sunday, wake up → turn on the microwave in the kitchen → sit at the dining table in the living room and have breakfast). Furthermore, signs of frailty can be detected by analyzing a decrease in the resident's walking speed or physical activity level from the time-series behavior information and current behavior information.

[0048] In this embodiment, when the action support device 1 recognizes the estimated action of the resident as movement, it moves the mobile object 2 ahead of the resident to the destination (for example, the dining table) so as to provide support to the resident. In particular, when it detects signs of frailty in the resident, it instructs the mobile object 2 to take an image of the resident including the mouth after moving so that the condition can be confirmed. In addition to the mouth, body movements that are manifested by signs of frailty may also be captured.

[0049] Furthermore, the behavior support device 1 confirms that the resident has arrived at the destination that the mobile body 2 has traveled to in advance, and if the resident has not arrived within a predetermined time, instructs the mobile body 2 to take a predetermined action (such as notifying the resident of non-arrival).

[0050] The arrival of a resident is detected by the in-house sensor 3 (human sensor) or by image recognition by the sensor 21 (camera) of the mobile object 2.

[0051] When the mobile object 2 detects the arrival of a resident, it may further perform a call operation to confirm with the resident whether they have been out or in another room.

[0052] Furthermore, the action support device 1 notifies the resident terminal 9a or the watcher terminal 9b of the confirmation result of the resident's movement and the captured image of the resident as watching information.

[0053] FIG. 4 is a flowchart of the process periodically performed by the action support device 1 of this embodiment. Steps S31 to S35 in FIG. 4 are the same as those in FIG. 3, and therefore will not be described here.

[0054] In step S41, the instruction information generation unit 132 analyzes the time-series behavior information and the current behavior information to determine whether the resident's future estimated behavior will involve movement based on their lifestyle rhythm, and if it does not involve movement (No in S41), the processing ends.

[0055] In step S42, the instruction information generating unit 132 analyzes the decrease in the walking speed and the decrease in the amount of physical activity of the resident in the time-series behavior information and the current behavior information to determine whether the resident has signs of frailty.

[0056] If there is a sign of frailty (Yes in S42), in step S43, the instruction information generating unit 132 sets the photographing operation including the mouth of the resident after movement in the instruction information of the moving body.

[0057] In step S44, the instruction information generation unit 132 estimates the destination position from the time-series behavior information and the current behavior information. For example, in the life rhythm recognized by the time-series behavior information, if it is estimated from the current behavior information (microwave on in the kitchen) that the user will "sit at the dining table in the living room and have breakfast" in the future, the instruction information generation unit 132 estimates the living room as the destination.

[0058] In step S45, the instruction information generating unit 132 sets the movement action to the estimated destination (living room) as instruction information for the moving object 2. This instruction information for the moving object 2 is notified to the moving object 2 in step S47, which will be described later.

[0059] In step S46, the instruction information generation unit 132 confirms that the resident has arrived at the destination that the mobile body 2 has advanced to, and sets in the instruction information of the mobile body 2 a confirmation operation for the resident's movement that will perform a predetermined operation (such as notifying the resident of non-arrival) if the resident has not arrived after a predetermined time has passed.

[0060] In step S47, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in steps S43, S45, and S46.

[0061] In step S48, the watching information notifying unit 14 notifies the resident terminal 9a or the watcher terminal 9b of the confirmation result of the resident's movement and the captured image of the resident as watching information, and ends the process.

[0062] With the above-described activity support device 1, the life support system can anticipate the resident's estimated destination and understand the resident's condition by complementing the in-home sensor 3. In particular, by capturing images of the mouth of a resident who shows signs of frailty, it is possible to monitor whether the resident is swallowing food. In addition, by estimating the number of chewing movements and chewing speed from the captured images, it is possible to determine whether the resident's muscles are weakening.

[0063] In the above description, the action support device 1 instructs the moving object 2 to check the resident's movement ahead of time and take a photograph of the resident, including the resident's mouth, but the action support device 1 may also detect the indoor temperature at the moving object's destination using the sensor 21 (temperature sensor, thermography) of the moving object 2 and control the air conditioner installed at the moving object. The moving object 2 may also perform an action to prompt the moved resident to operate the air conditioner.

[0064] Example 3 The time-series behavior information storage unit 12 stores time-series behavior information for each resident, so changes in the resident's physical condition (for example, fever, decreased frequency of eating, not waking up) can be recognized from the current behavior information.

[0065] In this embodiment, when the action support device 1 determines that the resident is in poor health, it instructs the mobile object 2 to measure the resident's condition using a sensor 21 (e.g., thermography, camera) provided in the mobile object 2. The action support apparatus 1 also instructs the moving object 2 to change the set temperature of the housing heater 24 of the moving object 2 in accordance with the measured body temperature of the resident.

[0066] Furthermore, when the action support device 1 determines that the resident is in poor physical condition, it notifies the resident terminal 9a or the watcher terminal 9b of the resident's physical condition information as watching information.

[0067] FIG. 5 is a flowchart of the processing periodically performed by the action support device 1 of this embodiment. Steps S31 to S35 in FIG. 5 are the same as those in FIG. 3, and therefore will not be described here.

[0068] In step S51, the instruction information generation unit 132 determines whether the resident is in poor health by analyzing the time-series behavior information and the current behavior information, and if the resident is not in poor health (No in S51), ends the processing.

[0069] If the resident is in poor health (Yes in S51), in step S52 the instruction information generator 132 sets in the instruction information an action of the mobile object 2 to measure the resident's condition using the sensor 21 (thermography, camera) mounted on the mobile object 2. This enables the action support apparatus 1 to know the resident's body temperature and observe their condition.

[0070] In step S53, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in step S52.

[0071] In step S54, when the moving object 2 notifies the resident of the body temperature, the instruction information generating unit 132 sets, in the moving object's instruction information, an operation to change the set temperature of the housing heater 24 of the moving object 2 in accordance with the resident's body temperature. For example, if the resident's body temperature is higher than normal, the surface temperature of the moving object 2 is increased.

[0072] In step S55, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in step S54.

[0073] In step S56, the watching information notification unit 14 notifies the resident terminal 9a or the watcher terminal 9b of the resident's physical condition information as watching information.

[0074] By using the above-described action support device 1, the life support system can make the resident aware of poor health (fever) without forcing the resident to take action.

[0075] Example 4 4, the mobile object 2 advances to the destination to provide lifestyle support by photographing the resident and confirming their movements, but environmental information about the destination may also be acquired. Specifically, when the estimated meal preparation behavior of the resident is analyzed from the current behavior information based on the life rhythm recognized from the time-series behavior information, the mobile object 2 advances to the kitchen at meal preparation time and photographs the opening and closing of the refrigerator door or the interior of the refrigerator, which is installed in the kitchen and whose location information is recorded in the map information 25, to provide lifestyle support such as food ingredient management and automatic ordering.

[0076] The location information of a refrigerator installed in the kitchen is acquired by using a sensor 21 (camera) to capture an image of a QR code attached to the refrigerator, which indicates the refrigerator's identification information. Specifically, the captured image of the QR code is analyzed to recognize the refrigerator's identification information, and the location information of the refrigerator is calculated from the distance information between the refrigerator and the moving object 2 detected by the sensor 21 (camera or distance sensor) and the location information of the moving object 2. The refrigerator whose location information has been identified is recorded in map information 25 of the moving object 2. The installation locations within the home can be acquired in the same way for not only refrigerators but also televisions, air conditioners, microwaves, etc. Note that the resident may input the installation location of each home appliance into the map information 25 via the resident terminal 9a without using a QR code.

[0077] FIG. 6 is a flowchart of the processing periodically performed by the action support device 1 of this embodiment. Steps S31 to S35 in FIG. 6 are the same as those in FIG. 3, and therefore will not be described here.

[0078] In step S61, the instruction information generation unit 132 analyzes the time-series behavior information and the current behavior information to determine whether the resident's future estimated behavior based on the resident's lifestyle rhythm is a move to the kitchen, and if it is not a move to the kitchen (No in S61), the processing ends.

[0079] If the moving object 2 is moving to the kitchen (Yes in S61), then in step S62, the instruction information generating unit 132 sets the moving action to the kitchen as the instruction information for the moving object 2.

[0080] In step S63, the instruction information generating unit 132 sets, as the instruction information for the moving object 2, the action of capturing an image of the opening and closing of the door of a refrigerator installed in the kitchen, or the action of capturing an image of the interior of the refrigerator.

[0081] In step S64, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in steps S62 and S63.

[0082] The above-described behavior support device 1 enables the life support system to provide life support to residents by managing ingredients and automatically ordering them.

[0083] Example 5 In the above-mentioned behavior support device 1, an example was described in which instruction information is notified to a mobile body 2 based on behavior information of a resident. In this embodiment, however, a case is described in which instruction information is notified to a mobile body 2 based on behavior information of a resident and environmental information within the home.

[0084] Specifically, when the behavior support device 1 acquires the operating status of the washing machine 4 as environmental information and estimates that the washing is finished, it calculates an estimated behavior from the resident's time-series behavior information and behavior information, determines whether the resident's estimated behavior can be changed to the behavior of hanging out laundry, and if it can be changed, performs an operation by the mobile body 2 to encourage the resident to change their behavior to hang out laundry.

[0085] FIG. 7 is a flowchart of the processing periodically performed by the action support apparatus 1 of this embodiment. Steps S31 to S34 in FIG. 7 are the same as those in FIG. 3, and therefore will not be described here.

[0086] In step S71, the human behavior detection unit 112 estimates the operating state of the washing machine 4 installed in the house for each resident identified by the person identification information based on the sensor information of the moving object 2 and the house sensor 3, and acquires environmental information (current environmental information) such as the completion of washing. For example, the human behavior detection unit 112 recognizes the operating sound and alarm sound of the washing machine 4 from the environmental sounds in the house detected by the house sensor 3 (microphone), and estimates the washing state and the completion of washing.

[0087] In step S72, the behavior estimation unit 131 compares the time-series behavior information for each resident with the current behavior information and environmental information to analyze the state of the resident.

[0088] In step S73, the instruction information generating unit 132 determines whether the current environmental information indicates that the washing has finished, and if the washing has not finished (No in S73), ends the process.

[0089] If the laundry is finished (Yes in S73), in step S74, the instruction information generation unit 132 determines whether the resident's future estimated behavior, analyzed based on the time-series behavior information and the current behavior information, is the behavior of hanging out laundry, and if the behavior is the behavior of hanging out laundry (Yes in S74), the processing ends.

[0090] In step S75, the instruction information generation unit 132 determines whether the resident's current behavior information can be changed to the behavior of hanging out laundry, and if the change is not possible (No in S75), the processing ends. In other words, the priority of the current behavior and the behavior of hanging out laundry is determined, and if the priority of the current behavior is higher, the processing ends. For example, if the current behavior is a relaxed state, it is determined that continuing the relaxed state has high priority, and the processing ends. Whether to prioritize the relaxed state may be determined based on the resident's past behavior information stored in the time-series behavior information storage unit 12. For example, if the resident has not been in a relaxed state recently, it is possible to prioritize the relaxed state.

[0091] In step S76, the instruction information generation unit 132 sets an action to prompt the resident to change their behavior to hanging out the laundry in the instruction information of the moving object 2. Specifically, the instruction information is set so that the moving object 2 outputs the message "The laundry is finished. Please hang out the laundry."

[0092] In step S77, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in step S76.

[0093] By using the above-described action support device 1, the life support system can prevent the resident from forgetting to take out the laundry from the washing machine 4. The action support device 1 may also be applied to a clothes dryer to prevent a resident from forgetting to remove dried clothes from the dryer when the drying is finished. The behavior support device 1 may also be applied to a microwave oven to prevent the resident from forgetting to remove the heated food from the microwave oven when heating is finished. In the behavior support device 1, the operation of home appliances such as a washing machine, clothes dryer, and microwave oven is referred to as a home appliance task, and the "washing completed" of a washing machine, the "drying completed" of a clothes dryer, the "heating completed" of a microwave oven, etc. are referred to as the end of a home appliance task.

[0094] Example 6 Next, we will explain an embodiment in which, when the environmental state within the home set by the resident's estimated behavior (e.g., turning on the air conditioner) analyzed based on time-series behavior information and current behavior information differs from the environmental state indicated by the current environmental information, the mobile body 2 is instructed to perform an action that prompts the resident to take the estimated behavior. In more detail, if the estimated behavior is behavior related to the operation of the air conditioner 6, and the air conditioner 6 is not operating, the mobile body 2 prompts the resident to change their behavior to one that will operate the air conditioner 6.

[0095] FIG. 8 is a flowchart of the processing periodically performed by the action support apparatus 1 of this embodiment. Steps S31 to S34 in FIG. 8 are the same as those in FIG. 3, and therefore will not be described here.

[0096] In step S81, the human behavior detection unit 112 estimates the operating state of the air conditioner 6 installed in the house for each resident identified by the person identification information based on the sensor information of the moving object 2 and the house sensor 3, and acquires environmental information (current environmental information) such as whether it is in operation. For example, the operating sound of the air conditioner 6 is recognized from the room temperature and the environmental sounds in the house detected by the house sensor 3 (microphone, temperature sensor), and the operating state of heating / cooling operation is estimated.

[0097] In step S82, the behavior estimation unit 131 compares the time-series behavior information for each resident with the current behavior information and environmental information to analyze the state of the resident.

[0098] In step S83, if the estimated future behavior of the resident analyzed based on the time-series behavior information and the current behavior information is behavior related to the operation of the air conditioner 6, the instruction information generation unit 132 determines whether the estimated behavior of the resident corresponds to the current operating state of the air conditioner 6 indicated by the current environmental information, and if they correspond (Yes in S83), ends the processing. Specifically, if the estimated behavior is behavior to operate the air conditioner and the air conditioner 6 is in air conditioning operation, ends the processing.

[0099] In step S84, the instruction information generation unit 132 determines whether the current behavior information of the resident is in a relaxed state, and if the current behavior information of the resident is in a relaxed state (Yes in S84), ends the processing. In other words, if the current behavior information of the resident is in a relaxed state, the instruction information generation unit 132 does not instruct the moving object 2 to perform the action of operating the air conditioner 6 in step S85 to encourage behavioral change.

[0100] If the occupant is not in a relaxed state (No in S84), in step S85 the instruction information generation unit 132 sets an action to prompt the occupant to operate the air conditioner in the instruction information for the moving object 2. Specifically, the moving object 2 is configured to output the message "Please operate the air conditioner."

[0101] In step S86, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in step S85.

[0102] The above-described behavior support device 1 can prevent the resident from forgetting to operate the air conditioner 6.

[0103] Example 7 In this embodiment, the behavior support device 1 compares the estimated behavior estimated from the resident's time-series behavior information with the current behavior state, and in the case of different behavior or abnormal behavior, instructs the mobile body 2 to photograph the resident's behavior using the sensor 21 of the mobile body 2.

[0104] Specifically, the behavioral support device 1 compares the estimated behavior of "sleeping" estimated from the time-series behavioral information of nighttime relaxation → moving to bedroom → bedroom with the current behavioral information of "going to the toilet" recognized from the behavior of waking up outside of bedtime and trying to get up from bed, and since these are different behaviors, it instructs the mobile body 2 to take a picture of the resident's behavior using the sensor 21 (camera) of the mobile body 2.

[0105] Furthermore, the action support device 1 compares the action information of the resident captured by the mobile object 2 with the past action information of the resident, and if it determines that there is a certain change over time (decline or improvement), it notifies the communication terminal of the resident or the person watching over the resident. The certain change over time is, for example, a change in posture or walking behavior.

[0106] FIG. 9 is a flowchart of the processing periodically performed by the action support apparatus 1 of this embodiment. Steps S31 to S35 in FIG. 9 are the same as those in FIG. 3, and therefore will not be described here.

[0107] In step S91, the instruction information generation unit 132 determines whether the estimated behavior of the resident estimated from the time-series behavior information is different from the current behavior information, and if they are the same or equivalent behavior (No in S91), ends the process. Specifically, the instruction information generation unit 132 compares the estimated behavior of "sleeping" with the current behavior information of "going to the toilet."

[0108] In step S92, the instruction information generating unit 132 sets the operation of photographing the resident in the instruction information for the moving object 2.

[0109] In step S93, the instruction information notification unit 133 notifies the moving object 2 of the instruction information set in step S92.

[0110] In step S94, the behavior recognition unit 11 acquires sensor information captured by the sensor 21 (camera) of the moving object 2 as a behavior image of the resident.

[0111] In step S95, the behavior estimation unit 131 compares the behavior information of the resident acquired in step S94 with past behavior information of the resident.

[0112] In step S96, the instruction information generation unit 132 determines the resident's risk of frailty by comparing the behavioral information, and if it determines that there is no certain change over time (decline or improvement) (No in S96), ends the processing.

[0113] In step S97, the watching information notifying unit 14 notifies the resident terminal 9a or the watcher terminal 9b of a certain change over time of the resident as watching information.

[0114] The above-described behavior support device 1 can notify a resident of certain changes over time in the resident as monitoring information.

[0115] In the above-described behavior support system, the mobile object 2 provides support for the resident's daily life based on time-series behavior information and current behavior information, but life support can also be provided based on time-series environmental information within the home.

[0116] For example, a robot vacuum cleaner as the mobile body 2 accumulates cleaning route information, and identifies the location information of the garbage that may cause the resident to fall based on the detection status of obstacles along the way, and takes a picture of the garbage with the sensor 21 (camera) mounted on the mobile body 2. Then, the mobile body 2 calls out to the resident to warn them. In addition, the location information of the garbage and the captured image of the garbage are notified to the resident terminal 9a or the watcher terminal 9b as monitoring information.

[0117] In addition, the system calculates changes in the degree of dirtiness in the home from time-series information on the amount of dirt suctioned by the robot vacuum cleaner as the mobile object 2, and determines that the resident is showing signs of frailty if the degree of dirtiness increases.The system also calculates changes in the degree of tidiness in the home from time-series information on the robot vacuum cleaner's obstacle detection, and determines that the resident is showing signs of frailty if the degree of tidiness decreases.The degree of dirtiness and the degree of tidiness are notified to the resident terminal 9a or the watcher terminal 9b as monitoring information.

[0118] The present invention is not limited to the above-described examples, and includes various modifications. The above-described examples have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. [Explanation of symbols]

[0119] 1 Behavior support device 2. Mobile 3a In-house sensor 3b In-house sensor 4 washing machines 5 refrigerators 6 Air conditioners 7 Home LAN 8. Internet 9a Resident terminal 9b Guardian terminal 21 Sensors 22 Driving means 23 Control Unit 24 Housing heater 25 Map Information 11 Behavior Recognition Department 111 Person Identification Department 112 Human behavior detection unit 12 Time-series behavior information storage unit 13 Mobile object instruction section 131 Behavior estimation section 132 Instruction information notification section 14 Monitoring Information Notification Department

Claims

1. a mobile object that moves autonomously within a home and has a sensor; In-home sensors and an action support device that controls the mobile object by detecting behavioral information of a resident and environmental information in the home based on sensor information acquired from a sensor of the mobile object and a sensor in the home; A life assistance system comprising: the action support device, a behavior recognition unit that recognizes current behavior information of the resident based on sensor information acquired from a sensor of the mobile object and a sensor in the home as current behavior information and sequentially stores the recognized behavior information as time-series behavior information; a mobile object instruction unit that analyzes the time-series behavior information to determine that the current behavior information has changed, and instructs the mobile object to perform an action that prompts the resident to change their behavior when the amount of change in the behavior information is equal to or greater than a threshold; A life support system comprising:

2. The life assistance system according to claim 1, The action support device further comprises: a monitoring information notification unit that notifies a communication terminal of the resident or a person watching over the resident of information about changes in the behavioral information of the resident or information about the behavioral change; A life support system comprising:

3. The life assistance system according to claim 1, The moving object instruction unit Based on the time-series behavior information and the current behavior information, the mobile body is instructed to move to a location where the resident is expected to move. A life support system characterized by:

4. 4. The life assistance system according to claim 3, The moving object instruction unit When a sign of frailty of the resident is detected based on the time-series behavior information and the current behavior information, an instruction is given to the mobile body to move to a location where the resident is estimated to move. A life support system characterized by:

5. 5. The life assistance system according to claim 3, The moving object instruction unit If the resident cannot be recognized at the destination even after a predetermined time has elapsed since the moving object moved, the moving object is instructed to perform a predetermined action for the resident. A life support system characterized by:

6. The life assistance system according to claim 1, The moving object instruction unit When it is determined that the resident is in poor health based on the time-series behavior information and the current behavior information, the mobile body is instructed to perform an operation of measuring the state of the resident with the sensor provided in the mobile body. A life support system characterized by:

7. 7. The life assistance system according to claim 6, The moving object instruction unit further Instructing the mobile body to change the set temperature of the housing heater of the mobile body according to the body temperature of the occupant. A life support system characterized by:

8. 4. The life assistance system according to claim 3, The moving object instruction unit a resident inputs the location of the refrigerator relative to map information via a resident terminal, and instructs the mobile body to move to the installation location of the refrigerator; Instructing the mobile body to perform a predetermined action including an action of photographing an opening and closing action of the refrigerator by the resident or an action of photographing the inside of the refrigerator using the sensor of the mobile body. A life support system characterized by:

9. The life assistance system according to claim 1, The behavior recognition unit acquiring task states of home appliances as home environment information based on sensor information acquired from sensors in the moving object and sensors in the home; The moving object instruction unit When the environmental information indicates the end of a task of the home appliance, determine whether the current behavior information of the resident can be changed to behavior associated with the task of the completed home appliance, and instruct the mobile body to perform an operation to prompt the behavior change of the resident to behavior associated with the task of the completed home appliance; The determination of whether the change is possible is made by: Between the current behavior information of the resident and the behavior associated with the task of the completed home appliance, a predetermined priority is followed based on the past behavior information of the resident. A life support system characterized by:

10. 10. The life assistance system according to claim 9, The moving object instruction unit If the current behavioral information of the resident is in a relaxed state, the mobile body is not instructed to perform an action that encourages the behavioral change of the resident. A life support system characterized by:

11. The life assistance system according to claim 1, The behavior recognition unit The operating status of the air conditioner is recognized as information on the home environment, The moving object instruction unit When the future predicted behavior of the resident analyzed based on the time-series behavior information and the current behavior information is behavior related to the operation of the air conditioner, the mobile body is instructed to perform an action to prompt the behavior of operating the air conditioner according to environmental information indicating the current operating state of the air conditioner. A life support system characterized by:

12. The life assistance system according to claim 11, The moving object instruction unit If the current behavioral information of the resident is in a relaxed state, the moving body is not instructed to perform an action to prompt the resident to operate the air conditioner. A life support system characterized by:

13. The life assistance system according to claim 1, The moving object instruction unit When it is determined that the estimated behavior estimated from the time-series behavior information differs from the current behavior information, the mobile body is instructed to take an image of the resident. A life support system characterized by:

14. The life assistance system according to claim 13, The action support device further comprises: a monitoring information notification unit that compares current behavioral information of the resident captured by instructing the mobile body with past behavioral information of the resident accumulated as time-series behavioral information, and notifies a communication terminal of the resident or a person watching over the resident of the fact that a certain change over time has occurred when the information is determined to be such; A life support system comprising:

Images