Action prediction system, action prediction program, action prediction method, and action prediction device

Abstract

This action prediction system comprises: an action acquisition unit that acquires action information, which is information indicating an action of a person to be monitored; a prediction unit that predicts, on the basis of the action information acquired by the action acquisition unit, a subsequent action, which is an action of the person to be monitored after the action indicated by the action information; a determination unit that determines, on the basis of the subsequent action predicted by the prediction unit, the necessity of assistance for the person to be monitored; and a notification unit that, when it is determined by the determination unit that assistance is necessary, issues a notification to make it known that assistance is necessary.

Description

Behavior prediction system, behavior prediction program, behavior prediction method, and behavior prediction device

[0001] The present disclosure relates to a behavior prediction system, a behavior prediction program, a behavior prediction method, and a behavior prediction device.

[0002] Patent Document 1 describes a system that displays, as a prediction icon, an icon corresponding to a prediction determination result that predicts a determination result of future movements of a monitored person.

[0003] The document describes that a fall prediction icon is displayed as a specific example of a prediction icon, and that the prediction icon flashes to indicate that the prediction icon does not correspond to the current state of the monitored person, but corresponds to the predicted future state of the monitored person.

[0004] JP 2019-145143 A

[0005] Even if a fall of the monitored person is predicted, assistance for the monitored person is not necessarily required. For example, if the monitored person is healthy and relatively young, or if they are skilled at dealing with their own falls, assistance may not be necessary.

[0006] Therefore, in the system described in Patent Document 1, every time a predicted icon for a monitored person is displayed, the monitor needs to determine whether or not the monitored person needs assistance, which places a mental burden on the monitor.

[0007] An object of the present disclosure is to reduce the burden on a monitor in a behavior prediction system, a behavior prediction program, a behavior prediction method, and a behavior prediction device that predict the behavior of a monitored person.

[0008] The behavior prediction system according to the present disclosure is characterized by comprising: a behavior acquisition unit that acquires behavior information, which is information indicating the behavior of a monitored person; a prediction unit that predicts a subsequent behavior, which is a behavior of the monitored person that occurs after the behavior indicated by the behavior information, based on the behavior information acquired by the behavior acquisition unit; a determination unit that determines whether assistance is required for the monitored person based on the subsequent behavior predicted by the prediction unit; and a notification unit that issues an assistance notification to notify the monitored person that assistance is required when the determination unit determines that assistance is required.

[0009] The behavior prediction program of the present disclosure is characterized in that it enables a computer to implement the following functions: a behavior acquisition function that acquires behavior information, which is information indicating the behavior of a monitored person; a prediction function that predicts a subsequent behavior of the monitored person that follows the behavior indicated by the behavior information based on the behavior information acquired by the behavior acquisition function; a determination function that determines whether assistance is required for the monitored person based on the subsequent behavior predicted by the prediction function; and a notification function that issues an assistance notification to notify the monitored person that assistance is required if the determination function determines that assistance is required.

[0010] The behavior prediction method according to the present disclosure is characterized in that it is a behavior prediction method in a behavior prediction system, and comprises a behavior acquisition step of acquiring behavior information, which is information indicating the behavior of a monitored person; a prediction step of predicting a subsequent behavior of the monitored person that follows the behavior indicated by the behavior information based on the behavior information acquired by the behavior acquisition step; a determination step of determining whether or not assistance is required for the monitored person based on the subsequent behavior predicted by the prediction step; and a notification step of issuing an assistance notification to notify the monitored person that assistance is required if the determination step determines that assistance is required.

[0011] The behavior prediction device according to the present disclosure is characterized by comprising: a behavior acquisition unit that acquires behavior information, which is information indicating the behavior of a monitored person; a prediction unit that predicts a subsequent behavior, which is a behavior of the monitored person that occurs after the behavior indicated by the behavior information, based on the behavior information acquired by the behavior acquisition unit; a determination unit that determines whether assistance is required for the monitored person based on the subsequent behavior predicted by the prediction unit; and an output unit that outputs a signal indicating that assistance is required when the determination unit determines that assistance is required.

[0012] According to this configuration, the behavior of the monitored person is predicted, and based on the prediction result, it is determined whether or not assistance is required for the monitored person. Then, when it is determined that assistance is required, a notification is issued.

[0013] This allows the monitor to know that assistance is required for the monitored person when a notification is issued. Therefore, the monitor does not need to determine whether assistance is required for the monitored person each time a notification is issued. As a result, the burden on the monitor can be reduced.

[0014] That is, according to this configuration, it is possible to reduce the burden on the monitor in a behavior prediction system, a behavior prediction program, a behavior prediction method, and a behavior prediction device that predict the behavior of a monitored person.

[0015] Furthermore, in the present disclosure, the notification unit is configured to be able to make a predictive notification to inform the user of the prediction result by the prediction unit, and when the determination unit determines that the assistance is necessary, the notification unit makes the assistance notification and the predictive notification, and when the determination unit determines that the assistance is not necessary, the notification unit does not make the assistance notification but makes the predictive notification.

[0016] According to this configuration, a predictive notification is made regardless of whether assistance is required, thereby enabling the monitor to grasp the predicted subsequent behavior.

[0017] Furthermore, with this configuration, if it is determined that assistance is needed, an assistance notification is made, and if it is determined that assistance is not needed, an assistance notification is not made, so that the monitor can easily understand whether assistance is needed or not.

[0018] Furthermore, in the present disclosure, the notification unit is configured to be able to make a predictive notification to inform the user of the prediction result by the prediction unit, and when the determination unit determines that the assistance is necessary, the notification unit makes the predictive notification to the monitor and the monitored person, and when the determination unit determines that the assistance is not necessary, the notification unit makes the predictive notification to the monitor rather than to the monitored person.

[0019] According to this configuration, when a subsequent behavior (such as a fall) that requires assistance is predicted, the monitored person can understand the prediction result, which makes it easier for the monitored person to avoid the subsequent behavior (such as a fall).

[0020] Furthermore, with this configuration, if a subsequent action that does not require assistance is predicted, a predictive notification is not sent to the monitored person, thereby preventing the monitored person from feeling uncomfortable due to unnecessary predictive notifications.

[0021] Furthermore, in the present disclosure, the notification unit is configured to be able to issue a prediction notification to notify the prediction result by the prediction unit, and the prediction unit is configured to be able to predict multiple types of candidates for the subsequent behavior, and when multiple types of candidates for the subsequent behavior are predicted by the prediction unit, it is preferable that the notification unit issues a multiple candidate notification as the prediction notification to notify the multiple types of candidates for the subsequent behavior that have been predicted.

[0022] According to this configuration, when multiple types of subsequent behavior candidates are predicted, each candidate is notified, which allows the observer to grasp each candidate and therefore facilitates appropriate response.

[0023] Furthermore, in the present disclosure, the notification unit is configured to be able to make a predictive notification to inform the user of the prediction result by the prediction unit, the prediction unit is configured to be able to calculate the probability of occurrence of the subsequent behavior, and it is preferable that the notification unit, when making the predictive notification, also notifies the user of the probability of occurrence.

[0024] According to this configuration, the probability of occurrence of the subsequent behavior is notified, which allows the observer to grasp the probability of occurrence of the subsequent behavior, making it easier for the observer to take appropriate action.

[0025] Furthermore, in the present disclosure, it is preferable that the judgment unit is configured to be able to calculate the probability that the monitored person will be in a state requiring the assistance based on the behavioral information, and to judge whether the assistance is required based on the probability.

[0026] This configuration makes it possible to determine that assistance is required when the probability that the monitored person will require assistance is relatively high, and to determine that assistance is not required when the probability is relatively low. This makes it easier to accurately determine whether assistance is required.

[0027] Furthermore, in the present disclosure, it is preferable to include an attribute acquisition unit that acquires attribute information that is information indicating the attributes of the monitored person, and the determination unit determines whether or not assistance is required based on the subsequent behavior predicted by the prediction unit and the attribute information acquired by the attribute acquisition unit.

[0028] According to this configuration, the need for assistance is determined based not only on the predicted subsequent behavior but also on the attributes of the monitored person (e.g., age, health condition, etc.), which tends to improve the accuracy of the determination of the need for assistance.

[0029] Furthermore, in the present disclosure, it is preferable to include a setting unit capable of setting criteria regarding the need for assistance, and the determination unit determines the need for assistance based on the subsequent behavior predicted by the prediction unit and the criteria set by the setting unit.

[0030] According to this configuration, the need for assistance is determined based not only on predicted subsequent behavior but also on set criteria. Therefore, if the monitor appropriately sets the criteria, a configuration can be realized in which appropriate determinations are made according to the actual situation at the site.

[0031] 1 is a diagram showing an overview of a behavior prediction system. FIG. 1 is a block diagram of a behavior prediction system. FIG. 2 is a diagram showing an example of a superimposed image. FIG. 3 is a diagram showing a notification screen when an assistance notification and a predicted notification are made. FIG. 4 is a diagram showing a notification screen after the assistance button is pressed. FIG. 5 is a diagram showing a notification screen when assistance is provided. FIG. 6 is a diagram showing a notification screen when the behavior result of the monitored person is determined. FIG. 7 is a diagram showing a notification screen when a predicted notification is made. FIG. 8 is a diagram showing a notification screen when the behavior result of the monitored person is determined. FIG. 9 is a diagram showing a notification screen when the behavior result of the monitored person is determined. FIG. 10 is a flowchart of a notification control flow. FIG. 11 is a block diagram of a behavior prediction system in another embodiment (3).

[0032] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present disclosure will be described with reference to the drawings

[0033] [Overall Configuration of Behavior Prediction System] As shown in Fig. 1, a behavior prediction system A in this embodiment includes a monitoring device 1 and a terminal 2. The monitoring device 1 and the terminal 2 are connected to each other via a predetermined network. The monitoring device 1 is installed in a monitored area B. A monitored person D exists in the monitored area B. The monitoring device 1 monitors the monitored person D who exists in the monitored area B in which the monitoring device 1 is installed.

[0034] Although not particularly limited, the terminal 2 in this embodiment has a touch panel 21. The touch panel 21 is capable of displaying various types of information. The touch panel 21 is also configured to receive manual operation input.

[0035] An image showing the situation within the monitored area B is transmitted from the monitoring device 1 to the terminal 2. A monitor E using the terminal 2 can grasp the situation within the monitored area B while being outside the monitored area B by looking at the image displayed on the touch panel 21 of the terminal 2.

[0036] In this embodiment, the monitored area B is a room in a welfare facility for the elderly, the monitor E is a staff member of the welfare facility for the elderly, and the terminal 2 is an information terminal (such as a smartphone or a personal computer) used by the staff member. In the example shown in Fig. 1, the monitored person D is a resident of the welfare facility for the elderly, and the monitor E, who is a staff member, is located in an area other than the room (for example, a waiting room).

[0037] The number of monitoring devices 1 included in the behavior prediction system A may be one or more. The monitoring devices 1 may be installed one in each room of the elderly welfare facility, or may be installed in areas other than the rooms (for example, corridors).

[0038] The number of monitors E may be one or more. The number of terminals 2 included in the behavior prediction system A may be one or more. The multiple monitors E may each use one terminal 2, or may share one terminal 2.

[0039] 1, the monitoring device 1 also has a speaker 7. The speaker 7 is capable of outputting sound to the monitored person D (monitored area B).

[0040] The behavior prediction system A will be described in detail below.

[0041] [Status Detection Unit] As shown in Fig. 2, the monitoring device 1 has an imaging unit 3. The imaging unit 3 captures an image of the monitoring target area B (see Fig. 1).

[0042] In this embodiment, the imaging unit 3 is configured with an infrared camera. However, the present disclosure is not limited thereto, and the imaging unit 3 may be any device other than an infrared camera as long as it is capable of capturing images of the monitored area B. The imaging unit 3 is preferably a ToF (Time of Flight) camera. The captured image acquired by the imaging unit 3 is preferably a depth image, which is an image including depth information. The captured image acquired by the imaging unit 3 may be a video or a still image.

[0043] As shown in Fig. 2, the monitoring device 1 includes a skeletal information generation unit 4 and a status detection unit 5. The captured image acquired by the imaging unit 3 is sent to the skeletal information generation unit 4. The skeletal information generation unit 4 generates skeletal information 10 (see Fig. 3) based on the captured image. The skeletal information 10 is information indicating the skeleton of a person (monitored person D) present in the monitoring target area B.

[0044] The skeletal information 10 in this embodiment is specifically a skeletal model. The skeletal model is generated by analyzing the captured image to recognize the three-dimensional positions of the joints of the monitored person D and connecting the joints with straight lines.

[0045] The skeleton information generating unit 4 generates a superimposed image 11 as shown in FIG. 3 by superimposing the skeleton information 10 on the captured image.

[0046] 2, the skeleton information 10 generated by the skeleton information generating unit 4 is sent to the state detecting unit 5. The state detecting unit 5 detects the state of the monitored person D based on the skeleton information 10.

[0047] Although not particularly limited, specific examples of the state detected by the state detection unit 5 include the body position of the monitored person D (e.g., standing, lying, sitting), falling, lying down, slipping down, boundary position, getting up, getting out of bed, moving to the toilet, etc. In this specification, the state of the monitored person D is considered to be a specific example of the "behavior of the monitored person D." For example, the monitored person D sitting in a chair or the monitored person D falling are both "behavior of the monitored person D." Furthermore, the "boundary position" is, for example, sitting beside the bed.

[0048] The status detection unit 5 generates status information based on the detected status of the monitored person D. In this way, the status detection unit 5 acquires the status information. The status information is information that indicates the status of the monitored person D. In other words, the status information is information that indicates the behavior of the monitored person D.

[0049] In this specification, the status information is a specific example of “behavior information.” The behavior information is information that indicates the behavior of the monitored person D.

[0050] In this way, the behavior prediction system A includes the state detection unit 5 as a behavior acquisition unit that acquires behavior information that indicates the behavior of the monitored person D.

[0051] 1 and 2, the behavior prediction system A includes a management computer 6. The management computer 6 is connected to the monitoring device 1 and the terminal 2 via a predetermined network.

[0052] As shown in FIG. 2, the management computer 6 has a status acquisition unit 61 and a history storage unit 62. The status acquisition unit 61 acquires the above-mentioned status information from the status detection unit 5. The status acquisition unit 61 sends the acquired status information to the history storage unit 62. The history storage unit 62 generates history information based on the received status information and stores the history information. In this way, the history storage unit 62 acquires the history information. Note that the history information is information that indicates the history of the status of the monitored person D. In other words, the history information is information that indicates the behavior of the monitored person D.

[0053] In this specification, history information is a specific example of "behavioral information."

[0054] In this way, the behavior prediction system A includes the history storage unit 62 as a behavior acquisition unit that acquires behavior information that indicates the behavior of the monitored person D.

[0055] The history information may be, for example, information indicating the transition of the state (behavior) of the monitored person D at each predetermined time interval (for example, every minute). In the history information, time may be associated with the state (behavior) of the monitored person D at that time.

[0056] Furthermore, if there are multiple monitored persons D, history information may be generated individually for each monitored person D.

[0057] As shown in FIG. 2 , the management computer 6 has a prediction unit 63. The prediction unit 63 acquires status information from the status acquisition unit 61. The prediction unit 63 also acquires history information from the history storage unit 62. The prediction unit 63 then predicts subsequent behavior based on the status information and history information. The subsequent behavior is behavior of the monitored person D that occurs after the behavior indicated by the behavior information (more specifically, the status information and history information). In other words, the subsequent behavior is future behavior of the monitored person D.

[0058] In this way, the behavior prediction system A is equipped with a prediction unit 63 that predicts subsequent behavior of the monitored person D, which is behavior that occurs after the behavior indicated by the behavior information, based on the behavior information acquired by the behavior acquisition unit (more specifically, the status detection unit 5 and the history memory unit 62).

[0059] The prediction unit 63 may generate tendency information indicating the behavioral tendency of the monitored person D based on the history information (e.g., that the monitored person D tends to go to the toilet when walking at night, or that the monitored person D tends to fall at a specific location in a welfare facility for the elderly, etc.), and may predict subsequent behavior based on the tendency information and the status information. Specific examples of subsequent behavior include the monitored person D's body position (e.g., standing, lying down, sitting), falling, lying down, slipping down, boundary position, getting up, getting out of bed, going to the toilet, etc.

[0060] Furthermore, the prediction unit 63 may predict the subsequent behavior by using a neural network trained using deep learning, for example.

[0061] 2, the management computer 6 has a determination unit 64, an attribute acquisition unit 65, and a setting unit 66. The prediction unit 63 sends information indicating the predicted subsequent behavior to the determination unit 64.

[0062] The attribute acquisition unit 65 acquires attribute information. The attribute information is information indicating the attributes of the monitored person D. The attributes may be, for example, age, sex, health condition, whether the monitored person is adept at dealing with falls, etc. The attribute information may be information input in advance by the monitor E or the like, or may be information generated by performing a known image recognition process or the like on the captured image acquired by the imaging unit 3.

[0063] In this way, the behavior prediction system A includes an attribute acquisition unit 65 that acquires attribute information that indicates the attributes of the monitored person D.

[0064] The setting unit 66 is configured to set criteria regarding whether assistance is required for the monitored person D in response to manual operation input on the touch panel 21 (terminal 2). That is, the behavior prediction system A includes the setting unit 66 that can set criteria regarding whether assistance is required.

[0065] 2, the determination unit 64 acquires attribute information from the attribute acquisition unit 65 and also acquires the above-mentioned criteria from the setting unit 66. Then, the determination unit 64 determines whether assistance is required for the monitored person D based on the predicted subsequent behavior, the attribute information, and the above-mentioned criteria.

[0066] In this way, the behavior prediction system A includes a determination unit 64 that determines whether assistance is required for the monitored person D based on the subsequent behavior predicted by the prediction unit 63. The determination unit 64 also determines whether assistance is required based on the subsequent behavior predicted by the prediction unit 63 and the attribute information acquired by the attribute acquisition unit 65. The determination unit 64 also determines whether assistance is required based on the subsequent behavior predicted by the prediction unit 63 and the criteria set by the setting unit 66.

[0067] Although not particularly limited, for example, when the predicted subsequent behavior is a fall, the judgment unit 64 may determine that assistance is not required if the attribute information indicates a predetermined content (for example, being good at dealing with one's own falls), and may determine that assistance is required if the attribute information does not indicate the predetermined content.

[0068] The criteria set by the setting unit 66 are not particularly limited, but may be, for example, "assistance is required if the monitored person D is over a specified age, and assistance is not required if the monitored person D is under the specified age," or "assistance is required if the predicted subsequent behavior is a specified behavior, and assistance is not required if the predicted subsequent behavior is other than the specified behavior."

[0069] 2, the management computer 6 has a first notification unit 67. The determination unit 64 sends the determination result of whether assistance is required and information indicating the subsequent behavior predicted by the prediction unit 63 to the first notification unit 67. The first notification unit 67 issues an assistance notification in accordance with the determination result. The first notification unit 67 also issues a predicted notification based on the information indicating the subsequent behavior.

[0070] The assistance notification is to notify the monitored person D that assistance is required. The prediction notification is to notify the monitored person D of the prediction result by the prediction unit 63.

[0071] More specifically, when the determination unit 64 determines that assistance is required, the first notification unit 67 issues an assistance notification and a predicted notification. At this time, the first notification unit 67 transmits a predetermined signal to the terminal 2. In response to the signal, the terminal 2 displays an assistance notification icon G1 (see FIG. 4) and a predicted notification icon G2 (see FIG. 4) on the touch panel 21. In addition, at this time, the first notification unit 67 transmits a predetermined signal to the speaker 7. In response to the signal, the speaker 7 outputs audio notifying the user of the predicted subsequent action.

[0072] That is, at this time, the first notification unit 67 notifies the monitor E of assistance via the touch panel 21 , and also makes a predicted notification to the monitor E and the monitored person D via the touch panel 21 and the speaker 7 .

[0073] The assistance notification icon G1 is an icon indicating that assistance is required for the monitored person D. The predicted notification icon G2 is an icon indicating the prediction result by the prediction unit 63. The display of the assistance notification icon G1 is a specific example of "assistance notification." The display of the predicted notification icon G2 is a specific example of "predictive notification." Outputting a voice to notify the monitored person D of predicted subsequent actions is a specific example of "predictive notification."

[0074] Furthermore, the assistance notification icon G1 and the prediction notification icon G2 may be individually displayed icons, or a single icon may be displayed as the assistance notification icon G1 and the prediction notification icon G2.

[0075] 4, a fall icon 41 is displayed on the behavior prediction display unit 31 displayed on the touch panel 21. The behavior prediction display unit 31 is an area where an assistance notification icon G1 and a prediction notification icon G2 are displayed. In this embodiment, the fall icon 41 displayed on the behavior prediction display unit 31 indicates that "assistance is required because a fall of the monitored person D is predicted" and "the prediction result by the prediction unit 63 is a fall." Therefore, the fall icon 41 displayed on the behavior prediction display unit 31 is both the assistance notification icon G1 and the prediction notification icon G2.

[0076] In this way, the behavior prediction system A includes the first notification unit 67 as a notification unit that issues an assistance notification to notify the monitor E and the monitored person D when the determination unit 64 determines that assistance is needed. Furthermore, the first notification unit 67 is configured to be able to issue a predictive notification that notifies the result of the prediction by the prediction unit 63. Furthermore, when the determination unit 64 determines that assistance is needed, the first notification unit 67 issues a predictive notification to the monitor E and the monitored person D.

[0077] Furthermore, when the determination unit 64 determines that assistance is not required, the first notification unit 67 does not issue an assistance notification but issues a predictive notification. At this time, the first notification unit 67 transmits a predetermined signal to the terminal 2. In response to the signal, the terminal 2 displays a predictive notification icon G2 (see FIG. 8 ) on the touch panel 21. At this time, the first notification unit 67 does not issue a predictive notification via the speaker 7.

[0078] That is, at this time, the first notification unit 67 does not issue an assistance notification. Also, at this time, the first notification unit 67 issues a predictive notification to the monitor E via the touch panel 21, but does not issue a predictive notification to the monitored person D.

[0079] 8 , a toilet icon 42 is displayed on the behavior prediction display unit 31 displayed on the touch panel 21. In this embodiment, the toilet icon 42 displayed on the behavior prediction display unit 31 indicates that the prediction result by the prediction unit 63 is a move to the toilet. In this embodiment, the toilet icon 42 displayed on the behavior prediction display unit 31 is the prediction notification icon G2, but is not the assistance notification icon G1.

[0080] In this way, when the determination unit 64 determines that assistance is not required, the first notification unit 67 does not issue an assistance notification but issues a predictive notification. Also, when the determination unit 64 determines that assistance is not required, the first notification unit 67 does not issue a predictive notification to the monitored person D but issues it to the monitor E.

[0081] As described above, in this embodiment, the fall icon 41 displayed on the behavior prediction display unit 31 is the assistance notification icon G1, and the toilet icon 42 displayed on the behavior prediction display unit 31 is not the assistance notification icon G1. However, the present disclosure is not limited to this. For example, if the monitored person D needs assistance to move to the toilet, the toilet icon 42 may be displayed on the behavior prediction display unit 31 as the assistance notification icon G1. In this case, if the monitor E recognizes that the monitored person D needs assistance to move to the toilet (in other words, if the monitored person D is a person who has the attribute "needs assistance to move to the toilet"), the toilet icon 42 displayed on the behavior prediction display unit 31 will function as the assistance notification icon G1.

[0082] Furthermore, in this embodiment, whether or not an icon displayed on the behavior prediction display unit 31 is the assistance notification icon G1 is indicated by the type of the icon. However, the present disclosure is not limited to this. Whether or not an icon displayed on the behavior prediction display unit 31 is the assistance notification icon G1 may be indicated by the display mode of the icon. For example, an icon that is the assistance notification icon G1 may be highlighted (for example, flashing or displayed in a different color) compared to icons that are not the assistance notification icon G1.

[0083] More specifically, for example, when the toilet icon 42 displayed on the behavior prediction display unit 31 is flashing, the toilet icon 42 indicates that the prediction result is a move to the toilet and that assistance is required. In this case, the flashing toilet icon 42 is both the assistance notification icon G1 and the prediction notification icon G2. When the toilet icon 42 displayed on the behavior prediction display unit 31 is displayed normally (in other words, not flashing), the toilet icon 42 indicates that the prediction result is a move to the toilet and that assistance is not required. In this case, the non-flashing toilet icon 42 is not the assistance notification icon G1 but the prediction notification icon G2.

[0084] Furthermore, the assistance notification is not limited to displaying the assistance notification icon G1. For example, the assistance notification may be a voice output by the terminal 2 notifying the user that assistance is needed, or a message notifying the user that assistance is needed may be printed on paper. Furthermore, the predictive notification is not limited to the above-described manner. For example, the predictive notification may be a message notifying the user of predicted subsequent actions that may be printed on paper.

[0085] The prediction unit 63 is configured to be able to predict multiple types of candidate subsequent actions. When multiple types of candidate subsequent actions are predicted by the prediction unit 63, the first notification unit 67 notifies the user of the multiple candidates as a prediction notification. The multiple candidate notification notifies the user of the multiple types of candidate subsequent actions that have been predicted.

[0086] As a multiple candidate notification, for example, multiple types of prediction notification icons G2 may be displayed on the above-mentioned behavior prediction display section 31.

[0087] In this way, when the prediction unit 63 predicts multiple types of candidate subsequent actions, the first notification unit 67 issues a multiple candidate notification as a prediction notification to notify the user of the multiple types of candidate subsequent actions that have been predicted.

[0088] The skeleton information generating unit 4, the state detecting unit 5, and each functional unit (such as the state acquiring unit 61) included in the management computer 6 may be a physical device such as a microcomputer, or may be a functional unit in software. For example, a configuration may be adopted in which a program corresponding to each functional unit included in the management computer 6 is stored in a ROM or non-volatile memory (not shown), and the program is loaded into and executed by a CPU, thereby executing a process corresponding to each functional unit.

[0089] 2 is configured to calculate the probability of occurrence of a subsequent behavior when predicting the subsequent behavior. The first notification unit 67 also notifies the occurrence probability when making a prediction notification.

[0090] 8, the probability of a trip to the toilet is calculated to be 90%. Therefore, the first notification unit 67 notifies the terminal 2 of this probability by transmitting a predetermined signal. More specifically, the first notification unit 67 displays "90%" on the behavior prediction display unit 31.

[0091] When the above-described multiple candidate notification is performed, the first notification unit 67 may notify the occurrence probability for each of the multiple types of subsequent action candidates.

[0092] [Notification Control Flow] The behavior prediction system A is configured to perform processes related to assistance notification and predicted notification in accordance with the notification control flow shown in FIG.

[0093] 11 will be described below assuming specific cases. First, a case where assistance is provided by the monitor E after a fall is predicted as a subsequent action will be described as case C1.

[0094] [Case C1] When the notification control flow starts, first, the process of step S01 is executed. In step S01, the prediction unit 63 acquires state information from the state acquisition unit 61. Thereafter, the process proceeds to step S02.

[0095] In step S02, the prediction unit 63 acquires history information from the history storage unit 62. After that, the process proceeds to step S03.

[0096] In step S03, the prediction unit 63 predicts the subsequent behavior as described above, and then the process proceeds to step S04.

[0097] In step S04, as described above, the determination unit 64 determines whether assistance is required for the monitored person D. In case C1, it is determined that assistance is required ("Yes" in step S04 of FIG. 11), and the process proceeds to step S05.

[0098] In step S05, the first notification unit 67 issues an assistance notification and a predictive notification. At this time, the first notification unit 67 displays a notification screen on the touch panel 21 of the terminal 2. Examples of the notification screens are shown in Figures 4 to 10. In this case C1, the notification screen shown in Figure 4 is displayed.

[0099] As shown in FIG. 4, the notification screen displays a monitoring location display section 30, a behavior prediction display section 31, a behavior result display section 32, an assistance schedule display section 33, an assistance button 34, and an image display section 35.

[0100] The monitoring location display section 30 is an area where the position of the monitoring target area B (see FIG. 1) is displayed. For example, the monitoring location display section 30 displays the room number of the monitored person D.

[0101] As described above, the behavior prediction display section 31 is an area in which the assistance notification icon G1 and the prediction notification icon G2 are displayed. In this case C1, a fall icon 41 is displayed in the behavior prediction display section 31. This fall icon 41 is both the assistance notification icon G1 and the prediction notification icon G2. In other words, the display of the fall icon 41 in the behavior prediction display section 31 results in assistance notification and prediction notification.

[0102] The behavior result display section 32 is an area where an icon indicating the actual behavior (behavior result) of the monitored person D after the subsequent behavior is predicted is displayed. In step S05, as shown in FIG. 4, nothing is displayed in the behavior result display section 32.

[0103] The assistance schedule display section 33 is an area that displays the name of the staff member (monitoring staff member E) who is scheduled to provide assistance to the monitored person D. In step S05, as shown in FIG. 4, the assistance schedule display section 33 displays "Undecided."

[0104] The assistance button 34 is a touch button that accepts operation input from the staff member (supervisor E) using the terminal 2 .

[0105] The image display unit 35 is an area where the above-mentioned superimposed image 11 is displayed. More specifically, as shown in FIG. 2 , the management computer 6 has an image transmission unit 60. The skeletal information generation unit 4 sends the generated superimposed image 11 to the image transmission unit 60. The image transmission unit 60 transmits the superimposed image 11 to the terminal 2. The terminal 2 displays the superimposed image 11 on the image display unit 35. In this embodiment, the superimposed image 11 is thereby displayed on the image display unit 35 in real time.

[0106] Next, the process proceeds to step S06 in Fig. 11. In step S06, the first notification unit 67 determines whether or not the assistance button 34 has been pressed.

[0107] In this case C1, it is assumed that the assistance button 34 has not been pressed when the process moves from step S05 to step S06 ("No" in step S06 in FIG. 11), and therefore the process moves to step S07.

[0108] 2, the management computer 6 has a collating unit 68 and a second notifying unit 69. In step S07 of Fig. 11, the collating unit 68 acquires the latest status information from the status acquiring unit 61. Thereafter, the process proceeds to step S08.

[0109] In step S08, the collation unit 68 determines whether the behavior result of the monitored person D has been determined. Although not particularly limited, this determination may be made based on the latest status information, or may be made based on whether a predetermined time (e.g., 10 seconds) has elapsed since the prediction unit 63 predicted the subsequent behavior in step S03. More specifically, for example, if the predetermined time (e.g., 10 seconds) has elapsed since the prediction unit 63 predicted a fall as the subsequent behavior, and if the monitored person D actually falls, the behavior result of the monitored person D may be determined to be determined.

[0110] If the behavioral result of the monitored person D has not been determined ("No" in step S08 of FIG. 11), the process returns to step S06. In this case C1, it is assumed that the assistance button 34 is pressed before the behavioral result of the monitored person D is determined. Therefore, after the process moves to step S06 for the first time, the process repeats steps S06, S07, and S08 until the assistance button 34 is pressed.

[0111] When the assistance button 34 is pressed, a predetermined signal is sent from the terminal 2 to the first notification unit 67, as shown in Fig. 2. Based on the signal, the first notification unit 67 determines that the assistance button 34 has been pressed ("Yes" in step S06 of Fig. 11), and identifies the staff member (supervisor E) who pressed the assistance button 34. Although not particularly limited, for example, the signal may include information indicating the user of the terminal 2, and the staff member (supervisor E) who pressed the assistance button 34 may be identified based on the information. In this case C1, it is assumed that staff member X pressed the assistance button 34.

[0112] Thereafter, the process proceeds to step S09. In step S09, the first notification unit 67 sends information indicating the staff member (supervisor E) who pressed the assistance button 34 to the terminal 2. Based on this information, the terminal 2 displays "Staff Member X" on the assistance schedule display unit 33, as shown in Fig. 5. At the same time, the assistance button 34 is made invisible. As a result, the assistance button 34 is placed in a state where it does not accept any operational input.

[0113] If the behavior prediction system A is equipped with a plurality of terminals 2, in step S09, the name of the staff member (supervisor E) who pressed the assistance button 34 is displayed on the assistance schedule display section 33 of each terminal 2.

[0114] Thereafter, the process proceeds to step S10. In step S10, the collation unit 68 acquires the latest state information from the state acquisition unit 61. Thereafter, the process proceeds to step S11.

[0115] In step S11, the collation unit 68 determines whether or not assistance has been provided by a staff member (monitor E). Although not particularly limited, this determination may be made based on the latest status information.

[0116] In this case C1, it is assumed that assistance is not being provided when the process moves from step S10 to step S11 ("No" in step S11 in FIG. 11), and therefore the process moves to step S12.

[0117] In step S12, the collation unit 68 determines whether or not the behavioral result of the monitored person D has been determined. This determination is the same as in step S08.

[0118] In this case C1, assistance is provided before the behavioral result of the monitored person D is determined. Therefore, after the process moves to step S10 for the first time, the process repeats steps S10, S11, and S12 until assistance is provided.

[0119] When assistance has been provided ("Yes" in step S11 of FIG. 11), the process proceeds to step S13. In step S13, as shown in FIG. 2, an assistance-completed signal indicating that assistance has been provided is sent from the collation unit 68 to the second notification unit 69. In response to the assistance-completed signal, the second notification unit 69 sends a predetermined signal to the terminal 2.

[0120] In response to the signal, the terminal 2 displays an assistance icon 43 on the behavior result display unit 32, as shown in Fig. 6. This causes the second notification unit 69 to notify the monitor E of the actual behavior (behavior result) of the monitored person D after the subsequent behavior is predicted via the touch panel 21. Note that the monitored person D being assisted by the monitor E is a specific example of "the behavior of the monitored person D." Thereafter, this notification control flow is temporarily terminated.

[0121] [Case C2] Next, a case where the result of the action of the monitored person D is determined after a fall is predicted as a subsequent action and before the assistance button 34 is pressed will be described as case C2.

[0122] The processing flow in case C2 is the same as that in case C1 up to step S05, so the description will start from the point where the processing moves from step S05 to step S06.

[0123] In this case C2, the behavioral result of the monitored person D is determined before the assistance button 34 is pressed. Therefore, after the process proceeds to step S06 for the first time, the process repeats steps S06, S07, and S08 until the behavioral result of the monitored person D is determined.

[0124] When the behavior result of the monitored person D is determined ("Yes" in step S08 of FIG. 11 ), the process proceeds to step S14. In step S14, the latest status information is sent from the collation unit 68 to the second notification unit 69. The second notification unit 69 then sends a predetermined signal to the terminal 2. In response to the signal, the terminal 2 displays an icon corresponding to the latest status information on the behavior result display unit 32. As a result, the second notification unit 69 notifies the monitor E, via the touch panel 21, of the actual behavior (behavior result) of the monitored person D after the subsequent behavior has been predicted.

[0125] For example, if the latest status information indicates a trip to the toilet, a toilet icon 42 is displayed in the behavior result display section 32, as shown in Fig. 7. Also, if the latest status information indicates a fall, a fall icon 41 is displayed in the behavior result display section 32.

[0126] In step S14, the assistance schedule display section 33 and the assistance button 34 are hidden as shown in Fig. 7. This puts the assistance button 34 in a state where it does not accept any operation input.

[0127] After that, this notification control flow ends for the time being.

[0128] In step S14, the predicted subsequent action may be compared with the confirmed action result (latest status information) by the comparison unit 68. Furthermore, information (e.g., a message) indicating whether or not the predicted subsequent action and the confirmed action result match may be displayed on the action result display unit 32 by the second notification unit 69.

[0129] In particular, if it is determined that assistance is required for the monitored person D and there is a mismatch between the predicted subsequent behavior and the confirmed behavior result (or if assistance is not actually required), information (e.g., a message) indicating the result of the matching by the matching unit 68 (in other words, that there was a mismatch) may be displayed on the behavior result display unit 32 by the second notification unit 69.

[0130] Moreover, the monitor E may be able to switch whether or not information indicating the result of the matching by the matching unit 68 is displayed by operating the terminal 2 .

[0131] [Case C3] Next, a case where the assistance button 34 is pressed after a fall is predicted as a subsequent action, but the action result of the monitored person D is determined before assistance is provided will be described as case C3.

[0132] The processing flow in case C3 is the same as that in case C1 up to step S09, so the description will start from the point where the processing moves from step S09 to step S10.

[0133] In this case C3, the behavioral result of the monitored person D is determined before assistance is provided. Therefore, after the process moves to step S10 for the first time, the process repeats steps S10, S11, and S12 until the behavioral result of the monitored person D is determined.

[0134] When the behavioral result of the monitored person D is determined ("Yes" in step S12 of FIG. 11), the process proceeds to step S14. Step S14 is as described above.

[0135] After that, this notification control flow ends for the time being.

[0136] [Case C4] Next, a case where a move to the toilet is predicted as a subsequent action will be described as Case C4.

[0137] The processing flow in case C4 is the same as that in case C1 up to step S03, so the description will begin from the point where the processing moves from step S03 to step S04.

[0138] In case C4, it is determined that assistance is not required ("No" in step S04 in FIG. 11), and the process proceeds to step S15.

[0139] In step S15, a predictive notification is made without an assistance notification by the first notification unit 67. At this time, the first notification unit 67 causes the touch panel 21 of the terminal 2 to display a notification screen shown in FIG.

[0140] 8, the notification screen at this time does not display the assistance plan display section 33 or the assistance button 34. Also, a toilet icon 42 is displayed in the behavior prediction display section 31. This toilet icon 42 is the prediction notification icon G2, but is not the assistance notification icon G1. That is, the display of the toilet icon 42 in the behavior prediction display section 31 results in a prediction notification.

[0141] Next, the process proceeds to step S16 in Fig. 11. In step S16, the collation unit 68 acquires the latest state information from the state acquisition unit 61. After that, the process proceeds to step S17.

[0142] In step S17, the collation unit 68 determines whether or not the behavioral result of the monitored person D has been determined. Although not particularly limited, this determination may be made based on the latest status information.

[0143] If the behavioral result of the monitored person D has not been determined ("No" in step S17 of FIG. 11), the process returns to step S16. That is, after the process proceeds to step S16 for the first time, the process repeats steps S16 and S17 until the behavioral result of the monitored person D is determined.

[0144] When the behavior result of the monitored person D is determined ("Yes" in step S17 of FIG. 11), the process proceeds to step S18. In step S18, the latest status information is sent from the collation unit 68 to the second notification unit 69. The second notification unit 69 then sends a predetermined signal to the terminal 2. In response to the signal, the terminal 2 displays an icon corresponding to the latest status information on the behavior result display unit 32. As a result, the second notification unit 69 notifies the monitor E, via the touch panel 21, of the actual behavior (behavior result) of the monitored person D after the subsequent behavior has been predicted.

[0145] For example, if the latest status information indicates a move to the toilet, a toilet icon 42 is displayed in the behavior result display section 32 as shown in Fig. 9. Also, if the latest status information indicates a move to another room, a room move icon 44 is displayed in the behavior result display section 32 as shown in Fig. 10.

[0146] In step S18, the predicted subsequent action may be compared with the confirmed action result (latest status information) by the comparison unit 68. Furthermore, information (e.g., a message) indicating whether or not the predicted subsequent action and the confirmed action result match may be displayed on the action result display unit 32 by the second notification unit 69.

[0147] In particular, if it is determined that assistance is not required for the monitored person D and there is a mismatch between the predicted subsequent behavior and the confirmed behavior result, the second notification unit 69 may display information (e.g., a message) indicating the result of the matching by the matching unit 68 (in other words, that there was a mismatch) and information (e.g., an icon) indicating the confirmed behavior result on the behavior result display unit 32.

[0148] According to the configuration described above, the behavior of the monitored person D is predicted, and based on the prediction result, it is determined whether or not assistance is required for the monitored person D. Then, a configuration can be realized in which a notification is issued when it is determined that assistance is required.

[0149] This allows the monitor E to understand, when a notification is issued, that assistance is required for the monitored person D. Therefore, the monitor E does not need to determine whether assistance is required for the monitored person D every time a notification is issued. As a result, the burden on the monitor E can be reduced.

[0150] That is, according to the configuration described above, in the behavior prediction system A that predicts the behavior of the monitored person D, it is possible to reduce the burden on the monitor E.

[0151] (1) The functions of the components in the above embodiments may be configured as a behavior prediction program that causes a computer to realize the functions of the components in the above embodiments. For example, the behavior prediction program may cause a computer to realize a behavior acquisition function corresponding to the state detection unit 5 and the history storage unit 62, a prediction function corresponding to the prediction unit 63, a determination function corresponding to the determination unit 64, and a notification function corresponding to the first notification unit 67.

[0152] (2) The behavior prediction method may be configured to perform the operations performed by each component in the above embodiment in one or more steps. For example, the behavior prediction method may be configured to include a behavior acquisition step corresponding to the state detection unit 5 and the history storage unit 62, a prediction step corresponding to the prediction unit 63, a determination step corresponding to the determination unit 64, and a notification step corresponding to the first notification unit 67.

[0153] (3) The behavior prediction system A may have the configuration shown in Fig. 12. In this configuration, the behavior prediction system A includes a monitoring device 1A as a behavior prediction device. The monitoring device 1A includes an image transmission unit 60, a status acquisition unit 61, a history storage unit 62, a prediction unit 63, a determination unit 64, an attribute acquisition unit 65, a setting unit 66, a comparison unit 68, and a second notification unit 69. The monitoring device 1A also includes a first notification unit 77 as an output unit. In other respects, the monitoring device 1A is similar to the monitoring device 1 in the above embodiment.

[0154] The first notification unit 77 outputs a signal indicating that assistance is required when the determination unit 64 determines that assistance is required. In other respects, the first notification unit 77 is similar to the first notification unit 67 in the above embodiment.

[0155] This signal is sent to, for example, the terminal 2. Then, in response to the signal, the terminal 2 displays the assistance notification icon G1 on the touch panel 21, similar to the above embodiment, thereby performing assistance notification.

[0156] In this way, the monitoring device 1A includes a first notification unit 77 that outputs a signal indicating that assistance is required when the determination unit 64 determines that assistance is required.

[0157] (4) The prediction unit 63 may predict the subsequent behavior based on only one of the state information and the history information.

[0158] (5) The determination unit 64 may be capable of acquiring status information from the status acquisition unit 61. The determination unit 64 may also be capable of acquiring history information from the history storage unit 62. The determination unit 64 may be configured to be capable of calculating the probability that the monitored person D will be in a state requiring assistance based on the behavioral information (more specifically, at least one of the status information and the history information), and may determine whether assistance is required based on the probability.

[0159] (6) The first notification unit 67 may be configured not to be able to provide predictive notification.

[0160] (7) The monitoring device 1 does not have to have a speaker 7.

[0161] (8) The assistance notification icon G1 may not indicate the prediction result by the prediction unit 63. For example, the assistance notification icon G1 may be a symbol (icon), such as an exclamation mark, that attracts the attention of the monitor E without indicating a specific subsequent action.

[0162] (9) When the determination unit 64 determines that assistance is not required, the first notification unit 67 may issue a predictive notification to the monitored person D.

[0163] (10) The prediction unit 63 may be configured not to predict multiple types of subsequent action candidates.

[0164] (11) The prediction unit 63 may be configured not to be able to calculate the probability of occurrence of a subsequent action.

[0165] (12) The attribute acquisition unit 65 does not have to be provided.

[0166] (13) The setting unit 66 does not have to be provided.

[0167] (14) Whether or not the behavior result display section 32 is displayed may be switchable by the monitor E's operation of the terminal 2.

[0168] The configurations disclosed in the above-described embodiments (including other embodiments, the same applies hereinafter) can be applied in combination with configurations disclosed in other embodiments, as long as no contradiction occurs. Furthermore, the embodiments disclosed in this specification are merely examples, and the embodiments of the present disclosure are not limited thereto and can be appropriately modified within the scope of the present disclosure.

[0169] The present disclosure can be used not only in elderly care facilities, but also in various other facilities such as medical facilities and public facilities, and can also be used outdoors.

Claims

1. A behavior prediction system comprising: a behavior acquisition unit that acquires behavior information, which is information indicating the behavior of a monitored person; a prediction unit that predicts a subsequent behavior of the monitored person that follows the behavior indicated by the behavior information based on the behavior information acquired by the behavior acquisition unit; a determination unit that determines whether or not assistance is required for the monitored person based on the subsequent behavior predicted by the prediction unit; and a notification unit that issues an assistance notification to notify the user that assistance is required when the determination unit determines that assistance is required.

2. The behavior prediction system of claim 1, wherein the notification unit is configured to be able to make a predictive notification to inform the user of the prediction result by the prediction unit, and when the determination unit determines that the assistance is necessary, the notification unit makes the assistance notification and the predictive notification, and when the determination unit determines that the assistance is not necessary, the notification unit does not make the assistance notification but makes the predictive notification.

3. The behavior prediction system of claim 1, wherein the notification unit is configured to issue a predictive notification to inform the user of the prediction result by the prediction unit, and when the determination unit determines that the assistance is necessary, the notification unit issues the predictive notification to the monitor and the monitored person, and when the determination unit determines that the assistance is not necessary, the notification unit issues the predictive notification to the monitor rather than to the monitored person.

4. The behavior prediction system of claim 1, wherein the notification unit is configured to be able to perform a prediction notification to notify the user of a prediction result by the prediction unit, the prediction unit is configured to be able to predict multiple types of candidates for the subsequent behavior, and when multiple types of candidates for the subsequent behavior are predicted by the prediction unit, the notification unit performs a multiple candidate notification as the prediction notification to notify the user of the multiple types of candidates for the subsequent behavior.

5. The behavior prediction system of claim 1, wherein the notification unit is configured to be able to make a prediction notification to inform the user of the prediction result by the prediction unit, the prediction unit is configured to be able to calculate the probability of occurrence of the subsequent behavior, and the notification unit also notifies the user of the occurrence probability when making the prediction notification.

6. A behavior prediction system as described in any one of claims 1 to 5, wherein the judgment unit is configured to calculate the probability that the monitored person will be in a state requiring the assistance based on the behavioral information, and judges whether or not the assistance is required based on the probability.

7. A behavior prediction system as described in any one of claims 1 to 5, further comprising an attribute acquisition unit that acquires attribute information which is information indicating attributes of the monitored person, and the judgment unit judges whether or not assistance is required based on the subsequent behavior predicted by the prediction unit and the attribute information acquired by the attribute acquisition unit.

8. A behavior prediction system as described in any one of claims 1 to 5, further comprising a setting unit capable of setting criteria regarding the need for assistance, wherein the judgment unit judges the need for assistance based on the subsequent behavior predicted by the prediction unit and the criteria set by the setting unit.

9. A behavior prediction program that causes a computer to realize the following functions: a behavior acquisition function that acquires behavior information, which is information indicating the behavior of a monitored person; a prediction function that predicts a subsequent behavior of the monitored person that follows the behavior indicated by the behavior information based on the behavior information acquired by the behavior acquisition function; a judgment function that determines whether or not assistance is required for the monitored person based on the subsequent behavior predicted by the prediction function; and a notification function that issues an assistance notification to notify the user that assistance is required when the judgment function determines that assistance is required.

10. A behavior prediction method in a behavior prediction system, comprising: a behavior acquisition step of acquiring behavior information, which is information indicating the behavior of a monitored person; a prediction step of predicting a subsequent behavior of the monitored person that follows the behavior indicated by the behavior information based on the behavior information acquired by the behavior acquisition step; a determination step of determining whether or not assistance is required for the monitored person based on the subsequent behavior predicted by the prediction step; and a notification step of issuing an assistance notification to notify the monitored person that assistance is required if it is determined by the determination step that assistance is required.

11. A behavior prediction device comprising: a behavior acquisition unit that acquires behavior information, which is information indicating the behavior of a monitored person; a prediction unit that predicts a subsequent behavior of the monitored person that follows the behavior indicated by the behavior information based on the behavior information acquired by the behavior acquisition unit; a judgment unit that judges whether or not assistance is required for the monitored person based on the subsequent behavior predicted by the prediction unit; and an output unit that outputs a signal indicating that the assistance is required when the judgment unit judges that the assistance is required.

Images