Monitoring system, monitoring device, monitoring method, and program

Abstract

To provide a monitoring system, a monitoring device, a monitoring method, and a program for enabling a user to appropriately grasp a behavior of an object person being a monitoring target object.SOLUTION: A monitoring system includes: a terminal device 2 carried by an object person being the object to be monitored by a user; a behavior message acquisition section 111 for acquiring behavior message identification information to identify a behavior message which indicates the behavior of the object person generated by the terminal device 2 in response to an operation of the object person with respect to the terminal device 2; a device state acquisition section 112 for acquiring device state information which indicates a state of each one of devices 4 owned by the object person; an elevator use state acquisition section 121 for acquiring elevator use state information which indicates a use state of the object person on an elevator which the object person uses; and an object person state estimation section 113 for determining a state of the object person based on the behavior message identification information, the device state information, and the elevator use state information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a monitoring system, a monitoring device, a monitoring method, and a program. [Background technology]

[0002] A message generation device has been proposed that determines whether an event has occurred and the type of event that has occurred by referring to power information, which is information about the power consumed in a home, the status of home appliances installed in the home, and sensor information obtained from sensors installed in the home, and determines the status of the person being monitored based on the determined event and time, and generates a message regarding the behavior of the person based on the determined status (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-060382 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the message generation device described in Patent Document 1 generates messages regarding the target person's behavior based only on the power consumed in the home and events that occur in devices such as home appliances, so if there is a mismatch between the correlation between the events occurring in the devices and the target person's actual behavior, there is a risk that an incorrect message regarding the target person's behavior will be generated, which could cause the person watching over the target person to be unable to properly understand the target person's behavior.

[0005] The present disclosure has been made in consideration of the above reasons, and aims to provide a monitoring system, monitoring device, monitoring method, and program that allow a user to properly understand the behavior of a subject that is the object of monitoring. [Means for solving the problem]

[0006] In order to achieve the above object, the monitoring system according to the present disclosure includes: a first terminal device carried by a person to be monitored by a user; a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message that indicates the behavior of the subject, the behavior message being generated by the first terminal device in response to an operation of the subject on the first terminal device; a device status acquisition unit that acquires device status information indicating the status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the target person; The elevator control system further includes a subject status estimation unit that estimates the subject's status based on the action message identification information, the equipment status information, and the elevator usage status information. [Effects of the Invention]

[0007] According to the present disclosure, a subject status estimation unit estimates the status of a subject based on behavioral message identification information from a first terminal device carried by the subject, who is the subject of monitoring, device status information for at least one device, and elevator usage status information indicating the subject's elevator usage status. This allows the subject's status to be estimated taking into account the subject's actual behavioral history, thereby enabling accurate estimation of whether the subject's status is abnormal. This allows the user to appropriately understand the behavior of the person being monitored. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic configuration diagram of a monitoring system according to an embodiment of the present disclosure; [Figure 2] FIG. 1 is a block diagram showing a hardware configuration of a monitoring system according to an embodiment. [Figure 3] FIG. 1 is a block diagram showing a functional configuration of a monitoring system according to an embodiment. [Figure 4]1A is a diagram showing an example of information stored in an action message storage unit according to an embodiment, and FIG. 1B is a diagram showing an example of information stored in a device status storage unit according to an embodiment. [Figure 5] FIG. 10 is a diagram showing an example of information stored in an elevator usage status storage unit according to an embodiment. [Figure 6] FIG. 1 is a diagram for explaining a subject situation estimation model according to an embodiment. [Figure 7] FIG. 10 is a diagram for explaining the operation of a subject situation estimation unit according to an embodiment. [Figure 8] FIG. 1 is a sequence diagram illustrating an operation of a monitoring system according to an embodiment. [Figure 9] FIG. 10 is a diagram showing an example of a portion of an operation screen image displayed on a display unit of a terminal device carried by a user according to an embodiment; [Figure 10] FIG. 1 is a sequence diagram illustrating an operation of a monitoring system according to an embodiment. [Figure 11] FIG. 1 is a sequence diagram illustrating an operation of a monitoring system according to an embodiment. [Figure 12] 10 is a flowchart showing an example of the flow of a monitoring process executed by a cloud server according to an embodiment. [Figure 13] 10 is a flowchart showing an example of the flow of a monitoring process executed by a cloud server according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] A monitoring system according to an embodiment of the present disclosure will be described below with reference to the drawings. The monitoring system according to the embodiment includes a first terminal device carried by a subject person to be monitored by a user, a behavior message acquisition unit that acquires behavior message identification information that identifies behavior message information indicating the subject person's behavior generated by the terminal device in response to the subject person's operation on the first terminal device, a device status acquisition unit that acquires device status information indicating the status of at least one device carried by the subject person, an elevator usage status acquisition unit that acquires elevator usage status information indicating the subject person's usage status of an elevator used by the subject person, and a subject person status estimation unit that estimates the subject person's status based on the behavior message identification information, the device status information, and the elevator usage status information.

[0010] As shown in FIG. 1, the monitoring system according to this embodiment includes a cloud server 1, terminal devices 2 and 3, and a device 4, and is used by a user U2 carrying the terminal device 3 to monitor a monitored person U1 carrying the terminal device 2. The device 4 is connected to a local network NW2. The local network NW2 is, for example, a wired local area network (LAN) or a wireless LAN, and is connected to a wide area network NW1 via a router 82 and a data circuit-terminating device 81. The data circuit-terminating device 81 is, for example, a modem, a gateway, or the like. This enables the device 4 to communicate with the cloud server 1 via the local network NW2, the router 82, the data circuit-terminating device 81, and the wide area network NW1. An elevator control server 8 that manages the elevator 5 used by the monitored person U1, and an image generation server 9 are also connected to the wide area network NW1.

[0011] The elevator 5 is installed in, for example, an apartment building where the target person U1 lives. When the elevator 5 acquires terminal device identification information identifying the terminal device 2 transmitted from the terminal device 2 carried by the target person U1 who has entered the elevator 5, the elevator 5 generates elevator usage status notification information including the acquired terminal device identification information and elevator usage status information indicating the usage status of the elevator 5. Here, the elevator usage status information includes boarding floor information indicating the floor where the target person U1 boarded the elevator and disembarking floor information indicating the floor where the target person U1 disembarked from the elevator 5. The elevator 5 then has a function of transmitting the generated elevator usage status notification information to the elevator management server 8 via the wide area network NW1.

[0012] The elevator management server 8 includes a user information storage unit (not shown) that stores terminal device identification information of the terminal device 2 carried by the target person U1 and user identification information identifying the target person U1 who is using the elevator 5, in association with each other, and an elevator usage status storage unit (not shown) that stores the above-mentioned elevator usage status information in association with the user identification information of the user who uses the elevator 5. The elevator usage status storage unit also stores time information indicating the time when the user used the elevator 5 in association with the elevator usage status information. When the elevator management server 8 acquires elevator usage status notification information transmitted from the elevator 5, it extracts the terminal device identification information and elevator usage status information included in the acquired elevator usage status notification information. The elevator management server 8 then identifies the user identification information stored in the user information storage unit that corresponds to the extracted terminal device identification information, and stores the extracted elevator usage status information, the identified user identification information, and time information indicating the time when the elevator usage status notification information was acquired in association with each other in the elevator usage status storage unit. Here, the elevator control server 8 manages one day in 24 unit time slots of one hour length, and stores time information indicating the start time of the unit time slot that includes the time when the elevator usage status notification information was acquired in the elevator usage status storage unit in association with the extracted elevator usage status information. Also, every time the elevator 5 is used, the elevator control server 8 generates elevator usage status notification information including user identification information of the user of the elevator 5, elevator usage status information, and time information, and transmits it to the cloud server 1.

[0013] When the image generation server 9 receives the abnormality notification information (described below) transmitted from the cloud server 1, it extracts the equipment status information, elevator usage status information, and action message identification information (described below) contained in the received abnormality notification information. Furthermore, the image generation server 9 generates an estimated situation image that estimates the target person's situation based on the extracted equipment status information, elevator usage status information, and action message identification information. Here, the image generation server 9 generates the estimated situation image using, for example, Generative AI. Specifically, the image generation server 9 generates the estimated situation image using an algorithm such as Transformer, Bidirectional Encoder Representations from Transformers (BERT), or Generative Pre-Training (GPT). The image generation server 9 then transmits estimated situation image notification information indicating the generated estimated situation image to the cloud server 1.

[0014] The devices 4 include, for example, an air conditioner 4A, a refrigerator 4B, and a water heater 4C, and generate device status notification information including device status information indicating the device's status and device identification information identifying the device each time a preset device status notification period arrives. The devices 4 then transmit the generated device status notification information to the cloud server 1. Here, the device status notification period arrives periodically, and can be set to arrive, for example, every hour.

[0015] The terminal device 2 is, for example, a smartphone. As shown in FIG. 2 , the terminal device 2 is a first terminal device including a CPU (Central Processing Unit) 201, a main memory 202, an auxiliary memory 203, a display 204, an input unit 205, a communication unit 206, and a bus 209 connecting these components to one another. The main memory 202 has a volatile memory such as a RAM (Random Access Memory) and is used as a work area for the CPU 201. The main memory 202 also has an image-only memory (not shown) that temporarily stores image information to be displayed on the display 204. The auxiliary memory 203 is a non-volatile memory such as a semiconductor flash memory and stores programs for the CPU 201 to execute various processes. The display 204 is a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The input unit 205 is, for example, a transparent touchpad placed on the display 204 and is a first terminal device input unit. The communication unit 206 is connected to the wide area network NW1 and transmits information transferred from the CPU 201 to the cloud server 1 via the wide area network NW1, and transfers information obtained from the cloud server 1 via the wide area network NW1 to the CPU 201.

[0016] The CPU 201 reads out the programs stored in the auxiliary storage unit 203 into the main storage unit 202 and executes them, thereby functioning as an action message notification unit 212, a confirmation message acquisition unit 213, a display control unit 215, and a confirmation response message notification unit 216, as shown in Fig. 3. The auxiliary storage unit 203 shown in Fig. 2 also has a confirmation message storage unit 231, as shown in Fig. 3. The confirmation message storage unit 231 stores confirmation message information transmitted from the terminal device 3. This confirmation message information is information including a message for the user U2 to confirm with the target person U1 whether or not an abnormality has actually occurred, for example, when an abnormality notification message is transmitted from the cloud server 1 to the terminal device 3 possessed by the user.

[0017] When the target person U1 performs a behavior notification operation on the input unit 205 to select a behavior message indicating the target person U1's behavior and notify the cloud server 1, the behavior message notification unit 212 accepts the behavior notification operation, specifies behavior message identification information for identifying the selected behavior message, and transmits the behavior message identification information to the cloud server 1. Here, the behavior message may be, for example, a message representing the target person U1's behavior, such as "Good morning," "I'm going out," "I'm home," or "Good night." Then, the behavior message notification unit 212 specifies behavior message identification information for identifying the selected behavior message from among the behavior message identification information preset for each behavior message, and transmits the behavior message identification information to the cloud server 1.

[0018] When the confirmation message acquisition unit 213 acquires confirmation message information transmitted from the terminal device 3, it stores the acquired confirmation message information in the confirmation message storage unit 231. When the confirmation message information is stored in the confirmation message storage unit 231, the display control unit 215 forms a confirmation message image for the target person U1 to confirm the appropriateness of the abnormality occurrence notification based on the confirmation message information. Then, the display control unit 215 causes the display unit 204 to display the formed confirmation message image.

[0019] When the target person U1 performs an acknowledgment operation via the input unit 205 to select a response message corresponding to the above-mentioned confirmation message information and notify the cloud server 1, the acknowledgment message notification unit 216 accepts the acknowledgment operation, specifies response message identification information that identifies the selected acknowledgment message, and transmits it to the cloud server 1. As the acknowledgment message, a message that represents the response content of the target person U1 is adopted, such as the response message "That's right" in response to the confirmation message from user U2 asking "Did you forget to turn off the air conditioner?" Then, the action message notification unit 212 specifies response message identification information that identifies the selected acknowledgment message from among the response message identification information that is preset for each acknowledgment message, and transmits it to the cloud server 1.

[0020] The terminal device 3 is, for example, a smartphone and has the same hardware configuration as the terminal device 2. As shown in FIG. 2, the terminal device 3 is a second terminal device including a CPU 301, a main memory 302, an auxiliary memory 303, a display 304, an input unit 305, a communication unit 306, and a bus 309 connecting these units to one another. The input unit 305 is, for example, a transparent touchpad disposed on the display 304 as an input unit of the second terminal device. The CPU 301 reads a program stored in the auxiliary memory 303 into the main memory 302 and executes it, thereby functioning as an abnormality notification acquisition unit 311, a display control unit 313, a confirmation message generation unit 315, a confirmation message transmission unit 316, a status acquisition unit 317, and an action notification information acquisition unit 318, as shown in FIG. 3. The auxiliary memory 203 shown in FIG. 2 also includes an abnormality notification storage unit 331, a status storage unit 332, and an action message storage unit 333, as shown in FIG. 3. The abnormality notification storage unit 331 stores abnormality notification message information included in the abnormality notification information transmitted from the cloud server 1 and abnormality occurrence time information included in the abnormality notification information, which indicates the time when the abnormality occurred, in association with each other.

[0021] The status storage unit 332 stores device status information indicating the device status of each device 4 in association with time information corresponding to the device status notification period. The status storage unit 332 also stores the elevator usage status information in association with the time information. The action message storage unit 333 stores the action message identification information included in the action notification information transmitted from the cloud server 1 in association with time information indicating the time when the cloud server 1 acquired the action message identification information, in association with each other.

[0022] When the abnormality notification acquisition unit 311 acquires the abnormality notification information transmitted from the cloud server 1, it extracts the aforementioned abnormality notification message information and abnormality occurrence time information contained in the acquired abnormality notification information, associates them with each other, and stores them in the abnormality notification storage unit 331.

[0023] The display control unit 313 forms a target person status notification image for notifying the user U2 of the target person U1's status, based on the equipment status information, elevator usage status information, and time information stored in the status storage unit 332, and the action message information and time information stored in the action message storage unit 333. Furthermore, when the abnormality notification storage unit 331 stores the abnormality notification message information and the abnormality occurrence time information, the display control unit 313 forms an abnormality occurrence notification image for notifying the target person U1 that an abnormality has occurred, based on the abnormality notification message information and the abnormality occurrence time information, and combines the image with the target person status notification image. The display control unit 313 causes the display unit 304 to display the target person status notification image and the abnormality occurrence notification image formed by the image forming unit 312.

[0024] When the user U2 inputs a confirmation message for confirming whether an abnormality has actually occurred for the target person U1 via the input unit 305 and performs an action confirmation operation to transmit the message to the terminal device 2, the confirmation message generation unit 315 accepts the action confirmation operation and generates confirmation message information indicating the confirmation message. The confirmation message transmission unit 316 then transmits the generated confirmation message information to the terminal device 2 and the cloud server 1. When the status acquisition unit 317 acquires the situation history notification information transmitted from the cloud server 1, it extracts the device status information and the corresponding time information, and the elevator usage status information and the corresponding time information, from the acquired situation history notification information, and stores the extracted information in the status storage unit 332 in association with each other. When the action notification information acquisition unit 318 acquires the action notification information transmitted from the cloud server 1, it extracts the action message information and the corresponding time information from the acquired action notification information, and stores the extracted action message information and the extracted time information in the action message storage unit 333 in association with each other.

[0025] Returning to FIG. 2, the cloud server 1 includes a CPU 101, a main memory unit 102, an auxiliary memory unit 103, a communication unit 106, a clock unit 108, and a bus 109 interconnecting these units. The CPU 101 is, for example, a multi-core processor. The main memory unit 102 is made up of volatile memory and is used as a work area for the CPU 101. The auxiliary memory unit 103 is made up of large-capacity non-volatile memory and stores programs for realizing various functions of the cloud server 1. The communication unit 106 is connected to the wide area network NW1. The clock unit 108 includes, for example, a real-time clock.

[0026] The CPU 101 reads out the programs stored in the auxiliary storage unit 103 into the main storage unit 102 and executes them, thereby functioning as an action message acquisition unit 111, an equipment status acquisition unit 112, a subject person status estimation unit 113, an abnormality notification information generation unit 114, an abnormality notification information transmission unit 115, a message acquisition unit 116, a validity determination unit 117, a model update unit 118, a status notification unit 119, and an action notification unit 120, as shown in Fig. 3. The auxiliary storage unit 103 shown in Fig. 2 also includes an action message storage unit 131, an equipment status storage unit 132, a subject person status estimation model storage unit 133, a content flag storage unit 134, an abnormality notification message storage unit 135, and an elevator usage status storage unit 136, as shown in Fig. 3. The auxiliary storage unit 103 shown in Fig. 2 also includes a user information storage unit (not shown) that stores device identification information for each device 4 in association with user identification information that identifies the user of the device 4. As shown in FIG. 4A, for example, the behavioral message storage unit 131 stores behavioral message identification information that identifies a behavioral message indicating the behavioral content of the target person U1 in association with time information indicating the time when the behavioral message identification information was acquired by the cloud server 1. The behavioral message identification information indicates, for example, an identification number uniquely assigned to each behavioral message. The time information indicates the start time of the unit time period that includes the time when the behavioral message notification information was acquired when a day is divided into 24 unit time periods of one hour. The example shown in FIG. 4A indicates that the target person U1 selected the behavioral message "Good morning" in the time period between 6:00 and 7:00.

[0027] As shown in FIG. 4(B), for example, the device status storage unit 132 stores device status information indicating the device status of each device 4 installed in the residence H where the subject U1 resides, in association with device identification information that identifies the device 4 and time information indicating the time at which the device entered the state indicated by the device status information. Here, the time information indicates the start time of each unit time slot when a day is divided into 24 unit time slots of one hour. The example shown in FIG. 4(B) indicates that the air conditioner is "ON" at 6:00, and that the refrigerator door is opened and closed between 6:00 and 7:00, and no hot water is being supplied from the water heater.

[0028] The elevator usage status storage unit 136 stores elevator usage status information indicating the usage status of the elevator 5 by each user, in association with time information and user identification information, as shown in FIG. 5, for example. The elevator usage status information includes the aforementioned boarding floor information and disembarking floor information. The time information indicates the start time of a unit time period that includes the time at which the user used the elevator, when a day is divided into 24 unit time periods of one hour each. The example shown in FIG. 5 indicates that a user identified by user identification information IDU[0] boarded the elevator 5 on the fifth floor and disembarked on the first floor between 6:00 and 7:00, and then boarded the elevator 5 on the first floor at 11:00 and disembarked on the fifth floor.

[0029] Returning to FIG. 3 , the subject person status estimation model storage unit 133 stores subject person status estimation model information indicating a subject person status estimation model used to estimate the status of the subject U1 from the aforementioned behavioral message identification information, device status information, and elevator usage status information. The subject person status estimation model is, for example, a forward propagation neural network. In this case, the subject person status estimation model storage unit 133 stores information indicating the structure of the neural network and information indicating weighting coefficients in the neural network. Here, the information indicating the structure of the neural network includes information indicating the number of nodes and layers of the neural network, as well as weighting coefficients and activation functions corresponding to each node. As shown in FIG. 6 , this neural network has an input layer L10, a hidden layer L20, and an output layer L30. The device status information of each device 4 stored in the state storage unit 332 and the behavioral message identification information stored in the behavioral message storage unit 333 are input to the input layer L10.

[0030] The hidden layer L20 is composed of N (N is a positive integer) layers, each containing a preset number M[j] of nodes x[j,i] (1≦i≦M[j], M[j] is a positive integer). That is, the hidden layer L20 has a structure in which each node column is connected to another. Here, the output y[j,i] of each node x[j,i] is expressed by the relational expression (1) below.

number

[0031] Returning to Fig. 3, the content flag storage unit 134 stores a plurality of preset types of response message identification information and content flag information in association with each other. Here, the content flag information is information indicating whether the confirmation response message indicated by the response message identification information is a positive message or a negative message in response to the confirmation message. The abnormality notification message storage unit 135 stores time information corresponding to the abnormality notification message information included in the abnormality notification information generated by the abnormality notification information generation unit 114 in association with the response message identification information indicating the confirmation response message of the target person U1 in response to the abnormality message information and the above-mentioned content flag information.

[0032] When the action message acquisition unit 111 acquires the action message notification information transmitted from the terminal device 2, it extracts the action message identification information and the target person identification information included in the acquired action message notification information. Then, the action message acquisition unit 111 stores the extracted action message identification information and the target person identification information in the action message storage unit 131 in association with time information indicating the time when the action message notification information was acquired. Here, the time information indicates the start time of the unit time period that includes the time when the action message notification information was acquired when one day is divided into 24 unit time periods of one hour length.

[0033] When the device status acquisition unit 112 acquires device status notification information transmitted from the device 4, it extracts the device status information and device identification information included in the acquired device status notification information. Furthermore, the device status acquisition unit 112 identifies the user identification information associated with the extracted device identification information from the user identification information stored in the user information storage unit. Then, the device status acquisition unit 112 stores the extracted device status information and device identification information in the device status storage unit 132 in association with time information indicating the time when the device status notification information was acquired and the user identification information of the target person U1 who is the user of the device 4 identified by the identified device identification information. Here, the time information indicates the start time of the unit time slot that includes the time when the device status notification information was acquired, when a day is divided into 24 unit time slots of one hour length.

[0034] When elevator usage status acquisition unit 121 acquires elevator usage status notification information transmitted from elevator management server 8, it extracts elevator usage status information, time information, and user identification information included in the acquired elevator usage status notification information, associates the extracted information with each other, and stores the information in elevator usage status storage unit 136. Furthermore, when elevator usage status acquisition unit 121 acquires multiple pieces of elevator usage status notification information within the aforementioned unit time period, elevator usage status acquisition unit 121 overwrites the elevator usage status information, time information, and user identification information included in the previously acquired elevator usage status notification information and already stored in elevator usage status storage unit 136 with the elevator usage status information, time information, and user identification information included in the later acquired elevator usage status notification information, and stores them in elevator usage status storage unit 136. Therefore, when elevator usage status acquisition unit 121 acquires multiple pieces of elevator usage status notification information within the aforementioned unit time period, elevator usage status storage unit 136 stores the elevator usage status information, time information, and user identification information included in the last acquired elevator usage status notification information.

[0035] Each time a predetermined subject person status estimation time for estimating the status of the subject U1 arrives, the subject person status estimation unit 113 estimates the status of the subject U1 from the most recent behavioral message identification information, the most recent equipment status information, and the most recent elevator usage status information using the subject person status estimation model indicated by the subject person status estimation model information stored in the subject person status estimation model storage unit 133. Specifically, the subject person status estimation unit 113 first identifies the behavioral message identification information and time information for the most recent 24 hours corresponding to the subject person identification information of the subject U1 stored in the behavior message storage unit 131, the equipment status information and time information for the most recent 24 hours corresponding to the user identification information of the subject U1 stored in the equipment status storage unit 132, and the elevator usage status information and time information for the most recent 24 hours corresponding to the user identification information of the subject U1 stored in the elevator usage status storage unit 136. Then, based on the identified behavioral message identification information, the device status information of each device 4, and the elevator usage status information, the subject status estimation unit 113 generates a vector Vin consisting of an element indicating the behavioral message identification information, an element representing the device status indicated by the device status information, and an element representing the usage status of the elevator 5 indicated by the elevator usage status information, as shown in Figure 7, for example. 7, vector Vin includes elements IN[0,0],..., IN[0,23], IN[1,0]... that numerically represent the ON / OFF states of air conditioner 4A and refrigerator 4B in each of 24 unit time slots in the most recent 24 hours, action message identification information IN[j,0]... that numerically represent the action messages of subject U1 in each of 24 unit time slots in the most recent 24 hours, and elements IN[j+k,0]... (j and k are positive integers) that numerically represent the usage status of elevator 5 by subject U1 in each of 24 unit time slots in the most recent 24 hours. The elements IN[0,0]... that represent the ON / OFF states of device 4 are set to "1" if device 4 is ON and to "0" if device 4 is OFF, for example. The elements IN[j,0], . . . representing the action message identification information are set to, for example, the identification number indicated by the above-mentioned action message identification information.The elements IN[j+k,0],..., representing the usage status of the elevator 5, are set to one of different numerical values ​​representing each combination of the floor where the elevator 5 is boarded and the floor where the elevator 5 is disembarked. For example, if the combination of the boarding floor and the disembarking floor is "floor 5" and "floor 1," the element is set to "p" (p is a positive integer), and if it is "floor 1" and "floor 5," the element is set to "p+q" (q is a positive integer). The subject situation estimation unit 113 then inputs the vector Vin into the subject situation estimation model described above to calculate the aforementioned expected value vector Vout, which has multiple preset types of subject situations as elements. The subject situation estimation unit 113 then identifies the status of the subject U1 corresponding to the element with the highest expected value among the multiple elements of the calculated expected value vector Vout. In the example shown in FIG. 7, when the status of the subject U1 is normal, the value of the element Q[0] of the expected value vector Vout is the highest. On the other hand, if the subject U1's situation is one of several types of abnormal situations set in advance, for example, if the subject U1 forgets to turn off the air conditioner 4A, if the subject U1's physical condition is abnormal, or if the subject U1 communicates, the values ​​of the elements Q[1], Q[m], and Q[m+n] (m and n are positive integers) will be highest, respectively. Then, the subject situation estimation unit 113 determines whether the identified subject U1's situation is normal or one of several types of abnormal situations set in advance. Furthermore, if the estimated subject U1's situation is an abnormal situation, the subject situation estimation unit 113 notifies the abnormality notification information generation unit 114 of subject situation information indicating the content of the abnormal situation and corresponding time information.

[0036] Returning to FIG. 3 , when the subject person status estimation unit 113 determines that the subject U1 is in an abnormal situation, the abnormality notification information generation unit 114 generates image generation request information including equipment status information, elevator usage status information, and behavior message identification information at the time of the determination. The abnormality notification information generation unit 114 also transmits the generated image generation request information to the image generation server 9 to acquire estimated situation image information transmitted from the image generation server 9. Based on the subject person status information transmitted from the subject person status estimation unit 113, the abnormality notification information generation unit 114 generates abnormality notification information including abnormality notification message information notifying the subject U1 of the abnormal situation, abnormality occurrence time information indicating the time when the subject U1 became in an abnormal situation, and the acquired estimated situation image information. The abnormality notification information transmission unit 115 then transmits the generated abnormality notification information to the terminal device 3. At this time, the abnormality notification information transmission unit 115 notifies the message acquisition unit 116 of the abnormality notification message information and the corresponding time information. After being notified of the abnormality notification message information and the time information, message acquirer 116 acquires the confirmation message information transmitted from terminal device 3, and then acquires the response message identification information transmitted from terminal device 2. Message acquirer 116 identifies content flag information corresponding to the acquired response message identification information from the content flag information stored in content flag storage unit 134. Message acquirer 116 then stores the notified abnormality notification message information and the corresponding time information, the acquired response message identification information, and the identified content flag information in abnormality notification message storage unit 135, associating them with each other.

[0037] The validity determination unit 117 determines whether the abnormality notification message information is valid based on the content flag information corresponding to the abnormality notification message information stored in the abnormality notification message storage unit 135. If the content flag information indicates a negative message for the confirmation message, the validity determination unit 117 determines that the abnormality notification message information is invalid, and notifies the model update unit 118 of the time information stored in the abnormality notification message storage unit 135 that corresponds to the abnormality notification message information.

[0038] When the model update unit 118 is notified of time information corresponding to the abnormality notification message information determined to be invalid by the validity determination unit 117, the model update unit 118 identifies the action message identification information, equipment status information, and elevator usage status information corresponding to the notified time information from the action message identification information stored in the action message storage unit 131 and the equipment status information stored in the equipment status storage unit 132. Then, the model update unit 118 generates a new subject person situation estimation model using the identified action message identification information, equipment status information, and elevator usage status information, and updates the subject person situation estimation model information stored in the subject person situation estimation model storage unit 133 with subject person situation estimation model information indicating the generated subject person situation estimation model. Specifically, the model update unit 118 first sets the expected values ​​constituting each element of the aforementioned expected value vector so that the expected value when the subject U1 is in a normal situation is higher than the expected value when other subjects U1 are in an abnormal situation. For example, the model updating unit 118 sets the expected value when the subject U1 is in a normal situation to a number greater than 0, and sets the expected value when the other subject U1 is in an abnormal situation to "0." Then, the model updating unit 118 calculates the error between the expected value vector calculated by the subject situation estimation unit 113 and the set expected value vector. Then, the model updating unit 118 determines a weighting coefficient of the subject situation estimation model by an error propagation method (backpropagation) based on the calculated error. Then, the model updating unit 118 updates the subject estimation model information stored in the subject situation estimation model storage unit 133 with subject estimation model information indicating the determined weighting coefficient.

[0039] The status notification unit 119 generates status history notification information including time information corresponding to the device status information stored in the device status storage unit 132 and elevator usage status information corresponding to the user identification information of the target person U1 stored in the elevator usage status storage unit 136, and time information, every time a predetermined target person status history notification period arrives, and transmits the generated status history notification information to the terminal device 3. Here, the target person status history notification period can be set to arrive, for example, every 30 minutes. Every time the behavior message storage unit 131 stores new behavior message identification information, the behavior notification unit 120 generates behavior notification information including behavior message information identified by the newly stored behavior message identification information and time information corresponding to the aforementioned behavior message identification information. Then, the behavior notification unit 120 transmits the generated behavior notification information to the terminal device 3.

[0040] Next, the operation of the monitoring system according to this embodiment will be described with reference to FIGS. 8 to 11. First, as shown in FIG. 8, when a preset device status notification time arrives, the device 4 generates device status notification information including device status information indicating its own status and device identification information of the device (step S1). Then, the generated device status notification information is transmitted from the device 4 to the cloud server 1 (step S2). Meanwhile, when the cloud server 1 acquires the device status notification information, it extracts the device status information and device identification information included in the acquired device status notification information. Then, the cloud server 1 stores the extracted device status information and device identification information in the device status storage unit 132 in association with time information indicating the time when the device status notification information was acquired (step S3). Thereafter, the series of processes from steps S1 to S3 is repeatedly executed every time the device status notification time arrives.

[0041] Furthermore, when the target person U1 uses the elevator 5, the elevator 5 acquires the terminal device identification information of the terminal device 2 transmitted from the terminal device 2 possessed by the target person U1, generates elevator usage status notification information including the acquired terminal device identification information and elevator usage status information, and transmits the information to the elevator management server 8. Here, the elevator management server 8 extracts the terminal device identification information and elevator usage status information contained in the acquired elevator usage status notification information, identifies user identification information corresponding to the extracted terminal device identification information, and stores the extracted elevator usage status information, the identified user identification information, and time information indicating the time when the elevator usage status notification information was acquired in the elevator usage status storage unit 136 in association with each other. Then, the elevator management server 8 generates elevator usage status notification information including the user identification information of the user of the elevator 5, the elevator usage status information, and time information (step S4). Next, the generated elevator usage status notification information is transmitted from the elevator management server 8 to the cloud server 1 (step S5). Meanwhile, when the cloud server 1 acquires the elevator usage status notification information, it extracts the elevator usage status information, the user identification information of the target person U1, and the time information contained in the acquired elevator usage status notification information. Then, the cloud server 1 associates the extracted elevator usage status information, user identification information, and time information with each other and stores them in the elevator usage status storage unit 136 (step S6). Thereafter, the series of processes from steps S4 to S6 are repeatedly executed every time the target person U1 uses the elevator 5.

[0042] Furthermore, when a preset target person status history notification time arrives, the cloud server 1 generates status history notification information including the device identification information, device status information, and time information stored in the device status storage unit 132, and the elevator usage status information and time information corresponding to the user identification information of the target person U1 stored in the elevator usage status storage unit 136 (step S7), and the generated status history notification information is transmitted from the cloud server 1 to the terminal device 3 (step S8). Meanwhile, upon acquiring the status history notification information, the terminal device 3 extracts the device status information and the corresponding time information, and the elevator usage status information and the corresponding time information, included in the acquired status history notification information, and stores the extracted device status information and time information and the elevator usage status information and time information in the status storage unit 332 in association with each other. Then, based on the device status information, elevator usage status information, and time information stored in the status storage unit 332, the terminal device 3 forms a target person status notification image for notifying the user U2 of the status of the target person U1, and displays the image on the display unit 304 (step S9). Here, the terminal device 3 displays, for example, a target person status notification image GA2 as shown in FIG. 9(A) on the display unit 304. The target person status notification image GA2 includes an icon image IC211 representing each device 4, an icon image IC212 representing the elevator 5, a timeline image GA2111 showing the status of each device 4 in chronological order, and a timeline image GA2112 showing the usage status of the elevator 5 in chronological order. The icon images IC211 represent the air conditioner 4A and the refrigerator 4B, respectively. The timeline image GA2111 represents rectangular icon images indicating the on / off state of the air conditioner 4A in each time period and whether or not the door of the refrigerator 4B has been opened or closed in each time period, arranged along a time axis. Furthermore, the timeline image GA2112 represents a plurality of rectangular icon images arranged along a time axis display, including a rectangular icon image IC21121 indicating the time period when the subject U1 used the elevator 5 to go out, and a rectangular icon image IC21122 indicating the time period when the subject U1 used the elevator 5 to go home.Here, the icon images IC21121 and IC21122 may be displayed in different colors, such as a combination of orange and green, so as to distinguish the purpose for which the target person U1 used the elevator 5. Furthermore, if the target person U1 lives in an apartment building where the elevator 5 is installed, the user U2 can grasp the information such as "what floor the target person U1 got on and what floor the target person U1 got off on," and can therefore determine whether the target person U1 has left home or returned home.

[0043] Next, suppose that the subject U1 performs the above-mentioned action notification operation on the input unit 205 of the terminal device 2. Here, the terminal device 2 displays, for example, an operation screen image GA1 as shown in FIG. 9(B) on the display unit 204. On the operation screen image GA1, button images BU11, BU12, BU13, and BU14 are arranged at positions corresponding to the portions of the input unit 205 that the subject U1 touches when selecting an action message. Also, on the operation screen image GA1, a button image BU15 is arranged at a position corresponding to the portion of the input unit 205 that the subject U1 touches when transitioning the operation screen image GA1 to a preset initial operation screen image. With the operation screen image GA1 displayed on the display unit 204 of the terminal device 2, the subject U1 can select an action message such as "Good morning," "I'm going to the hospital," "I'm home," or "Good night" by touching the portions of the input unit 205 that correspond to the button images BU11, BU12, BU13, and BU14.

[0044] Returning to FIG. 8, the terminal device 2 specifies action message identification information for identifying the selected action message (step S10), and transmits the specified action message identification information from the terminal device 2 to the cloud server 1 (step S11). Meanwhile, as shown in FIG. 10, upon acquiring the action message identification information, the cloud server 1 stores the acquired action message identification information in the action message storage unit 131 in association with time information indicating the time when the action message identification information was acquired (step S12). Subsequently, the cloud server 1 generates action notification information including the action message information identified by the newly stored action message identification information and the time information corresponding to the action message identification information (step S13). Thereafter, the generated action notification information is transmitted from the cloud server 1 to the terminal device 3 (step S14).

[0045] On the other hand, when the terminal device 3 acquires the behavior notification information, it stores the behavior message identification information and time information included in the acquired behavior notification information in the behavior message storage unit 333 in association with each other. Then, based on the device status information and time information stored in the status storage unit 332 and the acquired behavior message information and time information stored in the behavior message storage unit 333, the terminal device 3 forms a subject status notification image for notifying the user U2 of the status of the subject U1, and displays it on the display unit 304 (step S15). Here, for example, for the subject status notification image GA2 shown in FIG. 9(A), the terminal device 3 forms a subject status notification image (not shown) by superimposing message images representing the behavior messages of the subject U1, such as "Good morning," "I'm going to the hospital," "I'm home," and "Good night," on the timeline images GA2111 and GA2112, and displays the result on the display unit 304. Here, the message images are arranged according to the times indicated by the time information corresponding to the behavior message identification information. Then, by checking this target person status notification image, user U2 can understand that target person U1 was out to go to the hospital during the time period corresponding to the time between the message image ("I'm going to the hospital") and the message image ("I'm home") in the timeline image. Note that the timeline image included in the target person status notification image is not limited to this. For example, if the air conditioner 4A has a built-in human presence sensor, the timeline image may include a timeline image indicating the presence or absence of a person detected by the human presence sensor. The timeline image may also include a timeline image indicating a change in operation mode, such as the opening and closing of the door of the refrigerator 4B.

[0046] Returning to FIG. 10, next, when the preset subject person situation estimation time arrives, the cloud server 1 estimates the subject person U1's situation from the most recent behavioral message identification information, the most recent equipment status information, and the elevator usage status information using the subject person situation estimation model indicated by the subject person situation estimation model information stored in the subject person situation estimation model storage unit 133. Then, assume that the cloud server 1 determines that the estimated subject person U1's situation is abnormal (step S16). In this case, the cloud server 1 generates image generation request information including the equipment status information, elevator usage status information, and behavioral message identification information at the time when it determined that the subject person U1's situation is abnormal (step S17). Next, the generated image generation request information is transmitted from the cloud server 1 to the image generation server 9 (step S18).

[0047] On the other hand, when the image generation server 9 acquires the image generation request information, it extracts the equipment status information, elevator usage status information, and action message identification information contained in the acquired image generation request information. Then, the image generation server 9 generates an estimated situation image representing the estimated situation of the target person based on the extracted equipment status information, elevator usage status information, and action message identification information, and generates estimated situation image information indicating the generated estimated situation image (step S19). Thereafter, the generated estimated situation image information is transmitted from the image generation server 9 to the cloud server 1 (step S20).

[0048] On the other hand, when the cloud server 1 acquires the estimated situation image information, it generates abnormality notification information including the acquired estimated situation image information, abnormality notification message information notifying the subject U1 of the abnormal situation, and abnormality occurrence time information indicating the time when the subject U1 became in the abnormal situation (step S21). Next, the generated abnormality notification information is transmitted from the cloud server 1 to the terminal device 3 (step S22). On the other hand, when the terminal device 3 acquires the abnormality notification information, it extracts the estimated situation image, abnormality notification message information, and abnormality occurrence time information included in the acquired abnormality notification information, associates them with each other, and stores them in the abnormality notification storage unit 331. Then, the terminal device 3 forms an abnormality occurrence notification image for notifying the subject U1 that an abnormality has occurred based on the estimated situation image information, abnormality notification message information, and abnormality occurrence time information stored in the abnormality notification storage unit 331, and displays the image on the display unit 304 (step S23). Here, the terminal device 3 displays the abnormality occurrence notification image including the estimated situation image on the display unit 304.

[0049] Here, for example, suppose that an action message identification information of an action message indicating "going out" is selected from target person U1, and the elevator usage status information indicates that target person U1 is using elevator 5 to move from a floor where the residence is located to the first floor. In this case, if the device status information indicates that the hot water heated by the water heater is still being used, it is highly likely that target person U1 left the residence without turning off the faucet. Therefore, cloud server 1 transmits abnormality notification information to terminal device 3, including an abnormality notification message indicating "hot water is still running" and estimated situation image information showing an estimated situation image indicating, for example, a situation where hot water is overflowing from a bathtub and is likely to leak to the floor below. Terminal device 3 then generates an abnormality occurrence notification image including the estimated situation image and displays it on display unit 304. Furthermore, suppose that the elevator usage status information indicates that target person U1 frequently uses elevator 5 to move from a floor where the residence is located to the first floor during late-night hours, and the device status information indicates that device 4 is not operating. In this case, since it is suspected that the target U1 may be wandering around late at night, the cloud server 1 transmits abnormality notification information to the terminal device 3, including an abnormality notification message indicating that "the target may be behaving abnormally" and estimated situation image information showing, for example, an estimated situation image depicting the target U1 wandering around late at night. The terminal device 3 then forms an abnormality notification image including the estimated situation image and displays it on the display unit 304. Furthermore, if the target U1 is a child of elementary school age, the elevator usage status information indicates that the target U1 moved from the first floor to a floor where the residence is located during the school closing time, and the device status information indicates that the device 4 is operating. In this case, it can be assumed that the target U1 has returned home safely. Therefore, the cloud server 1 transmits abnormality notification information to the terminal device 3, including an abnormality notification message indicating that "the child has returned home" and estimated situation image information showing, for example, an estimated situation image depicting the target U1 relaxing at home. The terminal device 3 then forms an abnormality notification image including the estimated situation image and displays it on the display unit 304.In addition, the abnormality notification image may include, along with the aforementioned timeline image, a message image in the form of a speech bubble, such as "It appears the air conditioner was forgotten to be turned off" or "It appears the person has not returned home for xx days," superimposed on the timeline image.

[0050] 11, assume that the user U2 checks the abnormality notification image displayed on the display unit 304 of the terminal device 3 and then performs a behavior confirmation operation on the input unit 305 of the terminal device 3 to confirm whether or not an abnormality has actually occurred in the target person U1. In this case, the terminal device 3 accepts the behavior confirmation operation and generates confirmation message information indicating a confirmation message to be sent to the target person U1 (step S24). The generated confirmation message information is then transmitted from the terminal device 3 to the cloud server 1 (step S25) and to the terminal device 2 (step S26). Meanwhile, upon acquiring the confirmation message information, the terminal device 2 stores the acquired confirmation message information in the confirmation message storage unit 231. Based on the confirmation message information stored in the confirmation message storage unit 231, the terminal device 2 forms a confirmation message image for the target person U1 to confirm whether or not the abnormality notification is appropriate, and displays the confirmation message image on the display unit 204 (step S27).

[0051] Thereafter, suppose that the target person U1 checks the confirmation message image displayed on the display unit 204 of the terminal device 2 and then performs an acknowledgment response operation to respond to the confirmation message. In this case, the terminal device 2 specifies response message identification information that identifies the response message information selected by the acknowledgment response operation (step S28), and the specified response message identification information is transmitted from the terminal device 2 to the cloud server 1 (step S29) and also to the terminal device 3 (step S30). Meanwhile, upon acquiring the response message identification information, the terminal device 3 forms a response message image for notifying the user U2 of the confirmation response message of the target person U1 based on the acquired response message identification information and displays the response message image on the display unit 204 (step S31).

[0052] Furthermore, after transmitting the abnormality notification information, cloud server 1 acquires confirmation message information transmitted from terminal device 3, and then acquires response message identification information transmitted from terminal device 2, and identifies content flag information corresponding to the acquired response message identification information from the content flag information stored in content flag storage unit 134. Then, cloud server 1 stores the time information corresponding to the abnormality notification message information, the acquired response message identification information, and the identified content flag information in abnormality notification message storage unit 135, in association with each other (step S32).

[0053] Next, it is assumed that the cloud server 1 determines that the abnormality notification message information is invalid based on the content flag information corresponding to the abnormality notification message information stored in the abnormality notification message storage unit 135 (step S33). In this case, the cloud server 1 identifies the action message identification information and device status information corresponding to the time information corresponding to the abnormality notification message information determined to be invalid, stored in the abnormality notification message storage unit 135, from among the action message identification information stored in the action message storage unit 131 and the device status information stored in the device status storage unit 132. Then, the cloud server 1 generates a new subject person situation estimation model using the identified action message identification information and device status information, and updates the subject person situation estimation model information stored in the subject person situation estimation model storage unit 133 with subject person situation estimation model information indicating the generated subject person situation estimation model (step S34).

[0054] Next, the monitoring process executed by the cloud server 1 according to this embodiment will be described with reference to FIGS. 12 and 13. This monitoring process is started, for example, when a program for executing the monitoring process is started in the cloud server 1. First, as shown in FIG. 12, the device status acquisition unit 112 determines whether or not device status notification information transmitted from the device 4 has been acquired (step S101). If the device status acquisition unit 112 determines that device status notification information has not been acquired (step S101: No), the process of step S103, which will be described later, is executed. On the other hand, if the device status acquisition unit 112 determines that device status notification information has been acquired (step S101: Yes), the device status acquisition unit 112 extracts device status information and device identification information included in the acquired device status notification information. Then, the device status acquisition unit 112 associates the extracted device status information and device identification information with time information indicating the time when the device status notification information was acquired, and stores the extracted device status information and device identification information in the device status storage unit 132 (step S102).

[0055] Next, the elevator usage status acquisition unit 121 determines whether or not it has acquired elevator usage status notification information transmitted from the elevator management server 8 (step S103). If the elevator usage status acquisition unit 121 determines that it has not acquired elevator usage status notification information (step S103: No), it executes the processing of step S105, which will be described later. On the other hand, if the elevator usage status acquisition unit 121 determines that it has acquired elevator usage status notification information (step S103: Yes), it extracts the elevator usage status information, time information, and user identification information included in the acquired elevator usage status notification information, and stores the extracted information in the elevator usage status storage unit 136 in association with each other (step S104).

[0056] Next, the status notification unit 119 determines whether the aforementioned target person status history notification time has arrived based on the time measured by the timer unit 108 (step S105). If the status notification unit 119 determines that the target person status history notification time has not yet arrived (step S105: No), the processing of step S107 described below is executed. On the other hand, if the status notification unit 119 determines that the target person status history notification time has arrived (step S105: Yes), the status notification unit 119 generates status history notification information including time information corresponding to the device status information stored in the device status storage unit 132, and elevator usage status information and time information corresponding to the user identification information of the target person U1 stored in the elevator usage status storage unit 136. Then, the status notification unit 119 transmits the generated status history notification information to the terminal device 3 (step S106).

[0057] Thereafter, the action message acquiring unit 111 determines whether or not the action message identification information transmitted from the terminal device 2 has been acquired (step S107). If the action message acquiring unit 111 determines that the action message identification information has not been acquired yet (step S107: No), the process of step S108 described later is executed. On the other hand, if the action message acquiring unit 111 determines that the action message identification information has been acquired (step S107: Yes), the action message acquiring unit 111 stores the acquired action message identification information in the action message storage unit 131 in association with time information indicating the time when the action message identification information was acquired (step S108). Thereafter, the action notifying unit 120 generates action notification information including the action message information identified by the newly stored action message identification information and the time information corresponding to the above-mentioned action message identification information, and transmits the generated action notification information to the terminal device 3 (step S109).

[0058] Next, the subject person status estimation unit 113 determines whether the subject person status estimation time has arrived (step S110). If the subject person status estimation unit 113 determines that the subject person status estimation time has not yet arrived (step S110: No), the processing of step S115 described below is executed. On the other hand, if the subject person status estimation unit 113 determines that the subject person status estimation time has arrived (step S110: Yes), the subject person status estimation unit 113 estimates the status of the subject person U1 from the most recent behavioral message identification information, the most recent equipment status information, and the elevator usage status information, using the subject person status estimation model indicated by the subject person status estimation model information stored in the subject person status estimation model storage unit 133. Then, the subject person status estimation unit 113 determines whether the estimated status of the subject person U1 is abnormal (step S111). Here, if the subject person state estimation unit 113 determines that the estimated state of the subject person U1 is normal (step S111: No), the process of step S115 described below is executed.

[0059] On the other hand, suppose that the subject person status estimation unit 113 determines that the estimated status of the subject U1 is abnormal (step S111: Yes). In this case, the abnormality notification information generation unit 114 generates image generation request information including equipment status information, elevator usage status information, and action message identification information at the time when it was determined that the subject U1's status is abnormal, and transmits the image generation request information to the image generation server 9 (step S112). As a result, the abnormality notification information generation unit 114 acquires estimated status image information transmitted from the image generation server 9 (step S113). Next, the abnormality notification information generation unit 114 generates abnormality notification information including abnormality notification message information notifying the subject U1 of the abnormal status estimated by the subject person status estimation unit 113, abnormality occurrence time information indicating the time when the subject U1 became in the abnormal status, and the acquired estimated status image information. Then, the abnormality notification information transmission unit 115 transmits the generated abnormality notification information to the terminal device 3 (step S114). At this time, the abnormality notification information transmitting unit 115 notifies the message acquiring unit 116 of the abnormality notification message information and the corresponding time information.

[0060] 13, after the message acquisition unit 116 is notified of the abnormality notification message information and the time information, the message acquisition unit 116 determines whether or not it has acquired confirmation message information transmitted from the terminal device 3 within a preset confirmation message information waiting period (step S115). If the message acquisition unit 116 determines that it has not acquired the confirmation message information within the confirmation message information waiting period (step S115: No), the process of step S101 is executed again. On the other hand, if the message acquisition unit 116 determines that it has acquired confirmation message information within the confirmation message information waiting period (step S115: Yes), the message acquisition unit 116 determines whether or not it has acquired response message identification information transmitted from the terminal device 2 within a preset response message waiting period (step S116). If the message acquisition unit 116 determines that it has not acquired the response message identification information within the response message waiting period (step S116: No), the process of step S101 is executed again. On the other hand, when message acquirer 116 determines that response message identification information has been acquired within the response message waiting period (step S116: Yes), message acquirer 116 identifies content flag information corresponding to the acquired response message identification information from the content flag information stored in content flag storage unit 134. Then, message acquirer 116 stores the notified abnormality notification message information and the corresponding time information in abnormality notification message storage unit 135, associating the acquired response message identification information with the identified content flag information (step S117).

[0061] Thereafter, the validity determination unit 117 determines whether the abnormality notification message information is valid based on the content flag information corresponding to the abnormality notification message information stored in the abnormality notification message storage unit 135 (step S118). Here, if the content flag information corresponding to the abnormality notification message information indicates a positive message for the confirmation message, the validity determination unit 117 determines that the abnormality notification message information is valid (step S118: Yes), and the process of step S101 is executed again. On the other hand, if the content flag information corresponding to the abnormality notification message information indicates a negative message for the confirmation message, the validity determination unit 117 determines that the abnormality notification message information is invalid (step S118: No). In this case, when the model update unit 118 is notified of time information corresponding to the abnormality notification message information determined to be invalid by the validity determination unit 117, the model update unit 118 identifies the action message identification information, equipment status information, and elevator usage status information corresponding to the notified time information from the action message identification information stored in the action message storage unit 131 and the equipment status information stored in the equipment status storage unit 132. Then, the model update unit 118 generates a new subject person situation estimation model using the identified behavior message identification information, the equipment status information, and the elevator usage status information, and updates the subject person situation estimation model information stored in the subject person situation estimation model storage unit 133 with subject person situation estimation model information indicating the generated subject person situation estimation model (step S119). Next, the process of step S101 is executed again.

[0062] As described above, in the monitoring system according to this embodiment, the target person status estimation unit 113 estimates the status of the target person U1 based on the behavioral message identification information from the terminal device 2 carried by the target person U1, the device status information of each device 4, and the elevator usage status information indicating the usage status of the elevator 5 by the target person U1. This allows the target person U1's status to be estimated taking into account the target person U1's actual behavior history, thereby enabling accurate estimation of whether the target person U1's status is abnormal. This allows user U2 to appropriately understand the target person U1's behavior.

[0063] Furthermore, the terminal device 3 according to this embodiment generates a subject status notification image showing the past status of the device 4, the use of the elevator 5 by the subject U1, and the transition of the behavioral message selected by the subject U1 based on the above-described device status information, elevator usage status information, and behavioral message identification information, and displays the image on the display unit 304. Here, as shown in FIG. 9A, for example, the terminal device 3 displays a subject status notification image GA2 on the display unit 304, which includes a timeline image GA2112 showing the transition of the usage status of the elevator 5 in addition to a timeline image GA2111 showing the transition of the status of the device 4 used by the subject U1. This allows the user U2 to understand the status of the subject U1 through the subject status notification image GA2 without directly contacting the subject U1. Therefore, the user U2 can quietly keep an eye on the subject U1, and by displaying the message image, the user U2 and the subject U1 can communicate with each other while watching over the subject U1.

[0064] Furthermore, when it is estimated that the situation of the target person U1 is abnormal, the abnormality notification information generation unit 114 of the cloud server 1 according to this embodiment generates abnormality notification information notifying that an abnormality has occurred in the target person U1. Then, based on the abnormality notification information, the terminal device 3 forms an abnormality occurrence notification image notifying the user U2 that an abnormality has occurred in the target person U1, and displays the image on the display unit 304. This allows the user U2 to relatively quickly grasp the abnormality that has occurred in the target person U1 through the abnormality occurrence notification image, which has the advantage of making it easier for the user U2 to take appropriate action against the target person U1.

[0065] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments. For example, the terminal device 3 may further include a returning home behavior estimation unit that estimates whether the target person U1 is performing a behavior of returning home based on elevator usage status information, and a returning home behavior notification unit that generates returning home behavior notification information when it is estimated that the target person U1 is performing a behavior of returning home. The terminal device 3 may also form a returning home notification image that notifies the user that the target person U1 has returned home based on the returning home behavior notification information, and display the image on the display unit 304.

[0066] According to this configuration, the user U1 can confirm that the target person U1 has returned home from the homecoming notification image.

[0067] In an embodiment, the device may further include a door opening / closing operation status acquisition unit that acquires door opening / closing status information indicating the status of the opening / closing operation of a door installed at the entrance of the dwelling unit the subject lives in. The subject person status estimation unit may estimate the subject person's status based on the door opening / closing status information, as well as the behavioral message identification information, the equipment status information, and the elevator usage status information.

[0068] According to this configuration, the state of the subject U1 can be estimated with higher accuracy.

[0069] In an embodiment, the terminal device 3 carried by the user U2 may transmit response message request information requesting transmission of a response message to the terminal device 2 carried by the target person U1 in response to an operation by the user U2. Then, the target person status estimation unit 113 of the cloud server 1 may estimate the status of the target person U1 based on whether or not response message information indicating a response message is transmitted from the terminal device 2 to the terminal device 3 within a preset time after the response message request information is transmitted from the terminal device 3 to the terminal device 2. Here, for example, it is assumed that the elevator usage status information indicates that the target person U1 is not using the elevator 5, and the equipment status information indicates that the target person U1 is still using hot water boiled by the water heater. In this case, if response message request information is sent from terminal device 3 to terminal device 2 and then response message information indicating a response message is not sent from terminal device 2 to terminal device 3 within a preset time, the subject may be unwell and unconscious while at home, so the cloud server 1 sends abnormality notification information to terminal device 3, which includes an abnormality notification message indicating that "there is a possibility that an abnormality has occurred in the subject," and estimated situation image information showing an estimated situation image showing, for example, the subject U1 having lost consciousness and collapsed in the bathtub, and the terminal device 3 then forms an abnormality occurrence notification image including the aforementioned estimated situation image and displays it on the display unit 304.

[0070] In the embodiment, an example has been described in which an elevator 5 is installed in an apartment building where a target person U1 lives, and the elevator usage status information indicates the usage status of the elevator 5 installed in the apartment building by the target person U1. However, this is not limited to this, and the elevator usage status information may also indicate the usage status by the target person U1 of an elevator installed in another building, for example. For example, suppose the elevator usage status information is for an elevator installed in a hospital where the target person U1 visits, and the elevator usage status information indicates that the target person U1 has used the elevator installed in the hospital every day, and the equipment status information indicates that the elevator is not operating during the time period when the target person U1 is using the elevator. In this case, it can be inferred that the subject U1 is in an abnormal state and is visiting the hospital, and therefore it is considered that there is no need to issue a highly urgent alert. Therefore, the cloud server 1 transmits abnormality notification information to the terminal device 3, which includes an abnormality notification message indicating that "it appears that the subject U1 is visiting the hospital," and inferred situation image information showing an inferred situation image indicating, for example, that the subject U1 is injured or that the subject U1 is suffering from an illness, and the terminal device 3 then forms an abnormality occurrence notification image including the inferred situation image and displays it on the display unit 304.

[0071] The various functions of the cloud server 1 according to the present disclosure may be realized by software, firmware, or a combination of software and firmware. In this case, the software or firmware may be written as a program, and the program may be stored and distributed on a computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), or an MO (Magneto-Optical Disc). The program may then be read and installed on a computer to configure a computer capable of realizing the aforementioned functions. In addition, when each function is realized by sharing the work between an operating system (OS) and an application, or by cooperation between the OS and an application, only the parts other than the OS may be stored on the recording medium.

[0072] Furthermore, each program can be superimposed on a carrier wave and distributed over a network. For example, the program can be posted on a bulletin board system (BBS) on the network and distributed over the network. These programs can then be started and run under the control of the OS in the same way as other application programs, thereby enabling the above-mentioned processing to be performed.

[0073] Various aspects of the present disclosure are summarized below as appendices.

[0074] (Appendix 1) a first terminal device carried by a person to be monitored by a user; a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message that indicates the behavior of the subject, the behavior message being generated by the first terminal device in response to an operation of the subject on the first terminal device; a device status acquisition unit that acquires device status information indicating the status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the target person; a subject status estimation unit that estimates a status of the subject based on the action message identification information, the equipment status information, and the elevator usage status information, Surveillance system. (Appendix 2) Further provided is a second terminal device having a display unit and carried by the user, the second terminal device forms a target person status notification image showing a past status of the at least one device, a past usage status of the elevator by the target person, and a transition of a behavior message selected by the target person, based on the device status information, the elevator usage status information, and the behavior message identification information, and displays the image on the display unit; 1. A monitoring system as described in Appendix 1. (Appendix 3) a second terminal device having a display unit and carried by the user; and an abnormality notification information generation unit that generates abnormality notification information notifying the subject that an abnormality has occurred when the subject's condition is estimated to be abnormal, The second terminal device forms an abnormality occurrence notification image for notifying the user that an abnormality has occurred in the target person based on the abnormality notification information, and displays the image on the display unit. 1. A monitoring system as described in Appendix 1 or 2. (Appendix 4) the subject person status estimation unit estimates the subject person's status using a subject person status estimation model that estimates the subject person's status from the action message identification information, the equipment status information, and the elevator usage status information; 4. The monitoring system of any one of claims 1 to 3. (Appendix 5) the elevator usage status acquisition unit monitors the usage status of the elevator in association with a user who uses the elevator, and if the user is the target person, acquires the usage status information generated by an elevator management server that generates the elevator usage status information indicating the usage status of the elevator corresponding to the user; 5. The monitoring system of any of claims 1 to 4. (Appendix 6) a returning-home behavior estimation unit that estimates whether the subject is performing a behavior of returning home based on the elevator usage status information; a returning-home behavior notification unit that generates returning-home behavior notification information when it is estimated that the subject is performing a behavior of returning home, The second terminal device forms a homecoming notification image notifying the user that the target person has returned home based on the homecoming behavior notification information, and displays the image on the display unit. 10. A monitoring system as described in appendix 2 or 3. (Appendix 7) The method further includes an image generation server that generates a subject status notification image representing the subject's status based on the estimated subject's status, the second terminal device forms an abnormality occurrence notification image including the generated subject person status notification image and displays it on the display unit. 1. A monitoring system as described in Appendix 3. (Appendix 8) The device further includes a door opening / closing operation status acquisition unit that acquires door opening / closing status information indicating the status of the opening / closing operation of a door installed at the entrance of the dwelling unit where the subject resides, the target person status estimation unit estimates the target person's status based on the behavior message identification information, the equipment status information, the elevator usage status information, and the door opening / closing status information; 8. The monitoring system of any of claims 1 to 7. (Appendix 9) a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message indicating a behavior of a target person that is a target of monitoring by a user, the behavior message being generated by a first terminal device carried by the target person in response to an operation on the first terminal device; a device status acquisition unit that acquires device status information indicating the status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the target person; a subject status estimation unit that estimates a status of the subject based on the action message identification information, the equipment status information, and the elevator usage status information, Monitoring equipment. (Appendix 10) a step in which the monitoring system acquires behavior message identification information that identifies a behavior message indicating the behavior of a subject that is the subject of monitoring by a user, the behavior message being generated by a first terminal device carried by the subject in response to an operation on the first terminal device; The monitoring system acquires device status information indicating a status of each of at least one device owned by the subject; The monitoring system acquires elevator usage status information indicating the usage status of the elevator used by the subject; The monitoring system includes a step of estimating a status of the subject based on the behavior message identification information, the equipment status information, and the elevator usage status information. Monitoring method. (Appendix 11) Computer, a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message indicating a behavior of a target person that is a target of monitoring by a user, the behavior message being generated by a first terminal device carried by the target person in response to an operation on the first terminal device; a device status acquisition unit that acquires device status information indicating a status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the subject person; a subject situation estimation unit that estimates a situation of the subject based on the action message identification information, the equipment status information, and the elevator usage situation information; A program to function as a [Industrial Applicability]

[0075] The present disclosure is suitable as a monitoring system for a user to watch over a person to be watched over. [Explanation of symbols]

[0076] 1 Cloud server, 2, 3 Terminal device, 4 Equipment, 4A Air conditioner, 4B Refrigerator, 5 Elevator, 8 Elevator management server, 9 Image generation server, 81 Data circuit terminating device, 82 Router, 101, 201, 301 CPU, 102, 202, 302 Main memory unit, 103, 203, 303 Auxiliary memory unit, 106, 206, 306 Communication unit, 108 Timekeeping unit, 109, 209, 309 Bus, 111 Action message acquisition unit, 112 Equipment status acquisition unit, 113 Subject situation estimation unit, 114 Abnormality notification message generation unit, 115 Abnormality notification information transmission unit, 116 Message acquisition unit, 117 Validity determination unit, 118 Model update unit, 119 Status notification unit, 120 Action notification unit, 121 Elevator usage status acquisition unit, 131, 333 Action message storage unit, 132, 332 equipment status storage unit, 133 target person status estimation model storage unit, 134 content flag storage unit, 135 abnormality notification message storage unit, 136 elevator usage status storage unit, 204, 304 display unit, 205, 305 input unit, 212 action message notification unit, 213 confirmation message acquisition unit, 214, 312 image formation unit, 215, 313 display control unit, 216 confirmation response message generation unit, 217 confirmation response message transmission unit, 231 confirmation message storage unit, 311 abnormality notification acquisition unit, 315 confirmation message generation unit, 316 confirmation message transmission unit, 317 status acquisition unit, 318 action notification information acquisition unit, 331 abnormality notification storage unit, BU11, BU12, BU13, BU14, BU15 button image, GA1 operation screen image, GA2 Subject status notification image, GA2111,GA2112 timeline image, IC211,IC212 icon image, L10 input layer, L20 hidden layer, L30 output layer, NW1 wide area network, NW2 local area network

Claims

1. a first terminal device carried by a subject person to be monitored by a user; a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message that indicates the behavior of the subject, the behavior message being generated by the first terminal device in response to an operation of the subject on the first terminal device; a device status acquisition unit that acquires device status information indicating a status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the target person; a subject status estimation unit that estimates a status of the subject based on the action message identification information, the equipment status information, and the elevator usage status information, Surveillance system.

2. a second terminal device having a display unit and carried by the user; the second terminal device forms a target person status notification image showing a past status of the at least one device, a past usage status of the elevator by the target person, and a transition of a behavior message selected by the target person, based on the device status information, the elevator usage status information, and the behavior message identification information, and displays the image on the display unit; The monitoring system of claim 1 .

3. a second terminal device having a display unit and carried by the user; and an abnormality notification information generation unit that generates abnormality notification information notifying the subject that an abnormality has occurred when the subject's condition is estimated to be abnormal, the second terminal device forms an abnormality occurrence notification image for notifying the user that an abnormality has occurred in the target person based on the abnormality notification information, and displays the image on the display unit. The monitoring system of claim 1 .

4. the subject person status estimation unit estimates the subject person's status using a subject person status estimation model that estimates the subject person's status from the action message identification information, the equipment status information, and the elevator usage status information; A monitoring system according to any one of claims 1 to 3.

5. the elevator usage status acquisition unit monitors the usage status of the elevator in association with a user who uses the elevator, and if the user is the target person, acquires the usage status information generated by an elevator management server that generates the elevator usage status information indicating the usage status of the elevator corresponding to the user; A monitoring system according to any one of claims 1 to 3.

6. a returning-home behavior estimation unit that estimates whether the subject is performing a behavior of returning home based on the elevator usage status information; a returning-home behavior notification unit that generates returning-home behavior notification information when it is estimated that the subject is performing a behavior of returning home, the second terminal device forms a home-coming notification image notifying the user that the target person has returned home based on the home-coming behavior notification information, and displays the image on the display unit; 4. A monitoring system according to claim 2 or 3.

7. The method further includes an image generation server that generates a subject status notification image representing the subject's status based on the estimated subject's status, the second terminal device forms an abnormality occurrence notification image including the generated subject person status notification image and displays it on the display unit; The monitoring system of claim 3 .

8. The device further includes a door opening / closing operation status acquisition unit that acquires door opening / closing status information indicating the status of the opening / closing operation of a door installed at the entrance of the dwelling unit where the subject resides, the target person status estimation unit estimates the target person's status based on the behavior message identification information, the equipment status information, the elevator usage status information, and the door opening / closing status information; The monitoring system of claim 1 .

9. a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message indicating a behavior of a target person that is a target of monitoring by a user, the behavior message being generated by a first terminal device carried by the target person in response to an operation on the first terminal device; a device status acquisition unit that acquires device status information indicating a status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the target person; a subject status estimation unit that estimates a status of the subject based on the action message identification information, the equipment status information, and the elevator usage status information, monitoring equipment.

10. a step in which the monitoring system acquires behavior message identification information that identifies a behavior message indicating the behavior of a subject that is the subject of monitoring by a user, the behavior message being generated by a first terminal device carried by the subject in response to an operation on the first terminal device; The monitoring system acquires device status information indicating a status of each of at least one device owned by the subject; The monitoring system acquires elevator usage status information indicating the usage status of the elevator used by the subject; The monitoring system includes a step of estimating a status of the target person based on the behavior message identification information, the equipment status information, and the elevator usage status information. Monitoring method.

11. Computer, a behavior message acquisition unit that acquires behavior message identification information that identifies a behavior message indicating a behavior of a target person who is a target of monitoring by a user, the behavior message being generated by a first terminal device carried by the target person in response to an operation on the first terminal device; a device status acquisition unit that acquires device status information indicating a status of each of at least one device owned by the subject; an elevator usage status acquisition unit that acquires elevator usage status information indicating a usage status of the elevator used by the subject person; a subject situation estimation unit that estimates a situation of the subject based on the action message identification information, the equipment status information, and the elevator usage situation information; A program to function as a

Images