Display controller and monitoring system

Abstract

To provide a display controller and a monitoring system.SOLUTION: The display controller comprises a receiving unit that receives event information regarding events that have occurred at a monitored location, and a presentation unit that presents a history of the event information received by the receiving unit. The presentation unit presents details based on the authority set by a user for each monitored location.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display control device and a monitoring system. This application claims priority based on Japanese Patent Application No. 2021-167872, filed on October 13, 2021, the contents of which are incorporated herein by reference. [Background technology]

[0002] Conventionally, techniques relating to remote monitoring devices have been disclosed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-11698 Summary of the Invention

[0004] One embodiment of the present invention is a display control device comprising: a receiving unit that receives event information regarding events that occur at a monitored location; and a presentation unit that presents a history of the event information received by the receiving unit. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a diagram illustrating an example of a device configuration of a monitoring system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of a functional configuration of a monitoring system according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram illustrating an example of setting information of the monitoring system according to the present embodiment. [Figure 4A] FIG. 10 is a diagram illustrating an example of an initial setting screen according to the present embodiment. [Figure 4B] FIG. 10 is a diagram illustrating an example of an initial setting screen according to the present embodiment. [Figure 4C] FIG. 2 is a diagram illustrating an example of an operation screen of an emergency call function according to the present embodiment. [Figure 5A]10A to 10C are diagrams illustrating examples of displays for each monitoring mode according to the present embodiment. [Figure 5B] 10A to 10C are diagrams illustrating examples of displays for each monitoring mode according to the present embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of communication in an emergency response function of the present embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a pop-up display at the time of an emergency response according to the present embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of a live view display according to the present embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a correspondence selection screen according to the present embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a display screen of a response history according to the present embodiment. [Figure 11] FIG. 10 is a diagram illustrating another example of communication in the emergency response function of the present embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of an event history display according to the present embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a selection screen for events to be presented according to the present embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of communication using a conversation function according to the present embodiment. [Figure 15] FIG. 10 is a diagram illustrating an example of communication of the live view function in the private mode of the present embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of communication of the live view function in the public mode of the present embodiment. [Figure 17] 10A and 10B are diagrams illustrating an example of the operation of the monitoring system in a public mode and a private mode according to the present embodiment. [Figure 18] FIG. 10 is a diagram illustrating an example of communication in a voltage drop notification function of the present embodiment. [Figure 19] FIG. 10 is a diagram illustrating an example of a voltage drop notification screen according to the present embodiment. [Figure 20] FIG. 10 is a diagram illustrating an example of communication in a remote control device search function of the present embodiment. [Figure 21] 10A and 10B are diagrams illustrating an example of communication in the remote control device out-of-service area notification function of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0006] [Monitoring system configuration] FIG. 1 is a diagram showing an example of the device configuration of a monitoring system 1 according to this embodiment. The monitoring system 1 includes a terminal device 10, a server device 20, a communication device 30, and a remote control device 40. The monitoring system 1 provides a monitoring service that enables a monitor to monitor the status of a person to be monitored.

[0007] In the following description, the person to be monitored will also be referred to as the person being monitored or the person being monitored. The monitoring service provided by the monitoring system 1 will also be referred to as the monitoring service. The monitor will also be referred to as the monitor, the user of the monitoring service, or the user. That is, the monitoring system 1 provides a monitoring service that enables a watcher to monitor the state and behavior of a person being watched over. As an example, the person being watched over is a family member of the person being watched over who does not live with the person being watched over. The monitoring system 1 provides a function that enables the person being watched over to remotely monitor the status of the person being watched over.

[0008] The terminal device 10 is a device used by a watcher, that is, a user. The terminal device 10 is, for example, a smartphone, a tablet, or a personal computer, and performs information communication with the server device 20 via the network N1. The server device 20 is a computer device operated by a provider of a monitoring service. The server device 20 performs various settings in the monitoring system 1, and stores and provides information acquired by the monitoring system 1. The communication device 30 includes an imaging unit 350 (imaging device) such as a camera, and captures images of the monitoring target area A. The communication device 30 communicates information with the server device 20 via the network N2. The remote control device 40 is a device used by the person being watched over. The remote control device 40 communicates information with the communication device 30 via short-range wireless communication, infrared communication, or the like.

[0009] The monitoring system 1 may include a plurality of terminal devices 10, a plurality of communication devices 30, and a plurality of remote control devices 40. As an example, the monitoring system 1 can be configured to have multiple watchers, each of whom uses one terminal device 10. For example, if the person being watched has brothers or sisters, the monitoring system 1 can be configured so that multiple watchers (brothers or sisters) each watch over the parent who is the person being watched over. Furthermore, the monitoring system 1 can be configured to install a communication device 30 in each room of a building and monitor the status and behavior of the person being watched over in each room. For example, if a building has multiple rooms, the monitoring system 1 can be configured to monitor the status and behavior of the person being watched over in each room, with a first room (e.g., a living room) as the monitored area A1 and a second room (e.g., a bedroom) as the monitored area A2. Furthermore, the monitoring system 1 can be configured so that multiple people being watched over who live together in a building each use the remote control device 40. For example, if the people being watched over are the parents of the person being watched over, the monitoring system 1 can be configured so that the father and mother each use the remote control device 40. Furthermore, the monitoring system 1 can monitor not only one building but multiple buildings. For example, if the person being watched has a spouse, the monitoring system 1 can be configured to monitor the status and behavior of the person being watched in each building, with a first building where the person being watched's parents live as the monitored area A1 and a second building where the person being watched's spouse's parents live as the monitored area A2.

[0010] In the following description, when it is necessary to distinguish between the multiple terminal devices 10, the first terminal device 10 will be referred to as terminal device 10-1, the second terminal device 10 as terminal device 10-2, and so on. Furthermore, when distinguishing between the multiple communication devices 30, the first communication device 30 will be referred to as communication device 30-1, the second communication device 30 as communication device 30-2, and so on. Next, the functional configuration of each device in the monitoring system 1 will be described with reference to FIG.

[0011] [Monitoring system functional configuration] FIG. 2 is a diagram showing an example of the functional configuration of the monitoring system 1 of this embodiment.

[0012] [Functional configuration of terminal device] The terminal device 10 includes a terminal operation detection unit 110, a presentation unit 120, a terminal control unit 130, a terminal communication unit 140 (receiving unit), a terminal sound collection unit 150, and a terminal sound generation unit 160. The terminal operation detection unit 110 includes an operation device such as a touch panel or an operation switch, and detects an operation on the operation device by the user of the terminal device 10. When the terminal operation detection unit 110 detects an operation, it generates operation information indicating the detected operation and outputs the operation information to the terminal control unit 130. The presentation unit 120 includes, for example, a liquid crystal display, and presents an image under the control of the terminal control unit 130. The terminal control unit 130 includes, for example, a microcomputer, and controls each unit of the terminal device 10. The terminal communication unit 140 communicates information with the server device 20 under the control of the terminal control unit 130 .

[0013] The terminal sound collection unit 150 includes, for example, a microphone, and collects sounds around the terminal device 10. The terminal sound collection unit 150 outputs information about the collected sounds to the terminal control unit . The terminal device 10 may operate each function of the terminal control unit 130 using the collected sound (for example, performing voice recognition). In this case, the terminal sound collection unit 150 can be said to be an example of the terminal operation detection unit 110 that operates the terminal device 10 using sound. The terminal sound output unit 160 includes, for example, a speaker, and emits a sound based on the control of the terminal control unit 130. Note that the terminal device 10 may alert the user by the emitted sound (for example, by sounding an alarm). In this case, the terminal sound output unit 160 can be said to be an example of the presentation unit 120 that presents information by sound.

[0014] [Server device functional configuration] The server device 20 includes a server communication unit 210 and a server storage unit 220. The server communication unit 210 performs information communication with the terminal device 10 and with the communication device 30 . The server storage unit 220 includes a storage device such as a semiconductor memory, and stores various types of information. An example of information stored in the server storage unit 220 will be described with reference to FIG.

[0015] FIG. 3 is a diagram showing an example of setting information of the monitoring system 1 of this embodiment. As described above, the terminal device 10 can set multiple watching destinations as watching targets in the monitoring system 1. In this example, of the multiple watching destinations a to c, the terminal device 10-1 and the terminal device 10-2 are set to watch over the watching destinations a to c as watching targets. Furthermore, in the terminal device 10-3, the watching destinations a to b are set as the watching targets, but the watching destination c is not set as the watching target. Furthermore, when multiple terminal devices 10 are monitoring one monitoring destination, it is possible to set some of these multiple terminal devices 10 to function as a main monitoring device and some to function as sub monitoring devices. For example, for monitoring destination a, terminal device 10-1 functions as the main monitoring device, and terminal devices 10-2 and 10-3 function as sub monitoring devices. In the monitoring system 1, the main monitoring device and the sub monitoring devices can provide different functions. The server storage unit 220 stores setting information such as that shown in Fig. 3. This setting information is referenced by the terminal device 10 or the communication device 30 as needed.

[0016] Here, different operation rights may be given to the user of the main monitoring device and the user of the sub monitoring device. For example, the user of the main monitoring device may be given the right to view real-time video regardless of whether or not the user has given permission from the person being watched, and the user of the sub monitoring device may not be able to view real-time video without the permission of the person being watched. Also, although the operation authority has been described here as having two levels, main and sub, the present invention is not limited to this, and the operation authority may be set in three or more levels. That is, each of the multiple users is given one of at least two levels of operation authority, and the presenting unit 120 presents content based on the operation authority given to the user.

[0017] This operation authority may be given for each area A to be monitored (that is, each location to be monitored). That is, there are multiple locations to be monitored, and the user is given operation authority for each location to be monitored, and the presentation unit 120 of the terminal device 10 presents content based on the operation authority given to the user for each location to be monitored.

[0018] [Functional configuration of communication device] Returning to Figure 2, the communication device 30 has the functions of a first communication unit 310, a second communication unit 320, a communication device control unit 330, a measurement unit 340, an imaging unit 350, a communication device sound collection unit 360, a communication device sound generation unit 370, an operation detection unit 380, and a communication device memory unit 390.

[0019] First communication unit 310 communicates information with remote control device 40 under the control of communication device control unit 330 . The second communication unit 320 communicates information with the server device 20 under the control of the communication device control unit 330.

[0020] As described above, the second communication unit 320 communicates with the terminal device 10 via the server communication unit 210 of the server device 20. However, in the following description, the explanation that the server device 20 mediates the communication between the terminal device 10 and the communication device 30 will be omitted, and it may be described, for example, as the second communication unit 320 transmitting the first operation information to the terminal device 10.

[0021] The measurement unit 340 measures various conditions in the monitoring target area A. For example, the measurement unit 340 may include a temperature sensor and a humidity sensor to measure the temperature and humidity of the monitoring target area A. The measurement unit 340 may also include an infrared sensor to measure the movement of people in the monitoring target area A. The imaging unit 350 includes, for example, a camera, and captures an image of the monitoring target area A under the control of the communication device control unit 330.

[0022] The measuring unit 340 and the imaging unit 350 are collectively referred to as a detecting unit. That is, the communication device 30 detects various conditions in the monitoring target area A by means of the detecting unit. In addition to the imaging unit 350, the detection unit may have a determination unit (not shown) that determines the state of the monitored person based on the video captured by the imaging unit 350. The determination unit, for example, analyzes the posture of the monitored person based on the results of machine learning and determines whether or not an abnormality has occurred in the monitored person, such as the monitored person falling. The determination unit outputs the determination result to the communication device control unit 330.

[0023] The communication device control unit 330 includes, for example, a microcomputer, and controls each unit of the communication device 30 including the first communication unit 310, the second communication unit 320, and the imaging unit 350.

[0024] The communication device sound collection section 360 includes, for example, a microphone, and collects sounds around the communication device 30. The communication device sound collection section 360 outputs information about the collected sounds to the communication device control section 330. The communication device sound output unit 370 includes, for example, a speaker, and outputs sounds under the control of the communication device control unit 330 . The operation detection unit 380 detects an operation on an operation unit (not shown), such as a switch, provided in the communication device 30. As an example, the communication device 30 includes an operation mode changeover switch (not shown). The operation detection unit 380 detects the setting state of the operation mode changeover switch and outputs the result to the communication device control unit 330.

[0025] The communication device storage unit 390 includes a storage device such as a semiconductor memory, and stores various types of information. As an example, the communication device storage unit 390 stores information on images (e.g., moving images) captured by the imaging unit 350 and information on sounds collected by the communication device sound collection unit 360.

[0026] [Remote control device functional configuration] Remote control device 40 includes an emergency button 410, an OK button 420, an NG button 480, a remote control communication unit 430, a remote control control unit 440, a remote control sound collection unit 450, a remote control sound generation unit 460, and a voltage detection unit 470. The OK button 420 is an operation button that indicates agreement, approval, or a positive answer, and is also called a "Yes button." The NG button 480 is an operation button that indicates disagreement, rejection, or a negative answer, and is also called a "No button."

[0027] The emergency button 410 is a button that is operated by the person being watched over if any abnormality occurs to the person being watched over. The OK button 420 is a button that is operated by the person being watched over when the monitoring system 1 requests permission from the person being watched over and the request is to be granted.

[0028] The remote control communication unit 430 communicates information with the communication device 30 under the control of the remote control control unit 440 .

[0029] Remote control unit 440 includes, for example, a microcomputer, and controls each unit of remote control device 40. For example, when remote control unit 440 detects an operation on emergency button 410, it generates first operation information that is information indicating that emergency button 410 has been operated. Remote control unit 440 transmits the generated first operation information to communication device 30 via remote control communication unit 430. Similarly, when remote control unit 440 detects an operation on OK button 420, it generates second operation information that is information indicating that OK button 420 has been operated. Remote control control unit 440 transmits the generated second operation information to communication device 30 via remote control communication unit 430.

[0030] Remote control sound collection section 450 includes, for example, a microphone, and collects sounds around remote control device 40. Remote control sound collection section 450 outputs information on the collected sounds to remote control control section 440. Remote control sound generating section 460 includes, for example, a speaker, and generates sounds based on the control of remote control control section 440. Voltage detection unit 470 detects the voltage value of the operating power supply of remote control device 40. Voltage detection unit 470 outputs the detected voltage value of the operating power supply to remote control control unit 440. If the voltage value detected by voltage detection unit 470 is equal to or lower than a predetermined value, remote control control unit 440 transmits voltage drop information to communication device 30 via remote control communication unit 430.

[0031] [Explanation of each function] Next, each function provided by the monitoring system 1 and its operation will be described with reference to the drawings. In the following description, application software (hereinafter also simply referred to as an app) provided by the monitoring system 1 is installed in the terminal device 10. The terminal control unit 130 controls each unit of the terminal device 10 based on the installed app. The presentation unit 120 of the terminal device 10 displays a screen specified by the app. After the app is installed in the terminal device 10, the presentation unit 120 displays an initial setting screen when the app is launched for the first time.

[0032] [Initial Settings] 4A and 4B are diagrams showing an example of an initial setting screen of this embodiment. Fig. 4A shows an example of a notification setting screen. The monitoring system 1 provides a function for initially setting a situation related to the person being watched over that should be notified to the terminal device 10. An example of a situation to be notified is shown below.

[0033] · Emergency call: When the emergency button 410 is operated. Fall: When the person being watched falls within monitored area A. · Temperature and humidity abnormality: When the temperature or humidity in monitored area A is outside the specified range. - Safety confirmation abnormality: When the person being monitored does not respond to a call. Camera abnormality: When an abnormality such as a malfunction occurs in the imaging unit 350. Deviation from routine: When the monitored person's behavior is different from usual.

[0034] The user operates the notification setting screen displayed on the presentation unit 120 of the terminal device 10 and selects a situation to be notified from among multiple types of situations. The terminal device 10 transmits the selected notification setting to the server device 20. The server device 20 stores the received notification setting in the server storage unit 220. The monitoring system 1 notifies the terminal device 10 when the set situation occurs. In this example, the notification settings are described as being stored in the server storage unit 220, but this is not limiting. The notification settings may be stored in any of the terminal device 10, the server device 20, and the communication device 30, as long as any of the devices can determine whether or not a certain situation should be notified to the user when the situation occurs. Various pieces of information described below may also be stored in any of the devices as long as the functions using the information can be fulfilled.

[0035] 4B shows an example of the contact setting screen. The monitoring system 1 provides an emergency notification function that allows a pre-set contact to be contacted in an emergency with a few operations (for example, by tapping an icon displayed on the terminal device 10).

[0036] Fig. 4C shows an example of an operation screen for the emergency call function. When the above-mentioned situation to be notified occurs, the operation screen for the emergency call function (emergency response screen) shown in Fig. 4C is displayed on the display unit 120 of the terminal device 10. As an example, when three emergency call destinations are set, emergency call destination icons P11 to P13 are displayed on the display unit 120. The contacts for this emergency call function can be set on the contact setting screen shown in Fig. 4B. The user sets contacts by operating the contact setting screen displayed on the presentation unit 120 of the terminal device 10. The terminal device 10 stores the set contacts in a storage unit (not shown) of the terminal device 10.

[0037] [Monitoring mode settings] 5A and 5B are diagrams showing examples of displays for each monitoring mode of this embodiment. The monitoring system 1 provides multiple monitoring modes, including a private mode and a public mode. FIG. 5A shows an example of a screen displayed on the terminal device 10 in the private mode. In the private mode, an image of the person being watched is not displayed, and an image P21 showing the status of the person being watched in text is displayed. FIG. 5B shows an example of a screen displayed on the terminal device 10 in the public mode. In the public mode, an image P22 of the monitoring target area A captured by the imaging unit 350 of the communication device 30 is displayed. At this time, if a person being watched is present in the monitoring target area A, a moving image (real-time image) showing the current status of the person being watched is displayed in the image P22. Note that, for ease of explanation, the moving image of the person being watched is represented by a line drawing in FIG. 5B. In this embodiment, the terminal device 10 displays a moving image of the person being watched that is actually captured by the imaging unit 350, rather than a line drawing. The non-public mode and the public mode can be switched by the operation mode switch (not shown) of the communication device 30 described above. By making it possible to switch between monitoring modes, the monitoring system 1 can provide a monitoring function that allows the degree of privacy protection and the level of monitoring function to be selected depending on the relationship between the person being monitored and the person watching over, and the situation of the person being monitored. Next, each function after the initial setting will be explained.

[0038] [Monitoring function (1) Emergency response function] FIG. 6 is a diagram showing an example of communication in the emergency response function of this embodiment. (Step S101) When something abnormal occurs to the person being watched over, the person being watched over operates emergency button 410 on remote control device 40. When remote control device 40 detects that emergency button 410 has been operated, it transmits first operation information to communication device 30. The first operation information is information indicating that emergency button 410 has been operated.

[0039] (Step S102) When the communication device control section 330 receives information indicating that the emergency button 410 has been operated from the remote control device 40, the communication device control section 330 transmits first operation information from the second communication section 320 to the terminal device 10. In the following description, the information that the communication device control unit 330 transmits to the terminal device 10 when the emergency button 410 is operated is also referred to as "first operation information." This expression includes a case where the communication device control unit 330 transfers the first operation information received from the remote control device 40 to the terminal device 10 as is, and a case where the communication device control unit 330 transmits information generated in response to receiving the first operation information from the remote control device 40 to the terminal device 10.

[0040] Here, if the communication device 30 is equipped with an imaging unit 350, when the communication device control unit 330 receives first operation information from the remote control device 40, it transmits the image captured by the imaging unit 350 and the first operation information from the second communication unit 320 to the terminal device 10. Furthermore, communication device 30 may include communication device sound collection unit 360. In this case, when communication device control unit 330 receives first operation information from remote control device 40, it transmits information on the sound collected by communication device sound collection unit 360 and the first operation information to terminal device 10. Furthermore, the communication device control unit 330 may transmit information about the sound collected by the communication device sound collection unit 360, without transmitting the image captured by the imaging unit 350. Furthermore, the communication device control unit 330 may transmit both the image captured by the imaging unit 350 and the information about the sound collected by the communication device sound collection unit 360 to the terminal device 10. In addition, when the communication device 30 transmits the image captured by the imaging unit 350 and the sound collected by the communication device sound collection unit 360 immediately after the image capture and sound collection, that is, in real time, this is also called "live view image and sound."

[0041] In the following description, it is assumed that the communication device 30 includes an imaging unit 350 and a communication device sound collection unit 360, and transmits live view images and sound to the terminal device 10.

[0042] Note that sound in the monitored area A can also be collected by a remote control sound collection unit 450 included in the remote control device 40 operated by the person being watched over. In this case, the sound collected by the remote control sound collection unit 450 is transmitted from the remote control communication unit 430 and acquired by the communication device 30. Here, after the emergency button 410 is operated, communication between the remote control device 40 and the communication device 30 may be interrupted for some reason. By including the communication device sound collection unit 360 in the communication device 30, even when communication between the communication device 30 and the remote control device 40 is not possible, the communication device 30 can collect ambient sound and provide a live view image and sound to the terminal device 10.

[0043] (Step S103) When the terminal device 10 receives the first operation information and the live view image and sound from the communication device 30, it displays on the presentation unit 120 an image indicating that the emergency button 410 has been operated.

[0044] Fig. 7 is a diagram showing an example of a pop-up display during an emergency response in this embodiment. When the terminal device 10 receives the first operation information from the communication device 30, it displays a pop-up image P31 as shown in Fig. 7. According to the monitoring system 1 configured in this manner, even when the user is not operating the terminal device 10 or is performing an operation other than the app of the monitoring system 1, it is possible to notify the user that the person being watched over has operated the emergency button 410.

[0045] FIG. 8 is a diagram showing an example of a live view display according to this embodiment. (Step S104) When the pop-up image P31 displayed in step S103 is operated (for example, tapped), the terminal device 10 displays a live view image P41, a response selection icon P42, and a conversation icon P43. 8, for convenience of explanation, the moving image of the watching target in the live view image P41 is represented by a line drawing. In this embodiment, the terminal device 10 displays not a line drawing but a moving image of the watching target that is actually captured by the imaging unit 350.

[0046] FIG. 9 is a diagram showing an example of the correspondence selection screen of this embodiment. (Step S105) When the action selection icon P42 is operated, the terminal device 10 displays the action selection screen P51 shown in FIG.

[0047] Here, the first operation information is an example of event information indicating an event in which the emergency button 410 has been operated. The response selection screen P51 includes icons of response contents according to the type of event. In other words, the event information includes the type of event, and the presentation unit 120 presents content according to the type of event included in the received event information.

[0048] When the terminal device 10 receives the first operation information, that is, when the type of the event is “operation of the emergency button 410”, the terminal device 10 may, for example, - The user rushes to the location of the person being watched (icon P511) Report to public institutions (fire department, ambulance, police, health center, city hall, etc.) (icon P512) Contact a pre-defined contact (icon P513) A response selection screen P51 including various icons such as an icon P514 for starting the phone application is displayed. That is, the response includes at least one of moving the user to the location to be monitored, notifying a public institution, and notifying other users. When any icon on the response selection screen P51 is operated, the terminal device 10 performs an action corresponding to the operated icon (for example, calling a predetermined contact). Furthermore, the terminal device 10 transmits the selected response content to the server device 20.

[0049] In other words, when the device communication unit 140 receives event information, the presentation unit 120 pops up an icon indicating the event information, and presents content according to the type of event in response to a user operation after the icon is displayed.

[0050] As described above, if the communication device 30 has a judgment unit (not shown) that judges whether or not an abnormality has occurred based on the results of posture analysis of the person being monitored, the judgment result by the judgment unit that "the person being monitored has fallen" can also be considered an example of event information.

[0051] (Step S106) Returning to Fig. 6, when the server device 20 receives the response content from the terminal device 10, the server device 20 stores the received response content together with the time information transmitted by the terminal device 10 as a response history in the server storage unit 220. In addition, the server device 20 transmits to the communication device 30 information indicating that the user has responded to the operation of the emergency button 410. (Step S107) The communication device 30 releases the emergency state and returns to normal operation.

[0052] [Information sharing with other terminal devices] (Step S108) The other terminal devices 10 (for example, terminal device 10-2 and terminal device 10-3) acquire from the server device 20 the response history stored in step S106.

[0053] Here, the monitoring system 1 provides a function that allows the selection result of the response content selected in step S105 to be shared with other terminal devices 10. For example, if the terminal device 10 that selected the response content in step S105 is the terminal device 10-1, the other terminal devices 10 (for example, the terminal device 10-2 and the terminal device 10-3) can acquire the selection result (i.e., the response content of the user of the terminal device 10-1 in response to the operation of the emergency button 410).

[0054] That is, the event information is transmitted to a plurality of users, and the presentation unit 120 presents the status of a response to the event information by any one of the plurality of users.

[0055] 10 is a diagram showing an example of a display screen of the response history in this embodiment. The terminal device 10 displays a response history screen P61 acquired from the server device 20. This response history screen P61 includes the name of the user who responded, the time of the response, and the content of the response.

[0056] In other words, the presentation unit 120 of the terminal device 10 presents a screen (action selection screen P51) for selecting a plurality of actions, and presents the content of the action selected by any one of the plurality of users.

[0057] As described above, a plurality of destination terminal devices 10 can be set for one communication device 30. In this case, upon receiving event information (for example, first operation information) from the remote control device 40, the communication device 30 transmits the event information to each of the plurality of destination terminal devices 10. At this time, the fact that a user of one of the terminal devices 10 has taken some kind of action in response to the event information (for example, rushing to the location of the person being watched over in response to the operation of the emergency button 410) can be shared among the users of the multiple terminal devices 10, allowing each user to determine whether or not further action is necessary. The monitoring system 1 configured in this way can give users a sense of security.

[0058] [Video display when emergency button is pressed] The monitoring system 1 may provide a function to display images captured by the imaging section 350 in the time period before and after the panic button 410 is operated on the video display screen P62 shown in FIG.

[0059] FIG. 11 is a diagram showing another example of communication in the emergency response function of this embodiment. (Step S101) As described with reference to Fig. 6, when some abnormality occurs in the person being watched over, the person being watched over operates emergency button 410 of remote control device 40. When remote control device 40 detects that emergency button 410 has been operated, it transmits first operation information to communication device 30.

[0060] (Step S201) The communication device 30 of this embodiment always performs imaging by the imaging unit 350 and sound collection by the communication device sound collection unit 360, regardless of whether the emergency button 410 is operated or not. The communication device control unit 330 stores the captured images and collected sounds in the communication device storage unit 390. Depending on the capacity of the communication device storage unit 390, the communication device control unit 330 may erase information captured and collected more than a predetermined period of time ago and store new images and sounds. Here, the predetermined period is, for example, 12 hours, 24 hours, or 7 days. (Step S202) When operation of the emergency button 410 is detected in step S101, the communication device 30 stores, based on a predetermined recording time, images and sounds from the most recent period out of the past images and sounds stored in the communication device storage unit 390, as well as images and sounds captured and collected after the operation of the emergency button 410 is detected, in the communication device storage unit 390. For example, if the predetermined recording time is 10 minutes, the communication device 30 stores in the communication device storage unit 390 images and sounds from 5 minutes before and 5 minutes after the operation of the emergency button 410 is detected.

[0061] (Step S203) The terminal device 10 requests, from the communication device 30, images and audio before and after the operation of the emergency button 410 was detected. For example, the user taps the play button image on the video display screen P62 shown in Fig. 10. When the terminal device 10 detects the tap on the play button image on the video display screen P62, it requests, from the communication device 30, images and audio before and after the operation of the emergency button 410 was detected. (Step S204) When the communication device 30 receives a request for images and sounds before and after the operation of the emergency button 410 is detected, it transmits to the terminal device 10 the images and sounds from the time period before and after the operation of the emergency button 410 that were saved in step S202. (Step S205) When the terminal device 10 receives from the communication device 30 images and sounds from the time period before and after the operation of the emergency button 410 is detected, the terminal device 10 plays back the received images and sounds.

[0062] Information indicating the operation of the panic button 410 (first operation information) is an example of event information. The event information includes information indicating a video of a time range including the time when the event occurred, and the presentation unit 120 presents the video. Here, the time range including the time when the event occurred is, for example, 10 minutes, 5 minutes before and 5 minutes after the operation of the panic button 410 described above. Note that the time width and the ratio between before and after shown here are merely examples, and the time width may be longer or shorter than 10 minutes, and the ratio between the before and after times may be equal or unequal, with the before (or after) time being longer. Furthermore, the information indicating the video may be the video file itself, or may be link information indicating the storage location of the video.

[0063] The monitoring system 1 configured in this manner can present to the terminal device 10 the situation before the panic button 410 was operated, that is, the situation during the time period when the cause that led to the operation of the panic button 410 occurred. This allows the user to understand the cause of the abnormality that occurred in the person being watched over, and enables the user to take more appropriate action.

[0064] [Display emergency button operation history] 12 is a diagram showing an example of an event history display according to this embodiment. As described above, the terminal device 10 includes the presentation unit 120 and the terminal communication unit 140. The terminal communication unit 140 receives event information relating to an event that has occurred in a location to be monitored. This event information includes, for example, first operation information indicating that the emergency button 410 has been operated. The presentation unit 120 presents a history of event information received by the terminal communication unit 140. As an example, the presentation unit 120 of the terminal device 10 displays, on an event history screen P71, events that occurred in the past among the events to be notified, together with the dates and times of occurrence of the events. The monitoring system 1 configured in this way can not only notify the user when an event (for example, an operation of the emergency button 410) occurs, but also present the user with events that have occurred in the past. Furthermore, the terminal device 10 may have a function of excluding (that is, deleting) an event selected by the user from the event history to be displayed. The terminal device 10 may also display the timing of occurrence of events in a calendar format, or may display detailed information about events that occurred on a date selected from the calendar.

[0065] Here, the event information includes the time at which the event occurred, and the presentation unit 120 presents the history of the event information based on the time at which the event occurred. Here, presentation unit 120 can also be configured to present a history of event information based on the time at which the event information arrives at terminal device 10. However, because event information is transmitted from remote control device 40 to terminal device 10 via communication device 30 and server device 20, there may be a discrepancy between the time at which the event information arrives and the time at which the event occurred, depending on the communication conditions, etc. Furthermore, for example, when multiple events occur, the order in which the events occurred and the order in which the event information arrived may be reversed. According to the monitoring system 1 configured to present the history of event information based on the time of occurrence of an event as in this embodiment, the event occurrence history can be presented more accurately regardless of the communication situation.

[0066] As described above, a communication device 30 equipped with an imaging unit 350 is installed in the monitored area A. Event information (in the above example, information indicating that the emergency button 410 has been operated; that is, first operation information) transmitted to the terminal device 10 includes information on an image (for example, a thumbnail image or a video) captured by the imaging unit 350. The presentation unit 120 presents this image information.

[0067] Furthermore, the terminal device 10 may mark an unopened event P72 with an event whose content has not been confirmed by the user, that is, an unopened event, from among the event histories displayed on the event history screen P71. That is, the presentation unit 120 may present that the event information has not been viewed by the user.

[0068] Furthermore, for an unopened event, if a predetermined time has passed since the event occurred and the event has remained unopened, the terminal device 10 may notify the user that the predetermined time has passed by changing the color of the unopened display P72, displaying a pop-up image, making a sound, vibrating, or the like. That is, the presentation unit 120 presents that a predetermined time has passed since the event information was not viewed by the user.

[0069] Furthermore, the terminal device 10 may change the color or shape of the unopened display P72 depending on the type of event. That is, the presentation unit 120 may present the event in different ways depending on the type of event indicated by the event information.

[0070] 13 is a diagram showing an example of a selection screen for events to be presented according to this embodiment. The terminal device 10 may narrow down notifications when an event occurs based on a user's setting operation on the event selection screen P81. The presentation unit 120 selects event information to be presented based on at least one of the event occurrence time and the event type.

[0071] The monitoring system 1 configured as above allows the user to easily grasp the situation when an event occurs, and allows the user to take a more appropriate response to the event.

[0072] [Monitoring function (2) Conversation function using remote control device] The monitoring system 1 provides a conversation function using the remote control device 40. For example, when the user taps the conversation icon P43 shown in Fig. 8 described above, a conversation can be held between the user and the person being watched over.

[0073] As described above, the remote control device 40 includes the remote control sound collection unit 450. When a conversation request is received from the terminal device 10, the communication device control unit 330 transmits information about the sound collected by the remote control sound collection unit 450 to the terminal device 10 based on the second operation information. Here, the second operation information is information indicating that the OK button 420 has been operated. That is, when the remote control device 40 detects an operation on the OK button 420, it transmits the second operation information. More specific steps for the conversation function will be explained below.

[0074] FIG. 14 is a diagram showing an example of communication using the conversation function of this embodiment. (Step S301) When the terminal device 10 detects a tap on the conversation icon P43, the terminal device 10 transmits a conversation request to the communication device 30. (Step S302) When communication device 30 receives a conversation request from terminal device 10, communication device sound output unit 370 outputs a conversation request sound. The person being watched over hears the conversation request sound, knows that a conversation request has been made, and decides whether or not to accept the conversation. If the person being watched over accepts the conversation, they operate OK button 420 on remote control device 40. (Step S303) When remote control device 40 detects an operation on OK button 420, it notifies terminal device 10, via communication device 30 and server device 20, of information indicating that OK button 420 has been operated. Here, the information indicating that OK button 420 has been operated is also referred to as second operation information. In other words, when remote control device 40 detects an operation on OK button 420, it transmits the second operation information.

[0075] (Step S304) When the terminal device 10, the server device 20, the communication device 30, and the remote control device 40 detect that the conversation request has been accepted in step S303, they start a two-way conversation. Here, the two-way conversation function is a function that enables a two-way conversation between the user and the person being watched over by outputting a sound collected by the terminal sound collection unit 150 of the terminal device 10 from the communication device sound output unit 370 of the communication device 30 and outputting a sound collected by the remote control sound collection unit 450 of the remote control device 40 from the terminal sound output unit 160 of the terminal device 10. In the two-way conversation function, the monitoring system 1 may display, on the terminal device 10, an image of the person being watched over during a conversation captured by the imaging unit 350. (Step S305) The communication device 30 captures an image of the monitoring target area A using the imaging unit 350. (Step S306) Remote control device 40 causes remote control sound collection section 450 to collect the voice of the person being watched over. (Step S307) Remote control device 40 transmits the collected voice to communication device 30 as voice information. (Step S308) The communication device 30 transmits to the terminal device 10 the image captured in step S305 and the audio information received from the remote control device 40 in step S307. (Step S309) The terminal device 10 presents the image and sound received from the communication device 30 to the user. (Step S310) In parallel with steps S305 to S309, the terminal device 10 transmits the sound collected by the terminal sound collection unit 150 to the communication device 30 as sound information. (Step S311) The communication device 30 receives voice information from the terminal device 10 and outputs it from the voice output unit 370 of the communication device.

[0076] According to the monitoring system 1 configured as described above, the voice of the person being watched over is picked up using the remote control sound pickup unit 450 of the remote control device 40 carried by the person being watched over, and therefore the voice can be picked up more clearly than when the voice is picked up using, for example, the communication device sound pickup unit 360 of the communication device 30. Furthermore, according to the monitoring system 1, the voice is produced using the communication device sound pickup unit 370 of the communication device 30, and therefore the voice can be produced at a louder volume or with better sound quality than when the voice is produced using, for example, the remote control sound pickup unit 460 of the remote control device 40. Therefore, according to the monitoring system 1, the quality of the conversation between the user and the person being watched over can be improved.

[0077] As described above, the remote control device 40 includes the OK button 420. This OK button 420 is an operation button different from the emergency button 410. The remote control device 40 transmits second operation information indicating an operation of the OK button 420. The communication device control unit 330 of the communication device 30 controls the type of information to be transmitted to the terminal device 10 based on receiving the second operation information from the remote control device 40. In the example described above, when the OK button 420 is operated, the communication device control unit 330 controls the type of information to be transmitted to the terminal device 10 by transmitting an image and audio to the terminal device 10.

[0078] [Monitoring function (3) Live View function] The monitoring system 1 provides a live view function that presents current images and sounds of the monitoring target area A in real time to the terminal device 10. In the live view function, the terminal device 10 displays, for example, the live view image P41 shown in FIG. In the conversation function described above, remote control sound collection section 450 of remote control device 40 collects sound, but in the live view function, communication device sound collection section 360 of communication device 30 collects sound, which is different. Furthermore, the live view function operates differently in private mode and public mode. In private mode, a permission procedure is required from the person being watched. An example of this procedure is described below.

[0079] <Live View function: Private mode> FIG. 15 is a diagram showing an example of communication of the live view function in the private mode of this embodiment. (Step S401) When the terminal device 10 detects a user operation requesting the live view function (for example, selection of the “Live” tab or the “Current Status” tab), the terminal device 10 transmits a live view request to the communication device 30. (Step S402) When the communication device 30 receives a live view request from the terminal device 10, it issues a live view request sound from the communication device sound output unit 370. The person being watched over hears the live view request sound, knows that a live view request has been made, and decides whether to accept the live view. If the person being watched over accepts the live view, they operate the OK button 420 on the remote control device 40. (Step S403) When remote control device 40 detects an operation on OK button 420, it notifies terminal device 10, via communication device 30 and server device 20, of information indicating that OK button 420 has been operated. Here, the information indicating that OK button 420 has been operated is also referred to as second operation information. In other words, when remote control device 40 detects an operation on OK button 420, it transmits the second operation information. In the above-described conversation function, remote control device 40 also transmits second operation information. The meaning of the operation of the OK button 420, that is, the meaning of the second operation information, changes depending on the requested function.

[0080] (Step S404) The communication device 30 captures an image of the monitoring target area A using the imaging unit 350. The communication device 30 also collects sounds in the monitoring target area A using the communication device sound collection unit 360. (Step S405) The communication device 30 transmits the image and audio information to the terminal device 10. (Step S406) The terminal device 10 presents the image and sound received from the communication device 30 to the user.

[0081] <Live View function: Public mode> 16 is a diagram showing an example of communication using the live view function in the public mode of this embodiment. In the public mode, the communication device 30 constantly captures images of the monitoring target area A using the imaging unit 350.

[0082] (Step S501) When the terminal device 10 detects a user operation requesting the live view function (for example, selection of the “Live” tab or the “Current Status” tab), the terminal device 10 transmits a live view request to the communication device 30. (Step S502) When the communication device 30 receives a live view request from the terminal device 10, it issues a live view request sound from the communication device sound output unit 370. The person being watched over hears the live view request sound, knows that a live view request has been made, and decides whether to permit the live view. If the person being watched over permits the live view, they operate the OK button 420 on the remote control device 40. (Step S503) When remote control device 40 detects an operation on OK button 420, remote control device 40 notifies terminal device 10 via communication device 30 and server device 20 of information indicating that OK button 420 has been operated.

[0083] (Step S504) The communication device 30 collects sounds in the monitoring target area A using the communication device sound collection unit 360. (Step S505) The communication device 30 transmits the image and audio information to the terminal device 10. (Step S506) The terminal device 10 presents the image and sound received from the communication device 30 to the user.

[0084] The live view request is also referred to as an image transmission request. That is, when an image transmission request is received from the terminal device 10, the communication device control unit 330 of the communication device 30 transmits the image captured by the imaging unit 350 to the terminal device 10 based on the second operation information.

[0085] Here, differences in the operation of the monitoring system 1 between the public mode and the private mode will be described with reference to the drawings.

[0086] Fig. 17 is a diagram showing an example of the operation of the monitoring system 1 in the public mode and the private mode of this embodiment. In Fig. 17, "OFF" indicates a state in which the button is not operated, and "ON" indicates a state in which the button is operated. Regarding images, in the public mode, an image captured by the imaging unit 350 is presented to the terminal device 10 regardless of the operation of the emergency button 410 and the OK button 420. On the other hand, in the private mode, when the emergency button 410 is operated, or when the OK button 420 is operated in response to a conversation request or a live view request (that is, when the person being watched over gives permission), an image captured by the imaging unit 350 is presented to the terminal device 10. That is, in the private mode, when neither the emergency button 410 nor the OK button 420 is operated (when they are OFF), no image is presented. Regarding audio, in both the public mode and the private mode, when the emergency button 410 is operated or the OK button 420 is operated in response to a conversation request or a live view request (that is, when the watching target gives consent), the collected audio is presented on the terminal device 10. In other words, regarding audio, in both the public mode and the private mode, when neither the emergency button 410 nor the OK button 420 is operated (when they are OFF), the audio is not presented.

[0087] Regarding audio, in both public mode and private mode, when the emergency button 410 is operated, the sound collected by the communication device sound collection unit 360 of the communication device 30 is presented. Also, in both public mode and private mode, when the OK button 420 is operated (i.e., when the person being watched over gives permission), the sound collected by the remote control sound collection unit 450 is presented in the case of a conversation request, and the sound collected by the communication device sound collection unit 360 is presented in the case of a live view request.

[0088] As described above, the meaning of the information indicating the operation of the OK button 420, that is, the second operation information, changes depending on the content of the request from the user and the operation mode. The second operation information is information indicating an operation to permit at least one of information about the picked-up sound and the captured image to be provided to the terminal device 10 in a private mode in which the image captured by the imaging unit 350 is not made public.

[0089] The second operation information is information indicating an operation to permit information about the collected sound to be provided to the terminal device 10 in a public mode in which the image captured by the imaging section 350 is made public.

[0090] [Monitoring function (4) Remote control device voltage drop notification function] FIG. 18 is a diagram showing an example of communication in the voltage drop notification function of this embodiment. (Step S601) Voltage detection unit 470 of remote control device 40 detects the power supply voltage of remote control device 40. If the detected power supply voltage is equal to or lower than a predetermined value, remote control control unit 440 transmits a voltage drop notification to terminal device 10. That is, when communication device control unit 330 receives voltage drop information from remote control device 40, it transmits information indicating that the power supply voltage of remote control device 40 has dropped to terminal device 10. (Step S602) The terminal device 10 receives the voltage drop notification. (Step S603) The terminal device 10 notifies the user of the voltage drop.

[0091] 19 is a diagram showing an example of a voltage drop notification screen according to this embodiment. In step S603, the terminal device 10 displays a voltage drop notification screen P91 shown in FIG. When the power supply voltage of the remote control device 40 is restored by replacing or charging the battery, the remote control device 40 may transmit a voltage recovery notification to the communication device 30 or the terminal device 10. The communication device 30 or the terminal device 10 may manage the period from when it receives the voltage drop notification to when it receives the voltage recovery notification as the power supply voltage drop period of the remote control device 40.

[0092] [Monitoring function (5) Remote control device search function] The monitoring system 1 provides a remote control device search function that makes the remote control device 40 ring when the remote control device 40 goes missing, thereby assisting in the search for the remote control device 40.

[0093] FIG. 20 is a diagram showing an example of communication in the remote control device search function of this embodiment. (Step S701) Communication device 30 includes a remote control device search switch (not shown). Communication device 30 detects operation of the remote control device search switch. This remote control device search switch is an example of an operation receiving unit that receives an operation to search for remote control device 40. (Step S702) When the operation of the remote control device search switch is detected, the communication device 30 transmits a remote control device search instruction to the remote control device 40. (Step S703) When remote control device 40 receives a remote control device search instruction from communication device 30, it emits a sound from remote control sound output unit 460 to notify the person being watched over or the user of its location.

[0094] In addition, the remote control device 40 may be equipped with a vibration unit (not shown) or a light-emitting unit (not shown), and when it receives a remote control device search instruction from the communication device 30, it may notify its location by vibration or light.

[0095] [Monitoring function (6) Remote control device out-of-range notification function] The monitoring system 1 provides a remote control device out-of-service notification function that notifies the terminal device 10 when the remote control device 40 is out of the communication range of the communication device 30.

[0096] FIG. 21 is a diagram showing an example of communication in the remote control device out-of-service area notification function of this embodiment. (Step S801) The communication device 30 determines whether communication with the remote control device 40 is possible. The communication device 30 may determine whether communication is possible periodically or in response to a request from the terminal device 10. If the communication device 30 is unable to communicate with the remote control device 40, the communication device 30 determines that the remote control device 40 is out of communication range, and transmits remote control out-of-range information to the terminal device 10. In addition, by combining with the voltage drop notification function of the remote control device described above, the communication device 30 may determine that the remote control device 40 is out of communication range when the power supply voltage of the remote control device 40 has not dropped and communication with the remote control device 40 is not possible.

[0097] (Step S802) The terminal device 10 receives remote control out-of-service area information from the communication device 30. (Step S803) The terminal device 10 notifies the user that the remote control device 40 is outside the communication range of the communication device 30.

[0098] As described above, the monitoring system 1 can provide a monitoring system that is highly convenient for both monitoring personnel and non-monitoring personnel.

[0099] Although the embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment and can be appropriately modified without departing from the spirit of the present invention. The configurations described in the above-described embodiments may be combined.

[0100] Each unit included in each device in the above-described embodiments may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

[0101] In addition, each part of each device may be composed of a memory and a CPU (central processing unit), and the functions of each part of each device may be realized by loading a program for realizing the function of each part into memory and executing the program.

[0102] In addition, a program for realizing the functions of each part of each device may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed, thereby performing processing by each part of the control unit. The term "computer system" here includes hardware such as the OS and peripheral devices.

[0103] Furthermore, if a WWW system is used, the "computer system" also includes the homepage provision environment (or display environment). "Computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording media" also includes devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over networks like the Internet or over communication lines like telephone lines, and devices that store programs for a fixed period of time, such as volatile memory within computer systems that serve as servers or clients. Furthermore, the programs may be programs that implement some of the aforementioned functions, or may be programs that can realize the aforementioned functions in combination with programs already stored in the computer system. [Explanation of symbols]

[0104] 1...Monitoring system, 10...Terminal device, 110...Terminal operation detection unit, 120...Presentation unit, 130...Terminal control unit, 140...Terminal communication unit, 150...Terminal sound collection unit, 160...Terminal sound generation unit, 30...Communication device, 310...First communication unit, 320...Second communication unit, 330...Communication device control unit, 340...Measurement unit, 350...Imaging unit, 360...Communication device sound collection unit, 370...Communication device sound generation unit, 380...Operation detection unit, 390...Communication device memory unit, 40...Remote control device, 410...Emergency button, 420...OK button, 430...Remote control communication unit, 440...Remote control unit, 450...Remote control sound collection unit, 460...Remote control sound generation unit, 470...Voltage detection unit, 480...NG button

Claims

[Claim 1] a receiving unit for receiving event information relating to an event occurring at a monitored location; a presentation unit that presents a history of the event information received by the reception unit; Equipped with There are a plurality of locations to be monitored, and user authority can be set for each of the locations to be monitored; The presentation unit presents content based on the authority set for the user for each of the locations to be monitored. Display control device.

Images