Monitoring system and control device

Abstract

This overseeing system comprises an acquisition unit that acquires data relating to the state of a subject, an imaging unit that starts capturing an image including the subject when the state of the subject is a prescribed state, and a control unit that keeps the imaging unit in a state in which imaging of the subject is impossible until the state of the subject is the prescribed state.

Description

Technical Field

[0001] It relates to a monitoring system and a control device.

Background Art

[0002] A monitoring system has been proposed that can remotely monitor residents living alone, the elderly, patients, etc. who spend the day alone (for example, Patent Document 1).

[0003] There is a need for a monitoring system that takes into account the privacy of the target person.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

[0005] According to the first disclosed aspect, the monitoring system includes an acquisition unit that acquires data related to the state of a target person, an imaging unit that starts imaging an image including the target person when the state of the target person is a predetermined state, and a control unit that sets the imaging unit in a state where imaging of the target person is impossible until the state of the target person becomes the predetermined state.

[0006] According to the second disclosed aspect, the monitoring system includes an imaging unit that starts imaging an image including the target person when the state of the target person estimated from the data acquired by the acquisition unit that acquires data related to the state of the target person is a predetermined state, and a control unit that sets the imaging unit in a state where imaging of the target person is impossible until the estimated state of the target person becomes the predetermined state.

[0007] According to a third aspect of the disclosure, the control device includes a control unit that puts an imaging unit, which captures an image including the subject, into a state where it is impossible to image the subject until the state of the subject, estimated from the data acquired by an acquisition unit that acquires data relating to the state of the subject, reaches a predetermined state.

[0008] Furthermore, the configuration of the embodiments described later may be modified as appropriate, and at least a part of it may be replaced with other components. Moreover, the configuration elements whose arrangement is not particularly limited may be arranged in positions that can achieve their function, not limited to the arrangement disclosed in the embodiments. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a diagram showing the configuration of the monitoring system according to the first embodiment. [Figure 2] Figure 2 is a diagram illustrating an example of the installation of a sensor, a sub-camera, and a master camera. [Figure 3] Figure 3(A) is a block diagram illustrating the configuration of the master camera, and Figure 3(B) is a block diagram illustrating the configuration of the slave camera. [Figure 4] Figures 4(A) and 4(B) illustrate the master camera according to this embodiment, while Figures 4(C) and 4(D) illustrate another example of the master camera according to this embodiment. [Figure 5] Figure 5 is a flowchart showing an example of the processing performed by the control unit of the master camera. [Figure 6] Figure 6 is a flowchart showing the details of the first process. [Figure 7] Figure 7 is a flowchart showing the details of the second process. [Figure 8] Figure 8 is a flowchart showing an example of the processing performed by the control unit of the sub-camera. [Figure 9] Figures 9(A) to 9(D) are time charts showing an example of the processes performed in the monitoring system according to the first embodiment. [Figure 10]Figure 10 is a flowchart showing an example of processing performed by the control unit of the master camera in the monitoring system according to the second embodiment. [Figure 11] Figure 11 is a flowchart (part 1) showing the details of the third process. [Figure 12] Figure 12 is a flowchart (part 2) showing the details of the third process. [Figure 13] Figure 13 is a flowchart detailing the fourth process. [Figure 14] Figure 14 is a flowchart showing an example of processing performed by the control unit of the sub-camera in the monitoring system according to the second embodiment. [Figure 15] Figures 15(A) and 15(B) are time charts showing an example of the processes performed in the monitoring system according to the second embodiment. [Figure 16] Figure 16 shows an example of a list of broadcasters. [Figure 17] Figure 17 is a flowchart showing an example of processing performed by the control unit of the master camera in the monitoring system according to the third embodiment. [Figure 18] Figure 18 is a flowchart showing the details of the fifth process. [Figure 19] Figure 19 is a flowchart showing the details of the sixth process. [Figure 20] Figure 20 is a flowchart showing an example of processing performed by the control unit of the child camera in the monitoring system according to the third embodiment. [Figure 21] Figure 21 is a time chart showing an example of the processing performed in the monitoring system according to the third embodiment. [Figure 22] Figure 22(A) is a diagram illustrating the hardware configuration of the control unit of the master camera, and Figure 22(B) is a diagram illustrating the hardware configuration of the control unit of the slave camera. [Figure 23] Figure 23 shows the configuration of a modified monitoring system. [Figure 24] Figure 24 is a functional block diagram showing the configuration of the control device in a modified example.

Best Mode for Carrying Out the Invention

[0010] 《First Embodiment》 Hereinafter, the monitoring system 100 according to the first embodiment will be described in detail based on FIGS. 1 to 9(D). In FIG. 1, the configuration of the monitoring system 100 is shown in a block diagram.

[0011] The monitoring system 100 is a system for a person (family member, relative, helper, care manager, etc.) related to the person to be monitored TR1 (hereinafter referred to as the target person TR1) to monitor the target person TR1. In the following description, the person who monitors the person related to the target person TR1 is referred to as the monitor OB1. As shown in FIG. 1, the monitoring system 100 according to the present embodiment includes a home-side system 150, a service server SS, and a mobile terminal MT1.

[0012] The home-side system 150 (more specifically, the master camera 40), the service server SS of the monitoring system 100, and the mobile terminal MT1 are connected via a network NW including a public wireless LAN, the Internet, a mobile phone line network, and the like. The communication between the home-side system 150, the service server SS, and the mobile terminal MT1 can be, for example, communication based on the NICE (Network of Intelligent Camera Ecosystem) specification.

[0013] The mobile terminal MT1 is a mobile terminal possessed by the person OB1 who monitors the target person TR1, and is, for example, a smartphone, a tablet terminal, or a notebook PC (Personal Computer). Instead of the mobile terminal MT1, a terminal such as a desktop PC of the person OB1 who monitors may be used. An application for using the monitoring system is installed on the mobile terminal MT1. By the application, the person OB1 who monitors can give an instruction to the home-side system 150, receive a notification from the home-side system 150, or view images captured by the slave camera 20 and the master camera 40 provided in the home-side system 150.

[0014] The service server SS is a server that provides monitoring services. Based on requests from the home system 150, the service server SS pushes messages to the mobile terminal MT1 and sends instructions to the home system 150 in response to the operation of applications installed on the mobile terminal MT1.

[0015] The home system 150 is installed, for example, in the residence H1 where the subject TR1 resides. The home system 150 includes, for example, a sensor 10, a sub-camera 20, a master camera 40, and a remote controller 30.

[0016] Sensor 10, sub-camera 20, and master camera 40 are installed in the residence H1 where subject TR1 resides. Sensor 10, sub-camera 20, and master camera 40 are installed in places where subject TR1 stays and uses, such as living rooms, corridors, bathrooms, washrooms, and toilets.

[0017] Figure 2 is a diagram illustrating an example of the installation of sensor 10, slave camera 20, and master camera 40. In the example in Figure 2, dwelling H1 has three rooms R1 to R3. Sensor 10 is installed in each of rooms R1 to R3. In Figure 2, the sensors 10 installed in each of rooms R1 to R3 are referred to as sensors 10-1 to 10-3. Note that in Figure 2, there is one sensor 10 installed in each room, but there may be multiple sensors installed in each room.

[0018] At least one camera is installed in each room where sensor 10 is installed. In the example in Figure 2, a master camera 40 is installed in room R1 where sensor 10-1 is installed, and slave cameras 20 are installed in rooms R2 and R3 where sensors 10-2 and 10-3 are installed, respectively. In Figure 2, the slave cameras 20 installed in rooms R2 and R3 are referred to as slave cameras 20-1 and 20-2, respectively. The slave cameras 20 and master camera 40 may be installed on the ceiling, on the wall, or placed at any location within the room. In Figure 2, there is one camera installed in each room, but there may be multiple cameras installed in each room. For example, a master camera 40 and slave cameras 20 may be installed in room R1, or multiple slave cameras 20 may be installed in room R2.

[0019] The sensors 10 (sensors 10-1 to 10-3 in the example in Figure 2) and the master camera 40 are connected, for example, by DECT (Digital Enhanced Cordless Telecommunications). The sensors 10 and the master camera 40 may also be connected by proximity communication such as wired LAN (Local Area Network), wireless LAN, Wi-Fi, or Bluetooth (registered trademark).

[0020] The sub-cameras 20 (sub-cameras 20-1 and 20-2 in the example in Figure 2) and the master camera 40 are connected, for example, by Wi-Fi. The sub-cameras 20 and the master camera 40 may also be connected by wired LAN, wireless LAN, or proximity communication such as Bluetooth®.

[0021] The remote controller 30 and the master camera 40 are connected, for example, by DECT. The remote controller 30 and the master camera 40 may also be connected by proximity communication such as wireless LAN, Wi-Fi, or Bluetooth (registered trademark).

[0022] (Sensor 10) Sensor 10 acquires data regarding the state of the subject TR1. In this embodiment, sensor 10 acquires data regarding the state of the subject TR1 other than visible light images. Examples of sensor 10 include infrared array sensors, infrared cameras, depth sensors, radio wave sensors (millimeter-wave radar), vibration sensors, sound sensors, wearable sensors, thermometers, and hygrometers. In this embodiment, sensor 10 is described as an infrared array sensor. Note that a camera with a filter that cuts out visible light may be used as sensor 10.

[0023] In this embodiment, for example, the sensor 10 is mounted on the ceiling. If the sensor 10 is an infrared array sensor, the sensor 10 acquires, for example, temperature distribution data below the sensor 10. Unlike visible light images, this temperature distribution data does not allow the subject's face or clothing to be seen, thus protecting the privacy of the subject TR1. Furthermore, since the infrared array sensor does not have a lens, it can suppress the subject TR1 from feeling that they are being "monitored" or that their privacy is being violated, thereby reducing the psychological burden on the subject TR1. The sensor 10 may also be mounted on a wall or the like.

[0024] The data acquired by sensor 10 is transmitted to master camera 40.

[0025] (Remote controller 30) The remote controller 30 communicates with the master camera 40 and sends instructions to the master camera 40 in response to operations performed on the remote controller 30. The remote controller 30 may be equipped with, for example, buttons or a touch panel for inputting various instructions. The remote controller 30 may also be equipped with at least one of a microphone for voice input of instructions and a speaker for voice notification to the subject TR1. Alternatively, the remote controller 30 may be a terminal such as a smartphone with the monitoring system 100 application installed.

[0026] (Main camera 40) The master camera 40 captures a visible light image (hereinafter referred to as "image"). In this embodiment, the home system 150 is equipped with one master camera 40, but it may be equipped with multiple master cameras 40.

[0027] Figure 3(A) is a block diagram illustrating the configuration of the master camera 40. The master camera 40 comprises an imaging unit 41, a microphone 42, a speaker 43, a storage unit 44, a first communication module 45, a second communication module 46, a third communication module 47, a control unit 48, and a drive unit 49.

[0028] The imaging unit 41 includes a lens and an image sensor, and captures an image within the imaging range.

[0029] The microphone 42 acquires voice and other sounds emitted by the subject TR1 and transmits them to the control unit 48. The microphone 42 may also be used as the sensor 10.

[0030] The speaker 43 outputs a predetermined sound under the control of the control unit 48.

[0031] The storage unit 44 is, for example, a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores the images captured by the imaging unit 41.

[0032] The first communication module 45 is, for example, a DECT module, which enables communication with the sensor 10 and the remote controller 30.

[0033] The second communication module 46 is, for example, a Wi-Fi module, which enables communication with the slave camera 20.

[0034] The third communication module 47 is, for example, an LTE (Long Term Evolution) module, which enables communication with the service server SS and the mobile terminal MT1.

[0035] The control unit 48 controls the operation of the entire master camera 40. Details of the processes performed by the control unit 48 will be described later.

[0036] The drive device 49 is, for example, an actuator such as a motor, which is driven based on instructions from the control unit 48 to move the cover 422 (details will be described later) of the master camera 40.

[0037] (Sub-camera 20) The sub-camera 20 captures visible light images, similar to the master camera 40. In this embodiment, the home system 150 is equipped with one or more sub-cameras 20. However, if monitoring of the subject TR1 is possible with only the master camera 40, the sub-cameras 20 may be omitted.

[0038] Figure 3(B) is a block diagram illustrating the configuration of the sub-camera 20. The sub-camera 20 comprises an imaging unit 21, a microphone 22, a speaker 23, a storage unit 24, a second communication module 26, a control unit 28, and a drive unit 29.

[0039] The slave camera 20 differs from the master camera 40 in that the first communication module 45 and the third communication module 47 are omitted. In the slave camera 20, the second communication module 26 is, for example, a Wi-Fi module, enabling communication with the master camera 40. The other configurations are the same as those of the master camera 40, so a detailed explanation is omitted.

[0040] Next, the structural features of the slave camera 20 and the master camera 40 according to this embodiment will be described. Since the structures of the slave camera 20 and the master camera 40 are almost identical, the master camera 40 will be described here.

[0041] Figures 4(A) and 4(B) illustrate a master camera 40 according to this embodiment. As shown in Figures 4(A) and 4(B), for example, the master camera 40 includes a lens 421 and a cover 422 that covers the lens 421.

[0042] The cover 422 moves up and down when driven by a drive unit 49 based on the control of the control unit 48. The control unit 48 keeps the imaging unit 41 (lens 421) of the master camera 40 in a state where it cannot capture an image (as shown in Figure 4(B)) until it determines, based on data from the sensor 10, that there is an abnormality in the subject TR1. Specifically, it controls the drive unit 49 so that the cover 422 covers the lens 421. In other words, the control unit 48 keeps the lens 421 of the master camera 40 out of the view of the subject TR1 until it determines that there is an abnormality in the subject TR1. As a result, the imaging unit 41 (lens 421) of the master camera 40 is physically unable to capture an image.

[0043] Since the lens 421 is covered by the cover 422, the subject TR1 cannot see the lens 421. This suppresses the subject TR1 from having feelings such as "being monitored" or "having their privacy violated" during normal times (when no abnormality is occurring in the subject TR1).

[0044] If the control unit 48 determines that an abnormality has occurred in the subject TR1, it moves the cover 422 downwards, for example, so that the lens 421 is in a state where it can capture an image (the state shown in Figure 4(A)). In this way, the master camera 40 has a different appearance depending on whether it is in a state where it can capture an image or a state where it cannot. In other words, the master camera 40 has a first appearance (an appearance in which the lens 421 is visible) when an abnormality has occurred in the subject TR1, and a second appearance (an appearance in which the lens 421 is not visible) otherwise. In this way, the subject TR1 can know from the appearance of the master camera 40 that no image including themselves has been captured, so they can easily confirm whether their privacy is protected and gain a sense of security. The color of the cover 422 may be different from the color of the master camera 40 housing, or an "X" mark may be placed on the cover 422 so that it is easy to see that the lens 421 is covered by the cover 422.

[0045] In addition, Figures 4(A) and 4(B) describe an example in which the master camera 40 is equipped with a cover 422 that covers the lens 421 under normal conditions, but the configuration of the master camera 40 is not limited to this.

[0046] Figures 4(C) and 4(D) show alternative examples of the master camera 40. The master camera 40 shown in Figures 4(C) and 4(D) includes a lens 421 and an LED light 423.

[0047] In the master camera 40 shown in Figures 4(C) and 4(D), the control unit 48 illuminates the LED light 423 with a first color (for example, red) when the imaging unit 41 of the master camera 40 is unable to capture an image, and illuminates the LED light 423 with a second color (for example, green) that is different from the first color when the imaging unit 41 is able to capture an image. As a result, the subject TR1 can tell whether the master camera 40 is able to capture an image or not by the color of the LED light 423.

[0048] Furthermore, the control unit 48 may change the imaging direction of the master camera 40 (the orientation of the lens 421) to a direction different from the direction in which the preset imaging range exists, or it may keep the master camera 40 (or lens 421) in a location where the subject TR1 cannot see it (for example, under the bed). Even with this method, the master camera 40 or lens 421 cannot be seen by the subject TR1, thus suppressing the subject TR1 from having a feeling of being "monitored" or "having their privacy violated." Furthermore, if the orientation of the lens 421 is changed to a direction different from the direction in which the preset imaging range exists, the image may be captured in a way that does not include the subject TR1.

[0049] Next, we will explain the processes performed in the monitoring system 100. First, we will explain the processes performed by the control unit 48 of the master camera 40. Figure 5 is a flowchart showing an example of the processes performed by the control unit 48 of the master camera 40.

[0050] In the process shown in Figure 5, first, the control unit 48 acquires data regarding the state of the subject TR1 from the sensor 10 (step S11). In this embodiment, since the sensor 10 is an infrared array sensor, the control unit 48 acquires the temperature distribution data detected by the sensor 10.

[0051] Next, the control unit 48 estimates the state of the subject TR1 based on the acquired data (in this embodiment, temperature distribution data) (step S12). For example, the control unit 48 estimates that the subject TR1 is standing if the highest temperature included in the temperature distribution data is equal to or greater than the first temperature, and determines that the subject TR1 is sitting if the lowest temperature is equal to or greater than the second temperature, the highest temperature is equal to or less than the third temperature, and the area of ​​the region where the temperature is equal to or greater than the second temperature and less than or less than the third temperature is within a predetermined range. The control unit 48 also estimates that the subject TR1 is lying down if the lowest temperature is equal to or greater than the second temperature, the highest temperature is equal to or less than the third temperature, and the area of ​​the region where the temperature is equal to or greater than the second temperature and less than or less than the third temperature is larger than a predetermined area. For estimating the state of the subject TR1 using an infrared array sensor, for example, the method described in IEICE Technical Report, vol. 114, no. 166, ASN2014-81, pp. 219-224, July 2014 can be used.

[0052] Next, the control unit 48 determines whether the estimated state of subject TR1 is a state in which an abnormality has occurred in subject TR1 (a predetermined state) (step S13). If no abnormality has occurred in subject TR1 (step S13 / NO), the process returns to step S11. A state in which an abnormality has occurred in subject TR1 is, for example, a state in which subject TR1 has fallen or has remained motionless in the same posture for a long period of time.

[0053] If an abnormality occurs in subject TR1 (step S13 / YES), the control unit 48 identifies a camera located in the same place (room, etc.) as the sensor 10 that detected the abnormality in subject TR1 (step S14). For example, the storage unit 44 stores a table in which sensors and cameras installed in each room are associated, and the control unit 48 can identify a camera located in the same place (room, etc.) as the sensor 10 that detected the abnormality in subject TR1 by referring to this table.

[0054] Next, the control unit 48 determines whether the camera identified in step S14 is the master camera 40 (step S15). For example, in the example in Figure 2, if the sensor 10 that detected the abnormality of subject TR1 is sensor 10-1 installed in room R1, then the camera identified in step S14 is the master camera 40. If the camera identified in step S14 is the master camera 40 (step S15 / YES), the control unit 48 executes the first process (step S20).

[0055] Figure 6 is a flowchart detailing the first process. In the process shown in Figure 6, first, the control unit 48 sends a predetermined message to the mobile terminal MT1 (step S201). For example, the control unit 48 sends a message indicating that there may be an abnormality in the target person TR1. The message to the mobile terminal MT1 may be sent via the service server SS or directly from the control unit 48.

[0056] Next, the control unit 48 starts timing the image acquisition waiting time (step S203). The image acquisition waiting time is a predetermined time until the slave camera 20 or the master camera 40 starts acquiring an image including the subject TR1. The slave camera 20 and the master camera 40 may acquire images that do not include the subject TR1 before or after acquiring an image including the subject TR1. This allows the slave camera 20 and the master camera 40 to be used, for example, as security cameras or cameras for monitoring pets. Any time can be set as the image acquisition waiting time. During the image acquisition waiting time, the slave camera 20 and the master camera 40 do not acquire images including the subject TR1. However, for slave cameras 20 that are not located in the same place as the sensor 10 that detected the abnormality of the subject TR1 (in the example in Figure 2, slave cameras 20-1 and 20-2), images may be acquired because the images do not include the subject TR1. This allows the sub-camera 20 and the main camera 40 to be used, for example, as security cameras or cameras for monitoring pets.

[0057] Next, the control unit 48 notifies the subject TR1 about the start of imaging (step S205). Specifically, the control unit 48 notifies the subject TR1 about the start of imaging using the speaker 43. For example, the control unit 48 outputs a voice message such as "Imaging will start in 5 seconds" from the speaker 43 to notify the subject TR1 of the timing when imaging by the master camera 40 will start. Alternatively, the control unit 48 may notify the subject TR1 via the speaker 43 of the time until imaging starts, that the current time is the imaging waiting period, or that the current time is within the period when imaging can be prohibited (cancelled). This allows the subject TR1 to know the timing when image acquisition will start, the time until image acquisition starts, that image acquisition has not yet started, or that it is possible to prohibit imaging.

[0058] The order of processing in steps S201 to S205 can be changed arbitrarily.

[0059] Next, the control unit 48 determines whether or not it has received an input prohibiting image capture (step S207). Specifically, the control unit 48 performs speech recognition processing on the sound input from the microphone 42 and determines whether or not the input sound contains a predetermined voice for prohibiting image capture (for example, "No Image Capture," "Don't Take Pictures," "Stop," "False Detection," "No Abnormality," "Stop," etc.). If the voice for prohibiting image capture is included, the control unit 48 determines that it has received an input prohibiting image capture.

[0060] If an input to prohibit imaging is received (step S207 / YES), it is considered that the judgment in step S13 of Figure 5 was incorrect (misjudgment / false detection) because subject TR1 is in a state where an input to prohibit imaging is possible (no abnormality has occurred). In this case, the control unit 48 sends a predetermined message to the mobile terminal MT1 (step S225). For example, when the control unit 48 receives an input to prohibit imaging, it sends a message notifying that the abnormality was a false detection or that no abnormality has occurred in subject TR1. As a result, the person OB1 who possesses the mobile terminal MT1 (the person watching over subject TR1) can be assured that subject TR1 is safe.

[0061] Next, the control unit 48 stops timing the imaging waiting time (step S227) and returns to step S11 in Figure 5. As a result, imaging by the master camera 40 does not start. Therefore, it is possible to prevent subject TR1 from being imaged even if no abnormality has occurred, and the privacy of subject TR1 can be protected.

[0062] If no input prohibiting image capture has been received (step S207 / NO), the control unit 48 determines whether or not it has received an image capture request from the mobile terminal MT1 that includes the subject TR1 (step S209). For example, when the "Image Capture Request" button is pressed (tapped) on an application installed on the mobile terminal MT1, an image capture request is sent from the mobile terminal MT1. The image capture request may be sent directly from the mobile terminal MT1 to the master camera 40, or it may be sent via the service server SS.

[0063] If no imaging request has been received (step S209 / NO), the control unit 48 determines whether the imaging waiting time has elapsed (step S211). Specifically, it determines whether the imaging waiting time has elapsed since the timing of the imaging waiting time started in step S203. If the imaging waiting time has not elapsed (step S211 / NO), the process returns to step S207.

[0064] When an imaging request is received (step S209 / YES) or when the imaging waiting time has elapsed (step S211 / YES), the control unit 48 moves the cover 422 covering the lens 421 to expose the lens 421 and causes the imaging unit 41 to start capturing an image including the subject TR1 (step S213). At this time, the control unit 48 may also record the image captured by the imaging unit 41 in the storage unit 44.

[0065] When image acquisition including subject TR1 begins, the control unit 48 starts timing the distribution waiting time (step S215). The distribution waiting time is a predetermined time from the start of image acquisition until the start of image distribution. During the distribution waiting time, images including subject TR1 are not distributed. However, for example, if the sub-camera 20 is capturing images that do not include subject TR1, images that do not include subject TR1 may be distributed. This allows the sub-camera 20 and the master camera 40 to be used, for example, as security cameras or cameras for monitoring pets.

[0066] Next, the control unit 48 notifies the target user TR1 regarding the distribution (step S217). For example, the control unit 48 outputs an audio message to the speaker 43 such as "Distribution will begin in 5 seconds" to notify the target user TR1 of the timing when the image distribution will begin. Alternatively, instead of notifying the target user TR1 of the timing when the image distribution will begin, the time until the image distribution will begin, that the current time is within the distribution waiting period, or that the current time is within the period when distribution can be prohibited. This allows the target user TR1 to know when the image distribution will begin, how much time is left until the image distribution will begin, that the image distribution has not yet begun, or that distribution can be prohibited.

[0067] Next, the control unit 48 determines whether or not it has received an input prohibiting the distribution of images (step S219). For example, the control unit 48 performs speech recognition processing on the sound input from the microphone 42 and determines whether or not the input sound contains a predetermined voice for prohibiting the distribution of images (for example, "distribute prohibited," "do not distribute," "distribute stop," "false detection," "no abnormality," "stop," etc.). If the predetermined voice is included, the control unit 48 determines that it has received an input prohibiting the distribution of images.

[0068] If an input is received prohibiting the distribution of images (step S219 / YES), it is considered that the judgment in step S13 of Figure 5 was incorrect (misjudgment / false detection). In this case, the control unit 48 sends a predetermined message to the mobile terminal MT1 (step S231). For example, the control unit 48 sends a message notifying that the abnormality was a false detection, or that there is no abnormality in subject TR1. As a result, the person OB1 who possesses the mobile terminal MT1 (the person watching over subject TR1) can be assured that subject TR1 is safe.

[0069] Next, the control unit 48 stops timing the distribution waiting time and stops image acquisition by the imaging unit 41 (step S233). This prevents the distribution of images, thus preventing the distribution of images of subject TR1 even if no abnormality occurs in subject TR1, and thus protecting subject TR1's privacy. If images are recorded in the storage unit 44, the recorded images may be deleted. This further protects subject TR1's privacy.

[0070] Next, the control unit 48 drives the drive device 49 to cover the lens 421 with the cover 422 (step S235). This allows the subject TR1 to visually confirm that no image is being captured, thus giving the subject TR1 a sense of security that their privacy is protected. After that, the process returns to step S11 in Figure 5.

[0071] If no input to prohibit image distribution is received (step S219 / NO), the control unit 48 determines whether the distribution waiting time has elapsed (step S221). Specifically, it determines whether the distribution waiting time has elapsed since the start of timing the distribution waiting time in step S215. If the distribution waiting time has not elapsed (step S221 / NO), the process returns to step S219.

[0072] If the waiting time for distribution has elapsed (step S221 / YES), the control unit 48 starts distributing the image (step S223) and terminates the first process (step S20). The image distribution may be live view distribution, or it may be an image from a predetermined time (for example, a few seconds) before the current time. It may also be possible to view past images retrospectively.

[0073] On the other hand, if the camera identified in step S14 of Figure 5 is the sub-camera 20 (step S15 / NO), that is, if the camera installed in the same location as the sensor 10 that detected the abnormality of the subject TR1 is the sub-camera 20, the control unit 48 performs a second process (step S30).

[0074] Figure 7 is a flowchart detailing the second process. Figure 8 is a flowchart showing an example of the process performed by the control unit 28 of the slave camera 20. The second process will be explained in conjunction with the process performed by the control unit 28 of the slave camera 20.

[0075] The processes in steps S301 and S303 in Figure 7 are the same as steps S201 and S203 of the first process shown in Figure 6, so their explanation is omitted.

[0076] When the timing of the image acquisition waiting time begins (step S303), the control unit 48 instructs the slave camera 20 to notify it of the start of image acquisition (step S305).

[0077] Meanwhile, the control unit 28 of the slave camera 20 waits until it receives a notification instruction from the master camera 40 regarding the start of imaging (Figure 8: step S401 / NO). When the control unit 28 receives a notification instruction from the master camera 40 regarding the start of imaging (step S401 / YES), it uses the speaker 23 to make a notification regarding the start of imaging, similar to step S205 in Figure 6 (step S403).

[0078] Next, the control unit 28 determines whether or not it has received an imaging instruction from the master camera 40 (step S405). If it has not received an imaging instruction (step S405 / NO), the control unit 28 determines whether or not it has received an input from the subject TR1 prohibiting imaging (step S421). For example, the control unit 28 performs speech recognition processing on the sound input from the microphone 22 and determines whether or not the input sound contains a predetermined sound to prohibit image capture. If the predetermined sound is included, the control unit 28 determines that it has received an input to prohibit image capture.

[0079] If no input to prohibit imaging has been received (step S421 / NO), the process returns to step S405. If an input to prohibit imaging has been received (step S421 / YES), the control unit 28 transmits information indicating that an input to prohibit imaging has been received (imaging prohibition instruction) to the master camera 40 (step S423), and terminates the process shown in Figure 8.

[0080] Meanwhile, the control unit 48 of the master camera 40 determines whether or not it has received an image capture prohibition instruction from the slave camera 20 (Figure 7: step S307). If an image capture prohibition instruction is received from the slave camera 20 (step S307 / YES), the processing in steps S325 and S327 is executed, similar to steps S225 and S227 in Figure 6, and the process returns to step S11 in Figure 5.

[0081] If no instruction to disable imaging is received from the sub-camera 20 (step S307 / NO), the control unit 48 executes the processes of steps S309 and S311, similar to steps S209 and S211 in Figure 6.

[0082] When the control unit 48 receives an imaging request from the mobile terminal MT1 (step S309 / YES), or when the imaging waiting time has elapsed (step S311 / YES), it sends an imaging instruction to the slave camera 20 (step S313).

[0083] When the control unit 28 of the slave camera 20 receives an imaging instruction from the master camera 40 (Figure 8: step S405 / YES), it drives the drive device 29 to move the cover, thereby exposing the lens of the slave camera 20 and causing the imaging unit 21 to start imaging (step S407).

[0084] Subsequently, the control unit 28 waits until it receives instructions from the master camera 40 to send a notification regarding the distribution (step S409 / NO).

[0085] Meanwhile, the control unit 48 of the master camera 40 starts timing the distribution waiting time after transmitting the imaging instruction (Figure 7: step S315). Then, the control unit 48 instructs the slave camera 20 to send a notification regarding distribution (step S317).

[0086] When the control unit 28 of the slave camera 20 receives an instruction from the master camera 40 to send a notification regarding distribution (Figure 8: step S409 / YES), it sends a notification regarding distribution to the target user TR1 (step S411). For example, the control unit 28 outputs an audio message such as "Distribution will start in 5 seconds" to the speaker 23 to notify the target user TR1 of the timing when image distribution will start. Alternatively, instead of notifying the target user TR1 of the timing when image distribution will start, the time until image distribution will start, that the current time is within the distribution waiting period, or that the current time is within the period when distribution can be prohibited. This allows the target user TR1 to know when image distribution will start, how much time is left until image distribution starts, that image distribution has not yet started, or that distribution can be prohibited.

[0087] Next, the control unit 28 determines whether or not it has received an input that prohibits the distribution of images (step S413). For example, the control unit 28 performs speech recognition processing on the sound input from the microphone 22 and determines whether or not the input sound contains a predetermined sound for prohibiting the distribution of images. If the predetermined sound is included, the control unit 28 determines that it has received an input that prohibits the distribution of images.

[0088] If the control unit 28 receives an input to prohibit the distribution of images (step S413 / YES), it sends information to the master camera 40 indicating that it has received an input to prohibit distribution (distribution prohibition instruction) (step S425).

[0089] The control unit 28 stops imaging of the subject TR1 by the imaging unit 21 (step S417), drives the drive device 29 to cover the lens of the sub-camera 20 with a cover (step S419), and ends the process shown in Figure 8. If an image is recorded in the storage unit 24, the recorded image may be deleted. This further protects the privacy of the subject TR1.

[0090] Meanwhile, the control unit 48 of the master camera 40 determines whether or not it has received a distribution prohibition instruction from the slave camera 20 (Figure 7: step S319). If it receives a distribution prohibition instruction from the slave camera 20 (step S319 / YES), the control unit 48 sends a predetermined message to the mobile terminal MT1 (step S329). For example, the control unit 48 sends a message notifying that the abnormality was a false positive or that there is no abnormality in the subject TR1. This allows the person OB1 who is carrying the mobile terminal MT1 (the person watching over the subject TR1) to know that the subject TR1 is safe.

[0091] Next, the control unit 48 stops timing the distribution waiting time (step S331). This prevents the distribution of images, thus preventing the distribution of images of subject TR1 even though no abnormality has occurred in subject TR1, and thus protecting subject TR1's privacy. After that, the process returns to step S11 in Figure 5.

[0092] If no instruction to disable distribution has been received from the sub-camera 20 (step S319 / NO), the control unit 48 determines whether the distribution waiting time has elapsed (step S321). If the distribution waiting time has not elapsed (step S321 / NO), the process returns to step S319.

[0093] If the distribution waiting time has elapsed (step S321 / YES), the control unit 48 instructs the sub-camera 20 to continue capturing images including the subject TR1 (step S323), and starts distributing the images including the subject TR1 (step S324).

[0094] Meanwhile, in Figure 8, if the control unit 28 of the slave camera 20 has not received an input prohibiting distribution (step S413 / NO), it determines whether or not it has received a recording continuation instruction from the master camera 40 (step S415). If it has not received a recording continuation instruction (step S415 / NO), it returns to step S413.

[0095] If an instruction to continue shooting is received (step S415 / YES), the control unit 28 instructs the imaging unit 21 to continue capturing an image including the subject TR1 (step S416), and terminates the process shown in Figure 8.

[0096] Next, an example of the processes performed in the monitoring system 100 will be explained using the time charts in Figures 9(A) to 9(D). In Figures 9(A) to 9(D), the vertical axis represents time.

[0097] First, let's explain the example in Figure 9(A). In Figure 9(A), let's assume that at time t1, the control unit 48 determines that an abnormality has occurred in subject TR1. In this case, the control unit 48 starts timing the imaging waiting time at time t1.

[0098] Here, if an input to prohibit imaging is received at time t3, which is before time t2, when the imaging waiting time T1 has elapsed, the control unit 48 stops timing the imaging waiting time. Therefore, in the example of Figure 9(A), an image including subject TR1 is not captured, preventing an image from being captured even though there is no abnormality in subject TR1.

[0099] Next, let's explain the example in Figure 9(B). In Figure 9(B), let's assume that at time t1, the control unit 48 determines that there is an abnormality in subject TR1. In this case, the control unit 48 starts timing the imaging waiting time at time t1.

[0100] If an imaging request is received at time t3, which is before time t2, when the imaging waiting time T1 has elapsed, the imaging unit 21 or 41 starts capturing an image including the subject TR1, and the control unit 48 starts timing the distribution waiting time.

[0101] When the distribution waiting time T2 has elapsed at time t4, the control unit 48 starts distributing the image. In the example in Figure 9(B), an image including subject TR1 is distributed to the mobile terminal MT1 after time t4.

[0102] Next, let's explain the example shown in Figure 9(C). In Figure 9(C), let's assume that at time t1, the control unit 48 determines that an abnormality has occurred in subject TR1. In this case, the control unit 48 starts timing the imaging waiting time at time t1.

[0103] When the imaging waiting time T1 has elapsed at time t2, the imaging unit 21 or 41 starts capturing an image including the subject TR1, and the control unit 48 starts timing the distribution waiting time.

[0104] If the control unit 48 receives an input to prohibit distribution at time t4, which is before time t3, when the distribution waiting time T2 has elapsed, the control unit 48 stops timing the distribution waiting time. Therefore, in the example in Figure 9(C), the image including subject TR1 is not distributed to the mobile terminal MT1, preventing the image from being distributed even though there is no abnormality in subject TR1.

[0105] Next, let's explain the example shown in Figure 9(D). In Figure 9(D), let's assume that at time t1, the control unit 48 determines that an abnormality has occurred in subject TR1. In this case, the control unit 48 starts timing the imaging waiting time at time t1.

[0106] When the imaging waiting time T1 has elapsed at time t2, the imaging unit 21 or 41 starts capturing an image including the subject TR1, and the control unit 48 starts timing the distribution waiting time.

[0107] When the distribution waiting time T2 has elapsed at time t3, the control unit 48 starts distributing the image. In the example in Figure 9(D), an image including subject TR1 is distributed to the mobile terminal MT1 after time t3.

[0108] As described in detail above, according to the first embodiment, the monitoring system 100 includes a sensor 10 that acquires data on the state of the subject TR1, a master camera 40 and a slave camera 20 that start capturing images including the subject TR1 when the state of the subject TR1 is in a predetermined state (a state in which an abnormality has occurred in the subject TR1), and a control unit 48 and a control unit 28 that make it impossible to capture images of the subject TR1 with the master camera 40 and the slave camera 20 until the state of the subject TR1 is in a predetermined state. As a result, the subject TR1 is not captured under normal circumstances, thus protecting the privacy of the subject TR1.

[0109] Furthermore, in this first embodiment, the master camera 40 and the slave camera 20 have different appearances depending on whether they are in a state where they cannot image the subject TR1 or when they can (see Figures 4(A) to 4(D)). As a result, the subject TR1 can determine whether the master camera 40 and the slave camera 20 are taking images by checking their appearance.

[0110] Furthermore, in this first embodiment, when it is impossible to image the subject TR1, the lenses of the master camera 40 and the slave camera 20 are hidden from the subject TR1. For example, the control unit 48 keeps the lens 421 of the master camera 40 covered with the cover 422 until the subject TR1 reaches a predetermined state (until an abnormality occurs in the subject TR1). This suppresses the subject TR1 from having the feeling that they are being "monitored".

[0111] As mentioned above, the control units 48 and 28 may, until the state of the subject TR1 reaches a predetermined state, adjust the imaging direction of the master camera 40 and the slave camera 20 to a direction different from the direction in which the preset imaging range exists. Even in this way, it is possible to suppress the subject TR1 from having the feeling that they are being "monitored".

[0112] Furthermore, in this first embodiment, the sensor 10 acquires data other than visible light images as data related to the subject's condition. This makes it possible to determine whether or not there is an abnormality in the subject TR1 while protecting the subject TR1's privacy.

[0113] Furthermore, in this first embodiment, the control unit 48 causes the imaging unit 21 or 41 to start capturing an image including the subject TR1 when the state of the subject TR1, estimated from the data acquired by the sensor 10 that acquires data on the state of the subject TR1, reaches a predetermined state (a state in which an abnormality has occurred in the subject TR1) and an imaging waiting time (first predetermined time) has elapsed. Because there is a time lag between the determination that an abnormality has occurred in the subject TR1 and the start of image capture, if no abnormality has actually occurred in the subject TR1, measures can be taken to protect the privacy of the subject TR1. Therefore, the privacy of the subject TR1 can be protected more effectively compared to the case where imaging is started immediately after it is determined that an abnormality has occurred in the subject TR1.

[0114] Furthermore, in this first embodiment, if the control unit 48 receives an input to prohibit (cancel) imaging during the imaging waiting time, it does not perform the process of causing the imaging unit 21 or 41 to take an image. This makes it possible to prohibit the imaging unit 21 or 41 from capturing an image including the subject TR1, for example, when no abnormality is actually occurring in the subject TR1, thereby protecting the privacy of the subject TR1.

[0115] Furthermore, in this first embodiment, when the control unit 48 receives an input prohibiting imaging, it sends a predetermined message (for example, a message informing that the abnormality was a false positive or that no abnormality occurred) to the mobile terminal (external device) MT1. This allows the person OB1 (the person watching over the subject TR1) who possesses the mobile terminal MT1 to know that the subject TR1 is safe.

[0116] Furthermore, in this first embodiment, if it is determined that an abnormality has occurred in the subject TR1, the control unit 48 notifies the subject TR1 of the start of imaging. Specifically, in this first embodiment, the control unit 48 notifies the subject TR1 of the timing when imaging will start. This allows the subject TR1 to know when imaging will start, and if there is no abnormality in the subject TR1, it can take necessary measures to protect its privacy (for example, prohibit imaging, move outside the imaging range, etc.), thus protecting the privacy of the subject TR1 compared to when no notification regarding the start of imaging is given. In the above embodiment, the notification was made by the speaker 43 of the master camera 40 or the speaker 23 of the slave camera 20, but it is not limited to this. For example, the remote controller 30 may be equipped with a speaker, and the notification may be made via the speaker of the remote controller 30. Alternatively, the remote controller 30 may be equipped with a display unit, and the notification may be displayed on the display unit.

[0117] Furthermore, in this first embodiment, when the estimated state of subject TR1 reaches a predetermined state (a state in which subject TR1 is abnormal), the control unit 48 sends a notification regarding subject TR1 to the mobile terminal MT1. Specifically, it sends a message to the mobile terminal MT1 notifying that there is a possibility that subject TR1 is abnormal. As a result, person OB1, who possesses the mobile terminal MT1, can learn that subject TR1 is abnormal and take actions such as calling subject TR1 or visiting subject TR1.

[0118] Furthermore, in this first embodiment, if an imaging request is received from the mobile terminal MT1 during the imaging waiting time, the control unit 48 will cause the imaging unit 21 or imaging unit 41 to start imaging before the imaging waiting time elapses. This allows the image of the subject TR1 to be stored in response to a request from the caregiver OB1 who is holding the mobile terminal MT1. It should also be noted that the subject TR1 may be notified that imaging has started when the imaging unit 21 or imaging unit 41 is instructed to start imaging.

[0119] Furthermore, in this first embodiment, the control unit 48 distributes the image captured by the imaging unit 21 or 41 when a distribution waiting time (second predetermined time) has elapsed since the imaging unit 21 or 41 started capturing. Because there is a time lag between the start of capturing and the distribution of the image, if no abnormality occurs in the subject TR1, the subject TR1 can take necessary actions during the distribution waiting time. This protects the privacy of the subject TR1 compared to the case where the image distribution starts immediately after the imaging unit 21 or 41 starts capturing. Also, if an abnormality occurs in the subject TR1, the distribution stand-by After some time has passed, the image captured by the imaging unit 21 or 41 is distributed, allowing person OB1, who is holding the mobile terminal MT1, to check the condition of subject TR1 through the image.

[0120] Furthermore, in this first embodiment, if an input to prohibit (cancel) distribution is received within the distribution waiting time (second predetermined time), the control unit 48 stops timing the distribution waiting time. This prevents the distribution of images captured by the imaging unit 21 or 41, thereby preventing a situation in which the subject TR1's privacy is violated by the distribution of images when no abnormality is occurring in the subject TR1.

[0121] Furthermore, in this first embodiment, the control unit 48 notifies the target TR1 about the distribution of the image before the image distribution process. This allows the target TR1 to know that the image will be distributed and to take necessary actions to protect their privacy.

[0122] Furthermore, in this first embodiment, the control unit 48 notifies the subject TR1 of the start of image acquisition when the state of the subject TR1, estimated from the data acquired by the sensor 10 that acquires data on the state of the subject TR1, reaches a predetermined state (a state in which an abnormality has occurred in the subject TR1). As a result, the subject TR1 is aware that image acquisition is starting, and if, for example, the subject TR1 is not experiencing any abnormality, it can take the necessary actions to protect the subject TR1's privacy.

[0123] Furthermore, in this first embodiment, when the subject TR1 reaches a predetermined state and the imaging waiting time (first predetermined time) has elapsed, the control unit 48 causes the imaging unit 41 or imaging unit 21 to start capturing an image, and notifies the subject TR1 of the timing to start capturing an image during the imaging waiting time. This allows, for example, if no abnormality occurs in the subject TR1, the subject TR1 to take any necessary actions to protect their privacy before the timing when capturing is started.

[0124] As mentioned above, the control unit 48 may also notify that the current time is within the imaging waiting time or that imaging by the imaging unit 21 or 41 has not yet started. Such notification allows, for example, subject TR1 to take necessary actions to protect their privacy if no abnormality has occurred.

[0125] Furthermore, the control unit 48 may notify the user of the time remaining until imaging begins. Such notification allows the user TR1 to take necessary actions to protect their privacy, for example, if no abnormality occurs in the user TR1.

[0126] Furthermore, the control unit 48 may notify that the current time is within the period in which imaging can be prohibited. This allows, for example, subject TR1 to take the necessary actions to prohibit imaging if no abnormality has occurred in subject TR1.

[0127] Furthermore, in this first embodiment, the control unit 48 starts distributing the images captured by the imaging unit 21 or 41 after the imaging unit 21 or 41 starts capturing images, and provides notification regarding image distribution before starting image distribution. This allows the subject TR1 to know that image distribution is about to begin, and for example, if no abnormality occurs to the subject TR1, they can take the necessary actions to protect the subject TR1's privacy.

[0128] Furthermore, in this first embodiment, the control unit 48 starts distributing the captured image after the distribution waiting time has elapsed since the imaging unit 21 or 41 started capturing an image including the subject TR1, and notifies the subject of the timing to start distribution during the distribution waiting time. This allows the subject TR1 to take any necessary actions to protect their privacy before the image distribution starts, for example, if no abnormality occurs in the subject TR1.

[0129] The control unit 48 may also notify the user of the time remaining until distribution begins. Such notification allows the user TR1 to take necessary actions to protect their privacy, for example, if no abnormality has occurred.

[0130] Furthermore, the control unit 48 may notify the subject TR1 that the current time is within the period when distribution can be prohibited (cancelled). This allows the subject TR1 to know that distribution can be prohibited and to take the necessary actions to protect their privacy.

[0131] Furthermore, in this first embodiment, if the control unit 48 receives an input prohibiting imaging during the imaging waiting time, it does not perform the process to start image acquisition. Receiving an input prohibiting imaging means that the subject TR1 is in a state where they can give an input prohibiting imaging, that is, a state in which no abnormality has occurred. Therefore, by not performing the process to start image acquisition when an input prohibiting imaging is received during the imaging waiting time, the privacy of subject TR1, who is highly likely to be in a state in which no abnormality has occurred, can be protected.

[0132] Furthermore, in this first embodiment, if the control unit 48 receives an input prohibiting distribution during the distribution waiting time, it does not perform the process to start image distribution. Receiving an input prohibiting distribution indicates that the subject TR1 is in a state where they can give an input prohibiting distribution, that is, a state in which no abnormality has occurred. Therefore, by not performing the process to start image distribution when an input prohibiting distribution is received, the privacy of subject TR1, who is highly likely to be in a state in which no abnormality has occurred, can be protected.

[0133] 《Second Embodiment》 In the first embodiment, it was determined whether the state of subject TR1 estimated based on data acquired from sensor 10 was in a predetermined state (a state in which subject TR1 is experiencing an abnormality). In the first embodiment, if this determination was incorrect, subject TR1 would not notice the notification regarding imaging or distribution, and would not make either an input to prohibit imaging or an input to prohibit distribution, in which case imaging and distribution would occur, and subject TR1's privacy would not be protected.

[0134] Therefore, in the second embodiment, the determination of whether the judgment that "an abnormality has occurred in the subject" based on the data acquired from the sensor 10 is correct is made based on the image captured by the imaging unit 21 or 41. Specifically, the state of the subject TR1 is estimated based on the image captured by the imaging unit 21 or 41, and if the estimated state of the subject TR1 is a predetermined state (a state in which an abnormality has occurred in the subject TR1), the judgment of abnormality based on the data acquired from the sensor 10 is considered correct, and the timing of the delivery waiting time is started. Generally, the amount of information obtained from the image captured by the imaging unit 21 or 41 is greater than the amount of information obtained from the data acquired by the sensor 10, so the state of the subject TR1 estimated from the image captured by the imaging unit 21 or 41 is considered to have higher estimation accuracy than the state of the subject TR1 estimated from the data acquired by the sensor 10.

[0135] In the second embodiment, the processing performed by the control unit 28 and the control unit 48 differs from that in the first embodiment. The configuration of the monitoring system 100, the master camera 40, and the slave camera 20 are the same as in the first embodiment, so a detailed explanation is omitted.

[0136] Figure 10 is a flowchart showing an example of processing performed by the control unit 48 of the master camera 40 in the monitoring system 100 according to the second embodiment. The processing in Figure 10 differs from the processing in Figure 5 in that the processing performed when the camera identified in step S14 is the master camera 40 (step S15 / YES) (step S50: third processing) and the processing performed when the camera identified in step S14 is the slave camera 20 (step S15 / NO) (step S60: fourth processing).

[0137] Figures 11 and 12 are flowcharts detailing the third process. In Figure 11, steps S201 to S213 and steps S225 and S227 are the same as those in the first process shown in Figure 6, so they are denoted by the same reference numerals and detailed explanations are omitted.

[0138] In the third process, when imaging by the imaging unit 41 begins (step S213), the control unit 48 estimates the state of the subject TR1 based on the image (visible light image) captured by the imaging unit 41 (step S501). Next, the control unit 48 determines whether the estimated state of the subject TR1 is a predetermined state (a state in which an abnormality has occurred in the subject TR1) (step S503). For example, the control unit 48 performs pattern matching between the image captured by the imaging unit 41 and an image used in advance to determine abnormalities, and determines whether an abnormality has occurred in the subject TR1.

[0139] If no abnormality occurs in subject TR1 (step S503 / NO), the control unit 48 sends a predetermined message to the mobile terminal MT1 (step S507). For example, the control unit 48 sends a message notifying that the judgment in step S13 of Figure 10 was incorrect (misjudgment / false detection), or that there is no abnormality in subject TR1.

[0140] Next, the control unit 48 stops imaging by the imaging unit 41 (step S509) and drives the drive device 49 to move the cover 422 so that it covers the lens 421 (step S511). Note that the order of steps S507 to S511 may be changed. After step S511, the process returns to step S11 in Figure 10.

[0141] If it is determined that an abnormality has occurred in subject TR1 (step S503 / YES), the control unit 48 stores the image used to estimate the state of subject TR1 in the storage unit 44 (step S505). This allows person OB1, who is monitoring subject TR1, to later review and verify the image used to determine whether or not an abnormality has occurred in subject TR1. stand-by After a certain period of time has elapsed, the image used to determine whether or not an abnormality has occurred in subject TR1 may be delivered to the mobile terminal MT1.

[0142] The process from step S215 onwards in Figure 12 is the same as the process from step S215 onwards in the first process shown in Figure 6; therefore, the same reference numerals are used, and a detailed explanation is omitted.

[0143] Next, the details of the fourth process will be explained. Figure 13 is a flowchart illustrating the details of the fourth process. Figure 14 is a flowchart illustrating an example of the process performed by the control unit 28 of the sub-camera 20 in the monitoring system 100 according to the second embodiment. The fourth process will be explained in conjunction with the process performed by the control unit 28 of the sub-camera 20.

[0144] In the fourth process shown in Figure 13, steps S301 to S313 and steps S325 and S327 are the same processes as in the second process shown in Figure 7, so they are denoted by the same reference numerals and detailed explanations are omitted.

[0145] Furthermore, in the processing performed by the control unit 28 of the slave camera 20 shown in Figure 14, steps S401 to S407 and steps S421 and S423 are the same as the processing shown in Figure 8, so the same reference numerals are used and detailed explanations are omitted.

[0146] In Figure 14, the control unit 28 of the sub-camera 20 instructs the imaging unit 21 to start capturing an image including the subject TR1 (step S407), and then estimates the state of the subject TR1 based on the image (visible light image) captured by the imaging unit 21 (step S451). Next, the control unit 28 determines whether the estimated state of the subject TR1 is a predetermined state (a state in which an abnormality has occurred in the subject TR1) (step S453).

[0147] If no abnormality occurs in subject TR1 (step S453 / NO), the control unit 28 stops capturing images with the imaging unit 21 (step S461) and covers the lens of the sub-camera 20 with a cover (step S463). Next, the control unit 28 transmits the estimated result of the subject TR1's state to the master camera 40 (step S465), and terminates the process shown in Figure 14. In step S465, the estimated result that no abnormality occurs in subject TR1 is transmitted to the master camera 40.

[0148] On the other hand, if an abnormality occurs in subject TR1 (step S453 / YES), the control unit 28 stores the image used to estimate the state of subject TR1 in the storage unit 24 (step S455). This allows person OB1, who is monitoring subject TR1, to later review and verify the image used to determine whether or not an abnormality has occurred in subject TR1. stand-by After a certain period of time has elapsed, the image used to determine whether or not an abnormality has occurred in subject TR1 may be delivered to the mobile terminal MT1.

[0149] Next, the control unit 28 transmits the estimated status of subject TR1 to the master camera 40 (step S457) and waits until it receives a notification instruction regarding distribution (step S409). In step S457, the estimated result that an abnormality has occurred in subject TR1 is transmitted to the master camera 40.

[0150] Since the process from step S409 onward is the same as the process shown in Figure 8, the same reference numerals are used, and a detailed explanation is omitted.

[0151] Meanwhile, the control unit 48 of the master camera 40 waits after transmitting an imaging instruction to the slave camera 20 (Figure 13: step S313) until it receives the estimated result of the subject TR1's state based on the visible light image from the slave camera 20 (step S601 / NO).

[0152] Upon receiving the estimation result (step S601 / YES), the control unit 48 determines whether the estimation result indicates that an abnormality has occurred in subject TR1 (step S602).

[0153] If no abnormality occurs in subject TR1 (step S602 / NO), the control unit 48 sends a predetermined message to the mobile terminal MT1 (step S603). For example, the control unit 48 sends a message notifying that the judgment in step S13 of Figure 10 was incorrect (misjudgment / false detection), or that there is no abnormality in subject TR1.

[0154] If an abnormality occurs in the target user TR1 (step S602 / YES), the control unit 48 starts timing the delivery waiting time (step S315). The processing from step S315 onward is the same as the second processing shown in Figure 7, so the same reference numerals are used and detailed explanations are omitted.

[0155] Figures 15(A) and 15(B) are time charts showing an example of the processing performed in the monitoring system 100 according to the second embodiment.

[0156] First, let's explain the example in Figure 15(A). In Figure 15(A), let's assume that at time t1, the control unit 48 determined that an abnormality had occurred in subject TR1 based on the data acquired from sensor 10.

[0157] In this case, the control unit 48 starts timing the image acquisition waiting time from time t1. When the image acquisition waiting time T1 has elapsed at time t2, the imaging unit 21 or 41 starts acquiring an image including the subject TR1.

[0158] At time t3, if the control unit 28 or 48 determines, based on the captured image, that no abnormality has occurred in the subject TR1, it stops the imaging by the imaging unit 21 or 41. This prevents the imaging and distribution of images from continuing even though no abnormality has occurred in the subject TR1.

[0159] Next, let's explain the example in Figure 15(B). In Figure 15(B), let's assume that at time t1, the control unit 48 determined that an abnormality had occurred in subject TR1 based on the data acquired from sensor 10.

[0160] In this case, the control unit 48 starts timing the image acquisition waiting time from time t1. When the image acquisition waiting time T1 has elapsed at time t2, the imaging unit 21 or 41 starts acquiring an image including the subject TR1.

[0161] At time t3, if the control unit 28 or 48 determines that an abnormality has occurred in subject TR1 based on the captured image, the control unit 48 starts timing the delivery waiting time.

[0162] When the distribution waiting time T2 has elapsed at time t4, the control unit 48 starts distributing the image. This allows for image distribution when there is a high probability that an abnormality has occurred in the subject TR1.

[0163] As described in detail above, according to this second embodiment, when the state of subject TR1 estimated from the data acquired by the sensor 10, which acquires data on the subject's state, is a predetermined state (a state in which an abnormality has occurred in subject TR1), the control unit 48 causes the imaging unit 21 or 41 to start capturing an image including subject TR1, and estimates (detects) the state of subject TR1 based on the image captured by the imaging unit 21 or 41. This makes it possible to confirm whether the state of subject TR1 estimated from the data acquired by the sensor 10 is correct or not.

[0164] Furthermore, in this second embodiment, the control unit 48 continues to capture images when the state of the subject TR1 estimated (detected) based on the image captured by the imaging unit 21 or 41 is in a predetermined state (a state in which an abnormality has occurred in the subject TR1). This makes it possible to record an image of the subject TR1 in which an abnormality has occurred.

[0165] Furthermore, if image acquisition continues, the control unit 28 or 48 may notify the subject TR1 of information regarding image acquisition. For example, the control unit 28 or 48 may notify the subject TR1 of the timing when recording of the acquired image to the storage unit 24 or 44 begins. This allows the subject TR1 to know that the acquired image is being recorded.

[0166] Furthermore, the control unit 28 or 48 may notify the subject TR1 that image acquisition is taking place. This allows the subject TR1 to know that an image including themselves is being acquired.

[0167] Furthermore, in this second embodiment, the control unit 48 starts distributing the image captured by the imaging unit 21 or 41 when the state of the subject TR1 estimated (detected) based on the image is in a predetermined state (a state in which an abnormality has occurred in the subject TR1). This makes it possible to start distributing the image when there is a high probability that an abnormality has occurred in the subject TR1.

[0168] Furthermore, in this second embodiment, the control unit 28 or 48 stores the image used to estimate the state of subject TR1 when the state of subject TR1 estimated based on the image is a predetermined state (a state in which an abnormality has occurred in subject TR1). This allows for later confirmation and verification of the image in which it was determined that an abnormality has occurred in subject TR1.

[0169] Furthermore, in this second embodiment, the control unit 48 detects that the state of subject TR1 is in a predetermined state (a state in which subject TR1 is abnormal) based on the image, and after the distribution waiting time has elapsed, it starts distributing an image including subject TR1. Since there is a time lag between the determination that subject TR1 is abnormal based on the image and the distribution of the image, if subject TR1 is not abnormal, subject TR1 can take necessary actions to protect its privacy during the distribution waiting time.

[0170] Furthermore, in this second embodiment, the control unit 48 notifies the target TR1 of information regarding image distribution before the image distribution process begins. This allows the target TR1 to take necessary actions to protect their privacy before the image distribution process starts.

[0171] Furthermore, in this second embodiment, the control unit 28 or 48 stops capturing images if the state of the subject TR1 detected based on the image is not in a predetermined state (a state in which an abnormality has occurred in the subject TR1). That is, if it is determined based on the image that no abnormality has occurred in the subject TR1, the control unit 28 or 48 stops capturing images. This prevents a situation in which the capture of images including the subject TR1 continues even though no abnormality has occurred in the subject TR1, and the privacy of the subject TR1 is not protected.

[0172] Furthermore, in this second embodiment, if the state of the subject TR1 detected based on the image is not in a predetermined state (i.e., no abnormality has occurred in the subject TR1), the control unit 48 sends a predetermined message (for example, a message indicating that the abnormality was a false positive) to the mobile terminal MT1. This allows the person holding the mobile terminal MT1 (the person watching over the subject TR1) to know that the subject TR1 is safe.

[0173] In the second embodiment described above, when the imaging unit 21 or 41 starts imaging, the image captured by the imaging unit 21 or 41 may be stored in the storage unit 24 or 44. In this case, the control units 28 and 48 may delete the image stored in the storage unit 24 or 44 (including the image used to detect the state of the subject TR1) if the state of the subject TR1 detected based on the image is not in a predetermined state (a state in which an abnormality has occurred in the subject TR1). This protects the privacy of the subject TR1.

[0174] In addition, in the first and second embodiments described above, either the imaging waiting time or the distribution waiting time may be omitted. In this case, the subject TR1 should be able to input either an input to prohibit imaging or an input to prohibit distribution.

[0175] In addition, in the first and second embodiments described above, there may be multiple mobile terminals used to deliver messages and images.

[0176] 《Third Embodiment》 In the first and second embodiments, an example of distributing images to a single mobile terminal MT1 was described. However, if there are multiple people monitoring the subject TR1, the timing of image distribution may be changed according to the type (attribute) of the person monitoring.

[0177] The monitoring system 100 according to the third embodiment differs from the first and second embodiments in that the processing performed by the control unit 48 and the control unit 28 is different. Also, it differs from the first and second embodiments in that the storage unit 44 of the master camera 40 stores a list of broadcasters.

[0178] Figure 16 shows an example of a distributor list. The distributor list includes fields for ID, Name, Communication Information, and Group. The ID field stores an identifier to uniquely identify the person watching over Subject TR1. The Name field stores the name of the person watching over Subject TR1. The Communication Information field stores information necessary for directly distributing images from the main camera 40 to the mobile terminal (e.g., IP information of the mobile terminal). The Group field is used to classify the person identified by the ID, and in this embodiment, one of Groups 1 to 3 is stored. Here, Group 1 represents, for example, the family of Subject TR1; Group 2 represents, for example, the relatives of Subject TR1; and Group 3 represents, for example, the caregiver or care manager of Subject TR1.

[0179] In this third embodiment, the control unit 48 changes the image distribution timing and content according to the group. In this embodiment, images are distributed to groups 1 and 2, but not to group 3.

[0180] Figure 17 is a flowchart showing an example of processing performed by the control unit 48 of the master camera 40 in the monitoring system 100 according to the third embodiment. The processing in Figure 17 differs from the processing in Figures 5 and 10 in that the processing performed when the camera identified in step S14 is the master camera 40 (step S15 / YES) (step S70: fifth processing) and the processing performed when the camera identified in step S14 is the slave camera 20 (step S15 / NO) (step S80: sixth processing).

[0181] Figure 18 is a flowchart detailing the fifth process. In Figure 18, steps S201 to S213 and steps S225 and S227 are the same as those in the first process shown in Figure 6, so they are denoted by the same reference numerals and detailed explanations are omitted.

[0182] In the process shown in Figure 18, when image acquisition including subject TR1 begins (step S213), the control unit 48 starts distributing the image acquired by the imaging unit 41 to the mobile terminals of individuals belonging to group 1 (for example, the person with ID "OB1" in Figure 16) (step S701). This allows individuals belonging to group 1 to quickly confirm the status of subject TR1 through the image.

[0183] Next, the control unit 48 starts timing the delivery waiting time (step S703).

[0184] The control unit 48 notifies the subject TR1 of the timing when the image captured by the imaging unit 41 will be distributed to a person belonging to a group other than Group 1, and to a group for which the image will be distributed (in Figure 16, the person with ID "OB2") (step S705). For example, the control unit 48 causes the speaker 43 to output an audio message such as, "In 5 seconds, the distribution of the image to the person belonging to Group 2 will begin." This allows the subject TR1 to know that the distribution of the image captured by the imaging unit 41 to the person belonging to Group 2 will begin.

[0185] Next, the control unit 48 determines whether or not it has received an input to prohibit (cancel) the distribution of the image (step S707).

[0186] If the control unit 48 receives input to prohibit the distribution of images (step S707 / YES), it stops timing the distribution waiting time and stops imaging by the imaging unit 41 (step S713). This prevents the images captured by the imaging unit 41 from being distributed to people belonging to groups other than group 1. Since imaging by the imaging unit 41 is stopped, the distribution of images to people belonging to group 1 is also stopped. At this time, the control unit 48 may send a message to people belonging to each group informing them that the judgment in step S13 was incorrect, or that there is no abnormality in the subject TR1.

[0187] Subsequently, the control unit 48 covers the lens 421 of the master camera 40 with the cover 422 (step S715), and returns to step S11 in Figure 17.

[0188] If no input prohibiting image distribution is received (step S707 / NO), the control unit 48 determines whether the distribution waiting time has elapsed (step S709). Specifically, it determines whether the distribution waiting time has elapsed since the start of timing the distribution waiting time in step S703. If the distribution waiting time has not elapsed (step S709 / NO), the process returns to step S707.

[0189] If the distribution waiting time has elapsed (step S709 / YES), the control unit 48 starts distributing images to the mobile terminals of the people belonging to group 2 (second group) and notifies the mobile terminals of the people belonging to group 3 that, for example, distribution of images to group 2 has started (step S711). This allows the people belonging to group 2 to check the status of subject TR1 through images. Furthermore, the fact that distribution of images to group 2 has started indicates a high probability that something is wrong with subject TR1. The people belonging to group 3 can learn from the notification that there is a high probability that something is wrong with subject TR1 and can take action, for example, by visiting subject TR1.

[0190] Next, the details of the sixth process will be explained. Figure 19 is a flowchart showing the details of the sixth process. Figure 20 is a flowchart showing an example of the process performed by the control unit 28 of the slave camera 20 in the monitoring system 100 according to the third embodiment. The sixth process will be explained together with the process performed by the control unit 28 of the slave camera 20.

[0191] In Figure 19, steps S301 to S313 and steps S325 and S327 are the same as the second process in Figure 7, so they are denoted by the same reference numerals and detailed explanations are omitted.

[0192] Furthermore, in Figure 20, processes other than steps S471 and S473 are the same as those shown in Figure 8, so they are denoted by the same reference numerals and detailed explanations are omitted.

[0193] In Figure 19, when the control unit 48 of the master camera 40 transmits an imaging instruction to the slave camera 20 (step S313), it starts distributing the image captured by the imaging unit 21 of the slave camera 20 to the mobile terminal of a person belonging to Group 1 (for example, the person with ID "OB1" in Figure 16) (step S801). This allows the person belonging to Group 1 to quickly confirm the status of subject TR1 through the image.

[0194] Next, the control unit 48 starts timing the delivery waiting time (step S803).

[0195] The control unit 48 instructs the slave camera 20 to notify the person (in Figure 16, the person with ID "OB2") of the timing when the image captured by the imaging unit 21 will be distributed to a person belonging to a group other than group 1 and to which the image will be distributed (step S805).

[0196] Meanwhile, the control unit 28 of the sub-camera 20, after instructing the imaging unit 21 to start capturing an image including the subject TR1 (Figure 20: step S407), waits until it receives an instruction from the master camera 40 to notify it of the timing to start image distribution (step S471 / NO).

[0197] Then, upon receiving an instruction from the master camera 40 to notify the timing of the start of image distribution (step S471 / YES), the imaging unit 21 notifies the person (in Figure 16, the person with ID "OB2") of the timing when the image captured by the imaging unit 21 will be distributed to a person belonging to a group other than group 1 (step S473). For example, the control unit 28 causes the speaker 23 to output an audio message such as, "Image distribution to people belonging to group 2 will start in 5 seconds." As a result, the subject TR1 knows that the distribution of the image captured by the imaging unit 21 will start to people belonging to group 2.

[0198] The subsequent processing is the same as that shown in Figure 8, so a detailed explanation will be omitted.

[0199] Meanwhile, the control unit 48 of the master camera 40 determines whether or not it has received a transmission prohibition instruction from the slave camera 20 (Figure 19: Step S807).

[0200] If a distribution prohibition instruction is received (step S807 / YES), the control unit 48 stops timing the distribution waiting time (step S813). This prevents the images captured by the imaging unit 21 from being distributed to people belonging to groups other than group 1. Since imaging by the imaging unit 21 is stopped, the distribution of images to people belonging to group 1 is also stopped. At this time, the control unit 48 may send a message to people belonging to each group informing them that the judgment in step S13 of Figure 17 was incorrect, or that there is no abnormality in the subject TR1. After processing in step S813, the process returns to step S11 of Figure 17.

[0201] If no instruction to prohibit distribution has been received (step S807 / NO), the control unit 48 determines whether the distribution waiting time has elapsed (step S809). Specifically, it determines whether the distribution waiting time has elapsed since the start of timing the distribution waiting time in step S803. If the distribution waiting time has not elapsed (step S809 / NO), the process returns to step S807.

[0202] If the distribution waiting time has elapsed (step S809 / YES), the control unit 48 sends a command to the slave camera 20 to continue shooting (step S815), starts distributing images to the mobile terminals of people belonging to group 2 (second group), and notifies the mobile terminals of people belonging to group 3 that, for example, image distribution to group 2 has started (step S817). This allows people belonging to group 2 to check the status of subject TR1 through images. Furthermore, the start of image distribution to group 2 indicates a high probability that something is wrong with subject TR1. People belonging to group 3 can learn from the notification that there is a high probability that something is wrong with subject TR1 and can take action, for example, by visiting subject TR1.

[0203] Figure 21 is a time chart showing an example of processing in the monitoring system 100 according to the third embodiment. In Figure 21, it is assumed that at time t1, the control unit 48 determines that an abnormality has occurred in the subject TR1 based on the data acquired from the sensor 10.

[0204] In this case, the control unit 48 starts timing the image acquisition waiting time from time t1. When the image acquisition waiting time T1 has elapsed at time t2, the imaging unit 21 or 41 starts capturing an image including the subject TR1. The control unit 48 also starts distributing the image to the mobile terminals of the people belonging to group 1 and starts timing the distribution waiting time.

[0205] When the distribution waiting time T2 has elapsed at time t3, the control unit 48 starts distributing images to the mobile terminals of the people belonging to group 2 and sends a predetermined message to the mobile terminals of the people belonging to group 3. In this way, when there are multiple people watching over the subject TR1, the control unit 48 starts distributing images to the mobile terminals of the people belonging to group 2. People who watch over By changing the timing of image delivery and notification content according to the attributes of the subject, it is possible to achieve both monitoring of the subject TR1 and ensuring the privacy of the subject TR1.

[0206] As described in detail above, according to the third embodiment, when the imaging unit 21 or 41 starts imaging, the control unit 48 starts distributing the image captured by the imaging unit 21 or 41 to a person belonging to group 1, and when the distribution waiting time has elapsed since the imaging unit 21 or 41 started imaging, it starts distributing the image captured by the imaging unit 21 or 41 to a person belonging to group 2. This allows, for example, close relatives of subject TR1 to know the status of subject TR1 through images, and since the distribution of images to people other than close relatives is performed after the distribution waiting time has elapsed, the privacy of subject TR1 can be protected.

[0207] (Hardware configuration) Figure 22(A) shows the hardware configuration of the control unit 48. As shown in Figure 22(A), the control unit 48 includes a CPU 431, ROM 432, RAM 434, storage unit 436, network interface 437, etc. Each of these components of the control unit 48 is connected to the bus 438. The functions of the control unit 48 are realized when the CPU 431 executes a program stored in the ROM 432 or storage unit 436.

[0208] Figure 22(B) shows the hardware configuration of the control unit 28. The control unit 28 includes a CPU 231, ROM 232, RAM 234, storage unit 236, network interface 237, etc. Each of these components of the control unit 28 is connected to the bus 238. The functions of the control unit 28 are realized when the CPU 231 executes a program stored in the ROM 232 or storage unit 236.

[0209] In the first to third embodiments described above, an example was given in which the home system 150 comprises one master camera 40 and one or more slave cameras 20, but it is not limited to this. Figure 23 is a diagram showing the configuration of a modified monitoring system 100A. As shown in Figure 23, in the home system 150A, the master camera 40 is omitted, and the home system 150A comprises a sensor 10, a camera 80, a remote controller 30, and a control device 70 that controls the entire home system 150A. The camera 80 may have the same configuration as the slave camera 20 described above.

[0210] Figure 24 is a functional block diagram showing the configuration of the control device 70 in a modified example. The control device 70 comprises a storage unit 74, a first communication module 75, a second communication module 76, a third communication module 77, and a control unit 78. The first communication module 75, the second communication module 76, and the third communication module 77 are the same as the first communication module 45, the second communication module 46, and the third communication module 47, respectively, so a detailed explanation is omitted.

[0211] The storage unit 74 stores server address information necessary for communication with the outside of the home system 150A (e.g., service server SS), camera 80 identification information, past (e.g., the last month) operating status of the home system 150A, and past (e.g., the last month) detection results of the home system 150A's sensors 10 and camera 80 regarding the state of the subject TR1. The operating status of the home system 150A includes events that occurred in the home system 150A, such as system startup, system shutdown, and error occurrence.

[0212] The control unit 78 performs almost the same processing as the control unit 48, but in this modified example, the processing in steps S15 and S20 in Figure 5 is unnecessary. Also, the processing in Figure 6 is unnecessary. Since the hardware configuration of the control unit 78 is the same as that of the control unit 48, a detailed explanation is omitted.

[0213] In addition, the home system 150 may have multiple master cameras 40 and multiple slave cameras 20. Alternatively, the home system 150 may omit the slave cameras 20. In this case, there may be one or more master cameras 40. Furthermore, the control units for the master camera 40 and the slave cameras 20 may be provided separately from each camera. Alternatively, the control units for the master camera 40 and the slave cameras 20 may be shared. That is, one control unit may control both the master camera 40 and the slave cameras 20. Also, when the slave cameras 20 are omitted, one or more master cameras 40 may be controlled by one control unit.

[0214] In addition, in the first to third embodiments described above, the sensor 10 does not necessarily have to be included in the monitoring system 100 (home-side system 150). In this case, for example, an existing sensor installed in the residence of the subject TR1 can be connected to the monitoring system 100 to obtain data on the subject's condition from the existing sensor.

[0215] Furthermore, in the first to third embodiments described above, the sensor 10 and the master camera 40 and the slave camera 20 were separate units, but the sensor 10 and the master camera 40 may be integrated, or the sensor 10 and the slave camera 20 may be integrated.

[0216] Furthermore, in the first to third embodiments described above, the sub-camera 20 and the master camera 40 are equipped with filters that do not transmit visible light, and the control units 28 and 48 may control the filters to cover the lenses of the sub-camera 20 and the master camera 40 when it is not possible to image the subject TR1, and to expose the lenses when it is possible to image the subject TR1. This makes it possible to use the sub-camera 20 and the master camera 40 as sensors 10.

[0217] Furthermore, in the first to third embodiments described above, the sensor 10 and the sub-camera 20 do not necessarily have to be included in the monitoring system 100 (home-side system 150). In this case, for example, existing sensors and cameras installed in the residence of the subject TR1 can be connected to the home-side system 150 to acquire data on the subject's condition from the existing sensors and visible light images from the existing cameras. In this case, the home-side system 150 may include a cover that covers the lens of the existing camera, and the control unit 48 may set the existing camera to a state where it is physically impossible to image the subject TR1 (a state where the cover covers the lens of the existing camera) until the state of the subject TR1 estimated from the data acquired by the existing sensors reaches a predetermined state.

[0218] Furthermore, in the first to third embodiments described above, one of the sub-camera 20 and the master camera 40 may be a camera that cannot be made to a state in which it is impossible to image the subject TR1 under normal circumstances (for example, one that does not have a cover 422).

[0219] Furthermore, in the first to third embodiments described above, the master camera 40 does not necessarily have to be equipped with an imaging unit 21, a microphone 22, and a speaker 23. In this case, the master camera 40 functions as a control device that controls the entire home-side system 150.

[0220] Furthermore, while the first to third embodiments described above estimated the state of subject TR1 based on data acquired by the infrared array sensor, the system is not limited to this. For example, if sensor 10 is an infrared camera, a vector representing the movement of subject TR1 may be obtained by comparing frame images from the infrared camera, and the state of subject TR1 may be estimated based on the features of that vector. Alternatively, a thermograph may be used as sensor 10 instead of an infrared camera, or in addition to an infrared camera. For example, the state of subject TR1 may be determined by a combination of a vector representing the movement of subject TR1 detected by the infrared camera and changes in body temperature. Also, for example, if sensor 10 is a radio wave sensor, the measurable "heart rate," "respiratory rate," and "blood pressure" data may be compared with normal data (data measured in the past), and if they are similar, it may be determined that the state of subject TR1 is normal; if the difference from the normal data is above a threshold, it may be determined that an abnormality has occurred. Furthermore, if sensor 10 is a depth sensor, it may detect the posture of the subject TR1 and determine that the subject TR1 is normal if normal activities (standing, walking, sitting, sleeping) are detected, and determine that an abnormality has occurred if abnormal activities (falling, prolonged immobility) are detected. Also, if sensor 10 is a vibration sensor, it may determine that an abnormality has occurred if vibrations exceeding a threshold are detected. Also, if sensor 10 is a sound sensor, it may determine that an abnormality has occurred if impact sounds exceeding a threshold are detected. Also, if sensor 10 is a wearable sensor, it may determine that the state of the subject TR1 is normal if the measurable "heart rate," "respiratory rate," and "blood pressure" data are similar to normal data (data measured in the past), and determine that an abnormality has occurred if the difference from the normal data is above a threshold. Also, for example, a line sensor may be used as sensor 10.

[0221] Furthermore, in the first to third embodiments, different types of sensors 10 may be combined to estimate the state of the subject TR1. For example, a vibration sensor and a sound sensor may be combined to estimate the state of the subject TR1.

[0222] Furthermore, in the first to third embodiments described above, the predetermined state was explained as a state in which an abnormality has occurred in the subject TR1, but this is not the only possible state. For example, if the sensor 10 is a thermometer and a hygrometer, heatstroke is likely to occur if the measured temperature is above a predetermined value and the measured humidity is above a predetermined value. In this case, instead of determining whether or not the subject TR1 has heatstroke (whether or not an abnormality has occurred), the predetermined state may be a state in which there is a possibility that the subject TR1 has heatstroke (a state in which it is presumed that an abnormality has occurred in the subject TR1), and the timing of the imaging waiting time may be started.

[0223] Furthermore, in the first to third embodiments described above, input to prohibit imaging or broadcasting was received via the microphone 42 provided by the master camera 40 or the microphone 22 provided by the slave camera 20, but this is not limited to this. For example, the remote controller 30 may be equipped with a microphone and receive input to prohibit imaging via that microphone.

[0224] Furthermore, in the first embodiment described above, the input for prohibiting (canceling) imaging or transmission is not limited to a predetermined voice, but may also be, for example, a predetermined gesture or an operation on a predetermined device (remote controller 30, smartphone, etc.). If the input for prohibiting (canceling) imaging is a predetermined gesture, the control unit 48, if it determines that an abnormality has occurred in the subject TR1, should set the master camera 40 or the slave camera 20 to a state where it can image the subject TR1.

[0225] Furthermore, in the first to third embodiments described above, when image distribution has started, for example, if a predetermined operation is performed by the remote controller 30, image capture and image distribution may be stopped. For example, if a person TR1 who has fallen recovers from their fallen state, the person TR1 can instruct the master camera 40 to stop image capture and image distribution using the remote controller 30. Also, when a person OB1 who is watching over the person TR1 visits the person TR1's residence H1 after receiving a message or checking the distributed images, the person OB1 can instruct the master camera 40 to stop image capture and image distribution using the remote controller 30. Alternatively, the person OB1 may send an instruction to stop image distribution from their mobile terminal MT1.

[0226] Furthermore, in the first to third embodiments described above, while the image is being distributed, a notification that "distribution is in progress" may be continuously sent from a speaker or the like.

[0227] Furthermore, in the first to third embodiments described above, when the subject TR1 is asleep, the control unit 48 and the control unit 28 may refrain from performing the above processing. For example, if the subject TR1 is asleep in bed at a predetermined time, the control unit 48 and the control unit 28 may refrain from performing the above processing. Whether or not the subject TR1 is asleep in bed can be detected, for example, by a weight sensor installed in the bed. This prevents the system from misjudging the subject TR1's sleep as abnormal.

[0228] Furthermore, in the first to third embodiments described above, if a predetermined voice message such as "Help" is input during the imaging waiting time, imaging may be started without waiting for the imaging waiting time to elapse, and image distribution may also be started.

[0229] Furthermore, in the first to third embodiments described above, the control unit 48 may perform some or all of the processing performed by the control unit 28. Also, the second and third embodiments may be combined as appropriate.

[0230] The above processing functions can be implemented by a computer. In this case, a program describing the processing content of the functions that the processing unit (CPU) should have is provided. By executing this program on the computer, the above processing functions are implemented on the computer. The program describing the processing content can be recorded on a computer-readable recording medium (except for carrier waves).

[0231] When distributing a program, it may be sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. Alternatively, the program can be stored in the storage device of a server computer and transferred from the server computer to other computers via a network.

[0232] A computer executing a program stores the program, for example, on a portable storage medium or transferred from a server computer, in its own memory. The computer then reads the program from its memory and executes the processing according to the program. Alternatively, the computer can directly read the program from the portable storage medium and execute the processing according to that program. Furthermore, the computer can sequentially execute the processing according to the program received each time it is transferred from a server computer.

[0233] The embodiments described above are preferred examples of the present invention and can be combined as appropriate. However, the invention is not limited thereto, and various modifications are possible without departing from the spirit of the invention. [Explanation of symbols]

[0234] 10 sensors 20 Sub-camera 21 Imaging Department 28 Control Unit 30 Remote Controllers 40 Main camera 41 Imaging Unit 48 Control Unit 100 Monitoring System 150 Home-side system MT1 Mobile Terminal

Claims

1. An acquisition unit that acquires data regarding the subject's condition, When the subject's condition is in a predetermined state, the imaging unit starts capturing an image including the subject within a predetermined imaging range, A control unit that sets the imaging direction of the imaging unit to a direction different from the direction in which the predetermined imaging range exists until the state of the subject reaches the predetermined state, A monitoring system equipped with these features.

2. The imaging unit shall capture an image that does not include the subject if the imaging direction of the imaging unit is different from the direction in which the predetermined imaging range exists. The monitoring system according to claim 1.

3. The control unit shall, until the subject's state reaches the predetermined state, set the imaging direction of the imaging unit so that the subject cannot see the lens of the imaging unit. The monitoring system according to claim 1 or claim 2.

4. The data regarding the subject's condition is data other than visible light images. The monitoring system according to claim 1 or claim 2.

5. The monitoring system according to claim 1 or claim 2, wherein the acquisition unit includes at least one of a wearable sensor, an infrared sensor, a radio wave sensor, a depth sensor, a vibration sensor, a microphone, a temperature sensor, and a humidity sensor.

6. The imaging unit has a different appearance when it is not possible to image the subject and when it is possible to image the subject. The monitoring system according to claim 1 or claim 2.

7. A display unit that displays different information depending on whether the imaging unit is in a state where it is not possible to image the subject or whether the imaging unit is in a state where it is possible to image the subject. The monitoring system according to claim 6.

8. The aforementioned predetermined state is a state in which an abnormality is occurring in the subject, or a state in which it is suspected that an abnormality is occurring in the subject. The monitoring system according to claim 1 or claim 2.

9. A control device comprising: an acquisition unit that acquires data relating to the state of a subject, and when the state of the subject determined based on the acquired data is in a predetermined state, a control unit that sets the imaging direction of the imaging unit to a direction different from the direction in which the predetermined imaging range exists, until the state of the subject becomes the predetermined state.

Images