Program, monitoring system, and control device

Abstract

The program causes a computer to execute: when the state of a subject estimated on the basis of data acquired by an acquisition unit that acquires data relating to the state of the subject is a predetermined state, an imaging unit that captures an image including the subject starts imaging of the image; and detecting the state of the subject on the basis of the image captured by the imaging unit.

Description

Technical Field

[0001] Involves procedures, monitoring systems, and control equipment. Background Art

[0002] There has been proposed a monitoring system capable of remotely monitoring a resident living alone, an elderly person living alone during the day, a patient, and the like (for example, Patent Document 1).

[0003] Pursue a surveillance system that respects the privacy of the subject.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-84002 Summary of the invention

[0007] According to the first public method, this program causes the computer to perform the following processing: when it is inferred that the state of the aforementioned subject person is a specified state based on the aforementioned data acquired by the acquisition unit that acquires data related to the state of the subject person, the camera unit that captures the image including the aforementioned subject person starts capturing the aforementioned image, and the state of the subject person is detected based on the aforementioned image captured by the aforementioned camera unit.

[0008] According to the second disclosed method, the monitoring system includes: an acquisition unit, which acquires data related to the state of the subject person; a camera unit; and a control unit, which causes the camera unit to start capturing an image including the subject person when the state of the subject person is a specified state, and detects the state of the subject person based on the image captured by the camera unit.

[0009] According to the third disclosed method, the control device includes: a control unit, when it is inferred that the state of the aforementioned subject person is a specified state based on data acquired by an acquisition unit that acquires data related to the state of the subject person, the control unit causes the camera unit that captures an image including the aforementioned subject person to start capturing the aforementioned image, and detects the state of the subject person based on the aforementioned image captured by the aforementioned camera unit.

[0010] In addition, the configuration of the embodiments described below may be appropriately improved, and at least a portion thereof may be replaced by other configurations. Furthermore, the configuration of the components that are not particularly limited is not limited to the configuration disclosed in the embodiments, and can be configured at a position where its function can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a diagram showing the configuration of a monitoring system according to the first embodiment.

[0012] Figure 2is a diagram for explaining an example of the configuration of a sensor, a sub-camera, and a main camera.

[0013] Figure 3 (A) is a block diagram illustrating the configuration of a main camera. Figure 3 (B) is a block diagram showing the structure of the sub-camera.

[0014] Figure 4 (A) and Figure 4 (B) is a diagram illustrating a main camera according to the present embodiment. Figure 4 (C) and Figure 4 (D) is a diagram illustrating another example of the main camera involved in this embodiment.

[0015] Figure 5 : is a flowchart showing an example of processing performed by the control unit of the main camera.

[0016] Figure 6 is a flowchart showing the details of the first process.

[0017] Figure 7 is a flowchart showing the details of the second process.

[0018] Figure 8 : is a flowchart showing an example of processing performed by the control unit of the sub camera.

[0019] Fig. 9 (A) to Fig. 9 (D) is a sequence diagram showing an example of the process executed in the monitoring system according to the first embodiment.

[0020] Fig.10 This is a flowchart showing an example of processing executed by the control unit of the master camera in the monitoring system according to the second embodiment.

[0021] Fig.11 This is a flowchart (part 1) showing the details of the third process.

[0022] Fig.12 This is a flowchart (part 2) showing the details of the third process.

[0023] Fig.13 is a flowchart showing the details of the fourth process.

[0024] Fig.14 1 is a flowchart showing an example of processing executed by the control unit of the sub-camera in the monitoring system according to the second embodiment.

[0025] Fig.15 (A) and Fig.15(B) is a sequence diagram showing an example of processing executed in the monitoring system according to the second embodiment.

[0026] Fig.16 is a diagram showing an example of a transmitter list.

[0027] Fig.17 This is a flowchart showing an example of processing executed by the control unit of the main camera in the monitoring system according to the third embodiment.

[0028] Fig.18 is a flowchart showing the details of the fifth process.

[0029] Fig.19 : is a flowchart showing the details of the sixth process.

[0030] Fig. 20 This is a flowchart showing an example of processing executed by the control unit of the sub-camera in the monitoring system according to the third embodiment.

[0031] Fig.21 This is a sequence diagram showing an example of processing executed in the monitoring system according to the third embodiment.

[0032] Fig. 22 (A) is a diagram illustrating the hardware configuration of the control unit of the main camera. Fig. 22 (B) is a diagram illustrating the hardware configuration of the control unit of the sub-camera.

[0033] Fig.23 It is a diagram showing the configuration of a monitoring system according to a modification.

[0034] Fig.24 is a functional block diagram showing the configuration of a control device in a modified example. DETAILED DESCRIPTION

[0035] First embodiment

[0036] The following is based on Figures 1 to 9 (D) of the present invention will describe the monitoring system 100 according to the first embodiment in detail. Figure 1 , the configuration of the monitoring system 100 is shown in the form of a block diagram.

[0037] The monitoring system 100 is a system for allowing a person (family member, relative, care worker, care manager, etc.) related to the monitored person TR1 (hereinafter referred to as the monitored person TR1) to monitor the monitored person TR1. In the following description, the person related to the monitored person TR1 is referred to as the monitor OB1. Figure 1As shown, the monitoring system 100 according to the present embodiment includes a home system 150, a service server SS, and a mobile terminal MT1.

[0038] The home-side system 150 (more specifically, the main camera 40), the service server of the monitoring system 100, and the portable terminal MT1 are connected together via a network NW including a public wireless LAN, the Internet, a mobile phone line network, etc. The communication between the home-side system 150, the monitoring system 100, and the portable terminal MT1 can be set as communication based on the NICE (Network of Intelligent Camera Ecosystem) specification, for example.

[0039] The portable terminal MT1 is a portable terminal held by the monitor OB1 of the monitored person TR1, and may be, for example, a smart phone, a tablet terminal, or a notebook computer (Personal Computer). In addition, a terminal such as a desktop computer of the monitor OB1 may be used instead of the portable terminal MT1. An application for using the monitoring system is installed in the portable terminal MT1. Through the application, the monitor OB1 can give instructions to the home-end system 150, receive notifications from the home-end system 150, or view images (or pictures) taken by the sub-camera 20 and the main camera 40 of the home-end system 150.

[0040] The service server SS is a server that provides monitoring services. Based on a request from the home system 150, the service server SS pushes a message to the portable terminal MT1 or sends an instruction to the home system 150, the instruction corresponding to the operation of the application installed on the portable terminal MT1.

[0041] The home-side system 150 is installed in, for example, a house H1 where the subject person TR1 lives. The home-side system 150 includes, for example, a sensor 10 , a sub-camera 20 , a main camera 40 , and a remote controller 30 .

[0042] The sensor 10, sub-camera 20 and main camera 40 are installed in the house H1 where the subject TR1 lives, etc. The sensor 10, sub-camera 20 and main camera 40 are installed in places where the subject TR1 stays and uses, such as a living room, a corridor, a bathroom, a washroom, and a toilet.

[0043] Figure 2 1 is a diagram for explaining an example of the arrangement of the sensor 10, the sub-camera 20, and the main camera 40. Figure 2 In the example of FIG. 1 , the house H1 has three rooms R1 to R3. The sensors 10 are respectively installed in the rooms R1 to R3. Figure 2In the example, the sensors installed in the rooms R1 to R3 are referred to as sensors 10-1 to 10-3. Figure 2 In the embodiment, the number of sensors 10 installed in each room is one, but the number of sensors installed in each room may be multiple.

[0044] In the room where the sensor 10 is installed, at least one camera is installed. Figure 2 In the example of FIG. 1 , the main camera 40 is set in the room R1 where the sensor 10-1 is set, and the sub-cameras 20 are set in the rooms R2 and R3 where the sensors 10-2 and 10-3 are set respectively. Figure 2 In the example, the sub-cameras in rooms R1 and R3 are respectively referred to as sub-cameras 20-1 and 20-2. Sub-cameras 20 and main camera 40 can be installed on the ceiling, on the wall, or anywhere else in the living room. Figure 2 There is one camera set in each room, but the number of cameras set in each room can also be multiple. For example, a main camera 40 and a sub-camera 20 can be set in room R1, and multiple sub-cameras 20 can also be set in room R2.

[0045] Sensor 10 (at Figure 2 In the example, sensors 10-1 to 10-3) are connected to the main camera 40, for example, via DECT (Digital Enhanced Cordless Telecommunications). The sensor 10 and the main camera 40 may also be connected via near-field communications such as a wired LAN (Local Area Network), a wireless LAN, Wi-Fi, or Bluetooth (registered trademark).

[0046] Sub-camera 20 (in Figure 2 In the example of the present invention, the sub-cameras 20-1 and 20-2) and the main camera 40 are connected via Wi-Fi, for example. The sub-cameras 20 and the main camera may also be connected via near field communication such as wired LAN, wireless LAN, Wi-Fi or Bluetooth (registered trademark).

[0047] The remote controller 30 and the main camera 40 are connected, for example, via DECT. The remote controller 30 and the main camera 40 may also be connected via near field communication such as wired LAN, wireless LAN, Wi-Fi, or Bluetooth (registered trademark).

[0048] (Sensor 10)

[0049] The sensor 10 acquires data related to the state of the subject TR1. In the present embodiment, the sensor 10 acquires data related to the state of the subject TR1 in addition to the visible light image. Examples of the sensor 10 include an infrared array sensor, an infrared camera, a depth sensor, an electric wave sensor (millimeter wave radar), a vibration sensor, a sound sensor, a wearable sensor, a thermometer, and a hygrometer. In the present embodiment, the sensor 10 is an infrared array sensor as an example. In addition, a camera equipped with a filter that blocks visible light may also be used as the sensor 10.

[0050] In the present embodiment, the sensor 10 is installed on the ceiling, for example. When the sensor 10 is an infrared array sensor, the sensor 10 obtains temperature distribution data below the sensor 10, for example. Unlike visible light images, the temperature distribution data is data in which the face or clothing of the subject cannot be visually confirmed, so the privacy of the subject TR1 can be protected. In addition, since the infrared array sensor has no lens, it is possible to suppress the subject TR1 from having a feeling of "being monitored" or "privacy being invaded", thereby reducing the psychological burden of the subject TR1. In addition, the sensor 10 can also be installed on a wall, for example.

[0051] The data acquired by the sensor 10 is sent to the main camera 40 .

[0052] (Remote Control 30)

[0053] The remote controller 30 communicates with the main camera 40 and sends instructions corresponding to the operation of the remote controller 30 to the main camera. The remote controller 30, for example, has buttons or a touch screen (touch panel) for inputting various instructions. The remote controller 30 may also have at least one of a microphone for voice input instructions and a speaker for voice notification to the subject TR1. In addition, the remote controller 30 may also be a terminal device such as a smartphone installed with an application of the monitoring system 100.

[0054] (Main camera 40)

[0055] The main camera 40 is used to capture visible light images (or visible light images) (hereinafter referred to as “images”). In this embodiment, it is assumed that the home-end system 150 has one main camera, but it may also have multiple main cameras 40 .

[0056] Figure 3 (A) is a block diagram showing the configuration of the main camera 40 , which includes an imaging unit 41 , a microphone 42 , a speaker 43 , a storage unit 44 , a first communication module 45 , a second communication module 46 , a third communication module 47 , a control unit 48 , and a driving device 49 .

[0057] The imaging unit 41 includes a lens, an imaging element, and the like, and is used to capture an image within an imaging range.

[0058] The microphone 42 is used to acquire the voice uttered by the subject person TR1 and transmit it to the control unit 48. The microphone 42 may also be used as the sensor 10.

[0059] The speaker 43 outputs a designated voice under the control of the control unit 48 .

[0060] The storage unit 44 is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores the images captured by the imaging unit 41 .

[0061] The first communication module 45 is, for example, a DECT module, and is used to implement communication with the sensor 10 and the remote controller 30 .

[0062] The second communication module 46 is, for example, a Wi-Fi module, which is used to implement communication with the slave camera 20 .

[0063] The third communication module 47 is, for example, a LTE (Long Term Evolution) module, and is used to implement communication with the service server SS and the portable terminal MT1.

[0064] The control unit 48 controls the overall operation of the main camera 40. Details of the processing executed by the control unit 48 will be described later.

[0065] The driving device 49 is, for example, an actuator such as a motor, and is driven based on an instruction from the control unit 48 to move the cover plate 422 included in the main camera 40 (details will be described later).

[0066] (Sub-camera 20)

[0067] The sub-camera 20 captures visible light images similarly to the main camera 40. In the present embodiment, the home-side system 150 includes one or more sub-cameras 20. In addition, when the target person TR1 can be monitored only by the main camera 40, the sub-camera 20 may be omitted.

[0068] Figure 3 (B) is a block diagram illustrating the configuration of the sub-camera 20. The sub-camera includes an imaging unit 21, a microphone 22, a speaker 23, a storage unit 24, a second communication module 26, a control unit 28, and a driving device 29.

[0069] The difference between the sub-camera 20 and the main camera 40 is that the first communication module 45 and the third communication module 47 are omitted. In the sub-camera 20, the second communication module 26 is, for example, a Wi-Fi module, which can realize communication with the main camera 40. Since the other components are the same as the main camera 40, detailed description is omitted.

[0070] Next, the structural features of the sub-camera 20 and the main camera 40 involved in this embodiment will be described. Since the structures of the sub-camera and the main camera 40 are almost the same, the main camera 40 will be described here.

[0071] Figure 4 (A) and Figure 4 (B) is a diagram illustrating the main camera 40 involved in this embodiment. Figure 4 (A) and Figure 4 As shown in FIG. 5 (B), for example, the main camera 40 includes a lens 421 and a cover 422 that covers the lens 421 .

[0072] The cover plate 422 is driven by the driving device 49 based on the control of the control unit 48 so as to be able to move up and down. The control unit 48 makes the imaging unit 40 (lens 421) of the main camera 40 unable to capture images (such as the state in which the subject TR1 is abnormal) until it is determined based on the data from the sensor 10 that the subject TR1 is abnormal. Figure 4 Specifically, the driving device 49 is controlled so that the cover 422 covers the lens 421. That is, until it is determined that the subject person TR1 is abnormal, the control unit 48 sets the state in which the lens 421 of the main camera 40 cannot be seen from the subject person. In this way, the imaging unit 41 (lens 421) of the main camera 40 is in a state in which it is physically unable to capture an image.

[0073] Since the lens 421 is covered by the cover plate 422, the subject TR1 cannot visually check the lens 421. In this way, under normal circumstances (when the subject TR1 is not abnormal), the subject TR1 can be prevented from having a feeling of "being monitored" or "privacy being invaded".

[0074] When it is determined that the subject TR1 is abnormal, the control unit 48 moves the cover plate 422 downward, for example, so that the lens 421 is in a state where it can capture images (eg, Figure 4(A)). In this way, the main camera 40 has different appearances when it is able to shoot images and when it is unable to shoot images. In other words, when an abnormality occurs in the object TR1, the main camera 40 has a first appearance (the appearance of the lens 421 can be visually confirmed), and in other cases it has a second appearance (the appearance of the lens 421 cannot be visually confirmed). In this way, the subject TR1 can know through the appearance of the main camera 40 that the image including himself is not being captured, so that he can easily confirm whether his privacy is protected and gain a sense of security. In addition, by setting the color of the cover 422 to a different color from the frame of the main camera 40, or by providing an "×" mark on the cover 422, it is possible to easily visually confirm that the lens 421 is covered by the cover 422.

[0075] In addition, Figure 4 (A) and Figure 4 In (B), an example is described in which the main camera 40 normally includes a cover plate 422 that covers the lens 421, but the configuration of the main camera 40 is not limited to this.

[0076] Figure 4 (C) and Figure 4 (D) is a diagram showing another example of the main camera 40. Figure 4 (C) and Figure 4 The main camera 40 shown in (D) includes a lens 421 and an LED light 423 .

[0077] exist Figure 4 (C) and Figure 4 In the main camera 40 shown in (D), when the camera unit 41 of the main camera 40 is in a state where it cannot shoot an image, the control unit 48 makes the LED light 423 emit light in a first color (for example, red), and when the camera unit 41 is in a state where it can shoot an image, the control unit 48 makes the LED light 423 emit light in a second color (for example, green) different from the first color. In this way, the subject TR1 can know whether the main camera 40 can shoot an image by the color of the LED light.

[0078] In addition, until it is determined that the subject TR1 is abnormal, the control unit 48 can set the shooting direction of the main camera 40 (the direction of the lens 421) to a direction different from the direction of the preset camera range, and can also let the main camera 40 standby at a position (for example, under the bed) where the subject TR1 cannot visually confirm the main camera (or lens 421). In this way, since the subject TR1 cannot visually confirm the main camera 40 or the lens 421, it is possible to suppress the subject TR1 from feeling that he is "being monitored" or "his privacy is violated." In addition, when the direction of the lens 421 is set to a direction different from the direction of the preset camera range, an image that does not include the subject TR1 can also be captured.

[0079] Next, a description will be given of processes executed in the monitoring system 100. First, a description will be given of processes executed by the control unit 48 of the main camera 40. Figure 5 4 is a flowchart showing an example of processing performed by the control unit 48 of the host camera 40.

[0080] exist Figure 5 In the process of , first, the control unit 48 acquires data related to the state of the subject TR1 from the sensor 10 (step S11). In the present embodiment, since the sensor 10 is an infrared array sensor, the control unit 48 acquires temperature distribution data detected by the sensor 10.

[0081] Next, the control unit 48 infers the state of the subject TR1 based on the acquired data (temperature distribution data in this embodiment) (step S12). For example, when the highest temperature included in the temperature distribution data is above the first temperature, the control unit 48 infers that the subject TR1 is in a standing state; when the lowest temperature is above the second temperature, the highest temperature is below the third temperature, and the area of ​​the region from the second temperature to the third temperature is within a specified range, the control unit determines that the subject TR1 is in a sitting state. In addition, when the lowest temperature is above the second temperature, the highest temperature is below the third temperature, and the area of ​​the region from the second temperature to the third temperature is greater than the specified area, the control unit 48 infers that the subject TR1 is in a falling state. Regarding the method of inferring the state of the subject TR1 by using an infrared array sensor, for example, the method described in "Journal of Information Technology" (vol.114, no.166, ASN2014-81, pp.219-224, July 2014) can be used.

[0082] Next, the control unit 48 determines whether the estimated state of the subject TR1 is a state in which the subject TR1 is abnormal (predetermined state) (step S13). If the subject TR1 is not abnormal (step S13 / No), the process returns to step S11. The state in which the subject TR1 is abnormal refers to, for example, a state in which the subject TR1 falls or remains in the same posture for a long time.

[0083] When the subject TR1 is abnormal (step S13 / Yes), the control unit 48 identifies the camera installed in the same place (room, etc.) as the sensor 10 that detected the abnormality of the subject TR1 (step S14). For example, the storage unit 44 stores a table of associations between sensors and cameras installed in each room, and the control unit 48 can identify the camera in the same place (room, etc.) as the sensor that detected the abnormality of the subject TR1 by referring to the table.

[0084] Next, the control unit 48 determines whether the camera determined in step S14 is the main camera (step S15). Figure 2 In the example of FIG. 1 , when the sensor that detects that the subject TR1 is abnormal is the sensor 10-1 provided in the room R1, the camera determined in step S14 is the main camera 40. When the camera determined in step S14 is the main camera 40 (step S15), the control unit 48 executes the first process (step 20).

[0085] Figure 6 is a flowchart showing the details of the first process. Figure 6 In the process of, first, the control unit 48 sends a specified message to the portable terminal MT1 (step S201). For example, the control unit 48 sends a message indicating that the target person TR1 may have an abnormality. The message can be sent to the portable terminal MT1 via the service server SS or directly from the control unit 48.

[0086] Next, the control unit 48 starts counting the shooting standby time (step S203). The shooting standby time refers to the preset time until the sub-camera 20 or the main camera 40 starts shooting an image including the subject person TR1. In addition, the sub-camera 20 and the main camera 40 can shoot an image that does not include the subject person TR1 before or after shooting the image including the subject person TR1. In this way, the sub-camera 20 and the main camera 40 can be used as, for example, an anti-theft camera or a camera for monitoring the status of a pet. The shooting standby time can be set to an arbitrary time. During the shooting standby time, the sub-camera and the main camera 40 do not shoot an image including the subject person TR1. In addition, for a sub-camera 20 (in a location that is not located in the same place as the sensor 10 that detects the abnormality of the subject person TR1) Figure 2 In the example of FIG. 4 , the sub-cameras 20-1 and 20-2 are used. Since the image does not include the object TR1, the image can be captured. In this way, the sub-cameras and the main camera 40 can be used as, for example, anti-theft cameras or cameras for monitoring the status of pets.

[0087] Next, the control unit 48 issues a notification related to the start of filming to the subject TR1 (step 205). Specifically, the control unit 48 uses the speaker 43 to issue a notification related to the start of filming to the subject TR1. For example, the control unit 48 causes the speaker 43 to output a voice such as "filming will start in 5 seconds" to notify the subject TR1 of the timing when the main camera starts filming. In addition, instead of the timing of starting filming, the control unit 48 can notify the subject 43 through the speaker 43 of the time until the start of filming, the current time point is in the filming standby time, the current time point is in the period of prohibiting (cancelling) filming, etc. In this way, the subject TR1 can know the timing of starting image shooting, the time until the start of image shooting, the situation that the image has not started shooting, or the current shooting can be prohibited.

[0088] The processing order of steps S201 to S205 can be changed arbitrarily.

[0089] Next, the control unit 48 determines whether an input prohibiting shooting is received (step 207). Specifically, the control unit 48 performs voice recognition processing on the sound input from the microphone 42 to determine whether the input sound contains a prescribed voice for prohibiting shooting (for example, "prohibited shooting", "do not shoot", "stop", "false detection", "no abnormality", "stop", etc.). If the voice for prohibiting shooting is included, the control unit 48 determines that an input prohibiting shooting has been received.

[0090] When receiving the input of prohibiting shooting (step S207 / Yes), since the subject TR1 is in a state where the input of prohibiting shooting can be input (i.e., no abnormality occurs), it can be considered that Figure 5 The judgment in step S13 is wrong (misjudgment / false detection). In this case, the control unit 48 sends a predetermined message to the portable terminal MT1 (step S225). For example, when the control unit 48 receives an input prohibiting shooting, it sends a message notifying that the abnormality is a false detection or that the target person TR1 has no abnormality. In this way, the person OB1 holding the portable terminal MT1 (the person monitoring the target TR1) can know that the target person TR1 is safe.

[0091] Next, the camera unit 48 stops counting the shooting standby time (step S227) and returns to the Figure 5In this way, the main camera will not start shooting. Therefore, it is possible to prevent the subject TR1 from being shot even though there is no abnormality, and the privacy of the subject TR1 can be protected.

[0092] If no input for prohibiting shooting is received (step S207 / No), the control unit 48 determines whether a shooting request including an image of the subject TR1 is received from the portable terminal MT1 (step S209). For example, when a "shooting request" button is pressed (touched) in an application installed on the portable terminal MT1, a shooting request is sent from the portable terminal MT1. The shooting request can be sent directly from the portable terminal MT1 to the main camera 40, or can be sent via the service server SS.

[0093] If no shooting request is received (step S209 / No), the control unit 48 determines whether the shooting standby time has ended (step 211). Specifically, it determines whether the shooting standby time has ended since the shooting standby time was started in step S203. If the shooting standby time has not ended (step S211 / No), the process returns to step S207.

[0094] When receiving a shooting request (step 209 / Yes) or when the shooting standby time has ended (step 211 / Yes), the control unit 48 moves the cover 422 covering the lens 421 to expose the lens 421, and starts shooting the image of the object person TR1 by the camera unit 41 (step 213). At this time, the control unit 48 may also record the image shot by the camera unit 48 in the storage unit 44.

[0095] If the image including the subject TR1 is started to be captured, the control unit 48 starts to count the transmission standby time (step S215). The transmission standby time refers to the pre-set time from the start of image capture to the start of image transmission. During the transmission standby time, the image including the subject TR1 will not be transmitted. In addition, for example, when the sub-camera is capturing an image that does not include the subject TR1, the image that does not include the subject TR1 can also be transmitted. In this way, the sub-camera and the main camera can be used as, for example, an anti-theft camera or a camera for monitoring the status of a pet.

[0096] Next, the control unit 48 issues a notification related to the transmission to the subject TR1 (step S217). For example, the control unit 48 causes the speaker 43 to output a voice such as "transmission will start in 5 seconds" to notify the timing of the start of image transmission. In addition, instead of the timing of the start of image transmission, the control unit 48 may notify the subject TR1 of information such as the time until the image transmission starts, whether the current time point is within the transmission standby time, and whether the current time point is within the transmission prohibition period. In this way, the subject TR1 can know the time when the image transmission starts, the time until the image transmission starts, whether the image transmission has not started, or whether the transmission can be prohibited.

[0097] Next, the control unit 48 determines whether an input for prohibiting the transmission of the image has been received (step S219). For example, the control unit 48 performs voice recognition processing on the sound input from the microphone 42 to determine whether the input sound contains a prescribed voice for prohibiting the transmission of the image (e.g., "prohibit transmission", "do not transmit", "stop transmission", "false detection", "no abnormality", "stop", etc.). If the prescribed voice is contained, the control unit 48 determines that an input for prohibiting the transmission of the image has been received.

[0098] In the case of receiving an input for prohibiting image transmission (step S219 / Yes), it can be considered that Figure 5 The judgment in step S13 is wrong (misjudgment / false detection). In this case, the control unit 48 sends a predetermined message to the portable terminal MT1 (step S231). For example, the control unit 48 sends a message to notify that the abnormality is a false detection or that the target person TR1 has no abnormality. In this way, the person OB1 holding the portable terminal MT1 (the person monitoring the target TR1) can know that the target person TR1 is safe.

[0099] Next, the control unit 48 stops timing the transmission standby time and stops the image shooting by the camera unit 41 (step S233). In this way, since the image is not transmitted, it is possible to prevent the subject TR1 from being photographed even though there is no abnormality, thereby protecting the privacy of the subject TR1. In addition, if there is an image recorded in the storage unit 44, the recorded image can also be deleted. In this way, the privacy of the subject TR1 can be better protected.

[0100] Next, the control unit 48 drives the drive device 49 to cover the lens 421 with the cover plate 422 (step S235). In this way, the subject TR1 can visually confirm that the image is not being captured, so the subject TR1 can feel at ease that his privacy is protected. Figure 5 Step S11.

[0101] When no input for prohibiting image transmission is received (step S219 / No), the control unit 48 determines whether the transmission standby time has ended (step S221). Specifically, it determines whether the transmission standby time has ended since the transmission standby time was started in step S203. When the transmission standby time has not ended (step S211 / No), the process returns to step S219.

[0102] When the transmission standby time has ended (step S221 / Yes), the control unit 48 starts transmitting the image (step S223) and ends the first process (step 20). In addition, the image transmission may be a real-time view transmission, or an image from a predetermined time (e.g., several seconds) earlier than the current time may be transmitted. In addition, the past image may be viewed retrospectively.

[0103] On the other hand, Figure 5 When the camera determined in step S14 is a sub-camera 20 (step S12 / No), that is, when the camera set at the same place as the sensor 10 that detects the abnormality of the subject person TR1 is a sub-camera 20, the control unit will perform the second processing (step S30).

[0104] Figure 7 : is a flowchart showing the details of the second process. Figure 8 2 is a flowchart of an example of the process executed by the control unit 28 of the sub-camera 20. The second process will be described in conjunction with the process executed by the control unit 28 of the sub-camera 20.

[0105] because Figure 7 The processing of step S301 and step S303 is the same as Figure 6 Step S201 and step S203 in the first process shown are the same, so the description is omitted.

[0106] When the timing of the image capture standby time starts, the control unit 48 issues an instruction to the sub-camera 20 to notify the start of image capture (step S305 ).

[0107] On the other hand, the control unit 28 of the sub-camera waits until receiving a notification instruction related to the start of shooting from the main camera 40 ( Figure 8 : Step S401 / No). If the control unit 28 receives a notification instruction related to starting shooting from the main camera (Step S401 / Yes), it uses the speaker 23 as if Figure 6 In step 205, notification related to shooting is performed in the same manner (step S403).

[0108] Next, the control unit 28 determines whether a shooting instruction is received from the main camera 40. If a shooting instruction is not received (step S405 / No), the control unit 28 determines whether an input prohibiting shooting is received from the subject TR1 (step S421). For example, the control unit 28 performs voice recognition processing on the sound input from the microphone 22, and determines whether the input sound contains a prescribed voice for prohibiting the shooting of an image. If the prescribed voice is contained, the control unit 28 determines that an input prohibiting the shooting of an image is received.

[0109] If no input for prohibiting shooting is received (step S421 / No), the process returns to step S405. If an input for prohibiting shooting is received (step S421 / Yes), the control unit sends information indicating that an input for prohibiting shooting is received (a prohibiting shooting instruction) to the host camera 40 (step S423), and the process ends. Figure 8 processing.

[0110] On the other hand, the control unit 48 of the main camera determines whether it has received a prohibition instruction from the sub-camera 20 ( Figure 7 : Step S307). When a prohibition instruction from shooting is received from the sub-camera 20 (step S307 / yes), Figure 6 Similarly to steps S225 and S227, execute the processing of steps S325 and S327, and return Figure 5 Step S11.

[0111] When no prohibition instruction for shooting is received from the sub-camera 20 (step S307 / No), the control unit and Figure 6 Similarly to steps S209 and S211, the processing of steps S309 and S311 is performed.

[0112] When receiving a photographing request from the mobile terminal MT1 (step S309 / Yes), or when the photographing standby time has ended (step S311 / Yes), the control unit 48 sends a photographing instruction to the sub-camera 20 .

[0113] When receiving the shooting instruction from the main camera 40 , the control unit 28 of the sub-camera 20 drives the driving device 29 to move the cover plate to expose the lens of the sub-camera 20 and to start shooting with the camera unit 21 (step S407 ).

[0114] After that, the control unit 28 waits until there is an instruction from the main camera 40 to make a notification related to the transmission (step S409 / No).

[0115] On the other hand, after sending the shooting instruction, the control unit 48 of the main camera 40 starts counting the transmission standby time ( Figure 7 : Step S315). Then, the control unit 48 instructs the slave 20 to issue a notification related to the transmission (Step S317).

[0116] When an instruction to issue a notification related to transmission is received from the main camera 40 ( Figure 8 : Step S409 / Yes), the control unit 28 of the sub-camera sends a notification related to the transmission to the subject TR1 (step S411). For example, the control unit 28 causes the speaker 23 to output "transmission will start in 5 seconds" and the like, thereby notifying the subject TR1 of the timing of the start of image transmission. In addition, instead of the timing of the start of image transmission, the subject TR1 may be notified of information such as the time until the start of image transmission, whether the current time point is within the transmission standby time, and whether the current time point is within the period in which transmission can be prohibited. In this way, the subject TR1 can know the timing of the start of image transmission, the time until the start of image transmission, whether the image has not been transmitted, and whether transmission can be prohibited.

[0117] Next, the control unit 28 determines whether an input for prohibiting image transmission has been received (step S413). For example, the control unit 28 performs voice recognition processing on the sound input from the microphone to determine whether the input sound contains a prescribed voice for prohibiting image transmission. If the prescribed voice is contained, the control unit 28 determines that an input for prohibiting image transmission has been received.

[0118] When the input for prohibiting image transmission is received (step S413 / Yes), the control unit 28 transmits information indicating that the input for prohibiting transmission is received (transmission prohibition instruction) to the main camera (step S425 ).

[0119] The control unit 28 stops the image capture of the subject TR1 by the image capture unit 21 (step S147), drives the drive device 29 to cover the lens of the sub-camera 20 with the cover (step S419), and ends the operation. Figure 8 In addition, when there is an image recorded in the storage unit 24, the recorded image can also be deleted. In this way, the privacy of the subject TR1 can be better protected.

[0120] On the other hand, the control unit 48 of the main camera 40 determines whether a transmission prohibition instruction ( Figure 7 :Step S319). If a transmission prohibition instruction is received from the sub-camera 20 (step S319 / Yes), the control unit 48 sends a specified message to the portable terminal MT1 (step S329). For example, the control unit 48 sends a message to notify that the abnormality is a false detection, or to notify that the target person TR1 has no abnormality. In this way, the person OB1 holding the portable terminal MT1 (the person monitoring the target TR1) can know that the target person TR1 is safe.

[0121] Next, the control unit 48 stops counting the transmission standby time (step S331). In this way, since the transmission of the image is not performed, it is possible to prevent the subject TR1 from being photographed even though there is nothing wrong with it, thereby protecting the privacy of the subject TR1. Figure 5 Step S11.

[0122] If the transmission prohibition instruction is not received from the sub-camera (step S319 / No), the control unit 48 determines whether the transmission standby time has ended (step S321). If the transmission standby time has not ended (step S21 / No), the process returns to step S19.

[0123] When the transmission standby time has ended (step S21 / Yes), the control unit 48 instructs the sub-camera 20 to continue capturing the image including the target person TR1 (step S323 ), and starts transmitting the image including the target person TR1 (step S324 ).

[0124] On the other hand, Figure 8 In the case where the control unit 28 of the sub-camera 20 does not receive the input for prohibiting transmission (step S423 / No), it determines whether a continuous shooting instruction is received from the main camera (step S415). If the continuous shooting instruction is not received (step S415 / No), it returns to step 413.

[0125] When receiving the instruction to continue photographing (step S415 / Yes), the control unit 28 causes the imaging unit 21 to continue photographing the image including the subject TR1 (step S416), and ends the process. Figure 8 processing.

[0126] Next, use Fig. 9 (A) to Fig. 9 An example of the processing performed in the monitoring system 100 is described with reference to the timing diagram of (D). Fig. 9 (A) to Fig. 9 In (D), the vertical axis represents time.

[0127] First of all, Fig. 9 The example of (A) is used to illustrate. Fig. 9 In (A), at time t1, the control unit 48 determines that the subject person TR1 has an abnormality. In this case, the control unit 48 starts counting the imaging standby time at time t1.

[0128] Here, if an input for prohibiting photographing is received at time t3 before time t2 when the photographing standby time T1 has elapsed, the control unit 48 stops counting the photographing standby time. Fig. 9 In the example of (A), the image including the subject person TR1 is not captured, thereby preventing the subject person TR1 from being captured even though there is no abnormality.

[0129] Next, Fig. 9 The example of (B) is used to illustrate that Fig. 9 In (B), at time t1, the control unit 48 determines that the subject person TR1 is abnormal. In this case, the control unit 48 starts counting the imaging standby time at time t1.

[0130] When receiving a photographing request at time t3 before time t2 at which the photographing standby time T1 has elapsed, the imaging unit 21 or 41 starts photographing an image including the target person TR1 , and the control unit 48 starts counting the transmission standby time.

[0131] When the transmission standby time T2 has elapsed at time t4, the control unit 48 starts transmitting the image. Fig. 9 In the example of (B), after time t4, the video including the target person TR1 is transmitted to the mobile terminal MT1.

[0132] Next, Fig. 9 The example shown in (C) is used to illustrate this. Fig. 9 In (C), at time t1, the control unit 48 determines that the subject person TR1 has an abnormality. In this case, the control unit 48 starts counting the imaging standby time at time t1.

[0133] When the imaging standby time T1 has elapsed at time t2, the imaging unit 21 or 41 starts imaging an image including the target person TR1, and the control unit starts counting the transmission standby time.

[0134] If an input for prohibiting transmission is received at time t4 before time t3 when the transmission standby time T2 has elapsed, the control unit 48 stops counting the transmission standby time. Fig. 9 In the example of (C), the video including the subject person TR1 is not transmitted to the mobile terminal MT1, and it is possible to prevent the video from being transmitted even though there is no abnormality in the subject person TR1.

[0135] Next, Fig. 9 The example shown in (D) is used to illustrate this. Fig. 9 In (D), at time t1, the control unit 48 determines that the subject person TR1 has an abnormality. In this case, the control unit 48 starts counting the imaging standby time at time t1.

[0136] When the imaging standby time T1 has elapsed at time t2, the imaging unit 21 or 42 starts imaging an image including the target person TR1, and the control unit 48 starts counting the transmission standby time.

[0137] If the transmission standby time T2 has passed at time t3, the control unit 48 starts transmitting the image. Fig. 9 In the example of (D), after time t3, a video including the subject person TR1 is transmitted to the mobile terminal MT1.

[0138] As described in detail above, according to the first embodiment, the monitoring system 100 includes: the sensor 10 that acquires data related to the state of the subject TR1; the main camera 40 and the sub-camera 20 that start capturing an image including the subject TR1 when the subject TR1 is in a predetermined state (the subject TR1 is abnormal); and the control unit 48 and the control unit 28 that make the main camera 40 and the sub-camera 20 in a state where the subject TR1 cannot be captured until the state of the subject TR1 becomes a predetermined state. In this way, since the subject TR1 is not normally captured, the privacy of the subject TR1 can be protected.

[0139] In the first embodiment, the main camera 40 and the sub-camera 20 have different appearances when they are in a state where they cannot photograph the subject TR1 and when they are in a state where they can photograph the subject TR1 (see FIG. 1 ). Figure 4 (A)~ Figure 4 In this way, the subject TR1 can determine whether the main camera 40 and the sub-camera 20 are taking pictures by checking the appearance of the main camera 40 and the sub-camera 20.

[0140] Furthermore, in the first embodiment, when the main camera 40 and the sub-camera 20 cannot capture the subject TR1, the subject TR1 cannot see the lenses of the main camera 40 and the sub-camera 20. For example, the control unit 48 keeps the lens 421 of the main camera 40 covered by the cover plate 422 until the subject TR1 becomes in a predetermined state (until the subject TR1 becomes abnormal). In this way, it is possible to suppress the subject TR1 from having a feeling that he is being monitored.

[0141] In addition, as described above, the control units 48 and 28 may respectively set the shooting directions of the main camera 40 and the sub-camera 20 to directions different from the directions of the preset shooting ranges until the state of the subject TR1 becomes a predetermined state. In this way, it is also possible to suppress the subject TR1 from having a feeling of "being monitored".

[0142] In addition, in the first embodiment, the sensor 10 acquires data other than visible light images as data related to the state of the subject person. In this way, it is possible to determine whether the subject person TR1 is abnormal while protecting the privacy of the subject person TR1.

[0143] Furthermore, in the first embodiment, after the shooting standby time (first predetermined time) has passed since the state of the subject TR1 estimated by the data acquired by the sensor 10 that acquires data related to the state of the subject TR1 is a predetermined state (a state in which the subject TR1 is abnormal), the control unit 48 causes the camera unit 21 or 41 to start shooting an image including the subject TR1. Since there is a time lag between the determination that the subject TR1 is abnormal and the start of the image shooting, it is possible to take measures to protect the privacy of the subject TR1 when the subject TR1 is not actually abnormal. Therefore, compared with the case where the shooting is started immediately after the determination that the subject TR1 is abnormal, the privacy of the subject TR1 can be better protected.

[0144] Furthermore, in the first embodiment, if an input for prohibiting (cancelling) photographing is received during the photographing standby time, the control unit 48 does not execute the process of causing the camera unit 21 or 41 to photograph. In this way, for example, when the subject person TR1 is not actually abnormal, the camera unit 21 or 41 can be prohibited from photographing an image including the subject person TR1, thereby protecting the privacy of the subject person TR1.

[0145] In addition, in the first embodiment, when receiving an input prohibiting shooting, the control unit 48 sends a predetermined message (for example, a message notifying that the abnormality is a false detection or notifying that no abnormality has occurred) to the portable terminal (external device) MT1. In this way, the person (the person monitoring the target person TR1) OB1 holding the portable terminal MT1 can know that the target person TR1 is safe.

[0146] Furthermore, in the first embodiment, if it is determined that the subject person TR1 has an abnormality, the control unit 48 issues a notification related to the start of filming to the subject person TR1. Specifically, in the first embodiment, the subject person TR1 is notified of the timing of the start of filming. In this way, the subject person TR1 can know the timing of starting filming. In the case where the subject person TR1 has no abnormality, since the measures required to protect the privacy (such as prohibiting filming, moving outside the camera range, etc.) can be taken, the privacy of the subject person TR1 can be better protected compared to the case where no filming start notification is issued. In addition, in the above-mentioned embodiment, the notification is issued by the speaker 43 of the main camera 40 or the speaker 23 of the sub-camera 20, but it is not limited to this. For example, the remote control 30 may also be provided with a speaker, and the notification may be issued via the speaker of the remote control 30. In addition, the remote control 30 may also be provided with a display unit, and the notification may be displayed on the display unit.

[0147] In addition, in the first embodiment, when the estimated state of the subject TR1 becomes a predetermined state (a state in which the subject TR1 is abnormal), the control unit 48 sends a notification related to the subject TR1 to the portable device MT1. Specifically, a message is sent to the portable terminal MT1, and the message notifies that the subject TR1 may be abnormal. In this way, the person OB1 holding the portable terminal MT1 can know that the subject TR1 is abnormal, and can take actions such as calling the subject TR1 or visiting the subject TR1.

[0148] Furthermore, in the first embodiment, if a shooting request is received from the portable terminal MT1 during the shooting standby time, the control unit 48 causes the camera unit 21 or the camera unit 41 to start shooting before the shooting standby time has elapsed. In this way, it is possible to store the image of the subject TR1 in response to the request of the observer OB1 holding the portable terminal MT1. In addition, when the camera unit 21 or the camera unit 41 starts shooting, the subject TR1 may be notified to start shooting.

[0149] Furthermore, in the first embodiment, when the transmission standby time (second prescribed time) has passed since the camera unit 21 or 41 started shooting, the control unit 48 transmits the image shot by the camera unit 21 or 41. Since there is a time lag between the start of shooting and the transmission of the image, the subject TR1 can take necessary actions within the transmission standby time when there is no abnormality in the subject TR1. In this way, the privacy of the subject TR1 can be better protected compared to the case where the image transmission starts immediately after the camera unit 21 or 41 starts shooting. In addition, in the case of an abnormality in the subject TR1, the image shot by the camera unit 21 or 41 is transmitted after the transmission time ends, and the person OB1 holding the portable terminal MT1 can confirm the status of the subject TR1 through the image.

[0150] Furthermore, in the first embodiment, if an input for prohibiting (cancelling) transmission is received within the transmission standby time (second predetermined time), the control unit 48 stops counting the transmission standby time. In this way, the image captured by the camera unit 21 or 41 can be prevented from being transmitted, thereby preventing the privacy of the subject TR1 from being violated due to the transmission of the image when there is no abnormality in the subject TR1.

[0151] In addition, in the first embodiment, before the image transmission process is performed, the control unit 48 notifies the object TR1 of the image transmission (publishing or broadcasting). In this way, the object TR1 can know that the image will be transmitted and can take actions required to protect privacy.

[0152] Furthermore, in the first embodiment, when the state of the subject TR1 estimated from the data acquired by the sensor 10 that acquires data related to the state of the subject TR1 becomes a predetermined state (a state in which the subject TR1 is abnormal), the control unit 48 issues a notification related to the start of capturing an image including the subject TR1. In this way, the subject TR1 can be informed that the capturing will start, and can take actions necessary to protect the privacy of the subject TR1.

[0153] In the first embodiment, when the shooting standby time (first predetermined time) has passed since the state of the subject TR1 has changed to the predetermined state, the control unit 48 causes the camera unit 41 or the camera unit 21 to start shooting an image, and notifies the timing of starting shooting during the shooting standby time. In this way, for example, when the subject TR1 has no abnormality, the subject TR1 can take actions necessary to protect the privacy of the subject TR1 until the timing of starting shooting.

[0154] In addition, as described above, the control unit 48 may notify that the current time point is within the shooting standby time, or may notify that the current time point is before shooting is performed by the camera unit 21 or 41. With such notification, for example, when the subject person TR1 does not have any abnormality, the subject person TR1 can take actions necessary to protect the privacy of the subject person TR1.

[0155] In addition, the control unit 48 may notify the time until the start of imaging. With such notification, for example, when the subject person TR1 is not abnormal, the subject person TR1 can take actions necessary to protect the privacy of the subject person TR1.

[0156] Furthermore, the control unit 48 may notify that the current time point is within the period in which imaging can be prohibited. In this way, for example, when the subject TR1 is not abnormal, the subject TR1 can take necessary actions to prohibit imaging.

[0157] In addition, in the first embodiment, the control unit 48 starts transmitting the image captured by the camera unit 21 or 41 when the camera unit 21 or 41 starts capturing the image, and issues a notification related to the capturing of the image before the image transmission starts. In this way, for example, when the subject person TR1 does not have any abnormality, the subject person TR1 can take actions necessary to protect the privacy of the subject person TR1.

[0158] In addition, in the first embodiment, when the transmission standby time has passed since the camera unit 21 or 41 started capturing an image including the subject person TR1, the control unit 48 starts transmitting the captured image and notifies the timing of transmission start within the transmission standby time. In this way, for example, if the subject person TR1 does not have any abnormality, the subject person TR1 can take actions necessary to protect the privacy of the subject person TR1 until the timing of image transmission start.

[0159] In addition, the control unit 48 may notify the time until the transmission starts. With such notification, for example, when the subject TR1 has no abnormality, the subject TR1 can take necessary actions to protect the privacy of the subject TR1.

[0160] In addition, the control unit 48 may notify that the current time point is within the period in which transmission can be prohibited (cancelled). In this way, the subject TR1 can know that transmission can be prohibited and can take necessary actions to protect the privacy of the subject TR1.

[0161] Furthermore, in the first embodiment, if an input prohibiting shooting is received during the shooting standby time, the control unit 48 does not execute the process of starting to shoot an image. Receiving an input prohibiting shooting means that the subject TR1 is considered to be in a state where an input prohibiting shooting can be performed, that is, in a state where no abnormality has occurred. Therefore, if an input prohibiting shooting is received during the shooting standby time, by not executing the process of starting to shoot an image, the privacy of the subject TR1, who is likely to be free of abnormality, can be protected.

[0162] Furthermore, in the first embodiment, if an input prohibiting transmission is received during the transmission standby time, the control unit 48 does not execute the process of starting transmission. Receiving the process of prohibiting transmission means that the subject TR1 is in a state where an input prohibiting transmission can be made, that is, in a state where no abnormality has occurred. Therefore, if an input prohibiting transmission is received during the transmission standby time, the process of starting image transmission is not executed, thereby protecting the privacy of the subject TR1 who is likely to have no abnormality.

[0163] Second embodiment

[0164] In the first embodiment, it is determined whether the state of the subject TR1 estimated based on the data obtained from the sensor 10 is in a predetermined state (a state in which the subject TR1 is abnormal). In the first embodiment, if the determination is wrong, if the subject does not notice the notification related to shooting or the notification related to transmission and does not input an input to prohibit shooting or transmission, the image will be shot or transmitted, and there is a risk that the privacy of the subject TR1 cannot be protected.

[0165] Therefore, in the second embodiment, based on the image captured by the camera unit 21 or 41, it is determined whether the determination result of "the subject person is abnormal" obtained based on the data obtained from the sensor 10 is correct. Specifically, the state of the subject person TR1 is estimated based on the image captured by the camera unit 21 or 41. When the estimated state of the subject person TR1 is in a specified state (a state in which the subject person TR1 is abnormal), it is considered that the abnormality determination based on the data obtained by the sensor 10 is correct, and the transmission standby time is started to be counted. This is because, generally, since the amount of information obtained from the image captured by the camera unit 21 or 41 is greater than the amount of information obtained from the data obtained by the sensor 10, it can be considered that the state of the subject person TR1 estimated from the image captured by the camera unit 21 or 41 is more accurate than the state of the subject person TR1 estimated from the data obtained by the sensor 10.

[0166] In the second embodiment, the processes executed by the control unit 28 and the control unit 48 are different from those in the first embodiment. Since the configuration of the monitoring system 100, the configuration of the main camera, and the configuration of the sub-camera are the same as those in the first embodiment, detailed description thereof will be omitted.

[0167] Fig.10 FIG. 4 is a flowchart showing an example of processing performed by the main camera 48 in the monitoring system 100 according to the second embodiment. Fig.10As for the processing, the processing (step S50: third processing) performed when the camera determined in step S14 is the main camera 40 (step S15 / yes), and the processing (step S60: fourth processing) performed when the camera determined in step S14 is the sub-camera 20 (step S15 / no) are the same as Figure 5 The processing is different.

[0168] Fig.11 and Fig.12 is a flowchart showing the details of the third process. Fig.11 In the process of steps S201 to S213 and steps S225 and S227, Figure 6 The first processing shown is the same, so the same figure mark is marked and the detailed description thereof is omitted.

[0169] In the third process, when the camera unit 41 starts shooting (step S213), the control unit 48 estimates the state of the subject TR1 based on the image (visible light image) shot by the camera unit 41 (step S501). Next, the control unit 48 determines whether the estimated state of the subject TR1 is in a specified state (a state in which the subject TR1 is abnormal) (step S503). For example, the control unit 48 performs pattern matching between the image shot by the camera unit 41 and the image recorded in advance for determining abnormal conditions, thereby determining whether the subject TR1 is abnormal.

[0170] If the subject TR1 is not abnormal (step S503 / No), the control unit 48 sends a predetermined message to the portable terminal MT1 (step S507). For example, the control unit 48 sends a message notifying the subject TR1 of the abnormality. Fig.10 The judgment in step S13 is wrong (false judgment / false detection), or the notification object TR1 does not have any abnormality.

[0171] Next, the control unit 48 stops the shooting of the camera unit 41 (step S309), and drives the driving device 49 to move the cover plate 422 so that the cover plate 422 covers the lens 421 (step S511). In addition, the processing order of steps S507 to S511 can be interchanged. After step S511, the control unit 48 returns to step S309. Fig.10 Step S11.

[0172] When it is determined that the subject TR1 is abnormal (step S503 / Yes), the control unit 48 stores the image used to estimate the state of the subject TR1 in the storage unit 44. In this way, the person OB1 who monitors the subject TR1 can later confirm and verify the image used to determine whether the subject TR1 is abnormal. In addition, the image used to determine whether the subject TR1 is abnormal can be transmitted to the portable terminal MT1 after the transmission time has passed.

[0173] because Fig.12 The processing after step S215 is the same as Figure 6 The processes after step S215 in the first process shown are the same, so the same reference numerals are used and detailed description thereof is omitted.

[0174] Next, the details of the fourth process will be described. Fig.13 4 is a flowchart showing the details of the fourth process. Fig.14 1 is a flowchart showing an example of the process executed by the control unit 28 of the sub-camera 20 in the monitoring system 100 according to the second embodiment. The fourth process will be described in conjunction with the process executed by the control unit 28 of the sub-camera 20.

[0175] exist Fig.13 In the fourth process shown in FIG. 1 , since the processes of steps S301 to S313 and steps S325 and S327 are the same as those of steps S301 to S313, Figure 7 The second processing shown is the same, so the same figure mark is marked and the detailed description thereof is omitted.

[0176] In addition, Fig.14 In the process performed by the control unit 28 of the sub-camera 20 shown in FIG. 1 , since the processes of steps S401 to S407 and steps S421 and S423 are similar to those of steps S401 to S407, the processes of steps S421 and S423 are similar to those of steps S401 to S407. Figure 8 The processes shown are the same, so the same figure marks are marked and the detailed description thereof is omitted.

[0177] exist Fig.14 In the embodiment of the present invention, if the imaging unit 21 starts capturing an image including the subject person TR1 (step S407), the control unit 28 of the sub-camera estimates the state of the subject person TR1 based on the image (visible light image) captured by the imaging unit 21 (step S451). Next, the control unit 28 determines whether the estimated state of the subject person TR1 is in a predetermined state (a state in which the subject person TR1 is abnormal) (step S453).

[0178] If the subject TR1 is not abnormal (step S453 / No), the control unit 28 stops the image capture of the camera unit 21 (step S461) and covers the lens of the sub-camera 20 with a cover (step S463). Next, the control unit 28 sends the estimated result of the subject TR1's state to the main camera 40 (step S465), and ends the operation. Fig.14 In step S465, the inference result that the subject person has no abnormality is sent to the main camera 40.

[0179] On the other hand, when the subject TR1 is abnormal (step S453 / Yes), the control unit 28 stores the image used to estimate the state of the subject TR1 in the storage unit 24 (step S455). In this way, the person OB1 who monitors the subject TR1 can later confirm and verify the image used to determine whether the subject TR1 is abnormal. In addition, the image used to determine whether the subject TR1 is abnormal can also be transmitted to the portable terminal MT1 after the transmission time has passed.

[0180] Next, the control unit 28 sends the estimation result of the subject person TR1's state to the main camera 40 (step S457 ), and waits until receiving a notification instruction related to the transmission (step S409 ). In step S457 , the estimation result that the subject person is not abnormal is sent to the main camera 40 .

[0181] Since the processing after step S409 is the same as Figure 8 The processes shown are the same, so the same figure marks are marked and the detailed description thereof is omitted.

[0182] On the other hand, after the control unit 48 of the main camera 40 sends a shooting instruction to the sub-camera 20 ( Fig.13 : Step S313), wait until the sub-camera 20 receives the estimated result of the state of the subject TR1 based on the visible light image (step S601 / no).

[0183] If the estimation result is received (step S601 / Yes), the control unit 48 determines whether the estimation result indicates that the subject TR1 is abnormal (step S602).

[0184] If the subject TR1 is not abnormal (step S602 / No), the control unit 48 sends a specified message to the portable terminal MT1 (step S603). For example, the control unit 48 sends a message notifying the subject TR1 of the abnormality. Fig.10 The judgment in step S13 is wrong (false judgment / false detection), or the notification object TR1 does not have any abnormality.

[0185] If the subject TR1 is abnormal (step S602 / Yes), the control unit 48 starts counting the transmission standby time (step S315). The processing after step S315 is the same as Figure 7 The second processing shown is the same, so the same figure mark is marked and the detailed description thereof is omitted.

[0186] Fig.15 (A) and Fig.15 (B) is a sequence diagram showing an example of the process executed in the monitoring system 100 according to the second embodiment.

[0187] First of all, Fig.15 The example of (A) is used to illustrate. Fig.15 In (A), at time t1, the control unit 48 determines that the subject person TR1 has an abnormality based on the data obtained from the sensor 10.

[0188] In this case, the control unit 48 starts counting the imaging standby time from time t1. When the imaging standby time has elapsed at time t2, the imaging unit 21 or 41 starts imaging an image including the target person TR1.

[0189] At time t3, if the control unit 28 or 48 determines that the subject person TR1 is not abnormal based on the captured image, the image capturing unit 21 or 41 stops capturing. This prevents the subject person from being captured and transmitted even if the subject person is not abnormal.

[0190] Next, Fig.15 The example of (B) is used to illustrate. Fig.15 In (B), at time t1, the control unit 48 determines that the subject person TR1 has an abnormality based on the data obtained from the sensor 10.

[0191] In this case, the control unit 48 starts counting the imaging standby time from time t1. When the imaging standby time T1 elapses at time t2, the imaging unit 21 or 41 starts imaging an image including the target person TR1.

[0192] When the control unit 28 or 48 determines that the target person TR1 is abnormal based on the captured image at time t3, the control unit 48 starts counting the transmission standby time.

[0193] When the transmission standby time T2 has elapsed at time t4, the control unit 48 starts the transmission of the video. In this way, when there is a high possibility that the subject TR1 is abnormal, the video can be transmitted.

[0194] As described in detail above, according to the second embodiment, when the state of the subject person TR1 estimated from the data acquired by the sensor 10 that acquires data related to the state of the subject person is in a predetermined state (a state in which the subject person TR1 is abnormal), the control unit 48 causes the imaging unit 21 or 41 to start capturing an image including the subject person TR1, and estimates (detects) the state of the subject person TR1 based on the image captured by the imaging unit 21 or 41. In this way, it is possible to confirm whether the state of the subject person TR1 estimated from the data acquired by the sensor 10 is correct.

[0195] In the second embodiment, when the state of the subject TR1 estimated (detected) based on the image captured by the imaging unit 21 or 41 is in a predetermined state (a state where the subject TR1 is abnormal), the control unit 48 continues capturing the image. In this way, the image of the abnormal subject TR1 can be recorded.

[0196] In addition, when the image shooting is continued, the control unit 28 or 48 may also notify the subject TR1 of information related to the image shooting. For example, the control unit 28 or 48 may notify the subject TR1 of the timing to start recording the shot image in the storage unit 24 or 44. In this way, the subject TR1 can know that the shot image is recorded.

[0197] In addition, the imaging unit 28 or 48 may notify the subject person TR1 that the image is being captured. In this way, the subject person TR1 can know that an image including himself is being captured.

[0198] Furthermore, in the second embodiment, when the state of the subject person TR1 estimated (detected) based on the image is in a predetermined state (a state in which the subject person TR1 is abnormal), the control unit 48 starts transmitting the image captured by the imaging unit 21 or 41. In this way, when it is considered that there is a high possibility that the subject person is abnormal, the transmission of the image can be started.

[0199] Furthermore, in the second embodiment, when the state of the subject TR1 estimated based on the image is in a predetermined state (a state in which the subject TR1 is abnormal), the control unit 28 or 48 stores the image for estimating the state of the subject TR1. In this way, the image in which the subject TR1 is judged to be abnormal can be confirmed or inflamed later.

[0200] Furthermore, in the second embodiment, when the transmission standby time has passed since the state of the subject TR1 detected based on the image is in a predetermined state (a state in which the subject TR1 is abnormal), the control unit 48 starts transmitting the image including the subject TR1. Since there is a time lag between the determination based on the image that the subject TR1 is abnormal and the image being transmitted, if the subject TR1 is not abnormal, the subject TR1 can take actions necessary for his or her privacy within the transmission standby time.

[0201] In the second embodiment, the control unit 48 notifies the subject TR1 of information on video transmission before the video transmission process is performed. In this way, the subject TR1 can take necessary actions to protect his privacy before the video transmission process is performed.

[0202] Furthermore, in the second embodiment, when the state of the subject TR1 detected based on the image is not a predetermined state (a state in which the subject TR1 is abnormal), the control unit 28 or 48 stops the image capture. That is, when it is determined based on the image that the subject TR1 is not abnormal, the control unit 28 or 48 stops the image capture. In this way, it is possible to prevent the situation in which the privacy of the subject TR1 is not protected by continuing to capture an image including the subject TR1 even if the subject TR1 is not abnormal.

[0203] Furthermore, in the second embodiment, when the state of the subject TR1 detected based on the image is not a prescribed state (a state where the subject TR1 is not abnormal), the control unit 48 sends a prescribed message (for example, a message conveying that the abnormality is a false detection) to the portable terminal MT1. In this way, the person holding the portable terminal MT1 (the person monitoring the subject TR1) can ensure that the subject TR1 is safe.

[0204] In addition, in the second embodiment, when the camera unit 21 or 41 starts shooting, the camera unit 21 or 41 may store the shot image in the storage unit 24 or 44. In this case, when the state of the target person TR1 detected based on the image is not a predetermined state (the target person TR1 is abnormal), the control unit 28 or 48 may delete the image (including the image used to detect the target person TR1) stored in the storage unit 24 or 44. In this way, the privacy of the target person TR1 can be protected.

[0205] In the first and second embodiments, either the imaging standby time or the transmission standby time may be omitted. In this case, the subject TR1 only needs to be able to input either the imaging prohibition input or the transmission prohibition input.

[0206] Furthermore, in the first and second embodiments described above, there may be a plurality of portable terminals that receive messages and transmit images.

[0207] Third embodiment

[0208] In the first and second embodiments, an example in which a video is transmitted to one mobile terminal MT1 is described. However, when there are a plurality of monitored persons TR1, the timing of transmitting the video may be changed according to the type (attribute) of the monitored person.

[0209] The monitoring system 100 according to the third embodiment differs from the first and second embodiments in that the processes executed by the control unit 48 and the control unit 28 are different from the first and second embodiments. Also, the storage unit 44 of the main camera 40 stores a transmitter list, which is also different from the first and second embodiments.

[0210] Fig.16 : is a diagram showing an example of a transmitter list. The transmitter list has fields for ID, name, communication information, and group. In the ID field, an identifier that uniquely identifies the person of the monitored object TR1 is stored. In the name field, the name of the person of the monitored object TR1 is stored. In the communication information, information required for transmitting images directly from the main camera 40 to the portable terminal (for example, IP information of the portable terminal) and the like are stored. The group is used to classify the person identified by the ID, and in the present embodiment, any one of groups 1 to 3 is stored. Here, group 1 represents, for example, the family members of the object TR1, group 2 represents, for example, the relatives of the object TR1, and group 3 represents, for example, the caregiver or care manager of the object.

[0211] In the third embodiment, the control unit 48 changes the transmission timing and transmission content of the image in response to the above-mentioned group. In the present embodiment, it is assumed that the image is transmitted to the groups 1 and 2, and the image is not transmitted to the group 3.

[0212] Fig.17 FIG. 1 is a flowchart showing an example of processing performed by the control unit of the main camera in the monitoring system according to the third embodiment. Fig.17 As for the processing, the processing (step S70: fifth processing) performed when the camera determined in step S14 is the main camera 40 (step S15 / yes), and the processing (step S80: sixth processing) performed when the camera determined in step S14 is the sub-camera 20 (step S15 / no) are the same as Figure 5 and Fig.10 The processing is different.

[0213] Fig.18 is a flowchart showing the details of the fifth process. Fig.18In the process of steps S201 to S213, and steps S225 and S227, Figure 6 The first processing shown is the same, so the same figure mark is marked and the detailed description thereof is omitted.

[0214] exist Fig.18 In the process of FIG. 21, if the image including the subject person TR1 is started to be captured (step 213), the control unit 48 starts to send the image to the person belonging to group 1 (the first group) (for example, Fig.16 The mobile terminal of the target person TR1 (the person with ID "OB1" in the example) transmits the image captured by the camera unit 41. In this way, the person belonging to the group 1 can quickly confirm the status of the target person TR1 through the image.

[0215] Next, the control unit 48 starts counting the transmission standby time (step S703 ).

[0216] The control unit 48 notifies the subject TR1 of the person ( Fig.16 For example, the control unit 48 causes the speaker 43 to emit a voice message such as "I will start transmitting images to people belonging to group 2 in 5 seconds." In this way, the subject TR1 can know that the images captured by the camera unit 41 will start to be transmitted to people belonging to group 2.

[0217] Next, the control unit 48 determines whether an input for prohibiting (cancelling) image transmission is received (step S707 ).

[0218] When receiving the input for prohibiting the transmission of the image (step S707 / Yes), the control unit 48 stops counting the transmission standby time and stops the shooting of the camera unit 41. In this way, it is possible to prevent the transmission of the image captured by the camera unit 41 to the personnel belonging to the group other than the group 1. In addition, since the camera unit 41 stops shooting, the transmission of the image to the personnel belonging to the group other than the group 1 is also stopped. In addition, at this time, the control unit 48 may also send a message to the personnel belonging to each group, notifying that the judgment of step S13 is wrong or that the notification target person TR1 is not abnormal.

[0219] After that, the control unit 48 covers the lens 421 of the main camera 40 with the cover plate 422 (step S715), and returns to Fig.17 Step S11.

[0220] If no input for prohibiting image transmission is received (step S707 / No), the control unit 48 determines whether the transmission standby time has passed (step S709). Specifically, it determines whether the transmission standby time has passed since the transmission standby time was counted in step S703. If the transmission standby time has not passed (step S709 / No), the process returns to step S707.

[0221] When the transmission standby time has passed (step S709 / yes), the control unit 48 starts transmitting images to the portable terminals of the persons belonging to group 2 (the second group), and sends a notification to the portable terminals of the persons belonging to group 3, for example, notifying that the transmission of images to group 2 has started (step S711). In this way, the persons belonging to group 2 can confirm the state of the subject person TR1 through the images. In addition, the start of transmitting images to group 2 means that the subject person TR1 is likely to be abnormal. Since the persons belonging to group 3 can know through the notification that the subject person TR1 is likely to be abnormal, they can take countermeasures such as visiting the subject person TR1.

[0222] Next, the details of the sixth process will be described. Fig.19 : is a flowchart showing the details of the sixth process. Fig. 20 1 is a flowchart showing an example of the process executed by the control unit 28 of the sub-camera 20 in the monitoring system according to the third embodiment. The sixth process will be described in conjunction with the process executed by the control unit 28 of the sub-camera 20.

[0223] exist Fig.19 In the process of steps S301 to S313 and steps S325 and S327, Figure 7 The second processing is the same as that of the first processing, so the same figure mark is marked and the detailed description is omitted.

[0224] In addition, Fig. 20 In the process except steps S471 and S473, Figure 8 The processes shown are the same, so the same figure marks are marked and the detailed description thereof is omitted.

[0225] exist Fig.19 In the embodiment, if the control unit 48 of the main camera 40 sends a shooting instruction to the sub-camera 20 (step S313), the person belonging to group 1 (the first group) (for example, Fig.16 The portable terminal of the target person TR1 (the person whose ID is "OB1" in the figure) transmits the image captured by the camera unit 21 of the sub-camera 20. In this way, the people belonging to the group 1 can quickly confirm the status of the target person TR1 through the image.

[0226] Next, the control unit 48 starts counting the transmission standby time (step S803 ).

[0227] The control unit 48 instructs the sub-camera 20 to notify that the image captured by the camera unit 21 is transmitted to the personnel belonging to the group other than the group 1 who will transmit the image (in Fig.16 The timing is when the person whose ID is "OB2" is detected (step S805).

[0228] On the other hand, if the control unit 28 of the sub-camera 20 causes the imaging unit 21 to start capturing an image including the subject person TR1 ( Fig. 20 : Step S407), then wait until a notification indication of the start time of image transmission is received from the main camera 40 (step S471 / no).

[0229] Then, if an instruction to notify the start time of image transmission is received from the main camera 40 (step S471 / Yes), the image captured by the camera unit 21 is notified to the personnel belonging to the group other than group 1 who will transmit the image (in Fig.16 For example, the control unit 28 causes the speaker 23 to emit a voice message such as "in 5 seconds, the image will be transmitted to the person belonging to group 2". In this way, the subject TR1 can know that the image captured by the camera unit 21 will be transmitted to the person belonging to group 2.

[0230] After this processing due to and Figure 8 The processing shown is the same, so the detailed description thereof is omitted.

[0231] On the other hand, the control unit 48 of the main camera 40 determines whether a transmission prohibition instruction ( Fig.19 :Step S807).

[0232] When receiving a transmission prohibition instruction (step S807 / yes), the control unit 48 stops timing the transmission standby time (step S813). In this way, it is possible to prevent the image captured by the camera unit 21 from being transmitted to people belonging to groups other than group 1. In addition, since the shooting of the camera unit 21 is stopped, the image transmission to people belonging to group 1 is also stopped. In addition, at this time, the control unit 48 can also send a message to the people belonging to each group, and the aforementioned message notifies that the judgment of step S13 is wrong, or that the notification object TR1 has no abnormality. After the processing of step S813, return to Fig.17 Step S11.

[0233] If the transmission prohibition instruction is not received (step S807 / No), the control unit 48 determines whether the transmission standby time has passed (step S809). Specifically, it determines whether the transmission standby time has passed since the transmission standby time was counted in step S803. If the transmission standby time has not passed (step S809 / No), the process returns to step S807.

[0234] When the transmission standby time has passed (step S809 / yes), the control unit 48 sends a continue shooting instruction to the sub-camera 20 (step S815), starts transmitting images to the portable terminals of the people belonging to group 2 (the second group), and notifies the portable terminals of the people belonging to group 3 that, for example, image transmission to group 2 has started (step S817). In this way, the people belonging to group 2 can confirm the status of the subject person TR1 through the image. In addition, starting to transmit images to group 2 means that the subject person is likely to be abnormal. Since the people belonging to group 3 can know through the notification that the subject person TR1 is likely to be abnormal, they can take measures such as visiting the subject person TR1.

[0235] Fig.21 is a sequence diagram showing an example of processing in the monitoring system 100 according to the third embodiment. Fig.21 In the example, at time t1, the control unit 48 determines that the subject person TR1 has an abnormality based on the data acquired from the sensor 10.

[0236] In this case, the control unit 48 starts counting the shooting standby time from time t1. When the shooting standby time T1 has passed at time t2, the camera unit 21 or 41 starts shooting an image including the target person TR1. In addition, the control unit 48 starts transmitting the image to the portable terminal of the person belonging to group 1 and starts counting the transmission standby time.

[0237] When the transmission standby time T2 has elapsed at time t3, the control unit 48 starts transmitting the image to the portable terminal of the person belonging to group 2, and sends a message to the portable terminal of the person belonging to group 3. In this way, when there are multiple persons monitoring the target person TR1, by changing the transmission timing of the image or the notification content according to the attributes of the target person TR1, it is possible to balance monitoring the target person TR1 and protecting the privacy of the target person TR1.

[0238] As described in detail above, according to the third embodiment, when the camera unit 21 or 41 starts shooting, the control unit 48 starts transmitting the image captured by the camera unit 21 or 41 to the people belonging to group 1, and when the transmission standby time has passed since the camera unit 21 or 41 started shooting, the control unit 48 starts transmitting the image captured by the camera unit 21 or 41 to the people belonging to group 2. In this way, for example, the close relatives of the subject person TR1 can know the status of the subject person TR1 through the image, and at the same time, since the image is transmitted to people other than the close relatives after the transmission standby time has passed, the privacy of the subject person TR1 can be protected.

[0239] (Hardware Configuration)

[0240] Fig. 22 (A) is a diagram showing the hardware configuration of the control unit 28. Fig. 22 As shown in (A), the control unit 48 includes a CPU 431, a ROM 432, a RAM 434, a storage unit 436, a network interface 437, etc. These components of the control unit 48 are connected to a bus 438. The functions of the control unit 48 are realized by the CPU 431 executing a program stored in the ROM 432 or the storage unit 436.

[0241] Fig. 22 (B) is a diagram showing the hardware configuration of the control unit 28. The control unit 28 includes a CPU 231, a ROM 232, a RAM 234, a storage unit 236, a network interface 237, and the like. These components of the control unit 28 are connected to a bus 238. The functions of the control unit 28 are realized by the CPU 231 executing a program stored in the ROM 232 or the storage unit 236.

[0242] In addition, in the first to third embodiments described above, an example in which the home-side system 150 includes one main camera 40 and one or more sub-cameras 20 is described, but the present invention is not limited thereto. Fig.23 1 is a diagram showing a configuration of a monitoring system 100A according to a modification example. Fig.23 As shown, in the home end system 150A, the main camera 40 is omitted, and the home end system 150A includes a sensor 10, a camera 80, a remote controller 30, and a control device 70 for controlling the home end system 150A as a whole. The camera 80 may also have the same structure as the slave camera 20 described above.

[0243] Fig.247 is a functional block diagram showing the configuration of a control device 70 in a modified example. The control device 70 includes a storage unit 74, a first communication module 75, a second communication module 76, a third communication module 77, and a control unit 78. Since the first communication module 75, the second communication module 76, and the third communication module 77 are respectively the same as the first communication module 45, the second communication module 46, and the third communication module 47, their detailed descriptions are omitted.

[0244] The storage unit 74 stores address information of a server required for communicating with the outside of the home-side system 150A (e.g., a service server SS), identification information of the camera 80, the operating status of the home-side system 150A in the past (e.g., the last month, etc.), and the detection results of the state of the subject TR1 by the sensor 10 and the camera 80 of the home-side system 150A in the past (e.g., the last month, etc.). In addition, the operating status of the home-side system 150A refers to events that occur in the home-side system 150A, such as system startup, system termination, and error occurrence.

[0245] The control unit 78 performs substantially the same processing as the control unit 48, but in a modified example, the control unit 78 does not need to Figure 5 In addition, there is no need to process steps S15 and S20. Figure 6 In addition, since the hardware structure of the control unit 78 is the same as that of the control unit 48, its detailed description is omitted.

[0246] In addition, in the home-end system 150, at least one of the main camera 40 and the sub-camera 20 can be provided in plural units. In addition, the sub-camera 20 can also be omitted in the home-end system 150. In this case, the number of main cameras can be one or more. In addition, the control unit of the main camera 40 and the control unit of the sub-camera can be separately provided from their respective cameras. In addition, the control units of the main camera 40 and the sub-camera 20 can be shared. In other words, one control unit can be used to control the main camera 40 and the sub-camera 20. In addition, in the case where the main camera 20 is omitted, one control unit can also be used to control one or more main cameras 40.

[0247] In the first to third embodiments, the sensor 10 may not be included in the monitoring system 100 (home end system 150). In this case, for example, an existing sensor installed at the residence of the subject TR1 may be connected to the monitoring system 100, thereby acquiring data related to the subject's state from the existing sensor.

[0248] In addition, in the first to third embodiments described above, the sensor 10 and the main camera 40 and the sub-camera 20 are separate entities, but the sensor 10 and the main camera 40 may be integrated, and the sensor 10 and the sub-camera 20 may also be integrated.

[0249] In addition, in the first to third embodiments, the sub-camera 20 and the main camera 40 are provided with filters that do not transmit visible light, and the control units 28 and 48 can control the sub-camera 20 and the main camera 40 to cover the filters when the subject TR1 cannot be photographed, and to expose the lenses when the subject TR1 can be photographed. In this way, the sub-camera 20 and the main camera 40 can be used as the sensor 10.

[0250] In addition, in the first to third embodiments described above, the sensor 10 and the sub-camera 20 may not be included in the monitoring system 100 (home-side system 150). In this case, for example, by connecting an existing sensor and an existing camera installed at the residence of the subject person TR1 to the home-side system, data related to the state of the subject person can be obtained from the existing sensor, and a visible light image can be obtained from the existing camera. In addition, in this case, the home-side system 150 has a cover plate covering the lens of the existing camera, and before the state of the subject person inferred from the data obtained by the existing sensor becomes a specified state, the control unit 48 can make the existing camera physically unable to shoot the subject person TR1 (the state where the cover plate covers the lens of the existing camera).

[0251] In the first to third embodiments, one of the sub-camera 20 and the main camera 40 may be a camera that cannot be normally set to a state in which the subject person TR1 cannot be photographed (for example, without the cover 422 ).

[0252] In addition, in the first to third embodiments described above, the main camera 40 may not include the imaging unit 21, the microphone 22, and the speaker 23. In this case, the main camera 40 functions as a control device for controlling the entire home-side system 150.

[0253] In addition, in the first to third embodiments described above, the state of the subject TR1 is estimated based on the data obtained by the infrared array sensor, but the present invention is not limited thereto. For example, when the sensor 10 is an infrared camera, a vector representing the action of the subject TR1 can be obtained by comparing the image frames of the infrared camera, and the state of the subject TR1 can be estimated based on the feature value of the vector. In addition, a thermal imager can be used as the sensor 10 instead of the infrared camera, or on the basis of using the infrared camera. For example, the state of the subject TR1 can be determined by combining the vector representing the action of the subject TR1 detected by the infrared camera with the change in body temperature. In addition, when the sensor 10 is an electric wave sensor, the measurable "heart rate", "respiratory rate" and "blood pressure" data can be compared with the usual data (data measured in the past). If they are roughly the same, it is determined that the state of the subject TR1 is normal; if the difference with the usual data is greater than a threshold, it is determined that an abnormality has occurred. In addition, when the sensor 10 is a depth sensor, the posture of the subject TR1 can be detected. When it is detected that it is a daily behavior (standing, walking, sitting, lying down), it is judged to be normal; when it is detected that it is an abnormal behavior (falling, standing still for a long time), it is judged that an abnormality has occurred. In addition, when the sensor 10 is a vibration sensor, it can be judged that an abnormality has occurred when a vibration exceeding a threshold is detected. In addition, when the sensor 10 is a sound sensor, when an impact sound exceeding a threshold is detected, it is judged that an abnormality has occurred. When the sensor 10 is a wearable sensor, the "heart rate", "respiratory rate" and "blood pressure" data that can be measured can be compared with the usual data (data measured in the past). If they are roughly the same, it is judged that the state of the subject TR1 is normal; if the difference with the usual data exceeds the threshold, it is judged that an abnormality has occurred. In addition, for example, a line sensor can also be used as the sensor 10.

[0254] In the first to third embodiments, the state of the subject TR1 may be estimated by combining different types of sensors 10. For example, a vibration sensor and a sound sensor may be combined to estimate the state of the subject TR1.

[0255] In addition, in the first to third embodiments described above, the state in which the subject TR1 is abnormal is described as the prescribed state, but the present invention is not limited thereto. For example, when the sensor 10 is a thermometer and a hygrometer, heat stroke is likely to occur when the measured temperature is above a prescribed value and the measured humidity is above a prescribed value. In this case, it is also possible to start timing the shooting standby time based on the state in which the subject TR1 may have heat stroke (the state in which the subject TR1 is estimated to be abnormal) instead of whether the subject TR1 has heat stroke (whether an abnormality occurs).

[0256] In the first to third embodiments, the input for prohibiting shooting or transmission is received via the microphone 42 of the main camera 40 or the microphone of the sub-camera 20, but the present invention is not limited thereto. For example, the remote controller 30 may also include a microphone, and the input for prohibiting shooting is received via the microphone.

[0257] Furthermore, in the first embodiment, the input for prohibiting (canceling) photographing or transmitting is not limited to a predetermined voice, and may be, for example, a predetermined gesture, an operation on a predetermined device (remote controller 30, smart phone), etc. When the input for prohibiting (canceling) photographing is a predetermined gesture, when the control unit 48 determines that the subject TR1 is abnormal, the main camera 40 or the sub-camera 20 can be set to a state where the subject TR1 can be photographed.

[0258] Furthermore, in the first to third embodiments described above, when the transmission of an image starts, for example, it may be set so that the image capture and the image transmission are terminated if a designated operation is performed through the remote controller 30. For example, when the subject TR1 who has fallen down recovers from the fallen state, the subject TR1 can instruct the main camera 40 to terminate the image capture and the image transmission through the remote controller 30. Furthermore, when the person OB1 who has received a message or confirmed the transmitted image of the monitored subject TR1 visits the residence H1 of the subject TR1, the monitor OB1 can instruct the main camera 40 to terminate the image capture and the image transmission through the remote controller 30. Furthermore, an instruction to terminate the image transmission may be sent from the portable terminal MT1 of the monitor OB1.

[0259] Furthermore, in the first to third embodiments described above, a notification of "transmission in progress" may be continuously given from a speaker or the like during transmission of a video.

[0260] In addition, in the first to third embodiments, when the subject TR1 is sleeping, the control unit 48 and the control unit 28 may not perform the above-mentioned processing. For example, when the subject TR1 lies on the bed and sleeps at the designated time, the control unit 48 and the control unit 28 may not perform the above-mentioned processing. Whether the subject TR1 lies on the bed and sleeps, for example, can be detected by a weight sensor provided on the bed. In this way, it is possible to prevent the subject TR1's sleep from being misjudged as abnormal.

[0261] Furthermore, in the first to third embodiments, when a predetermined voice such as "Help" is input during the shooting standby time, shooting may be started without waiting for the shooting standby time to end, and video transmission may be started.

[0262] Furthermore, in the first to third embodiments described above, part or all of the processing executed by the control unit 28 may also be executed by the control unit 48. Furthermore, the second embodiment and the third embodiment may also be combined according to actual circumstances.

[0263] In addition, the above-mentioned processing functions can be realized by a computer. In this case, a program is provided that describes the processing content of the processing device (CPU). By executing the program on a computer, the above-mentioned processing functions can be realized on the computer. The program describing the processing content can be recorded in a computer-readable recording medium (but not including a carrier).

[0264] When the program is distributed, it may be sold in the form of a removable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. Alternatively, the program may be stored in a storage device of a server computer and transferred from the server computer to other computers via a network.

[0265] The computer executing the program stores the program recorded in the removable recording medium or the program transmitted from the server computer in its own storage device, for example. Then, the computer reads the program from its own storage device and executes the processing in accordance with the program. In addition, the computer can also directly read the program from the removable recording medium and execute the processing in accordance with the program. In addition, the computer can also sequentially execute the processing in accordance with the received program each time the program is transmitted from the server computer.

[0266] The above-mentioned embodiments are preferred embodiments of the present invention and can be appropriately combined. However, the present invention is not limited thereto and can be variously modified without departing from the gist of the present invention.

[0267] Description of Reference Numerals

[0268] 10 Sensors

[0269] 20 sub-cameras

[0270] 21 Camera Department

[0271] 28 Control Department

[0272] 30 Remote Control

[0273] 40 Main camera

[0274] 41 Camera Department

[0275] 48 Control Department

[0276] 100 monitoring system

[0277] 150 Home Terminal System

[0278] MT1 portable terminal.

Claims

1. A program that causes a computer to execute the following processing: When the state of the subject person estimated based on the data acquired by the acquisition unit that acquires data related to the state of the subject person is a predetermined state, causing a camera unit that captures an image including the subject person to start capturing the image; and The state of the subject person is detected based on the image captured by the imaging unit. 2 . The program according to claim 1 , causing the computer to execute a process of continuing to capture the image when the state of the subject person detected from the image captured by the imaging unit is the predetermined state. 3 . The program according to claim 2 , causing the computer to execute a process of notifying the subject person of information related to the photographing of the video when the photographing of the video continues. 4 . The program according to claim 1 , causing the computer to execute a process of starting transmission of the image captured by the imaging unit when the state of the subject person detected based on the image is the predetermined state. 5 . The program according to claim 1 , causing the computer to execute a process of storing an image used for detecting the state of the subject person when the state of the subject person detected based on the image is a predetermined state. 6 . The program according to claim 1 , wherein the imaging unit starts capturing the video when a first predetermined time has elapsed since the state of the subject person estimated based on the data becomes the predetermined state. 7 . The program according to claim 4 , causing the computer to execute a process of starting transmission of the image when a second predetermined time has elapsed since the state of the subject person was detected to be the predetermined state based on the image.

8. The image according to claim 7 causes the computer to perform the following processing: Prior to the process of transmitting the image, the subject person is notified of information related to the transmission of the image. 9 . The program according to claim 1 , causing the computer to execute a process of stopping capturing of the image when the state of the subject person detected based on the image is not the predetermined state. 10 . The program according to claim 1 , causing the computer to execute a process of transmitting a predetermined notification to an external device when the state of the subject person detected based on the image is not the predetermined state.

11. The program according to claim 9, causing the computer to execute processing for deleting an image used for detecting the state of the subject person when the state of the subject person detected based on the image captured by the imaging unit is not the predetermined state.

12. The program according to any one of claims 1 to 3, The data related to the state of the subject is data other than the visible light image.

13. A monitoring system comprising: an acquisition unit that acquires data related to the state of the subject; Camera Department; and The control unit causes the imaging unit to start capturing an image including the subject person when the state of the subject person is a predetermined state, and detects the state of the subject person based on the image captured by the imaging unit.

14. A control device comprising a control unit, When the state of the subject person inferred from the data acquired by the acquisition unit for acquiring data related to the state of the subject person is a specified state, the control unit causes the camera unit that captures an image including the subject person to start capturing the image, and detects the state of the subject person based on the image captured by the camera unit.

Images