Surveillance system, surveillance method, and surveillance program

Abstract

To appropriately control a camera even when a location of a mobile terminal user changes or when a monitored object is different from the mobile terminal user.SOLUTION: A monitoring system 1000 includes a first mobile terminal, a second mobile terminal, and a management device 1200 that communicates with the first mobile terminal and the second mobile terminal. The first mobile terminal has a first transmitting unit that transmits location information of the first mobile terminal to the management device 1200. The management device 1200 has a control unit that predicts a future location of the first mobile terminal from the location information, controls orientation of a camera so that the first mobile terminal is imaged on the basis of the future location, extracts video acquired from the camera, and transmits the video to the second mobile terminal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a monitoring system, a monitoring method, and a monitoring program. [Background technology]

[0002] In the above technical field, Patent Document 1 discloses a technology in which, upon detecting an abnormal state of a user using a mobile communications terminal, the user's location information is transmitted to a surveillance camera, and an image including the user captured by the surveillance camera is transmitted to a management center. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-328333 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned Patent Document 1, the user of the mobile communication terminal is the subject of monitoring, and the situation and state of the user can be detected. However, the above-mentioned Patent Document 1 is based on the premise that the user of the mobile communication terminal does not move. Therefore, when the user of the mobile communication terminal moves, even if the camera is controlled to a position obtained from the mobile communication terminal and an image is taken, there is a problem that the user may not be included in the captured image.

[0005] An object of the present invention is to provide a monitoring system, a monitoring method, and a monitoring program for solving the above problems. [Means for solving the problem]

[0006] A surveillance system according to the present invention includes a first mobile terminal, a second mobile terminal, and a management device that communicates with the first mobile terminal and the second mobile terminal. The first mobile terminal has a first transmission unit that transmits location information of the first mobile terminal to the management device. The management device has a control unit that predicts a future location of the first mobile terminal from the location information, controls a camera orientation so that the first mobile terminal is imaged based on the future location, extracts an image acquired from the camera, and transmits the image to the second mobile terminal. When the first mobile terminal detects a predetermined motion of a person carrying the first mobile terminal, the first mobile terminal transmits to the management device position information of the first mobile terminal and control switching information for switching camera control of the camera. The management device has a normal mode for monitoring a predetermined range and a tracking mode for monitoring based on the position information. When the control switching information is received, the control unit switches from the normal mode to the tracking mode and controls the orientation of the camera based on the future position of the first mobile terminal. It is characterized by:

[0007] In a monitoring method according to the present invention, a first mobile terminal transmits location information of the first mobile terminal to a management device, and the management device predicts a future location of the first mobile terminal from the location information, controls a camera orientation so that the first mobile terminal is imaged based on the future location, extracts an image acquired by the camera, and transmits the image to a second mobile terminal. When the first mobile terminal detects a predetermined motion of a person carrying the first mobile terminal, the first mobile terminal transmits to the management device position information of the first mobile terminal and control switching information for switching camera control of the camera, and when the management device receives the control switching information, the management device switches from a normal mode for monitoring a predetermined range to a tracking mode for monitoring based on the position information, and controls the orientation of the camera based on the future position of the first mobile terminal. It is characterized by:

[0008] A monitoring program according to the present invention includes a process for predicting a future location of a first mobile terminal from location information of the first mobile terminal transmitted by the first mobile terminal, controlling a camera orientation so that an image of the first mobile terminal is captured based on the future location, extracting an image captured by the camera, and transmitting the image to the second mobile terminal, in a computer provided in a management device that communicates with a first mobile terminal and a second mobile terminal. a process of receiving position information of the first mobile terminal and control switching information for switching camera control of the camera, the control switching information being transmitted when the first mobile terminal detects a predetermined motion of a person carrying the first mobile terminal, and a process of switching from a normal mode in which a predetermined range is monitored to a tracking mode in which monitoring is performed based on the position information, when the control switching information is received, and controlling the orientation of the camera based on the future position of the first mobile terminal. The present invention is characterized in that the above-mentioned is executed. Effect of the Invention

[0009] According to the present invention, even when the position of the terminal user changes or when the monitored subject is different from the terminal user, the camera can be appropriately controlled. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a configuration of a monitoring system according to a first embodiment. [Diagram 2] FIG. 4 is a flow diagram showing information transmission of a mobile terminal according to the first embodiment. [Diagram 3]FIG. 4 is a flow diagram showing camera control of the management device according to the first embodiment. [Figure 4] 4 is a flow diagram showing how the management device according to the first embodiment tracks a tracking target. [Diagram 5] 1 is a bird's-eye view showing a specific example of an operation performed by a security guard in embodiment 1 when he discovers a suspicious person. [Figure 6] 4 is an overhead view showing a specific example of camera control of the management device according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing a specific example of an image captured by camera 300-1 in FIG. [Figure 8] FIG. 7 is a diagram showing a specific example of an image captured by camera 300-2 in FIG. [Figure 9] FIG. 7 is a diagram showing a specific example of an image captured by camera 300-3 in FIG. [Figure 10] 4 is a diagram showing a specific example of a display unit of the management device according to the first embodiment. FIG. [Figure 11] 13 is a diagram showing a modified example of the display unit of the management device according to the first embodiment. FIG. [Figure 12] FIG. 11 is a block diagram illustrating a configuration of a monitoring system according to a second embodiment. [Figure 13] FIG. 11 is a flow diagram showing information transmission of a mobile terminal according to the second embodiment. [Figure 14] FIG. 11 is a flow diagram showing camera control of the management device according to the second embodiment. [Figure 15] 11 is a flow diagram showing how a management device according to a second embodiment tracks a tracking target. [Figure 16] 13 is a bird's-eye view showing a specific example of camera control of the management device according to the second embodiment. [Figure 17] FIG. 11 is a diagram showing a specific example of a display unit of a management device according to a second embodiment. [Figure 18] FIG. 11 is a flow diagram showing information transmission of a mobile terminal according to a third embodiment. [Figure 19] 13 is a flow diagram showing how a management device according to a third embodiment tracks a tracking target. [Figure 20] 1 is a diagram illustrating a hardware configuration of a monitoring system according to a first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Embodiment 1] (Features of the invention) In this embodiment, in a surveillance system, a surveillance camera is controlled to capture the location of a mobile terminal based on the location information of the mobile terminal held by a security guard, and tracking candidates extracted from the video captured by the surveillance camera are displayed to an operator of a management device. The system is characterized in that the operator of the management device is prompted to select a tracking candidate from the displayed candidates. (composition) FIG. 1 is a block diagram illustrating the configuration of a monitoring system 1000 according to this embodiment. In this embodiment, the monitoring system 1000 is a system for monitoring facilities (important facilities such as ports, airports, platforms, power plants, plants and dams, warehouse districts, leisure facilities, stadiums, commercial facilities, buildings, cities, streets, etc.). More specifically, an embodiment for monitoring a facility in which security guards perform patrol monitoring and camera monitoring will be described. Referring to FIG. 1, the monitoring system 1000 is composed of a mobile terminal 1100, a management device 1200, and cameras 300-1 to 300-N.

[0012] The mobile terminal 1100 includes a position acquisition unit 1110, a motion detection unit 1120, and a communication unit 1130. Examples of the mobile terminal 1100 include a wireless device, a mobile phone, a smartphone, and a tablet terminal. The position acquisition unit 1110, the motion detection unit 1120, and the like can be realized by making a computer constituting these devices execute a computer program that executes each process using the hardware. That is, as shown in FIG. 20, a CPU (Central Processing Unit) 100 included in the mobile terminal 1100 may be connected to a memory 120 included in the mobile terminal 1100, and the CPU 100 may function based on a predetermined program or various data read from the memory 120, thereby realizing the position acquisition unit 1110, the motion detection unit 1120, and the like. More specific examples of the CPU 100 include a processor and a microprocessor. Furthermore, the CPU 100 of the mobile terminal 1100 described above may be connected to the communication unit 1130, the input reception unit 110, and the display unit 130 via an input / output circuit. Here, the memory 120 may be configured as a RAM (Random Access Memory) or a ROM (Read Only Memory).

[0013] The position acquisition unit 1110 is connected to the motion detection unit 1120 and the communication unit 1130. The position acquisition unit 1110 acquires real world coordinates, which are position information, from a GPS (Global Positioning System) satellite. When the position acquisition unit 1110 acquires the position information of the mobile terminal 1100, the position acquisition unit 1110 transmits the position information of the mobile terminal 1100 to the communication unit 1130. In this embodiment, the position information is acquired using a GPS satellite, but it is also possible to use a system that transmits and receives data using wireless communication and calculate the position information from information on the access location. Here, the position information is set to real world coordinates, which are unified coordinates in the real world, but it is not necessary to be limited to this information alone, and it is also possible to configure the information by adding information such as the current time. Here, the real world coordinates may be configured by information indicating longitude and latitude.

[0014] The motion detection unit 1120 is connected to the position acquisition unit 1110 and the communication unit 1130. The motion detection unit 1120 detects a predetermined motion of the security guard. When the motion detection unit 1120 detects a predetermined motion of the security guard, it sends control switching information requesting a mode switch of the camera control of each of the cameras 300-1 to 300-N to the communication unit 1130. Here, the predetermined motion is the operation of the mobile terminal 1100 by the security guard. More specifically, when the security guard is patrolling and finds a suspicious person or a suspicious object, the security guard operates the mobile terminal and sends control switching information requesting a mode switch of the camera control of each of the cameras 300-1 to 300-N to the communication unit 1130. At this time, the security guard may take an image of the suspicious person or the suspicious object with the camera (not shown) of the mobile terminal 1100, and the taken image may be sent to the communication unit 1130 together with the position information of the mobile terminal 1100. The camera control modes will be described later.

[0015] The communication unit 1130 includes a first communication unit 1131 and a second communication unit 1132. The communication unit 1130 communicates with the management device 1200 via a communication network.

[0016] The first communication unit 1131 is connected to the position acquisition unit 1110. The first communication unit 1131 transmits the position information of the mobile terminal 1100 acquired by the position acquisition unit 1110 to the management device 1200 via the communication network. It is also possible to receive information from the management device 1200. The first communication unit 1131 may be configured to transmit an image of a suspicious person, suspicious object, etc., captured by a camera (not shown) of the mobile terminal 1100 to the management device 1200 together with the position information of the mobile terminal 1100.

[0017] The second communication unit 1132 is connected to the motion detection unit 1120. When the motion detection unit 1120 detects a predetermined motion of a security guard, the second communication unit 1132 transmits control switching information to the management device 1200 via a communication network. It is also possible to receive information from the management device 1200. Here, the first communication unit 1131 and the second communication unit 1132 are functionally separate and only need to be able to transmit and receive independently. In other words, they do not need to be separate in terms of hardware configuration, and can be implemented in a single piece of hardware. The second communication unit 1132 may also be configured to transmit the control switching information together with an image of a suspicious person, suspicious object, etc. captured by a camera (not shown) of the mobile terminal 1100.

[0018] The management device 1200 includes a communication unit 1210, a control unit 1220, and a display unit 1230. The management device 1200 is installed in a building where a security company that monitors the entire facility is located, or in a security room installed in the facility to be monitored. The management device 1200 can also be realized even when installed outside the facility as a management server. Furthermore, it can exist not only as an independent device, but also be incorporated as one of the components of a camera 300-1 described later. Here, it can also be realized by a computer program that causes a computer constituting these devices to execute each process using the hardware. That is, as shown in FIG. 20, the CPU 200 provided in the management device 1200 is connected to a memory 220 provided in the management device 1200, and the CPU 200 functions based on a predetermined program or various data read from the memory 220, thereby realizing the control unit 1220. More specific examples of the CPU 200 include a processor and a microprocessor. Furthermore, the CPU 200 of the management device 1200 described above may be connected to the communication unit 1210, the input reception unit 210, and the display unit 1230 via an input / output circuit. Furthermore, the connection unit 230 of the management device 1200 is connected to each of the cameras 300-1 to 300-N. Here, the memory 220 may be configured as a RAM or a ROM.

[0019] The communication unit 1210 includes a first communication unit 1211 and a second communication unit 1212. The communication unit 1210 can transmit and receive information to and from the communication unit 1130 of the mobile terminal 1100. In addition, the communication unit 1210 sends the received information to the control unit 1220.

[0020] The first communication unit 1211 communicates with the first communication unit 1131 of the mobile terminal 1100 and receives the above-mentioned location information of the mobile terminal 1100. The first communication unit 1211 sends the received location information of the mobile terminal 1100 to the control unit 1220. The first communication unit 1211 can also transmit information to the first communication unit 1131. The first communication unit 1211 may also be configured to acquire the location information of the mobile terminal 1100 together with an image of a suspicious person, suspicious object, etc. captured by a camera (not shown) of the mobile terminal 1100.

[0021] The second communication unit 1212 communicates with the second communication unit 1132 of the mobile terminal 1100 via a communication network and receives the above-mentioned control switching information. The second communication unit 1212 sends the control switching information to the control unit 1220. The second communication unit 1212 can also transmit information to the second communication unit 1132. Here, the first communication unit 1211 and the second communication unit 1212 are functionally separate and can transmit and receive information independently. In other words, the first communication unit 1211 and the second communication unit 1212 do not need to be separate in terms of hardware configuration, and can be implemented in one hardware. In addition, the second communication unit 1212 may be configured to acquire the control switching information together with an image of a suspicious person, suspicious object, etc. captured by a camera (not shown) of the mobile terminal 1100.

[0022] The control unit 1220 includes a tracking unit 1221 and a storage unit 1222. In addition, the control unit 1220 is connected to the communication unit 1210, the display unit 1230, and each of the cameras 300-1 to 300-N. The control unit 1220 controls the camera based on the control switching information acquired from the communication unit 1210. More specifically, the control unit 1220 switches the camera control from the normal mode to the tracking mode when the control switching information arrives. Here, in the normal mode, a monitoring area is set for each of the cameras 300-1 to 300-N, and the control unit 1220 controls the cameras 300-1 to 300-N to capture the monitoring area set therein. On the other hand, in the tracking mode, the control unit 1220 controls the capture range of each of the cameras 300-1 to 300-N to a position indicated by the acquired position information of the mobile terminal 1100, and tracks the tracking target. That is, the control unit 1220 controls the PTZ (Pan-Tilt-Zoom) of the cameras 300-1 to 300-N so that the cameras 300-1 to 300-N can capture the position indicated by the position information of the mobile terminal 1100. Furthermore, the control unit 1220 receives the images captured by the cameras 300-1 to 300-N and sends them to the display unit 1230. Here, tracking refers to judging whether or not an object detected in a certain image frame, for example, the previous image frame, n-1 frame (n is a natural number), can be matched with an object detected in the current image frame (n frame) as the same object, and measuring the movement status of the object considered to be the same. More specifically, it refers to controlling the pan, tilt, and zoom of the camera so that the tracked object is included in the image obtained by the camera 300-1, etc.

[0023] When the tracking unit 1221 is in the tracking mode, it selects a tracking candidate based on the images captured by the cameras 300-1 to 300-N. Furthermore, the tracking unit 1221 starts tracking a tracking target designated by the operator of the management device 1200 from among the tracking candidates. A detailed tracking flow will be described later.

[0024] The storage unit 1222 is a storage medium such as a hard disk drive (HDD). The storage unit 1222 accumulates images captured by the cameras 300-1 to 300-N, location information of the cameras 300-1 to 300-N, the traveling direction of the mobile terminal 1100, the acceleration direction of the mobile terminal 1100, control switching information, location information of the mobile terminal 1100, features of specific objects (people, animals, small airplanes, helicopters, etc.), features of the security guards, their clothes, etc.

[0025] The display unit 1230 includes a liquid crystal display device. The display unit 1230 displays the camera image obtained by the control unit 1220. The display unit 1230 also displays information on tracking candidates that are candidates for tracking targets, and displays information to prompt the operator to select a tracking candidate to be the tracking target.

[0026] Each of the cameras 300-1 to 300-N is a CMOS (Complementary Metal Oxide Semiconductor) camera. In addition, each of the cameras 300-1 to 300-N can change the shooting range by PTZ (Pan-Tilt-Zoom) control. Here, although a CMOS camera is used in this embodiment, a CCD (Charge Coupled Devices) camera or an infrared camera may be used, and any camera may be used as long as it is an image acquisition means that can acquire an image from which a feature can be extracted. In addition, each of the cameras 300-1 to 300-N does not need to be the same camera, and various cameras can be adopted. In addition, in this embodiment, an example in which the camera 300-1 and the management device 1200 are configured as separate devices has been shown, but the monitoring system 1000 may be configured to adopt an intelligent camera in which the camera 300-1 and the management device 1200 are integrated. Furthermore, the surveillance system 1000 may be configured such that the management device 1200 is embedded in each of the cameras 300-1 to 300-N, or such that the management device 1200 is built into only the camera 300-1, and the management device 1200 also controls the other cameras. (Mobile terminal control) Next, a control flow in the mobile terminal 1100 will be described with reference to Fig. 2. Fig. 2 is a flow diagram in which the mobile terminal 1100 transmits information to the management device 1200.

[0027] In step S110, the location acquisition unit 1110 acquires location information of the own device from a GPS satellite. The acquired location information is sent to the first communication unit 1131. Here, the location acquisition unit 1110 is configured to acquire location information at a predetermined interval (for example, every 10 seconds). Here, a variable cycle or a random cycle may be adopted as the predetermined interval. As another example of the predetermined interval, the location acquisition unit 1110 may be configured to acquire location information when a request for the mobile terminal 1100 is made from a security guard. In this case, the predetermined interval is set to the time when a request for the mobile terminal 1100 is made from a security guard.

[0028] In step S120, the motion detection unit 1120 receives control switching information, which is a request to switch the monitoring control (more specifically, camera control) from the normal mode to the tracking mode, from the security guard. The motion detection unit 1120 transmits the acquired control switching information to the second communication unit 1132. Here, in steps S110 and S120, the position acquisition unit 1110 and the motion detection unit 1120 are configured to acquire the position information and the control switching information of the mobile terminal 1100 separately and independently in order. The position acquisition unit 1110 and the motion detection unit 1120 can also be configured to acquire them together and simultaneously. Also, the motion detection unit 1120 may be configured to acquire the control switching information together with an image of a suspicious person, suspicious object, etc. captured by a camera (not shown) of the mobile terminal 1100.

[0029] In step S130, the first communication unit 1131 transmits the acquired location information of the mobile terminal 1100 to the management device 1200 via the first communication unit 1211. Furthermore, the second communication unit 1132 transmits the acquired control switching information to the management device 1200 via the second communication unit 1212. Here, the first communication unit 1131 is configured to transmit information to the management device 1200 every time the location acquisition unit 1110 acquires the location information of the mobile terminal 1100 at a predetermined interval (for example, periodically). That is, the first communication unit 1131 transmits the location information of the mobile terminal 1100 to the management device 1200 at a predetermined interval (for example, periodically). Furthermore, the first communication unit 1131 or the second communication unit 1132 may be configured to transmit an image of a suspicious person, a suspicious object, or the like captured by a camera (not shown) of the mobile terminal 1100. Here, the predetermined interval at which position acquisition section 1110 acquires the position information and the predetermined interval at which first communication section 1131 transmits the position information may be synchronized or asynchronous. (Camera control by management device) Next, a flow of control by management device 1200 for cameras 300-1 to 300-N will be described with reference to Fig. 3. Fig. 3 is a control flow diagram showing how management device 1200 controls the shooting ranges of cameras 300-1 to 300-N.

[0030] In step S210, the first communication unit 1211 receives location information of the mobile terminal 1100 from the mobile terminal 1100, and sends the acquired information to the control unit 1220. Furthermore, the second communication unit 1212 receives control switching information from the mobile terminal 1100, and sends the acquired information to the control unit 1220. Furthermore, the first communication unit 1211 or the second communication unit 1212 may be configured to also acquire images of a suspicious person, suspicious object, etc. captured by a camera (not shown) of the mobile terminal 1100.

[0031] In step S220, the control unit 1220 switches the camera control mode from the normal mode to the tracking mode based on the control switching information. The control unit 1220 also controls the pan, tilt, and zoom of the cameras 300-1 to 300-N based on the acquired position information of the mobile terminal 1100. A more detailed algorithm is shown below. First, the storage unit 1222 stores in advance the positions at which the cameras can capture images. The control unit 1220 extracts combinations of pan, tilt, and zoom in which the real world coordinates, which are the acquired position information, are within the captureable range from among the combinations of pan, tilt, and zoom stored in the storage unit 1222. Then, the control unit 1220 controls the pan, tilt, and zoom of the camera 300-1 by setting the combination closest to the current combination as a target value from among the extracted combinations of pan, tilt, and zoom. Here, the configuration is such that the combination closest to the current combination is selected, but the control unit 1220 may select a combination in which the real world coordinates, which are the acquired position information, are in the center of the camera.

[0032] Furthermore, the control unit 1220 can detect the moving direction of the security guard and perform camera control by predicting the current position of the security guard. Specifically, the control unit 1220 periodically receives the position information and transmission time information of the mobile terminal 1100. Thereafter, the control unit 1220 calculates the moving direction and moving speed of the security guard from the currently received position information and transmission time of the mobile terminal 1100 and the previously received position information and transmission time of the mobile terminal 1100. Next, the control unit 1220 calculates the expected movement amount based on the difference between the time when the current position information was received and the current time, and the moving speed. The control unit 1220 predicts the position where the mobile terminal 1100 will be located at the current time based on the calculated moving direction and expected movement amount of the security guard. Finally, the control unit 1220 controls the pan, tilt, and zoom of each of the cameras 300-1 to 300-N in the above-mentioned manner so that the predicted position information is included in the shooting range.

[0033] Furthermore, the control unit 1220 can also detect the moving direction of the security guard and control the camera by predicting the future position of the security guard. Specifically, the control unit 1220 predicts the future position of the security guard by carrying out the same process as the above-mentioned means for predicting the current position for a future time.

[0034] In step S230, control unit 1220 transmits the control amount calculated in step S220 to cameras 300-1 to 300-N. The flow then ends. Here, control unit 1220 may be configured to successively update the camera positions of cameras 300-1 to 300-N based on periodically acquired position information of the mobile terminal. With this configuration, even if the security guard carrying mobile terminal 1100 has moved away from the previously acquired position, the camera's shooting range is updated, thereby improving the possibility that each of cameras 300-1 to 300-N will capture the security guard. (Tracking control by management device) Next, a flow of tracking control by the management device 1200 will be described with reference to Fig. 4. Fig. 4 is a flow diagram of tracking control in which the management device 1200 performs tracking.

[0035] In step S310, when the control unit 1220 receives a mode change request, it changes the camera control from the normal mode to the tracking mode.

[0036] In step S320, the tracking unit 1221 acquires the images captured by the cameras 300-1 to 300-N. The tracking unit 1221 stores in the storage unit 1222 the images acquired from the cameras 300-1 to 300-N.

[0037] In step S330, the tracking unit 1221 extracts features from the images acquired from the cameras 300-1 to 300-N. The tracking unit 1221 compares the extracted features with features for identifying people that are stored in advance in the storage unit 1222 to extract candidates for people. The tracking unit 1221 selects the extracted candidates for people as tracking candidates. Here, the tracking unit 1221 may obtain a motion vector indicating the amount of movement and the direction of movement of a moving object, and select only the moving object as a tracking candidate. More specifically, the tracking unit 1221 obtains the motion vector by reading out a plurality of frames and comparing image data between frames. The tracking unit 1221 can also be configured to extract features from an image of a suspicious person captured by a camera (not shown) of the mobile terminal 1100 acquired by the first communication unit 1211 or the second communication unit 1212, and to extract tracking candidates by comparing the extracted features with features extracted from the video acquired from each of the cameras 300-1 to 300-N. In addition, the tracking control of the management device 1200 is configured to extract people as tracking candidates, but is not limited to people, and may also be animals, moving objects (cars, motorcycles), and flying objects (airplanes, helicopters).

[0038] In step S340, the tracking unit 1221 outputs the images of the cameras 300-1 to 300-N captured in step S320 and the tracking candidates extracted in step S330 to the display unit 1230. The display unit 1230 displays the images of the cameras 300-1 to 300-N and highlights the tracking candidates extracted in step S330 in the images.

[0039] In step S350, the display unit 1230 accepts designation of one or more tracking targets by the operator. Here, the display unit 1230 accepts the designation of the operator, but this is not particularly limited, and the control unit 1220 may be configured to accept the designation of the operator.

[0040] In step S360, the tracking unit 1221 tracks the target based on the designation of the target. A specific detection algorithm for the target is shown below. The tracking unit 1221 predicts the position of the target on the current frame based on the position of the target detected in the previous frame. As this method, various existing methods such as a method using a Kalman filter or a particle filter can be used. Then, the tracking unit 1221 performs tracking based on the predicted position of the target on the current frame and the image of the current frame. Note that the tracking is not limited to the above-mentioned form, and tracking can also be performed by associating the target with the similarity or likelihood of features between the target and an object in the current frame. In addition, the control unit 1220 may be configured to transmit the images captured by each of the cameras 300-1 to 300-N to the mobile terminal 1100 via the communication unit 1210. With the above-mentioned configuration, even if the security guard loses sight of the suspicious person, it is possible to start tracking again. Also, when there are multiple security guards and multiple mobile terminals, the control unit 1220 can be configured to transmit video to a mobile terminal other than the mobile terminal that acquired the location information. Furthermore, the control unit 1220 can be configured to analyze the video captured by each of the cameras 300-1 to 300-N while tracking the tracking target with the tracking unit 1221, generate metadata related to the tracking target, or associate it with the video and store it in the storage unit 1222. Here, the metadata refers to color data of the tracking target's clothes, facial features, etc. (Specific example of embodiment 1) Next, a specific example of the first embodiment will be described with reference to Fig. 5 to Fig. 10. Fig. 5 is an overhead view showing a specific example of the operation when a security guard in the first embodiment finds a suspicious person.

[0041] As a specific example, a certain facility has buildings A, B, and C. Building A is equipped with cameras 300-1 and 300-2. Building B is equipped with camera 300-3. A suspicious person is present in the center of FIG. 5. A security guard is present in the lower right of FIG. 5. When the security guard discovers a suspicious person, the security guard executes an input operation to request a mode change in mobile terminal 1100. Mobile terminal 1100 that receives the input transmits control switching information together with location information of mobile terminal 1100 to management device 1200. Thereafter, the security guard starts moving in the direction of the suspicious person to capture the suspicious person. Even while the suspicious person is moving, mobile terminal 1100 successively transmits the acquired location information to management device 1200.

[0042] Fig. 6 shows an overhead view of camera control of the management device according to the first embodiment. In Fig. 6, when the management device 1200 acquires control switching information from the mobile terminal 1100, the management device 1200 switches the mode for controlling the cameras 300-1 to 300-3 to the tracking mode. Furthermore, based on the acquired position information of the mobile terminal 1100, the management device 1200 directs the line of sight of the cameras 300-1 to 300-3 to the position acquired from the mobile terminal 1100 (where a security guard is likely to be present).

[0043] FIG. 7 is a diagram showing an example of an image captured by camera 300-1. Camera 300-1 captures a security guard and a suspicious person at a long distance. FIG. 8 is a diagram showing an example of an image captured by camera 300-2. Camera 300-2 captures a security guard and a suspicious person at a medium distance. FIG. 9 is a diagram showing an example of an image captured by camera 300-3. Camera 300-3 captures a security guard at a close distance, but only the upper body of the suspicious person. Since control unit 1220 controls the focus target of camera 300-3 to the position of mobile terminal 1100, camera 300-3 cannot capture the suspicious person. The images captured by each of cameras 300-1 to 300-3 are sent to management device 1200 and stored in storage unit 1222.

[0044] FIG. 10 is a diagram showing a specific example of the display unit of the management device according to the first embodiment. The control unit 1220 displays the images captured by the cameras 300-1 to 300-3 on the display unit 1230. The control unit 1220 also extracts tracking candidates 1 to 5 captured in the images of the cameras 300-1 to 300-3 by displaying them in a surrounding manner. The control unit 1220 also highlights the tracking candidates 1 to 5 in the images of the cameras 300-1 to 300-3. In addition, one or more tracking candidates can be selected so that a tracking target can be determined from tracking candidates 1 to 5. In this specific example, the management device operator selects tracking candidate 1 and tracking candidate 3 from the displayed tracking candidates 1 to 5 and presses the determination button to determine tracking candidate 1 and tracking candidate 3 as tracking targets. The tracking unit 1221 determines the determined tracking target as a tracking target and performs tracking on the determined tracking target. More specifically, the control unit 1220 controls the camera 300-1 so that tracking target 1 is captured near the center of the screen. Furthermore, the control unit 1220 controls the camera 300-2 so that the tracking target 3 is photographed near the center of the screen.

[0045] Further, a modified example is shown in FIG. 11. In FIG. 10, the operator needs to execute a selection step and a decision step, and two operations are required. Here, as shown in FIG. 11, a configuration may be adopted in which the tracking candidates highlighted in the images acquired by the cameras 300-1 to 300-3 can be directly selected. In more detail, "tracking target 1" in FIG. 11 is gray, indicating a state in which it is selected as a tracking target. Also, "tracking target 2" is white, indicating a state in which it is not selected as a target. When the operator presses "tracking target 1" in FIG. 11, it turns gray as shown in FIG. 11, and tracking starts. Furthermore, when the operator presses "tracking target 1" in FIG. 11 again, it turns white, and tracking stops. By adopting the above-mentioned configuration, the operator's visibility and operability can be improved. (Action and effect) Since the position information of the mobile terminal can be acquired successively, the possibility that the camera can capture the security guard or the suspicious person is improved even if the security guard is moving. Also, by controlling the camera based on the position information of the mobile terminal and displaying tracking candidates in a selectable manner, it is possible to select a tracking target other than the security guard who is the user of the mobile terminal. Furthermore, by the mobile terminal sending an image of the tracking target such as a suspicious person that it has taken to the management device in conjunction with the transmission of the position information, the reliability of the extraction of tracking candidates in the management device is increased, and the tracking candidates can be narrowed down. [Embodiment 2] In this embodiment, the monitoring system uses an acceleration sensor of a mobile terminal to estimate whether a security guard is being tracked, and when it is determined that the security guard is tracking, the mobile terminal transmits control switching information to a management device. Also, one of multiple monitoring cameras is selected to be controlled. Furthermore, tracking candidates are extracted, and tracking is started with the security guard as a provisional tracking target before the management device operator selects the tracking target. (composition) Fig. 12 is a block diagram illustrating the configuration of a monitoring system 2000 according to this embodiment. Referring to Fig. 12, the monitoring system 2000 includes a mobile terminal 2100, a management device (management server) 2200, and cameras 300-1 to 300-N.

[0046] The mobile terminal 2100 includes at least a position acquisition unit 1110, a motion detection unit 2120, a communication unit 1130, and an acceleration sensor 2140. The mobile terminal 2100 includes all of the hardware configuration of the mobile terminal 1100 of the first embodiment shown in FIG.

[0047] The position acquisition unit 1110 and the communication unit 1130 are the same as those in the first embodiment.

[0048] The motion detection unit 2120 is connected to the position acquisition unit 1110, the communication unit 1130, and the acceleration sensor 2140. The motion detection unit 2120 detects a predetermined motion of the security guard based on the output value of the acceleration sensor 2140. When the motion detection unit 2120 detects a predetermined motion of the security guard, it sends control switching information to the communication unit 1130, requesting a change in the camera control mode to a tracking mode. The motion detection unit 2120 also sends the control switching information to the position acquisition unit 1110. Here, the predetermined motion refers to the security guard finding a suspicious person or object and entering a tracking state. The motion detection unit 2120 may also be configured to independently execute a process of transmitting the control switching information by the operation of the security guard, as in the first embodiment. The acceleration sensor 2140 may be expressed as a motion detection device that detects the acceleration in the vertical direction and the acceleration in the horizontal direction of the mobile terminal 2100.

[0049] The management device 2200 includes a communication unit 1210, a control unit 2220, and a display unit 1230. The management device 2200 has the same hardware configuration as the management device 1200 according to the first embodiment shown in FIG.

[0050] The communication unit 1210 and the display unit 1230 are the same as those in the first embodiment.

[0051] The control unit 2220 includes a tracking unit 2221, a storage unit 1222, and a selection unit 2223. The control unit 2220 performs the same camera control as the control unit 1220 of the first embodiment. The control unit 2220 controls the camera based on control switching information acquired by the communication unit 1210. More specifically, the control unit 2220 changes the camera control mode from the normal mode to the tracking mode in the same manner as the first embodiment, and controls the camera.

[0052] When the tracking unit 2221 is in the tracking mode, it extracts a provisional tracking target, which is a provisional tracking target, from the images of the cameras 300-1 to 300-N. In addition, when the tracking unit 2221 is in the tracking mode, it extracts tracking candidates from the images of the cameras 300-1 to 300-N. Furthermore, the tracking unit 2221 tracks the tracking target designated by the operator of the management device 2200 from among the extracted tracking candidates.

[0053] The storage unit 1222 is the same as in the first embodiment.

[0054] The selection unit 2223 selects the camera closest to the position of the mobile terminal 2100 based on the acquired position information of the mobile terminal 2100 and the position information of each of the cameras 300-1 to 300-N previously stored in the storage unit 1222. Here, the configuration is such that the closest camera is selected, but it is not limited to the closest camera. Specifically, it may be such that the farthest camera is selected. Also, it is not necessary to select one camera, and it may be such that all cameras within a predetermined distance from the mobile terminal 1100 are selected.

[0055] Each of the cameras 300-1 to 300-N is the same as in the first embodiment. (Mobile terminal control) Next, a control flow in the mobile terminal 2100 will be described with reference to Fig. 13. Fig. 13 is a flow diagram in which the mobile terminal 1100 transmits information to the management device 2200.

[0056] In step S410, the same process as in step S110 in FIG. 2 is carried out.

[0057] In step S 420 , the motion detection unit 2120 receives the acceleration of the mobile terminal 2100 from the acceleration sensor 2140 .

[0058] In step S430, the motion detection unit 2120 determines whether the acquired acceleration is equal to or greater than a predetermined value. If the acceleration is equal to or greater than the predetermined value, the motion detection unit 2120 determines that the security guard has performed a predetermined motion. The motion detection unit 2120 transmits control switching information to the second communication unit 1132 and proceeds to step S440. If the acceleration is less than the predetermined value, the motion detection unit 2120 determines that the security guard has not performed a predetermined motion, and returns to step S420 to detect acceleration again. Here, the motion detection unit 2120 determines that the security guard has performed a predetermined motion by simply checking the predetermined value of the acceleration, but may also determine that the security guard has performed a predetermined motion if the period during which the acceleration is equal to or greater than the predetermined value exceeds a certain period.

[0059] In step S440, the same process as in step S130 in FIG. 2 is carried out, and the flow ends. (Camera control by management device) Next, the control flow in which the management device 2200 controls the cameras 300-1 to 300-N will be described with reference to Fig. 14 and the following Table 1. Fig. 14 is a control flow diagram in which the management device 2200 controls the shooting range of each of the cameras 300-1 to 300-N. Table 1 shows the position information of each of the cameras 300-1 to 300-N stored in the memory unit 1222.

[0060] In step S510, the same process as in step S210 in FIG. 3 is carried out.

[0061] In step S520, the selection unit 2223 compares the acquired position information of the mobile terminal 2100 with the position information of each of the cameras 300-1 to 300-N as shown in Table 1 below, and selects the camera closest to the position of the mobile terminal 2100. More specifically, when the real world coordinates, which are the acquired position information of the mobile terminal 2100, are 35 degrees 63 minutes 81.100 seconds north latitude and 139 degrees 74 minutes 72.894 seconds east longitude, the selection unit 2223 compares the degrees of north latitude and east longitude of each camera with the degrees of north latitude and east longitude of the acquired position information of the mobile terminal 2100, and calculates the difference. The selection unit 2223 selects the camera with the smallest calculated difference from among the cameras 300-1 to 300-N. When there are multiple selected cameras, the difference is calculated using the degrees of north latitude and the degrees of east longitude, and the camera with the smallest difference is selected. If there are multiple selected cameras, the seconds of north latitude and seconds of east longitude are compared, and the camera with the smallest difference is selected. At this point, if there are multiple cameras with the smallest difference, all of them are selected as the target cameras. In this specific example, camera 300-N is selected.

[0062] [Table 1]

[0063] In step S530, the same process as step S220 in FIG. 3 is performed. Here, in this step, it is also possible to detect the moving direction of the security guard and perform camera control that predicts the current position of the security guard. Specifically, the control unit 2220 periodically receives the position information and transmission time information of the mobile terminal 2100. In addition, the control unit 2220 receives the acceleration and acceleration direction of the mobile terminal 2100 acquired by the acceleration sensor 2140 of the mobile terminal 2100. Thereafter, the control unit 2220 calculates the predicted movement amount of the security guard from the position information, transmission time information, and acceleration of the mobile terminal 2100 received this time. Next, the control unit 2220 predicts the position where the mobile terminal 2100 will be located at the current time based on the calculated predicted movement amount and the acquired acceleration direction. Finally, the control unit 2220 controls the pan, tilt, and zoom of each of the cameras 300-1 to 300-N in the above-mentioned manner so that the predicted position of the mobile terminal 2100 is included in the shooting range.

[0064] In step S540, the same process as in step S230 in FIG. 3 is carried out, and the flow ends. (Tracking control by management device) Next, the flow of tracking control by the management device 2200 will be described with reference to Fig. 15. Fig. 15 is a flow diagram of tracking control in which the management device 2200 performs tracking.

[0065] In step S610, the same process as in step S310 in FIG. 4 is carried out.

[0066] In step S620, the same process as in step S320 in FIG. 4 is carried out.

[0067] In step S630, the tracking unit 2221 extracts features from the acquired video. The tracking unit 2221 compares the acquired features with features of the security guards stored in advance in the storage unit 1222 to extract the security guards. Then, the tracking unit 2221 determines the extracted security guards as tentative tracking targets. Furthermore, the tracking unit 2221 tracks the determined tentative tracking target. The tracking method is the same as the tracking algorithm in the first embodiment.

[0068] In step S640, the same process as in step S330 in FIG. 4 is carried out.

[0069] In step S650, the tracking unit 2221 outputs the images of the cameras 300-1 to 300-N acquired in step S620, the tentative tracking target set in step S630, and the tracking candidates extracted in step S640 to the display unit 1230. The display unit 1230 displays the images of the cameras 300-1 to 300-N acquired in step S620, and also highlights the tentative tracking target set in step S630 and the tracking candidates extracted in step S640 on the screen by surrounding them. Various methods can be used for highlighting, such as surrounding them with a line or filling in the detected target with a marker or the like.

[0070] In step S660, the same process as in step S350 in FIG. 4 is carried out.

[0071] In step S670, the tracking unit 2221 changes the tracking target from the tentative tracking target to the designated tracking target based on the designation of the tracking target, and continues tracking. A specific detection algorithm for the tracking target is the same as that in the first embodiment. (Specific example of embodiment 2) Next, a specific example of the second embodiment will be described with reference to Figs. 5, 16, and 17. Fig. 5 shows a configuration similar to that described in the first embodiment. In Fig. 5, when a security guard finds a suspicious person, the security guard moves in the direction of the suspicious person to capture the suspicious person without operating the mobile terminal 1100. The acceleration sensor 2140 of the mobile terminal 1100 acquires the acceleration of the security guard. When the motion detection unit 2120 detects that the acquired acceleration is equal to or greater than a predetermined acceleration, it transmits control switching information to the management device 2200 via the second communication unit 1132.

[0072] Fig. 16 shows an overhead view of camera control of a management device according to embodiment 2. In Fig. 16, control unit 2220 acquires video from each of cameras 300-1 to 300-3. Based on the position information of each of cameras 300-1 to 300-3 and the position information of mobile terminal 2100, control unit 2220 selects camera 300-2 that is closest to the position of mobile terminal 2100. Furthermore, control unit 2220 directs the line of sight of camera 300-2 to the position of mobile terminal 1100 that has been acquired, until the position of mobile terminal 2100 is reached.

[0073] FIG. 17 is a diagram showing a specific example of the display unit of the management device according to the second embodiment. The control unit 2220 displays the images captured by the cameras 300-1 to 300-3 on the display unit 1230. The control unit 2220 also extracts the security guards captured in the respective camera images and determines them as tentative tracking targets. The control unit 2220 also extracts the tracking candidate 1 captured in each camera image. The display unit 1230 displays the images of the cameras 300-1 to 300-3, and also surrounds and highlights the tentative tracking target and the tracking candidate 1. The control unit 2220 also causes the display unit 1230 to display one or more selectable tracking candidates so that the tracking target can be determined from among the tracking candidates. That is, the management device operator selects the tracking candidate 1 from the tracking candidate 1 displayed on the display unit 1230 this time, and changes the tracking candidate 1 to the tracking target by pressing the determination button. The control unit 2220 also performs camera control to track the determined tracking target. (Action and effect) The mobile terminal is configured to use an acceleration sensor to estimate whether a security guard is being tracked, and when it is estimated that a security guard is being tracked, the mobile terminal transmits control switching information to the management device. Therefore, even in a situation where the security guard cannot directly operate the mobile terminal to send control switching information, that is, when a suspicious person or object needs to be tracked immediately, the management device can be informed of the need to change the mode. Furthermore, by selecting a surveillance camera to be controlled from among multiple surveillance cameras, it is possible to limit the number of surveillance cameras and capture the target with a minimum number of cameras. In addition, by extracting tracking candidates and starting tracking of the security guard as a provisional tracking target before having the operator of the management device select a tracking target, the amount of communication with the mobile terminal and the communication time can be reduced. [Embodiment 3] (Features of the invention) In a configuration in which a mobile terminal periodically transmits location information to a management device, when control switching information is transmitted, the period for acquiring the location of the mobile terminal is made faster. Also, when an operator of the management device determines a tracking target, the mobile terminal ends the control for making the period faster. In addition, the mobile terminal is configured to automatically switch from the tentative tracking target to the tracking target that is the target to be actually tracked. (composition) The functional blocks and hardware configuration of the third embodiment are the same as those of the second embodiment. (Mobile terminal control) Next, a control flow in the mobile terminal 2100 will be described with reference to Fig. 18. Fig. 18 is a flow diagram in which the mobile terminal 2100 transmits information to the management device 2200.

[0074] In step S710, the same process as in step S110 in FIG. 2 is carried out.

[0075] In step S720, the same process as in step S120 in FIG. 2 is carried out.

[0076] In step S730, the position acquisition unit 1110 acquires control switching information from the motion detection unit 2120. When the position acquisition unit 1110 acquires the control switching information, the position acquisition unit 1110 changes the acquisition cycle of the position information acquired by itself to a faster cycle than the slow cycle, which is the normal acquisition cycle. Specifically, when the position acquisition unit 1110 updates the position information at a 10-second cycle (slow cycle) in the absence of the control switching information, the position acquisition unit 1110 switches its own position acquisition cycle to acquisition at a 2-second cycle (fast cycle) upon receiving the control switching information. Here, the first communication unit 1131 is configured to transmit the position information to the management device 2200 every time the position information acquired by the position acquisition unit 1110 is acquired, but the first communication unit 1131 may also be configured to set a cycle and change the cycle according to the control switching information.

[0077] In step S740, the same process as in step S130 in FIG. 2 is executed.

[0078] In step S750, the location acquisition unit 1110 judges whether or not the tracking unit 2221 of the control unit 2220 has acquired information indicating that the tracking target has been set. If the location acquisition unit 1110 has not acquired information indicating that the tracking unit 2221 has set the tracking target, the process remains at step S750. If the location acquisition unit 1110 has acquired information indicating that the tracking unit 2221 has set the tracking target, the process proceeds to step S760.

[0079] In step S760, when the position acquisition unit 1110 acquires information indicating that the tracking unit 2221 has set a tracking target, the position acquisition unit 1110 changes the cycle from the high speed to the low speed. Then, the flow ends. (Camera control by management device) The camera control by the management device has a control flow similar to that shown in FIG. 14 of the second embodiment, and therefore a description thereof will be omitted. (Tracking control by management device) Next, a control flow of the tracking control in the management device 2200 will be described with reference to Fig. 19. Fig. 19 is a flow diagram of tracking control in which the management device 2200 performs tracking.

[0080] In step S810, the same process as in step S310 in FIG. 4 is carried out.

[0081] In step S820, the same process as in step S320 in FIG. 4 is carried out.

[0082] In step S830, the same process as in step S630 in FIG. 15 is carried out.

[0083] In step S840, the control unit 2220 transmits information indicating that a tentative tracking target has been set to the mobile terminal 2100 via the communication unit 1210. Here, the control unit 2220 transmits information indicating that a tentative tracking target has been set to the mobile terminal 2100, but when a tracking target is determined in step S870 described later, the control unit 2220 may transmit information indicating that a tracking target has been set to the mobile terminal 2100.

[0084] In step S850, the tracking unit 2221 detects a moving object whose comparison result with the provisional tracking target satisfies a predetermined condition from the video of each of the cameras 300-1 to 300-N. Then, the tracking unit 2221 sets the detected moving object as a new tracking target. That is, the tracking unit 2221 changes the tracking target to the moving object detected from the provisional tracking target, which is a security guard. A more specific determination algorithm is shown below. First, the tracking unit 2221 detects the provisional tracking target and at least one other moving object from a plurality of frames in the video of each of the cameras 300-1 to 300-N. Furthermore, the tracking unit 2221 obtains a motion vector indicating the movement amount and movement direction of the provisional tracking target in the image (within the angle of view) and a motion vector indicating the movement amount and movement direction of the other moving object by comparing the image data between the read frames. Next, the tracking unit 2221 sets the other moving object, whose motion vector of the provisional tracking target and the motion vector of the other moving object are in substantially the same direction and whose difference in movement amount is equal to or less than a predetermined value, as a new tracking target. At this time, when a moving object exists but no other moving object that satisfies the predetermined condition is present, or when multiple other moving objects that satisfy the predetermined condition are detected, the tracking unit 2221 highlights the extracted moving object as a tracking candidate on the display unit 1230. The control unit 2220 requests the operator of the management device 2200 to select a tracking target.

[0085] In step S860, the tracking unit 2221 changes the tracking target set in step S850 to a new tracking target and continues tracking. After that, the flow ends. The specific tracking target detection algorithm is the same as in the first embodiment. (Action and effect) By speeding up the cycle for acquiring the position of the mobile terminal until the tracking target is determined, the real-timeness of the position information that can be transmitted to the management device is improved, and by using the end of the control for speeding up the cycle as a trigger for determining the tracking target, it is possible to reduce the excessive lengthening of the cycle for acquiring the position of the mobile terminal.In addition, since it is possible to automatically switch from the provisional tracking target to the tracking target that is the target to be actually tracked, it is possible to quickly capture the tracking target.

[0086] The present invention has been described above as an example of application to the exemplary embodiments described above. However, the technical scope of the present invention is not limited to the scope described in each of the above-mentioned embodiments. It is clear that a person skilled in the art can make various modifications or improvements to the embodiments. In such cases, new embodiments with such modifications or improvements may also be included in the technical scope of the present invention. This is clear from the matters described in the claims.

[0087] [Appendix 1] A mobile terminal; A management device that communicates with the mobile terminal, The mobile terminal, a first transmission unit that transmits location information of the mobile terminal to the management device at a predetermined interval; The management device includes: A control unit controls a camera orientation based on the location information when the location information is acquired. A monitoring system comprising:

[0088] [Appendix 2] The surveillance system according to claim 1, wherein the control unit controls the orientation of the camera to a shooting range in which the position indicated by the position information is captured.

[0089] [Appendix 3] The mobile terminal, a second transmission unit that transmits control switching information for switching camera control of the camera to the management device when a predetermined operation is detected; When the control unit receives the control switching information, the control unit controls the orientation of the camera based on the position information of the mobile terminal. 2. A monitoring system as described in claim 1.

[0090] [Appendix 4] the management device has a normal mode for monitoring a predetermined range and a tracking mode for monitoring based on the position information, When the control unit receives the control switching information, the control unit switches from the normal mode to the tracking mode and controls the orientation of the camera based on the position information of the mobile terminal. 3. A surveillance system according to claim 1 or 2.

[0091] [Appendix 5] The control unit acquires position information of the camera, The control unit selects at least one camera to be controlled from among the cameras based on the acquired position information of the camera and the acquired position information of the mobile terminal. 5. A monitoring system according to any one of claims 1 to 4.

[0092] [Appendix 6] The management device includes: A tracking unit that extracts candidates for a tracking target based on the image acquired by the camera; a display unit that displays the image and selectably highlights the candidates of the tracking target included in the image; 6. A monitoring system according to any one of claims 1 to 5.

[0093] [Appendix 7] The tracking unit sets the selected candidate for the tracking target as a tracking target and tracks the tracking target. 6. A monitoring system as described in claim 6.

[0094] [Appendix 8] The tracking unit identifies a holder of the mobile terminal based on the image acquired by the camera, and tracks the tentative tracking target by treating the identified holder as the tentative tracking target. 8. The monitoring device according to claim 6 or 7.

[0095] [Appendix 9] The tracking unit further extracts a moving object different from the owner of the mobile terminal based on the image acquired by the camera, The display unit highlights the extracted moving object as a candidate for the tracking target. 9. The monitoring device of any of claims 6 to 8.

[0096] [Appendix 10] When the mobile terminal detects the predetermined motion, the mobile terminal shortens an interval for acquiring the location information compared to when the mobile terminal does not detect the predetermined motion. 10. A monitoring system according to any one of claims 1 to 9.

[0097] [Appendix 11] When the tracking unit has determined the tracking target, the management device changes an interval for acquiring the location information of the management device to an acquisition interval when the predetermined motion is not detected. 7. A monitoring system as described in claim 7.

[0098] [Appendix 12] The first transmission unit transmits location information of the mobile terminal to the management device at predetermined intervals when an input to the mobile terminal is received. 12. A monitoring system according to any one of claims 2 to 11.

[0099] [Appendix 13] The mobile terminal further includes an acceleration sensor; The first transmission unit transmits location information of the mobile terminal to the management device at predetermined intervals when the acceleration acquired by the acceleration sensor becomes equal to or greater than a predetermined value. 12. A monitoring system according to any one of claims 2 to 11.

[0100] [Appendix 14] A mobile terminal carried by a security guard monitoring a specific facility; A management device that communicates with the mobile terminal and monitors the specific facility; At least one camera capable of photographing at least a part of the specific facility; The mobile terminal, a first transmission unit that transmits location information of the mobile terminal to the management device at a predetermined interval; The management device includes: A control unit controls a camera orientation based on the location information when the location information is acquired. A facility monitoring system comprising:

[0101] [Appendix 15] a first transmission unit that transmits location information of the device to a management device at a predetermined interval; The first transmission unit starts transmitting the location information to the management device when a predetermined action is detected. A mobile terminal comprising:

[0102] [Appendix 16] The first transmission unit transmits the location information to the management device at a predetermined interval when an input to the mobile terminal is received. 16. The mobile terminal of claim 15.

[0103] [Appendix 17] Further comprising an acceleration sensor; The first transmission unit transmits the location information to the management device at a predetermined interval when the acceleration acquired by the acceleration sensor becomes equal to or greater than a predetermined value. 16. The mobile terminal of claim 15.

[0104] [Appendix 18] an acquisition unit that acquires location information of a mobile terminal at a predetermined interval; and a control unit that controls the orientation of the camera based on the location information when the location information of the mobile terminal is acquired. A management device comprising:

[0105] [Appendix 19] The control unit controls the orientation of the camera based on the position information of the mobile terminal when control switching information for switching camera control is received from the mobile terminal. 19. The management device of claim 18.

[0106] [Appendix 20] Obtaining location information of the mobile terminal at predetermined intervals; When the position information is acquired, the orientation of the camera is controlled based on the position information. A monitoring method comprising:

[0107] [Appendix 21] A process of acquiring location information of a mobile terminal at a predetermined interval; and a process of controlling a camera orientation based on the location information when the location information is acquired; A monitoring program that causes a computer to execute the following.

[0108] [Appendix 22] A mobile terminal; A management device that communicates with the mobile terminal, The mobile terminal, a first transmission unit that transmits location information of the mobile terminal to the management device; The management device includes: a control unit that controls a camera orientation based on the location information when the location information is acquired; A tracking unit that extracts candidates for a tracking target based on the image acquired by the camera; a display unit that displays the image and selectably highlights the candidates of the tracking target included in the image; A monitoring system comprising:

[0109] [Appendix 23] The tracking unit sets the selected candidate for the tracking target as a tracking target and tracks the tracking target. 23. The monitoring system of claim 22.

[0110] [Appendix 24] The tracking unit identifies the owner of the mobile terminal based on the image acquired by the camera, and tracks the identified owner as a tentative tracking target. 24. The monitoring system of claim 22 or 23.

[0111] [Appendix 25] The tracking unit further extracts a moving object different from the owner of the mobile terminal based on the image acquired by the camera, The display unit highlights the extracted moving object as a candidate for the tracking target. 25. The monitoring system of any of claims 22 to 24.

[0112] This application claims priority based on Japanese Patent Application No. 2015-052795, filed on March 17, 2015, the disclosure of which is incorporated herein in its entirety. [Industrial Applicability]

[0113] The present invention can be used in a monitoring device, a monitoring method, a monitoring program, a monitoring system, and a facility monitoring system for performing monitoring. [Explanation of symbols]

[0114] 1000, 2000 Surveillance System 1100, 2100 Mobile terminal 1110 Position acquisition part 1120, 2120 Motion detection unit 1130 Communications Department 1131 1st Communications Department 1132 2nd Communications Department 2140 Acceleration Sensor 1200, 2200 management device 1210 Communications Department 1211 First Communications Department 1212 2nd Communications Department 1220, 2220 Control unit 1221, 2221 Tracking Department 1222 Storage section 2223 Selection section 1230 Display section 300-1~300-N Camera 100, 200 CPUs 130 Display section 110, 210 Input reception section 120, 220 Memory 230 Connection

Claims

1. A first mobile terminal; A second mobile terminal; a management device that communicates with the first mobile terminal and the second mobile terminal; The first mobile terminal comprises: a first transmission unit that transmits location information of the first mobile terminal to the management device; The management device includes: a control unit that predicts a future location of the first mobile terminal from the location information, controls a camera orientation so that the first mobile terminal is imaged based on the future location, extracts an image acquired by the camera, and transmits the image to the second mobile terminal; when detecting a predetermined action of a person carrying the first mobile terminal, the first mobile terminal transmits, to the management device, location information of the first mobile terminal and control switching information for switching camera control of the camera; the management device has a normal mode for monitoring a predetermined range and a tracking mode for monitoring based on the position information, When the control unit receives the control switching information, the control unit switches from the normal mode to the tracking mode and controls the orientation of the camera based on the future position of the first mobile terminal. A monitoring system comprising:

2. The control unit controls the direction of the camera to a shooting range in which the future position is shot. The monitoring system of claim 1 .

3. The control unit predicts the future position from a traveling direction of the first mobile terminal. The monitoring system of claim 1 .

4. The control unit is Acquire position information of the camera; At least one camera to be controlled is selected from among the cameras based on the acquired position information of the cameras and the acquired future position of the first mobile terminal. The monitoring system of claim 1 .

5. A first mobile terminal transmits location information of the first mobile terminal to a management device; the management device predicts a future location of the first mobile terminal from the location information, controls a camera orientation so that the first mobile terminal is imaged based on the future location, extracts an image acquired from the camera, and transmits the image to a second mobile terminal; When the first mobile terminal detects a predetermined action of a person carrying the first mobile terminal, the first mobile terminal transmits, to the management device, location information of the first mobile terminal and control switching information for switching camera control of the camera; When the management device receives the control switching information, the management device switches from a normal mode in which a predetermined range is monitored to a tracking mode in which monitoring is performed based on the position information, and controls the orientation of the camera based on the future position of the first mobile terminal. A monitoring method comprising:

6. A computer included in a management device that communicates with the first mobile terminal and the second mobile terminal, A process of predicting a future location of the first mobile terminal from location information of the first mobile terminal transmitted by the first mobile terminal, controlling a camera orientation so that the first mobile terminal is imaged based on the future location, extracting an image acquired by the camera, and transmitting the image to the second mobile terminal; A process of receiving location information of the first mobile terminal and control switching information for switching camera control of the camera, the location information being transmitted by the first mobile terminal when a predetermined motion of a person carrying the first mobile terminal is detected; and A process of switching from a normal mode in which a predetermined range is monitored to a tracking mode in which monitoring is performed based on the position information when the control switching information is received, and controlling the orientation of the camera based on the future position of the first mobile terminal. A monitoring program for executing the following:

Images