Video monitoring device, display control method, and video monitoring system

Abstract

To provide a video monitoring device that enables a user to confirm a tracking target from video displayed on a small screen of a user terminal.SOLUTION: A video monitoring device 3 comprises: a video acquisition unit 33 for acquiring a plurality of pieces of video from a plurality of cameras 50; a tracking target identification unit 34 for identifying a tracking target from the plurality of pieces of video acquired by the video acquisition unit 33; a camera selection unit 35 for selecting video in which a tracking target is captured: and a display control unit 31 for performing control of displaying the video selected by the camera selection unit 35 on a user terminal 1.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video monitoring device, a display control method, and a video monitoring system. [Background technology]

[0002] Conventionally, a video monitoring system is generally installed that transmits information reporting the status of the building from a management server of the building to a mobile terminal carried by a worker and performs monitoring activities. In the conventional system, when an abnormality is detected in a building being monitored, the status of the part where the abnormality occurred is sent by email to the terminal carried by the worker, and the abnormality in the building can be grasped by displaying both text information and image information. However, the worker cannot arbitrarily specify a camera installed in the building, specify the date and time when the specified camera took the image, and display the image information, so there is a problem that the surrounding situation of the part where the abnormality occurred and the situation before and after the abnormality occurred cannot be grasped.

[0003] Therefore, a technology aimed at allowing a manager to quickly check where an abnormality has occurred is disclosed in Patent Document 1. Patent Document 1 discloses a video monitoring system in which a worker operates a mobile terminal equipped with a mobile terminal operation unit that accepts an operation to specify a specific camera from multiple cameras, and a mobile terminal display control unit that displays video data received from a video distribution server via a network on the video display unit, thereby causing the video display unit to display video data captured by a specific camera at a specified shooting date and time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-219923 A Summary of the Invention [Problem to be solved by the invention]

[0005] The mobile terminal disclosed in Patent Document 1 is small, and the screen of a mobile terminal is generally small. For this reason, it is difficult for a user to specify a specific camera from among multiple cameras using the technology disclosed in Patent Document 1. In addition, when multiple images are displayed in reduced size on the screen of a mobile terminal, it is difficult for a user to confirm a person captured in a small image.

[0006] The present invention has been made in view of the above circumstances, and has an object to enable a user to easily check a tracking target through a user terminal. [Means for solving the problem]

[0007] The video monitoring device according to the present invention includes a video acquisition unit that acquires a plurality of videos from a plurality of cameras, and a tracking target identification unit that identifies a tracking target from the plurality of videos acquired by the video acquisition unit; A camera management unit records positional relationship information that correlates the installation positions of a plurality of cameras; a movement direction detection unit detects the movement direction of a tracking target and estimates the movement route of the tracking target based on the positional relationship information recorded in the camera management unit; and a camera installed in the movement direction captures the image, The camera selection unit selects an image showing a target to be tracked, and the display control unit controls the image selected by the camera selection unit to be displayed on a user terminal. The display control unit displays on the user terminal a main display unit that displays an image selected by the camera selection unit and including the tracking target tracked by the tracking target identification unit, a sub-display unit that displays in list form a plurality of images sorted according to the moving direction detected by the moving direction detection unit, and a moving route display unit that displays the moving route, and displays on the sub-display unit a plurality of images acquired by the image acquisition unit in the order of the cameras installed along the moving route, and the tracking target is specified by selecting an image showing the tracking target from the plurality of images displayed on the sub-display unit, and when an image showing the tracking target is captured by a camera not set on the moving route, the moving direction detection unit re-estimates the moving route. . Effect of the Invention

[0008] According to the present invention, since an image showing the tracking target is displayed on the user terminal, the user can easily check the tracking target through the user terminal. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] 1 is a system configuration diagram showing an example of the overall configuration of a video monitoring system according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing an example of a hardware configuration of a user terminal according to an embodiment of the present invention. FIG. [Diagram 3] 1 is a block diagram showing an example of a hardware configuration of a video monitoring device according to an embodiment of the present invention. [Figure 4]FIG. 2 is a diagram showing installation positions of cameras according to an embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram showing the positional relationship between cameras according to an embodiment of the present invention. [Figure 6] 11 is a table configuration diagram showing an example of positional relationship information and weighting information according to an embodiment of the present invention. FIG. [Figure 7] FIG. 2 is a diagram showing an example of a default display screen of a user terminal according to an embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing an example of a process for estimating a person's movement route according to an embodiment of the present invention. [Figure 9] 10 is a flowchart illustrating an example of a person tracking process according to an embodiment of the present invention. [Figure 10] FIG. 2 is a diagram showing an example of a travel route according to an embodiment of the present invention; [Figure 11] 1A to 1C are diagrams showing examples of screen displays of a route taken by a person who has left an elevator car and is moving within a building in accordance with an embodiment of the present invention; [Figure 12] 1A to 1C are diagrams showing examples of screen displays of a route taken by a person who has left an elevator car and is moving within a building in accordance with an embodiment of the present invention; [Figure 13] 11A and 11B are diagrams showing an example of a screen display of a movement route of a person moving from outside a building into the building in accordance with an embodiment of the present invention. [Figure 14] 11A and 11B are diagrams showing an example of a screen display of a movement route of a person moving from outside a building into the building in accordance with an embodiment of the present invention. [Figure 15] FIG. 13 is a diagram showing an example of a screen display for a re-predicted travel route according to an embodiment of the present invention. [Figure 16] FIG. 13 is a diagram showing a modified example of a screen displayed on a user terminal according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0011] [One embodiment] FIG. 1 is a system configuration diagram showing an example of the overall configuration of a video monitoring system 10 according to an embodiment. The video monitoring system 10 includes a user terminal 1, a video monitoring device 3, and a group of cameras 5. The user terminal 1 is used as an example of a user terminal that can be carried and operated by a user. An example of the user is assumed to be, for example, a manager of a building in which an area for monitoring video is provided. The manager may manage multiple buildings, and in order to check the status of each building while minimizing movement, the manager checks the status of suspicious persons by viewing the video displayed on the user terminal 1. In the following description, the suspicious person is also referred to as a "tracking target."

[0012] Examples of the user terminal 1 include smart devices such as smartphones and tablet terminals. A user can check information displayed on a touch panel display of the user terminal 1 and input various instructions. A user can monitor the situation of a building at night, or the situation of a passageway leading to an area with restricted entry such as a server room, by watching the video displayed on the user terminal 1.

[0013] A user terminal 1 and a video monitoring device 3 are connected via an Internet line 2 and are capable of transmitting and receiving data to and from each other. Furthermore, the video monitoring device 3 and the group of cameras 5 are connected via an Internet line 4, and are capable of transmitting and receiving data to and from each other.

[0014] The camera group 5 includes six cameras 51 to 56. In the following description, the camera name of camera 51 is "camera A", the camera name of camera 52 is "camera B", and the camera name of camera 53 is "camera C". Similarly, the camera name of camera 54 is "camera D", the camera name of camera 55 is "camera E", and the camera name of camera 56 is "camera F". When there is no need to distinguish between the cameras 51 to 56, they will be referred to as cameras 50. Each camera 50 included in the camera group 5 is installed inside the building or corridor to be monitored, as well as on the grounds outside the building.

[0015] The video monitoring device 3 is a device constructed on a cloud server by an administrator of the video monitoring system 10. This video monitoring device 3 includes a display control unit 31, a camera management unit 32, a video acquisition unit 33, a tracking target identification unit 34, a camera selection unit 35, and a movement direction detection unit 36.

[0016] The display control unit 31 controls the display of the video selected by the camera selection unit 35 on the user terminal 1. Then, the display control unit 31 generates a screen to be displayed on the user terminal 1 based on the video output from the video acquisition unit 33, and transmits the screen data to the user terminal 1 via the Internet line 2. Therefore, the user terminal 1 displays, for example, a screen shown in FIG. 7, which will be described later, based on the received screen data. Since the screen generated by the display control unit 31 is displayed on the user terminal 1, the user looking at the screen can confirm the state of the tracking target instructed by the user terminal 1.

[0017] The display control unit 31 itself may not generate a screen, but may transmit an image of a camera 50 installed on the movement route of the monitored object shown in Fig. 10 described later to the user terminal 1, and may transmit additional information indicating an image including a person tracked by the tracking target identification unit 34 to the user terminal 1. In this case, the user terminal 1 generates a screen (see Fig. 7 described later) based on the received additional information, which displays an image including a person identified by the tracking target identification unit 34 on the main display unit 401, and displays an image of a camera 50 installed on the movement route on the sub-display unit 403.

[0018] Furthermore, when a command to track a person is input to the user terminal 1, the display control unit 31 receives data of the command from the user terminal 1. Then, the display control unit 31 instructs the tracking target identification unit 34 to identify the tracking target person from the video including the person instructed to be tracked, and to track the person.

[0019] The camera management unit 32 records positional relationship information 321 in which the installation positions of the multiple cameras 50 are related to each other, and weighting information 322 in which the installation position of each camera 50 is weighted. Note that, as shown in Fig. 6 described later, the positional relationship information 321 and the weighting information 322 are used in combination.

[0020] Positional relationship information 321 is information that defines the positional relationship of each camera 50. The installation position of each camera 50 is registered in advance in positional relationship information 321. Based on the installation position of each camera 50, the relative positional relationship between adjacent cameras 50 is defined. When the installation position of a camera 50 is changed, the user inputs the changed installation position through user terminal 1, and positional relationship information 321 is updated.

[0021] The weighting information 322 is set with a weighting according to the position of the camera 50. When multiple cameras 50 are installed on the movement route estimated by the movement direction detection unit 36, the camera selection unit 35 selects the video captured by the camera 50 with a high weighting based on the weighting information 322 recorded in the camera management unit 32. Here, if the area is available only to specific people, such as a security area or a server room, a high weighting is set for the camera 50 installed in this area. The weighting of the cameras 50 can be set arbitrarily by the user.

[0022] The tracking target identifying unit 34 identifies the tracking target from the multiple videos acquired by the video acquisition unit 33. The tracking target identifying unit 34 according to this embodiment identifies the tracking target specified by the user terminal 1 from the multiple videos. The person to be tracked is a person appearing in the video for which a tracking command is input from the user terminal 1. The tracking target identifying unit 34 tracks the identified person from the multiple videos acquired from the multiple cameras 50, and outputs information identifying the video in which the tracked person appears to the display control unit 31.

[0023] The camera selection unit 35 selects an image showing the tracking target. Here, when the movement direction detection unit 36 ​​detects the movement direction of the tracking target, the camera selection unit 35 selects an image captured by the camera 50 installed in the movement direction. For example, when a person to be tracked moves along a passageway, the movement direction detection unit 36 ​​can determine the camera 50 installed at the person's destination simply by detecting whether the person is moving left or right. Then, the camera selection unit 35 can select the image of the camera 50 installed at the person's destination as an image showing the tracking target.

[0024] The video acquisition unit 33 acquires multiple videos captured by each camera 50 from the camera group 5 (multiple cameras 50) via the Internet line 4. The multiple acquired videos are output to the display control unit 31, the video acquisition unit 33, the tracking target identification unit 34, the camera selection unit 35, and the movement direction detection unit 36. The video acquisition unit 33 acquires the videos (moving images) captured by each camera 50 in real time. However, the video acquisition unit 33 may acquire the videos (still images) captured by each camera 50 every few seconds at each timing of capture.

[0025] <Example of hardware configuration of user terminal 1> Next, an example of the hardware configuration of the user terminal 1 will be described. FIG. 2 is a block diagram showing an example of the hardware configuration of the user terminal 1. As shown in FIG.

[0026] The user terminal 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, and a RAM (Random Access Memory) 13, each of which is connected to a bus 14. Furthermore, the user terminal 1 includes a display device 15, an input device 16, a non-volatile storage 17, and a network interface 18.

[0027] The CPU 11 reads out program code of software for implementing each function according to the present embodiment from the ROM 12, loads it into the RAM 13, and executes it. Variables, parameters, etc. generated during the arithmetic processing of the CPU 11 are temporarily written into the RAM 13, and these variables, parameters, etc. are read out by the CPU 11 as appropriate. However, an MPU (Micro Processing Unit) may be used instead of the CPU 11.

[0028] The display device 15 is, for example, a liquid crystal display monitor, and displays to the user the results of processing performed in the user terminal 1. The input device 16 is, for example, a keyboard, a mouse, etc., and allows the user to input predetermined operations and give instructions. The user terminal 1 uses a touch panel display device configured by overlapping the display device 15 and the input device 16.

[0029] For example, a hard disk drive (HDD), a solid state drive (SSD), or a nonvolatile memory is used as the nonvolatile storage 17. In addition to an operating system (OS) and various parameters, a program for operating the user terminal 1 is recorded in this nonvolatile storage 17. The ROM 12 and the nonvolatile storage 17 record programs, data, and the like required for the CPU 11 to operate, and are used as an example of a computer-readable non-transient storage medium that stores a program executed by the user terminal 1.

[0030] The network interface 18 may be, for example, a network interface card (NIC), and various data may be transmitted and received between devices via a local area network (LAN) or dedicated line connected to a terminal of the NIC.

[0031] <Example of hardware configuration for video surveillance equipment> Next, the hardware configuration of the video monitoring device 3 will be described. Fig. 3 is a block diagram showing an example of a hardware configuration of the video monitoring device 3. In the video monitoring device 3 according to the present embodiment, a computer constituting the video monitoring device 3 executes a program, thereby realizing a display control method in which the respective functional blocks shown in Fig. 1 cooperate with each other.

[0032] The video monitoring device 3 includes a CPU 61, a ROM 62, and a RAM 63, each of which is connected to a bus 64. The video monitoring device 3 further includes a non-volatile storage 65 and a network interface 66.

[0033] The CPU 61 reads out program code of software for implementing each function according to the present embodiment from the ROM 62, loads it into the RAM 63, and executes it. Variables, parameters, etc. generated during the arithmetic processing of the CPU 61 are temporarily written into the RAM 63, and these variables, parameters, etc. are read out by the CPU 61 as appropriate. However, an MPU may be used instead of the CPU 61.

[0034] The non-volatile storage 65 may be, for example, an HDD, an SSD, a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a non-volatile memory. In addition to the OS and various parameters, a program for operating the video monitoring device 3 is recorded in this non-volatile storage 65. The ROM 62 and the non-volatile storage 65 record programs, data, and the like required for the CPU 61 to operate, and are used as an example of a computer-readable non-transient storage medium that stores a program executed by the video monitoring device 3.

[0035] The network interface 66 may be, for example, a NIC, and various data can be transmitted and received between devices via a LAN, dedicated line, or the like connected to a terminal of the NIC.

[0036] <Information about where the camera will be installed> Next, the contents of the positional relationship information 321 and the weighting information 322 will be described with reference to FIGS. FIG. 4 is a diagram showing the installation positions of the cameras 50. As shown in FIG. 4, the positional relationship of cameras A to F is shown as each camera 50. In the drawing, camera A is represented by two different symbols, such as camera 51(A).

[0037] As described above, positional relationship information 321 is defined for each of the multiple adjacent cameras 50. Cameras A to F are installed inside and outside the building, and face a predetermined shooting direction to shoot a predetermined shooting range (range indicated by a circular arc). Cameras A to F are attached to the walls of the building or corridor. The shooting direction and shooting angle of view of cameras A to F can be changed arbitrarily.

[0038] As shown in FIG. 4, camera A is installed on the wall near the ceiling of the elevator car and captures images inside the car. Camera B is installed on the wall of the corridor and captures the inside of the corridor. Camera C is installed on the wall opposite Camera B and captures the interior of the corridor. Camera D is installed next to camera B and captures the inside of the corridor. Camera E is installed in an area that requires more vigilance than other areas and photographs this area. Camera F is installed outside the building and captures the parking lot.

[0039] Fig. 5 is a diagram showing the positional relationship of each camera 50. In Fig. 5, the description of the structure of the building and the like is omitted, and only the positional relationship of each camera 50 is shown.

[0040] Each camera 50 is provided with positional relationship information, such as R (right side), L (left side), or U (front side), indicating the direction in which the adjacent camera 50 is located. Furthermore, the lines connecting each camera 50 represent part of the route along which a person can move. For example, a person can move from camera A to camera B, and from camera B to camera C. On the other hand, there is no line connecting camera E and camera F. This indicates that when a person moves, he or she cannot go directly from camera E to camera F, but can only move from camera E to camera D to camera F (or in the reverse order).

[0041] When the camera 50 is attached to the ceiling of a passageway, a room, or the like, positional relationship information may also be added to the rear direction, which is the opposite direction to the front direction, as the positional relationship of the camera 50 in FIG.

[0042] FIG. 6 is a table configuration diagram showing an example of the positional relationship information 321 and the weighting information 322. As shown in FIG. The table shown in Fig. 6 is composed of the items Camera, Left (L), Front (U), Right (R), and Weighting. Note that the No. item on the left side of the table is added to identify the records in the table, and does not have to be included in the actual table.

[0043] In the camera item, a camera name such as “camera A” or “camera B” is stored as identification information of the camera 50. The left side (L), front side (U), and right side (R) items store identification information (any of the codes A to F) of the cameras 50 located on the left side (L), front side (U), and right side (R) of each camera 50, with each camera 50 as the starting point. The weighting item stores the weighting assigned to each camera 50.

[0044] 6 is registered by an administrator according to the layout of the cameras 50 installed in a building when constructing the video monitoring device 3. Furthermore, if a camera 50 is added or removed, changing the layout of the cameras 50, the administrator re-registers the information for each camera 50 according to the changed layout.

[0045] 4, camera B is installed to the right of camera A, so the code "B" representing camera B is stored in the right (R) column of the record for camera A in row No. 1. Also, since no other cameras 50 are installed to the left and in front of camera A, "-" is stored in the left (L) and front (U) columns.

[0046] Additionally, camera A is installed to the left of camera B, so in the record for camera B in row No. 2, the code "A" representing camera A is stored in the left (L) column. Additionally, camera C is installed in front of camera B, so the code "C" representing camera C is stored in the front (U) column. Additionally, cameras D and E are installed to the right of camera B, so the codes "D,E" representing cameras D and E are stored in the right (R) column. As for the other cameras 50, the positional relationship of each camera 50 is stored in the table in accordance with the positional relationship information 321 shown in FIG.

[0047] The weighting assigned to each camera 50 is managed by weighting information 322. In the table shown in Fig. 6, cameras A to D, and F are weighted at "1," and camera E is weighted at "2." In this example, camera E is installed in an area that is more important than the areas in which the other cameras 50 are installed (for example, a warehouse where valuables are stored). For this reason, camera E is weighted higher than the other cameras.

[0048] The weighting is information that changes depending on the operation of the site where the camera 50 is installed. For example, when there are multiple cameras 50 capturing the same area, if the image captured by a camera 50 from a specific direction is important, the weighting of that camera 50 is set high. Also, if important areas change depending on the time of day, the weighting may be changed according to the time of day when the important areas change. Such weighting can be set through the user terminal 1.

[0049] Furthermore, when multiple cameras 50 are installed in a building, it is not possible to display all of the images captured by all of the cameras 50 on the sub-display unit 403. For this reason, it is possible to preferentially display the cameras 50 installed in a specific area on the default display screen by setting the weighting of the cameras 50 installed in a specific area higher than that of the cameras 50 installed in other areas, for example.

[0050] <User terminal display screen> Next, an example of the display screen of the user terminal 1 will be described. FIG. 7 is a diagram showing an example of a default display screen of the user terminal 1. As shown in FIG.

[0051] The display screen of the user terminal 1 is a screen that the user terminal 1 acquires from the video monitoring device 3 by accessing the video monitoring device 3 via the Internet line 2 as a result of the user operating the user terminal 1. This default display screen includes a main display section 401, a person tracking function switching button 402, and a sub-display section 403. The display control section 31 displays, on the user terminal 1, the main display section 401 that displays the video selected by the camera selection section 35, the sub-display section 403 that displays, in order, a plurality of videos rearranged according to the movement direction of the tracking target detected by the movement direction detection section 36, and the person tracking function switching button 402.

[0052] A video of a person (tracking target) currently being watched by the user is displayed on the main display unit 401. The user can check the enlarged video displayed on the main display unit 401 showing the tracking target. The user can arbitrarily change the camera 50 that captures the video displayed on the main display unit 401. However, the video may be automatically switched under the control of the display control unit 31.

[0053] The person tracking function switching button 402 is used by the user to switch the person tracking function ON or OFF. In the default state shown in FIG. 7, the person tracking function is OFF. Therefore, an image in which a person is not reflected may be displayed on the main display unit 401. When the user touches the person tracking function switching button 402, the person tracking function is turned ON. In this case, an image in which a person is reflected is displayed on the main display unit 401. Therefore, the operation of the user touching the person tracking function switching button 402 is regarded as an instruction to start tracking the tracking target. Then, the tracking target identification unit 34 starts tracking the tracking target based on the instruction to start tracking received from the user terminal 1. Since the tracking target identification unit 34 starts tracking the tracking target upon receiving the instruction to start tracking, only the image of the tracking target that the user wants to pay attention to is displayed on the main display unit 401. When the person moves and moves out of the image displayed on the main display unit 401, another image in which the person is reflected is displayed on the main display unit 401.

[0054] The sub-display unit 403 displays the names of the cameras 50 installed in the building (e.g., camera A, camera B) and multiple images captured by each camera 50 in a vertically arranged list format. This makes it easier for the user to check the multiple images captured by each camera 50 in the list format. Here, the main display unit 401 displays images larger than the sub-display unit 403. This makes it easier for the user to check the tracking target that is selected from the sub-display unit 403 and appears in the image displayed on the main display unit 401.

[0055] The images displayed on the sub-display unit 403 include all images captured by the six cameras 50, camera A to camera F. All images displayed on the sub-display unit 403 are acquired by the image acquisition unit 33 from each camera 50 in the camera group 5 via the Internet line 4. Furthermore, on the default display screen, the images displayed in a list on the sub-display unit 403 may be in the order of identifiers for identifying the cameras 50.

[0056] Then, the movement direction detection unit 36 ​​estimates the movement route of the tracking target based on the positional relationship information 321 recorded in the camera management unit 32. The display control unit 31 displays the multiple images acquired by the image acquisition unit 33 on the sub-display unit 403 in the order of the cameras 50 installed along the movement route. Therefore, the user can understand the estimated movement route from the images displayed on the sub-display unit 403. Then, the tracking target is specified by selecting an image showing the tracking target from the multiple images displayed on the sub-display unit 403. This makes it easy for the tracking target identification unit 34 to identify the person to be tracked.

[0057] Furthermore, a radio button is displayed next to the camera name displayed on the sub-display section 403. When the user selects an image from the sub-display section 403, the radio button provided corresponding to the camera 50 capturing the image displayed on the main display section 401 is turned ON.

[0058] The video displayed on the main display unit 401 and the video of each camera 50 displayed on the sub-display unit 403 are linked. When the user selects any video from the sub-display unit 403, the video selected by the user is displayed on the main display unit 401. At this time, the radio button corresponding to the camera name of the camera 50 capturing the video displayed on the main display unit 401 is turned ON. Then, the user can check, from the sub-display unit 403, the video captured by the cameras 50 installed on the movement route of the tracking target.

[0059] <Example of route estimation process when tracking function is ON> Next, an example of a process for estimating a travel route when the tracking function is ON will be described with reference to the flowcharts shown in FIGS. 8 and 9 and examples of display screens of the user terminal 1 shown in FIGS.

[0060] FIG. 8 is a diagram showing an example of a process for estimating a person's movement route.

[0061] Before starting this process, it is assumed that the user presses the person tracking function switching button 402 on the user terminal 1. When the switching button 402, which has the person tracking function turned OFF, is pressed, the person tracking function is turned ON. Then, person tracking command data indicating that the person tracking function has been turned ON is input to the display control unit 31 of the video monitoring device 3 shown in FIG. 1 via the Internet line 2.

[0062] Therefore, the video monitoring device 3 starts a process of estimating a movement route (S1). First, when the display control unit 31 receives person tracking command data from the user terminal 1, it outputs a person tracking command to the tracking target identification unit 34 and the movement direction detection unit 36. The tracking target identification unit 34 and the movement direction detection unit 36 ​​start tracking a person appearing in the image displayed on the main display unit 401 or a person selected and input by the user terminal 1 from among the images displayed on the sub-display unit 403, as a tracking target.

[0063] Next, the movement direction detection unit 36 ​​detects the movement direction of the person appearing in the image displayed on the main display unit 401. Then, the movement direction detection unit 36 ​​determines whether or not a camera 50 other than the camera 50 capturing the image displayed on the main display unit 401 is installed in the detected movement direction (also called the movement destination) (S2).

[0064] If the movement direction detection unit 36 ​​determines in step S2 that no camera 50 is installed at the destination of the person (NO in S2), the process proceeds to step S8, which will be described later. On the other hand, if the movement direction detection unit 36 ​​determines in step S2 that a camera 50 is installed at the destination of the person (YES in S2), it determines whether the number of cameras 50 installed at the destination is two or more (S3).

[0065] If the number of cameras 50 installed at the destination is not two or more (NO in S3), the movement direction detection unit 36 ​​adds the cameras 50 at the destination to the movement route (S4). On the other hand, if the number of cameras 50 installed at the destination is two or more (YES in S3), the movement direction detection unit 36 ​​adds the camera 50 with the highest weighting among the cameras 50 at the two or more destinations to the movement route (S5).

[0066] After step S4 or S5, the movement direction detection unit 36 ​​determines whether or not a next camera 50 is installed at the destination, starting from the camera 50 at the destination (S6). If the next camera 50 is not installed at the destination (NO in S6), the process proceeds to step S8, which will be described later.

[0067] On the other hand, if the movement direction detection unit 36 ​​determines in step S6 that the next camera 50 is installed at the movement destination (YES in S6), this camera 50 represents a camera 50 that has already been registered in the movement route, and is the person's return route. Therefore, the movement direction detection unit 36 ​​excludes the next camera 50 installed at the movement destination determined in step S6 from the movement route (S7). Thereafter, the movement direction detection unit 36 ​​returns to step S3, and repeats the processes of steps S3 to S7.

[0068] After NO in step S2 or NO in step S6, the movement direction detection unit 36 ​​completes adding all the cameras 50 on the movement route. Then, the camera selection unit 35 transmits the images captured by the cameras 50 selected along the movement route to the display control unit 31. The display control unit 31 performs processing for displaying the images captured by each camera 50 on the sub-display unit 403 of the user terminal 1 based on the information on the cameras 50 received from the camera selection unit 35 (S8). Thereafter, the tracking target identification unit 34 starts a person tracking process (S9).

[0069] FIG. 9 is a flowchart showing an example of a person tracking process. First, the tracking target identification unit 34 determines whether or not a person photographed by the camera 50 displayed on the main display unit 401 has disappeared from the shooting range of the camera 50, i.e., from the image captured by the camera 50 (S11).

[0070] When it is determined that the person has not disappeared from the shooting range of the camera 50 (NO in S11), the tracking target identification unit 34 continues the process of step S11 again. On the other hand, when it is determined that the person has disappeared from the shooting range of the camera 50 (YES in S11), the tracking target identification unit 34 continues to monitor whether the tracking target person appears in the video captured by the camera 50 from which the person has disappeared, that is, the camera 50 displayed on the main display unit 401, and the video captured by the camera 50 installed in the moving direction of the camera 50 (S12).

[0071] Then, the tracking target identification unit 34 judges whether or not the tracking target person appears in the image of the range captured by the camera 50 installed at the moving direction destination among the images of the cameras 50 monitored in step S12 (S13). If the tracking target person does not appear in the image (NO in S13), the tracking target identification unit 34 continues the process of step S12.

[0072] On the other hand, if the person to be tracked appears in the video (YES in S13), the camera selection unit 35 outputs information about the camera 50 capturing the video in which the person to be tracked appears to the display control unit 31. Then, the display control unit 31 displays the video of the camera 50 capturing the video in which the person appears, acquired from the video acquisition unit 33 based on the information of the camera 50 input from the camera selection unit 35, on the main display unit 401 of the user terminal 1 (S14).

[0073] Next, the camera selection unit 35 judges whether the camera 50 capturing the video in which the tracking target person appears is the camera 50 added to the moving route in step S4 or S5 of Fig. 8 (S15). If the camera 50 is the camera 50 added to the moving route (YES in S15), the tracking target specification unit 34 returns the process to step S11.

[0074] On the other hand, if the camera 50 is not one that has been added to the movement route (NO in S15), the movement route is estimated incorrectly. Therefore, the movement direction detection unit 36 ​​re-estimates the movement route (S16) and returns the process to step S11.

[0075] In this way, when a tracking target that has disappeared from the image on the main display unit 401 appears in an image captured by a camera 50 that is not installed on the moving route, the moving direction detection unit 36 ​​again estimates the moving route starting from the camera 50 that captured the image capturing the tracking target. Then, the display control unit 31 displays a plurality of images on the sub-display unit 403 in the order of the cameras 50 installed along the re-estimated moving route. Therefore, the user can check the images of the cameras 50 installed on the re-estimated moving route without performing any special operation.

[0076] Next, based on the flowcharts of FIG. 8 and FIG. 9, examples of screens displayed on the user terminal 1 when a person to be tracked moves will be described with reference to FIG. 10 to FIG.

[0077] Fig. 10 is a diagram showing an example of a movement route. In each diagram of Fig. 10, the movement route is represented by a thick arrow. Fig. 10A shows an example of a route taken by a person who leaves an elevator car and moves within a building. Examples of screen displays corresponding to Fig. 10A are shown in Figs. 11 and 12, which will be described later. Fig. 10B is an example of a route of a person moving from outside a building to inside the building. Display examples of a screen corresponding to Fig. 10B are shown in Figs. 13 and 14, which will be described later. Fig. 10C is an example of a re-estimated travel route. An example of a screen display corresponding to Fig. 10C is shown in Fig. 15, which will be described later.

[0078] In the following description, the cameras 50 will be referred to as "camera A" to "camera F" in accordance with the display names of the cameras 50 displayed on the user terminal 1. In addition, as for the video captured by each camera 50, for example, the video captured by camera A will be referred to as the "video of camera A."

[0079] <Route of a person leaving the car and moving around the building> FIG. 11A is a diagram showing an example in which an image from camera A is displayed on main display unit 401. In FIG.

[0080] (Camera A footage) When the display screen of the user terminal 1 is in the display state shown in Fig. 7 described above, an image captured by camera A showing a person riding in a car is displayed on the main display section 401. When the user of the user terminal 1 presses the person tracking function switching button 402, the person tracking function switching button 402 displays "Person tracking ON" as shown in Fig. 11A.

[0081] Then, movement direction detection unit 36 ​​adds camera B installed to the right of camera A, camera E with the highest weighting ahead of camera B, camera D at the destination, and camera F at the destination to the movement route, according to the flowchart shown in Fig. 8. As shown in Fig. 11A, main display unit 401 displays the image of camera A. Then, sub-display unit 403 displays the images of camera A, camera B, camera E, camera D, and camera F added to the movement route in the order of the movement route. Also, the selection of the image of camera A as the image to be displayed on main display unit 401 is indicated by the inverted color of the radio button.

[0082] (Camera B footage) Next, suppose that the tracking target person disappears from the image of camera A, and then the person appears in the image of camera B. In this case, tracking target identification unit 34 determines that the person has appeared in the image of camera B according to the flowchart shown in FIG.

[0083] 11B, main display section 401 displays the image from camera B. Sub-display section 403 displays the images from cameras A, B, E, D, and F in the order of the movement route. The selection of the image from camera B as the image to be displayed on main display section 401 is indicated by the color of the radio button being inverted.

[0084] (Footage from Camera E) Next, suppose that the tracking target person disappears from the image of camera B, and then the person appears in the image of camera E. In this case, tracking target identification unit 34 determines that the person has appeared in the image of camera E according to the flowchart shown in FIG.

[0085] 12C, main display section 401 displays the image from camera E. Sub-display section 403 displays the images from cameras A, B, E, D, and F in the order of the movement route. The selection of the image from camera E as the image to be displayed on main display section 401 is indicated by the color of the radio button being inverted.

[0086] (Camera D footage) Suppose that after a tracking target person disappears from the image of camera B, this person appears in the image of camera D. Camera D is not set on the movement route after camera B. Therefore, tracking target identification unit 34 determines that a person has appeared in the image of camera D according to the flowchart shown in FIG. 9, and display control unit 31 displays the image of camera D on main display unit 401. Furthermore, movement direction detection unit 36 ​​re-estimates the movement route according to the flowchart shown in FIG. 9. At this time, movement direction detection unit 36 ​​adds camera E, which has the highest weighting as shown in FIG. 6, of cameras E and F next to camera D, to the movement route.

[0087] 12D, main display section 401 displays the image from camera D. Then, sub-display section 403 displays the images from cameras A, B, D, and E along the newly estimated movement route in the order of the movement route. Furthermore, the selection of the image from camera D as the image to be displayed in main display section 401 is indicated by the inverted color of the radio button. Thereafter, although not shown, it is assumed that the person to be added moves outside the building. Note that, as shown in FIG. 5, cameras E and F are not adjacent to each other, so camera F is removed from the movement route. For this reason, the image from camera F is not included in sub-display section 403 in FIG. 12D.

[0088] <Route of movement of person moving from outside the building to inside the building> Next, examples of images displayed on the user terminal 1 when a person moves from outside the building to inside the building along the movement route shown in FIG. 10B will be described with reference to FIG. 13 and FIG.

[0089] (Footage from Camera F) As shown in Fig. 13A, when a person to be tracked is outside the building, the person is captured in the image from camera F. Here, person tracking function switch button 402 is displayed as "OFF", so the person tracking function is turned off. Also, in Fig. 13A, since the movement route has not yet been estimated, images from camera A, camera B, camera C, camera D, camera E, and camera F are displayed in that order in sub-display unit 403. Also, the selection of the image from camera F as the image to be displayed in main display unit 401 is indicated by the color of the radio button being inverted.

[0090] Next, when the user of user terminal 1 presses person tracking function switching button 402, as shown in Fig. 13B, person tracking function switching button 402 displays "Person Tracking ON". Then, movement direction detection unit 36 ​​adds camera B installed to the left of camera F, camera E with the highest weighting ahead of camera B, camera D at the destination, camera B at the destination, camera C, and camera A to the movement route in accordance with the flowchart shown in Fig. 8.

[0091] As shown in Fig. 10B, cameras B and C are installed at the movement destination from camera B. However, as shown in Fig. 6, camera B and camera C have the same weighting of "1". Therefore, the movement direction detection unit 36 ​​adds both camera B and camera C to the movement route. Therefore, two movement routes are inferred.

[0092] 13B, main display unit 401 displays the image from camera F. Then, sub-display unit 403 displays the images from cameras F, D, E, B, C, and A along the newly estimated movement route in the order of the movement route. Also, the selection of the image from camera F as the image to be displayed on main display unit 401 is indicated by the inverted color of the radio button.

[0093] (Camera D footage) Next, suppose that the tracking target person disappears from the image of camera F, and then the person appears in the image of camera D. In this case, tracking target identification unit 34 determines that the person has appeared in the image of camera D according to the flowchart shown in FIG.

[0094] 14C, main display section 401 displays the image from camera D. Sub-display section 403 displays the images from cameras F, D, E, B, C, and A in the order of the movement route. The selection of the image from camera D as the image to be displayed on main display section 401 is indicated by the color of the radio button being inverted.

[0095] (Footage from Camera E) Next, suppose that the tracking target person disappears from the image of camera D, and then the person appears in the image of camera E. In this case, tracking target identification unit 34 determines that the person has appeared in the image of camera E, which has the highest weighting, according to the flowchart shown in FIG. 9, and display control unit 31 displays the image of camera E on main display unit 401.

[0096] 14D, main display unit 401 displays the image from camera E. Sub-display unit 403 displays the images from cameras F, D, E, B, C, and A in the order of the movement route. The selection of the image from camera E as the image to be displayed on main display unit 401 is indicated by the color of the radio button being inverted.

[0097] <Re-estimated travel route> Next, an example of an image displayed on the user terminal 1 when the travel route shown in FIG. 10B is re-estimated and the travel route shown in FIG. 10C is estimated will be described with reference to FIG.

[0098] (Camera B footage) Suppose that after a tracking target person disappears from the image of camera D, the person appears in the image of camera B. Camera B is not the camera set on the movement route next to camera D. Therefore, tracking target identification unit 34 determines that a person has appeared in the image of camera B according to the flowchart shown in FIG. 9, and re-estimates the movement route. Then, display control unit 31 displays the image of camera B on main display unit 401. At this time, movement direction detection unit 36 ​​adds camera E, which has the highest weighting shown in FIG. 6, of cameras E and F next to camera B, to the movement route.

[0099] 15, main display unit 401 displays the image from camera B. Then, sub-display unit 403 displays the images from cameras F, D, B, and E along the newly estimated movement route in the order of the movement route. Also, the selection of the image from camera B as the image to be displayed on main display unit 401 is indicated by the color of the radio button being inverted.

[0100] In the video monitoring system 10 according to the embodiment described above, the video monitoring device 3 selects a video in which the tracking target is captured from the videos captured by the multiple cameras 50, and displays this video on the user terminal 1. Here, the video monitoring device 3 automatically tracks the suspicious person (tracking target), and displays a large image of the camera 50 showing the tracking target identified by the tracking target identification unit 34 on the user terminal 1. Therefore, the user can easily confirm the tracking target from the video of the camera 50 even on the user terminal 1.

[0101] As the suspicious person moves, the route the suspicious person took is inferred. Then, images captured by multiple cameras 50 installed along the route of movement are displayed in a list on the sub-display unit 403. This allows the user to simultaneously check multiple images displayed on the user terminal 1. If the user wishes to focus on the movements of a suspicious person captured in a certain image, the user can select this image to instruct the system to start tracking the suspicious person captured in this image. This allows the user to check the image of the suspicious person enlarged and displayed on the main display unit 401 with simple operations.

[0102] Furthermore, after an instruction to start tracking is given, the image is automatically switched so that the image shown on the main display unit 401 reflects the suspicious person. This makes it easier for the user to check the appearance of the suspicious person from the image displayed on the main display unit 401. Furthermore, the image displayed on the main display unit 401 is larger than the image displayed on the sub-display unit 403. This makes it easier for the user to check the image shown on the main display unit 401 reflecting the suspicious person, even if the display screen of the user terminal 1 is small.

[0103] Furthermore, the camera management unit 32 records positional relationship information 321. When the movement direction detection unit 36 ​​estimates the movement route of a suspicious individual from a video image showing the suspicious individual for which a user has instructed to start tracking, the movement direction detection unit 36 ​​automatically estimates the movement direction of the suspicious individual by using the positional relationship information 321. This allows the movement direction detection unit 36 ​​to estimate a movement route that matches the positional relationship of the cameras 50.

[0104] Furthermore, weighting information 322 is recorded in the camera management unit 32. A higher weighting is set for the camera 50 that captures an important area where the user wants to check the situation with particular attention than for the camera 50 that captures other areas. The movement route estimated by the movement direction detection unit 36 ​​preferentially includes the installation positions of the cameras 50 that capture areas with high weightings. This allows the user to monitor whether a suspicious person has entered an important area and, if so, what kind of behavior the person is engaged in.

[0105] Furthermore, if the suspicious person deviates from the estimated movement route after an instruction to start tracking the suspicious person is input from the user terminal 1, the movement direction detection unit 36 ​​re-estimates the movement route, starting from the camera 50 that captures the image that newly captures the suspicious person. Then, the display control unit 31 updates the image displayed on the sub-display unit 403 based on the re-estimated movement route so as to display a list of images captured by the cameras 50 installed on this movement route. This makes it easy for the user to continue monitoring the image in which the suspicious person is captured even if the suspicious person disappears from the image.

[0106] Moreover, the video of the camera 50 installed at a position off the moving route is not displayed on the sub-display unit 403. Therefore, the user only needs to check the sub-display unit 403 that displays the video of the camera 50 installed at a position along the moving route, and the burden on the user when checking the video is reduced.

[0107] [Variations] FIG. 16 is a diagram showing a modified example of the screen displayed on the user terminal 1. In FIG. The user terminal 1 includes a main display section 401, a person tracking function switching button 402, a sub-display section 403, and in addition, a movement route display section 404. The movement route display section 404 displays the movement route estimated by the movement direction detection section 36 and the code (A to F) representing the camera name of each camera together with the map of the premises. By checking the movement route displayed in the movement route display section 404, the user can specifically grasp the location within the premises of the suspicious person who instructed the start of tracking. In addition, the user can easily grasp what kind of movement route has been estimated.

[0108] The area monitored by the camera 50 is not limited to the corridors and rooms inside the building, but also includes the parking lot outside the building and the inside and outside of the premises.

[0109] Furthermore, the tracking targets whose actions are monitored by the video monitoring device 3 are not limited to people such as suspicious individuals. For example, moving objects such as drones, robots, and cars may be the tracking targets. In this way, moving objects including at least one of people and drones are the tracking targets, so the user can check the status of various types of tracking targets depending on the area to be monitored.

[0110] Furthermore, the size of the main display unit 401 may be approximately the same as that of the sub-display unit 403. If the user terminal 1 is a tablet terminal with a large screen, the position of the main display unit 401 may be fixed to a specific position within the screen (for example, the upper left), and the position of the sub-display unit 403 may be below the main display unit 401, so that the sub-display unit 403 arranges the images of the cameras 50 in a tiled manner. Even in this form, the user can correctly confirm the behavior of a suspicious person from the image on the main display unit 401.

[0111] In the above-described embodiment, the user selects the video of the camera 50 showing the tracking target, but the tracking target identification unit 34 may automatically identify the tracking target. The tracking target identification unit 34 may, for example, identify abnormal behavior and set a person engaging in abnormal behavior as a tracking target. The tracking target identification unit 34 may also set a person who intrudes during a time when no one is supposed to be present as a tracking target. By the tracking target identification unit 34 automatically identifying the tracking target in this way, the user does not need to keep watching the video displayed on the user terminal 1, and the monitoring burden on the user is reduced.

[0112] Incidentally, the present invention is not limited to the above-described embodiment, and it goes without saying that various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-mentioned embodiment describes the system configuration in detail and specifically in order to explain the present invention in an easily understandable manner, and is not necessarily limited to a system having all of the described configurations. In addition, it is also possible to add, delete, or replace part of the configuration of the present embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0113] 1...user terminal, 3...video monitoring device, 5...camera group, 10...video monitoring system, 31...display control unit, 32...camera management unit, 33...video acquisition unit, 34...tracking target identification unit, 35...camera selection unit, 36...movement direction detection unit, 50-56...camera, 321...positional relationship information, 322...weighting information, 401...main display unit, 402...person tracking function switching button, 403...sub-display unit

Claims

1. an image acquisition unit that acquires a plurality of images from a plurality of cameras; a tracking target identification unit that identifies a tracking target from the plurality of images acquired by the image acquisition unit; a camera management unit that records positional relationship information that correlates the installation positions of the plurality of cameras; a movement direction detection unit that detects a movement direction of the tracking target and estimates a movement route of the tracking target based on the positional relationship information recorded in the camera management unit; a camera selection unit that selects the image captured by the camera installed in the moving direction and in which the tracking target is captured; a display control unit that controls displaying the video selected by the camera selection unit on a user terminal, The display control unit displays, on the user terminal, a main display unit that displays the image selected by the camera selection unit and including the tracking target tracked by the tracking target identification unit, a sub-display unit that displays the multiple images sorted according to the moving direction detected by the moving direction detection unit in a list format in order, and a moving route display unit that displays the moving route, and displays, on the sub-display unit, the multiple images acquired by the image acquisition unit in the order of the cameras installed along the moving route; The tracking target is designated by selecting the image in which the tracking target appears from the plurality of images displayed on the sub-display unit, and when the image in which the tracking target appears is captured by a camera that is not set on the movement route, the movement direction detection unit re-estimates the movement route. Video surveillance equipment.

2. the camera management unit manages weighting information in which the installation positions of the cameras are weighted; The camera selection unit, when a plurality of cameras are installed on the movement route, selects the video captured by the camera with a large weight based on the weighting information recorded in the camera management unit.

2. The video monitoring device according to claim 1.

3. when the tracking target that has disappeared from the image on the main display unit is captured in the image captured by the camera that is not installed on the moving route, the moving direction detection unit again estimates the moving route starting from the camera that captures the image in which the tracking target is captured, The display control unit displays the plurality of images on the sub-display unit in the order of the cameras installed along the estimated movement route.

3. A video monitoring device according to claim 1 or 2.

4. The tracking target identification unit starts tracking of the tracking target based on a tracking start instruction received from the user terminal.

4. The video monitoring device according to claim 3.

5. The main display section displays the image larger than the sub-display section, and the sub-display section displays a plurality of the images in a list format. The video monitoring device according to any one of claims 1 to 4.

6. The tracking target is a moving object including at least one of a person and a drone. The video monitoring device according to any one of claims 1 to 5.

7. A process of acquiring multiple images from multiple cameras; A process of identifying a tracking target from the plurality of videos; A process of detecting a moving direction of the tracked object and estimating a moving route of the tracked object based on positional relationship information relating the installation positions of the plurality of cameras; A process of selecting the image captured by the camera installed in the moving direction and showing the tracking target; A process of controlling display of the selected video on a user terminal; A process of displaying on the user terminal a main display unit that displays the selected image including the tracking target, a sub-display unit that displays the multiple images sorted according to the detected movement direction in order in a list format, and a movement route display unit that displays the movement route, and displaying the multiple images on the sub-display unit in the order of the cameras installed along the movement route; A process of selecting an image in which the tracking target is shown from the plurality of images displayed on the sub-display unit, thereby indicating the tracking target; and a process of re-estimating the movement route when the image showing the tracking target is captured by the camera that is not set on the movement route. Display control method.

8. A video monitoring system including a user terminal and a video monitoring device, The user terminal includes: a display unit that displays a plurality of images received from the image monitoring device; The video monitoring device includes: an image acquisition unit that acquires a plurality of the images from a plurality of cameras; a tracking target identification unit that identifies a tracking target from the plurality of images acquired by the image acquisition unit; a camera management unit that records positional relationship information that correlates the installation positions of the plurality of cameras; a movement direction detection unit that detects a movement direction of the tracking target and estimates a movement route of the tracking target based on the positional relationship information recorded in the camera management unit; a camera selection unit that selects the image captured by the camera installed in the moving direction and in which the tracking target is captured; a display control unit that controls displaying the video selected by the camera selection unit on the user terminal, The display control unit displays, on the user terminal, a main display unit that displays the image selected by the camera selection unit and including the tracking target tracked by the tracking target identification unit, a sub-display unit that displays the multiple images sorted according to the moving direction detected by the moving direction detection unit in a list format in order, and a moving route display unit that displays the moving route, and displays, on the sub-display unit, the multiple images acquired by the image acquisition unit in the order of the cameras installed along the moving route; The tracking target is designated by selecting the image in which the tracking target appears from the plurality of images displayed on the sub-display unit, and when the image in which the tracking target appears is captured by a camera that is not set on the movement route, the movement direction detection unit re-estimates the movement route. Video surveillance system.

Images