Monitor target identification apparatus, monitor target identification system, and monitor target identification method

Abstract

To provide a monitor target identification apparatus, a monitor target identification system, and a monitor target identification method for allowing individual identification of a target included in a monitor image.SOLUTION: A monitor information management server 10 as a monitor target identification apparatus stores transmitter management data in which identification information of a transmitter 40 is associated with target identification information being information identifiably indicating a monitor target holding a transmitter 40. Also, the monitor information management server 10 specifies a position of a shooting target included in a monitor image, and specifies a position of the transmitter 40 which has transmitted the identification information received by a receiver 30 on the basis of a radio wave condition from the transmitter 40 detected by the receiver 30. Further, the monitor information management server 10 performs the individual identification of the monitor target included in the monitor image by specifying correspondence between the shooting target and the monitor target held by the transmitter 40 on the basis of the transmitter management data, a specified position of the shooting target, and a specified position of the transmitter 40.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a monitoring target identification device, a monitoring target identification system, and a monitoring target identification method used for monitoring a target by video.

Background Art

[0002] A system is known that uses a network camera, which is a photographing terminal connected to a network such as the Internet, for monitoring a target. The photographing terminal photographs a target area, and the photographed data is transmitted to a display terminal via a server connected to the network. Then, the display terminal displays a video based on the received photographed data. Thereby, the user can check the state of the target area using the display terminal.

[0003] Furthermore, systems that use various sensors such as infrared sensors in addition to the photographing terminal have also been proposed. In such systems, changes such as the entry and exit of people and equipment abnormalities in the target area are detected by sensors, and the user can check the state of the target area when these changes are detected by the video photographed by the photographing terminal (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, among various situations that require the above-described monitoring, there are situations where it is desired to monitor the position and movement of a predetermined person or article. For example, at a factory, a construction site, or the like, there is a demand to monitor who among a plurality of workers is working at which location. On the other hand, relying only on the video from the above-described imaging terminal to distinguish a person or article included in the video places a heavy burden on the user who checks the video. On the other hand, if an infrared sensor is used as in the system described in Patent Document 1, although the position and movement of a person can be detected, it is difficult to detect the position and movement of an article having the same temperature as the surroundings, and also, depending on the infrared sensor, individual identification of a person or article cannot be performed.

[0006] An object of the present invention is to provide a monitoring target identification device, a monitoring target identification system, and a monitoring target identification method that enable individual identification of a target included in a monitoring video.

Means for Solving the Problems

[0007] A monitoring target identification device for solving the above problems is connected to an imaging terminal that captures an area to be monitored and outputs it as a monitoring video, and a second communication device that receives identification information from a first communication device held by a monitoring target located within the imaging range of the imaging terminal by wireless communication at a short distance. The monitoring target identification device includes: a storage unit that stores management data in which the identification information of the first communication device is associated with target identification information that is information for identifying the monitoring target holding the first communication device; a monitoring video processing unit that specifies the position of a shooting target included in the monitoring video; a communication device specifying unit that specifies the position of the first communication device that transmitted the identification information received by the second communication device based on the state of radio waves from the first communication device detected by the second communication device; and an individual identification unit that performs individual identification of the monitoring target included in the monitoring video by specifying the correspondence between the shooting target and the monitoring target holding the first communication device based on the management data, the position of the shooting target specified by the monitoring video processing unit, and the position of the first communication device specified by the communication device specifying unit.

[0008] The monitoring target identification system for solving the above problems includes a photographing terminal that photographs the inside of a monitoring area and outputs it as a monitoring video, a first communication device held by a monitoring target, and a second communication device that receives the identification information from the first communication device located within the photographing range of the photographing terminal by wireless communication at a short distance, and a monitoring target identification device connected to the photographing terminal and the second communication device. The monitoring target identification device includes a storage unit that stores management data in which the identification information of the first communication device is associated with target identification information that is information indicating that the monitoring target holding the first communication device can be identified, a monitoring video processing unit that specifies the position of a photographing target included in the monitoring video, a communication device specifying unit that specifies the position of the first communication device that transmitted the identification information received by the second communication device based on the state of radio waves from the first communication device detected by the second communication device, and an individual identification unit that performs individual identification of the monitoring target included in the monitoring video by specifying the correspondence between the photographing target and the monitoring target holding the first communication device based on the management data, the position of the photographing target specified by the monitoring video processing unit, and the position of the first communication device specified by the communication device specifying unit.

[0009] The monitoring target identification method for solving the above problems includes: a photographing terminal photographing within a monitoring area and outputting a monitoring video, which is the video, to a monitoring target identification device; a first communication device held by a monitoring target transmitting identification information of the first communication device; a second communication device receiving the identification information from the first communication device located within the photographing range of the photographing terminal by short-range wireless communication, and outputting the identification information and information indicating the state of radio waves from the first communication device detected by the second communication device to the monitoring target identification device; the monitoring target identification device specifying the position of a photographing target included in the monitoring video; the monitoring target identification device specifying the position of the first communication device that transmitted the identification information received by the second communication device based on the information indicating the state of the radio waves; and the monitoring target identification device identifying the correspondence between the photographing target and the monitoring target holding the first communication device based on data in which the identification information of the first communication device is associated with target identification information that is information capable of identifying the monitoring target holding the first communication device, the specified position of the photographing target, and the specified position of the first communication device, thereby performing individual identification of the monitoring target included in the monitoring video.

[0010] According to the above configuration, by the cooperation of specifying the position of the photographing target included in the monitoring video and specifying the position of the monitoring target based on the state of radio waves from the first communication device, individual identification of the monitoring target included in the monitoring video becomes possible. Therefore, even in a situation where it is difficult to individually identify the monitoring target included in the monitoring video based only on the monitoring video, individual identification of the monitoring target is possible.

[0011] In the above configuration, the first communication device may be a transmitter that spontaneously emits radio waves, and the second communication device may be a receiver that receives the radio waves. According to the above configuration, compared with the case where the first communication device is activated by receiving radio waves from the second communication device, the constraints on the distance and direction in which communication between the first communication device and the second communication device is possible are smaller. Therefore, the versatility of the monitoring target identification system is enhanced.

[0012] In the monitoring target identification device, the monitoring target identification device may be connected to a display device, and may include a display data generation unit that generates display data of an image including the monitoring video and information indicating the target identification information of the monitoring target for which the correspondence between the monitoring video and the imaging target has been specified, in association with the position of the imaging target in the monitoring video, and an output unit that outputs the display data to the display device.

[0013] According to the above configuration, the user of the display device can easily grasp the individual identification of the monitoring target included in the monitoring video by viewing the image displayed on the display device. In the monitoring target identification device, when the imaging targets whose positions have been specified by the monitoring video processing unit include a target for which the correspondence with the monitoring target by the individual identification unit cannot be specified, a notification to the outside may be made.

[0014] According to the above configuration, an external monitor or the like can grasp that a target that does not hold the first communication device, that is, a target that is not assumed to exist in the monitoring area, has entered the monitoring area.

[0015] In the monitoring target identification device, when the imaging targets whose positions have been specified by the monitoring video processing unit include an unconfirmed target for which the correspondence with the monitoring target by the individual identification unit cannot be specified, the monitoring target identification device may be connected to a display device, and may include a display data generation unit that generates display data of an image including the monitoring video and information indicating the position of the unconfirmed target in the monitoring video, and an output unit that outputs the display data to the display device.

[0016] According to the above configuration, the user of the display device can visually grasp that a target that is not assumed to exist in the monitoring area has entered the monitoring area by viewing the image displayed on the display device. Therefore, it is easy to accurately proceed with countermeasures against the intrusion of such targets.

[0017] In the monitoring target identification device, artificial intelligence may be used for the specific processing of the imaging target by the monitoring video processing unit. According to the above configuration, the imaging target included in the monitoring video can be more accurately identified.

[0018] In the monitoring target identification device, the state of the radio wave may include the intensity and direction of the radio wave detected by the second communication device. According to the above configuration, since it is possible to calculate the distance and direction of the first communication device with respect to the second communication device, it is possible to accurately identify the position of the first communication device.

Effect of the Invention

[0019] According to the present invention, it is possible to perform individual identification of the target included in the monitoring video.

Brief Description of the Drawings

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0021] With reference to FIGS. 1 to 12, an embodiment of a monitoring target identification device, a monitoring target identification system, and a monitoring target identification method will be described. [Configuration of Monitoring Target Identification System] With reference to FIG. 1, the overall configuration of the monitoring target identification system will be described.

[0022] As shown in FIG. 1, the monitoring target identification system includes a monitoring information management server 10, which is an example of a monitoring target identification device, a shooting terminal 20, a receiver 30, a transmitter 40, a registration terminal 50, and a monitoring terminal 60. The transmitter 40 is an example of a first communication device, and the receiver 30 is an example of a second communication device. The number of each of the shooting terminal 20, the receiver 30, the transmitter 40, the registration terminal 50, and the monitoring terminal 60 may be one or more.

[0023] The monitoring information management server 10 is connected to the shooting terminal 20, the receiver 30, the registration terminal 50, and the monitoring terminal 60 via a network, and the monitoring information management server 10 transmits and receives data to and from each of these devices through the network. The network used for communication of each device may include a plurality of independent networks or may be a common single network. These networks may be general-purpose communication lines such as the Internet or dedicated communication lines for communication of each device.

[0024] The imaging terminal 20 has a video imaging function and a function of transmitting data to the monitoring information management server 10 through a network. The imaging terminal 20 is embodied as a network camera. The imaging terminal 20 stores the imaging terminal ID which is the identification information of the imaging terminal 20. The imaging terminal ID is unique information for each imaging terminal 20 and is pre-stored in the imaging terminal 20.

[0025] The imaging terminal 20 sequentially, that is, in real time, transmits the video data generated by imaging to the monitoring information management server 10 together with the imaging terminal ID stored in the imaging terminal 20.

[0026] The receiver 30 has a function of receiving data from the transmitter 40 by short-range wireless communication and a function of transmitting data to the monitoring information management server 10 through a network. As the communication standard for the wireless communication, Bluetooth (registered trademark) is used, and in particular, BLE (Bluetooth Low Energy) is preferably used. Each of the receiver 30 and the transmitter 40 corresponding to Bluetooth incorporates a battery, and the transmitter 40 spontaneously emits radio waves and the receiver 30 receives the radio waves.

[0027] Specifically, the receiver 30 receives the transmitter ID which is the identification information of the transmitter 40 from the transmitter 40. The range within which the radio waves emitted by the transmitter 40 reach the receiver 30 within the range with the receiver 30 as the reference is the reception range of the receiver 30. The receiver 30 receives the transmitter ID transmitted at a predetermined interval from the transmitter 40 located within the reception range of the receiver 30, using the radio waves emitted by the transmitter 40 as a carrier wave.

[0028] Also, the receiver 30 stores the receiver ID which is the identification information of the receiver 30. The receiver ID is unique information for each receiver 30 and is pre-stored in the receiver 30. When the receiver 30 receives the transmitter ID from the transmitter 40, it transmits the transmitter ID to the monitoring information management server 10 together with the receiver ID stored in the receiver 30. At this time, the receiver 30 also transmits to the monitoring information management server 10 the radio wave intensity information indicating the intensity of the radio wave from the transmitter 40 detected at the time of receiving the transmitter ID, and the radio wave direction information indicating the direction in which the radio wave has propagated.

[0029] The imaging terminal 20 and the receiver 30 are installed in a monitoring area which is an area to be monitored. The monitoring area is, for example, an area where workers perform various operations at a factory, a construction site, a medical site, etc., and is an area where the actions of the monitored objects, who are such workers, are desired to be monitored.

[0030] The monitoring area may include a plurality of partitioned rooms, or may be a single continuous space without partitions, or may include a plurality of spaced-apart spaces. Also, the monitoring area may be indoors or outdoors.

[0031] The receiver 30 is installed so that its reception range extends within the monitoring area. When the monitoring object identification system includes a plurality of receivers 30, the plurality of receivers 30 are installed such that the reception range of each receiver 30 is a different area within the monitoring area, and the combined range of the reception ranges of these receivers 30 extends over the entire monitoring area. Note that a part of the reception range of each receiver 30 may overlap with a part of the reception range of another receiver 30.

[0032] The range captured by the imaging terminal 20 is the capture range, and the imaging terminal 20 is installed so that its capture range extends within the monitoring area. When the monitoring object identification system includes a plurality of imaging terminals 20, the plurality of imaging terminals 20 are installed such that the capture range of each imaging terminal 20 is a different area within the monitoring area, and the combined range of the capture ranges of these imaging terminals 20 extends over the entire monitoring area. Note that a part of the capture range of each imaging terminal 20 may overlap with a part of the capture range of another imaging terminal 20.

[0033] The shooting range of each imaging terminal 20 overlaps with the reception range of one or more receivers 30. Hereinafter, as an example, the case where the shooting range of one imaging terminal 20 overlaps with the reception range of one receiver 30 will be described. That is, one imaging terminal 20 and one receiver 30 are associated with each other.

[0034] Specifically, for example, as shown in FIG. 1, a monitoring area is composed of two divided areas, area RA and area RB. In area RA, an imaging terminal 20 with a shooting terminal ID of C01 is installed. In area RB, an imaging terminal 20 with a shooting terminal ID of C02 is installed. That is, area RA is shot by the imaging terminal 20 of C01, and area RB is shot by the imaging terminal 20 of C02. And in area RA, a receiver 30 with a receiver ID of BR01 is installed. In area RB, a receiver 30 with a receiver ID of BR02 is installed. The reception range of the receiver 30 of BR01 substantially overlaps with the whole of area RA, that is, substantially overlaps with the whole of the shooting range of the imaging terminal 20 of C01. The reception range of the receiver 30 of BR02 substantially overlaps with the whole of area RB, that is, substantially overlaps with the whole of the shooting range of the imaging terminal 20 of C02. In this way, the imaging terminal 20 and the receiver 30 are associated with each other due to the overlap between the shooting range and the reception range.

[0035] The transmitter 40 has a function of transmitting data by wireless communication corresponding to the receiver 30. Specifically, the transmitter 40 continuously transmits radio waves that are carrier waves of data, and transmits the transmitter ID as the data at a predetermined interval. The transmitter ID functions as a signal for notifying the existence of the transmitter 40 itself to the surroundings of the transmitter 40. The transmitter ID is unique information for each transmitter 40 and is pre-stored in the transmitter 40.

[0036] The transmitter 40 is carried by a worker who is the monitoring target, that is, who works in the monitoring area. One transmitter 40 is assigned to each monitoring target. The registration terminal 50 is a terminal used for associating a monitoring target with the transmitter 40, and has a function of receiving data by wireless communication corresponding to the transmitter 40 and a function of transmitting and receiving data to and from the monitoring information management server 10 through a network. For example, the registration terminal 50 is embodied in a personal computer, a tablet terminal, or a smartphone.

[0037] The monitoring terminal 60 is an example of a display device. The monitoring terminal 60 is a terminal used for checking a monitoring video that is a video captured by the imaging terminal 20 within the monitoring area, and has a function of transmitting and receiving data to and from the monitoring information management server 10 through a network, and a function of displaying an image based on the data received from the monitoring information management server 10. The monitoring terminal 60 is embodied in, for example, a personal computer, a tablet terminal, or a smartphone.

[0038] The monitoring information management server 10 is a server that performs individual identification of a monitoring target included in a monitoring video based on the information received from the imaging terminal 20 and the receiver 30. Further, the monitoring information management server 10 transmits display data generated in response to a request from the monitoring terminal 60 to the monitoring terminal 60. The display data is data for causing the monitoring terminal 60 to display an integrated image that is an image including the monitoring video.

[0039] [Detailed Configuration of Each Device] With reference to FIG. 2, the detailed configurations of the monitoring information management server 10, the registration terminal 50, and the monitoring terminal 60 will be described.

[0040] The monitoring information management server 10 includes a communication unit 11, a storage unit 12, and a control unit 13. The communication unit 11 executes connection processing between the monitoring information management server 10 and each of the above-described devices through a network, and transmits and receives data between the monitoring information management server 10 and each device. The communication unit 11 functions as a receiving unit that receives information from the imaging terminal 20 and the receiver 30, and an output unit that outputs display data to the monitoring terminal 60.

[0041] The storage unit 12 has a configuration including a non-volatile memory such as a hard disk drive, and stores programs and data necessary for the processes executed by the control unit 13. As part of such data, the storage unit 12 stores receiver management data 12a, transmitter management data 12b, surveillance video data 12c, and detection data 12d.

[0042] The receiver management data 12a includes data for managing the association between the receiver 30 and the imaging terminal 20 where the reception range and the imaging range overlap. For example, in the receiver management data 12a, a receiver ID and an imaging terminal ID are associated with each other. Further, the receiver management data 12a includes data for managing the positions within the space where the receiver 30 and the imaging terminal 20 are located, for the receiver 30 and the imaging terminal 20 associated with each other, using, for example, a three-dimensional orthogonal coordinate system.

[0043] The transmitter management data 12b is data for managing the association between the transmitter 40 and the monitoring target holding the transmitter 40. In the transmitter management data 12b, a transmitter ID and target information which is information regarding the monitoring target are associated with each other.

[0044] The surveillance video data 12c is data for managing video data indicating the surveillance video captured by the imaging terminal 20 in association with the imaging terminal ID of the imaging terminal 20 that captured the surveillance video. The above video data includes time information indicating the time when the surveillance video was captured.

[0045] The detection data 12d includes information acquired by the surveillance information management server 10 from the receiver 30 during the capture of the surveillance video by the imaging terminal 20, and information acquired by the surveillance information management server 10 through the analysis of the above information and the surveillance video. Specifically, the detection data 12d includes the transmitter ID received by the receiver 30 and the period during which the reception of the transmitter ID was continued, the identification result of the monitoring target included in the surveillance video, and the like.

[0046] The control unit 13 has a configuration including a CPU and a volatile memory such as a RAM, and based on the programs and data stored in the storage unit 12, controls the communication by the communication unit 11, reads and writes information in the storage unit 12, performs various arithmetic processes, etc., and controls each part included in the monitoring information management server 10. Such a control unit 13 includes a management data registration unit 13a, a monitoring video processing unit 13b, a receiver information processing unit 13c, and a display data generation unit 13d.

[0047] The management data registration unit 13a associates the transmitter ID and the target information received by the monitoring information management server 10 from the registration terminal 50, and stores them in the storage unit 12 as transmitter management data 12b. Also, the management data registration unit 13a generates receiver management data 12a and stores it in the storage unit 12. The input of the receiver ID and the shooting terminal ID required for generating the receiver management data 12a may be performed through any computer terminal connected to the monitoring information management server 10 via a network.

[0048] The receiver management data 12a is constructed before the shooting of the monitoring area by the shooting terminal 20 is started, and the transmitter management data 12b is constructed before the monitoring target enters the monitoring area. The monitoring video processing unit 13b analyzes the monitoring video received by the monitoring information management server 10 from the shooting terminal 20, identifies the type of the shooting target included in the monitoring video, and specifies the position of the shooting target. For such identification, it is preferable to use artificial intelligence (AI: Artificial Intelligence). Specifically, for example, object detection (Real Time Object Detection) technologies such as YOLO are used. The type of the shooting target is, for example, a classification such as a person or a predetermined article, and the type of the identified shooting target may be preset in the artificial intelligence program. When the monitoring target is a person, the monitoring video processing unit 13b detects at least the shooting target that is a person among the targets included in the monitoring video.

[0049] Also, the monitoring video processing unit 13b associates the video data indicating the monitoring video with the shooting terminal ID, and stores it in the storage unit 12 as monitoring video data 12c. The receiver information processing unit 13c functions as a communication device identification unit and an individual identification unit. Specifically, the receiver information processing unit 13c uses the radio wave intensity information and radio wave direction information received by the monitoring information management server 10 from the receiver 30 to identify the position of the transmitter 40 corresponding to the transmitter ID received by the receiver 30. Further, the receiver information processing unit 13c compares the identification result of the position of the transmitter 40, the detection result of the shooting target by the monitoring video processing unit 13b, and the transmitter management data 12b to perform individual identification of the monitoring target included in the monitoring video.

[0050] In addition, the receiver information processing unit 13c stores the information received from the receiver 30 and the result of individual identification of the monitoring target in the storage unit 12 as detection data 12d. The monitoring video data 12c and the detection data 12d are sequentially constructed during the shooting of the monitoring area by the shooting terminal 20.

[0051] The display data generation unit 13d generates display data using each data stored in the storage unit 12 in response to a request from the monitoring terminal 60. The display data is transmitted to the monitoring terminal 60 via the communication unit 11.

[0052] Note that in the control unit 13, each of the management data registration unit 13a, the monitoring video processing unit 13b, the receiver information processing unit 13c, and the display data generation unit 13d may be separately embodied by various hardware such as a plurality of CPUs and a memory including a RAM, etc., and software that functions these, or may be embodied by software that gives a plurality of functions to one common hardware. Such software is stored in the storage unit 12 as a monitoring information management program.

[0053] The registration terminal 50 includes a communication unit 51 that controls communication, an operation unit 52 that accepts user operations, a display unit 53 that displays images, and a storage unit 54 that stores programs and data. It also includes a control unit 55 that performs control and arithmetic processing of these units based on programs and data stored in the storage unit 54. The storage unit 54 has a configuration including a non-volatile memory, and the control unit 55 has a configuration including a volatile memory such as a CPU and a RAM.

[0054] The communication unit 51 has a function of connecting the registration terminal 50 and the monitoring information management server 10 through a network and performing data transmission and reception between these devices, and a function of connecting the registration terminal 50 and the transmitter 40 using short-range wireless communication corresponding to the transmitter 40 and performing data transmission and reception between these devices.

[0055] The communication unit 51 receives a transmitter ID from the transmitter 40 by the above wireless communication. Also, target information is input to the registration terminal 50 through an operation on the operation unit 52, and the control unit 55 causes the communication unit 51 to transmit the transmitter ID and the target information to the monitoring information management server 10.

[0056] The monitoring terminal 60 includes a communication unit 61 that controls communication, an operation unit 62 that accepts user operations, a display unit 63 that displays images, a storage unit 64 that stores programs and data, and a control unit 65 that performs control and arithmetic processing of these units based on programs and data stored in the storage unit 64. The storage unit 64 has a configuration including a non-volatile memory, and the control unit 65 has a configuration including a volatile memory such as a CPU and a RAM.

[0057] The communication unit 61 has a function of connecting the monitoring terminal 60 and the monitoring information management server 10 through a network and performing data transmission and reception between these devices. Also, the communication unit 61 receives display data from the monitoring information management server 10, and the control unit 65 causes the display unit 63 to display an image using the display data.

[0058] Note that the functions of the control unit 55 of the registration terminal 50 and the control unit 65 of the monitoring terminal 60 may be separately embodied by various types of hardware such as a plurality of CPUs and a memory including a RAM, etc., and software that causes these to function, or may be embodied by software that gives a plurality of functions to a common single piece of hardware. Such software may be application software installed in each terminal, or may be application software provided as a web application from the monitoring information management server 10.

[0059] Also, the functions borne by the registration terminal 50 and the functions borne by the monitoring terminal 60 may be realized by one computer terminal. That is, the registration terminal 50 and the monitoring terminal 60 may be the same terminal.

[0060] [Configuration of Transmitter Management Data] With reference to FIG. 3, the data configuration of the transmitter management data 12b stored in the monitoring information management server 10 will be described.

[0061] As shown in FIG. 3, the transmitter management data 12b includes a transmitter ID and target information, and the transmitter ID and the target information are associated with each other. The target information includes target identification information that is information capable of identifying the monitoring target. The target identification information is, for example, a target ID that is identification information assigned to each worker who is the monitoring target, the name of the monitoring target, a nickname, etc. The target information may further include information indicating the attributes of the monitoring target in addition to the target identification information. Information indicating the attributes of the monitoring target includes, for example, the group to which the monitoring target belongs, the position or role of the monitoring target, etc.

[0062] In the transmitter management data 12b, the target information associated with the transmitter ID is information about the monitoring target to which the transmitter 40 having the transmitter ID is assigned and which holds the transmitter 40. For example, in the example shown in FIG. 3, the transmitter ID of BC01 is associated with the target ID of P01 and the name of the monitoring target, "Ichiro Akiba", as the target identification information in the target information. This indicates that the target ID assigned to the monitoring target that holds the transmitter 40 with the transmitter ID of BC01 is P01, and the name of the monitoring target is "Ichiro Akiba".

[0063] The target identification information differs for each transmitter ID. On the other hand, the information indicating the attributes of the monitoring target may be set commonly for a plurality of transmitter IDs. [Operation of Monitoring Target Identification System] With reference to FIGS. 4 to 12, the operation of the monitoring target identification system, that is, the procedure of the monitoring target identification method will be described. The operation of the monitoring target identification system includes the process of registering the transmitter management data 12b, the process of identifying the individual of the monitoring target, and the process of displaying the integrated image. Note that the receiver management data 12a is constructed at an appropriate timing before the shooting of the monitoring area by the shooting terminal 20 is started.

[0064] First, the process of registering the transmitter management data 12b will be described with reference to FIG. 4. The process of registering the transmitter management data 12b is, in other words, the process of registering the monitoring target, and is performed, for example, by the cooperation of the registration terminal 50 and the monitoring information management server 10 before an operator starts work in a factory or the like. The process shown in FIG. 4 is performed for each operator who is the monitoring target. The process of each step shown in FIG. 4 is implemented, for example, by displaying a screen for registering the monitoring target on the registration terminal 50 based on a predetermined operation, and an operator or a registration administrator performing an operation according to the instructions on this screen.

[0065] As shown in FIG. 4, target information is input to the registration terminal 50 through the operation of the registration terminal 50 (step S10). That is, information such as the name of the operator to be monitored is input to the registration terminal 50. Note that the target information input to the registration terminal 50 may be part of the target information associated with the transmitter ID in the transmitter management data 12b. For example, the target ID may not be input to the registration terminal 50 and may be assigned by the management data registration unit 13a of the monitoring information management server 10 at the time of registration of the transmitter management data 12b.

[0066] Subsequently, based on the transmitter 40 assigned to the operator to be monitored approaching the registration terminal 50, the registration terminal 50 acquires the transmitter ID of this transmitter 40 through short-range wireless communication (step S11).

[0067] Note that the order of input of the target information and acquisition of the transmitter ID is not limited to the above order, and the target information may be input to the registration terminal 50 after the registration terminal 50 has acquired the transmitter ID. When the target information and the transmitter ID are acquired, the registration terminal 50 transmits the target information and the transmitter ID to the monitoring information management server 10 through the network (step S12). When receiving the target information and the transmitter ID, the management data registration unit 13a of the monitoring information management server 10 associates the target information and the transmitter ID and stores them in the storage unit 12 as the transmitter management data 12b. At this time, the management data registration unit 13a assigns a target ID to each set of the target information and the transmitter ID, includes the target ID in the target information, and generates the transmitter management data 12b.

[0068] Thereby, the registration of the monitoring target, that is, the registration of the transmitter management data 12b is completed. The transmitter 40 used for registration is handed over to the operator who is the monitoring target, that is, the monitoring target corresponding to the target information input to the registration terminal 50, and is carried by the monitoring target.

[0069] Next, the process of individual identification of the monitoring target will be described with reference to FIGS. 5 to 9. As shown in FIG. 5, when the imaging terminal 20 starts imaging the monitoring area (step S20), it transmits the video data of the monitoring video generated by imaging, together with the imaging terminal ID of the imaging terminal 20, to the monitoring information management server 10 through the network (step S21).

[0070] Also, when the receiver 30 receives the transmitter ID from the transmitter 40 located within the reception range (step S22), it transmits the transmitter ID, the radio wave intensity information, and the radio wave direction information, together with the receiver ID of the receiver 30, to the monitoring information management server 10 (step S23). The transmission of the information including the transmitter ID from the receiver 30 to the monitoring information management server 10 is performed, for example, at a predetermined interval, for all the transmitter IDs received by the receiver 30 at that time point, by transmitting the transmitter ID, the radio wave intensity information, and the radio wave direction information to the monitoring information management server 10.

[0071] When receiving the video data of the monitoring video, the monitoring video processing unit 13b of the monitoring information management server 10 analyzes the monitoring video and detects the imaging objects included in the monitoring video for each type of imaging object (step S24). Thereby, in the monitoring video, the type and position of the imaging object are specified. Specifically, in the monitoring video, the position of the imaging object that is a person is specified.

[0072] When receiving information from the receiver 30, the receiver information processing unit 13c of the monitoring information management server 10 specifies the position of the transmitter 40 corresponding to the transmitter ID received by the receiver 30 using the radio wave intensity information and the radio wave direction information (step S25).

[0073] Subsequently, the receiver information processing unit 13c performs individual identification of the monitoring objects included in the monitoring video using the detection result of the imaging object in step S24, the specification result of the position of the transmitter 40 in step S25, and the transmitter management data 12b for the imaging terminal 20 and the receiver 30 that are associated with each other (step S26).

[0074] The monitoring information management server 10 stores, in the storage unit 12, various data received from the imaging terminal 20 and the receiver 30, and data such as the individual identification result of the monitoring target obtained by analyzing such data (step S27).

[0075] Note that the order of the processes of step S24 and step S25 is not particularly limited. The processes of step S24 to step S27 are repeatedly performed while the monitoring information management server 10 continues to receive information such as video data from the imaging terminal 20 and the receiver 30.

[0076] When the monitoring target identification system includes a plurality of imaging terminals 20, the processes of step S21 and step S24 are performed for each imaging terminal 20. When the monitoring target identification system includes a plurality of receivers 30, the processes of step S23 and step S25 are performed for each receiver 30. Then, for the imaging terminal 20 and the receiver 30 that are associated with each other, that is, the imaging terminal 20 and the receiver 30 whose imaging range and reception range overlap, the detection result of the imaging target in step S24 for the monitoring video captured by the imaging terminal 20 and the specific result of the position of the transmitter 40 in step S25 corresponding to the transmitter ID received by the receiver 30 are used to perform the individual identification of the monitoring target in step S26. The association between the imaging terminal 20 and the receiver 30 is specified based on the receiver management data 12a.

[0077] With reference to FIGS. 6 to 9, the details of the process of detecting the imaging target in step S24 and the process of specifying the position of the transmitter 40 in step S25 described in the flow of FIG. 5 above, and the process of individually identifying the monitoring target in step S26 will be described.

[0078] In the process of specifying the position of the transmitter 40, when the receiver 30 receives the transmitter ID, the radio wave intensity information and radio wave direction information of the radio wave from the transmitter 40 carrying the transmitter ID are used. Then, as shown in FIG. 6, the position of the transmitter 40 based on the position of the receiver 30 is specified. Specifically, by using the radio wave intensity information, the distance L from the receiver 30 to the transmitter 40 is calculated. Also, by using the radio wave direction information, in the three-dimensional orthogonal coordinate system centered on the receiver 30, the direction in which the transmitter 40 is located with respect to the receiver 30 is calculated. For example, when the x-axis direction and y-axis direction of the three-dimensional orthogonal coordinate system are set along the horizontal direction in the monitoring area and the z-axis direction is set along the vertical direction in the monitoring area, based on the calculation of the azimuth angle θ1 in the horizontal direction and the elevation angle θ2 in the vertical direction of the transmitter 40 with respect to the receiver 30, the direction in which the transmitter 40 is located with respect to the receiver 30 is obtained. Thereby, the position of the transmitter 40 with respect to the receiver 30 is specified.

[0079] Note that in FIG. 6, for ease of understanding, the elevation angle θ2 is shown as the elevation angle, but the receiver 30 is preferably installed above the monitoring area such as on the ceiling or the like. In this case, the elevation angle θ2 becomes the depression angle.

[0080] When the receiver 30 receives the transmitter IDs of a plurality of transmitters 40, for each transmitter ID, the position of the transmitter 40 corresponding to the transmitter ID is specified by using the radio wave intensity information and radio wave direction information. As shown in FIG. 7, even when there are a plurality of transmitters 40 in the same direction with respect to the receiver 30, due to the difference in the radio wave intensity information, the difference in the distance between the receiver 30 and each transmitter 40 can be detected, so that each transmitter 40 can be identified. Also, as shown in FIG. 8, even when there are a plurality of transmitters 40 at the same distance from the receiver 30, due to the difference in the radio wave direction information, the difference in the direction in which each transmitter 40 is located with respect to the receiver 30 can be detected, so that each transmitter 40 can be identified. Thus, by using the radio wave intensity information and radio wave direction information, the position of the transmitter 40 with respect to the receiver 30 can be accurately specified.

[0081] On the one hand, through the analysis of surveillance videos, the shooting targets included in the surveillance videos are detected for each type, and thereby, the positions of the shooting targets within the surveillance videos are identified. Based on this, as shown in FIG. 9, the position of the shooting target Ob with respect to the shooting terminal 20 is calculated. When a plurality of shooting targets Ob are included in the surveillance video, the positions of each shooting target Ob are identified.

[0082] For example, a three-dimensional orthogonal coordinate system centered on the shooting terminal 20 is preset in advance, whereby the position coordinates of fixed objects such as walls and pillars in the surveillance area are determined. When the shooting target Ob is detected, furthermore, the positional relationship of the shooting target Ob with respect to the fixed object is obtained by image analysis, whereby the position coordinates of the shooting target Ob are obtained.

[0083] In the process of individual identification of the surveillance target, the position of the transmitter 40 with respect to the receiver 30 and the position of the shooting target Ob with respect to the shooting terminal 20 are compared with each other, whereby the correspondence between the shooting target Ob and the transmitter 40 is identified. At this time, the shooting target Ob of the same type as the surveillance target held by the transmitter 40 is selected as the specific target. Specifically, when the surveillance target is a person, the shooting target Ob determined to be a person becomes the target whose correspondence with the transmitter 40 is identified.

[0084] For example, when the axial direction of the three-dimensional orthogonal coordinate system with the receiver 30 as the reference and the axial direction of the three-dimensional orthogonal coordinate system centered on the imaging terminal 20 are aligned, and the position of the transmitter 40 and the position of the imaging target Ob are close to each other, it is determined that the transmitter 40 and the imaging target Ob correspond to each other. The position of the imaging target Ob may be specified as a point or as a region. When the position of the imaging target Ob is specified as a point, for example, when the imaging target Ob is a person, the point at the approximate center of its head or torso may be used as the position of the imaging target Ob, or a rectangular parallelepiped, sphere, or ellipsoid surrounding the entire imaging target Ob may be set and its center of gravity may be used as the position of the imaging target Ob, or the position of the imaging target Ob may be determined based on the distribution of points located at the extracted outer shape or characteristic parts of the imaging target Ob. Also, when the position of the imaging target Ob is specified as a region, for example, a three-dimensional object such as a rectangular parallelepiped, sphere, or ellipsoid surrounding the entire imaging target Ob may be set, and the region where this three-dimensional object is located may be used as the position of the imaging target Ob, or the range based on the distribution of points located at the extracted outer shape or characteristic parts of the imaging target Ob may be used as the position of the imaging target Ob.

[0085] When the position of the imaging target Ob is specified as a point, for example, when the position of the transmitter 40 is included within a predetermined range from the position of the imaging target Ob, it is determined that the transmitter 40 and the imaging target Ob correspond to each other. When the position of the imaging target Ob is specified as a region, when the position of the transmitter 40 is included within the region, it is determined that the transmitter 40 and the imaging target Ob correspond to each other. Also, the position of the transmitter 40 may be specified as a point or as a region.

[0086] Note that the position of the imaging target Ob does not necessarily have to be defined as three-dimensional coordinates, and may be defined by a part of the distance and direction of the imaging target Ob with respect to the imaging terminal 20. For example, in a monitoring area, if the receiver 30 and the imaging terminal 20 are arranged on the same vertical axis, the axial direction of the coordinate system based on the receiver 30 and the axial direction of the coordinate system based on the imaging terminal 20 can be physically aligned. When the imaging terminal 20 is installed at the upper part of the monitoring area and the monitoring target is a person located on the ground, even if the elevation angle of the imaging target Ob with respect to the imaging terminal 20 in the vertical direction is not obtained, if the azimuth angle of the imaging target Ob with respect to the imaging terminal 20 in the horizontal direction is known, it is possible to compare with the position of the transmitter 40.

[0087] In this way, the position of the imaging target Ob with respect to the imaging terminal 20 is not limited to the above-described example, and may be defined so that it can be compared with the position of the transmitter 40 according to the arrangement of the imaging terminal 20 and the expected arrangement of the monitoring target.

[0088] The imaging target Ob and the transmitter 40 corresponding to each other indicate that the transmitter 40 emits radio waves from the position of the imaging target Ob, in other words, that the imaging target Ob holds the transmitter 40.

[0089] Furthermore, by referring to the transmitter management data 12b, the target information associated with the transmitter ID is specified. As a result, the correspondence between the imaging target Ob and the target identification information is specified via the transmitter ID, and as a result, the target identification information corresponding to the imaging target Ob, that is, the individual who is the monitoring target corresponding to the imaging target Ob is specified. Thus, the individual identification of the monitoring target included in the monitoring video is completed. Note that the specification of the target information associated with the transmitter ID may be performed before the correspondence between the imaging target Ob and the transmitter 40 is specified.

[0090] In the surveillance video, among the shooting targets Ob detected as the same type as the surveillance target, that is, people, there may be a shooting target Ob for which the correspondence with the transmitter 40 cannot be specified. In other words, there may be a shooting target Ob for which the transmitter 40 existing at the position of the shooting target Ob cannot be confirmed. In this case, it is preferable that the position of the shooting target Ob and information indicating that the corresponding transmitter 40 could not be confirmed are included in the detection data 12d and stored in the storage unit 12.

[0091] Next, the process of displaying the integrated image will be described with reference to FIG. 10. The process of displaying the integrated image is performed through the cooperation of the surveillance terminal 60 and the surveillance information management server 10. As shown in FIG. 10, through the operation of the surveillance terminal 60, an integrated image is requested from the surveillance terminal 60 to the surveillance information management server 10 (step S30). When the surveillance target identification system includes a plurality of shooting terminals 20, which shooting terminal 20 requests an integrated image based on the surveillance video captured may be determined by designating an area within the surveillance area at the surveillance terminal 60.

[0092] Upon receiving the request from the surveillance terminal 60, the display data generation unit 13d of the surveillance information management server 10 generates display data using each data stored in the storage unit 12 (step S31). The integrated image is an image that shows the surveillance video together with the individual identification results of the surveillance targets, and the display data is data for causing the surveillance terminal 60 to display such an integrated image.

[0093] The surveillance information management server 10 transmits the generated display data to the surveillance terminal 60 (step S32), and the surveillance terminal 60 uses the received display data to display the integrated image on the display unit (step S33). Thereby, the surveillant, who is the user of the surveillance terminal 60, can confirm the state of the surveillance targets in the surveillance area while individually identifying the surveillance targets.

[0094] Referring to FIG. 11, the configuration of the integrated image will be described. As shown in FIG. 11, the integrated image IM includes the surveillance video Ia captured by the imaging terminal 20 and the identification information Ib indicating the position of the surveillance target included in the surveillance video and the target identification information corresponding to the surveillance target. The identification information Ib indicates the position of the imaging target Ob in the surveillance video for the imaging target Ob and the target identification information that are specified as corresponding to each other as a result of the above-described individual identification process of the surveillance target, and indicates the target identification information in association with this position.

[0095] The display data generation unit 13d generates display data for displaying the integrated image IM so as to display the identification information Ib on the same screen as the surveillance video Ia. Such display data may be, for example, data for displaying the identification information Ib superimposed on the surveillance video Ia by synthesizing a plurality of layers, or data for displaying at least a part of the surveillance video Ia and the identification information Ib in different regions within one screen.

[0096] In the example shown in FIG. 11, the position of the surveillance target is clearly shown by a display surrounding the imaging target Ob included in the surveillance video Ia, and the target identification information, which is the name of the surveillance target, is displayed near the display. As a result, the monitor viewing the integrated image IM on the monitoring terminal 60 can identify each imaging target Ob included in the surveillance video as to who among the surveillance targets they are, while confirming the state of the surveillance targets.

[0097] Also, when there is an imaging target Ob among the imaging targets Ob detected as a person in the surveillance video for which the correspondence with the transmitter 40 cannot be confirmed, as shown in FIG. 12, it is preferable that the integrated image IM includes non-identification information Ic indicating that the position of the imaging target Ob and the correspondence identification information corresponding to the imaging target Ob cannot be specified. The display data generation unit 13d generates display data for displaying the integrated image IM so as to display the non-identification information Ic on the same screen as the surveillance video Ia.

[0098] The existence of a photographed object Ob for which correspondence with the transmitter 40 cannot be confirmed, that is, indicates that the photographed object Ob does not hold the transmitter 40, and suggests that the photographed object Ob is likely a person who is not assumed to be present in the monitoring area, such as an outsider. By including the non-identification information Ic in the integrated image IM, the monitor can easily grasp the existence of such a photographed object Ob, so that it is possible to early detect and take measures against the fact that a person who should not be present in the monitoring area has entered the monitoring area.

[0099] In addition, when a photographed object Ob for which correspondence with the transmitter 40 cannot be confirmed exists in the monitoring video, the monitoring information management server 10 may cause the monitoring terminal 60 to issue a warning by voice or text, or may notify the monitor by email or the like. In this way, the existence of a photographed object Ob for which correspondence with the transmitter 40 cannot be confirmed is notified to the outside by the display of the non-identification information Ic in the integrated image IM, the above warning, and the above notification.

[0100] As described above, according to the present embodiment, by the cooperation between specifying the position of the photographed object in the monitoring video and specifying the position of the monitoring object based on the position of the transmitter 40, individual identification of the monitoring object included in the monitoring video becomes possible. In other words, by associating the transmitter 40 with the photographed object whose position is specified in the monitoring video based on the position specified from the radio wave state, the correspondence between the photographed object and the monitoring object holding the transmitter 40 can be specified.

[0101] Then, together with the monitoring video, the individual identification result of the monitoring object is displayed on the monitoring terminal 60 as an integrated image. Therefore, even in a situation where it is difficult to individually identify the monitoring object included in the monitoring video by the monitoring video alone, the individual identification of the monitoring object is possible, and the monitoring object can be accurately monitored.

[0102] In particular, at work sites such as factories, construction sites, and medical sites, the clothing and accessories such as helmets worn by workers often look similar among different workers. Relying solely on videos taken at such work sites to distinguish the people included in the videos places a heavy burden on the monitor. Therefore, by using the monitoring target identification system of this embodiment, the burden on the monitor can be greatly reduced. Also, for example, although it is possible to perform individual identification of the monitored person based only on the surveillance video by using face recognition technology, there are significant constraints on the conditions required for shooting the surveillance video, such as the shooting direction of the monitored target, in order to identify the monitored target by face recognition. Also, when the monitored target is wearing common accessories required for work, such as a hat or a mask, the accuracy of identification inevitably decreases. In contrast, according to this embodiment, since individual identification of the monitored target is possible without being restricted by shooting conditions or the accessories of the monitored target, it is highly useful.

[0103] Also, in the above description, the case where the monitored target holding the transmitter 40 is a person has been exemplified. However, by attaching the transmitter 40 to an article, the article can be set as the monitored target, and by detecting the shooting target Ob of the type corresponding to the article in the surveillance video, individual identification of the article is also possible. That is, as long as the transmitter 40 can be held and detected in the surveillance video, the type of the monitored target is not limited. When the monitored target is an article, the target identification information included in the integrated image may be information that can identify the monitored target, such as a target ID or the name of the article. In this embodiment, in order to specify the position of the monitored target through the exchange of information between the transmitter 40 and the receiver 30 using short-range wireless communication, unlike the case of detecting the monitored target using an infrared sensor, not only people but also articles can be set as the monitored target.

[0104] In particular, by using BLE for short-range wireless communication, the receiver 30 can receive the radio waves of the transmitter 40 within a range with a maximum radius of about 20 m to 30 m centered on the receiver 30, and the reception range can be set within a desired range within this range. For example, as another example of a short-range wireless communication system, a passive RFID system using the HF band or the UHF band can be cited. In such a system, the range within which the receiver can detect the RFID tag acting as the transmitter, that is, the communication distance between the receiver and the RFID tag is at most about 1 m in the HF band and at most about 10 m in the UHF band. In comparison, by using BLE, the degree of freedom in setting the reception range of the receiver 30 is increased, and it is possible to secure a wider reception range. Therefore, even when the monitoring area is large, the number of receivers 30 required for the system can be reduced.

[0105] Also, in an RFID system, in order for the receiver to detect the RFID tag, the receiver and the RFID tag need to face each other, so there are large restrictions on the orientation between the receiver for detection and the RFID tag. On the other hand, in a configuration using BLE, the restrictions on the orientation between the receiver 30 for detection and the transmitter 40 are reduced, so it is possible to suppress an increase in the load required for setting the installation position of the receiver 30 and a decrease in the work efficiency of the operator being monitored for detection. Furthermore, in recent years, the number of terminals such as smartphones and tablet terminals compatible with BLE has been increasing, so the degree of freedom in selecting terminals that can be used as the registered terminal 50 is also increased. Therefore, the versatility of the monitoring target identification system is enhanced by using BLE.

[0106] [Other Forms Regarding Display of Integrated Images] When requesting an integrated image from the monitoring terminal 60 to the monitoring information management server 10, display conditions, which are conditions for generating the integrated image, may be specified. The display data generation unit 13d of the monitoring information management server 10 generates display data according to the display conditions.

[0107] Specifically, the display conditions include the designation of the viewing mode of the surveillance video and the conditions for extracting the surveillance video. The viewing mode is either the live mode or the archive mode. The live mode is a mode in which the surveillance video captured by the imaging terminal 20 is sequentially displayed on the monitoring terminal 60, that is, a mode in which the monitor can view the surveillance video being captured by the imaging terminal 20 in real time.

[0108] The archive mode is a mode in which the surveillance video captured by the imaging terminal 20 in the past and stored in the surveillance information management server 10 is displayed on the monitoring terminal 60, that is, a mode in which the monitor can view the surveillance video captured by the imaging terminal 20 in the past. By using the archive mode, the monitor can accurately confirm the locations and targets that need to be confirmed regarding the past surveillance areas and surveillance targets. Therefore, the convenience of the monitor is enhanced, and the past surveillance videos can be effectively utilized as records of the surveillance targets in the past surveillance areas.

[0109] The conditions for extracting the surveillance video include at least one of the information for designating an area within the surveillance area and the information for designating the surveillance target. The designation of an area within the surveillance area is performed, for example, through the designation of the imaging terminal ID or the receiver ID, and the designation of the surveillance target is performed, for example, through the designation of the target ID, the target identification information, or the transmitter ID. When the archive mode is designated, the conditions for extracting the surveillance video also include the time specification information for specifying the time when the surveillance video was captured.

[0110] When the live mode is designated as the display condition, the display data generation unit 13d generates display data using the video data sequentially received from the imaging terminal 20. When the archive mode is designated as the display condition, the display data generation unit 13d generates display data using the video data captured during the period specified by the time specification information among the video data stored in the storage unit 12 as the surveillance video data 12c.

[0111] In addition, when an area within the monitoring area is specified as a display condition, the display data generation unit 13d extracts the monitoring video of the specified area as the monitoring video to be used for generating the integrated image, and generates the display data of the integrated image using the individual identification result of the monitoring target for the monitoring video. When the receiver ID is specified as the specification of the area within the monitoring area, the imaging terminal 20 that has captured the monitoring video of the specified area can be identified by referring to the association between the receiver ID and the imaging terminal ID in the receiver management data 12a. According to such a configuration, since a specific area within the monitoring area can be specified and the monitoring video Ia of that area captured can be displayed on the monitoring terminal 60 together with the identification information Ib, the monitor can easily check the location to be confirmed.

[0112] In addition, when a monitoring target is specified as a display condition, the display data generation unit 13d extracts the monitoring video including the specified monitoring target as the monitoring video to be used for generating the integrated image, and generates the display data of the integrated image using the individual identification result of the monitoring target for the monitoring video. The imaging terminal 20 that has captured the monitoring video including the specified monitoring target can be identified by referring to the association between the target ID and target identification information and the transmitter ID in the transmitter management data 12b, the record of the transmitter ID received by the receiver 30 in the detection data 12d, and the association between the receiver ID and the imaging terminal ID in the receiver management data 12a. When the specified monitoring target moves within the imaging ranges of different imaging terminals 20, the display data is configured such that a video in which the monitoring videos captured by each imaging terminal 20 are joined together is displayed. According to such a configuration, since a specific monitoring target can be specified and the monitoring video Ia of that monitoring target captured can be displayed on the monitoring terminal 60 together with the identification information Ib, the monitor can easily check the monitoring target to be confirmed.

[0113] Furthermore, when an area within the monitoring area and a monitoring target are specified as display conditions, the monitoring video that includes the specified area and the specified monitoring target is extracted as the monitoring video to be used for generating the integrated image.

[0114] As described above, according to the present embodiment, the following effects can be obtained. (1) By the cooperation of specifying the position of the shooting target included in the surveillance video and specifying the position of the surveillance target based on the state of the radio wave from the transmitter 40, it is possible to identify the individual of the surveillance target included in the surveillance video.

[0115] (2) The integrated image IM displayed on the monitoring terminal 60 includes the monitoring video Ia and the identification information Ib that shows the identification information of the monitoring target whose correspondence with the shooting target is specified, in association with the position of the shooting target in the monitoring video. According to such a configuration, the monitor can easily grasp the individual identification of the monitoring target included in the monitoring video by looking at the integrated image.

[0116] (3) When the shooting targets whose positions are specified in the surveillance video include unidentified targets that are shooting targets whose correspondence with the surveillance target cannot be specified, a notification to the outside is made. With such a configuration, the monitor or the like can grasp that a target that is not supposed to exist in the surveillance area has invaded the surveillance area.

[0117] (4) If the non-identification information Ic indicating the position of the unidentified target in the surveillance video is included in the integrated image IM as the above-mentioned notification to the outside, the monitor can visually grasp that a target that is not supposed to exist in the surveillance area has invaded the surveillance area. Therefore, it is easy to accurately proceed with countermeasures against the intrusion of such targets.

[0118] (5) Artificial intelligence is used for the process of specifying the type and position of the shooting target in the surveillance video. According to such a form, the shooting target included in the surveillance video can be specified more accurately.

[0119] (6) The state of the radio wave used for specifying the position of the transmitter 40 includes the intensity and direction of the radio wave of the transmitter 40 detected by the receiver 30. According to such a configuration, since it is possible to calculate the distance and direction of the transmitter 40 with respect to the receiver 30, it is possible to accurately specify the position of the transmitter 40.

[0120] [Modification Example] The above embodiment can be implemented with the following modifications. · In the above embodiment, an example in which one imaging terminal 20 and one receiver 30 are associated with each other has been described. However, one imaging terminal 20 may be associated with a plurality of receivers 30, or a plurality of imaging terminals 20 may be associated with one receiver 30. In other words, one imaging terminal 20 may image the reception range of one receiver 30, or may image the reception ranges of a plurality of receivers 30. Further, the reception range of one receiver 30 may be imaged by a plurality of imaging terminals 20. In short, as long as the imaging terminal 20 and the receiver 30 are arranged and data for position calculation in the monitoring information management server 10 is set so that the imaging target and the monitoring target can be associated with each other by the cooperation between the identification of the position of the imaging target in the monitoring video and the identification of the position of the monitoring target based on the position of the transmitter 40.

[0121] · The integrated image may include a plurality of monitoring videos Ia and identification information Ib for each monitoring video Ia. For example, when a plurality of regions or a plurality of monitoring targets are specified as display conditions, or when a plurality of imaging terminals 20 are associated with one receiver 30, a plurality of monitoring videos are extracted, and display data of an integrated image including the plurality of monitoring videos is generated using the individual identification results of the monitoring targets for each monitoring video.

[0122] · In the above embodiment, an example in which the monitoring target identification device is a server has been described. However, the monitoring target identification device may be a computer terminal that is connected to the imaging terminal 20 and the receiver 30 by wire or wirelessly to exchange information.

[0123] ·In addition to the surveillance video and the target identification information, the integrated image displayed on the surveillance terminal 60 may include information indicating the attributes of the surveillance target associated with the transmitter ID together with this target identification information, information about the receiver 30 that has received the transmitter ID corresponding to this target identification information, and information about the imaging terminal 20 that has captured the surveillance video. That is, the display data generation unit 13d of the surveillance information management server 10 may generate display data so as to display this information within one screen.

[0124] ·In the above embodiment, the transmitter 40 that spontaneously transmits radio waves is used as the first communication device, and the receiver 30 that receives the radio waves is used as the second communication device. However, the second communication device is not limited to this, and it may be configured to be able to receive the identification information of the first communication device by short-range wireless communication, and the wireless communication may be wireless communication capable of specifying the position of the first communication device according to the state of the radio wave. For example, the first communication device may be activated by receiving radio waves from the second communication device, reflect the identification information of the first communication device to the second communication device, and as a result, the second communication device may receive the identification information of the first communication device. The first communication device is held by the surveillance target, and the second communication device is installed within the surveillance area.

[0125] Specifically, the wireless communication used for the exchange of information between the first communication device and the second communication device is not limited to BLE, and for example, it may be contactless communication using the HF band or UHF band used in an RFID system. In this case, as the first communication device, a communication medium such as an IC tag used in an RFID system is used, and as the second communication device, a communication device such as a reader / writer device used in an RFID system is used. Then, through command communication using the radio waves emitted by the communication device, the communication device receives a response including its identification information from the communication medium. The position of the communication medium can be specified from the intensity and direction of the radio wave when the communication device receives the response from the communication medium. Also, by installing a plurality of communication devices within the surveillance area and specifying the position of the communication medium from the state of the radio wave during communication between each communication device and the communication medium, highly accurate positioning of the position of the communication medium can also be realized.

Explanation of Reference Numerals

[0126] Ia… Surveillance video, Ib… Identification information, Ic… Non-identification information, IM… Integrated image, Ob… Object to be photographed, 10… Surveillance information management server, 11… Communication unit, 12… Memory unit, 12a… Receiver management data, 12b… Transmitter management data, 12c… Surveillance video data, 12d… Detection data, 13… Control unit, 13a… Management data registration unit, 13b… Surveillance video processing unit, 13c… Receiver information processing unit, 13d… Display data generation unit, 20… Photographing terminal, 30… Receiver, 40… Transmitter, 50… Registration terminal, 60… Surveillance terminal.

Claims

1. A monitoring target identification device connected to an imaging terminal that captures an image of a monitoring area and outputs the captured image as a monitoring video, a second communication device that receives identification information of a monitoring target located within an imaging range of the imaging terminal from a first communication device held by the monitoring target by short-distance wireless communication, and a display device, a storage unit that stores management data in which identification information of the first communication device, target identification information that is information that identifiably indicates the monitoring target that holds the first communication device, and a group to which the monitoring target belongs are associated with each other; a surveillance video processing unit that identifies a position of a person or other target included in the surveillance video; a communication device identification unit that identifies a location of the first communication device that has transmitted the identification information received by the second communication device based on a state of radio waves from the first communication device detected by the second communication device; an individual identification unit that identifies an individual of the surveillance target included in the surveillance video by identifying a correspondence between the surveillance target and the surveillance target having the first communication device based on the management data, the position of the surveillance target identified by the surveillance video processing unit, and the position of the first communication device identified by the communication device identification unit; a display data generating unit that generates display data of an image including the surveillance video and information indicating the group of the surveillance subjects whose correspondence with the shooting subject has been identified; an output unit that outputs the display data to the display device, The surveillance image processing unit specifies a position of the subject to be photographed as an area, When the location of the first communication device identified by the communication device identification unit is included in an area identified as the location of the photographing target, the individual identification unit determines that the photographing target corresponds to the monitoring target that holds the first communication device. Surveillance target identification device.

2. The image for which the display data is generated by the display data generating unit further includes information on the second communication device that has received the identification information of the first communication device held in the monitoring target, and information on the imaging terminal that has captured the monitoring video. The apparatus according to claim 1 .

3. When the surveillance image processing unit identifies a position of the captured image of the subject, and the individual identification unit cannot identify the correspondence with the surveillance subject, the surveillance image processing unit notifies the outside.

3. The monitored object identification device according to claim 1 or 2.

4. When an unconfirmed object whose correspondence with the surveillance object cannot be identified by the individual identification unit is included among the photographic objects whose positions have been identified by the surveillance video processing unit, the display data generating unit generates display data of an image including the surveillance video and information indicating the position of the unconfirmed object in the surveillance video.

3. The monitored object identification device according to claim 1 or 2.

5. Artificial intelligence is used in the process of identifying the subject to be photographed by the surveillance image processing unit. The monitored object identification device according to any one of claims 1 to 4.

6. A monitoring target identification device connected to an imaging terminal that captures an image of a monitoring area and outputs the captured image as a monitoring video, a second communication device that receives identification information of a monitoring target located within an imaging range of the imaging terminal from a first communication device held by the monitoring target by short-distance wireless communication, and a display device, a storage unit that stores management data in which identification information of the first communication device, target identification information that is information that identifiably indicates the monitoring target that holds the first communication device, and a group to which the monitoring target belongs are associated with each other; a surveillance video processing unit that identifies a position of a person or other target included in the surveillance video; a communication device identification unit that identifies a location of the first communication device that has transmitted the identification information received by the second communication device based on a state of radio waves from the first communication device detected by the second communication device; an individual identification unit that identifies an individual of the surveillance target included in the surveillance video by identifying a correspondence between the surveillance target and the surveillance target having the first communication device based on the management data, the position of the surveillance target identified by the surveillance video processing unit, and the position of the first communication device identified by the communication device identification unit; a display data generating unit that generates display data for an image including the surveillance video, information indicating the group of the surveillance subjects whose correspondence with the surveillance subjects has been identified, information of the second communication device that has received the identification information of the first communication device held in the surveillance subject, and information of the imaging terminal that has captured the surveillance video; an output unit that outputs the display data to the display device; Surveillance target identification device.

7. an imaging terminal that captures an image within a surveillance area and outputs the image as surveillance video; a first communication device held by a monitoring target; a second communication device that receives the identification information from the first communication device located within a photographing range of the photographing terminal by short-distance wireless communication; A surveillance target identification device connected to the photographing terminal and the second communication device; A monitoring target identification system comprising: a display device connected to the monitoring target identification device, The monitoring target identification device includes: a storage unit that stores management data in which identification information of the first communication device, target identification information that is information that identifiably indicates the monitoring target that holds the first communication device, and a group to which the monitoring target belongs are associated with each other; a surveillance video processing unit that identifies a position of a person or other target included in the surveillance video; a communication device identification unit that identifies a location of the first communication device that has transmitted the identification information received by the second communication device based on a state of radio waves from the first communication device detected by the second communication device; an individual identification unit that identifies an individual of the surveillance target included in the surveillance video by identifying a correspondence between the surveillance target and the surveillance target having the first communication device based on the management data, the position of the surveillance target identified by the surveillance video processing unit, and the position of the first communication device identified by the communication device identification unit; a display data generating unit that generates display data of an image including the surveillance video and information indicating the group of the surveillance subjects whose correspondence with the shooting subject has been identified; an output unit that outputs the display data to the display device, The surveillance image processing unit specifies a position of the subject to be photographed as an area, When the location of the first communication device identified by the communication device identification unit is included in an area identified as the location of the photographing target, the individual identification unit determines that the photographing target corresponds to the monitoring target that holds the first communication device. Surveillance Identification System.

8. an imaging terminal that captures an image within a surveillance area and outputs the image as surveillance video; a first communication device held by a monitoring target; a second communication device that receives the identification information from the first communication device located within a photographing range of the photographing terminal by short-distance wireless communication; A surveillance target identification device connected to the photographing terminal and the second communication device; A monitoring target identification system comprising: a display device connected to the monitoring target identification device, The monitoring target identification device includes: a storage unit that stores management data in which identification information of the first communication device, target identification information that is information that identifiably indicates the monitoring target that holds the first communication device, and a group to which the monitoring target belongs are associated with each other; a surveillance video processing unit that identifies a position of a person or other target included in the surveillance video; a communication device identification unit that identifies a location of the first communication device that has transmitted the identification information received by the second communication device based on a state of radio waves from the first communication device detected by the second communication device; an individual identification unit that identifies an individual of the surveillance target included in the surveillance video by identifying a correspondence between the surveillance target and the surveillance target having the first communication device based on the management data, the position of the surveillance target identified by the surveillance video processing unit, and the position of the first communication device identified by the communication device identification unit; a display data generating unit that generates display data for an image including the surveillance video, information indicating the group of the surveillance subjects whose correspondence with the surveillance subjects has been identified, information of the second communication device that has received the identification information of the first communication device held in the surveillance subject, and information of the imaging terminal that has captured the surveillance video; an output unit that outputs the display data to the display device; Surveillance Identification System.

9. The photographing terminal photographs the area to be monitored and outputs the photographed image to the monitoring target identification device; A first communication device held by a monitoring target transmits identification information of the first communication device; A second communication device receives the identification information from the first communication device located within a photographing range of the photographing terminal by short-distance wireless communication, and outputs the identification information and information indicating a state of radio waves from the first communication device detected by the second communication device to the monitored object identification device; The surveillance target identification device identifies a position of a target, which is a person, included in the surveillance video; The monitored object identification device identifies a position of the first communication device that transmitted the identification information received by the second communication device based on the information indicating the state of the radio wave; the monitoring target identification device identifies an individual of the monitoring target included in the surveillance video by identifying a correspondence between the photographed target and the monitoring target holding the first communication device based on data in which identification information of the first communication device, target identification information which is information identifiably indicating the monitoring target holding the first communication device, and a group to which the monitoring target belongs, the identified position of the photographed target, and the identified position of the first communication device; The monitoring target identification device generates display data of an image including the monitoring video and information indicating the group of the monitoring targets whose correspondence with the photographing target has been identified; The monitored object identification device outputs the display data to a display device; The surveillance target identification device identifies the position of the target to be photographed as an area, Identifying the correspondence between the subject to be photographed and the monitoring subject holding the first communication device includes determining that the subject to be photographed and the monitoring subject holding the first communication device correspond to each other when the specified position of the first communication device is included in an area specified as the position of the subject to be photographed. Target identification method.

10. The photographing terminal photographs the area to be monitored and outputs the photographed image to the monitoring target identification device; A first communication device held by a monitoring target transmits identification information of the first communication device; A second communication device receives the identification information from the first communication device located within a photographing range of the photographing terminal by short-distance wireless communication, and outputs the identification information and information indicating a state of radio waves from the first communication device detected by the second communication device to the monitored object identification device; The surveillance target identification device identifies a position of a target, which is a person, included in the surveillance video; The monitored object identification device identifies a position of the first communication device that transmitted the identification information received by the second communication device based on the information indicating the state of the radio wave; the monitoring target identification device identifies an individual of the monitoring target included in the surveillance video by identifying a correspondence between the photographed target and the monitoring target holding the first communication device based on data in which identification information of the first communication device, target identification information which is information identifiably indicating the monitoring target holding the first communication device, and a group to which the monitoring target belongs, the identified position of the photographed target, and the identified position of the first communication device; The monitoring target identification device generates display data of an image including information indicating the group of the monitoring targets whose correspondence with the monitoring video and the photographing targets has been identified, information of the second communication device that has received the identification information of the first communication device held in the monitoring target, and information of the photographing terminal that photographed the monitoring video; The monitored object identification device outputs the display data to a display device. Target identification method.

Images