A device for identifying monitored targets, a system for identifying monitored targets, and a method for identifying monitored targets.

Abstract

To provide a monitor target identification apparatus, a monitor target identification system, and a monitor target identification method for allowing individual identification of a target included in a monitor image.SOLUTION: A monitor information management server 10 as a monitor target identification apparatus stores transmitter management data in which identification information of a transmitter 40 is associated with target identification information being information identifiably indicating a monitor target holding a transmitter 40. Also, the monitor information management server 10 specifies a position of a shooting target included in a monitor image, and specifies a position of the transmitter 40 which has transmitted the identification information received by a receiver 30 on the basis of a radio wave condition from the transmitter 40 detected by the receiver 30. Further, the monitor information management server 10 performs the individual identification of the monitor target included in the monitor image by specifying correspondence between the shooting target and the monitor target held by the transmitter 40 on the basis of the transmitter management data, a specified position of the shooting target, and a specified position of the transmitter 40.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a monitoring target identification device, a monitoring target identification system, and a monitoring target identification method used for monitoring a target by video.

Background Art

[0002] A system that uses a network camera, which is a photographing terminal connected to a network such as the Internet, for monitoring a target is known. The photographing terminal photographs a target area to be monitored, and the photographed data is transmitted to a display terminal via a server connected to the network. Then, the display terminal displays a video based on the received photographed data. Thereby, the user can check the state of the target area using the display terminal.

[0003] Furthermore, in addition to the photographing terminal, systems that use various sensors such as an infrared sensor have also been proposed. In such systems, changes such as the entry and exit of people and equipment abnormalities in the target area are detected by the sensors, and the user can check the state of the target area when these changes are detected by the video photographed by the photographing terminal (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Incidentally, among the various situations requiring monitoring as described above, there are situations where it is desirable to monitor the location and movement of predetermined individuals or objects. For example, in factories and construction sites, there is a need to monitor which of multiple workers is working where. On the other hand, relying solely on the video from the aforementioned camera terminal to distinguish between individuals and objects in the video is burdensome for the user reviewing the footage. On the other hand, while infrared sensors, as in the system described in Patent Document 1 above, can detect the location and movement of individuals, it is difficult to detect the location and movement of objects that have the same temperature as the surroundings, and infrared sensors cannot individually identify individuals or objects.

[0006] The present invention aims to provide a surveillance target identification device, a surveillance target identification system, and a surveillance target identification method that enable individual identification of targets included in surveillance video. [Means for solving the problem]

[0007] A surveillance target identification device that solves the above problems is connected to a shooting terminal that takes pictures of the area within a surveillance area and outputs them as surveillance images, and a second communication device that receives identification information from a first communication device held by a surveillance target located within the shooting range of the shooting terminal via short-range wireless communication, and comprises a storage unit that stores management data associated with the identification information of the first communication device and target identification information which is information that identifies the surveillance target holding the first communication device; a surveillance image processing unit that identifies the location of the shooting target included in the surveillance images; a communication device identification unit that identifies the location of the first communication device that transmitted the identification information received by the second communication device based on the state of radio waves from the first communication device detected by the second communication device; and an individual identification unit that performs individual identification of the surveillance target included in the surveillance images by identifying the correspondence between the shooting target and the surveillance target holding the first communication device based on the management data, the location of the shooting target identified by the surveillance image processing unit, and the location of the first communication device identified by the communication device identification unit.

[0008] A surveillance target identification system that solves the above problems comprises: a shooting terminal that photographs the area within the surveillance area and outputs it as surveillance video; a first communication device held by the surveillance target; a second communication device that receives identification information from the first communication device located within the shooting range of the shooting terminal via short-range wireless communication; and a surveillance target identification device connected to the shooting terminal and the second communication device, wherein the surveillance target identification device stores management data in which the identification information of the first communication device and target identification information, which is information that identifies the surveillance target holding the first communication device, are associated. The system includes a memory unit, a surveillance video processing unit that identifies the location of a target to be filmed included in the surveillance video, a communication device identification unit that identifies the location of the first communication device that transmitted the identification information received by the second communication device based on the state of radio waves from the first communication device detected by the second communication device, and an individual identification unit that identifies individual targets of the surveillance video included in the surveillance video by identifying the correspondence between the target to be filmed and the surveillance target holding the first communication device based on the management data, the location of the target to be filmed identified by the surveillance video processing unit, and the location of the first communication device identified by the communication device identification unit.

[0009] A method for identifying a target being monitored that solves the above problem involves: a camera terminal capturing images within a monitoring area and outputting the resulting video to a target identification device; a first communication device held by the target being monitored transmitting identification information of the first communication device; a second communication device receiving the identification information from the first communication device located within the shooting range of the camera terminal via short-range wireless communication, and outputting the identification information and information indicating the state of the radio waves from the first communication device detected by the second communication device to the target identification device; and the target identification device identifying the location of the target being captured in the monitoring video. The monitoring target identification device includes identifying the location of the first communication device that transmitted the identification information received by the second communication device based on information indicating the state of the radio waves, and identifying the individual monitoring target included in the monitoring video by identifying the location of the shooting target and the monitoring target holding the first communication device based on data in which the identification information of the first communication device and target identification information which is information that identifies the monitoring target holding the first communication device are associated, the location of the identified shooting target and the location of the identified first communication device.

[0010] According to the above configuration, individual identification of the monitored object included in the surveillance video becomes possible through the coordination of identifying the location of the object captured in the surveillance video and identifying the location of the monitored object based on the location of the first communication device using the radio wave conditions from the first communication device. Therefore, even in situations where individual identification of the monitored object included in the surveillance video is difficult based on the surveillance video alone, individual identification of the monitored object becomes possible.

[0011] In the above configuration, the first communication device may be a transmitter that spontaneously emits radio waves, and the second communication device may be a receiver that receives those radio waves. With the above configuration, compared to the case where the first communication device is activated by receiving radio waves from the second communication device, there are fewer constraints on the distance and direction from which communication between the first and second communication devices can occur. Therefore, the versatility of the monitoring target identification system is increased.

[0012] In the above-described surveillance target identification device, the surveillance target identification device may also include a display data generation unit connected to a display device that generates display data of an image including the surveillance video and information indicating the target identification information of the surveillance target whose correspondence with the target being filmed has been identified, in correspondence with the position of the target being filmed in the surveillance video, and an output unit that outputs the display data to the display device.

[0013] According to the above configuration, users of the display device can easily identify individual monitored objects included in the surveillance video by viewing the images displayed on the display device. In the above-described surveillance target identification device, if the surveillance video processing unit determines the location of the target being filmed and there is a target that cannot be identified as corresponding to the surveillance target by the individual identification unit, the device may notify an external party.

[0014] According to the above configuration, external observers can detect when an object that does not possess the first communication device, that is, an object that is not expected to be in the monitoring area, has entered the monitoring area.

[0015] In the above-described surveillance target identification device, the surveillance target identification device may also include a display data generation unit connected to a display device, which generates display data of an image including the surveillance video and information indicating the position of the unidentified target in the surveillance video when the surveillance video processing unit has identified the location of the target being filmed and the target is an unidentified target which cannot be identified as corresponding to the surveillance target by the individual identification unit, and an output unit which outputs the display data to the display device.

[0016] With the above configuration, users of the display device can visually understand that an object not expected to be present in the monitoring area has intruded into the monitoring area by viewing the image displayed on the display device. Therefore, it becomes easier to take appropriate countermeasures against such intrusions.

[0017] In the above monitoring target identification device, artificial intelligence may be used for the specific processing of the shooting target by the monitoring video processing unit. According to the above configuration, the shooting target included in the monitoring video can be specified more accurately.

[0018] In the above monitoring target identification device, the state of the radio wave may include the intensity and direction of the radio wave detected by the second communication device. According to the above configuration, since it is possible to calculate the distance and direction of the first communication device with respect to the second communication device, it is possible to accurately specify the position of the first communication device.

Effect of the Invention

[0019] According to the present invention, it is possible to enable individual identification of the target included in the monitoring video.

Brief Description of the Drawings

[0020] [Figure 1] A diagram showing the overall configuration of a monitoring target identification system for one embodiment of the monitoring target identification system. [Figure 2] A diagram showing the configuration of each of the monitoring information management server, registration terminal, and monitoring terminal in one embodiment. [Figure 3] A diagram showing an example of transmitter management data in one embodiment. [Figure 4] A sequence chart showing the procedure for registering transmitter management data in a monitoring video display system in one embodiment. [Figure 5] A sequence chart showing the procedure for individual identification of a monitoring target in a monitoring video display system in one embodiment. [Figure 6] A diagram schematically showing the position of the transmitter specified in a monitoring video display system in one embodiment. [Figure 7] A diagram showing an example of the arrangement of transmitters in a monitoring video display system in one embodiment. [Figure 8] A diagram showing an example of the arrangement of transmitters in a monitoring video display system in one embodiment. [Figure 9]A diagram schematically showing the location of a target being filmed in a surveillance video display system according to one embodiment. [Figure 10] A sequence chart illustrating the procedure for displaying integrated images in a surveillance video display system according to one embodiment. [Figure 11] A diagram showing an example of an integrated image in a surveillance video display system according to one embodiment. [Figure 12] A diagram showing an example of an integrated image in a surveillance video display system according to one embodiment. [Modes for carrying out the invention]

[0021] An embodiment of a monitored target identification device, a monitored target identification system, and a monitored target identification method will be described with reference to Figures 1 to 12. [Configuration of the monitored target identification system] Referring to Figure 1, the overall configuration of the monitored target identification system will be explained.

[0022] As shown in Figure 1, the monitoring target identification system includes a monitoring information management server 10, a camera terminal 20, a receiver 30, a transmitter 40, a registration terminal 50, and a monitoring terminal 60, which are examples of monitoring target identification devices. The transmitter 40 is an example of a first communication device, and the receiver 30 is an example of a second communication device. The number of each of the camera terminal 20, receiver 30, transmitter 40, registration terminal 50, and monitoring terminal 60 can be one or more.

[0023] The monitoring information management server 10, the imaging terminal 20, the receiver 30, the registration terminal 50, and the monitoring terminal 60 are connected to a network, and the monitoring information management server 10 transmits and receives data from each of these devices via the network. The network used for communication between each device may include multiple independent networks or may be a single common network. These networks may be general-purpose communication lines such as the Internet, or they may be dedicated communication lines for communication between each device.

[0024] The recording terminal 20 has a video recording function and a function to transmit data to the monitoring information management server 10 via the network. The recording terminal 20 is embodied in a network camera. The recording terminal 20 stores a recording terminal ID, which is the identification information of the recording terminal 20. The recording terminal ID is unique to each recording terminal 20 and is stored in the recording terminal 20 in advance.

[0025] The recording terminal 20 transmits the video data generated by the recording, along with the recording terminal ID stored in the recording terminal 20, to the monitoring information management server 10 sequentially, i.e., in real time.

[0026] The receiver 30 has the function of receiving data from the transmitter 40 via short-range wireless communication and the function of transmitting data to the monitoring information management server 10 via the network. Bluetooth® is used as the communication standard for the above wireless communication, and in particular, BLE (Bluetooth Low Energy) is preferred. Both the Bluetooth-compatible receiver 30 and transmitter 40 have built-in batteries, and the transmitter 40 spontaneously emits radio waves, which the receiver 30 receives.

[0027] In detail, the receiver 30 receives the transmitter ID, which is the identification information of the transmitter 40, from the transmitter 40. The receiver 30's reception range is the range within which the radio waves emitted by the transmitter 40 reach the receiver 30, within the range relative to the receiver 30. The receiver 30 receives the transmitter ID, which is transmitted at predetermined intervals from the transmitter 40 located within the receiver 30's reception range, using the radio waves emitted by the transmitter 40 as the carrier wave.

[0028] Furthermore, the receiver 30 stores a receiver ID, which is the identification information for the receiver 30. The receiver ID is unique to each receiver 30 and is pre-stored in the receiver 30. When the receiver 30 receives a transmitter ID from the transmitter 40, it transmits the transmitter ID, along with the receiver ID stored in the receiver 30, to the monitoring information management server 10. At the same time, the receiver 30 also transmits to the monitoring information management server 10 radio wave strength information indicating the strength of the radio waves detected from the transmitter 40 when receiving the transmitter ID, and radio wave direction information indicating the direction from which the radio waves propagated.

[0029] The camera terminal 20 and receiver 30 are installed in the monitoring area, which is the area where monitoring takes place. The monitoring area is, for example, an area in a factory, construction site, medical facility, etc., where workers perform various tasks, and where it is desirable to monitor the actions of these workers who are the targets of monitoring.

[0030] The monitoring area may include multiple partitioned rooms, a single continuous space without partitions, or multiple spaces that are spaced apart from each other. Furthermore, the monitoring area may be indoors or outdoors.

[0031] The receiver 30 is installed so that its reception range extends within the monitoring area. If the monitoring target identification system includes multiple receivers 30, the receivers 30 are installed so that the reception range of each receiver 30 is in a different area within the monitoring area, and the combined reception range of these receivers 30 extends across the entire monitoring area. Note that a portion of the reception range of each receiver 30 may overlap with a portion of the reception range of another receiver 30.

[0032] The area captured by the imaging terminal 20 is the imaging range, and the imaging terminal 20 is installed so that its imaging range extends within the monitoring area. If the monitoring target identification system includes multiple imaging terminals 20, the imaging ranges of each imaging terminal 20 are located in different areas within the monitoring area, and the combined imaging ranges of these imaging terminals 20 extend across the entire monitoring area. Note that a portion of the imaging range of each imaging terminal 20 may overlap with a portion of the imaging range of another imaging terminal 20.

[0033] Each shooting range of the shooting terminal 20 overlaps with the reception range of one or more receivers 30. Below, as an example, we will explain the case where the shooting range of one shooting terminal 20 overlaps with the reception range of one receiver 30. That is, one shooting terminal 20 is associated with one receiver 30.

[0034] Specifically, as shown in Figure 1, for example, the monitoring area is composed of two partitioned areas, area RA and area RB. An imaging terminal 20 with imaging terminal ID C01 is installed in area RA, and an imaging terminal 20 with imaging terminal ID C02 is installed in area RB. That is, area RA is imaged by imaging terminal 20 C01, and area RB is imaged by imaging terminal 20 C02. A receiver 30 with receiver ID BR01 is installed in area RA, and a receiver 30 with receiver ID BR02 is installed in area RB. The reception range of receiver 30 BR01 almost overlaps with the entirety of area RA, that is, almost overlaps with the entirety of the imaging range of imaging terminal 20 C01. The reception range of receiver 30 BR02 almost overlaps with the entirety of area RB, that is, almost overlaps with the entirety of the imaging range of imaging terminal 20 C02. In this way, the imaging terminal 20 and the receiver 30 are associated with each other by the overlap of their imaging ranges and reception ranges.

[0035] The transmitter 40 has a function to transmit data via wireless communication corresponding to the receiver 30. Specifically, the transmitter 40 continuously transmits radio waves that serve as the data carrier, and transmits the transmitter ID as data at predetermined intervals. The transmitter ID functions as a signal to notify those around the transmitter 40 of its presence. The transmitter ID is unique information for each transmitter 40 and is stored in the transmitter 40 in advance.

[0036] The transmitter 40 is carried by the person being monitored, i.e., the worker performing the work in the monitored area. One transmitter 40 is assigned to each person being monitored. The registration terminal 50 is a terminal used to associate the monitored object with the transmitter 40, and has the function of receiving data via wireless communication corresponding to the transmitter 40, and the function of sending and receiving data to and from the monitoring information management server 10 via the network. For example, the registration terminal 50 can be embodied in a personal computer, tablet terminal, or smartphone.

[0037] The monitoring terminal 60 is an example of a display device. The monitoring terminal 60 is a terminal used to check the monitoring video, which is video captured by the camera terminal 20 within the monitoring area. It has the function of sending and receiving data to and from the monitoring information management server 10 via the network, and the function of displaying images based on the data received from the monitoring information management server 10. The monitoring terminal 60 can be embodied in, for example, a personal computer, a tablet terminal, or a smartphone.

[0038] The monitoring information management server 10 is a server that performs individual identification of monitored objects included in the monitoring video based on information received from the camera terminal 20 and the receiver 30. The monitoring information management server 10 also transmits display data generated in response to a request from the monitoring terminal 60 to the monitoring terminal 60. The display data is data for displaying the integrated image, which is an image including the monitoring video, on the monitoring terminal 60.

[0039] [Detailed configuration of each device] Referring to Figure 2, the detailed configurations of the monitoring information management server 10, registration terminal 50, and monitoring terminal 60 will be described.

[0040] The monitoring information management server 10 comprises a communication unit 11, a storage unit 12, and a control unit 13. The communication unit 11 performs connection processing between the monitoring information management server 10 and each of the above-mentioned devices via the network, and transmits and receives data between the monitoring information management server 10 and each of the devices. The communication unit 11 functions as a receiving unit that receives information from the imaging terminal 20 and the receiver 30, and as an output unit that outputs display data to the monitoring terminal 60.

[0041] The storage unit 12 has a configuration that includes non-volatile memory such as a hard disk drive, and stores programs and data necessary for processing executed by the control unit 13. As part of this data, the storage unit 12 stores receiver management data 12a, transmitter management data 12b, surveillance video data 12c, and detection data 12d.

[0042] The receiver management data 12a includes data for managing the correspondence between the receiver 30 and the imaging terminal 20, where the reception range and imaging range overlap. For example, in the receiver management data 12a, the receiver ID and the imaging terminal ID are associated. The receiver management data 12a also includes data for managing the positions of the receiver 30 and the imaging terminal 20, which are associated with each other, in the space in which the receiver 30 and the imaging terminal 20 are located, for example using a three-dimensional Cartesian coordinate system.

[0043] Transmitter management data 12b is data for managing the association between transmitter 40 and the monitored object that holds the transmitter 40. In transmitter management data 12b, the transmitter ID is associated with the monitored object, which is information about the monitored object.

[0044] The surveillance video data 12c is data that manages video data showing surveillance footage captured by the camera terminal 20, in association with the camera terminal ID of the camera terminal 20 that captured the surveillance footage. The video data includes time information indicating the time the surveillance footage was captured.

[0045] The detection data 12d includes information acquired by the monitoring information management server 10 from the receiver 30 while the camera terminal 20 is capturing surveillance video, and information acquired by the monitoring information management server 10 through analysis of the above information and the surveillance video. Specifically, the detection data 12d includes the transmitter ID received by the receiver 30, the period during which the reception of that transmitter ID continued, and the identification result of the monitored object included in the surveillance video.

[0046] The control unit 13 has a configuration that includes a CPU and volatile memory such as RAM, and controls various parts of the monitoring information management server 10, such as controlling communication by the communication unit 11, reading and writing information in the storage unit 12, and various calculation processes, based on programs and data stored in the storage unit 12. This control unit 13 includes a management data registration unit 13a, a monitoring video processing unit 13b, a receiver information processing unit 13c, and a display data generation unit 13d.

[0047] The management data registration unit 13a associates the transmitter ID received by the monitoring information management server 10 from the registration terminal 50 with the target information and stores it in the storage unit 12 as transmitter management data 12b. The management data registration unit 13a also generates receiver management data 12a and stores it in the storage unit 12. The input of the receiver ID and the shooting terminal ID required for generating the receiver management data 12a can be performed through any computer terminal connected to the monitoring information management server 10 via a network.

[0048] Receiver management data 12a is constructed before the imaging terminal 20 starts imaging the monitoring area, and transmitter management data 12b is constructed before the monitored object enters the monitoring area. The surveillance video processing unit 13b analyzes the surveillance video received by the surveillance information management server 10 from the shooting terminal 20 to identify the type of object being filmed in the surveillance video and to pinpoint the location of the object. It is preferable to use artificial intelligence (AI) for this identification. Specifically, for example, Real Time Object Detection (YLO) technology is used. The type of object being filmed refers to classifications such as people or specific objects, and the types of objects to be identified can be pre-set in the AI ​​program. If the object being filmed is a person, the surveillance video processing unit 13b detects at least one object in the surveillance video that is a person.

[0049] Furthermore, the surveillance video processing unit 13b associates video data representing the surveillance video with the shooting terminal ID and stores it in the storage unit 12 as surveillance video data 12c. The receiver information processing unit 13c functions as a communication device identification unit and an individual identification unit. Specifically, the receiver information processing unit 13c uses radio wave strength information and radio wave direction information received by the monitoring information management server 10 from the receiver 30 to identify the location of the transmitter 40 corresponding to the transmitter ID received by the receiver 30. Furthermore, the receiver information processing unit 13c compares the result of identifying the location of the transmitter 40 with the result of detecting the target being filmed by the monitoring video processing unit 13b and the transmitter management data 12b to perform individual identification of the monitored targets included in the monitoring video.

[0050] Furthermore, the receiver information processing unit 13c stores the information received from the receiver 30 and the results of individual identification of the monitored target as detection data 12d in the storage unit 12. The surveillance video data 12c and the detection data 12d are sequentially constructed while the shooting terminal 20 is capturing images of the monitored area.

[0051] The display data generation unit 13d generates display data using the data stored in the storage unit 12 in response to a request from the monitoring terminal 60. The display data is transmitted to the monitoring terminal 60 via the communication unit 11.

[0052] In the control unit 13, the management data registration unit 13a, the surveillance video processing unit 13b, the receiver information processing unit 13c, and the display data generation unit 13d may each be individually implemented by various hardware components such as multiple CPUs and RAM, and software to enable them to function, or they may be implemented by software that provides multiple functions to a single common piece of hardware. Such software is stored in the storage unit 12 as a surveillance information management program.

[0053] The registered terminal 50 includes a communication unit 51 for controlling communication, an operation unit 52 for receiving user input, a display unit 53 for displaying images, a storage unit 54 for storing programs and data, and a control unit 55 that controls these units and performs calculations based on the programs and data stored in the storage unit 54. The storage unit 54 has a configuration that includes non-volatile memory, and the control unit 55 has a configuration that includes a CPU and volatile memory such as RAM.

[0054] The communication unit 51 has the function of connecting the registered terminal 50 and the monitoring information management server 10 via a network and transmitting and receiving data between these devices, and the function of connecting the registered terminal 50 and the transmitter 40 using short-range wireless communication corresponding to the transmitter 40 and transmitting and receiving data between these devices.

[0055] The communication unit 51 receives the transmitter ID from the transmitter 40 via the above wireless communication. In addition, target information is input to the registration terminal 50 through operations on the operation unit 52, and the control unit 55 causes the communication unit 51 to transmit the transmitter ID and target information to the monitoring information management server 10.

[0056] The monitoring terminal 60 includes a communication unit 61 for controlling communications, an operation unit 62 for receiving user input, a display unit 63 for displaying images, a storage unit 64 for storing programs and data, and a control unit 65 that controls and performs calculations on each of these units based on the programs and data stored in the storage unit 64. The storage unit 64 has a configuration that includes non-volatile memory, and the control unit 65 has a configuration that includes a CPU and volatile memory such as RAM.

[0057] The communication unit 61 connects the monitoring terminal 60 and the monitoring information management server 10 via the network and has the function of sending and receiving data between these devices. The communication unit 61 also receives display data from the monitoring information management server 10, and the control unit 65 uses the display data to display an image on the display unit 63.

[0058] The functions of the control unit 55 of the registered terminal 50 and the control unit 65 of the monitoring terminal 60 may be individually implemented by various hardware components such as multiple CPUs and RAM, and software that enables them to function, or they may be implemented by software that provides multiple functions to a single common piece of hardware. Such software may be application software installed on each terminal, or it may be application software provided as a web application from the monitoring information management server 10.

[0059] Furthermore, the functions performed by the registration terminal 50 and the functions performed by the monitoring terminal 60 may be implemented by a single computer terminal. In other words, the registration terminal 50 and the monitoring terminal 60 may be the same terminal.

[0060] [Configuration of transmitter management data] Referring to Figure 3, the data structure of the transmitter management data 12b stored in the monitoring information management server 10 will be explained.

[0061] As shown in Figure 3, the transmitter management data 12b includes a transmitter ID and target information, with the transmitter ID and target information being associated with each other. The target information includes target identification information, which is information that identifies the monitored target. Target identification information may include, for example, a target ID, which is identification information assigned to each worker who is being monitored, or the name or nickname of the monitored person. In addition to target identification information, the target information may further include information indicating the attributes of the monitored person. Information indicating the attributes of the monitored person may include, for example, the group to which the monitored person belongs, or the position or role of the monitored person.

[0062] In the transmitter management data 12b, the target information associated with the transmitter ID is information about the monitored target to which the transmitter 40 with that transmitter ID is assigned and which possesses that transmitter 40. For example, in the example shown in Figure 3, the transmitter ID BC01 is associated with the target ID P01 and the name of the monitored target, "Ichiro Akiba," as target identification information in the target information. This indicates that the target ID assigned to the monitored target that possesses the transmitter 40 with the transmitter ID BC01 is P01, and the name of the monitored target is "Ichiro Akiba."

[0063] The target identification information differs for each transmitter ID. On the other hand, information indicating the attributes of the monitored target may be set in common for multiple transmitter IDs. [Operation of the monitored target identification system] Referring to Figures 4 to 12, the operation of the monitored target identification system, that is, the procedure for the monitored target identification method, will be explained. The operation of the monitored target identification system includes the process of registering transmitter management data 12b, the process of identifying individual monitored targets, and the process of displaying integrated images. The receiver management data 12a is constructed at an appropriate timing before the imaging terminal 20 starts capturing images of the monitored area.

[0064] First, the process of registering the transmitter management data 12b will be explained with reference to Figure 4. The process of registering the transmitter management data 12b is, in other words, the process of registering the monitored items, and is performed, for example, in a factory, before an employee starts work, through the cooperation of the registration terminal 50 and the monitoring information management server 10. The process shown in Figure 4 is performed for each employee to be monitored. The process of each step shown in Figure 4 is performed, for example, when a screen for registering the monitored items is displayed on the registration terminal 50 based on a predetermined operation, and the employee or the registration manager performs the operation according to the instructions on this screen.

[0065] As shown in Figure 4, target information is entered into the registration terminal 50 through the operation of the registration terminal 50 (step S10). That is, the name of the worker to be monitored is entered into the registration terminal 50. Note that the target information entered into the registration terminal 50 may be part of the target information associated with the transmitter ID in the transmitter management data 12b. For example, the target ID is not entered into the registration terminal 50, but is assigned by the management data registration unit 13a of the monitoring information management server 10 when the transmitter management data 12b is registered.

[0066] Next, based on the fact that the transmitter 40 assigned to the worker to be monitored is brought close to the registration terminal 50, the registration terminal 50 obtains the transmitter ID of the transmitter 40 via short-range wireless communication (step S11).

[0067] The order in which the target information is entered and the transmitter ID is obtained is not limited to the order described above; the target information may be entered into the registration terminal 50 after the registration terminal 50 has obtained the transmitter ID. Upon obtaining the target information and the transmitter ID, the registration terminal 50 transmits the target information and the transmitter ID to the monitoring information management server 10 via the network (step S12). Upon receiving the target information and the transmitter ID, the management data registration unit 13a of the monitoring information management server 10 associates the target information and the transmitter ID and stores them in the storage unit 12 as transmitter management data 12b. At this time, the management data registration unit 13a assigns a target ID to each pair of target information and transmitter ID, includes the target ID in the target information, and generates the transmitter management data 12b.

[0068] This completes the registration of the monitored target, i.e., the registration of the transmitter management data 12b. The transmitter 40 used for registration is passed to the worker who is the monitored target, i.e., the monitored target corresponding to the target information entered into the registration terminal 50, and is carried by the monitored target.

[0069] Next, the process of identifying the individual being monitored will be explained with reference to Figures 5 to 9. As shown in Figure 5, when the shooting terminal 20 starts shooting the monitoring area (step S20), it transmits the video data of the monitoring image generated by the shooting, along with the shooting terminal ID of the shooting terminal 20, to the monitoring information management server 10 via the network (step S21).

[0070] Furthermore, when the receiver 30 receives a transmitter ID from a transmitter 40 located within its reception range (step S22), it transmits the transmitter ID, radio wave strength information, and radio wave direction information, along with the receiver ID of the receiver 30, to the monitoring information management server 10 (step S23). This transmission of information, including the transmitter ID, from the receiver 30 to the monitoring information management server 10 is performed, for example, by transmitting the transmitter ID, radio wave strength information, and radio wave direction information to the monitoring information management server 10 for all transmitter IDs that the receiver 30 has received at that time, at predetermined intervals.

[0071] Upon receiving video data of the surveillance footage, the surveillance video processing unit 13b of the surveillance information management server 10 analyzes the surveillance footage and detects the subjects included in the surveillance footage for each type of subject (step S24). This identifies the type and location of the subjects in the surveillance footage. Specifically, the location of subjects that are people is identified in the surveillance footage.

[0072] When information is received from the receiver 30, the receiver information processing unit 13c of the monitoring information management server 10 uses the radio wave strength information and radio wave direction information to identify the location of the transmitter 40 corresponding to the transmitter ID received by the receiver 30 (step S25).

[0073] Next, the receiver information processing unit 13c uses the detection result of the target being filmed in step S24, the location identification result of the transmitter 40 in step S25, and the transmitter management data 12b to identify individual targets included in the surveillance video for the corresponding filming terminal 20 and receiver 30 (step S26).

[0074] The monitoring information management server 10 stores various data received from the camera terminal 20 and receiver 30, as well as data such as individual identification results of the monitored targets obtained by analyzing this data, in the storage unit 12 (step S27).

[0075] The order in which the processing in step S24 and the processing in step S25 are performed is not particularly limited. The processing in steps S24 to S27 is repeated as long as the monitoring information management server 10 continues to receive information such as video data from the shooting terminal 20 and the receiver 30.

[0076] Furthermore, if the surveillance target identification system includes multiple shooting terminals 20, the processes in steps S21 and S24 are performed for each shooting terminal 20, and if the surveillance target identification system includes multiple receivers 30, the processes in steps S23 and S25 are performed for each receiver 30. Then, for shooting terminals 20 and receivers 30 that are associated with each other, that is, shooting terminals 20 and receivers 30 whose shooting range and reception range overlap, the detection result of the target in step S24 for the surveillance video captured by the shooting terminal 20 and the location identification result of the transmitter 40 in step S25 corresponding to the transmitter ID received by the receiver 30 are used to perform individual identification of the surveillance target in step S26. The association between the shooting terminals 20 and receivers 30 is determined based on the receiver management data 12a.

[0077] Referring to Figures 6 to 9, we will now explain in detail the process of detecting the target to be photographed in step S24 and identifying the location of the transmitter 40 in step S25, as described in the flow chart of Figure 5 above, and the process of identifying the individual target to be monitored in step S26.

[0078] In the process of determining the position of the transmitter 40, the radio wave strength information and radio wave direction information of the radio wave from the transmitter 40 that carries the transmitter ID, when the receiver 30 receives the transmitter ID, are used. Then, as shown in Figure 6, the position of the transmitter 40 is determined with respect to the position of the receiver 30. Specifically, the distance L from the receiver 30 to the transmitter 40 is calculated using the radio wave strength information. In addition, the direction in which the transmitter 40 is located relative to the receiver 30 in a three-dimensional Cartesian coordinate system centered on the receiver 30 is calculated using the radio wave direction information. For example, if the x-axis and y-axis directions of the above three-dimensional Cartesian coordinate system are set to align with the horizontal direction in the monitoring area, and the z-axis direction is set to align with the vertical direction in the monitoring area, the direction in which the transmitter 40 is located relative to the receiver 30 is determined based on the calculation of the horizontal azimuth angle θ1 and vertical elevation angle θ2 of the transmitter 40 relative to the receiver 30. This determines the position of the transmitter 40 relative to the receiver 30.

[0079] Note that in Figure 6, the elevation angle θ2 is shown as the elevation angle for ease of understanding. However, it is preferable that the receiver 30 be installed in the upper part of the monitoring area, such as on the ceiling, in which case the elevation angle θ2 becomes the depression angle.

[0080] When the receiver 30 receives the transmitter IDs of multiple transmitters 40, the location of the transmitter 40 corresponding to each transmitter ID is determined using radio wave strength information and radio wave direction information. As shown in Figure 7, even if multiple transmitters 40 are located in the same direction relative to the receiver 30, the difference in radio wave strength information allows for the detection of the difference in distance between the receiver 30 and each transmitter 40, making it possible to identify each transmitter 40. Furthermore, as shown in Figure 8, even if multiple transmitters 40 are located at the same distance from the receiver 30, the difference in radio wave direction information allows for the detection of the difference in the direction in which each transmitter 40 is located relative to the receiver 30, making it possible to identify each transmitter 40. In this way, by using radio wave strength information and radio wave direction information, the location of the transmitter 40 relative to the receiver 30 can be accurately determined.

[0081] On the other hand, by analyzing the surveillance video, the types of subjects included in the video are detected, and the positions of the subjects within the surveillance video are determined. Based on this, as shown in Figure 9, the position of the subject Ob is calculated relative to the camera terminal 20. If the surveillance video contains multiple subjects Ob, the position of each subject Ob is determined.

[0082] For example, a three-dimensional Cartesian coordinate system centered on the imaging terminal 20 is pre-set, which determines the position coordinates of fixed objects such as walls and pillars in the monitoring area. When the target object Ob is detected, the positional relationship of the target object Ob with respect to the fixed objects is determined by image analysis, thereby obtaining the position coordinates of the target object Ob.

[0083] In the process of identifying individual monitored objects, the position of the transmitter 40 relative to the receiver 30 and the position of the object Ob relative to the imaging terminal 20 are compared to determine the correspondence between the object Ob and the transmitter 40. At this time, objects Ob of the same type as the monitored object holding the transmitter 40 are selected as the objects to be identified. Specifically, when the monitored object is a person, objects Ob determined to be a person are the objects whose correspondence with the transmitter 40 is identified.

[0084] For example, if the axis direction of a three-dimensional Cartesian coordinate system based on the receiver 30 is aligned with the axis direction of a three-dimensional Cartesian coordinate system centered on the imaging terminal 20, and the position of the transmitter 40 and the position of the object to be photographed Ob are close together, then it is determined that the transmitter 40 and the object to be photographed Ob correspond to each other. The position of the object to be photographed Ob may be specified as a point or as a region. When the position of the object to be photographed Ob is specified as a point, for example, if the object to be photographed Ob is a person, the position of the object to be photographed Ob may be a point determined to be approximately the center of the head or torso, or a cuboid, sphere, or ellipsoid enclosing the entire object to be photographed Ob may be set and its centroid may be used as the position of the object to be photographed, or the outline or characteristic parts of the object to be photographed Ob may be extracted and the position of the object to be photographed may be determined based on the distribution of points located therein. Furthermore, when the location of the object to be photographed (Ob) is specified as a region, for example, a three-dimensional object such as a rectangular prism, sphere, or ellipsoid may be defined to enclose the entire object to be photographed (Ob), and the region where this object is located may be defined as the location of the object to be photographed (Ob). Alternatively, the outline or characteristic parts of the object to be photographed (Ob) may be extracted, and the range based on the distribution of points located thereon may be defined as the location of the object to be photographed (Ob).

[0085] If the location of the object to be photographed (Ob) is specified as a point, for example, if the location of the transmitter 40 is included within a predetermined range from the location of the object to be photographed (Ob), then it is determined that the transmitter 40 and the object to be photographed (Ob) correspond. If the location of the object to be photographed (Ob) is specified as a region, then it is determined that the transmitter 40 and the object to be photographed (Ob) correspond if the location of the transmitter 40 is included within that region. Furthermore, the location of the transmitter 40 may also be specified as a point or as a region.

[0086] Furthermore, the position of the object to be photographed, Ob, does not necessarily have to be defined as a three-dimensional coordinate system; it may be defined by part of the distance and direction of Ob relative to the imaging terminal 20. For example, if the receiver 30 and the imaging terminal 20 are arranged on the same vertical axis in the monitoring area, the axis direction of the coordinate system based on the receiver 30 and the axis direction of the coordinate system based on the imaging terminal 20 can be physically aligned. If the imaging terminal 20 is installed at the top of the monitoring area and the object to be monitored is a person located on the ground, it is possible to compare the position of Ob with that of the transmitter 40 if the azimuth angle of Ob relative to the imaging terminal 20 in the horizontal direction is known, even without determining the elevation angle of Ob relative to the imaging terminal 20 in the vertical direction.

[0087] Thus, the position of the target Ob relative to the imaging terminal 20 is not limited to the example described above, but should be defined in such a way that it can be compared with the position of the transmitter 40, depending on the arrangement of the imaging terminal 20 and the expected arrangement of the monitored targets.

[0088] The corresponding objects to be photographed Ob and the transmitter 40 indicate that the transmitter 40 is emitting radio waves from the location of the object to be photographed Ob, or in other words, that the object to be photographed Ob is holding the transmitter 40.

[0089] Furthermore, by referring to the transmitter management data 12b, the target information associated with the transmitter ID is identified. This identifies the correspondence between the filmed object Ob and the target identification information via the transmitter ID, and as a result, the target identification information corresponding to the filmed object Ob, i.e., the individual who is the subject of surveillance corresponding to the filmed object Ob, is identified. With this, the individual identification of the subject of surveillance included in the surveillance video is completed. Note that the identification of the target information associated with the transmitter ID may be performed before the correspondence between the filmed object Ob and the transmitter 40 is identified.

[0090] Furthermore, in surveillance footage, among the filmed objects Ob detected as the same type as the surveillance target, i.e., as a person, there may be filmed objects Ob whose correspondence with the transmitter 40 cannot be identified, in other words, filmed objects Ob for which the transmitter 40 located at the location of the filmed object Ob cannot be confirmed. In this case, it is preferable that the location of the filmed object Ob and information indicating that the corresponding transmitter 40 could not be confirmed are included in the detection data 12d and stored in the storage unit 12.

[0091] Next, the process for displaying the integrated image will be explained with reference to Figure 10. The process for displaying the integrated image is carried out through cooperation between the monitoring terminal 60 and the monitoring information management server 10. As shown in Figure 10, an integrated image is requested from the monitoring information management server 10 through the operation of the monitoring terminal 60 (step S30). If the monitoring target identification system includes multiple imaging terminals 20, the monitoring terminal 60 may determine which imaging terminal 20 has captured the integrated image to request by specifying an area within the monitoring area.

[0092] When a request is received from the monitoring terminal 60, the display data generation unit 13d of the monitoring information management server 10 generates display data using the data stored in the storage unit 12 (step S31). The integrated image is an image that shows the monitoring video together with the individual identification result of the monitored target, and the display data is data for displaying such an integrated image on the monitoring terminal 60.

[0093] The monitoring information management server 10 transmits the generated display data to the monitoring terminal 60 (step S32), and the monitoring terminal 60 uses the received display data to display an integrated image on its display unit (step S33). This allows the monitor, who is a user of the monitoring terminal 60, to confirm the status of the monitored objects in the monitoring area while individually identifying each monitored object.

[0094] The structure of the integrated image will be explained with reference to Figure 11. As shown in Figure 11, the integrated image IM includes surveillance video Ia captured by the shooting terminal 20 and identification information Ib that indicates the location of the monitored object included in the surveillance video and the object identification information corresponding to the monitored object. The identification information Ib indicates the location of the captured object Ob and the object identification information that have been identified as corresponding to each other as a result of the individual identification process of the monitored object described above, and also indicates the object identification information in association with this location.

[0095] The display data generation unit 13d generates display data for displaying the integrated image IM so that the identification information Ib is displayed on the same screen as the surveillance video Ia. This display data may, for example, be data that displays the identification information Ib overlaid on the surveillance video Ia by combining multiple layers, or it may be data that displays the surveillance video Ia and at least a part of the identification information Ib in different areas within a single screen.

[0096] In the example shown in Figure 11, the location of the monitored object Ob included in the surveillance video Ia is clearly indicated by a display surrounding it, and object identification information, which is the name of the monitored object, is displayed near the display. As a result, the monitor viewing the integrated image IM on the surveillance terminal 60 can individually identify which of the monitored objects Ob included in the surveillance video is, and confirm the status of the monitored objects.

[0097] Furthermore, if there are any subjects Ob detected as people in the surveillance video that cannot be identified as corresponding to the transmitter 40, it is preferable that the integrated image IM includes non-identification information Ic indicating that the location of the subject Ob and the corresponding identification information for that subject Ob cannot be identified, as shown in Figure 12. The display data generation unit 13d generates display data for displaying the integrated image IM so that the non-identification information Ic is displayed on the same screen as the surveillance video Ia.

[0098] The presence of a subject Ob whose correspondence with the transmitter 40 cannot be confirmed indicates that the subject Ob does not possess the transmitter 40, suggesting that the subject Ob is likely a person who is not expected to be present in the surveillance area, such as an outsider. By including non-identifiable information Ic in the integrated image IM, the surveillance officer can easily grasp the presence of such subject Obs, thereby enabling early detection and action against individuals who should not be present in the surveillance area entering the surveillance area.

[0099] Furthermore, if there is a target object Ob in the surveillance video that cannot be identified as corresponding to the transmitter 40, the surveillance information management server 10 may have the surveillance terminal 60 issue an audio or text warning, or it may notify the monitor via email or other means. In this way, the display of non-identifiable information Ic in the integrated image IM, the above warning, and the above notification notify the outside that there is a target object Ob that cannot be identified as corresponding to the transmitter 40.

[0100] As described above, according to this embodiment, individual identification of monitored objects included in surveillance video becomes possible through the coordination of identifying the location of the object being filmed in the surveillance video and identifying the location of the monitored object based on the location of the transmitter 40. In other words, by associating the transmitter 40 with the object being filmed whose location has been identified in the surveillance video, based on the location identified from the radio wave conditions, the correspondence between the object being filmed and the monitored object holding the transmitter 40 can be identified.

[0101] Then, along with the surveillance video, the individual identification results of the monitored target are displayed on the surveillance terminal 60 as an integrated image. Therefore, even in situations where it is difficult to identify the individual monitored target based solely on the surveillance video, it is possible to identify the individual monitored target, and the monitored target can be accurately monitored.

[0102] In particular, in workplaces such as factories, construction sites, and medical facilities, the clothing and equipment worn by workers, such as helmets, are often similar among different workers. Relying solely on video footage of such workplaces to distinguish individuals within the footage is burdensome for supervisors. The surveillance target identification system of this embodiment can significantly reduce the burden on supervisors. Furthermore, while it is possible to identify individual surveillance targets using facial recognition technology based solely on surveillance footage, facial recognition imposes significant constraints on the conditions required for filming the surveillance footage, such as the direction in which the surveillance target is filmed. Moreover, if the surveillance targets are wearing common work-related equipment such as hats or masks, the accuracy of identification inevitably decreases. In contrast, this embodiment enables individual identification of surveillance targets without being constrained by filming conditions or the equipment worn by the surveillance targets, making it highly useful.

[0103] Furthermore, while the above description exemplified a case where the monitored object holding the transmitter 40 is a person, it is also possible to attach the transmitter 40 to an object to make the object the monitored object, and to detect the type of target Ob corresponding to the object in the monitored video, individual identification of the object is also possible. In other words, as long as the transmitter 40 can be held and detection in the monitored video is possible, the type of monitored object is not limited. When the monitored object is an object, the object identification information included in the integrated image can be any information that can identify the monitored object, such as the object ID or the name of the object. In this embodiment, the location of the monitored object is determined through the exchange of information between the transmitter 40 and the receiver 30 using short-range wireless communication, so unlike the case where an infrared sensor is used to detect the monitored object, not only people but also objects can be monitored.

[0104] In particular, by using BLE for short-range wireless communication, the receiver 30 can receive radio waves from the transmitter 40 within a radius of up to approximately 20m to 30m centered on the receiver 30, and the reception range can be set to a desired range within this range. For example, another example of a short-range wireless communication system is a passive RFID system using the HF band or UHF band. In such a system, the range in which the receiver can detect the RFID tag acting as a transmitter, that is, the communication distance between the receiver and the RFID tag, is approximately 1m at most in the HF band and approximately 10m at most in the UHF band. In comparison, by using BLE, the degree of freedom in setting the reception range of the receiver 30 is increased, and it is possible to secure a wider reception range, so that even when the monitoring area is large, the number of receivers 30 required for the system can be reduced.

[0105] Furthermore, in RFID systems, the receiver and RFID tag must be facing each other for the receiver to detect the RFID tag, thus imposing significant constraints on the orientation of the receiver and RFID tag for detection. In contrast, a configuration utilizing BLE reduces the constraints on the orientation of the receiver 30 and transmitter 40 for detection, thereby minimizing the burden required to set the placement of the receiver 30 and reducing the work efficiency of the monitored worker due to detection. Moreover, with the increasing number of BLE-compatible devices such as smartphones and tablets in recent years, the degree of freedom in selecting devices that can be used as registration terminals 50 is also increased. Therefore, the use of BLE enhances the versatility of the monitored object identification system.

[0106] [Other forms of displaying integrated images] When the monitoring terminal 60 requests an integrated image from the monitoring information management server 10, display conditions, which are the conditions for generating the integrated image, may also be specified. The display data generation unit 13d of the monitoring information management server 10 generates display data according to the display conditions.

[0107] Specifically, the display conditions include specifying the mode for viewing the surveillance video and the conditions for extracting the surveillance video. The viewing mode is either live mode or archive mode. Live mode is a mode in which the surveillance video being captured by the camera terminal 20 is displayed sequentially on the surveillance terminal 60, that is, a mode in which the monitor can view the surveillance video being captured by the camera terminal 20 in real time.

[0108] Archive mode is a mode in which the monitoring terminal 60 displays monitoring video previously captured by the recording terminal 20 and stored in the monitoring information management server 10, allowing the monitor to view monitoring video previously captured by the recording terminal 20. By using archive mode, the monitor can accurately confirm the locations and targets they wish to check regarding past monitoring areas and targets. Therefore, the convenience for the monitor is enhanced, and past monitoring video can be effectively used as a record of targets monitored in past monitoring areas.

[0109] The conditions for extracting surveillance footage include at least one of the following: information specifying the area within the surveillance domain, and information specifying the target of surveillance. The area within the surveillance domain is specified, for example, through the specification of the shooting terminal ID or receiver ID, and the target of surveillance is specified, for example, through the specification of the target ID, target identification information, or transmitter ID. In addition, if archive mode is specified, the conditions for extracting surveillance footage also include time specification information that specifies the time when the surveillance footage was captured.

[0110] When live mode is specified as the display condition, the display data generation unit 13d generates display data using video data that is sequentially received from the shooting terminal 20. When archive mode is specified as the display condition, the display data generation unit 13d generates display data using video data that was shot during the period specified by the time specification information from the video data stored in the storage unit 12 as surveillance video data 12c.

[0111] Furthermore, when a region within the monitoring area is specified as a display condition, the display data generation unit 13d extracts the monitoring video of the specified region as the monitoring video to be used to generate the integrated image, and generates display data for the integrated image using the individual identification result of the monitored object for said monitoring video. When a receiver ID is specified as the region within the monitoring area, the shooting terminal 20 that captured the monitoring video of the specified region can be identified by referring to the correspondence between the receiver ID and the shooting terminal ID in the receiver management data 12a. With this configuration, a specific region within the monitoring area can be specified, and the monitoring video Ia of that region can be displayed on the monitoring terminal 60 along with the identification information Ib, so that the monitor can easily confirm the location they want to check.

[0112] Furthermore, when a monitoring target is specified as a display condition, the display data generation unit 13d extracts the monitoring video containing the specified monitoring target as the monitoring video to be used for generating the integrated image, and generates display data for the integrated image using the individual identification result of the monitoring target for said monitoring video. The shooting terminal 20 that captured the monitoring video containing the specified monitoring target can be identified by referring to the correspondence between the target ID and target identification information and the transmitter ID in the transmitter management data 12b, the record of the transmitter ID received by the receiver 30 in the detection data 12d, and the correspondence between the receiver ID and the shooting terminal ID in the receiver management data 12a. If the specified monitoring target moves within the shooting range of different shooting terminals 20, the display data is configured so that an image is displayed which is a composite of the monitoring videos captured by each shooting terminal 20. With this configuration, a specific monitoring target can be specified, and the monitoring video Ia of that monitoring target can be displayed on the monitoring terminal 60 along with the identification information Ib, so that the monitor can easily confirm the monitoring target they want to check.

[0113] Furthermore, if the display conditions specify an area within the monitoring region and a target to be monitored, the monitoring video containing the specified area and the specified target to be monitored will be extracted as the monitoring video used to generate the integrated image.

[0114] As described above, the following effects can be obtained according to this embodiment. (1) By coordinating the identification of the location of the subject being filmed in the surveillance video with the identification of the location of the subject being monitored based on the location of the transmitter 40 using the radio wave conditions from the transmitter 40, it becomes possible to identify individual subjects being monitored in the surveillance video.

[0115] (2) The integrated image IM displayed on the monitoring terminal 60 includes a monitoring video Ia and identification information Ib which shows the identification information of the monitored object whose correspondence with the filmed object has been identified, in correspondence with the position of the filmed object in the monitoring video. With this configuration, the monitor can easily grasp the individual identification of the monitored objects included in the monitoring video by looking at the integrated image.

[0116] (3) If the surveillance footage includes unidentified objects whose location has been identified, and which cannot be identified as corresponding to the surveillance targets, an external notification will be sent. With this configuration, the monitors can be aware that objects that are not expected to be in the surveillance area have entered the surveillance area.

[0117] (4) If the integrated image IM includes non-identifiable information Ic indicating the location of the unidentified object in the surveillance video as the notification to the external party, the monitor can visually grasp that an object that is not expected to be present in the surveillance area has entered the surveillance area. Therefore, it is easier to take appropriate measures to deal with the intrusion of such an object.

[0118] (5) Artificial intelligence is used to identify the type and location of objects being filmed in surveillance footage. This configuration allows for more accurate identification of objects being filmed in surveillance footage.

[0119] (6) The radio wave conditions used to determine the location of the transmitter 40 include the intensity and direction of the radio waves from the transmitter 40 detected by the receiver 30. With this configuration, it is possible to calculate the distance and direction of the transmitter 40 relative to the receiver 30, and thus the location of the transmitter 40 can be accurately determined.

[0120] [Differentiation] The above embodiment can be implemented with the following modifications. In the above embodiment, an example was described in which one imaging terminal 20 is associated with one receiver 30. However, one imaging terminal 20 may be associated with multiple receivers 30, or multiple imaging terminals 20 may be associated with one receiver 30. In other words, one imaging terminal 20 may capture the reception range of one receiver 30, or it may capture the reception range of multiple receivers 30. Also, the reception range of one receiver 30 may be captured by multiple imaging terminals 20. In short, the imaging terminal 20 and receiver 30 should be arranged in such a way that the imaging target and the monitored target can be associated by coordinating the identification of the position of the target in the surveillance video with the identification of the position of the monitored target based on the identification of the position of the transmitter 40, and the data for position calculation should be set in the surveillance information management server 10.

[0121] The integrated image may include multiple surveillance video Ia and identification information Ib for each surveillance video Ia. For example, if multiple areas or multiple surveillance targets are specified as display conditions, or if multiple shooting terminals 20 are associated with one receiver 30, multiple surveillance video is extracted, and display data for an integrated image containing multiple surveillance video is generated using the individual identification results of the surveillance targets for each surveillance video.

[0122] • In the above embodiment, an example was described in which the monitored target identification device is a server, but the monitored target identification device can be any computer terminal that is connected to the shooting terminal 20 and the receiver 30 by wire or wireless connection to exchange information.

[0123] The integrated image displayed on the monitoring terminal 60 may include, in addition to the monitoring video and target identification information, information indicating the attributes of the monitored target associated with the transmitter ID along with this target identification information, information about the receiver 30 that received the transmitter ID corresponding to this target identification information, and information about the shooting terminal 20 that captured the monitoring video. In other words, the display data generation unit 13d of the monitoring information management server 10 may generate display data so that this information is displayed on a single screen.

[0124] In the above embodiment, a transmitter 40 that spontaneously emits radio waves is used as the first communication device, and a receiver 30 that receives the radio waves is used as the second communication device. However, the second communication device only needs to be configured to receive identification information of the first communication device by short-range wireless communication, and the wireless communication only needs to be wireless communication that can determine the location of the first communication device by the state of the radio waves. For example, the first communication device may be activated by receiving radio waves from the second communication device, reflect the identification information of the first communication device to the second communication device, and as a result, the second communication device may receive the identification information of the first communication device. The first communication device is held by the object being monitored, and the second communication device is installed within the monitoring area.

[0125] Specifically, the wireless communication used for exchanging information between the first and second communication devices is not limited to BLE; for example, it may be contactless communication using the HF or UHF band, as used in RFID systems. In this case, the first communication device is a communication medium such as an IC tag used in an RFID system, and the second communication device is a communication device such as a reader / writer used in an RFID system. The communication device receives a response containing its identification information from the communication medium via command communication using radio waves emitted by the communication device. The location of the communication medium can be determined from the strength and direction of the radio waves when the communication device receives the response from the communication medium. Furthermore, by installing multiple communication devices within the monitoring area and determining the location of the communication medium from the state of radio waves during communication between each communication device and the communication medium, highly accurate positioning of the communication medium can be achieved. [Explanation of symbols]

[0126] Ia... Surveillance video, Ib... Identification information, Ic... Non-identification information, IM... Integrated image, Ob... Target to be filmed, 10... Surveillance information management server, 11... Communication unit, 12... Storage unit, 12a... Receiver management data, 12b... Transmitter management data, 12c... Surveillance video data, 12d... Detection data, 13... Control unit, 13a... Management data registration unit, 13b... Surveillance video processing unit, 13c... Receiver information processing unit, 13d... Display data generation unit, 20... Filming terminal, 30... Receiver, 40... Transmitter, 50... Registration terminal, 60... Surveillance terminal.

Claims

1. A monitoring target identification device connected to a display device, comprising: a shooting terminal that captures images within a monitoring area and outputs them as monitoring images; a second communication device that receives identification information from a first communication device held by a monitoring target located within the shooting range of the shooting terminal via short-range wireless communication; and a display device, A storage unit that stores management data associated with the identification information of the first communication device, the target identification information which is information that identifies the monitored object holding the first communication device, and the group to which the monitored object belongs. A surveillance video processing unit that identifies the location of the subject, which is a person included in the aforementioned surveillance video, A communication device identification unit that identifies the location of the first communication device that transmitted the identification information received by the second communication device, based on the state of radio waves from the first communication device detected by the second communication device, An individual identification unit performs individual identification of the monitored object included in the surveillance video by identifying the correspondence between the captured object and the monitored object holding the first communication device based on the management data, the location of the captured object identified by the surveillance video processing unit, and the location of the first communication device identified by the communication device identification unit, A display data generation unit generates image display data including the aforementioned surveillance video and information indicating the group of surveillance targets whose correspondence with the subject being filmed has been identified. The system includes an output unit that outputs the display data to the display device, The aforementioned surveillance video processing unit identifies the location of the object to be filmed as a region, The individual identification unit determines that the object being photographed and the monitored object holding the first communication device correspond when the location of the first communication device identified by the communication device identification unit is included in the region identified as the location of the object being photographed. A device for identifying monitored targets.

2. The image for which the display data is generated by the display data generation unit further includes information of the second communication device that received the identification information of the first communication device held by the monitored object, and information of the shooting terminal that captured the monitored video. The monitoring target identification device according to claim 1.

3. When the surveillance video processing unit determines the location of an object to be filmed, and that object does not correspond to the surveillance object as determined by the individual identification unit, it notifies an external party. The monitoring target identification device according to claim 1 or 2.

4. The display data generation unit generates an image display data that includes the surveillance video and information indicating the location of the unidentified object in the surveillance video when the surveillance video processing unit has identified the location of the target being filmed and the target that cannot be identified in correspondence with the surveillance target by the individual identification unit. The monitoring target identification device according to claim 1 or 2.

5. Artificial intelligence is used in the process of identifying the target being filmed by the aforementioned surveillance video processing unit. A monitoring target identification device according to any one of claims 1 to 4.

6. A monitoring target identification device connected to a display device, comprising: a shooting terminal that captures images within a monitoring area and outputs them as monitoring images; a second communication device that receives identification information from a first communication device held by a monitoring target located within the shooting range of the shooting terminal via short-range wireless communication; and a display device, A storage unit that stores management data associated with the identification information of the first communication device, the target identification information which is information that identifies the monitored object holding the first communication device, and the group to which the monitored object belongs. A surveillance video processing unit that identifies the location of the subject, which is a person included in the aforementioned surveillance video, A communication device identification unit that identifies the location of the first communication device that transmitted the identification information received by the second communication device, based on the state of radio waves from the first communication device detected by the second communication device, An individual identification unit performs individual identification of the monitored object included in the surveillance video by identifying the correspondence between the captured object and the monitored object holding the first communication device based on the management data, the location of the captured object identified by the surveillance video processing unit, and the location of the first communication device identified by the communication device identification unit, A display data generation unit generates image display data including the aforementioned surveillance video, information indicating the group of surveillance targets whose correspondence with the subject being filmed has been identified, information of the second communication device that received the identification information of the first communication device held in the surveillance target, and information of the filming terminal that filmed the surveillance video. The system includes an output unit that outputs the display data to the display device, The aforementioned monitoring target identification device is connected to a plurality of the aforementioned shooting terminals, The aforementioned display data generation unit, The device receives the designation of the target to be monitored from the display device, extracts the monitoring video including the designated target to generate the display data, When the designated monitored object moves within the shooting range of different shooting terminals, the display data is generated so that an image including the combined footage of the monitoring images captured by each of these shooting terminals is displayed on the display device. When the designated monitored object is being captured by multiple of the capturing terminals, the display data is generated so that an image including multiple of the monitoring images captured by these capturing terminals is displayed on the display device. A device for identifying monitored targets.

7. A camera terminal that captures images within the surveillance area and outputs them as surveillance video, The first communication device held by the monitored object, A second communication device receives identification information from the first communication device located within the shooting range of the aforementioned shooting terminal via short-range wireless communication, The aforementioned imaging terminal and the second communication device are connected to a monitoring target identification device, A monitoring target identification system comprising a display device connected to the aforementioned monitoring target identification device, The aforementioned monitoring target identification device is A storage unit that stores management data associated with the identification information of the first communication device, the target identification information which is information that identifies the monitored object holding the first communication device, and the group to which the monitored object belongs. A surveillance video processing unit that identifies the location of the subject, which is a person included in the aforementioned surveillance video, A communication device identification unit that identifies the location of the first communication device that transmitted the identification information received by the second communication device, based on the state of radio waves from the first communication device detected by the second communication device, An individual identification unit performs individual identification of the monitored object included in the surveillance video by identifying the correspondence between the captured object and the monitored object holding the first communication device based on the management data, the location of the captured object identified by the surveillance video processing unit, and the location of the first communication device identified by the communication device identification unit, A display data generation unit generates image display data including the aforementioned surveillance video and information indicating the group of surveillance targets whose correspondence with the subject being filmed has been identified. The system includes an output unit that outputs the display data to the display device, The aforementioned surveillance video processing unit identifies the location of the object to be filmed as a region, The individual identification unit determines that the object being photographed and the monitored object holding the first communication device correspond when the location of the first communication device identified by the communication device identification unit is included in the region identified as the location of the object being photographed. A system for identifying monitored targets.

8. A camera terminal that captures images within the surveillance area and outputs them as surveillance video, The first communication device held by the monitored object, A second communication device receives identification information from the first communication device located within the shooting range of the aforementioned shooting terminal via short-range wireless communication, The aforementioned imaging terminal and the second communication device are connected to a monitoring target identification device, A monitoring target identification system comprising a display device connected to the aforementioned monitoring target identification device, The aforementioned monitoring target identification device is A storage unit that stores management data associated with the identification information of the first communication device, the target identification information which is information that identifies the monitored object holding the first communication device, and the group to which the monitored object belongs. A surveillance video processing unit that identifies the location of the subject, which is a person included in the aforementioned surveillance video, A communication device identification unit that identifies the location of the first communication device that transmitted the identification information received by the second communication device, based on the state of radio waves from the first communication device detected by the second communication device, An individual identification unit performs individual identification of the monitored object included in the surveillance video by identifying the correspondence between the captured object and the monitored object holding the first communication device based on the management data, the location of the captured object identified by the surveillance video processing unit, and the location of the first communication device identified by the communication device identification unit, A display data generation unit generates image display data including the aforementioned surveillance video, information indicating the group of surveillance targets whose correspondence with the subject being filmed has been identified, information of the second communication device that received the identification information of the first communication device held in the surveillance target, and information of the filming terminal that filmed the surveillance video. The system includes an output unit that outputs the display data to the display device, The aforementioned monitoring target identification device is connected to a plurality of the aforementioned shooting terminals, The aforementioned display data generation unit, The device receives the designation of the target to be monitored from the display device, extracts the monitoring video including the designated target to generate the display data, When the designated monitored object moves within the shooting range of different shooting terminals, the display data is generated so that an image including the combined footage of the monitoring images captured by each of these shooting terminals is displayed on the display device. When the designated monitored object is being captured by multiple of the capturing terminals, the display data is generated so that an image including multiple of the monitoring images captured by these capturing terminals is displayed on the display device. A system for identifying monitored targets.

9. The camera terminal captures images within the monitoring area and outputs the resulting video footage to the monitoring target identification device. The first communication device held by the monitored object transmits identification information of the first communication device, The second communication device receives the identification information from the first communication device located within the shooting range of the shooting terminal via short-range wireless communication, and outputs the identification information and information indicating the state of the radio waves from the first communication device detected by the second communication device to the monitored target identification device. The aforementioned surveillance target identification device identifies the location of the subject being filmed, which is a person included in the surveillance video. The monitoring target identification device identifies the location of the first communication device that transmitted the identification information received by the second communication device, based on the information indicating the state of the radio waves. The monitoring target identification device identifies individual monitoring targets included in the monitoring video by identifying the correspondence between the shooting target and the monitoring target holding the first communication device, based on the identification information of the first communication device, target identification information which is information that identifies the monitoring target holding the first communication device, data which associates the group to which the monitoring target belongs, the location of the identified shooting target, and the location of the identified first communication device. The monitoring target identification device generates image display data that includes the monitoring video and information indicating the group of monitoring targets whose correspondence with the filming target has been identified. The monitoring target identification device includes outputting the display data to a display device, The aforementioned monitoring target identification device identifies the location of the target to be photographed as a region, Identifying the correspondence between the subject to be photographed and the monitoring object holding the first communication device includes determining that the subject to be photographed and the monitoring object holding the first communication device correspond if the location of the first communication device, as identified, is included in the region identified as the location of the subject to be photographed. Method for identifying monitored targets.

10. The camera terminal captures images within the monitoring area and outputs the resulting video footage to the monitoring target identification device. The first communication device held by the monitored object transmits identification information of the first communication device, The second communication device receives the identification information from the first communication device located within the shooting range of the shooting terminal via short-range wireless communication, and outputs the identification information and information indicating the state of the radio waves from the first communication device detected by the second communication device to the monitored target identification device. The aforementioned surveillance target identification device identifies the location of the subject being filmed, which is a person included in the surveillance video. The monitoring target identification device identifies the location of the first communication device that transmitted the identification information received by the second communication device, based on the information indicating the state of the radio waves. The monitoring target identification device identifies individual monitoring targets included in the monitoring video by identifying the correspondence between the shooting target and the monitoring target holding the first communication device, based on the identification information of the first communication device, target identification information which is information that identifies the monitoring target holding the first communication device, data which associates the group to which the monitoring target belongs, the location of the identified shooting target, and the location of the identified first communication device. The monitoring target identification device generates image display data including the monitoring video, information indicating the group of monitoring targets whose correspondence with the target being filmed has been identified, information of the second communication device that received the identification information of the first communication device held by the monitoring target, and information of the filming terminal that filmed the monitoring video. The monitoring target identification device includes outputting the display data to a display device, The aforementioned monitoring target identification device is connected to a plurality of the aforementioned shooting terminals, The generation of the aforementioned display data is The monitoring target identification device receives a designation of the monitoring target from the display device, extracts the monitoring video including the designated monitoring target, and generates the display data, When the designated monitored object moves within the shooting range of different shooting terminals, the display data is generated so that an image including the combined footage of the monitoring images captured by each of these shooting terminals is displayed on the display device. The system includes generating display data such that, when the designated monitored object is being captured by multiple capture terminals, an image containing multiple monitoring videos captured by these capture terminals is displayed on the display device. Method for identifying monitored targets.

Images