Server, program, monitoring support system and monitoring support method

Abstract

To provide a server capable of appropriately detecting abnormality in a monitoring target region, using a glass-type device.SOLUTION: A server communicating with a glasses-type device worn by an observer, has: a memory unit that stores map information; a device information acquisition unit that acquires information indicating a position of the glasses-type device and the observer's line of sight from the glasses-type device; an estimation unit that estimates a distance from the observer to an object that the observer is gazing at based on information about the line of sight direction; an identification unit that identifies a position of the object based on the position of the glasses-type device, the observer's line of sight and the estimated distance; and an output unit that outputs information about the position of the object in the map information.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a server, a program, a monitoring support system, and a monitoring support method. [Background technology]

[0002] In recent years, wide-area surveillance systems have been introduced that install surveillance cameras in large areas such as residential areas, forests, and river areas to monitor them. Although surveillance cameras act as a psychological deterrent against crime, they are no match for human visual surveillance when it comes to proactively detecting suspicious people and objects to prevent crimes from occurring. Therefore, there is a demand for such systems to proactively detect abnormalities in the area being monitored.

[0003] Meanwhile, there are known glasses-type display devices that are worn on the head, such as head-mounted displays and smart glasses. Some of these glasses-type devices have a function of detecting the line of sight of the user. These glasses-type devices detect the line of sight of workers in facilities such as plants to monitor and support their work (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-5660 A Summary of the Invention [Problem to be solved by the invention]

[0005] It is believed that by using a glasses-type device to detect a person's gaze direction, it is possible to actively detect abnormalities in a monitored area even in wide-area surveillance.

[0006] An object of the present invention is to provide a server, a program, a monitoring support system, and a monitoring support method that enable appropriate detection of abnormalities in a monitored area using a glasses-type device. [Means for solving the problem]

[0007] A server according to an embodiment of the present invention is a server that communicates with a glasses-type device worn by an observer, and has a memory unit that stores map information, a device information acquisition unit that acquires information indicating the position of the glasses-type device and the observer's line of sight from the glasses-type device, an estimation unit that estimates the distance from the observer to an object that the observer is gazing at based on the line of sight direction information, an identification unit that identifies the position of the object based on the position of the glasses-type device, the observer's line of sight and the estimated distance, and an output unit that outputs information regarding the position of the object in the map information.

[0008] It is also preferable that the device information acquisition unit acquires the horizontal and vertical gaze directions of the monitor, and the identification unit further identifies the height of the object based on the vertical gaze direction.

[0009] In addition, it is preferable that the server further has a calculation unit that calculates the surveillance density of multiple areas included in the map information based on the positions of the object identified for multiple times, and the output unit outputs the surveillance density as information regarding the position of the object in the map information.

[0010] It is also preferable that the server further has a designated information acquisition unit that acquires information designating a surveillance area from among a plurality of areas, and that the output unit outputs information indicating a surveillance area identified based on the surveillance density and the designation of the surveillance area.

[0011] In addition, it is preferable that the server further has a setting unit that sets the patrol route of the monitor so as to include the area to be monitored or the vicinity of the area to be monitored, and a notification unit that notifies the monitor of the patrol route.

[0012] A program according to an embodiment of the present invention is a computer program that communicates with a glasses-type device worn by an observer and has a memory unit that stores map information, and causes the computer to obtain information indicating the position of the glasses-type device and the observer's line of sight from the glasses-type device, estimate the distance from the observer to an object at which the observer is gazing based on the line of sight direction information, identify the position of the object based on the position of the glasses-type device, the observer's line of sight and the estimated distance, and output information relating to the position of the object in the map information.

[0013] A surveillance support system according to an embodiment of the present invention has a glasses-type device worn by an observer and a server that communicates with the glasses-type device, wherein the glasses-type device has a position sensor that detects the position of the glasses-type device, a gaze direction sensor that detects the gaze direction of the observer, and a transmission unit that transmits information indicating the position of the glasses-type device and the gaze direction of the observer to the server, and the server has a memory unit that stores map information, an acquisition unit that acquires information indicating the position of the glasses-type device and the gaze direction of the observer, an estimation unit that estimates the distance from the observer to an object that the observer is gazing at based on the gaze direction information, an identification unit that identifies the position of the object based on the position of the glasses-type device, the gaze direction of the observer and the estimated distance, and an output unit that outputs information regarding the position of the object in the map information.

[0014] A surveillance support method according to an embodiment of the present invention is a surveillance support method executed by a surveillance support system having a glasses-type device worn by an observer and a server that communicates with the glasses-type device, in which the glasses-type device detects the position of the glasses-type device and the observer's gaze direction, and transmits information indicating the position of the glasses-type device and the observer's gaze direction to the server, and the server stores map information, acquires information indicating the position of the glasses-type device and the observer's gaze direction, estimates the distance from the observer to an object at which the observer is gazing based on the gaze direction information, identifies the position of the object based on the position of the glasses-type device, the observer's gaze direction and the estimated distance, and outputs information regarding the position of the object in the map information. Effect of the Invention

[0015] The server, the program, the monitoring support system, and the monitoring support method according to the present invention make it possible to appropriately detect abnormalities in a monitored area using a glasses-type device. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a monitoring support system 1. [Diagram 2] FIG. 2 is a schematic front view of the glasses-type device 2. [Diagram 3] 2 is an exemplary functional block diagram of the glasses-type device 2. FIG. [Figure 4] FIG. 2 is an exemplary functional block diagram of the server 3. [Diagram 5] 13 is a diagram showing an example of the data structure of a monitoring area table T1. FIG. [Figure 6] FIG. 13 is a diagram showing an example of the data structure of a target object candidate table T2. [Figure 7] FIG. 13 is a diagram showing an example of the data structure of a surveillance camera table T3. [Figure 8] FIG. 11 is a flow diagram showing an example of the flow of a patrol monitoring process. [Figure 9] FIG. 11 is a sequence diagram showing an example of the flow of an acquisition process. [Figure 10] FIG. 13 is a diagram showing an example of a monitoring density map M1. [Figure 11] FIG. 11 is a flow diagram showing an example of the flow of a fixed point monitoring process. [Figure 12] FIG. 11 is a flowchart showing an example of the flow of a travel route setting process. [Figure 13] FIG. 11 is a flow diagram showing an example of a monitoring area map M2. [Figure 14] FIG. 13 is a diagram showing an example of a travel route setting screen G1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments, but covers the inventions described in the claims and their equivalents.

[0018] 1 is a diagram showing an example of a schematic configuration of a monitoring support system 1 according to an embodiment of the present invention. The monitoring support system 1 includes a glasses-type device 2, a server 3, a user terminal 4, a monitoring camera 5, and a monitoring terminal 6. The glasses-type device 2, the server 3, the user terminal 4, the monitoring camera 5, and the monitoring terminal 6 communicate with each other via a network N, which is the Internet or an intranet.

[0019] The surveillance support system 1 supports the surveillance of the area to be monitored by combining patrol surveillance by a patrol monitor P who wears a glasses-type device 2 and patrols a predetermined patrol route, and fixed-point surveillance by a fixed-point monitor F who monitors the video captured by a surveillance camera 5 via a monitoring terminal 6. The surveillance support system 1 appropriately detects abnormalities in the area to be monitored by updating the patrol route to include the area to be monitored or its vicinity, which is identified based on the results of the patrol surveillance and the fixed-point surveillance.

[0020] The glasses-type device 2 is worn on the head of the patrol observer P. The glasses-type device 2 detects the position of the glasses-type device 2 and the line of sight of the patrol observer P. The glasses-type device 2 transmits information indicating the position of the glasses-type device 2 and the line of sight of the patrol observer P to the server 3.

[0021] The server 3 is an information processing device. The server 3 may be a PC (Personal Computer), a tablet terminal, a smartphone, a mobile phone, a game machine, or the like. The server 3 stores map information. The server 3 identifies the position in the map information of an object that the patrol observer P is gazing at, based on information indicating the position of the glasses-type device 2 and the line of sight direction of the patrol observer P. The server 3 also identifies an area requiring monitoring from among the areas to be monitored, and sets a patrol route to include the area requiring monitoring or its vicinity.

[0022] The user terminal 4 is an information processing terminal such as a PC, tablet terminal, smartphone, mobile phone, game console, etc., carried by the patrol monitor P. The user terminal 4 receives information on the patrol route from the server 3 and displays it.

[0023] The surveillance camera 5 is installed at a predetermined position in the area to be monitored. The position where the surveillance camera 5 is installed and the shooting direction of the surveillance camera 5 are stored in advance in the server 3. In addition, the image captured by the surveillance camera 5 is transmitted to the monitoring terminal 6.

[0024] The monitoring terminal 6 is an information processing terminal such as a PC, a tablet terminal, a smartphone, a mobile phone, a game machine, etc. The monitoring terminal 6 displays images captured by the monitoring camera 5. In addition, the monitoring terminal 6 transmits a signal indicating an abnormality in the monitored area to the server 3 in response to an operation by the fixed point monitor F.

[0025] Fig. 2 is a schematic front view of the glasses-type device 2, and Fig. 3 is an exemplary functional block diagram of the glasses-type device 2. The glasses-type device 2 has a frame 211, lenses 212, a position sensor 22, an orientation sensor 23, a gaze direction sensor 24, a memory unit 25, a communication unit 26, a processing unit 27, and the like.

[0026] The frame 211 holds each component of the glasses-type device 2 and enables the glasses-type device 2 to be worn on the head. The lens 212 is made of translucent resin or glass.

[0027] The position sensor 22 is configured to detect the position of the glasses-type device 2. For example, the position sensor 22 includes a Global Navigation Satellite System (GNSS) receiver that detects the position of the glasses-type device 2. The position sensor 22 generates information indicating the detected position of the glasses-type device 2 and supplies the information to the processing unit 27.

[0028] The orientation sensor 23 is configured to detect a rough line-of-sight direction of the patrol monitor P. For example, the orientation sensor 23 includes a geomagnetic sensor that detects the direction of the glasses-type device 2. The orientation sensor 23 generates information indicating the detected direction of the glasses-type device 2 as information indicating a rough line-of-sight direction of the patrol monitor P, and supplies the information to the processing unit 27.

[0029] The gaze direction sensor 24 is configured to detect the precise gaze direction of the patrol observer P. For example, the gaze direction sensor 24 includes an eye tracking sensor that detects the gaze direction of each of the eyes and an electromagnetic signal sensor that detects the motion state of the ciliary muscles of each of the eyes.

[0030] The eye tracking sensor includes a near-infrared LED (Light Emitting Diode) light source, a camera, and a processing circuit. The LED light source irradiates near-infrared light onto the eyeballs of the patrol observer P wearing the glasses-type device 2. The camera captures the eyeballs of the patrol observer P, which reflect the near-infrared light, to generate an image. The processing circuit calculates the line of sight of the patrol observer P by applying a predetermined algorithm to the image of the eyeball captured by the camera. For example, the processing circuit calculates which of a plurality of preset angle ranges the line of sight of the patrol observer P is included in. For example, five angle ranges are set as the plurality of angle ranges in the horizontal direction: left direction, left front direction, front direction, right front direction, and right direction, and five angle ranges are set in the vertical direction: upward direction, diagonally upward direction, front direction, diagonally downward direction, and downward direction. In this case, the processing circuit calculates which of the five angle ranges in the horizontal direction and which of the five angle ranges in the vertical direction the line of sight of both eyes is included in. The processing circuit generates information indicating the calculated angle range as information indicating the line of sight of both eyes, and supplies the information to the processing unit 27.

[0031] The electromagnetic signal sensor includes an antenna, an amplifier, and a processing circuit. The antenna is, for example, a single loop antenna, and detects an electromagnetic signal in a predetermined frequency band generated by the movement of the ciliary muscle. The amplifier amplifies the signal detected by the antenna to a predetermined amplitude. The processing circuit determines the motion state of the ciliary muscle by applying a predetermined algorithm to the signal amplified by the amplifier. For example, the processing circuit determines whether the ciliary muscle is in one of five states: a strong tension state, a weak tension state, a normal state, a weak relaxation state, or a strong relaxation state. The processing circuit generates information indicating the result of the determination as information indicating the motion state of the ciliary muscle, and supplies the information to the processing unit 27.

[0032] Generally, when a person looks at a nearby object, the eyes move inward (convergence movement) with respect to each other. When a person looks at a distant object, the eyes move outward (divergence movement) with respect to each other. Therefore, the distance to the object is estimated based on the gaze direction of both eyes.

[0033] In addition, when a person looks at a nearby object, the ciliary muscle surrounding the crystalline lens of the eye tenses and contracts. This causes the crystalline lens to be pushed by the ciliary muscle and thicken, shortening the focal length of the crystalline lens and bringing the object into focus. On the other hand, when a person looks at a distant object, the ciliary muscle relaxes and stretches. This causes the crystalline lens to be pulled by the ciliary muscle and become thinner, lengthening the focal length of the crystalline lens and bringing the object into focus. Therefore, the distance to the object can be estimated based on the movement of the ciliary muscle.

[0034] The storage unit 25 is configured to store programs and data, and includes, for example, a semiconductor memory. The storage unit 25 stores, as programs, an operating system program, a driver program, an application program, and the like used in processing by the processing unit 27. The programs are installed in the storage unit 25 from a computer-readable, non-transitory portable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a DVD-ROM (Digital Versatile Disc Read Only Memory).

[0035] The communication unit 26 is configured to enable the glasses-type device 2 to communicate with other devices, and includes a communication interface circuit. The communication interface circuit included in the communication unit 26 is a communication interface circuit for a mobile communication system, a wireless LAN (Local Area Network), or the like. The communication unit 26 receives data from other devices and supplies the data to the processing unit 27, and transmits data supplied from the processing unit 27 to other devices.

[0036] The processing unit 27 is configured to comprehensively control the operation of the glasses-type device 2, and includes one or more processors and their peripheral circuits. The processing unit 27 includes, for example, a CPU (Central Processing Unit). The processing unit 27 may include a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. The processing unit 27 controls the operation of each component so that various processes of the glasses-type device 2 are executed in an appropriate order based on the programs stored in the storage unit 25.

[0037] The processing unit 27 includes, as functional blocks, an acquisition unit 271 and a transmission unit 272. These functional blocks are functional modules realized based on a program executed by the processing unit 27. Each of these units may be implemented in the glasses-type device 2 as a dedicated processing circuit.

[0038] 4 is an exemplary functional block diagram of the server 3. The server 3 includes a storage unit 31, a communication unit 32, a processing unit 33, and the like.

[0039] The storage unit 31 is configured to store programs and data, and includes, for example, a semiconductor memory. The storage unit 31 stores, as programs, an operating system program, a driver program, an application program, and the like used in processing by the processing unit 33. The programs are installed in the storage unit 31 from a computer-readable, non-transitory, portable storage medium such as a CD-ROM or a DVD-ROM.

[0040] The communication unit 32 is configured to enable the server 3 to communicate with other devices, and includes a communication interface circuit. The communication interface circuit included in the communication unit 32 is a communication interface circuit for a mobile communication system, a wireless LAN (Local Area Network), or the like. The communication unit 32 receives data from other devices and supplies the data to the processing unit 33, and transmits data supplied from the processing unit 33 to other devices.

[0041] The processing unit 33 is configured to comprehensively control the operation of the glasses-type device 2, and includes one or more processors and their peripheral circuits. The processing unit 33 includes, for example, a CPU. The processing unit 33 may include a GPU (Graphics Processing Unit), a DSP, an LSI, an ASIC, an FPGA, or the like. The processing unit 33 controls the operation of each component so that various processes of the server 3 are executed in an appropriate order based on the programs stored in the storage unit 31.

[0042] The processing unit 33 includes, as functional blocks, a device information acquisition unit 331, an estimation unit 332, an object identification unit 333, a calculation unit 334, an output unit 335, an area information acquisition unit 336, an area identification unit 337, a setting unit 338, and a notification unit 339. These functional blocks are functional modules realized based on a program executed by the processing unit 33. Each of these units may be implemented in the server 3 as a dedicated processing circuit.

[0043] 5 is a diagram showing an example of the data structure of the monitored area table T1 stored in the storage unit 31. The monitored area table T1 stores information on a plurality of areas included in the monitored area. The monitored area table T1 stores area identification information, coordinates, monitoring density, alert area designation, etc. in association with each other.

[0044] The area identification information is information that identifies each of the multiple areas. The multiple areas are, for example, meshes such as 10m meshes and 50m meshes defined by the Geospatial Information Authority of Japan. In this case, the area identification information is a mesh code. This example is not limited to this, and the multiple areas may be geographical areas of any shape and size. The coordinates are information that indicates the positions of the multiple areas. For example, the coordinates are the latitude and longitude of the center point of each area. The area identification information and the coordinates are examples of map information.

[0045] The surveillance density is information indicating the degree to which the patrol monitors monitored each area. The alert area designation is information indicating whether each area has been designated as an alert area. The surveillance density and alert area designation information are used to identify areas requiring increased surveillance density.

[0046] The area identification information and coordinate information are stored in advance in the storage unit 31. The surveillance density information is calculated and stored in the patrol surveillance processing described later. The security area designation information is set in advance, or is stored in the fixed point surveillance processing described later. For example, an area that is not included in the shooting range of the surveillance camera 5 is designated as a security area in advance. In the example shown in FIG. 5, the security area designation information is stored as information indicating whether the area was designated as a security area in the fixed point surveillance processing, or whether the area was designated as a security area in advance because it is not included in the shooting range of the surveillance camera 5.

[0047] 6 is a diagram showing an example of the data structure of the object candidate table T2 stored in the storage unit 31. The object candidate table T2 stores information on object candidates that are included in the monitored area and monitored by the monitor. The objects are, for example, buildings, parks, etc. The object candidate table T2 stores the name, coordinates, height, and gaze level of the object candidate in association with each other.

[0048] The coordinates are information indicating the position of the object candidate, such as latitude and longitude. The height is information indicating the height of the object candidate relative to a reference plane. The reference plane is mean sea level, ground level, etc. If the object candidate is an object identified by a planar shape, such as a park, the height does not need to be stored. The degree of attention is information indicating the degree to which the patrol monitor P has gazed upon the object candidate. The degree of attention may be information indicating the number of times or the time for which the patrol monitor P has gazed upon the object candidate during patrol monitoring.

[0049] Information on the name, coordinates, and height of the object candidate is stored in advance in the storage unit 31. Information on the gaze degree is calculated in a patrol monitoring process described later, and stored in the storage unit 31. Note that instead of the coordinates of the object candidate, graphic data indicating the two-dimensional shape of the object may be stored. Also, instead of the coordinates and height of the object candidate, graphic data indicating the three-dimensional shape of the object may be stored.

[0050] 7 is a diagram showing an example of the data structure of the surveillance camera table T3 stored in the storage unit 31. The surveillance camera table T3 stores information about the surveillance cameras 5. The surveillance camera table T3 stores camera identification information, coordinates, and shooting directions in association with each other.

[0051] The camera identification information is information for identifying each of the multiple surveillance cameras 5. The coordinates are information indicating the position of the surveillance camera 5. For example, the coordinates are the latitude and longitude of the position of the surveillance camera 5. The shooting direction is information indicating the shooting direction of the surveillance camera 5. For example, the shooting direction is the azimuth angle of the shooting direction of the surveillance camera 5 based on a predetermined orientation.

[0052] The camera identification information, coordinates, and shooting direction information are stored in the storage unit 31 in advance.

[0053] 8 is a diagram showing an example of the flow of a patrol monitoring process executed by the server 3. The patrol monitoring process is realized by the processing unit 33 cooperating with each component of the server 3 based on a program stored in the storage unit 31.

[0054] First, the device information acquisition unit 331 of the server 3 determines whether or not a start signal indicating that patrol monitoring has started has been received via the communication unit 32 (step S11). For example, the start signal is transmitted from the user terminal 4 in response to an operation by the patrol monitor P.

[0055] If the start signal is not received (step S11-No), the tour monitoring process returns to step S11. That is, the device information acquisition unit 331 waits until the start signal is received.

[0056] When the start signal is received (step S11-Yes), the device information acquisition unit 331 executes an acquisition process (step S12). The acquisition process is a process of acquiring device information indicating the position of the glasses-type device 2 and the line of sight direction of the patrol observer P from the glasses-type device 2.

[0057] 9 is a sequence diagram showing an example of the flow of an acquisition process executed by the glasses-type device 2 and the server 3. The acquisition process is executed in step S12 of the tour monitoring process.

[0058] First, the device information acquisition unit 331 of the server 3 transmits an instruction to transmit device information to the glasses-type device 2 via the communication unit 32 (step S21).

[0059] Next, the acquisition unit 271 of the glasses-type device 2 acquires device information (step S22). The acquisition unit 271 acquires, as the device information, information indicating the position of the glasses-type device 2 from the position sensor 22. The acquisition unit 271 acquires, as the device information, information indicating a rough line-of-sight direction of the patrol observer P from the orientation sensor 23. The acquisition unit 271 acquires, as the device information, information indicating a precise line-of-sight direction of the patrol observer P from the line-of-sight direction sensor 24.

[0060] Next, the transmission unit 272 of the glasses-type device 2 transmits the device information and time information indicating the time when the device information was acquired in association with each other to the server 3 via the communication unit 26 (step S23).

[0061] Next, the device information acquisition unit 331 of the server 3 acquires the device information and the time information by receiving them via the communication unit 32 (step S24). The device information acquisition unit 331 associates the acquired device information with the time information and stores them in the storage unit 31. This ends the acquisition process.

[0062] 8, next, the device information acquisition unit 331 determines whether or not an end signal indicating that the patrol monitoring has ended has been received via the communication unit 32 (step S13). For example, the end signal is transmitted from the user terminal 4 in response to an operation by the patrol monitor P.

[0063] If the end signal is not received (step S13-No), the tour monitoring process returns to step S12. That is, the device information acquisition unit 331 repeatedly executes the acquisition process until the end signal is received.

[0064] When the end signal is received (step S13-Yes), the estimation unit 332 selects one of the acquired device information as the device information to be processed (step S14). For example, the estimation unit 332 selects, from the device information stored in the storage unit 31, the device information associated with time information indicating the earliest time as the device information to be processed.

[0065] Next, the estimation unit 332 estimates the distance to the object that the patrol observer P is gazing at (step S15). For example, the estimation unit 332 estimates the distance to the object based on information indicating the gaze direction of both eyes and information indicating the movement state of the ciliary muscle, among the device information to be processed.

[0066] For example, the storage unit 31 stores in advance a table that associates a combination of the gaze direction in the horizontal direction of both eyes and the motion state of the ciliary muscle with an estimated distance to an object. For example, the information indicating the gaze direction in the horizontal direction of both eyes is information indicating which of five angle ranges the gaze direction is included in, and the information indicating the motion state of the ciliary muscle is information indicating which of five motion states the ciliary muscle is in. In this case, the number of these combinations is 25. An estimated distance to an object is associated in advance with each of these 25 combinations. The estimation unit 332 estimates the distance to an object by acquiring an estimated distance associated with a combination of the gaze direction in the horizontal direction of both eyes and the motion state of the ciliary muscle indicated in the acquired device information.

[0067] Next, the object identification unit 333 identifies the position of the object in the map information (step S16). The object identification unit 333 identifies the position of the object based on the position of the glasses-type device 2 and the line of sight of the patrol observer P indicated in the device information to be processed, and the distance to the object estimated for the device information to be processed. For example, the object identification unit 333 identifies a position that is separated from the position of the glasses-type device 2 by an estimated distance in the line of sight of the patrol observer P as the position of the object. For example, the line of sight of the patrol observer P is a rough line of sight direction detected by the orientation sensor 23 of the glasses-type device 2.

[0068] Next, the object identification unit 333 determines whether or not all of the acquired device information has been selected as a processing target (step S17).

[0069] If all of the device information has not been selected as the device information to be processed (step S17-No), the patrol monitoring process returns to step S14, and the estimation unit 332 selects one of the device information that has not yet been selected as the device information to be processed as the device information to be processed.

[0070] If all device information is selected as the processing target (step S17-Yes), the calculation unit 334 calculates the attention degree of the object and the surveillance density of multiple areas based on the distance to the object for multiple times (step S18).

[0071] The calculation unit 334 refers to the object candidate table T2 and identifies the object gazed upon by the patrol observer P based on the position of the object identified for each piece of acquired device information. For example, the calculation unit 334 identifies, among the object candidates included in the object candidate table T2, the object whose position is closest to the position estimated in step S16 as the object gazed upon by the patrol observer P. The calculation unit 334 calculates the number of times each of the multiple object candidates has been identified as an object gazed upon by the patrol observer P, and calculates the degree of gaze of the object based on the calculated number of times. For example, the calculation unit 334 calculates that the degree of gaze is high when the calculated number of times is equal to or greater than a threshold, and that the degree of gaze is low when the calculated number of times is less than the threshold.

[0072] Furthermore, the calculation unit 334 refers to the area table T1 and identifies an area that includes an object that the patrol monitor P has gazed upon as an area monitored by the patrol monitor P. The calculation unit 334 calculates the number of times that each of the multiple areas has been identified as an area monitored by the patrol monitor P, and calculates the surveillance density of each area based on the calculated number of times. For example, the calculation unit 334 calculates that the surveillance density is high when the calculated number of times is equal to or greater than a threshold, and calculates that the surveillance density is low when the calculated number of times is less than the threshold.

[0073] Next, the output unit 335 outputs the surveillance density (step S19). For example, the output unit 335 generates a surveillance density map that displays a list of surveillance densities of a plurality of areas. The output unit 335 outputs the surveillance density map by transmitting it to another device via the communication unit 32. For example, the output unit 335 transmits the surveillance density map to a display device (not shown) to display it. Note that the surveillance density is an example of information related to the position of an object.

[0074] FIG. 10 is a diagram showing an example of a surveillance density map M1. The surveillance density map M1 includes a map image M11, a surveillance density M12, and a patrol route M13. The map image M11 is stored in advance in the storage unit 31. The map image M11 may be received from a map information server (not shown). The surveillance density M12 is a graphic that is superimposed on the map image M11 and indicates the surveillance density of each area. In the example shown in FIG. 10, the surveillance density is indicated by the color of the rectangle superimposed on each area. The patrol route M13 is a graphic that indicates the route patrolled by the patrol monitor P in the map image M11.

[0075] For example, the output unit 335 acquires a map image corresponding to a predetermined geographical area including the patrol route of the patrol monitor P. The output unit 335 identifies, in the acquired map image, areas corresponding to the multiple areas for which the surveillance density has been calculated. The output unit 335 places, in the identified areas of the map image, a rectangle of a color according to the surveillance density of the corresponding area. The output unit 335 also places, in the map image, a figure indicating the patrol route. In this manner, the output unit 335 generates a surveillance density map.

[0076] 11 is a flow diagram showing an example of the flow of fixed-point monitoring processing executed by the server 3. The fixed-point monitoring processing is executed in response to a signal indicating an abnormality in a monitored area being transmitted from the monitoring terminal 6. The fixed-point monitoring processing is realized by the processing unit 33 cooperating with each component of the server 3 based on a program stored in the storage unit 31.

[0077] First, the area information acquisition unit 336 acquires an abnormality signal indicating an abnormality in the area to be monitored (step S31). For example, the area information acquisition unit 336 acquires the abnormality signal by receiving it from the monitoring terminal 6 via the communication unit 32. For example, the abnormality signal is transmitted in response to a fixed point monitor F monitoring the video of the monitoring camera 5 displayed on the monitoring terminal 6 inputting to the monitoring terminal 6 that he has found an abnormality. The abnormality signal includes the camera identification information of the monitoring camera 5 that captured the video in which the abnormality was found.

[0078] Next, the area information acquisition unit 336 designates a warning area based on the abnormal signal (step S32). The area information acquisition unit 336 acquires the position and shooting direction of the monitoring camera 5 indicated by the camera identification information included in the abnormal signal from the monitoring camera table T3. The area information acquisition unit 336 identifies the area including the shooting range of the monitoring camera 5 based on the position and shooting direction of the monitoring camera 5. Note that if the shooting range of the monitoring camera 5 is included in multiple areas, the area information acquisition unit 336 may identify each of the multiple areas, or may identify only one area according to a predetermined order. The area information identification unit 336 designates the identified area as a warning area.

[0079] Next, the area information acquisition unit 336 stores information indicating that the area identified in step S32 has been designated as a warning area in the storage unit 31 (step S33). This ends the fixed point monitoring process.

[0080] 12 is a flow diagram showing an example of the flow of a tour route setting process executed by the server 3. The tour route setting process is executed periodically (for example, once a week). The tour route setting process is realized by the processing unit 33 working in cooperation with each component of the server 3 based on a program stored in the storage unit 31.

[0081] First, the area information acquisition unit 336 acquires information on the monitoring density of a plurality of areas from the storage unit 31 (step S41).

[0082] Next, the area specification unit 337 acquires information on the designation of a plurality of areas as alert areas from the storage unit 31 (step S42).

[0083] Next, the area specifying unit 337 specifies an area to be monitored from among the multiple areas based on the information on the monitoring density and the alert area designation (step S43). For example, the area specifying unit 337 specifies an area that satisfies a monitoring requirement condition from among the multiple areas as an area to be monitored. The monitoring requirement condition is that at least one of the following is satisfied: the monitoring density is low, and the area is designated as an alert area.

[0084] Next, the area identification unit 337 outputs information indicating the areas requiring monitoring (step S44). For example, the area identification unit 337 generates a monitoring area map that displays a list of areas requiring monitoring. The area identification unit 337 outputs the monitoring area map by transmitting it to another device via the communication unit 32. For example, the area identification unit 337 transmits the monitoring area map to a display device (not shown) to display it.

[0085] FIG. 13 is a diagram showing an example of a monitoring area map M2. The monitoring area map M2 includes a map image M21, a surveillance density object M22, an alert area object M23, and a monitoring area object M24. The map image M21 is stored in advance in the storage unit 31. The map image M21 may be received from a map information server (not shown). The surveillance density object M22 is a figure superimposed on the map image M21 and indicates the surveillance density of each area. The alert area object M23 is a figure placed in the alert area. In the example shown in FIG. 13, a first alert area object M23a is placed in the alert area specified in the fixed point monitoring process. In addition, a second alert area object M23b, which is displayed in a manner different from the first alert area object M23a, is placed in the alert area specified in advance because it is not included in the shooting range (shown in a sector shape) of the monitoring camera 5. The monitoring area object M24 is a figure placed in the monitoring area.

[0086] Returning to FIG. 12, next, the setting unit 338 sets a tour route so as to include the monitoring area or the vicinity of the monitoring area (step S45). For example, the setting unit 338 acquires road network information previously stored in the storage unit 31. The road network information is, for example, data conforming to the national digital road map database standard defined by the Japan Digital Road Map Association. The setting unit 338 sets at least one passing point within each of the monitoring areas with reference to the road network information. For example, the passing point is a road node such as an intersection. The setting unit 338 sets a tour route that passes through all the passing points. For example, the setting unit 338 sets the tour route by sequentially calculating the shortest route between the passing points using Dijkstra's algorithm or the like.

[0087] Next, the notification unit 339 determines whether or not the tour route set in step S45 is the same as the current tour route (step S46).

[0088] If the set tour route is the same as the current tour route (step S46-Yes), the tour route setting process ends.

[0089] If the set tour route is not the same as the current tour route (step S46-No), the notification unit 339 notifies the tour monitor P of the set tour route (step S47). For example, the notification unit 339 generates display data for a tour route display screen showing the tour route. The notification unit 339 transmits the display data for the tour route display screen to the user terminal 4 via the communication unit 32, thereby causing the tour route display screen to be displayed on the user terminal 4. This ends the tour route setting process.

[0090] Fig. 14 is a diagram showing an example of a tour route setting screen G1. The tour route setting screen G1 includes a map image G11, a current tour route G12, and a new tour route G13. The current tour route G12 and the new tour route G13 are displayed superimposed on the map image G11. In the example shown in Fig. 14, the tour route has been changed so as to pass through a newly designated security area (shown by a dashed line) in the fixed point monitoring process.

[0091] As described above, the server 3 has the object identification unit 333 that identifies the position of the object based on the position of the glasses-type device 2 and the distance from the patrol observer P to the object, and the calculation unit 334 that calculates the surveillance density of multiple areas. This enables the server 3 to detect an area where an abnormality is estimated to exist based on the surveillance density.

[0092] The server 3 further includes an area information acquisition book 336 that acquires information specifying a security area from among a plurality of areas, and an output unit 335 that outputs information indicating a surveillance area specified based on the surveillance density and the designation of the security area. This enables the server 3 to detect an area where an abnormality has been found during fixed point surveillance, or an area that is not included in the shooting range of the surveillance camera 5 and therefore requires patrol surveillance.

[0093] The server 3 further includes a setting unit 338 that sets the patrol route of the patrol monitor P to include the area requiring monitoring or the vicinity of the area requiring monitoring, and a notification unit 339 that notifies the monitor of the patrol route. This enables the server 3 to urge the patrol monitor P to monitor the area requiring patrol monitoring.

[0094] The monitoring support system 1 may be modified as follows.

[0095] In the above-described embodiment, in step S19 of the patrol monitoring process, the output unit 335 outputs the monitoring density. The output unit 335 may output information indicating the position of the object in the map information as information regarding the position of the object. In this case, the output unit 335 superimposes an object indicating the object on the map image and places it at the position of the object identified in step S16. The object indicating the object is displayed in a manner according to the degree of attention to the object. An object indicating the object may be placed only for objects having a degree of attention equal to or greater than a threshold. This makes it possible to detect the range gazed upon by the patrol monitor P in more detail. Note that the output unit 335 may output only information indicating the position of the object instead of the monitoring density.

[0096] In this case, in step S16 of the patrol monitoring process, the estimation unit 332 may further estimate the height of the object based on the line of sight in the vertical direction of both eyes. For example, the estimation unit 332 estimates the height of the part of the object that the patrol monitor P gazes at based on the elevation angle of the line of sight and the distance to the estimated object. In step S19, the output unit 335 further displays the height of the part that the patrol monitor P gazes at. This makes it possible to detect the range that the patrol monitor P gazes at in the height direction as well, and to detect a floor that is estimated to have an abnormality in an object that is, for example, a high-rise building.

[0097] In the above-described embodiment, in the patrol monitoring process, steps S14 to S16 are executed after the end signal is received in step S13, and the position of the object is identified. This is not a limitation, and the process of steps S14 to S16 may be executed each time device information is acquired, and the position of the object may be identified. In other words, the position of the object may be identified in real time during patrol monitoring.

[0098] The above-mentioned functions of the user terminal 4 may be realized by the glasses-type device 2. For example, in steps S11 and S13 of the patrol monitoring process, the glasses-type device 2 may transmit a start signal or an end signal. In this case, the glasses-type device 2 has an operation unit that is a tactile switch, and transmits a start signal and an end signal in response to the patrol monitor P operating the operation unit. Also, in step S47 of the patrol route setting process, the notification unit 339 may notify the glasses-type device 2 of the patrol route. In this case, the glasses-type device 2 has a lens 212 that is a light guide plate, a diffraction element, and a display device such as an LCD (Liquid Crystal Display) or LCOS (Liquid Crystal On Silicon), and displays a patrol route setting screen so that it can be viewed by the patrol monitor P.

[0099] The above-described embodiment and various modified examples may be appropriately combined and implemented within the scope of the present invention. The above-described various processes may be appropriately executed in different orders within the scope of the present invention. [Explanation of symbols]

[0100] 2. Glasses-type device 22 Position Sensor 23 Orientation Sensor 24 Gaze direction sensor 271 Acquisition Department 272 Transmitter 3 Server 331 Device Information Acquisition Unit 332 Estimation Department 333 Object Identification Department 334 Calculation Department 335 Output section 336 Area Information Acquisition Department 337 Area Identification Department 338 Settings 339 Notification Department

Claims

1. A server that communicates with a glasses-type device worn by a monitor, A storage unit that stores map information; a device information acquisition unit that acquires information indicating a position of the glasses-type device and a line of sight direction of the observer from the glasses-type device; an estimation unit that estimates a distance from the observer to an object that the observer is gazing at based on the information on the line of sight; an identification unit that identifies a position of the object based on a position of the glasses-type device, a line of sight direction of the observer, and the estimated distance; an output unit that outputs information regarding a position of the object in the map information; A server having:

2. The device information acquisition unit acquires the gaze directions of the monitor in the horizontal and vertical directions, The determination unit further determines a height of the object based on a line of sight direction in the vertical direction. The server of claim 1 .

3. A calculation unit is further provided for calculating a surveillance density of a plurality of areas included in the map information based on the positions of the object identified for a plurality of times, The output unit outputs the surveillance density as information regarding a position of an object in the map information. The server of claim 1 .

4. a zone information acquisition unit that acquires information designating a warning zone from among the plurality of zones; The output unit outputs information indicating a surveillance area identified based on the surveillance density and the information on the designation of the alert area. The server according to claim 3.

5. A setting unit that sets a patrol route for the monitor so as to include the monitoring area or the vicinity of the monitoring area; A notification unit that notifies the monitor of the patrol route. The server according to claim 4.

6. A computer program having a storage unit that communicates with a glasses-type device worn by a monitor and stores map information, acquiring information indicating a position of the glasses-type device and a line of sight direction of the observer from the glasses-type device; estimating a distance from the observer to an object that the observer is gazing at based on the information on the line of sight; determining a position of the object based on a position of the eyeglass-type device, a gaze direction of the observer, and the estimated distance; outputting information regarding the position of the object in the map information; A program for causing the computer to execute the above steps.

7. A surveillance support system having a glasses-type device worn by a surveillance person and a server that communicates with the glasses-type device, The glasses-type device includes: a position sensor for detecting a position of the glasses-type device; a gaze direction sensor for detecting the gaze direction of the observer; a transmission unit that transmits information indicating a position of the glasses-type device and a line of sight direction of the observer to the server, The server, A storage unit that stores map information; an acquisition unit that acquires information indicating a position of the glasses-type device and a line of sight direction of the observer; an estimation unit that estimates a distance from the observer to an object that the observer is gazing at based on the information on the line of sight; an identification unit that identifies a position of the object based on a position of the glasses-type device, a line of sight direction of the observer, and the estimated distance; an output unit that outputs information regarding the position of the object in the map information; Surveillance support system.

8. A surveillance assistance method executed by a surveillance assistance system having a glasses-type device worn by a surveillance person and a server that communicates with the glasses-type device, The glasses-type device comprises: Detecting a position of the glasses-type device and a line of sight direction of the observer; Transmitting information indicating a position of the glasses-type device and a line of sight direction of the observer to the server; The server, Stores map information, Acquire information indicating a position of the glasses-type device and a line of sight direction of the observer; estimating a distance from the observer to an object that the observer is gazing at based on the information on the line of sight; determining a position of the object based on a position of the eyeglass-type device, a gaze direction of the observer, and the estimated distance; Outputting information regarding the position of the object in the map information A monitoring support method comprising:

Images