Monitoring system, monitoring device, monitoring method, and program

Abstract

To provide a technology for efficiently monitoring a monitoring position in a monitored space.SOLUTION: A monitoring system includes point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged, point cloud display means for displaying the point cloud on a display device of an information terminal, monitoring position acquisition means for acquiring a monitoring position within the monitored space specified via the information terminal, and selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of capturing images of the monitoring position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a monitoring system, a monitoring device, a monitoring method, and a program. [Background technology]

[0002] Patent Document 1 discloses an abnormality location display device that improves the visibility of abnormal locations on an object. Specifically, the abnormality location display device is composed of a laser distance measuring device that measures the distance to the object and generates a point cloud, a photographing device that photographs the object and generates a photographed image, and a display device that simultaneously outputs the point cloud and the photographed image. The laser distance measuring device and the photographing device are arranged as close to each other as possible. This is said to improve the visibility of abnormal locations.

[0003] Patent Document 1 also discloses a configuration in which laser distance measuring devices and photographing devices are provided at multiple locations, and the user selects a point cloud or photographed image to be displayed on a display device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-143875 Summary of the Invention [Problem to be solved by the invention]

[0005] Although installing multiple camera devices as in Patent Document 1 reduces blind spots that cannot be captured by the camera devices, it is still troublesome for users to quickly select the camera device that can actually capture the area they want to check.

[0006] An object of the present disclosure is to provide a technology for efficiently monitoring a monitoring position in a monitored space. [Means for solving the problem]

[0007] a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Including, A monitoring system is provided.

[0008] a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Including, A monitoring device is provided.

[0009] The computer, Acquire a point cloud of a monitored space in which multiple imaging devices are arranged, Displaying the point cloud on a display device of an information terminal; acquiring a monitoring position in the monitored space designated via the information terminal; selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; A monitoring method is provided.

[0010] Computer, a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; To function as, Programs are offered. [Effects of the Invention]

[0011] According to the present disclosure, a monitoring position in a monitored space can be monitored efficiently. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a monitoring system. [Figure 2] 1 is a control flow of a monitoring system. [Figure 3] FIG. 2 is a perspective view of a monitored space. [Figure 4] FIG. 1 is a schematic diagram illustrating the configuration of a monitoring system. [Figure 5] FIG. 2 is a block diagram of a monitoring server. [Figure 6] FIG. 2 is a block diagram of a user terminal. [Figure 7] FIG. 1 is a sequence diagram of a monitoring system. [Figure 8] FIG. 1 is a sequence diagram of a monitoring system. [Figure 9] This is an example of a display in a web browser. [Figure 10] This is an example of a display in a web browser. [Figure 11] This is an example of a display in a web browser. [Figure 12] This is an example of a display in a web browser. [Figure 13] FIG. 1 is a sequence diagram of a monitoring system. [Figure 14] FIG. 1 is a sequence diagram of a monitoring system. [Figure 15] This is an example of a display in a web browser. [Figure 16] This is an example of a display in a web browser. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Summary of the Disclosure) An outline of the present disclosure will be described below. Fig. 1 is a block diagram of a monitoring system 100. Fig. 2 is a control flow of the monitoring system 100.

[0014] As shown in Fig. 1, the surveillance system 100 includes a point cloud acquisition means 101 that acquires a point cloud of a monitored space in which multiple imaging devices are arranged. The surveillance system 100 also includes a point cloud display means 102 that displays the point cloud on a display device of an information terminal. The surveillance system 100 also includes a monitoring position acquisition means 103 that acquires a monitoring position within the monitored space specified via the information terminal. The surveillance system 100 also includes a selection means 104 that selects at least one selected imaging device from the multiple imaging devices as an imaging device that can capture the monitoring position.

[0015] As shown in Fig. 2, the point cloud acquisition means 101 acquires a point cloud of a monitored space in which multiple image capture devices are arranged (S101). The point cloud display means 102 displays the point cloud on a display device of an information terminal (S102). The monitoring position acquisition means 103 acquires a monitoring position in the monitored space specified via the information terminal (S103). The selection means 104 selects at least one selected image capture device from the multiple image capture devices as an image capture device capable of capturing the monitoring position (S104).

[0016] According to the above configuration, an imaging device capable of capturing an image of the monitoring position is selected, so that the monitoring position in the monitored space can be monitored efficiently.

[0017] (First embodiment) A first embodiment of the present disclosure will be described below. FIG. 3 shows a monitored space 2 including a substation 1. The substation 1 is a specific example of monitored equipment. The substation 1 includes multiple substation equipment 3. The multiple substation equipment 3 typically includes, but is not limited to, transformers, circuit breakers, disconnectors, lightning arresters, etc. The multiple substation equipment 3 includes substation equipment 3a, substation equipment 3b, and substation equipment 3c. The monitored space 2 is provided with multiple PTZ cameras 4 and multiple LiDAR (Light Detection and Ranging) devices 5 to monitor the substation 1. The PTZ camera 4 is a specific example of an imaging device. The LiDAR device 5 is a specific example of a point cloud generation device. At least two PTZ cameras 4 are provided in the monitored space 2. In this embodiment, the multiple PTZ cameras 4 include PTZ camera 4a, PTZ camera 4b, PTZ camera 4c, PTZ camera 4d, PTZ camera 4e, and PTZ camera 4f. At least one LiDAR device 5 is provided in the monitored space 2. In this embodiment, the multiple LiDAR devices 5 include a LiDAR device 5a, a LiDAR device 5b, a LiDAR device 5c, a LiDAR device 5d, a LiDAR device 5e, and a LiDAR device 5f.

[0018] As shown in FIG. 3, the multiple substation equipment 3 are arranged in a row, for example. The multiple PTZ cameras 4 are arranged in two rows, for example, with the multiple substation equipment 3 on either side. This makes it possible to monitor the substation 1 with fewer blind spots. The multiple LiDAR devices 5, like the multiple PTZ cameras 4, are arranged in two rows, for example, with the multiple substation equipment 3 on either side. This makes it possible to monitor the substation 1 with fewer blind spots. The arrangement of the substation equipment 3, PTZ cameras 4, and LiDAR devices 5 is not limited to the arrangement exemplified in FIG. 3. For convenience of explanation, it is assumed that a foreign object 6 is present in contact with the substation equipment 3a.

[0019] FIG. 4 shows a configuration diagram of a monitoring system 7. The monitoring system 7 is a system for monitoring a substation 1. As shown in FIG. 4, the monitoring system 7 includes a monitoring server 8, a user terminal 9 operated by a monitor, and the above-mentioned multiple PTZ cameras 4 and multiple LiDAR devices 5. The monitoring server 8, the user terminal 9, the multiple PTZ cameras 4, and the multiple LiDAR devices 5 are typically configured to be able to communicate with each other via a WAN (Wide Area Network) or a LAN (Local Area Network). The user terminal 9 is a specific example of an information terminal.

[0020] Each PTZ camera 4 is a camera that can be remotely controlled by control commands from the monitoring server 8. Each PTZ camera 4 is configured to be able to pan, tilt, and zoom freely based on control commands from the monitoring server 8. Each PTZ camera 4 captures an image of the substation 1 to generate a captured image, and transmits the generated captured image together with identification information of the PTZ camera 4 to the monitoring server 8. It is assumed that the installation location of each PTZ camera 4 is known.

[0021] Each LiDAR device 5 is a device that can be remotely controlled by a control command from the monitoring server 8. The LiDAR device 5 generates a point cloud by measuring the distance to the substation 1 based on a control command from the monitoring server 8, and transmits the generated point cloud to the monitoring server 8 together with identification information of the LiDAR device 5. It is assumed that the installation position of each LiDAR device 5 is known. As described above, the LiDAR device 5 is one specific example of a point cloud generation device. The point cloud generation device is not limited to the LiDAR device 5, but may also be a radar device (radio detection and ranging), may be configured with a stereo camera, or may be configured by combining multiple PTZ cameras 4 with SfM (structure from motion) technology or MVS (multi-view stereo) technology.

[0022] FIG. 5 shows a block diagram of the monitoring server 8. As shown in FIG. 5, the monitoring server 8 includes a processor 8a, a memory 8b, and a communication interface 8c. The monitoring server 8 may be configured as a single device or may be realized by distributed processing using multiple devices. The processor 8a can access the memory 8b. The processor 8a communicates with the user terminal 9 and the like via the communication interface 8c. The processor 8a loads and executes programs stored in the memory 8b. As a result, the processor 8a causes the hardware, such as the processor 8a, to function as a point cloud acquisition unit 20, a point cloud synthesis unit 21, a point cloud image generation unit 22, a point cloud image transmission unit 23, a viewpoint information reception unit 24, a PTZ control unit 25, a monitoring position information reception unit 26, a camera selection unit 27, a camera selection information reception unit 28, a captured image acquisition unit 29, and a captured image transmission unit 30.

[0023] FIG. 6 shows a block diagram of the user terminal 9. As shown in FIG. 6, the user terminal 9 includes a processor 9a, a memory 9b, a communication interface 9c, an LCD (Liquid Crystal Display) 9d, and an input interface 9e. The input interface 9e is typically a pointing device or a keyboard. The processor 9a can access the memory 9b. The processor 9a communicates with the monitoring server 8 and the like via the communication interface 9c. The processor 9a loads and executes programs stored in the memory 9b. As a result, the processor 9a causes hardware such as the processor 9a to function as a point cloud image receiving unit 40, a point cloud image display unit 41, a viewpoint information transmitting unit 42, a monitoring position acquisition unit 43, a monitoring position information transmitting unit 44, a camera selecting unit 45, a camera selection information transmitting unit 46, a captured image receiving unit 47, and a captured image display unit 48. The LCD 9d is a specific example of a display device.

[0024] 5, the point cloud acquisition unit 20 acquires three-dimensional point clouds from the multiple LiDAR devices 5. Typically, the point cloud acquisition unit 20 acquires point clouds from the multiple LiDAR devices 5 twice a day. However, instead of this, the point cloud acquisition unit 20 may acquire point clouds from the multiple LiDAR devices 5 every five minutes.

[0025] The point cloud synthesis unit 21 synthesizes the point clouds acquired by the point cloud acquisition unit 20. Specifically, the point cloud synthesis unit 21 synthesizes the point clouds received from the LiDAR device 5a, the point cloud received from the LiDAR device 5b, the point cloud received from the LiDAR device 5c, the point cloud received from the LiDAR device 5d, the point cloud received from the LiDAR device 5e, and the point cloud received from the LiDAR device 5f, typically by aligning them using ICP (Iterative Closest Point) technology. The point cloud synthesis unit 21 stores the synthesized point cloud generated by the synthesis in the memory 8b. Hereinafter, the synthesized point cloud will also be simply referred to as a point cloud.

[0026] The point cloud image generator 22 reads the point cloud from the memory 8b, and captures the point cloud under predetermined imaging conditions in a computer to generate a point cloud image, which is a two-dimensional image. The imaging conditions are composed of a standpoint, a gaze point, and an angle of view.

[0027] The point cloud image transmission unit 23 transmits the point cloud image generated by the point cloud image generation unit 22 to the user terminal 9.

[0028] The viewpoint information receiving unit 24 receives viewpoint information from the user terminal 9. The viewpoint information is information indicating the above-mentioned imaging conditions.

[0029] The PTZ control unit 25 performs PTZ control of the multiple PTZ cameras 4.

[0030] The monitoring position information receiving unit 26 receives, from the user terminal 9, monitoring position information indicating a monitoring position designated by the monitor using the user terminal 9.

[0031] The camera selection unit 27 selects at least one PTZ camera 4 from the plurality of PTZ cameras 4 as an imaging device capable of capturing an image of the monitoring position.

[0032] Specifically, the camera selection unit 27 first generates voxel data of the monitored space 2 by voxelizing the point cloud. In this case, the camera selection unit 27 divides the monitored space 2 into multiple cubes, and when the number of point clouds contained in each cube reaches a predetermined number or more, it determines that an object exists in that cube, and leaves the cube in which the object exists as a voxel while deleting the other cubes. In short, the voxel data can be said to be a collection of voxels corresponding to the objects existing in the monitored space 2.

[0033] Then, the camera selection unit 27 determines whether or not an object exists between each PTZ camera 4 and the monitoring position based on the voxel data, and if it determines that no object exists between the PTZ camera 4 and the monitoring position, it determines that the PTZ camera 4 is capable of capturing an image of the monitoring position. In this way, the camera selection unit 27 selects at least one PTZ camera 4 from the multiple PTZ cameras 4 as an imaging device capable of capturing an image of the monitoring position. In the example of Fig. 3, of the multiple PTZ cameras 4, the PTZ cameras 4 that can capture an image of foreign object 6 at the monitoring position are PTZ camera 4a and PTZ camera 4d. PTZ camera 4a and PTZ camera 4d are specific examples of selected imaging devices.

[0034] The camera selection information receiving unit 28 receives from the user terminal 9 camera selection information indicating the PTZ camera 4 selected by the monitor from among multiple PTZ cameras 4 (PTZ camera 4a and PTZ camera 4d) that are capable of capturing images of a foreign object 6 at the monitoring position.

[0035] The captured image acquisition unit 29 acquires captured images from the PTZ camera 4 selected by the camera selection unit 27 or specified by the camera selection information.

[0036] The captured image transmitting unit 30 transmits the captured image acquired by the captured image acquiring unit 29 to the user terminal 9 .

[0037] Referring to FIG. 6, the point cloud image receiving unit 40 receives the point cloud image from the monitoring server 8.

[0038] The point cloud image display unit 41 displays the point cloud image received by the point cloud image receiving unit 40 on the LCD 9d.

[0039] When the monitor changes the viewpoint of the point cloud using the input interface 9e, the viewpoint information transmission unit 42 generates viewpoint information indicating the viewpoint after the change and transmits it to the monitoring server 8.

[0040] The monitoring position acquisition unit 43 acquires a monitoring position in the monitored space 2 that is specified by the monitor using the input interface 9e. For example, the monitor clicks on any point shown in the point cloud image using the input interface 9e. The monitoring position acquisition unit 43 identifies the monitoring position based on the clicked position.

[0041] The monitoring position information transmitting unit 44 transmits the monitoring position information indicating the monitoring position acquired by the monitoring position acquiring unit 43 to the monitoring server 8 .

[0042] The camera selection unit 45 accepts the monitor's selection of one of the multiple PTZ cameras 4 (PTZ camera 4a and PTZ camera 4d) that can capture an image of the foreign object 6 at the monitoring position, using the input interface 9e.

[0043] The camera selection information transmitting unit 46 transmits to the surveillance server 8 camera selection information indicating the PTZ camera 4 selected by the surveillance person.

[0044] The captured image receiving unit 47 receives the captured image from the monitoring server 8. The captured image display unit 48 displays the captured image received by the captured image receiving unit 47 on the LCD 9d.

[0045] Next, the operation of the monitoring system 7 will be described with reference to Fig. 7 to Fig. 12. Fig. 7 and Fig. 8 are sequence diagrams of the monitoring system 7. Fig. 9 to Fig. 12 are examples of screen displays on the LCD 9d.

[0046] First, the point cloud acquisition unit 20 acquires point clouds from the multiple LiDAR devices 5 (S300). Next, the point cloud synthesis unit 21 synthesizes the point cloud received from the LiDAR device 5a, the point cloud received from the LiDAR device 5b, the point cloud received from the LiDAR device 5c, the point cloud received from the LiDAR device 5d, the point cloud received from the LiDAR device 5e, and the point cloud received from the LiDAR device 5f (S310). Next, the point cloud image generation unit 22 captures the point clouds under initial imaging conditions to generate a two-dimensional point cloud image (S320). At this time, the point cloud image generation unit 22 simultaneously captures images of the multiple PTZ cameras 4 arranged in the monitored space 2 in addition to the point cloud to generate a two-dimensional point cloud image. Since the installation positions of the multiple PTZ cameras 4 are known, the point cloud image generation unit 22 may capture the point cloud after placing three-dimensional objects representing the cameras at the installation positions of the multiple PTZ cameras 4 in the point cloud coordinate system. The above initial imaging conditions are typically set in advance so that at least all of the substation equipment 3 and all of the PTZ cameras 4 are within the field of view, in other words, so that all of the substation equipment 3 and all of the PTZ cameras 4 can be viewed from above.

[0047] Next, the point cloud image transmitting unit 23 transmits the point cloud image generated by the point cloud image generating unit 22 to the user terminal 9 (S330). The point cloud image receiving unit 40 receives the point cloud image from the monitoring server 8 (S330). The point cloud image display unit 41 displays the point cloud image received by the point cloud image receiving unit 40 on the LCD 9d (S340). Specifically, as shown in FIG. 9, the point cloud image display unit 41 displays a point cloud image p in a point cloud image display area 51 of the display area 50a of the web browser 50. As shown in FIG. 9, in the point cloud image p, a plurality of substation facilities 3 and foreign objects 6 are depicted as point clouds, and a plurality of PTZ cameras 4 are depicted as icons. Furthermore, in the point cloud image p, identification names are attached to the plurality of PTZ cameras 4.

[0048] Next, if the monitor changes the viewpoint using the input interface 9e (S350: YES), the viewpoint information transmission unit 42 of the user terminal 9 generates viewpoint information indicating the changed viewpoint and transmits the viewpoint information to the monitoring server 8 (S360). Upon receiving the viewpoint information from the user terminal 9 (S360), the point cloud image generation unit 22 regenerates a point cloud image in accordance with the imaging conditions defined in the viewpoint information (S320).

[0049] If the observer has not performed a viewpoint change operation (S350: NO), the monitoring position acquisition unit 43 acquires the monitoring position specified by the observer using the input interface 9e (S360). Then, the monitoring position information transmission unit 44 transmits monitoring position information indicating the monitoring position to the monitoring server 8 (S370). The monitoring position information reception unit 26 receives the monitoring position information from the user terminal 9 (S370).

[0050] Next, the camera selection unit 27 of the monitoring server 8 selects at least one PTZ camera 4 from the plurality of PTZ cameras 4 that can capture an image of the monitoring position specified by the monitoring position information (S380).

[0051] If the camera selection unit 27 has selected, from the multiple PTZ cameras 4, multiple PTZ cameras 4 that can capture images of the monitoring position specified by the monitoring position information (S400: YES), the process proceeds to step S410. On the other hand, if the camera selection unit 27 has selected, from the multiple PTZ cameras 4, only one PTZ camera 4 that can capture images of the monitoring position specified by the monitoring position information (S400: NO), the process proceeds to step S500.

[0052] In step S410, the PTZ control unit 25 controls the imaging directions of the multiple PTZ cameras 4 capable of imaging the monitoring position so that the multiple PTZ cameras 4 capable of imaging the monitoring position can image the monitoring position (S410). Next, the captured image acquisition unit 29 acquires captured images from the multiple PTZ cameras 4 capable of imaging the monitoring position (S420). Next, the point cloud image generation unit 22 generates a point cloud image in which icons representing the multiple PTZ cameras 4 capable of imaging the monitoring position are highlighted in accordance with the initial imaging conditions described above (S430). Next, the captured image transmission unit 30 and the point cloud image transmission unit 23 transmit the multiple captured images acquired by the captured image acquisition unit 29 and the point cloud image generated by the point cloud image generation unit 22 to the user terminal 9 (S440). The captured image receiving unit 47 and the point cloud image receiving unit 40 receive the multiple captured images and point cloud image from the monitoring server 8 (S440). Next, as shown in Fig. 10, the point cloud image display unit 41 displays the point cloud image p received by the point cloud image receiving unit 40 in the point cloud image display area 51 (S450). As shown in Fig. 10, the PTZ camera 4a and the PTZ camera 4d are highlighted in the point cloud image p. Similarly, the taken image display unit 48 displays the multiple taken images q received by the taken image receiving unit 47 side by side in the taken image display area 52 of the display area 50a of the web browser 50 (S450). As an example, the point cloud image display area 51 and the taken image display area 52 are adjacent to each other in the display area 50a of the web browser 50.

[0053] Next, the camera selection unit 45 accepts the monitor's selection, using the input interface 9e, of one of the multiple PTZ cameras 4 (PTZ camera 4a and PTZ camera 4d) capable of capturing an image of the foreign object 6 at the monitoring position (S460). Specifically, the monitor operates the input interface 9e to click on one of the multiple PTZ cameras 4 highlighted in the point cloud image display area 51. Alternatively, the monitor clicks on one of the multiple captured images q displayed in the captured image display area 52. This selects the PTZ camera 4 that the monitor will use for monitoring. Next, the camera selection information transmission unit 46 transmits camera selection information indicating the PTZ camera 4 selected by the monitor to the monitoring server 8 (S470).

[0054] In step S500, the PTZ control unit 25 controls the imaging direction of the PTZ camera 4 selected by the camera selection unit 27 or the PTZ camera 4 selected by the monitor so that the PTZ camera 4 can capture an image of the monitoring position (S500). Next, the captured image acquisition unit 29 acquires captured images from the PTZ camera 4 (S510). Next, the point cloud image generation unit 22 captures a point cloud under the same imaging conditions as those of the PTZ camera 4 to generate a point cloud image (S520). Then, the captured image transmission unit 30 and the point cloud image transmission unit 23 of the monitoring server 8 transmit the captured images acquired by the captured image acquisition unit 29 in step S510 and the point cloud image generated by the point cloud image generation unit 22 in step S520 to the user terminal 9 (S530). The captured image reception unit 47 and the point cloud image reception unit 40 of the user terminal 9 receive the captured images and point cloud image from the monitoring server 8 (S530). 11, the point cloud image display unit 41 displays the point cloud image p received by the point cloud image receiving unit 40 in the point cloud image display area 51 (S540). Similarly, the captured image display unit 48 displays the captured image q received by the captured image receiving unit 47 in the captured image display area 52 (S540). In the example of FIG. 11, the observer selects PTZ camera 4d from among the multiple PTZ cameras 4 (PTZ camera 4a and PTZ camera 4d) that can capture an image of foreign object 6 at the monitoring position. The observer observes foreign object 6 by, for example, mutually observing point cloud image p and captured image q displayed on the web browser 50 as shown in FIG. 11. As a result of this observation, the observer may take any action necessary to remove foreign object 6, for example.

[0055] Next, if the monitor performs a ZOOM operation using the input interface 9e (S550: YES), the viewpoint information transmitting unit 42 generates ZOOM operation information and transmits it to the monitoring server 8 (S560). The viewpoint information receiving unit 24 receives the ZOOM operation information from the user terminal 9 (S560). Then, the PTZ control unit 25 performs ZOOM control of the PTZ camera 4 based on the ZOOM operation information received in step S560 (S500). Next, the captured image acquiring unit 29 acquires captured images from the PTZ camera 4 (S510). Next, the point cloud image generating unit 22 captures a point cloud under the same imaging conditions as those of the PTZ camera 4 and generates a point cloud image (S520). Then, the captured image transmitting unit 30 and the point cloud image transmitting unit 23 of the monitoring server 8 transmit the captured image acquired by the captured image acquiring unit 29 in step S510 and the point cloud image generated by the point cloud image generating unit 22 in step S520 to the user terminal 9 (S530). The captured image receiving unit 47 and the point cloud image receiving unit 40 of the user terminal 9 receive the captured image and the point cloud image from the monitoring server 8 (S530). Then, as shown in FIG. 12 , the point cloud image display unit 41 displays the point cloud image p received by the point cloud image receiving unit 40 in the point cloud image display area 51 (S540). Similarly, the captured image display unit 48 displays the captured image q received by the captured image receiving unit 47 in the captured image display area 52 (S540). In this way, the point cloud image p is also enlarged or reduced in conjunction with a ZOOM operation on the captured image q in the web browser 50, so that the foreign object 6 can be efficiently observed.

[0056] The first embodiment of the present disclosure has been described above. The above embodiment has the following features.

[0057] The surveillance server 8 includes a point cloud acquisition unit 20 as a point cloud acquisition means, a point cloud image transmission unit 23 as a point cloud display means, a monitoring position information reception unit 26 as a monitoring position acquisition means, and a camera selection unit 27 as a selection means. The point cloud acquisition unit 20 acquires a point cloud of the monitored space 2 in which multiple PTZ cameras 4 (imaging devices) are arranged. The point cloud image transmission unit 23 displays the point cloud on an LCD 9d (display device) of a user terminal 9 (information terminal). The monitoring position information reception unit 26 acquires a monitoring position in the monitored space 2 specified via the user terminal 9. The camera selection unit 27 selects PTZ camera 4a and PTZ camera 4d (at least one selected imaging device) from the multiple PTZ cameras 4 as PTZ cameras 4 capable of capturing images of the monitoring position. With the above configuration, the PTZ camera 4 capable of capturing images of the monitoring position is selected, thereby enabling efficient monitoring of the monitoring position in the monitored space 2.

[0058] The surveillance server 8 further includes a captured image acquisition unit 29 (captured image acquisition means) that acquires captured images from the multiple PTZ cameras 4, and a captured image transmission unit 30 (captured image display means) that displays the captured images acquired from the PTZ cameras 4 on the LCD 9d. With the above configuration, the surveillance person can observe the surveillance position through the captured images on the LCD 9d of the user terminal 9.

[0059] The surveillance server 8 further includes a PTZ control unit 25 (imaging device control means) that controls the multiple PTZ cameras 4. The PTZ control unit 25 controls the imaging direction of the PTZ cameras 4 so that the PTZ cameras 4 capture images of the monitoring position. With the above configuration, the surveillance person can view captured images in which the monitoring position is included in the angle of view without adjusting the imaging direction of the PTZ cameras 4.

[0060] 9 and 10, the point cloud image transmission unit 23 displays the point cloud on the LCD 9d of the user terminal 9 together with the multiple PTZ cameras 4. Specifically, the point cloud image transmission unit 23 transmits a point cloud image containing the point cloud and the multiple PTZ cameras 4 to the user terminal 9, thereby displaying the point cloud on the LCD 9d of the user terminal 9 together with the multiple PTZ cameras 4. With the above configuration, the monitor can easily grasp the positional relationship between the monitoring position and the multiple PTZ cameras 4.

[0061] 10, the point cloud image transmission unit 23 highlights the PTZ camera 4a and the PTZ camera 4d selected by the camera selection unit 27 on the LCD 9d. With the above configuration, the monitor can easily identify the PTZ camera 4 that is suitable for observing the monitoring position among the multiple PTZ cameras 4.

[0062] When the camera selection unit 27 selects a plurality of PTZ cameras 4 (PTZ camera 4a and PTZ camera 4d), the captured image transmission unit 30 displays, on the LCD 9d, a plurality of captured images corresponding to the plurality of PTZ cameras 4 selected by the camera selection unit 27, as shown in Fig. 10. With the above configuration, the monitoring position can be observed from different angles, and therefore the monitoring position can be monitored efficiently.

[0063] When the camera selection unit 27 selects a plurality of PTZ cameras 4 (PTZ camera 4a and PTZ camera 4d), the captured image transmission unit 30 displays on the LCD 9d the captured image corresponding to the PTZ camera 4 selected via the user terminal 9 from the plurality of PTZ cameras 4 selected by the camera selection unit 27, as shown in Figures 11 and 12. With the above configuration, the monitor can display on the LCD 9d the captured image that is most suitable for observing the monitoring position.

[0064] The camera selection unit 27 generates voxel data of the monitored space 2 by voxelizing the point cloud, and selects at least one PTZ camera 4 based on the voxel data. With the above configuration, it is possible to efficiently select a PTZ camera 4 that can capture images of the monitoring position.

[0065] (Second embodiment) Next, a second embodiment of the present disclosure will be described. The following description will focus on the differences between this embodiment and the first embodiment, and will omit redundant explanations.

[0066] 11 and 12, the imaging conditions of the captured image q displayed in the captured image display area 52 are made to match the imaging conditions of the point cloud image p displayed in the point cloud image display area 51. When a zoom operation is performed in the captured image display area 52, the point cloud image p in the point cloud image display area 51 is enlarged or reduced in conjunction with the zoom operation.

[0067] In contrast to this, in this embodiment, the point cloud image display area 51 displays a point cloud image p generated based on imaging conditions that allow a bird's-eye view of all of the substation equipment 3 and all of the PTZ cameras 4, and the imaging conditions of the captured image q and the point cloud image p are not made to match. This makes it possible to monitor the foreign object 6 without losing track of the positional relationship between the PTZ cameras 4 and the foreign object 6. Specifically, this is as follows.

[0068] The operation of the monitoring system 7 will be described below with reference to Figs. 13 to 16. Figs. 13 and 14 are sequence diagrams of the monitoring system 7. Figs. 15 and 16 are examples of screen displays on the LCD 9d. Note that the operations from step S300 to step S380 shown in Fig. 7 are similarly applied to the second embodiment, and therefore a description of these operations will be omitted. In Fig. 7, when the camera selection unit 27 of the monitoring server 8 selects at least one PTZ camera 4 from the multiple PTZ cameras 4 that can capture images of the monitoring position specified by the monitoring position information (S380), step S600 is then executed.

[0069] If the camera selection unit 27 has selected, from the multiple PTZ cameras 4, multiple PTZ cameras 4 capable of capturing images of the monitoring position specified by the monitoring position information (S600: YES), the process proceeds to step S610. In this case, for convenience of explanation, it is assumed that the camera selection unit 27 selects PTZ camera 4a and PTZ camera 4d from the multiple PTZ cameras 4 as PTZ cameras 4 capable of capturing images of the monitoring position specified by the monitoring position information. On the other hand, if the camera selection unit 27 has selected only one PTZ camera 4 from the multiple PTZ cameras 4 capable of capturing images of the monitoring position specified by the monitoring position information (S600: NO), the process proceeds to step S710. In this case, for convenience of explanation, it is assumed that the camera selection unit 27 selects PTZ camera 4a from the multiple PTZ cameras 4 as the PTZ camera 4 capable of capturing images of the monitoring position specified by the monitoring position information.

[0070] In step S610, the PTZ control unit 25 controls the imaging directions of the multiple PTZ cameras 4 capable of imaging the monitoring position so that the multiple PTZ cameras 4 capable of imaging the monitoring position can image the monitoring position (S610). Next, the captured image acquisition unit 29 acquires multiple captured images from the multiple PTZ cameras 4 capable of imaging the monitoring position (S620). Next, the point cloud image generation unit 22 generates a point cloud image in which icons representing the multiple PTZ cameras 4 capable of imaging the monitoring position are highlighted in accordance with the initial imaging conditions described above (S630). Next, the captured image transmission unit 30 and the point cloud image transmission unit 23 transmit the multiple captured images acquired by the captured image acquisition unit 29 and the point cloud image generated by the point cloud image generation unit 22 to the user terminal 9 (S640). The captured image reception unit 47 and the point cloud image reception unit 40 receive the multiple captured images and point cloud image from the monitoring server 8 (S640). Next, as shown in Fig. 15, the point cloud image display unit 41 displays the point cloud image p received by the point cloud image receiving unit 40 in the point cloud image display area 51 (S650). As shown in Fig. 15, the PTZ camera 4a and the PTZ camera 4d are highlighted in the point cloud image p. Similarly, the taken image display unit 48 displays the multiple taken images q received by the taken image receiving unit 47 in a vertical arrangement in the taken image display area 52 (S650).

[0071] Next, when the monitor changes the viewpoint in the point cloud image display area 51 using the input interface 9e (S660: YES), the viewpoint information transmitter 42 of the user terminal 9 generates viewpoint information indicating the changed viewpoint and transmits the viewpoint information to the monitoring server 8 (S670). When the point cloud image generator 22 receives the viewpoint information from the user terminal 9 (S670), it regenerates a point cloud image in accordance with the imaging conditions defined by the viewpoint information (S630). As a result, as shown in FIGS. 15 and 16 , the monitor can freely change the viewpoint (stand point, focus point, and angle of view) from which to observe the point cloud in the point cloud image display area 51 (S660), thereby monitoring the foreign object 6 by referring to the multiple captured images q displayed in the captured image display area 52 while fully understanding the positional relationship of the PTZ cameras 4a and 4d with respect to the foreign object 6.

[0072] If the monitor has not performed a viewpoint change operation in step S660 (S660: NO), the viewpoint information transmitter 42 determines whether the monitor has performed a ZOOM operation in the captured image display area 52 using the input interface 9e (S680). If it is determined that the monitor has performed a ZOOM operation in the captured image display area 52 using the input interface 9e (S680: YES), the viewpoint information transmitter 42 generates ZOOM operation information and transmits it to the monitoring server 8 (S690), and the process returns to step S610. The viewpoint information receiver 24 receives the ZOOM operation information from the user terminal 9 (S690). Then, the PTZ control unit 25 performs ZOOM control on the PTZ camera 4 based on the ZOOM operation information received in step S690 (S610). As a result, as shown in Figures 15 and 16, each time the monitor performs a ZOOM operation in the captured image display area 52, the captured image q shown in the captured image display area 52 is enlarged or reduced, so that the foreign object 6 can be displayed at an appropriate size in the captured image display area 52.

[0073] In step S710, the PTZ control unit 25 controls the imaging direction of the PTZ camera 4 capable of imaging the monitoring position so that the PTZ camera 4 capable of imaging the monitoring position can image the monitoring position (S710). Next, the captured image acquisition unit 29 acquires captured images from the PTZ camera 4 capable of imaging the monitoring position (S720). Next, the point cloud image generation unit 22 generates a point cloud image in which icons indicating the PTZ cameras 4 capable of imaging the monitoring position are highlighted in accordance with the initial imaging conditions described above (S730). Next, the captured image transmission unit 30 and the point cloud image transmission unit 23 transmit the captured images acquired by the captured image acquisition unit 29 and the point cloud images generated by the point cloud image generation unit 22 to the user terminal 9 (S740). The captured image reception unit 47 and the point cloud image reception unit 40 receive the captured images and point cloud images from the monitoring server 8 (S740). Next, the point cloud image display unit 41 displays the point cloud image p received by the point cloud image receiving unit 40 in the point cloud image display area 51 (S750). The PTZ camera 4a is highlighted in the point cloud image p. Similarly, the captured image display unit 48 displays the captured image q received by the captured image receiving unit 47 in the captured image display area 52 (S750).

[0074] Next, when the monitor changes the viewpoint in the point cloud image display area 51 using the input interface 9e (S760: YES), the viewpoint information transmitter 42 of the user terminal 9 generates viewpoint information indicating the changed viewpoint and transmits the viewpoint information to the monitoring server 8 (S770). When the point cloud image generator 22 receives the viewpoint information from the user terminal 9 (S770), it regenerates a point cloud image in accordance with the imaging conditions defined by the viewpoint information (S730). This allows the monitor to freely change the viewpoint (stand point, focus point, and angle of view) from which to observe the point cloud in the point cloud image display area 51 (S760), thereby enabling the monitor to monitor the foreign object 6 by referring to the captured image q displayed in the captured image display area 52 while fully understanding the positional relationship of the PTZ camera 4a with respect to the foreign object 6.

[0075] If the monitor has not performed a viewpoint change operation in step S760 (S760: NO), the viewpoint information transmission unit 42 determines whether the monitor has performed a zoom operation in the captured image display area 52 using the input interface 9e (S780). If it is determined that the monitor has performed a zoom operation in the captured image display area 52 using the input interface 9e (S780: YES), the viewpoint information transmission unit 42 generates zoom operation information and transmits it to the monitoring server 8 (S790), and the process returns to step S710. The viewpoint information receiving unit 24 receives the zoom operation information from the user terminal 9 (S790). Then, the PTZ control unit 25 performs zoom control on the PTZ camera 4 based on the zoom operation information received in step S790 (S710). As a result, each time the monitor performs a zoom operation in the captured image display area 52, the captured image q displayed in the captured image display area 52 is enlarged or reduced, so that the foreign object 6 can be displayed at an appropriate size in the captured image display area 52.

[0076] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0077] For example, in the first and second embodiments described above, the monitoring server 8 generates a point cloud image and transmits the generated point cloud image to the user terminal 9. However, instead of this, the monitoring server 8 may transmit the point cloud to the user terminal 9, and the user terminal 9 may capture the point cloud to generate a point cloud image, which may then be displayed on the LCD 9d.

[0078] Furthermore, when a ZOOM operation is performed in the captured image display area 52, the captured image display section 48 may omit re-capturing the image with the PTZ camera 4 and simply enlarge or reduce the captured image q.

[0079] In the above example, the program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Non-transitory computer-readable media include, for example, magnetic recording media, magneto-optical recording media, CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories. Semiconductor memories include, for example, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be supplied to a computer by various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable media can be supplied to a computer via wired communication paths such as electrical wires and optical fibers, or wireless communication paths.

[0080] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0081] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Including, Surveillance system. (Appendix 2) 10. The monitoring system of claim 1, captured image acquisition means for acquiring captured images from the plurality of imaging devices; a captured image display means for displaying at least one captured image acquired from the at least one selected imaging device on the display device; Further comprising: Surveillance system. (Appendix 3) 3. The monitoring system of claim 2, further comprising: further comprising an imaging device control means for controlling the plurality of imaging devices; the imaging device control means controls the imaging direction of the at least one imaging device so that the at least one selected imaging device images the monitoring position; Surveillance system. (Appendix 4) 10. The monitoring system of claim 1, the point cloud display means displays the point cloud together with the plurality of imaging devices on the display device. Surveillance system. (Appendix 5) 5. The monitoring system of claim 4, the point cloud display means highlights the at least one selected imaging device on the display device. Surveillance system. (Appendix 6) 6. The monitoring system of claim 5, further comprising: the at least one selected imaging device includes a plurality of selected imaging devices; the captured image display means displays, on the display device, a captured image corresponding to a selected imaging device selected via the information terminal from among the plurality of selected imaging devices. Surveillance system. (Appendix 7) 10. The monitoring system of claim 1, the selection means generates voxel data of the monitored space by voxelizing the point cloud, and selects the at least one imaging device based on the voxel data. Surveillance system. (Appendix 8) a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Including, monitoring equipment. (Appendix 9) The computer, Acquire a point cloud of a monitored space in which multiple imaging devices are arranged, Displaying the point cloud on a display device of an information terminal; acquiring a monitoring position in the monitored space designated via the information terminal; selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Monitoring method. (Appendix 10) Computer, a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; To function as, program.

[0082] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 7 that are dependent on Supplementary Notes 1 may also be dependent on Supplementary Notes 8 to 10 in the same dependency relationship as Supplementary Notes 2 to 7. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0083] p point cloud image q Captured image 1. Substation 2. Monitored Space 3. Substation equipment 3a Substation equipment 3b Substation equipment 3c Substation equipment 4 PTZ cameras 4a PTZ camera 4b PTZ camera 4c PTZ camera 4D PTZ Camera 4e PTZ camera 4f PTZ camera 5 LiDAR device 5a LiDAR device 5b LiDAR device 5c LiDAR device 5d LiDAR device 5e LiDAR device 5f LiDAR device 6 Foreign object 7. Surveillance System 8 Monitoring Server 8a processor 8b Memory 8c Communication Interface 9 User Devices 9a processor 9b Memory 9c Communication Interface 9d LCD 9e Input Interface 20 point cloud acquisition section 21 Point group synthesis section 22 Point cloud image generation unit 23 Point cloud image transmission unit 24 Viewpoint information receiving unit 25 PTZ control unit 26 Monitoring location information receiving unit 27 Camera Selection Department 28 Camera selection information receiver 29 Image acquisition unit 30 Captured image transmission unit 40 Point cloud image receiving unit 41 Point cloud image display section 42 Viewpoint information transmission unit 43 Monitoring position acquisition unit 44 Monitoring location information transmission unit 45 Camera selection section 46 Camera selection information transmitter 47 Image receiving unit 48 Captured image display section 50 Web Browsers 50a Display Area 51 Point cloud image display area 52 Captured image display area

Claims

1. a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Including, Surveillance system.

2. 2. The monitoring system of claim 1, captured image acquisition means for acquiring captured images from the plurality of imaging devices; a captured image display means for displaying at least one captured image acquired from the at least one selected imaging device on the display device; Further comprising: Surveillance system.

3. 3. The monitoring system of claim 2, further comprising an imaging device control means for controlling the plurality of imaging devices; the imaging device control means controls the imaging direction of the at least one imaging device so that the at least one selected imaging device images the monitoring position; Surveillance system.

4. 2. The monitoring system of claim 1, the point cloud display means displays the point cloud together with the plurality of imaging devices on the display device. Surveillance system.

5. 5. The monitoring system of claim 4, the point cloud display means highlights the at least one selected imaging device on the display device; Surveillance system.

6. 3. The monitoring system of claim 2, the at least one selected imaging device includes a plurality of selected imaging devices; the captured image display means displays, on the display device, a captured image corresponding to a selected imaging device selected via the information terminal from among the plurality of selected imaging devices. Surveillance system.

7. 2. The monitoring system of claim 1, the selection means generates voxel data of the monitored space by voxelizing the point cloud, and selects the at least one imaging device based on the voxel data. Surveillance system.

8. a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Including, monitoring equipment.

9. The computer Acquire a point cloud of a monitored space in which multiple imaging devices are arranged, Displaying the point cloud on a display device of an information terminal; acquiring a monitoring position in the monitored space designated via the information terminal; selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; Monitoring method.

10. Computer, a point cloud acquisition means for acquiring a point cloud of a monitored space in which a plurality of imaging devices are arranged; a point cloud display means for displaying the point cloud on a display device of an information terminal; a monitoring position acquisition means for acquiring a monitoring position in the monitored space designated via the information terminal; a selection means for selecting at least one selected imaging device from the plurality of imaging devices as an imaging device capable of imaging the monitoring position; To function as, program.

Images