surveillance system

Abstract

To provide a monitoring system capable of reducing / shortening adjustment work until system operation by automating registration of positional information in a local coordination system on a LiDAR and the like and positional information in the local coordination system on a monitoring camera in installation work by automatically collecting the positional information on the LiDAR and the like and the positional information on the monitoring camera and expanding them into a global coordination system of the whole monitoring space.SOLUTION: A monitoring system comprises an integrated management server 4 for: creating a global coordination system from information on latitudes / longitudes of a plurality of LiDARs 1; synthesizing respective local coordination systems created at the respective LiDARs 1 so that detected objects including monitoring cameras 2 are overlapped; further expanding the synthesized local coordination systems into the created global coordination system; identifying positions of the monitoring cameras 2 on the basis of the global coordination system; and controlling the monitoring cameras 2.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a surveillance system using multiple LiDARs (Light Detection And Ranging / remote sensing devices) and mobile surveillance cameras, and more particularly to a surveillance system that allows easy setting of parameters required for system construction. [Background technology]

[0002] [Prior Art] With conventional surveillance systems, parameters for the location information of sensors such as LiDAR and surveillance cameras had to be manually registered into the system during installation, which required time before the system could begin operation.

[0003] Furthermore, while there are systems that link various sensors with surveillance cameras, there are no systems that can synthesize and display images of objects detected by LiDAR and other sensors with images from surveillance cameras.

[0004] [Related Technology] As a related prior art, there is Japanese Patent No. 6823211 "Method for localizing an autonomous vehicle including multiple sensors" (Patent Document 1). Patent Document 1 discloses that a plurality of sensors are used to identify the position of a ground vehicle for autonomous driving of an autonomous vehicle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6823211 Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, with conventional surveillance systems, the location information of sensors such as LiDAR and the location information of surveillance cameras had to be manually registered during installation work, which required a great deal of adjustment work before the system could be put into operation, resulting in the problem of it taking time and costing money from installation work to the start of system operation.

[0007] In addition, there is no system that can combine and display the objects detected by LiDAR or other devices with the images from the surveillance camera, so the objects cannot be overlaid on the surveillance images, which creates the problem of not being able to immediately identify the detected objects.

[0008] Incidentally, Patent Document 1 does not describe a configuration for automating the registration of position information from LiDAR or the like and position information from a surveillance camera, and overlaying a surveillance object detected by LiDAR or the like on surveillance footage from the surveillance camera.

[0009] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a surveillance system that can automate the registration of position information of LiDAR, etc. and surveillance cameras during installation work by automatically collecting position information of the local coordinate system of LiDAR, etc. and position information of the local coordinate system of surveillance cameras and expanding it into a global coordinate system of the entire monitored space, thereby reducing and shortening the adjustment work required until the system is operational.

[0010] Another object of the present invention is to provide a surveillance system that can overlay a surveillance object detected by a LiDAR or the like onto the surveillance image of a surveillance camera, making it easy to understand the detected object. [Means for solving the problem]

[0011] The present invention, which aims to solve the problems of the above-mentioned conventional example, is a surveillance system in which a plurality of remote sensing devices, a plurality of surveillance cameras, and an integrated management server that performs surveillance-related processing are connected via a wireless network, The remote sensing device has a built-in GPS to acquire location information of the device, detects the direction and distance of the detected object and the surveillance camera, generates a local coordinate system indicating the positions of the detected object and the surveillance camera from the direction and distance of the detected object and the surveillance camera, with the position of the remote sensing device as the origin, and transmits the location information, the direction and distance of the detected object and the surveillance camera, and the local coordinate system to the integrated management server; The integrated management server Multiple remote sensing equipment Multiple sent from Location information By synthesizingA global coordinate system is generated, and each remote sensing device acquires a local coordinate system; The overlapping detected in each of the acquired local coordinate systems Detected object and the location of surveillance cameras The local coordinate systems are synthesized so that they overlap, and the synthesized local coordinate systems are expanded into the global coordinate system.

[0012] In the monitoring system, the integrated management server performs the following for each acquired local coordinate system: Duplicate Detected object and surveillance cameras place Even if the positions are overlapped, If it is misaligned, The integrated management server acquires the local coordinate system and the remote sensing device detects the The average value of the distance from the remote sensing device is calculated, and the position is determined by the difference between the average value and the detected distance. Place The local coordinate system synthesized by correcting the deviation amount is expanded into the global coordinate system.

[0013] The present invention is characterized in that in the above monitoring system, the integrated management server periodically repeats a process of expanding the synthesized local coordinate system into a global coordinate system.

[0014] The present invention is characterized in that in the above-mentioned monitoring system, the integrated management server controls the monitoring camera so as to point it toward the position coordinates of the detected object shown in the global coordinate system.

[0015] The present invention provides the above-mentioned monitoring system, A terminal device that displays images from the surveillance camera is connected to the wireless network. The integrated management server collects point cloud data of objects detected by remote sensing equipment. all When a rectangle including the object is formed in the global coordinate system and the surveillance camera is directed to the center coordinate of the rectangle, end In the image, Detected object The feature of this method is that a rectangle corresponding to the image is overlaid.

[0016] In the above-mentioned present invention, the integrated management server When an object is detected by the remote sensing device, a warning message will be displayed on the terminal device. Report When a correct or incorrect report is input from the terminal device in response to the report, Accumulating data from remote sensing devices in the case of correct reports, and conducting machine learning using the data, Detection of detected objects The method is characterized by using the results of the machine learning. [Effects of the Invention]

[0017] According to the present invention, there is provided a surveillance system in which a plurality of remote sensing devices, a plurality of surveillance cameras, and an integrated management server that performs surveillance-related processing are connected via a wireless network, The remote sensing device has a built-in GPS to acquire location information of the device, detects the direction and distance of the detected object and the surveillance camera, generates a local coordinate system indicating the positions of the detected object and the surveillance camera from the direction and distance of the detected object and the surveillance camera, with the position of the remote sensing device as the origin, and transmits the location information, the direction and distance of the detected object and the surveillance camera, and the local coordinate system to the integrated management server; The integrated management server Multiple remote sensing equipment Multiple sent from Location information By synthesizing A global coordinate system is generated, and each remote sensing device acquires a local coordinate system; The overlapping detected in each of the acquired local coordinate systems Detected object and the location of surveillance cameras This surveillance system combines the local coordinate systems so that they overlap, and then expands the combined local coordinate systems into a global coordinate system, which has the effect of identifying the positions of multiple surveillance cameras and automating the registration of surveillance camera position information. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a schematic diagram of the system. [Figure 2] FIG. 2 is a flow chart showing the outline of the processing of the present system. [Figure 3] FIG. 1 is an image diagram of a global coordinate system. [Figure 4] FIG. 1 is an image diagram of a local coordinate system. [Figure 5] This is an image of the global coordinate system when a suspicious person is detected. [Figure 6] FIG. 10 is an image of a detection image when a suspicious person is detected. [Figure 7] FIG. 10 is a process image diagram for overlaying a detection frame on an image. [Figure 8] This is an image of the XY plane of the global coordinate system. [Figure 9] This is an image of the XZ plane of the global coordinate system. DETAILED DESCRIPTION OF THE INVENTION

[0019] An embodiment of the present invention will be described with reference to the drawings. [Outline of the embodiment] According to an embodiment of the present invention Monitoring The system (this system) generates a global coordinate system from latitude / longitude information from multiple remote sensing devices (LiDAR), synthesizes the local coordinate systems generated by each LiDAR so that detected objects, including detected surveillance cameras, overlap, and then expands the synthesized local coordinate system into the generated global coordinate system, identifies the positions of multiple surveillance cameras based on the global coordinate system, and controls the surveillance cameras.This system can automate the registration of surveillance camera position information and also automates the control of surveillance cameras in response to detected objects.

[0020] In addition, this system forms a rectangle containing point cloud data of the object detected by the LiDAR in a global coordinate system, and when the surveillance camera is pointed at the center coordinates of the rectangle, the system overlays the rectangle corresponding to the detected object on the image displayed on the terminal device that displays the surveillance camera image, making it easier to understand the detected object.

[0021] [This system: Figure 1] The configuration of this system will be described with reference to Figure 1. Figure 1 is a schematic diagram of the system. As shown in Figure 1, this system basically comprises multiple remote sensing devices (LiDAR) 1 that serve as sensors, multiple mobile surveillance cameras (monitoring cameras) 2, a wireless network 3, an integrated management server 4, and an operation terminal 5.

[0022] The LiDAR 1 and the surveillance camera 2 are wirelessly connected to a wireless network 3. Furthermore, the integrated management server 4 and the operation terminal 5 are connected to the wireless network 3 via a wired connection.

[0023] Here, the prerequisites for this system will be explained. The LiDAR 1 is installed horizontally to the ground in the monitored space. The ground is assumed to be level and without slope. Furthermore, the individual coordinate system formed by sensing by the LiDAR 1 is referred to as a "local coordinate system," and the coordinate system integrated by the integrated management server 4 as the entire monitored space is referred to as a "global coordinate system."

[0024] [Parts of this system] Next, each part of this system will be specifically described. [LiDAR1] The LiDAR 1 is a remote sensing device equipped with a laser emitter and an optical sensor, and the optical sensor receives the reflected light of the laser light emitted (emitted) from the laser emitter to detect the direction and distance of the monitored object (detected object) and transmits this information to the integrated management server 4. The integrated management server 4 then acquires movement trajectory information, movement speed information, and point cloud density information based on the direction and distance of the detected monitored object, as well as time information. This direction and distance of the monitored object are used to generate a local coordinate system for each LiDAR 1.

[0025] In addition, the LiDARs 1 are arranged so that their detection ranges overlap each other. The LiDAR 1 also has a built-in GPS (Global Positioning System) function, and transmits location information (latitude / longitude information) of the latitude / longitude of the installation position to other LiDARs 1 and the integrated management server 4 via the wireless network 3. The integrated management server 4 generates a global coordinate system based on the latitude / longitude information collected from all of the LiDARs 1.

[0026] Furthermore, the LiDAR 1 calculates the position information (local position information) of the adjacent LiDAR 1 in the local coordinate system from the direction of the laser light emitted from the adjacent LiDAR 1 and the latitude / longitude information of the adjacent LiDAR 1. A specific calculation method will be described later.

[0027] [Surveillance Camera 2] The surveillance camera 2 is a movable surveillance camera that receives pan / tilt / zoom control values from the integrated management server 4 and directs itself toward a detected object according to these control values. The control of directionality toward a detected object will be described later.

[0028] [Wireless Network 3] The wireless network 3 is a communication network that realizes wireless communication such as the fifth generation (5G). The wireless network 3 is connected to the LiDAR 1 and the monitoring camera 2 by wireless connection, and to the integrated management server 4 and the operation terminal 5 by wired connection.

[0029] [Master Management Server 4] The integrated management server 4 of this system is basically configured to perform global coordinate system generation processing, local coordinate system integration processing, global coordinate system expansion (reflection / conversion) processing, detection processing, alarm processing, learning processing, etc. These processes may be distributed among a plurality of processing devices.

[0030] The integrated management server 4 comprises a control unit, a storage unit, and an interface unit, and the processing programs stored in the storage unit run on the control unit to realize the functions of the above-mentioned processes. The storage unit also stores data on the local coordinate system and the global coordinate system, as well as data related to detection and data for controlling the surveillance camera 2. The interface unit is used to connect the integrated management server 4 to the wireless network 3 .

[0031] [Global coordinate system generation process] The integrated management server 4 generates a global coordinate system for the entire monitored space from the latitude / longitude information of all the LiDARs 1. The specific generation of the global coordinate system will be described later.

[0032] [Local coordinate system integration processing] Furthermore, the integrated management server 4 acquires the coordinates of adjacent LiDARs 1, obstacles, and the monitoring camera 2 obtained by sensing from each LiDAR 1, acquires a local coordinate system in which these coordinates are plotted, and further synthesizes the local coordinate system. The specific synthesis of the local coordinate system will be described later.

[0033] [Global coordinate system expansion processing] The integrated management server 4 then expands (reflects) the synthesized local coordinate system into a global coordinate system. Since the position coordinates of the surveillance camera 2 are specified in the global coordinate system, the position information of the surveillance camera 2 can be automatically registered in the integrated management server 4. The specific generation and expansion of the global coordinate system will be described later.

[0034] [Detection process] In addition, when the LiDAR 1 detects a monitored object, the integrated management server 4 receives information on the direction and distance of the detected object from the LiDAR 1, identifies the coordinates (position) in the local coordinate system, and then expands it into the global coordinate system to identify the coordinates (position) in the global coordinate system, and controls the surveillance camera 2 to point in the direction of those coordinates.

[0035] [Alarm processing] Furthermore, the integrated management server 4 displays the video captured by the surveillance camera 2 whose directionality has been controlled on the display unit of the operation terminal 5, and issues an alert to call attention. Then, the monitor checks the content of the issued report on the operation terminal 5 and inputs "true report" or "false report", and the input result is sent to the integrated management server 4.

[0036] [Learning process] Furthermore, the integrated management server 4 stores the sensing data of the LiDAR 1 in the "true report" inputted to the operation terminal 5 in a storage unit, and uses the data as training data to train an AI (Artificial Intelligence) on it, so that the detection process is performed using the trained AI model. This makes it possible to improve the accuracy of detection as the "true report."

[0037] [Operation terminal 5] The operation terminal 5 is a terminal device that displays the images captured by the directionally controlled surveillance camera 2 on the display unit under control of the integrated management server 4, issues an alarm from the speaker, and, when the monitor inputs a "true report" or "false report" from the input unit, transmits the information of "true report" or "false report" to the integrated management server 4.

[0038] [Processing overview of this system: Figure 2] Next, an outline of the processing in this system will be explained with reference to Fig. 2. Fig. 2 is a flow chart showing the outline of the processing in this system. As shown in FIG. 2, the integrated management server 4 of this system generates a global coordinate system from the latitude / longitude information of the LiDAR 1 connected to the wireless network 3 (S1).

[0039] Next, the integrated management server 4 acquires each local coordinate system from the measurement results obtained by sensing each LiDAR 1 (S2). Furthermore, the integrated management server 4 synthesizes each of the local coordinate systems (S3), and corrects errors during this synthesis. Then, the synthesized local coordinate system is expanded (reflected / transformed) into the generated global coordinate system (S4).

[0040] Furthermore, when a new object is detected by the LiDAR 1, the integrated management server 4 performs a detection process (S5). Specifically, when LiDAR1 detects a suspicious event (such as a suspicious person or object), it transmits the detected coordinates of the suspicious event in the local coordinate system to the integrated management server 4, and the integrated management server 4 receives the detected coordinates in the local coordinate system.

[0041] Next, the integrated management server 4 controls the monitoring cameras 2 in response to suspicious events (S6). Specifically, the integrated management server 4 calculates the distance and direction from the position coordinates of the surveillance camera 2 in the global coordinate system and the detection coordinates in the local coordinate system, and controls the surveillance camera 2 to point towards the suspicious event.

[0042] The integrated management server 4 then issues a warning to the operation terminal 5, and causes the operation terminal 5 to display and output an image captured by the surveillance camera 2 whose directionality has been controlled (S7). Here, on the operation terminal 5, the monitor registers the reported event as a "true report" if it is a suspicious event, or as a "false report" if it is a plant or tree, etc., and this information is sent to the integrated management server 4.

[0043] Furthermore, the integrated management server 4 links the movement trajectory information, movement speed information, and point cloud density information for each alert and stores them in the memory unit, and in the case of a "true alert," stores this information in the memory unit as training data for machine learning (S8). This training data will then be subjected to machine learning using AI to detect only suspicious events, reduce false alarms, and improve detection accuracy. When the above process is completed, the process returns to step S2 and is repeated periodically.

[0044] [Global coordinate system: Figure 3] Next, the global coordinate system generated by the integrated management server 4 will be described with reference to Fig. 3. Fig. 3 is an image diagram of the global coordinate system. The integrated management server 4 calculates the average value of the latitude / longitude of all LiDARs 1 connected to the wireless network 3 from the latitude / longitude information transmitted from the LiDARs 1, and generates a global coordinate system with the coordinate point of that average value as the origin (center point).

[0045] In Fig. 3, the center point of the global coordinate system is found from the latitude / longitude information of the LiDARs 1a to 1d, a mesh of the global coordinate system is formed, and the positions of the LiDARs 1a to 1d are set on the mesh. The size of the mesh is arbitrary, and although the position of the surveillance camera 2 is shown in Fig. 3, this is displayed after synthesis of the local coordinate system.

[0046] [Local Coordinate System Merging] If the laser irradiation range of LiDAR1 is 180° in front, the direction of LiDAR1's local coordinate system is recognized from the laser irradiation direction of adjacent LiDAR1s detected by LiDAR1, and the local coordinate systems are synthesized so that the position coordinates of the overlapping detected LiDAR1 and obstacles overlap, and then expanded (converted) into the generated global coordinate system. At this time, surveillance camera 2 is also detected in the same way as the obstacle, and its position information is also set in the local coordinate system and then further set in the global coordinate system. In other words, the position information of surveillance camera 2 is automatically acquired. The synthesis of the local coordinate systems will be explained with reference to FIG.

[0047] [Local coordinate system: Figure 4] Next, the local coordinate system formed by each LiDAR 1 will be described with reference to Fig. 4. Fig. 4 is an image diagram of the local coordinate system. FIG. 4(a) shows the local coordinate system of the sensing result of the LiDAR 1a, and FIG. 4(b) shows the local coordinate system of the sensing result of the LiDAR 1b.

[0048] Comparing Figures 4(a) and (b), it can be seen that the position of the obstacle 6 and the position sensed by the other LiDAR 1 are misaligned (do not match). This is due to a detection error occurring in each LiDAR 1. Therefore, the integrated management server 4 corrects the local coordinate system and combines them.

[0049] [Error correction] Specifically, the average value of the distances from the LiDAR 1 to the obstacles 6 simultaneously detected in each local coordinate system (two local coordinate systems in FIG. 4) is calculated. The difference between this average value and the detected distance is used as the deviation (error) between the two coordinate systems, and when the local coordinate systems are combined to generate a global coordinate system, the deviation is corrected and combined.

[0050] The integrated management server 4 registers a global coordinate system including the LiDAR 1, the surveillance camera 2, and the obstacle 6 as background data. The integrated management server 4 periodically performs background subtraction processing on the background data and issues a warning when it detects a change. If the position coordinates of a fixed object such as an obstacle 6 change over time, it considers that the orientation of the LiDAR 1 has shifted due to vibration or aging, causing a change in the laser irradiation angle, and issues an alarm.

[0051] [Global coordinate system for suspicious person detection: Figure 5] Figure 5 shows an image of the global coordinate system when a suspicious person is detected. 5 is depicted on a plane, but in reality it is a three-dimensional coordinate system, so the suspicious person detection coordinates 7 have a Z coordinate (height information). The integrated management server 4 calculates the center coordinates of a rectangle that encompasses all of the point cloud data detected by the LiDAR 1, and controls and points the surveillance camera 2 toward the center coordinates.

[0052] One technique for pointing the surveillance camera 2 toward a suspicious person involves multiple sensors detecting the suspicious person, a control device pre-registering multiple preset positions, predicting the suspicious person's movements based on the position coordinates output from the multiple sensors, and calculating the direction of the suspicious person's face from the direction of movement of the suspicious person, and tracking the suspicious person by changing the viewing angle of the surveillance camera relative to the position of the suspicious person detected by the sensors (see Patent Publication No. 2009-273006, "Monitoring Device and Monitoring System"). Other technologies than those mentioned above are described in Patent No. 5161045, "Monitoring System and Monitoring Method."

[0053] [Detection image of suspicious person: Figure 6] 6 is an image diagram of a detection image when a suspicious person is detected. Here, a detection image 22 of the suspicious person is set as a rectangle within a detection image 21 detected by the monitoring camera 2. The range displayed as an image within a plane perpendicular to the ground to which the center coordinate belongs is the angle of view of the movable camera relative to the distance to the center coordinate.

[0054] The minimum point of this angle of view is the coordinate point at the bottom left of the angle of view, and the maximum point is the coordinate point at the top right of the angle of view. The integrated management server 4 calculates the size and position of the rectangle on the entire screen from the bottom left and top right coordinate points of the rectangle and the bottom left and top right coordinate points of the field of view, and notifies the operation terminal 5 of this information. The operation terminal 5 overlays and displays the rectangle on the video based on the position information of the rectangle.

[0055] [Overlay display: Figures 7-9] Next, the overlay display will be described with reference to Figures 7 to 9. Figure 7 is a processing image diagram for overlaying a detection frame on an image, Figure 8 is an image diagram of the XY plane of the global coordinate system, and Figure 9 is an image diagram of the XZ plane of the global coordinate system. To simplify the explanation, the position of surveillance camera 2 is assumed to be the origin of the global coordinate system, and the Z coordinate of the center of the detection frame is assumed to be "0" (zero). Surveillance camera 2 automatically orients itself according to the position (distance and direction) of the object to be detected (suspicious person), so the object to be detected (suspicious person) is captured at the center of the detection screen.

[0056] 7, a suspicious person and a detection frame 22 are shown in a detection image 21 detected by the surveillance camera 2. Also shown are a point 211 at the bottom left of the detection image 21, a point 212 at the top right of the detection image 21, a point 221 at the bottom left of the detection frame 22, a point 222 at the top right of the detection frame 22, a point 213 at the left end of the horizontal angle of view, a point 214 at the right end of the horizontal angle of view, a point 215 at the bottom end of the vertical angle of view, and a point 216 at the top end of the vertical angle of view. In addition, in the XY plane of the global coordinate system, in FIG. 8, the horizontal angle of view of the surveillance camera 2 (horizontal angle: θ H ) 23 is shown, and in FIG. 9, the vertical angle of view (horizontal angle: θ V )24 is shown.

[0057] If the X coordinate of the suspicious person is L, the coordinates (X, Y, Z) of each point 213 to 216 are as follows: 213 coordinates (L, -Ltan(θ H / 2),0) 214 coordinates (L, Ltan(θ H / 2),0) 215 coordinates (L, 0, -Ltan(θV / 2)) 216 coordinates (L, 0, Ltan(θ V / 2))

[0058] From the above, the coordinates of 211 and 212 are as follows. 211 coordinates (L, -Ltan(θ H / 2),-Ltan(θ V / 2)) 212 coordinates (L, Ltan(θ H / 2),Ltan(θ V / 2))

[0059] From the above, the coordinates corresponding to the points at each edge of the detected image can be determined. Next, the coordinates corresponding to points 221 and 222 in the detection frame 22 are calculated. The point cloud coordinates of the suspicious person detected by LiDAR1 are (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), (X n ,Y n ,Z n ) and the maximum absolute value of X, Y, and Z in the point cloud is X max , Y max , Z max Let's say.

[0060] If the detection frame 22 is a rectangle on the YZ plane (X is fixed) and is a rectangle 20% larger than the suspicious person, the coordinates (X, Y, Z) of 221 and 222 are as follows: Coordinates of 221 (L, -1.2Y max ,-1.2Z max ) Coordinates of 222 (L, 1.2Y max ,1.2Z max )

[0061] In the above explanation, a 20% margin is provided in the detection frame for the detection target, but the amount of margin is determined depending on the system based on the detection and capture capabilities of LiDAR1 and the movement speed of the detection target. From the above, the coordinate values of each point in the detection image 21 are obtained, and a detection frame 22 encompassing the detection target (suspicious person) can be overlaid and displayed in the image according to the coordinate values.

[0062] With this system, when installing the LiDAR 1 and the surveillance camera 2, there is no need to manually configure and adjust the settings required for detection, which shortens the construction period and reduces construction costs. In addition, by performing machine learning during operation, it is possible to improve detection accuracy over the operation period. Furthermore, by overlaying the object detected by the LiDAR 1 on the monitoring image on the display unit of the operation terminal 5, it becomes possible to immediately grasp the detected object when an alarm is issued.

[0063] [Effects of the embodiment] According to this system, the integrated management server 4 generates a global coordinate system from the latitude / longitude information of multiple LiDARs 1, synthesizes the local coordinate systems generated by each LiDAR 1 so that detected objects including the detected surveillance cameras 2 overlap, and then expands the synthesized local coordinate system into the generated global coordinate system, identifies the positions of the multiple surveillance cameras 2 based on the global coordinate system, and controls the orientation of the surveillance cameras 2, which has the effect of automating the registration of surveillance camera position information and also automating the control of the surveillance cameras in response to detected objects. [Industrial Applicability]

[0064] The present invention is suitable for surveillance systems that can automatically collect position information in the local coordinate system of LiDAR, etc. and that of surveillance cameras, and expand it into a global coordinate system for the entire monitored space, thereby automating the registration of position information for LiDAR, etc. and surveillance cameras during installation work and reducing and shortening the adjustment work required before the system can be put into operation. [Explanation of symbols]

[0065] 1 (1a to 1d)... LiDAR, 2 (2A, 2B)... Surveillance camera, 3... Wireless network, 4... Integrated management server, 5... Operation terminal, 6... Obstacle, 7... Detected image, 21... Detected image, 22... Detection frame, 23... Horizontal angle of view (horizontal angle: θ H), 24...Vertical angle of view (horizontal angle: θ V ), 211...bottom left point of the detected image, 212...top right point of the detected image, 213...left end point of the horizontal angle of view, 214...right end point of the horizontal angle of view, 215...bottom end point of the vertical angle of view, 216...top end point of the vertical angle of view, 221...bottom left point of the detection frame, 222...top right point of the detection frame

Claims

1. A surveillance system in which a plurality of remote sensing devices, a plurality of surveillance cameras, and an integrated management server that performs surveillance-related processing are connected via a wireless network, The remote sensing device has a built-in GPS to acquire location information of the device, detects the direction and distance of the detected object to be monitored and the surveillance camera, generates a local coordinate system indicating the positions of the detected object and the surveillance camera, with the position of the remote sensing device as the origin, from the direction and distance of the detected object and the surveillance camera, and transmits the location information, the direction and distance of the detected object and the surveillance camera, and the local coordinate system to the integrated management server, The integrated management server generates a global coordinate system by synthesizing multiple pieces of location information transmitted from the multiple remote sensing devices, each of the remote sensing devices acquires its own local coordinate system, synthesizes the local coordinate systems so that the positions of the detected objects and the surveillance cameras that are detected in the acquired local coordinate systems overlap, and expands the synthesized local coordinate system into the global coordinate system.

2. The monitoring system described in claim 1, characterized in that if the positions of the overlapping detected objects and the surveillance cameras are misaligned even when the positions are superimposed on each acquired local coordinate system, the integrated management server calculates an average value of the distances from the remote sensing devices detected by the remote sensing devices in each acquired local coordinate system, corrects the amount of positional misalignment using the difference between the average value and the detected distance, and expands the synthesized local coordinate system into a global coordinate system.

3. 3. The monitoring system according to claim 1, wherein the integrated management server periodically repeats a process of expanding the synthesized local coordinate system into the global coordinate system.

4. 4. The monitoring system according to claim 1, wherein the integrated management server controls the monitoring camera so that it points toward the position coordinates of the detected object shown in the global coordinate system.

5. A terminal device that displays images from a surveillance camera is connected to the wireless network, A monitoring system as described in any one of claims 1 to 4, characterized in that the integrated management server forms a rectangle in a global coordinate system that includes all point cloud data of detected objects detected by a remote sensing device, and when a monitoring camera is pointed at the center coordinates of the rectangle, causes the terminal device to overlay the rectangle corresponding to the detected object on the image.

6. The monitoring system described in claim 5, characterized in that when an object is detected by a remote sensing device, the integrated management server issues a warning to the terminal device, and when a correct or false report is input from the terminal device in response to the report, the integrated management server accumulates data from the remote sensing device in the case of the correct report, performs machine learning using the data, and uses the results of the machine learning to detect the object.

Images