Monitoring system

Abstract

This invention provides a monitoring system 1 that efficiently detects the intrusion of a person into a monitored area. [Solution] The surveillance system 1 comprises a camera unit 10 that outputs images of the surveillance area, a detection unit 20 that detects the intrusion of a person into the surveillance area by inputting the images into a trained model that has learned from a plurality of training images including a plurality of correct images and a plurality of incorrect images, and a warning unit 30 that generates an alarm based on the detection result of the detection unit 20. The camera unit 10 outputs visible light images and infrared images as the images, and the plurality of training images include a plurality of visible light images and a plurality of infrared images.

Description

Technical Field

[0001] The present invention relates to a monitoring system for detecting the intrusion of a person into a monitoring area.

Background Art

[0002] Based on the demand for reducing carbon dioxide emissions, the utilization of solar power generation as a renewable energy has been progressing. In particular, by installing large-area batteries on vast unused land, the construction of so-called megasolar power plants with large-scale power generation capacity has been successively carried out. In order to secure vast unused land with good sunlight, megasolar power plants are often constructed in mountainous areas away from populated areas.

[0003] However, along with the soaring prices of metal resources such as copper, thefts of copper wires from megasolar power plants have occurred frequently. Monitoring megasolar power plants located far from populated areas is not easy.

[0004] Japanese Patent Laid-Open No. 2010-211710 discloses a security system including a camera installed to monitor an access road to a water storage area where solar panels are installed, and a transmission means for transmitting image data captured by the camera to a security guard.

[0005] In addition, there is known a monitoring system that detects the intrusion of a person into a monitoring area by AI by inputting an image of the monitoring area into a learned model.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] In the aforementioned security system, security personnel who receive image data captured by cameras must review the images each time to determine whether action is necessary. Furthermore, false positives are unavoidable in AI-based surveillance systems. It takes time and money for security personnel to arrive at remote mega solar power plants, assess the situation on-site, and take appropriate action.

[0008] Embodiments of the present invention aim to provide a surveillance system that efficiently detects the intrusion of a person into a surveillance area. [Means for solving the problem]

[0009] The surveillance system of the embodiment comprises a camera unit that outputs images of a surveillance area, a detection unit that detects the intrusion of a person into the surveillance area by inputting the images into a trained model that has learned multiple training images including multiple correct images and multiple incorrect images, and a warning unit that generates an alarm based on the detection result of the detection unit, wherein the camera unit outputs visible light images and infrared images, and the multiple training images include multiple visible light images and multiple infrared images. Furthermore, the detection unit uses a trained model for visible light and a trained model for infrared light. , In model training, the multiple correct images of the multiple training images do not include images in which the entire figure of a person is obscured by a predetermined ratio or more, and the second ratio, which is the predetermined ratio in infrared light model training, is set to be smaller than the first ratio, which is the predetermined ratio in visible light model training. [Effects of the Invention]

[0010] According to embodiments of the present invention, a surveillance system can be provided that efficiently detects the intrusion of a person into a surveillance area. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows the configuration of the monitoring system according to the embodiment. [Figure 2] Figure 2 shows an example image of the monitoring area of ​​the monitoring system according to the embodiment. [Figure 3] Figure 3 shows a predetermined monitoring area of ​​the monitoring system of the embodiment. [Figure 4] Figure 4 is a diagram illustrating the ground truth image of the monitoring system according to the embodiment. [Figure 5]Figure 5 is a diagram illustrating the correct image of the monitoring system according to the embodiment. [Figure 6] Figure 6 is a diagram illustrating the ground truth image of the monitoring system according to the embodiment. [Figure 7] Figure 7 is a diagram illustrating incorrect images of the monitoring system according to the embodiment. [Figure 8] Figure 8 shows an example of a slow chart of the monitoring system according to the embodiment. [Best Mode for Carrying Out the Invention]

[0012] Embodiments of the present invention will be described below with reference to the drawings. The drawings based on the embodiments are schematic. The illustration and reference numerals of some components have been omitted. <First Embodiment>

[0013] As shown in Figure 1, the monitoring system 1 of this embodiment mainly comprises a camera unit 10, a detection unit 20, a warning unit 30, and a notification unit 40. The monitoring area of ​​the monitoring system 1 is, for example, the site of a solar power plant as shown in Figure 2.

[0014] Some of the components of the monitoring system 1 may consist of internal circuits of semiconductor elements processed by software, or dedicated hardware circuits. For example, the detection unit 20 may include an FPGA (Field Programmable Gate Array) which contains hardware circuits, and a GPU (Graphics Processing Unit) which contains software circuits.

[0015] The camera unit 10 includes a visible light camera 11 and an infrared light camera 12, and outputs images of the monitored area. The visible light camera 11, which receives visible light with wavelengths from 400nm to 800nm, outputs a color image. The infrared light camera 12, which receives infrared light with wavelengths from 800nm ​​to 1000nm, outputs a black and white image from luminance data.

[0016] The camera unit 10 outputs a visible light image during the day and an infrared image at night. In addition, when the camera unit 10 detects that the visibility has deteriorated due to, for example, fog or the like based on a decrease in the contrast of the visible light image, it preferably outputs an infrared image instead of the visible light image even during the day. Further, the camera unit 10 may output a composite image obtained by synthesizing the color data of the visible light image and the luminance data of the infrared image.

[0017] The detection unit 20 detects the intrusion of a person into the monitoring area by inputting the image output by the camera unit 10 into a machine-learned neural network model (hereinafter, a learned model). When the camera unit 10 outputs a video, the detection unit 20 does not need to process all frames and may process still images at a predetermined time interval, for example, at an interval of one minute.

[0018] The monitoring system 1 may have a plurality of camera units 10 that capture different fields of view. The detection unit 20 processes a plurality of images output by each of the plurality of camera units 10.

[0019] The learned model has learned a plurality of teacher images including a plurality of correct images and a plurality of incorrect images. For example, a correct image is an image annotated with an annotation indicating "intruder present", "person", or other correct answers. An incorrect image is an image without such an annotation. In addition, an annotation indicating "not a person" or other incorrect answers may be added to the incorrect image. In this way, in addition to learning an image in which no person such as an intruder is shown as an incorrect image, an image that is likely to be misjudged as having a person shown or an image that shows something confusing with a person and does not correspond to an intrusion can also be learned as an incorrect image by methods such as not adding an annotation, so that it can be learned as "not a person". At this time, data augmentation (data expansion) images may be used as teacher data.

[0020] Multiple training images include multiple visible light images and multiple infrared images. The detection unit 20 has a trained model for visible light and a trained model for infrared light, and it is preferable to use the trained model for visible light during the day and the trained model for infrared light at night.

[0021] The warning unit 30 generates an alarm based on the detection result of the detection unit 20. The warning unit 30 emits alarm illumination light from the illumination unit 31 or generates an alarm sound from the speaker 32. The alarm illumination light is preferably different from normal illumination light and is a light that intimidates intruders, such as a flashing light or a blinking light.

[0022] The alarm sound can be a general siren sound, but it is especially preferable that it be an unpleasant sound that humans find offensive. Examples of unpleasant sounds include sounds in the 2000Hz to 4000Hz frequency range, such as the sound produced when fingernails are scratched on a blackboard, and mosquito sounds (17000Hz), such as the buzzing of a mosquito. By using unpleasant sounds rather than just intimidation sounds, it is possible to encourage the departure of intruders. It should be noted that the ability to generate unpleasant sounds from speaker 32 is a characteristic of surveillance systems for remote sites.

[0023] The notification unit 40 notifies the remote monitoring center 100 of the intrusion detection via the communication line 50. The monitoring center 100 then notifies the relevant parties, such as the security company and the manager of the solar power plant.

[0024] The surveillance system 1 preferably includes a drone 33 that is operated remotely via a communication line. The drone 33, operated from the surveillance center 100, can acquire detailed images of intruders and images outside the field of view of the camera unit 10, such as images of the intruder's car's license plate.

[0025] In AI-based intrusion detection by the detection unit 20, false positives are inevitable. False positives can be classified into two types: Type 1 errors, where an intrusion occurs but is not detected, and Type 2 errors, where an intrusion is detected when it does not exist.

[0026] As already explained, it takes time and money for security guards to arrive at remote mega solar power plants, assess the situation on-site, and take appropriate action. In addition, intruders often work on the premises for relatively long periods. For this reason, the detection unit 20 is designed to tolerate the occurrence of Type 1 errors in order to reduce the occurrence of Type 2 errors. As a result, the detection unit 20 has a higher error rate for Type 1 errors than for Type 2 errors.

[0027] In the image of the monitoring area shown in Figure 3, the detection unit 20 does not monitor areas other than the hatched predetermined region. Therefore, the detection unit 20 does not mistakenly detect ghosts caused by sunlight reflecting off solar panels or white clouds in the sky as intruders.

[0028] In monitoring images of solar power plants, human figures may be partially obscured by solar panels. To reduce the occurrence of Type II errors, it is preferable that the ground truth images used as training images for model learning do not include images where the entire figure of a person is largely obscured, as shown in Figure 4. For example, images in which the entire figure of a person is obscured by a predetermined ratio or more, e.g., 80% or more, should not be included as ground truth images.

[0029] Because infrared images are black and white, they are more prone to false detections than visible light images. Therefore, it is preferable that infrared light trained models are trained to not assume a person unless the situation is more certain than that of visible light trained models.

[0030] For this reason, the second ratio, which is a predetermined ratio in infrared light model training, is set smaller than the first ratio, which is a predetermined ratio in visible light model training. For example, the first ratio is 80%, and the second ratio is 50%. In other words, in the visible light model, images in which more than 80% of the overall image is hidden are not included in the ground truth images, whereas in the infrared light model, images in which more than 50% of the overall image is hidden are not included in the ground truth images.

[0031] For example, 60% of the figure shown in Figure 5 is obscured. Therefore, the image of the figure in Figure 5 is included in the ground truth images for visible light model training, but not in the ground truth images for infrared light model training.

[0032] Furthermore, multiple incorrect images include images showing rain or snow adhering to the lens of the camera unit 10, as shown in Figure 6, and images where the trajectory of rain is displayed as a line. In addition, images of spiders in spiderwebs in front of the camera unit 10 are also prone to false detection and should therefore preferably be included in the incorrect images.

[0033] On the other hand, since the purpose of an intruder into a solar power plant is to obtain copper wires, the intruder is often in a crouching position. For this reason, it is preferable that the multiple correct images of multiple training images include multiple images of a person in a crouching position, as shown in Figure 7.

[0034] As explained above, the occurrence of errors in a single still image by the detection unit 20 is tolerated up to a predetermined level. Therefore, the monitoring system 1 may reduce the probability of a first-class error occurring by having the detection unit 20 process multiple still images. The processing by the detection unit 20 may use multiple images captured in succession, or multiple consecutive frames of a captured video.

[0035] In other words, in order to reduce the occurrence of Type 2 errors, it is preferable that the warning unit 30 generates an alarm only when the detection unit 20 detects intrusion a predetermined number of times or more at predetermined time intervals. The predetermined time interval and predetermined number of times can be set appropriately, taking into consideration the balance between the occurrence of Type 1 errors and Type 2 errors. The predetermined number of times may be multiple. Furthermore, it is preferable that the predetermined number of times be set more at night, when false detections are more likely to occur, than during the day. In addition, in a surveillance system 1 having multiple camera units 10, it is preferable that the warning unit 30 generates an alarm only when intrusion is detected from the images of each of two or more camera units 10.

[0036] For example, workers may enter the solar power plant premises for maintenance and inspection. To prevent false alarms, monitoring system 1 needs to suspend its detection operation. To do this, it is necessary to contact the administrator of monitoring system 1 in advance and register the inspection schedule.

[0037] Preferably, the detection unit 20 interrupts the detection process when it detects a person in a predetermined first posture, and resumes the detection process when it detects a person in a predetermined second posture. The first and second postures are selected from special postures that humans do not normally assume during work. Examples of special postures include making a circle with both hands above the head or crossing both hands in front of the face.

[0038] As shown in Figure 8, the detection process starts (step S10), and when the detection unit 20 detects an intrusion (step S20: YES), the warning unit 30 generates an alarm (step S30). The detection process ends when an end instruction is given (step S40: YES).

[0039] If the detection unit 20 does not detect a person in the first posture (step S50: NO), it continues the detection process. On the other hand, if the detection unit 20 detects a person in the first posture (step S50: YES), it interrupts the detection process and enters a standby state (step S60). If the detection unit 20 detects a person in the second posture (step S70: YES), it resumes the detection process.

[0040] By communicating predetermined first and second postures to the worker, the workload of the administrator of monitoring system 1 can be reduced. The first and second postures can be easily changed by training the AI ​​model of the detection unit 20 using images. The first and second postures may also be videos of predetermined movements.

[0041] The present invention is not limited to the embodiments described above, and various changes, combinations, and modifications can be made without altering the essence of the invention. [Explanation of Symbols]

[0042] 1. Surveillance system 10. Camera Unit 11. Visible light camera 12. Infrared light camera 20...Detection unit 30...Warning section 31. Lighting Department 32 speakers 33. Drone 40...Notification section 50...Communication lines 100... Monitoring Center

Claims

1. A camera unit that outputs images of the monitoring area, A detection unit detects a person entering the monitoring area by inputting the image into a trained model that has learned from multiple training images, including multiple correct images and multiple incorrect images. The system comprises a warning unit that generates an alarm based on the detection result of the aforementioned detection unit, The camera unit outputs a visible light image and an infrared image as the image. The aforementioned plurality of training images include a plurality of visible light images and a plurality of infrared images, The detection unit uses a trained model for visible light and a trained model for infrared light, In model learning, the multiple ground truth images of the multiple training images do not include images in which the entire figure of a person is obscured by a predetermined ratio or more. A monitoring system characterized in that the second ratio, which is a predetermined ratio in infrared light model learning, is set to be smaller than the first ratio, which is a predetermined ratio in visible light model learning.

2. The monitoring system according to claim 1, characterized in that the monitoring area is the site of a solar power plant.

3. The monitoring system according to claim 1, further comprising a notification unit that notifies a remote location of intrusion detection via a communication line.

4. The surveillance system according to claim 1, further comprising a drone that is operated remotely via a communication line.

5. The monitoring system according to claim 1, characterized in that the warning unit generates at least one of light and sound.

6. The monitoring system according to claim 5, characterized in that the warning unit generates an unpleasant sound.

7. The monitoring system according to claim 1, characterized in that the detection unit does not monitor areas other than a predetermined region of the image in the monitoring area.

8. The monitoring system according to claim 1, characterized in that the first ratio is 80% and the second ratio is 50%.

9. The monitoring system according to claim 1, characterized in that the plurality of correct images of the plurality of training images include images of a person in a crouching position.

10. The surveillance system according to claim 1, characterized in that the plurality of incorrect images include an image of rain or snow adhering to the camera unit.

11. The monitoring system according to claim 1, characterized in that the detection unit has a first type error rate in which it fails to detect an intrusion, which is higher than the second type error rate in which it falsely detects an intrusion.

12. The monitoring system according to claim 1, characterized in that the warning unit generates an alarm when the detection unit detects an intrusion a predetermined number of times or more at predetermined time intervals.

13. The monitoring system according to claim 1, characterized in that the detection unit interrupts the detection process when it detects a person in a predetermined first posture, and resumes the detection process when it detects a person in a predetermined second posture.

Images