Surveillance system

Abstract

To provide a monitoring system capable of appropriately coping with the deviation of the angle of view of a 3D sensor.SOLUTION: The monitoring system of the present example includes a 3D sensor 20 for detecting an object present in a monitoring area, a reference reflecting object 60 fixedly installed so as to fit in an angle of view of the 3D sensor 20, and a server 40 that calculates measured coordinates, which are current coordinates of the reference reflecting object 60 in the angle of view of the 3D sensor 20, based on sensor information obtained from the 3D sensor 20, and performs processing related to a shift in the angle of view of the 3D sensor 20 according to a variation amount of the measured coordinates with respect to initial coordinates set in advance for the reference reflecting object 60.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a surveillance system that uses a 3D sensor to detect objects within a surveillance area. [Background technology]

[0002] Conventionally, systems that monitor a monitoring area using video captured by cameras have been in practical use. Such monitoring systems often use visible light cameras to detect intruders, such as suspicious individuals, within the monitoring area, with subsequent image processing technology used to determine the detection. However, the accuracy of detection using image processing is significantly affected by factors such as differences in illuminance (brightness) between day and night, as well as by the installation environment of the visible light camera and the performance of the visible light camera itself. Furthermore, in important locations where more reliable capture of events is desired, fixed-point monitoring using fixed cameras rather than rotating cameras has often been used as the visible light monitoring camera.

[0003] In recent years, the use of 3D sensors has been increasing in the surveillance systems described above as an alternative to (or in combination with) visible light cameras. 3D sensors use lasers, ultrasound, or other sensors to obtain three-dimensional information about a subject. 3D sensors can obtain information such as the position, distance, and shape of a subject based on the return value of a laser beam irradiated onto the subject.

[0004] These 3D sensors are not limited to surveillance systems; they are also used in a variety of applications, including detecting dangerous areas at construction sites, controlling industrial robots, and controlling autonomous driving in automobiles. 3D sensors in surveillance systems are often used primarily for fixed-point monitoring, but in some cases they are mounted on moving objects and their usage varies depending on the system's operation. Naturally, the specifications of the 3D sensor itself are also selected to be optimal for the system requirements.

[0005] The following are examples of conventional technologies in the technical field related to the present invention: For example, Patent Document 1 discloses a monitoring system that can shorten the time required for re-registering presets by automatically correcting deviations in the camera's angle of view. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2023-38480 Summary of the Invention [Problem to be solved by the invention]

[0007] Systems that provide 24-hour surveillance require high detection accuracy, day or night. However, in systems that use visible light cameras, factors such as how the subject appears in the camera image have a significant impact on the image processing results. Therefore, by using or combining 3D sensors, which are less affected by factors such as ambient brightness, development is underway to achieve systems that can provide 24-hour surveillance, day or night, which is a weakness of visible light cameras.

[0008] In systems that use 3D sensors, the state of objects within a monitoring area is judged (detected) based on the return value obtained by irradiating the monitoring area with a laser that scans the area. For this reason, the return value obtained when the laser light hits the subject and returns has a significant effect on detection accuracy. In this case, if the return value of the laser irradiation against a fixed object in the background that serves as the reference appears different each time, it becomes impossible to determine the amount of change (difference) from the reference, which creates the problem of not being able to correctly judge the state of the object.

[0009] For this reason, when using 3D sensors for fixed-point monitoring, it is desirable that the return values ​​from the 3D sensors always remain the same under normal conditions (when there are no intruders, etc.). However, if the installation status of the 3D sensor changes due to external factors or over time, the return values ​​may differ from the intended values, raising concerns that detection accuracy may decrease. If the amount of change in the return values ​​could be grasped systematically, it would be possible to notify users of abnormalities or take preventative measures, allowing for subsequent rapid response (such as readjustment), but such a system has not yet existed.

[0010] The present invention has been made in consideration of the above-described conventional circumstances, and aims to provide a surveillance system that can appropriately deal with deviations in the angle of view of a 3D sensor. [Means for solving the problem]

[0011] In order to achieve the above object, a surveillance system according to one aspect of the present invention is configured as follows: That is, the surveillance system includes a 3D sensor for detecting an object present in a surveillance area, a reference reflector fixedly installed so as to fit within the angle of view of the 3D sensor, and a server that calculates measurement coordinates, which are the current coordinates of the reference reflector within the angle of view of the 3D sensor, based on sensor information obtained from the 3D sensor, and performs processing related to deviations in the angle of view of the 3D sensor in accordance with the amount of variation of the measurement coordinates relative to initial coordinates that are preset for the reference reflector.

[0012] Here, in the above monitoring system, the server may correct the angle of view of the 3D sensor in accordance with the amount of variation of the measurement coordinates relative to the initial coordinates.

[0013] In addition, in the above monitoring system, the server may be configured to output an alert to notify of a deviation in the angle of view of the 3D sensor when the amount of variation of the measurement coordinates relative to the initial coordinates is equal to or greater than a predetermined threshold.

[0014] In the above monitoring system, a plurality of the reference reflecting objects may be installed so that they fit within the angle of view of the 3D sensor. [Effects of the Invention]

[0015] According to the present invention, it is possible to provide a surveillance system that can appropriately deal with deviations in the angle of view of a 3D sensor. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a diagram showing a schematic configuration of a monitoring system according to an embodiment of the present invention; [Figure 2] 2 is a diagram showing an example of the arrangement of reference reflecting objects in the monitoring system of FIG. 1. FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of a server in the monitoring system of FIG. [Figure 4] FIG. 2 is a reference diagram relating to angle of view correction in the monitoring system of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0017] An embodiment of the present invention will be described with reference to the drawings. Here, a configuration will be described in which a detection function using a 3D sensor is added to a monitoring system that performs 24-hour monitoring using a general visible light camera. An example of such a monitoring system is a track monitoring system that detects intruders and flying objects in restricted areas such as railroad tracks to ensure the safety of train operations. For example, the track monitoring system monitors restricted areas using a 3D sensor, and when an intruder or flying object is detected, the system photographs the detected object using a visible light camera and displays it on a monitoring terminal.

[0018] Fig. 1 shows a schematic configuration of a monitoring system according to one embodiment of the present invention. The monitoring system in Fig. 1 includes a visible light camera 10, a 3D sensor 20, a network switch 30, a server 40, a monitoring terminal 50, and a reference reflecting object 60.

[0019] The server 40 and the monitoring terminal 50 are realized by a computer equipped with hardware resources such as a processor and memory, and are configured to implement functions and processes related to the present invention by reading a predetermined program from the memory and executing it with the processor. The server 40 and the monitoring terminal 50 may be realized by a single computer, or may be realized by multiple computers operating in cooperation with each other.

[0020] One or more visible light cameras 10 are installed so as to capture images of a monitored area, including restricted areas. The camera images captured by the visible light cameras 10 are transmitted to a server 40 and a monitoring terminal 50 via a network switch 30. Users such as monitors can check the status of the monitored area by viewing the camera images displayed on the monitoring terminal 50.

[0021] One or more 3D sensors 20 are installed near the visible camera 10 so as to detect the state of objects within the monitoring area. The 3D sensors 20 are used, for example, to detect intruders and flying objects. In this example, a LiDAR (Light Detection and Ranging) sensor is used as the 3D sensor 20, but this is just one example, and various types of 3D sensors can be used. The 3D sensor 20 generates sensor information by irradiating the monitoring area with laser light and acquiring the return value (e.g., reflection intensity). The sensor information is generated for each frame in which the monitoring area is scanned once. Examples of the sensor information include point cloud data representing three-dimensional coordinate information and brightness information of points where reflection intensity equal to or greater than a predetermined value is obtained. This sensor information is transmitted to the server 40 and the monitoring terminal 50 via the network switch 30.

[0022] The server 40 determines the state of objects in the monitoring area based on the sensor information received from the 3D sensor 20. For example, the server 40 determines the presence or absence of foreign objects in the monitoring area and acquires characteristic information of foreign objects (such as the position, distance, and shape of the foreign objects) by calculating the difference between the current sensor information received from the 3D sensor 20 and sensor information (background information) obtained when no foreign objects are present. Note that the background information data that can be obtained from the 3D sensor 20 is based on the use of a sensor that can always acquire stable data for each frame.

[0023] Furthermore, when the server 40 determines that a foreign object is present, it controls the visible camera 10 to capture an image of the foreign object based on the characteristic information of the foreign object, and displays the camera image on the monitoring terminal 50 for a user, such as a monitor, to check. At this time, the sensor information obtained by the 3D sensor 20 may also be displayed on the monitoring terminal 50. Note that it is difficult to determine the detailed state of a specific object simply by visualizing and displaying sensor information such as point cloud data. Therefore, in order to be able to determine the state of the object, for example, a detailed data determination process may be performed on the application side of the monitoring terminal 50, and the results may also be displayed.

[0024] This concludes the explanation of foreign object detection, the basic operation of this surveillance system. Conventionally, regardless of whether a 3D sensor is used, users were required to review all reported events. In such cases, the user's initial judgment is based on visible camera footage. However, when an event occurs far from the visible camera, it is often difficult to assess the situation using camera footage alone. In a surveillance system using a 3D sensor, if the cause cannot be identified from the visible camera footage, the situation must be confirmed using sensor information obtained by the 3D sensor. While it depends on the situation, identifying an event from sensor information such as point cloud data can be difficult, and it often takes a significant amount of time to reach a conclusion. If it can be determined initially that a false alarm is due to an incorrect installation angle of the 3D sensor, subsequent measures, such as physically adjusting the sensor angle, can be clarified, significantly reducing the burden of user confirmation work.

[0025] As explained in the background art, adopting a 3D sensor is an ideal solution for solving the problems of visible light cameras, but the reliability of the return values ​​is important when using a 3D sensor. To ensure the reliability of the return values, the surveillance system in this example is configured to detect deviations in the angle of view of the 3D sensor, as will be explained in detail below.

[0026] The surveillance system of this example stores a reference value for the return value of the laser light emitted from the 3D sensor 20 and implements a mechanism for constantly comparing the return value with this reference value. Comparing all data output from the 3D sensor 20 increases the processing load, potentially reducing overall performance (e.g., the actual intruder detection process). Therefore, a reference reflector 60 is fixedly installed so that it fits within the field of view of the 3D sensor 20. The coordinates of the reference reflector 60 detected by the 3D sensor 20 during operation are compared with the original coordinates of the reference reflector 60 to calculate the amount of variation, thereby detecting any deviation in the field of view of the 3D sensor 20. In this example, as shown in FIG. 2, the reference reflector 60 is fixedly installed within the laser irradiation range 71 of the 3D sensor 20.

[0027] The reference reflecting object 60 is, for example, a reflecting plate made of a material with high reflectivity, and is installed in a position that allows it to efficiently reflect the laser light arriving from the 3D sensor 20 toward the 3D sensor 20. The reference reflecting object 60 is not limited to this, and may be any object that can efficiently reflect the laser light from the 3D sensor 20.

[0028] 3 shows an example of the configuration of a processing unit related to the detection of a field of view deviation of the 3D sensor 20, among the processing units included in the server 40. The server 40 in FIG. 3 includes a sensor information acquisition unit 41, a reflector coordinate calculation unit 42, an initial coordinate storage unit 43, a field of view deviation detection unit 44, a field of view correction unit 45, and an alert output unit 46.

[0029] The sensor information acquisition unit 41 acquires sensor information from the 3D sensor 20 via the network switch 30. Since a reference reflector 60 is installed within the laser irradiation range 71 of the 3D sensor 20, the sensor information includes at least data related to the return value of the reference reflector 60 (for example, point cloud data representing three-dimensional coordinate information, brightness information, etc. of the reference reflector 60).

[0030] The reflector coordinate calculation unit 42 calculates the coordinates of the reference reflector 60 within the angle of view of the 3D sensor 20 based on the sensor information acquired by the sensor information acquisition unit 41. In this specification, the coordinates of the reference reflector 60 calculated before system operation are referred to as "initial coordinates," and the coordinates of the reference reflector 60 calculated during system operation are referred to as "measurement coordinates." The initial coordinates are reference values ​​to be compared with the measurement coordinates calculated during system operation, and are stored in advance in the initial coordinate storage unit 43.

[0031] During system operation, the angle-of-view deviation detection unit 44 detects a deviation in the angle of view of the 3D sensor 20 based on the measurement coordinates of the reference reflecting object 60 calculated by the reflector coordinate calculation unit 42 and the initial coordinates of the reference reflecting object 60 stored in the initial coordinate storage unit 43. In this example, the amount of variation of the measurement coordinates relative to the initial coordinates set in advance for the reference reflecting object 60 is calculated. This amount of variation represents the degree of deviation in the angle of view of the 3D sensor 20. The angle-of-view deviation detection unit 44 detects a deviation in the angle of view every N frames (here, N is an integer equal to or greater than 1). The processing result of the angle-of-view deviation detection unit 44 is provided to the angle-of-view correction unit 45 and the alert output unit 46.

[0032] The angle-of-view correction unit 45 corrects the angle of view of the 3D sensor 20 in accordance with the amount of variation calculated by the angle-of-view deviation detection unit 44 to deal with minor deviations in the angle of view due to vibrations or the like. Here, as shown in FIG. 4 , a sensor monitoring range 72 slightly smaller than the laser irradiation range 71 is set as the range targeted for object detection by the 3D sensor 20, and the reference reflecting object 60 is set within this sensor monitoring range 72. When the measurement coordinates of the reference reflecting object 60 within the sensor monitoring range 72 differ from the initial coordinates (i.e., when the amount of variation is greater than 0), the angle-of-view correction unit 45 performs a process of sliding the sensor monitoring range 72 so as to cancel out the variation of the measurement coordinates relative to the initial coordinates of the reference reflecting object 60. Specifically, the angle-of-view correction unit 45 performs a process of sliding the sensor monitoring range 72 in the direction opposite to the direction of variation by the amount of variation. This allows the sensor monitoring range 72 to function like a stabilizer when the deviation is equal to or less than a certain amount, thereby reducing the need to readjust the angle of view of the 3D sensor 20. In addition, any deviation in the angle of view caused by vibrations due to the movement of the train can be automatically corrected, preventing a decrease in detection accuracy.

[0033] The alert output unit 46 outputs alert information to notify a user, such as a monitor, of a deviation in the angle of view of the 3D sensor 20 when the amount of variation in the measurement coordinates relative to the initial coordinates is equal to or greater than a predetermined threshold. The threshold value, for example, is a value that can detect the occurrence of a deviation in the angle of view (or a deviation close to that) that cannot be addressed by the angle of view correction by the angle of view correction unit 45. Examples of alert information include an instruction to display a text message or image message, an instruction to output a voice message, and an instruction to turn on a warning lamp. Such alert information is transmitted to an output device (the monitoring terminal 50 in this example) via the network switch 30, and the output device performs output processing according to the alert information. This allows the user to quickly recognize the need to readjust the angle of view of the 3D sensor 20.

[0034] As described above, the monitoring system of this example includes the 3D sensor 20 for detecting objects present in the monitoring area, the reference reflecting object 60 fixedly installed so as to fit within the angle of view of the 3D sensor 20, and the server 40 that calculates measurement coordinates, which are the current coordinates of the reference reflecting object 60 within the angle of view of the 3D sensor 20, based on sensor information obtained from the 3D sensor 20, and performs processing related to deviations in the angle of view of the 3D sensor 20 according to the amount of variation of the measurement coordinates from initial coordinates that are set in advance for the reference reflecting object 60. As a more specific example, the server 40 has a function to correct the angle of view of the 3D sensor 20 according to the amount of variation of the measurement coordinates from the initial coordinates, and a function to output an alert when the amount of variation of the measurement coordinates from the initial coordinates is equal to or greater than a predetermined threshold.

[0035] This configuration makes it possible to appropriately deal with misalignment of the 3D sensor's angle of view. In other words, minor misalignment of the angle of view can be automatically corrected, and an alert is issued when misalignment of the angle of view that cannot be addressed by angle of view correction occurs, thereby reducing false detections caused by misalignment of the 3D sensor's angle of view due to external factors, etc. Furthermore, because misalignment of the 3D sensor's angle of view can be determined by the system rather than by the user, this improves system reliability while significantly reducing the burden on the user.

[0036] In the above description, one reference reflecting object 60 is placed within the angle of view of the 3D sensor 20, but multiple reference reflecting objects 60 may be placed within the angle of view of the 3D sensor 20. This provides multiple reference points within the angle of view of the 3D sensor 20, making it possible to determine in detail any deviation in the angle of view of the 3D sensor 20. Furthermore, setting multiple reference points is also effective in detecting deviation in the angle of view due to roll rotation (rotation around the front axis) of the 3D sensor 20, which is difficult to detect using only one reference point.

[0037] The sensor information of the 3D sensor 20 may also be used for other purposes. For example, the luminance information of the reference reflector 60 included in the sensor information may be used as information for determining the lifespan of the laser diode of the 3D sensor 20. That is, the luminance information of the reference reflector 60 measured before the system is put into operation may be stored in advance, and the luminance information may be compared with the luminance information measured during system operation to determine the deterioration of the 3D sensor 20. Furthermore, the degree of luminance decrease over time may be determined, and the time remaining until the end of the lifespan of the 3D sensor 20 may be calculated.

[0038] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications thereof are included in the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0039] Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for realizing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable manner. [Industrial Applicability]

[0040] The present invention can be used in a surveillance system that uses a 3D sensor to detect objects within a surveillance area. [Explanation of symbols]

[0041] 10: Visible camera, 20: 3D sensor, 30: Network switch, 40: Server, 41: Sensor information acquisition unit, 42: Reflector coordinate calculation unit, 43: Initial coordinate storage unit, 44: View angle deviation detection unit, 45: View angle correction unit, 46: Alert output unit, 50: Monitoring terminal, 60: Reference reflector

Claims

1. a 3D sensor for detecting objects present in the monitoring area; a reference reflecting object fixedly installed so as to fit within the angle of view of the 3D sensor; a server that calculates measurement coordinates, which are the current coordinates of the reference reflector in the angle of view of the 3D sensor, based on sensor information obtained from the 3D sensor, and performs processing related to the angle of view deviation of the 3D sensor in accordance with the amount of variation of the measurement coordinates relative to initial coordinates previously set for the reference reflector; A monitoring system comprising:

2. 2. The monitoring system according to claim 1, The server corrects the angle of view of the 3D sensor according to the amount of variation of the measurement coordinates relative to the initial coordinates.

3. 2. The monitoring system according to claim 1, The monitoring system is characterized in that the server outputs an alert to notify of a deviation in the angle of view of the 3D sensor when the amount of variation of the measurement coordinates relative to the initial coordinates is greater than or equal to a predetermined threshold.

4. 2. The monitoring system according to claim 1, A surveillance system characterized in that a plurality of the reference reflecting objects are installed so as to fit within the angle of view of the 3D sensor.

Images