Monitoring device

Abstract

To provide a monitoring device that can quickly and accurately detect the entry of objects such as people into a monitoring area. [Solution] The monitoring device 10 includes a LiDAR sensor 20 that measures the position of object B1 and the distance to object B1, wide-angle cameras 21 and 22, narrow-angle cameras 31 and 32 that have a field of view narrower than that of the wide-angle cameras 21 and 22 and are configured to change the position of the field of view, and a detection unit 27 that detects the entry of object B1 into a preset danger area 102. The detection unit 27 determines whether or not object B1 has entered the danger area 102 based on point cloud data from the LiDAR sensor 20, and determines whether or not object B1 is a human based on image data from at least one of the wide-angle camera 21(22) and the narrow-angle camera 31(32).

Description

Technical Field

[0001] The present invention relates to a monitoring device for monitoring the entry of an object into a preset area.

Background Art

[0002] There is known a technique for generating useful data for improving the efficiency of a worker's work process at a work site by using the measurement results of a LiDAR (Light Detection And Ranging) sensor and the imaging results of a camera (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when a worker works on the rooftop of a factory, if the worker enters a place where the intensity such as taking lights is not high (prohibited entry area), the worker may fall due to damage to the place where the intensity is not high. As a safety measure to prevent such a fall, a measure may be taken in which the worker wears a safety belt on a safety rope for fall prevention. In addition, as a measure to prevent entry into the prohibited entry area, a measure may be taken in which a monitor monitors the worker with a net for preventing entry into the prohibited entry area provided.

[0005] However, in the above safety measures, it is preferable that the safety measures are mechanically performed without relying on the attention of the worker himself or the monitor who is a human. That is, it is preferable that there is a device that can more reliably monitor that the worker accidentally enters the prohibited entry area. It cannot be said that Patent Document 1 discloses a configuration for detecting the entry of a worker into a prohibited entry area in such high-place work or the like.

[0006] In view of the above circumstances, the present invention aims to provide a monitoring device that can quickly and accurately detect the entry of objects such as people into a monitoring area. [Means for solving the problem]

[0007] The present invention is essentially a monitoring device described below.

[0008] (1) A distance measuring sensor that measures the position of an object and the distance to the object, Wide-angle camera, A narrow-angle camera is configured to have a field of view narrower than that of the wide-angle camera and to be able to change the position of the field of view. It includes a detection unit that detects the entry of an object into a pre-set monitoring area, The detection unit determines whether the object has entered the monitoring area based on data from the distance measuring sensor, and determines whether the object is a human being based on data from at least one of the wide-angle camera and the narrow-angle camera, in this monitoring device.

[0009] (2) The monitoring area is configured to be set in multiple locations, The wide-angle camera is configured to capture at least two of the aforementioned monitoring areas, The monitoring device according to (1), wherein the narrow-angle camera is configured to move its field of view to the monitoring area in which the distance measuring sensor has determined that an object has entered, among a plurality of monitoring areas.

[0010] (3) The monitoring device according to (1) or (2), wherein the detection unit determines whether the object is a human being based on data from the narrow-angle camera when it is not possible to determine whether the object is a human being based on data from the wide-angle camera.

[0011] (4) The monitoring device according to any one of items (1) to (3) above, wherein the narrow-angle camera is equipped with a zoom mechanism that can change the shooting magnification. [Effects of the Invention]

[0012] According to the present invention, the entry of objects such as people into a monitoring area can be detected quickly and accurately. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is a schematic diagram showing a monitoring device according to one embodiment of the present invention installed at a work site. [Figure 2] Figure 2 is a side view of the monitoring device. [Figure 3] Figure 3 is a block diagram showing the configuration of a monitoring system including monitoring devices. [Figure 4] Figure 4 shows an example of the layout on a mobile device's display screen, and includes an example of a LiDAR screen displayed on the mobile device. [Figure 5] Figure 5 shows an example of the first operation section displayed on the screen of a mobile device. [Figure 6] Figure 6 shows an example of the second operation section displayed on the screen of a mobile device. [Figure 7] Figure 7 is a flowchart illustrating an example of the operation of a monitoring system including a monitoring device, showing the case where both camera modes are set. [Figure 8] Figure 8 is a flowchart illustrating an example of the operation of a monitoring system including a monitoring device, showing the case where both camera modes are set. [Figure 9] Figure 9 is a flowchart illustrating another example of the operation of a monitoring system including a monitoring device, showing the case where a single-camera mode (wide-angle camera mode or narrow-angle camera mode) is set. [Figure 10] Figure 10 is a flowchart illustrating another example of the operation of a monitoring system including a monitoring device, showing the case where a single-camera mode (wide-angle camera mode or narrow-angle camera mode) is set. [Modes for carrying out the invention]

[0014] Hereinafter, embodiments for implementing the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a schematic diagram showing a state where a monitoring device 10 according to an embodiment of the present invention is installed at a work site 100. FIG. 2 is a side view of the monitoring device 10. FIG. 3 is a block diagram showing the configuration of a monitoring system 1 including the monitoring device 10. As shown in FIGS. 1 to 3, the work site 100 is a high place such as the roof of a factory, and in a part of the work site 100, there is a caution-required area 101 where the intensity, such as light picking, is relatively weak. If an operator P1 gets on the caution-required area 101, the caution-required area 101 may crack and the operator P1 may fall. Therefore, it is required that the operator P1 does not enter such a caution-required area 101. The monitoring device 10 sets a predetermined area including such a caution-required area 101 as a danger area 102, and issues a warning when an operator P1 (human) enters this danger area 102. Note that the operator P1 does not refer to a specific person, but indicates a person present at the work site 100.

[0016] The monitoring system 1 includes a monitoring device 10 and a mobile terminal 50.

[0017] In the present embodiment, the mobile terminal 50 is used to operate the monitoring device 10 and to receive the monitoring results of the monitoring device 10. The mobile terminal 50 is carried by the operator P1 or a supervisor P2 who supervises the operator P1. Note that the mobile terminal 50 may be held by the operator P1 and the supervisor P2 respectively. Also, instead of the mobile terminal 50, a fixed terminal not premised on portability may be used, and the mobile terminal 50 is merely an example of a terminal.

[0018] The mobile terminal 50 is operated by the person who possesses it. The mobile terminal 50 may be connected to the monitoring device 10 via a network 2 such as the Internet or a LAN (Local Area Network), or it may be composed of the same terminal as the monitoring device 10. Examples of the mobile terminal 50 include a PC (personal computer), smartphone, tablet, feature phone, smart glasses, and smart goggles.

[0019] The monitoring device 10 is installed, for example, on the upper surface 100a of the work site 100. Preferably, the monitoring device 10 is lightweight and compact, weighing, for example, about 10 kg to 20 kg and having a height of about 1 m to 1.5 m, so that a worker P1 can carry it by themselves.

[0020] The monitoring device 10 comprises a base 11, a support column 12 attached to the base 11, a sensor unit 13, a control unit 14, and a storage unit 15 supported on the support column 12, and a warning device 16 and an antenna 17 attached to the support column 12.

[0021] The base 11 is, for example, a tripod and is installed on the upper surface 100a of the work site 100. The upper surface 100a may be horizontal or inclined with respect to the horizontal plane. If the upper surface 100a is inclined, a weight is placed on the base 11 to prevent it from tipping over. The base 11 may have a configuration other than a tripod, as long as it can support the control unit 14, sensor unit 13, warning device 16, and antenna 17. In this embodiment, the base 11 also supports the power supply 3, but the power supply 3 may be installed on a separate component from the base 11, or a power supply (power plant) supplied by a power company may be used. If the power supply 3 is a power plant, an outlet extends from the base 11 to the socket. A support column 12 extends upward from the top of the base 11.

[0022] An antenna 17 is installed, for example, near the base of the support column 12. The antenna 17 is provided to connect the monitoring device 10 to the network 2. Also, a power supply 3, a control unit 14, and a storage unit 15 are installed, for example, near the base of the antenna 17. The power supply 3 is the power source for the monitoring device 10 and includes, for example, a secondary battery, which powers the monitoring device 10. The specific configuration of the control unit 14 will be described later. A warning device 16 is installed above the control unit 14.

[0023] The warning device 16 is provided to notify when an object B1, such as luggage S1 or worker P1, enters the hazardous area 102. In this embodiment, the warning device 16 is both a warning light and an alarm. The warning device 16 is installed to emit light and sound outwards from the support column 12. The warning device 16 is connected to the control unit 14, and when it detects that an object B1 has entered the hazardous area 102, it emits a first warning, for example, an orange light and a first warning sound. Furthermore, when it detects that a worker P1 (human) has entered the hazardous area 102, the warning device 16 emits a second warning, for example, a red light and a second warning sound.

[0024] The first and second warning sounds are, for example, buzzer sounds. The first and second warning sounds only need to be distinguishable from each other, and the specific types of sounds are not limited. The warning device 16 is not limited to emitting both light and sound when issuing a warning; it may also be configured to emit only one of them. If the warning devices 16 are arranged at multiple locations (for example, two locations) along the circumference of the support column 12, the warning from the warning devices 16 can be more clearly communicated to the surrounding area. A sensor unit 13 is installed, for example, above the warning device 16.

[0025] The sensor unit 13 measures and photographs the area around the monitoring device 10, including the hazardous area 102. The sensor unit 13 includes a LiDAR sensor 20 as a distance measuring sensor, a first wide-angle camera 21, a first narrow-angle camera 31, a second wide-angle camera 22, and a second narrow-angle camera 32. These cameras 21, 31, 22, and 32 are imaging cameras such as CCD cameras and CMOS cameras, and generate image data that identifies the shooting results.

[0026] LiDAR stands for "Light Detection And Ranging," and the LiDAR sensor 20 is a sensor that performs detection and distance measurement using light. The LiDAR sensor 20 emits laser light, measures the time it takes for the light to hit an object and bounce back, and measures the distance and direction to the object. Unlike radar, which uses radio waves for measurement, LiDAR uses laser light for measurement.

[0027] The LiDAR sensor 20 is an example of a distance measuring sensor that measures the position of object B1 and the distance to object B1. In this embodiment, object B1 refers collectively to the luggage S1 placed on the upper surface 100a of the work site 100 and the worker P1. However, object B1 is not limited to the above, and can be anything that can be detected by the LiDAR sensor 20. The LiDAR sensor 20 includes a light-emitting unit and a light-receiving unit. The light-emitting unit transmits laser light as a beam, and the light-receiving unit receives the reflected light generated by transmitting this beam. The LiDAR sensor 20 outputs the detection result as, for example, point cloud data, but the specific output format of the detection result is not limited. In this embodiment, the LiDAR sensor 20 measures a 360° radius around the support column 12. Alternatively, n (where n is an integer of 2 or more) LiDAR sensors may be provided, and each LiDAR sensor may measure (360 / n)° around the support column 12 to measure a 360° radius around the support column 12. The LiDAR sensor 20 has a predetermined step angle (for example, about 0.1°), and emits a beam while varying the angle at which it emits laser light for each step angle. This allows the LiDAR sensor 20 to detect the direction relative to the LiDAR sensor 20 and the distance to the signal reflection point. The LiDAR sensor 20 detects the distance to the signal reflection point for each of multiple directions by repeating the scanning operation. The measurement results of the LiDAR sensor 20 are output to the control unit 14.

[0028] The LiDAR sensor 20 may also be a radar sensor or the like that emits radio waves such as microwaves or millimeter waves. A first wide-angle camera 21 is installed below the LiDAR sensor 20.

[0029] The first wide-angle camera 21 is, for example, a camera equipped with a fisheye lens, and in this embodiment, it captures a 180° range around the support column 12. The first wide-angle camera 21 captures at least a range in the longitudinal direction (vertical direction) of the support column 12 in which a hazard area 102 can be set. The shooting magnification of the first wide-angle camera 21 is fixed, but the shooting magnification may be variable. The first wide-angle camera 21 is configured to capture at least two hazard areas 102. In Figure 1, as an example, four hazard areas 102 are set. The first narrow-angle camera 31 is installed, for example, below the first wide-angle camera 21.

[0030] The first narrow-angle camera 31 is a movable camera configured to have a field of view narrower than that of the first wide-angle camera 21 and to allow the position of the field of view of the first narrow-angle camera 31 to be changed. The first narrow-angle camera 31 is, for example, a PTZ camera and is capable of panning (moving the field of view around the vertical axis), tilting (moving the field of view in the longitudinal direction (up and down direction) of the support column 12, and zooming in and out. The first narrow-angle camera 31 is capable of capturing at least the range that the first wide-angle camera 21 is capturing.

[0031] The first narrow-angle camera 31 has a zoom function that allows the shooting magnification to be changed from approximately 1x to 4x, but the specific shooting magnification is not limited. For the first narrow-angle camera 31, the angle of view (field of view) around the support column 12 is, for example, 30° to 120° depending on the shooting magnification, but the specific angle of view is not limited. For the first narrow-angle camera 31, the pan range (shooting location movement range) around the support column 12 is at least the angle of view of the first wide-angle camera 21 around the support column 12 (180°), and in this embodiment, it is approximately 360°. For the first narrow-angle camera 31, the tilt range (shooting location movement range) in the longitudinal direction (up and down direction) of the support column 12 is, for example, 90°.

[0032] The first narrow-angle camera 31 only needs to have a narrower field of view than the first wide-angle camera 21 around the support column 12 and be able to change its position within the field of view of the first wide-angle camera 21; its specific configuration is not limited. It is preferable that the first narrow-angle camera 31 and the first wide-angle camera 21 are installed in the same position around the support column 12 in order to align the fields of view of these cameras 21 and 31.

[0033] Image data, which is the result of the first wide-angle camera 21 and the first narrow-angle camera 31, is output to the control unit 14. The first wide-angle camera 21 and the first narrow-angle camera 31 constitute the first camera unit 41. The second wide-angle camera 22 and the second narrow-angle camera 32 constitute the second camera unit 42. Each camera unit 41, 42 is a combination of a wide-angle camera and a narrow-angle camera, and captures a 180° range around the support column 12. The first camera unit 41 and the second camera unit 42 capture different 180° ranges around the support column 12, thereby capturing a 360° (full circumference) range around the support column 12 as a whole.

[0034] The second wide-angle camera 22 is positioned 180° differently from the first wide-angle camera 21 around the support column 12, and captures areas around the support column 12 that are outside the shooting range of the first wide-angle camera 21. Since the second wide-angle camera 22 has the same configuration as the first wide-angle camera 21, the explanation of its configuration, which is the same as that of the first wide-angle camera 21, will be omitted as it would be redundant.

[0035] The second narrow-angle camera 32 is positioned 180° differently from the first narrow-angle camera 31 around the support column 12, and is capable of capturing at least the shooting range of the second wide-angle camera 22 around the support column 12. Since the second narrow-angle camera 32 has the same configuration as the first narrow-angle camera 31, the explanation of its configuration, which is the same as that of the first narrow-angle camera 31, will be omitted as it would be redundant.

[0036] In this embodiment, we will describe an example in which two camera units 41 and 42 are used to capture 360° (the entire circumference) around the support column 12, but this is not required. Alternatively, m (where m is an integer of 3 or more) camera units may be arranged at equal pitches around the support column 12, and each camera unit may capture a range of (360 / m)° around the support column 12, so that multiple camera units cooperate to capture 360° (the entire circumference) around the support column 12.

[0037] In this embodiment, an umbrella member 18 is positioned above the LiDAR sensor 20. The umbrella member 18 is configured to cover the LiDAR sensor 20 and each camera unit 41, 42 from above. The umbrella member 18 protects the sensor unit 13 and the control unit 14 from rain. In addition, the umbrella member 18 can suppress direct sunlight from hitting the sensor unit 13, thereby suppressing halation in the detection results of the sensor unit 13.

[0038] In this embodiment, the control unit 14 is a computer including a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory). The control unit 14 may also include an FPGA (Field Programmable Gate Array), be formed using a PLC (Programmable Logic Controller), or be formed using sequence circuits, etc. The control unit 14 performs various functions programmatically by the CPU executing programs stored in a storage device such as ROM or a hard disk drive (HDD).

[0039] The control unit 14 is connected to the sensor unit 13, the memory unit 15, and the warning device 16.

[0040] The control unit 14 includes a setting unit 25 for configuring settings related to the monitoring device 10, a display control unit 26 for controlling displays on the mobile terminal 50 and the warning device 16, and a detection unit 27 for detecting the entry of object B1 into a preset danger area 102 (monitoring area).

[0041] In this embodiment, the setting unit 25 performs various settings in response to the operation of the mobile terminal 50 by the worker P1 or supervisor P2. Specific examples of the settings performed by the setting unit 25 will be described later.

[0042] Figure 4 shows an example of the layout on the display screen D1 of the mobile terminal 50, and includes an example of the LiDAR screen D2 displayed on the mobile terminal 50. Figure 5 shows an example of the first operation unit 51 displayed on the display screen D1 of the mobile terminal 50. Figure 6 shows an example of the second operation unit 52 displayed on the display screen D1 of the mobile terminal 50. As shown in Figures 3 to 6, the display control unit 26 displays various settings and monitoring results from the monitoring device 10 on the display screen (display unit) D1 of the mobile terminal 50.

[0043] On the display screen D1 of the mobile terminal 50, for example, the LiDAR screen D2 is displayed from the center to the left side of the screen. In addition, either the first operation unit 51 or the second operation unit 52 is displayed on the right side of the display screen D1, for example. Note that the layout on the display screen D1 is merely an example, and other layouts may be adopted.

[0044] The first operation unit 51 and the second operation unit 52 are user interfaces operated by the operator P1 or supervisor P2 operating the mobile terminal 50, such as a touch panel, keyboard, or mouse. The first operation unit 51 and the second operation unit 52 are operated by the operator P1 or supervisor P2 based on the display shown on the mobile terminal 50, and command signals corresponding to the operations are output from the mobile terminal 50 to the control unit 14, for example, the setting unit 25.

[0045] When a predetermined operation is performed by worker P1 or supervisor P2, the program for the monitoring device 10 on the mobile terminal 50 is launched, and the first operation unit 51 is displayed on the mobile terminal 50. The first operation unit 51 includes various operation units. In this embodiment, the hazard area read button 53 is a button that requests the setting unit 25 to recall the setting if a hazard area 102 has already been set. The START button 54 is a button that requests the display control unit 26, etc., to start the operation of the monitoring device 10 and to display the measurement results from the LiDAR sensor 20 on the LiDAR screen D2. The STOP button 55 is a button that terminates the operation of the monitoring device 10.

[0046] The Hazard Area Creation Button 56 is a button that requests the setting unit 25 to start setting the Hazard Area 102. In this embodiment, the Hazard Area 102 is configured to be set at multiple locations around the support column 12. The Hazard Area Setting Method Specification Button 57 is a button that sets whether the position and range of the Hazard Area 102 is set by operating the LiDAR screen D2 (clicking or tapping on the screen) or by operating the first operation unit 51 (clicking or tapping on the screen). If the Hazard Area Setting Method Specification Button 57 is used to select that the position and range of the Hazard Area 102 be set by operating the LiDAR screen D2, the worker P1 or supervisor P2 clicks or taps multiple points (for example, two points on opposite sides) on the LiDAR screen D2 which displays the work site 100 in a plan view. As a result, the setting unit 25 sets the area specified by the click or tap as a plan view rectangular Hazard Area 102. In this case, the height range in the Z direction (vertical direction) of the hazardous area 102 is, for example, predetermined, with the lower end being, for example, 0.25m above the top surface 100a of the work site 100, and the range from the lower end upwards to, for example, 2m. The Z-direction height range may also be configured to be set by clicking or tapping on the LiDAR screen D2.

[0047] On the other hand, if the operation of the hazard area setting method specification button 57 selects that the position and range of the hazard area 102 be set by operating the first operation unit 51, then the worker P1 or supervisor P2 sets the reference points (coordinate points in the horizontal plane) in the X and Y directions, as well as the reference point in the Z direction (coordinate point in the vertical direction), according to the display on the first operation unit 51. Furthermore, the worker P1 or supervisor P2 sets the width, length (range of the rectangular hazard area in the horizontal plane), and height of the hazard area 102 by operating the first operation unit 51. This sets the rectangular hazard area 102. The first operation unit 51 includes three slide bars 58. By operating these slide bars 58, the position of the hazard area 102 in the X, Y, and Z directions may be adjusted. The X, Y, and Z axes may be relative coordinate axes that change along the orientation of the monitoring device 10, or they may be absolute coordinate axes based on the cardinal directions (east, west, north, and south).

[0048] The area center acquisition button 59 is a button that outputs a signal to the setting unit 25 requesting it to acquire the center position of the set hazardous area 102. The area size acquisition button 60 is a button that outputs a signal to the setting unit 25 requesting it to acquire the size (size) of the set hazardous area 102. The hazardous area save button 61 is a button that requests the setting unit 25 to save data identifying the set hazardous area 102, linking it to a predetermined area number as an index. The cancel button 62 is a button that requests the control unit 14 not to save the set hazardous area 102. The hazardous area delete button 63 is a button that requests the setting unit 25 to delete the data of the hazardous area 102 that has already been saved, by specifying an index and linking it to that index.

[0049] Furthermore, the LiDAR screen D2 may be configured to allow switching between a side view and a plan view from a predetermined direction, depending on the settings in the first operation unit 51 or the second operation unit 52, or the LiDAR screen D2 may be configured to display a three-dimensional image. By displaying a three-dimensional image on the LiDAR screen D2, the operator P1 or supervisor P2 can more easily and visually set the hazard area 102 in the Z direction.

[0050] Furthermore, when a hazardous area 102 is set, the setting unit 25 stores the point cloud data from the LiDAR sensor 20, excluding the point cloud data that identifies light reflected from the upper surface 100a of the work site 100, as reference point cloud data in the storage unit 15. This suppresses false detections when the monitoring device 10 is shaken by wind or the like. It is preferable that at least multiple sets (multiple time points) of reference point cloud data are stored.

[0051] Furthermore, when a predetermined operation is performed by worker P1 or supervisor P2, the second operation unit 52 is displayed on the mobile terminal 50. The second operation unit 52 includes various operation units. In this embodiment, the detection area setting button 65 is a button that causes the setting unit 25 to set whether to detect the entire 360° (all around) of the support column 12 with both the first and second camera units 41 and 42, to detect 180° of the support column 12 with the sensor unit 13 using the first camera unit 41 but not the second camera unit 42, or to detect 180° of the support column 12 with the sensor unit 13 using the second camera unit 42 but not the first camera unit 41.

[0052] The camera setting button 66 is a button that allows the setting unit 25 to set whether to use the wide-angle camera 21(22) or the narrow-angle camera 31(32) for monitoring the hazardous area 102. Note that "wide-angle camera 21(22)" refers to the wide-angle camera 21 or 22 selected by the second operation unit 52, and "narrow-angle camera 31(32)" refers to the narrow-angle camera 31 or 32 selected by the second operation unit 52.

[0053] The camera setting button 66 allows selection of three camera modes: (i) both wide-angle camera 21(22) and narrow-angle camera 31(32) are used (dual camera mode); (ii) wide-angle camera mode is used but the narrow-angle camera 31(32) is not (wide-angle camera mode); and (iii) narrow-angle camera mode is used but the wide-angle camera 31(32) is not (narrow-angle camera mode). When both camera modes are selected, the monitoring device 10 monitors for the presence or absence of objects entering the hazardous area 102 using both the wide-angle camera 21(22) and the narrow-angle camera 31(32). When wide-angle camera mode is selected, the monitoring device 10 monitors for the presence or absence of objects entering the hazardous area 102 using the wide-angle camera 21(22). When narrow-angle camera mode is selected, the monitoring device 10 monitors for the presence or absence of objects entering the hazardous area 102 using the narrow-angle camera 31(32). In the following, both the wide-angle camera mode and the narrow-angle camera mode will be collectively referred to as "single-camera mode." In either camera mode, monitoring using the LiDAR sensor 20 will be performed.

[0054] The detection sensitivity threshold adjustment unit 67 is a setting unit for setting the detection sensitivity of the LiDAR sensor 20 at various distances from the sensor unit 13. Since the LiDAR sensor 20 emits light signals radially, even small signal changes are detected as large changes the further away you are from the LiDAR sensor 20. For this reason, the detection sensitivity threshold adjustment unit 67 can be set to lower the detection sensitivity as the distance from the LiDAR sensor 20 increases. In this embodiment, the detection sensitivity threshold adjustment unit 67 has multiple distance ranges (5 steps) set, and the detection sensitivity can be set for each distance range.

[0055] The reset button 68 resets (initializes) the settings of the second operation unit 52. The change button 69 confirms the settings changed in the second operation unit 52 in the setting unit 25.

[0056] The detection unit 27 is provided to detect the entry of object B1 into the hazardous area 102, which is a pre-set monitoring area. Based on the point cloud data, which is the detection signal from the LiDAR sensor 20, the detection unit 27 detects (determines) whether or not object B1 has entered the hazardous area 102. For example, the detection unit 27 refers to the time change of the point cloud data in the hazardous area 102 and determines that object B1 has entered the hazardous area 102 if the amount of change of the point cloud data with respect to time in the hazardous area 102 is greater than a certain amount. In other words, the detection unit 27 determines that object B1 has entered the hazardous area 102 if the difference in the point cloud data in the hazardous area 102 exceeds a threshold. At this time, the detection unit 27 compares each of the above-mentioned reference point cloud data stored in multiple sets with the point cloud data and determines that object B1 has entered the hazardous area 102 if the difference exceeds the threshold multiple times, thereby suppressing false detections when the monitoring device 10 is shaken by the wind.

[0057] The method by which the detection unit 27 detects whether or not object B1 has entered the hazardous area 102 using the measurement results of the LiDAR sensor 20 is not limited, and other general detection methods such as pattern matching may be used. In this embodiment, the detection unit 27 detects whether or not object B1 has entered the hazardous area 102 using the measurement results of the LiDAR sensor 20, but does not determine whether object B1 is luggage S1 or worker P1 (human).

[0058] The detection unit 27 uses image data acquired by the wide-angle camera 21(22) to detect (determine) whether or not a worker P1 (human) has entered the hazardous area 102. The detection unit 27 determines whether or not object B1 is worker P1 (human) using general person detection methods that use image data, such as pattern matching or feature point extraction. In this way, the detection unit 27 determines whether or not object B1 that has entered the hazardous area 102 is worker P1 based on image data from the wide-angle camera 21(22).

[0059] Similar to person detection using the wide-angle camera 21(22), the detection unit 27 uses image data acquired by the narrow-angle camera 31(32) to detect (determine) whether or not worker P1 has entered the hazardous area 102. The detection unit 27 determines whether or not object B1 is worker P1 (human) using general person detection methods that use image data, such as pattern matching or feature point extraction. In this way, the detection unit 27 determines whether or not object B1 that has entered the hazardous area 102 is worker P1 based on image data from the narrow-angle camera 31(32).

[0060] In this embodiment, in both camera modes, the detection unit 27 determines whether or not object B1 has entered the hazardous area 102 based on data from the LiDAR sensor 20. When the detection unit 27 detects that object B1 has entered the hazardous area 102 based on data from the LiDAR sensor 20, it determines whether or not object B1 is worker P1 based on data from at least one of the wide-angle camera 21(22) and the narrow-angle camera 31(32).

[0061] In particular, in this embodiment, in both camera modes, if the detection unit 27 cannot determine whether object B1 is a worker P1 (human) based on image data from the wide-angle camera 21 (22), it determines whether object B1 is a worker P1 (human) based on image data from the narrow-angle camera 31 (32).

[0062] Furthermore, in this embodiment, in single-camera mode, the detection unit 27 determines whether or not object B1 has entered the hazardous area 102 based on data from the LiDAR sensor 20. When the detection unit 27 detects that object B1 has entered the hazardous area 102 based on data from the LiDAR sensor 20, it determines whether or not object B1 is worker P1 based on data from the wide-angle camera 21(22) or the narrow-angle camera 31(32).

[0063] Furthermore, in this embodiment, the detection unit 27 can determine whether or not object B1 is a human being using the image data recorded in the storage unit 15 in both camera modes and single camera modes.

[0064] The storage unit 15 is, for example, a storage device such as an HDD (hard disk drive) or an SSD (solid state drive), and stores point cloud data from the LiDAR sensor 20 obtained by the sensor unit 13, and image data obtained by imaging from each camera 21, 31, 22, and 32. The storage unit 15 is connected to the sensor unit 13 and the control unit 14. The storage unit 15 may store only one of the above point cloud data or image data.

[0065] The above is a general overview of the monitoring device 10.

[0066] (device operation) (In both camera modes) Next, an example of the operation of the monitoring system 1, including the monitoring device 10, will be explained using Figures 7 and 8. Figures 7 and 8 are flowcharts showing an example of the operation of the monitoring system 1, including the monitoring device 10, and illustrate the case where both camera modes are set. In the following explanation, Figures 1 to 8 will be referred to as appropriate.

[0067] In the monitoring system 1, the hazardous area 102 is set in the setting unit 25 when the mobile terminal 50 is operated by worker P1 or supervisor P2 (step S1). Also, both camera modes are set when the mobile terminal 50 is operated by worker P1 or supervisor P2 (step S2).

[0068] Next, when the START button 54 is operated, monitoring by the monitoring device 10, which includes the sensor unit 13 and the control unit 14, begins (step S3). Specifically, the LiDAR sensor 20, the wide-angle camera 21(22), and the narrow-angle camera 31(32) are activated. As a result, the LiDAR sensor 20 measures the dangerous area 102, and the wide-angle camera 21(22) and narrow-angle camera 31(32) take images. The data from the LiDAR sensor 20, the wide-angle camera 21(22), and the narrow-angle camera 31(32) are then provided to the detection unit 27 and the storage unit 15.

[0069] When monitoring begins, the detection unit 27 first determines whether or not object B1 has entered the hazard area 102 based on point cloud data from the LiDAR sensor 20 (step S4). The detection unit 27 repeats the process in step S4 until it determines that object B1 has entered the hazard area 102 (NO in step S4).

[0070] If the detection unit 27 determines that object B1 has entered the hazardous area 102 based on point cloud data from the LiDAR sensor 20 (YES in step S4), the detection unit 27 issues a first warning signal to instruct the warning device 16 and the mobile terminal 50 to issue a first warning (step S5). The first warning refers to the operation of the warning device 16, for example, by emitting an orange light and a first warning sound, and the operation of the mobile terminal 50 by emitting an orange light across the entire screen or a predetermined area on the screen and emitting a warning sound. The first warning is not limited to the above specific example, and only requires that at least one of the warning device 16 and the mobile terminal 50 notify that object B1 has entered the hazardous area 102.

[0071] When the first warning is issued, the control unit 14 operates the narrow-angle camera 31(32) so that its field of view includes the area in the hazardous area 102 where object B1 has been detected (step S6). Through this process, the narrow-angle camera 31(32) is configured to move its field of view and zoom (adjust magnification) to the hazardous area 102 in which the LiDAR sensor 20 has determined that object B1 has entered, among the multiple hazardous areas 102.

[0072] In conjunction with the processing in step S6, the detection unit 27 uses image data from the wide-angle camera 21(22) to determine whether or not the entity that entered the hazardous area 102 is a worker P1 (human) (step S7). If the detection unit 27 determines, based on the image data from the wide-angle camera 21(22), that the object B1 that entered the hazardous area 102 is a worker P1 (human) (YES in step S7), it issues a second warning signal to instruct the warning device 16 and the mobile terminal 50 to issue a second warning (step S8). The second warning is a clearer warning than the first warning. The second warning refers to the operation of the warning device 16, for example, by emitting a red light and a second warning sound, and also refers to the operation of the mobile terminal 50 by emitting a red light across the entire screen or at a predetermined location on the screen, and by emitting a louder warning sound than the first warning. Furthermore, the second warning is not limited to the specific examples above, and only requires that the warning device 16 and the portable terminal 50 notify the operator that worker P1 has entered the hazardous area 102.

[0073] On the other hand, if the detection unit 27 does not determine that the object B1 that entered the hazardous area 102 is a worker P1 (human) based on the image data from the wide-angle camera 21 (22) (NO in step S7), it uses the image data from the narrow-angle camera 31 (32) to determine whether or not the person who entered the hazardous area 102 is a worker P1 (human) (step S9). If the detection unit 27 determines that the person who entered the hazardous area 102 is a worker P1 (human) based on the image data from the narrow-angle camera 31 (32) (YES in step S9), it issues a second warning signal to instruct the warning device 16 and the mobile terminal 50 to issue a second warning (step S8).

[0074] Thus, in this embodiment, when the detection unit 27 determines that object B1 has entered the hazardous area 102, it determines whether object B1 is worker P1 or not based on image data from the wide-angle camera 21(22) or the narrow-angle camera 31(32).

[0075] Furthermore, in this embodiment, in steps S7 and S9, the detection unit 27 determines whether object B1 is worker P1 or not based on image data from the wide-angle camera 21 (22) or the narrow-angle camera 31 (32), without using point cloud data from the LiDAR sensor 20. Alternatively, in steps S7 and S9, the detection unit 27 may also use data from the LiDAR sensor 20 to determine whether object B1 is worker P1 or not.

[0076] Next, the detection unit 27 determines whether object B1 or worker P1 has evacuated from the hazardous area 102 (step S10). If the detection unit 27 determines that object B1 or worker P1 has not evacuated from the hazardous area 102 (NO in step S10), it waits. For example, if object B1 detected in the hazardous area 102 is luggage S1 or something other than worker P1 (a person), the detection unit 27 detects that object B1 has evacuated from the hazardous area 102 when worker P1 removes it (YES in step S10). Also, if object B1 detected in the hazardous area 102 is worker P1, the detection unit 27 detects that worker P1 has evacuated from the hazardous area 102 when worker P1 evacuates from the hazardous area 102 (YES in step S10).

[0077] If the answer in step S10 is YES, the detection unit 27 outputs a warning cancellation signal, thereby canceling the warnings from the warning device 16 and the portable terminal 50 (step S11). This cancels the warning displays on the warning device 16 and the portable terminal 50. Next, if the STOP button 55 on the portable terminal 50 is not operated (NO in step S12), the steps from step S4 onwards are repeated. On the other hand, if the STOP button 55 on the portable terminal 50 is operated (YES in step S12), the monitoring device 10 stops detecting object entry into the hazardous area 102.

[0078] (In single-camera mode) Next, another example of the operation of the monitoring system 1, including the monitoring device 10, will be described using Figures 9 and 10. Figures 9 and 10 are flowcharts showing another example of the operation of the monitoring system 1, including the monitoring device 10, and illustrate the case where a single-camera mode (wide-angle camera mode or narrow-angle camera mode) is set. In the following description, Figures 1 to 7, 9 and 10 will be referred to as appropriate.

[0079] When single-camera mode is set, the process of setting the hazardous area 102 (step S21) is the same as step S1 in both-camera modes. Also, single-camera mode (wide-angle camera mode or narrow-angle camera mode) is set when the mobile terminal 50 is operated by worker P1 or supervisor P2 (step S22).

[0080] Next, when the START button 54 is operated, monitoring by the monitoring device 10, which includes the sensor unit 13 and the control unit 14, begins (step S23). Specifically, the LiDAR sensor 20 operates, and the wide-angle camera 21(22) or narrow-angle camera 31(32) also operates. As a result, point cloud data from the LiDAR sensor 20 and image data from the wide-angle camera 21(22) or narrow-angle camera 31(32) are provided to the detection unit 27 and the storage unit 15.

[0081] When monitoring begins, the detection unit 27 first determines whether or not object B1 has entered the hazardous area 102 based on data from the LiDAR sensor 20 (step S24). The detection unit 27 repeats the process in step S24 until it determines that object B1 has entered the hazardous area 102 (NO in step S24).

[0082] If the detection unit 27 determines that object B1 has entered the hazardous area 102 based on point cloud data from the LiDAR sensor 20 (YES in step S24), the detection unit 27 issues a first warning signal to instruct the warning device 16 and the mobile terminal 50 to issue a first warning (step S25).

[0083] When the first warning is issued and the system is in narrow-angle camera mode (narrow-angle camera mode in step S26), the control unit 14 operates the narrow-angle camera 31(32) so that its field of view includes the location in the hazard area 102 where object B1 has been detected (step S27). Through this process, the narrow-angle camera 31(32) is configured to move its field of view to the hazard area 102 in which the LiDAR sensor 20 has determined that object B1 has entered, among the multiple hazard areas 102.

[0084] On the other hand, when in wide-angle camera mode (wide-angle camera mode in step S27), the operation (field of view movement) of the narrow-angle camera 31 (32) is not performed.

[0085] Next, the detection unit 27 uses image data from the selected wide-angle camera 21(22) or narrow-angle camera 31(32) to determine whether or not the object B1 that entered the hazardous area 102 is a worker P1 (human) (step S28). If the detection unit 27 determines that the object B1 that entered the hazardous area 102 is a worker P1 (human) (YES in step S28), it issues a second warning signal to instruct the warning device 16 and the mobile terminal 50 to issue a second warning (step S29).

[0086] On the other hand, if the detection unit 27 does not determine that the object B1 that entered the hazardous area 102 is a worker P1 (human) based on the image data from the selected wide-angle camera 21 (22) or narrow-angle camera 31 (32) (NO in step S28), it maintains the first warning.

[0087] Next, the detection unit 27 determines whether the object B1, such as the luggage S1 or worker P1, has moved out of the hazardous area 102 (step S30). If the detection unit 27 determines that the object B1 or worker P1 has not moved out of the hazardous area 102 (NO in step S30), it waits.

[0088] If the answer in step S30 is YES, the detection unit 27 outputs a warning cancellation signal, thereby canceling the warnings from the warning device 16 and the portable terminal 50 (step S31). This cancels the warning displays on the warning device 16 and the portable terminal 50. Next, if the STOP button 55 on the portable terminal 50 is not operated (NO in step S32), the steps from step S24 onwards are repeated. On the other hand, if the STOP button 55 on the portable terminal 50 is operated (YES in step S32), the monitoring device 10 stops detecting object entry into the hazardous area 102.

[0089] (effect) As described above, according to this embodiment, the detection unit 27 determines whether or not object B1 has entered the hazardous area 102 based on point cloud data from the LiDAR sensor 20, and determines whether or not object B1 is worker P1 based on image data from at least one of the wide-angle camera 21(22) and the narrow-angle camera 31(32). With this configuration, the LiDAR sensor 20 can quickly detect the entry of an object into the hazardous area 102. Furthermore, while it is difficult to determine whether or not object B1 that has entered the hazardous area 102 is worker P1 based on point cloud data from the LiDAR sensor 20, by using image data from at least one of the two types of cameras (wide-angle cameras 21, 22, and narrow-angle cameras 31, 32), it is possible to more accurately determine whether or not object B1 that has entered the hazardous area 102 is worker P1. In this way, by using both the data from the LiDAR sensor 20 and the image data from at least one of the two types of cameras (wide-angle cameras 21, 22 and narrow-angle cameras 31, 32) in coordination, it is possible to detect more quickly and accurately whether or not an object B1 (worker P1) has entered the hazardous area 102.

[0090] Furthermore, according to this embodiment, the narrow-angle camera 31(32) is configured to move its field of view to the hazardous area 102 in which the LiDAR sensor 20 has determined that object B1 has entered, among the multiple hazardous areas 102. With this configuration, the detection unit 27 can more quickly determine whether object B1 that has entered the hazardous area 102 is worker P1 or not by using data from the wide-angle camera 21(22). On the other hand, since the narrow-angle camera 31(32) generally has a higher resolution than the wide-angle camera 21(22), the narrow-angle camera 31(32) can capture object B1 that has entered the hazardous area 102 more clearly. Therefore, the detection unit 27 can more quickly detect whether object B1 is worker P1 or not using image data from the wide-angle camera 21(22), and furthermore, it can more accurately determine whether object B1 is worker P1 or not using image data from the narrow-angle camera 31(32).

[0091] Furthermore, according to this embodiment, if the detection unit 27 cannot determine whether object B1 is worker P1 based on image data from the wide-angle camera 21(22), it determines whether object B1 is worker P1 based on image data from the narrow-angle camera 31(32). This configuration allows for faster person detection by using image data obtained more quickly.

[0092] Furthermore, according to this embodiment, the narrow-angle camera 31(32) is equipped with a zoom mechanism that can change the shooting magnification. With this configuration, the detection unit 27 can obtain clearer image data for determining whether or not object B1 is worker P1 by zooming in on the narrow-angle camera 31(32).

[0093] Furthermore, according to this embodiment, the detection unit 27 determines whether or not object B1 has entered the hazardous area 102 based on point cloud data from the LiDAR sensor 20, and when it is determined that object B1 has entered the hazardous area 102, it determines whether or not object B1 is worker P1 based on image data from the wide-angle camera 21(22) and the narrow-angle camera 31(32). With this configuration, the LiDAR sensor 20 can quickly detect the entry of an object into the hazardous area 102. Moreover, while it is difficult to determine whether or not object B1 that has entered the hazardous area 102 is worker P1 based on point cloud data from the LiDAR sensor 20, by using image data from cameras 21, 31, 22, and 32, it is possible to more accurately determine whether or not object B1 that has entered the hazardous area 102 is worker P1. In this way, the detection unit 27 can more quickly and accurately detect whether object B1 and worker P1 have entered the hazardous area 102 by using both point cloud data from the LiDAR sensor 20 and image data from cameras 21, 31, 22, and 32 in cooperation.

[0094] Furthermore, according to this embodiment, the detection unit 27 issues a first warning signal when it determines that object B1 has entered the hazardous area 102 based on point cloud data from the LiDAR sensor 20, and issues a second warning signal when it determines that object B1 is worker P1 based on image data from the wide-angle camera 21(22) or narrow-angle camera 31(32). With this configuration, the monitoring device 10 can provide a rapid warning of object B1 entering the hazardous area 102 based on point cloud data from the LiDAR sensor 20. In addition, the detection unit 27 can clearly notify that worker P1 has entered the hazardous area 102 based on processing using image data from the wide-angle camera 21(22) or narrow-angle camera 31(32), which takes more time than detecting the entry of object B1 using point cloud data from the LiDAR sensor 20. Therefore, worker P1 who has entered the hazardous area 102 and people around them can be more clearly notified that worker P1 has entered the hazardous area 102.

[0095] Furthermore, according to this embodiment, the detection unit 27 uses the data recorded in the storage unit 15 to determine whether or not object B1 is worker P1. With this configuration, after the entry of object B1 into the hazardous area 102 is detected by point cloud data from the LiDAR sensor 20, the most recent image data already recorded in the storage unit 15 can be used to more quickly determine whether or not object B1 is worker P1 based on the image data.

[0096] Furthermore, according to this embodiment, in steps S7, S9, and S28, the detection unit 27 can determine whether object B1 is worker P1 or not based on image data from the wide-angle camera 21(22) or the narrow-angle camera 31(32), without using point cloud data from the LiDAR sensor 20. With this configuration, the algorithm for determining whether object B1 is worker P1 or not in the detection unit 27 can be simplified, and the determination of whether object B1 is worker P1 or not can be made more quickly.

[0097] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be modified in various ways as long as they are described in the claims.

[0098] For example, the control unit 14 in this embodiment can be implemented not by a computer on which the program is installed, but by using hardware corresponding to each part, such as electronic circuits. Furthermore, the control unit 14 may be partially implemented by a program and the remaining part by hardware. [Industrial applicability]

[0099] This invention can be applied as a monitoring device. [Explanation of Symbols]

[0100] 10 Monitoring equipment 20 LiDAR sensors (distance measuring sensors) 21,22 Wide-angle camera 27 Detection unit 31,32 Narrow-angle camera 102 Dangerous Area (Monitoring Area) B1 Object P1 Worker (Human)

Claims

1. A distance measuring sensor that measures the position of an object and the distance to the object, Wide-angle camera, A narrow-angle camera is configured to have a field of view narrower than that of the wide-angle camera and to be able to change the position of the field of view. It includes a detection unit that detects the entry of an object into a pre-set monitoring area, The detection unit determines whether the object has entered the monitoring area based on data from the distance measuring sensor, and determines whether the object is a human being based on data from at least one of the wide-angle camera and the narrow-angle camera, in this monitoring device.

2. The aforementioned monitoring area is configured to be configurable in multiple locations. The wide-angle camera is configured to capture at least two of the aforementioned monitoring areas, The monitoring device according to claim 1, wherein the narrow-angle camera is configured to move its field of view to the monitoring area in which the distance measuring sensor has determined that an object has entered, among a plurality of monitoring areas.

3. The monitoring device according to claim 1, wherein the detection unit determines whether or not the object is a human being based on data from the narrow-angle camera when it is not possible to determine whether or not the object is a human being based on data from the wide-angle camera.

4. The monitoring device according to any one of claims 1 to 3, wherein the narrow-angle camera is equipped with a zoom mechanism capable of changing the shooting magnification.

Images