Heavy equipment monitoring system

Abstract

To provide a heavy machine monitoring system that can easily detect abnormalities in and around a heavy machine even during remote control when no operator is on the heavy machine.SOLUTION: A monitoring system 400 for heavy machines operating in an excavation area has a three-dimensional scanner 201 that measures three-dimensional distances within the excavation area in which a heavy machine is deployed and outputs three-dimensional data including the heavy machine, a thermographic camera 202 that acquires temperature distribution images of the excavation area including at least the heavy machine, a heavy machine detection part 405 that identifies the shape of the heavy machine from the three-dimensional data and detects the position information of the heavy machine within the excavation area, and an abnormality determination part 407 that determines the presence or absence of abnormal areas in and around the heavy machine based on the shape and position information of the heavy machine and temperature distribution images.SELECTED DRAWING: Figure 5

Description

Technical Field

[0006] , ,

[0005] , , , , ,

[0001] The present invention relates to a monitoring system for heavy machinery used at a construction site.

Background Art

[0002] Heavy machinery such as backhoes, bulldozers, and drill jumbos used at construction sites or tunnel construction sites need to be operated so as to avoid contact with other heavy machinery and workers. Therefore, many technologies for monitoring the surrounding area of heavy machinery have been proposed.

[0003] For example, in the system disclosed in Patent Document 1, a communication terminal of an operator detects magnetism from a heavy machine and transmits a magnetic detection signal, and a control device located away from the construction area receives the magnetic detection signal to monitor the state of the operator and the heavy machine. Further, in the system disclosed in Patent Document 2, the alarm output is optimized by adjusting the monitoring area according to the change in the outer shape of the heavy machine.

Prior Art Documents

[0007] This invention was devised in view of the above circumstances, and the object of this invention is to provide a heavy equipment monitoring system that can easily detect abnormalities in heavy equipment and its surroundings, even in the case of remote control where an operator is not on board the heavy equipment. [Means for solving the problem]

[0008] To achieve the above-mentioned objectives, one embodiment of the present invention is a monitoring system for heavy machinery operating in an excavation area, comprising: a 3D scanner that measures the 3D distance within the excavation area where the heavy machinery is located and outputs 3D data including the heavy machinery; a thermographic camera that acquires a temperature distribution image of the excavation area including at least the heavy machinery; a heavy machinery detection unit that identifies the shape of the heavy machinery from the 3D data and detects the position information of the heavy machinery within the excavation area; and an abnormality determination unit that determines the presence or absence of abnormal areas in and around the heavy machinery based on the shape and position information of the heavy machinery and the temperature distribution image. Furthermore, according to one embodiment of the present invention, the abnormality determination unit can determine the occurrence of an abnormal region if at least one of a predetermined number of regions of the heavy machine exceeds a predetermined threshold temperature. Furthermore, according to one embodiment of the present invention, a predetermined plurality of areas of the heavy machine may include the hydraulic cylinder of the heavy machine, a hydraulic oil tank for driving the hydraulic cylinder, an oil pan of the engine which is the power source of the heavy machine, the radiator and exhaust pipe of the engine, a bearing part of the drive system for moving the heavy machine, and an electric motor. Furthermore, according to one embodiment of the present invention, the invention further includes a noise and vibration sensor for detecting noise and vibration in the excavation area and the heavy machinery, and the abnormality determination unit can determine whether or not there is an abnormal area in and around the heavy machinery based on the temperature distribution image and the noise and vibration in the excavation area and the heavy machinery. Furthermore, according to one embodiment of the present invention, when the abnormality detection unit determines that an abnormality area has occurred, it can issue an alarm both inside and outside the excavation area 10. Furthermore, according to one embodiment of the present invention, the heavy machinery can be remotely controlled by a remote control system. Furthermore, according to one embodiment of the present invention, the remote control system can remotely operate the heavy machinery according to the position information from the heavy machinery detection unit. [Effects of the Invention]

[0009] According to one embodiment of the present invention, the presence or absence of abnormal areas in and around the heavy machinery is determined based on the shape and position information of the heavy machinery and the temperature distribution image. Therefore, even in the case of remote control where an operator is not on board the heavy machinery, abnormalities in and around the heavy machinery can be easily detected. Furthermore, according to one embodiment of the present invention, if the temperature of at least one of a predetermined number of regions of the heavy machinery exceeds a predetermined threshold, the occurrence of an abnormal region is determined. This allows for early detection of abnormalities and prevents fires and other incidents. Furthermore, according to one embodiment of the present invention, by designating areas within heavy machinery that become particularly hot or areas where the danger increases when temperatures are high as the monitoring target area, abnormalities can be detected early, and fires and other incidents can be prevented. Furthermore, according to one embodiment of the present invention, when an abnormal area is detected, an alarm can be issued both inside and outside the excavation area 10 to notify the danger, thereby further enhancing the safety of the construction work. Furthermore, according to one embodiment of the present invention, construction safety can be further enhanced by remotely controlling heavy machinery. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram illustrating the overall configuration of a heavy equipment monitoring system according to one embodiment of the present invention. [Figure 2] This is a block diagram showing an example of the functional configuration of the heavy machinery used in this embodiment. [Figure 3] This is a block diagram showing an example of the hydraulic system of heavy machinery used in this embodiment. [Figure 4] This block diagram shows an example of the functional configuration of the remote control system in this embodiment. [Figure 5] This block diagram shows an example of the functional configuration of the drilling management system in this embodiment. [Figure 6] This figure shows an example of temperature distribution using thermography in the drilling management system of this embodiment. [Figure 7] This flowchart shows an example of the operation of the heavy equipment monitoring system according to this embodiment. [Modes for carrying out the invention]

[0011] 1. System Configuration The following describes in detail a heavy machinery monitoring system according to one embodiment of the present invention, using an excavation project as shown in Figure 1 as an example. However, the heavy machinery monitoring system according to this embodiment is applicable not only to tunnel excavation projects but also to construction projects in general that use heavy machinery. Furthermore, the heavy machinery may be any of the following: backhoe, bulldozer, drill jumbo, etc.

[0012] As shown in Figure 1, the heavy machinery 100 is positioned in front of the excavation surface 11 of the excavation area 10. The heavy machinery 100 has a boom arm 101, attachments such as a bucket or drilling machine provided at its tip, an imaging unit 112, etc. The movement of the heavy machinery 100 itself, the operation of the boom and arm (hereinafter referred to as boom arm 101), and the hydraulic attachments can be automated by a remote control system 300, which will be described later.

[0013] Further, a mobile trolley 200 is arranged behind the heavy machine 100 within the excavation area 10, and at least a 3D scanner 201 and a thermographic camera 202 are mounted on the trolley 200. The 3D scanner 201 is installed at the top of the trolley 200. By moving the trolley 200, the three-dimensional distances of the inner wall portion of the excavation area 10, the excavation surface 11, and the excavation area 10 including the heavy machine 100 can be measured. The thermographic camera 202 is an infrared camera that uses infrared rays to acquire temperature distribution images of the heavy machine 100 and the excavation area 10. Further, a noise and vibration sensor 203 may be provided on the trolley 200 or within the excavation area 10.

[0014] The monitoring system 400 in the present embodiment can detect the relative position of the heavy machine 100 within the excavation area 10 by using the measurement data of the three-dimensional distances (hereinafter referred to as 3D data) acquired by the 3D scanner 201. The remote control system 300 can move the heavy machine 100 to a predetermined position by using the position data of the heavy machine 100 detected by the monitoring system 400. Further, the monitoring system 400 can acquire temperature distribution images of the heavy machine 100 and the excavation area 10 from the thermographic camera 202, and can detect abnormal areas around the heavy machine 100 and its surroundings based on the temperature distribution images and the position data of the heavy machine 100.

[0015] Note that the remote control system 300 and the monitoring system 400 according to the present embodiment may be configured by one device, or may be configured such that separate devices are connected so as to be capable of data communication. Further, the remote control system 300 and the monitoring system 400 according to the present embodiment can be constructed on a general-purpose server or a personal computer having one or more processors such as a CPU (Central Processing Unit) and an MPU (Micro Processor Unit).

[0016] Hereinafter, the functional configurations and operations of the heavy machine 100, the remote control system 300, and the monitoring system 400 in the present embodiment will be described with reference to FIGS. 2 to 5.

[0017] 2. Heavy Machine As illustrated in Figure 2, the heavy machinery 100 is remotely controllable by a remote control system 300 and includes a boom arm 101 for performing excavation work, a drive / braking system 110 for moving the heavy machinery 100, and an imaging unit 112.

[0018] The boom arm 101 is driven by a hydraulic system 111, which is controlled by an excavation control unit 121, as will be described later. For example, if the heavy machine 100 is a drill jumbo used in a tunnel or other underground space, a drilling machine is attached to the tip of the boom arm 101 to perform tasks such as drilling explosive holes in blasting methods, drilling rock bolt insertion holes, driving rock bolts into rock bolt insertion holes, and filling rock bolt insertion holes with grout.

[0019] The drive / braking system 110 is equipped with steering wheels and drive wheels for propelling the main body of the heavy machinery 100, and is controlled by the travel control unit 122. The steering wheels are located on both sides in the width direction at the front of the main body and change the direction of travel by steering. The drive wheels are located on both sides in the width direction at the rear of the main body and move the heavy machinery 100 forward and backward by rotational drive. The drive / braking system 110 also includes a power source (not shown), steering actuators, brakes, and brake actuators. The power source provides rotational driving force to the drive wheels and is either an engine (internal combustion engine) or a motor (electric motor). The steering actuators change the steering angle of the steering wheels. The brakes provide braking for the steering wheels and drive wheels. The brake actuators drive the brakes. In this embodiment, the case where the machine travels using steering wheels and drive wheels has been described, but the main body may also travel using crawlers.

[0020] The imaging unit 112 is mounted on the upper part of the main body of the heavy machinery 100, and captures images of the excavation surface 11 of the excavation area 10 to generate image information, which is then transmitted to the remote control system 300 via the communication unit 130. The image information includes images of the excavation surface 11 and the surrounding wall surface in its vicinity. In this embodiment, one imaging unit 112 is provided, but it is optional to provide two or more imaging units with different imaging ranges.

[0021] The various operations of the heavy machinery 100 are controlled by control units such as the excavation control unit 121 and the travel control unit 121 according to control signals received from the remote control system 300 via the communication unit 130. Communication between the communication unit 130 and the remote control system 300 may be via a wired line using signal cables. However, using a wireless line is preferable because it eliminates the need for cable laying work and avoids situations where cables interfere with work inside the tunnel.

[0022] Furthermore, the control units such as the excavation control unit 121 and the travel control unit 121 described above can also be implemented by launching programs that perform their respective functions on at least one processor 131.

[0023] As illustrated in Figure 3, the hydraulic system 111 of the heavy machinery 100 includes a plurality of hydraulic cylinders 102 that perform operations such as extension, rotation, and luffing of the boom arm 101, a control valve 104 that controls the hydraulic cylinders 102 and hydraulic motors 103, a hydraulic pump 105, and a hydraulic oil tank 106. The control valve 104 is controlled by the excavation control unit 121. The operation of the boom arm 101, such as extension, rotation, and luffing, is controlled by the control valve 104 by changing the hydraulic cylinder to which the pressurized oil is supplied. The hydraulic pump 105 is driven by the engine 107, which is the power source, and supplies pressurized oil from the hydraulic oil tank 106 to the control valve 104. The pressurized oil is supplied to the hydraulic cylinders 102 and hydraulic motors 103 through the control valve 104. It is recovered in the hydraulic oil tank 106 and then circulates within the hydraulic system 111.

[0024] The travel control unit 122 controls the movement of the heavy machinery 100. Specifically, it controls the on / off state of the engine 107, which is the power source, adjusts the power transmission, controls the rotational speed, controls the steering amount, and controls the braking. The coolant and lubricating oil of the engine 107 are cooled by the radiator 108, and the exhaust gas is discharged from the exhaust pipe 109.

[0025] Generally, in heavy machinery 100, the hydraulic cylinder 102, hydraulic oil tank 106, radiator 108, and exhaust pipe 109 are exposed to the outside, and their temperatures rise as the heavy machinery 100 operates. As will be described later, in this embodiment, the monitoring system 400 can monitor the temperature distribution of the heavy machinery 100 to determine whether or not there is an abnormality in the heavy machinery 100.

[0026] 3. Remote control system As illustrated in Figure 4, the remote control system 300 comprises communication units 301 and 302, a control unit 303, an operation unit 304, a display unit 305, and a program storage unit 306. The remote control system 300 is installed at a location away from the excavation site, for example, in a location inside the tunnel away from the tunnel face, or outside the tunnel.

[0027] The communication unit 301 enables wireless communication with the heavy machinery 100 described above, transmits control signals to control the heavy machinery 100, and can receive status signals indicating the posture and operating state of the heavy machinery 100, as well as imaging information such as the excavation surface 11. The communication unit 302 enables communication with the monitoring system 400, which will be described later, and can receive location information of the heavy machinery 100 from the monitoring system 400. The communication unit 302 only needs to enable communication with the monitoring system 400, and can employ a wired or wireless interface. The control unit 303 controls the overall operation of the remote control system 300.

[0028] The control unit 304 receives input from an operator remotely controlling the heavy machinery 100 and generates control signals. These control signals are transmitted to the heavy machinery 100 via the communication unit 301, thereby controlling the operation of the heavy machinery 100. Routine operations may also be stored as programs in, for example, the program storage unit 306, and the control unit 303 may read them upon request and transmit them to the heavy machinery 100 as control signals.

[0029] The display unit 305 displays video footage of the excavation area 10 or the excavation surface 11 received from the monitoring system 400. The operator of the remote control system 300 operates the control unit 304 to move the heavy machinery 100 and carry out the necessary work while checking the video footage showing the conditions inside the excavation area 10 displayed on the display unit 305.

[0030] Furthermore, the heavy machinery 100 may be equipped with a pan / tilt head device for remotely controlling the shooting direction of the imaging unit 112 and a pan / tilt head control unit for controlling the operation of the pan / tilt head device, and the remote control system 300 may be equipped with a pan / tilt head operation unit for remotely controlling the pan / tilt head device.

[0031] 4. Monitoring System As illustrated in Figure 4, the monitoring system 400 has an input unit 401 for inputting various types of data. Here, the input unit 401 is connected by wired or wireless means to a 3D scanner 201, a thermographic camera 202, and a noise / vibration sensor 203, and inputs 3D data, thermographic data, and noise / vibration data. The input unit 401 is also connected to input devices such as the system's keyboard and pointing device.

[0032] The monitoring system 400 has a control unit 402 that controls the operation of the entire system, and the control unit 402 is connected to an input unit 401, an excavation area extraction unit (excavation surface shape extraction unit 403 and excavation area side shape extraction unit 404), a heavy equipment detection unit 405, a temperature distribution analysis unit 406, an anomaly determination unit 407, a data storage unit 408, an output unit 409, and a communication unit 410.

[0033] The drilling surface shape extraction unit 403 and the drilling area side shape extraction unit 404, which constitute the drilling area extraction unit, extract the drilling surface shape and the drilling area side shape, respectively, from the 3D data acquired by the 3D scanner 201, and store them in chronological order as drilling area shape data in the data storage unit 408.

[0034] The heavy equipment detection unit 405 identifies the heavy equipment 100 from the 3D data acquired by the 3D scanner 201 and detects its relative position within the excavation area 10. This heavy equipment position data is stored chronologically in the data storage unit 408 along with the excavation area shape data. The position of the heavy equipment 100 can also be detected by, for example, installing a predetermined reflector on the heavy equipment 100 and measuring the distance to the 3D scanner 201 from the 3D data. It is also possible to identify the part of the heavy equipment 100 from the 3D data within the excavation area 10 using AI analysis and detect the position of the heavy equipment 100. The data stored in the data storage unit 408 can be used, for example, to build machine learning models in AI analysis.

[0035] The temperature distribution analysis unit 406 analyzes the temperature distribution image IMG within the excavation area 10 input from the thermographic camera 202. As described later, the temperature distribution data divided into temperature regions is stored chronologically in the data storage unit 408 along with the heavy equipment position data and the excavation area shape data. The abnormality determination unit 407 determines whether or not an abnormality has occurred in the heavy equipment 100 and the excavation area 10 by referring to the temperature distribution within the excavation area 10, including the heavy equipment 100, and its time-series changes. If an abnormality is determined to have occurred, an alarm signal is output.

[0036] The output unit 409 is a display unit such as a computer monitor and an alarm generation unit. When an alarm signal is output from the abnormality detection unit 407, an alarm can be generated both inside and outside the excavation area 10. The output unit 409 can also display the shape of the excavation area 10 extracted from 3D data, and can display thermographic images of the excavation area 10 and the heavy machinery 100.

[0037] The communication unit 410 enables communication with the remote control system 300 and transmits the aforementioned heavy equipment position information and thermographic images to the remote control system 300.

[0038] The functions of the control unit 102, the excavation surface shape extraction unit 403, the excavation area side shape extraction unit 404, the heavy equipment detection unit 405, the temperature distribution analysis unit 406, and the abnormality determination unit 407 can be realized by executing a program stored in a memory (not shown) on the processor.

[0039] As illustrated in Figure 6, the temperature distribution analysis unit 406 analyzes the temperature distribution image input from the thermographic camera 202 and divides it into temperature regions according to a predetermined concentration level 406a. The abnormality determination unit 407 determines whether or not there are any areas in the heavy machinery 100 where the temperature has risen abnormally, based on the position information of the heavy machinery 100 within the excavation area 10 identified by the heavy machinery detection unit 405 and the temperature distribution image IMG divided into temperature regions within the excavation area 10.

[0040] For example, the system determines whether the temperature in the area corresponding to the hydraulic cylinder 102, the area corresponding to the hydraulic oil tank 106, the area corresponding to the oil pan of the engine 107, the area corresponding to the radiator 108, the area corresponding to the exhaust pipe 109, the area corresponding to the drive / braking system 110 (for example, the bearing section), and the area corresponding to the electric motor have exceeded the thresholds that indicate their respective normal ranges. For example, if the temperature in at least one of these areas exceeds a predetermined threshold, the system determines that an abnormality has occurred and the abnormality determination unit 407 outputs an alarm signal.

[0041] Furthermore, the abnormality detection unit 407 can not only detect abnormalities in the heavy machinery 100, but also monitor the size of areas of abnormally high or low temperature locations within the excavation area 10 other than the heavy machinery 100 from the temperature distribution image IMG shown in Figure 6. By detecting the occurrence of abnormally high temperature locations, an increase in the area of ​​low temperature locations, the occurrence of abnormal noise, etc., it can determine the occurrence of an abnormality in the excavation area 10 and output an alarm signal.

[0042] 5. Heavy equipment monitoring operation In Figure 7, the excavation surface shape extraction unit 403 and the excavation area side shape extraction unit 404 of the monitoring system 400 extract the shape of the excavation area 10 from 3D data acquired by the 3D scanner 201, and the heavy equipment detection unit 405 identifies the heavy equipment 100 and measures its relative position within the excavation area 10 (step 501). Alternatively, an image of the excavation surface may be acquired and displayed from the imaging unit 107.

[0043] The remote control system 300 moves the heavy machinery 100 from its current position to the position where the excavation work will be performed (step 502). In this embodiment, since the shape of the excavation area 10 and the position of the heavy machinery 100 are measured, the control unit 303 of the remote control system 300 controls the drive / braking system 110 of the heavy machinery 100 via the communication unit 301, and the heavy machinery 100 can be automatically moved to a working position in front of the excavation surface 11 within the excavation area 10. Alternatively, the operator may move the heavy machinery 100 to a predetermined working position by operating the travel control unit of the control unit 304 while visually checking the situation in the excavation area 10 displayed on the display unit 305.

[0044] Once the heavy machinery 100 reaches the designated working position, the remote control system 300 operates the boom arm 101 based on the positional relationship between the excavation surface 11 and the heavy machinery 100 to perform the excavation work (step 503).

[0045] Next, the temperature distribution analysis unit 406 analyzes the temperature distribution image input from the thermographic camera 202 and acquires a temperature distribution image IMG divided into temperature regions according to a predetermined concentration level 406a (step 504). Based on the position information of the heavy machinery 100 within the excavation area 10 identified by the heavy machinery detection unit 405 and the temperature distribution image IMG acquired by the temperature distribution analysis unit 406, the anomaly detection unit 407 determines whether there are any areas in the heavy machinery 100 where the temperature has risen abnormally, and whether there are any abnormalities in the noise and vibration data (steps 505, 506).

[0046] As described above, if the temperature of the hydraulic cylinder 102, hydraulic oil tank 106, radiator 108, exhaust pipe 109, and drive / braking system 110 exceeds the threshold indicating their respective normal ranges (YES in step 506), the abnormality detection unit 407 will issue an alarm inside and outside the excavation area 10 (step 507). The abnormality detection unit 407 may also issue an alarm inside and outside the excavation area 10 when at least one of abnormal high temperature and abnormal noise and vibration is detected. Alternatively, an alarm may be issued if the sound or vibration inside the excavation area 10 contains a component indicating the possibility of collapse. If no abnormality is detected (NO in step 506), the process returns to step 501, and steps 501 to 506 are repeated to perform the monitoring until the work is completed.

[0047] As described above, the monitoring system according to this embodiment can detect abnormalities in the heavy machinery 100 using a 3D scanner 201, a thermographic camera 202, and, if necessary, a noise / vibration sensor 203. Even in the case of remote control where no operator is on board the heavy machinery, abnormalities in the heavy machinery and its surroundings can be easily and accurately detected. Early detection of such abnormalities and the resulting alarm can prevent damage to the heavy machinery and the occurrence of fires due to overheating. [Explanation of Symbols]

[0048] 10 Excavation Area 11 Excavation surface 100 heavy machinery 101 Boom Arm 102 Hydraulic Cylinder 103 Hydraulic motor 104 Control valve 105 Hydraulic pump 106 Hydraulic oil tank 107 Engine 108 Radiator 109 Exhaust pipe 110 Drive and Control Systems 111 Hydraulic System 112 Imaging Unit 121 Excavation Control Unit 122 Driving Control Unit 130 Communications Department 131 processors 200 trolleys 201 3D Scanner 202 Thermal imaging camera 203 Noise and Vibration Sensor 300 Remote Control Systems 301, 302 Communications Department 303 Control Unit 304 Operation section 305 Display section 306 Program Storage Unit 400 monitoring systems 401 Input section 402 Control Unit 403 Excavation surface shape extraction unit (Excavation area shape extraction unit) 404 Haiku composition area side shape extraction unit (excavation area shape extraction unit) 405 Heavy equipment detection unit 406 Temperature distribution analysis section 407 Abnormality determination section 408 Data Storage Unit 409 Output section 410 Communications Department

Claims

1. A monitoring system for heavy machinery operating in an excavation area, A 3D scanner measures the three-dimensional distance within the excavation area where the heavy machinery is positioned and outputs three-dimensional data including the heavy machinery. A thermographic camera that acquires a temperature distribution image of the excavation area including at least the heavy machinery, A heavy equipment detection unit identifies the shape of the heavy equipment from the three-dimensional data and detects the position information of the heavy equipment within the excavation area, An abnormality determination unit that determines the presence or absence of abnormal areas in and around the heavy machinery based on the shape and position information of the heavy machinery and the temperature distribution image, A monitoring system characterized by having the following features.

2. The monitoring system according to claim 1, characterized in that the abnormality determination unit determines the occurrence of an abnormal area if the temperature exceeds a predetermined threshold in at least one of a plurality of predetermined areas of the heavy machine.

3. The monitoring system according to claim 2, characterized in that the predetermined plurality of areas of the heavy machinery include the hydraulic cylinder of the heavy machinery, a hydraulic oil tank for driving the hydraulic cylinder, an oil pan of the engine which is the power source of the heavy machinery, the radiator and exhaust pipe of the engine, a bearing part of the drive system for moving the heavy machinery, and an electric motor.

4. The system further includes noise and vibration sensors that detect noise and vibration in the excavation area and the heavy machinery, The monitoring system according to any one of claims 1 to 3, characterized in that the abnormality determination unit determines the presence or absence of abnormal areas in and around the heavy machinery based on the temperature distribution image, the excavation area, and the noise and vibration of the heavy machinery.

5. The monitoring system according to any one of claims 1 to 3, characterized in that when the abnormality detection unit determines that an abnormality has occurred, it issues an alarm inside or outside the excavation area 10.

6. The monitoring system according to any one of claims 1 to 3, characterized in that the heavy machinery is remotely controlled by a remote control system.

7. The monitoring system according to claim 6, characterized in that the remote control system remotely operates the heavy machinery according to the position information from the heavy machinery detection unit.