Work site management system and work site management method

The work site management system addresses unattended maintenance issues by identifying and repairing road surface abnormalities, thereby maintaining productivity.

WO2026234884A1PCT designated stage Publication Date: 2026-11-12KOMATSU LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2026/006328
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-07
Filing Date
2026-02-20
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

The productivity of a work site, such as a mine or quarry, can decrease if maintenance-required portions on the road surface are left unaddressed.

Method used

A work site management system that includes a processor to acquire and display video from a camera on a work vehicle, identifying maintenance-required portions and issuing commands for maintenance using a display device and work machines like bulldozers or motor graders.

Benefits of technology

The system effectively suppresses a decrease in productivity by promptly addressing maintenance needs, ensuring timely repair of road surface abnormalities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2026006328_12112026_PF_FP_ABST
    Figure JP2026006328_12112026_PF_FP_ABST
Patent Text Reader

Abstract

This work site management system comprises a processor. The processor acquires images captured by a camera provided to a work vehicle traveling in a work site, and causes a display device to display images including a maintenance-required location which is at the work site and which requires maintenance.
Need to check novelty before this filing date? Find Prior Art

Description

Work site management system and work site management method

[0001] The present disclosure relates to a work site management system and a work site management method.

[0002] As disclosed in Patent Document 1, large mobile bodies and small mobile bodies may travel in mines. The road surface condition is monitored by the small mobile body patrolling the travel area, and portions with large unevenness on the road surface are repaired by a road surface repair dedicated mobile body.

[0003] Japanese Unexamined Patent Application Publication No. 2024-118103

[0004] If a state where a maintenance-required portion requiring maintenance exists at the work site is left unaddressed, the productivity of the work site may decrease.

[0005] The present disclosure aims to suppress a decrease in the productivity of a work site.

[0006] According to the present disclosure, a work site management system is provided. The management system includes a processor. The processor acquires video captured by a camera provided on a work vehicle traveling at the work site, and causes a display device to display the video including a maintenance-required portion that requires maintenance at the work site.

[0007] According to the present disclosure, a decrease in the productivity of a work site is suppressed.

[0008] FIG. 1 is a diagram schematically illustrating a work site according to an embodiment. FIG. 2 is a diagram schematically illustrating the management system according to the embodiment. FIG. 3 is a hardware configuration diagram illustrating a control server according to the embodiment. FIG. 4 is a block diagram illustrating a dump truck according to the embodiment. FIG. 5 is a block diagram illustrating the control server according to the embodiment. FIG. 6 is a diagram for explaining travel data of the dump truck according to the embodiment. FIG. 7 is a diagram for explaining the maintenance-required portion according to the embodiment. FIG. 8 is a diagram illustrating an example of a display device that displays video including the maintenance-required portion according to the embodiment. FIG. 9 is a flowchart illustrating the work site management method according to the embodiment.

[0009] The embodiments of this disclosure will be described below with reference to the drawings, but this disclosure is not limited to these embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

[0010] [Work Site] Figure 1 is a schematic diagram showing a work site 1 according to an embodiment. A mine or quarry is an example of a work site 1. A mine is a place or business establishment where minerals are extracted. A quarry is a place or business establishment where stone materials are extracted. Examples of mines include metal mines where metals are extracted, non-metallic mines where limestone is extracted, and coal mines where coal is extracted.

[0011] At work site 1, a dump truck 2, a type of work vehicle, performs the work. The dump truck 2 may be a manned or unmanned dump truck. A manned dump truck is a dump truck that performs work based on the driver's operation. An unmanned dump truck is a dump truck that performs work without the driver's operation. In this embodiment, the dump truck 2 is an unmanned dump truck. The dump truck 2 has a dump body. The work performed by the dump truck 2 includes transporting the cargo loaded in the dump body and soil removal work to discharge the cargo from the dump body.

[0012] At work site 1, a type of work machine, an excavator 3, a bulldozer 4, and a motor grader 5, will perform their duties. Each of the excavator 3, bulldozer 4, and motor grader 5 has a work implement. The work performed by the excavator 3 includes excavation work, where the work implement is used to excavate the work target, and loading work, where the work implement is used to load cargo onto the dump truck 2. The work performed by the bulldozer 4 includes excavation work, where the work implement is used to excavate the work target, leveling work, where the work implement is used to prepare the terrain of work site 1, and embankment formation work, where the work implement is used to form an embankment. The work performed by the motor grader 5 includes excavation work, where the work implement is used to excavate the work target, and leveling work, where the work implement is used to prepare the terrain of work site 1. The leveling work of the bulldozer 4 and the leveling work of the motor grader 5 include preparation work, where the road surface of work site 1 is prepared. Preparing the road surface of work site 1 includes leveling the road surface.

[0013] Excavator 3 may be a manned or unmanned work machine. A manned work machine is a work machine that performs work based on the operation of an operator. An unmanned work machine is a work machine that performs work without the operation of an operator. A manned work machine may perform work through the operation of an operator who is seated in the driver's cab of the work machine, or it may perform work through remote operation by an operator who is located outside the work machine. Bulldozer 4 and motor grader 5 may each be a manned or unmanned work machine. In this embodiment, excavator 3, bulldozer 4, and motor grader 5 are each manned work machines.

[0014] A light vehicle 6, a type of vehicle, travels within the work site 1. A light vehicle 6 is a lightweight vehicle that travels within the work site 1. The light vehicle 6 travels to inspect the work site 1 and to transport workers. The light vehicle 6 may be a manned vehicle or an unmanned vehicle. A manned vehicle is a vehicle that travels based on the driver's operation. An unmanned vehicle is a vehicle that travels without a driver's operation. In this embodiment, the light vehicle 6 is assumed to be a manned vehicle.

[0015] A loading area 7, a soil removal area 8, and a transport route 9 are provided at the work site 1. The loading area 7, soil removal area 8, and transport route 9 are work areas where a dump truck 2, an excavator 3, a bulldozer 4, a motor grader 5, and a light vehicle 6 can operate, respectively.

[0016] Loading area 7 refers to the work area where loading operations are carried out to load cargo onto dump truck 2. An example of cargo is excavated material excavated in loading area 7. Excavator 3 performs excavation and loading operations in loading area 7.

[0017] The soil removal area 8 is the work area where soil removal operations are carried out, in which the dump truck 2 unloads its cargo. The bulldozer 4 performs excavation, leveling, and embankment formation in the soil removal area 8. The motor grader 5 may also perform excavation and leveling in the soil removal area 8.

[0018] The transport path 9 refers to the road on which the dump truck 2 travels. The transport path 9 leads to the loading area 7 and the soil removal area 8, respectively. The transport path 9 is provided to connect at least the loading area 7 and the soil removal area 8. The dump truck 2 traveling towards at least one of the loading area 7 and the soil removal area 8 travels along the transport path 9. The dump truck 2 travels along the transport path 9, for example, by going back and forth between the loading area 7 and the soil removal area 8. At least one of the bulldozer 4 and the motor grader 5 performs excavation and leveling work on the transport path 9.

[0019] Furthermore, the bulldozer 4 may perform excavation, leveling, and embankment formation work in the loading area 7. The motor grader 5 may also perform excavation and leveling work in the loading area 7.

[0020] [Management System] Figure 2 is a schematic diagram showing the management system 10 according to the embodiment. The management system 10 manages the work site 1. The management system 10 manages the dump truck 2, shovel 3, bulldozer 4, motor grader 5, and light vehicle 6. In Figure 2, the dump truck 2 is shown, but the shovel 3, bulldozer 4, motor grader 5, and light vehicle 6 are not shown. In the following description, for the sake of simplicity, an example in which the management system 10 manages the dump truck 2 will be described.

[0021] The management system 10 comprises a control server 12 and a communication system 13. The control server 12 includes a computer. The control server 12 is located outside the dump truck 2. The control server 12 is installed in the control facility 120 at the work site 1. The control server 12 manages the work site 1. The control server 12 manages the dump truck 2. The communication system 13 includes at least one of the following: the internet, a mobile phone network, a satellite network, and a local area network (LAN).

[0022] The dump truck 2 comprises a body 14, a running gear 15, a dump body 16, and a controller 17. The body 14 is supported by the running gear 15. The running gear 15 supports the body 14 and travels on the road surface of the work site 1. The running gear 15 includes four wheels on which tires are mounted. The running gear 15 is driven by power generated by an engine (not shown). The running gear 15 is braked by the operation of a brake device (not shown). The running gear 15 turns by a steering device (not shown). The dump truck 2 travels on the work site 1 by rotating with the tires of the running gear 15 in contact with the road surface of the work site 1.

[0023] The dump body 16 is the component into which the cargo is loaded. The dump body 16 is supported by the vehicle body 14. The dump body 16 performs dumping and lowering operations. Dumping operation refers to the operation of moving the dump body 16 away from the vehicle body 14 and tilting it in the dumping direction. Lowering operation refers to the operation of moving the dump body 16 closer to the vehicle body 14. When loading operations are performed, the dump body 16 performs a lowering operation. When soil removal operations are performed, the dump body 16 performs a dumping operation.

[0024] The controller 17 includes a computer. The control server 12 and the controller 17 of the dump truck 2 communicate wirelessly via the communication system 13.

[0025] [Computer] Figure 3 is a hardware configuration diagram showing a control server 12 according to an embodiment. The control server 12 includes a computer 30. The computer 30 has a processor 30A such as a CPU (Central Processing Unit), a main memory 30B including non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory), a storage 30C such as a semiconductor disk, an input / output interface 30D including an input / output circuit, and a communication interface 30E including a communication circuit. The functions of the control server 12 are stored in the storage 30C as a computer program 30F. The processor 30A reads the computer program 30F from the storage 30C, loads it into the main memory 30B, and executes processing according to the computer program 30F. The computer program 30F may be distributed to the control server 12 via a network.

[0026] Similar to the control server 12, the controller 17 of the dump truck 2 also includes a computer 30. The controller 17 of the dump truck 2 also includes a processor 30A, main memory 30B, storage 30C for storing computer programs 30F, input / output interface 30D, and communication interface 30E.

[0027] Figure 4 is a block diagram showing a dump truck 2 according to an embodiment. The dump truck 2 includes a controller 17, a driving device 15, a dump body 16, a position sensor 31, a compass sensor 32, a speed sensor 33, an obstacle sensor 34, and a camera 35.

[0028] The position sensor 31 detects the position of the dump truck 2. The position sensor 31 is located on the body 14 of the dump truck 2. The position sensor 31 detects the position of the dump truck 2 using the Global Navigation Satellite System (GNSS). The Global Navigation Satellite System includes the Global Positioning System (GPS). The Global Navigation Satellite System detects the position in a global coordinate system defined by latitude, longitude, and altitude coordinate data. A global coordinate system refers to a coordinate system fixed to the Earth. The position sensor 31 includes a GNSS receiver located on the body 14. The position sensor 31 detects the position of the dump truck 2 in the global coordinate system.

[0029] The orientation sensor 32 detects the orientation of the dump truck 2. The orientation sensor 32 is positioned on the body 14 of the dump truck 2. The orientation of the dump truck 2 includes the orientation angle relative to a reference orientation. The reference orientation is, for example, north. An inertial measurement unit (IMU) is exemplified as the orientation sensor 32. The orientation sensor 32 may also include a calculator that calculates the orientation from position data detected by two GNSS antennas installed on the dump truck 2. The calculator can calculate the orientation from a vector connecting the two GNSS antennas. The orientation sensor 32 may also detect the tilt angle of the dump truck 2 with respect to the horizontal plane. If the orientation sensor 32 is an inertial sensor (IMU), it can detect the tilt angle of the dump truck 2 with respect to the horizontal plane.

[0030] The speed sensor 33 detects the travel speed of the dump truck 2. The speed sensor 33 detects the travel speed of the dump truck 2 by, for example, detecting the rotational speed of the drive shaft connected to the wheels of the running gear 15.

[0031] The obstacle sensor 34 detects objects around the dump truck 2 non-contact. As shown in Figure 2, the obstacle sensor 34 is positioned at the front of the dump truck 2's body 14. The obstacle sensor 34 may also be positioned at the rear of the body 14 or at the side of the body 14. Multiple obstacle sensors 34 may be positioned on the body 14. The obstacle sensor 34 detects objects by emitting energy waves. Examples of obstacle sensors 34 include a laser sensor (LiDAR: Light Detection and Ranging) that detects objects by emitting laser light, a radar sensor (RADAR: Radio Detection and Ranging) that detects objects by emitting radio waves, and an ultrasonic sensor that detects objects by emitting ultrasonic waves. The obstacle sensor 34 can detect the presence or absence of an object, its relative position (relative distance and direction), and its relative velocity. The obstacle sensor 34 may also include a three-dimensional sensor that detects the three-dimensional shape of an object.

[0032] The obstacle sensor 34 detects the condition of the road surface at least at the work site 1 where the dump truck 2 is traveling. The condition of the road surface includes the presence or absence of abnormal parts 72 on the road surface. Abnormal parts 72 on the road surface include ruts and potholes. Abnormal parts 72 on the road surface include protrusions or recesses on the road surface. Protrusions on the road surface include obstacles. Fallen rocks are an example of an obstacle. Abnormal parts 72 include parts of an embankment that has collapsed beside the road surface.

[0033] The obstacle sensor 34 detects abnormal sections 72 on the road surface in the path of the dump truck 2. Based on the detection data from the obstacle sensor 34, the controller 17 stops the dump truck 2 from moving. If the obstacle sensor 34 detects an abnormal section 72 on the road surface while the dump truck 2 is moving, the controller 17 outputs a control command to stop the dump truck 2. For example, if the obstacle sensor 34 detects an obstacle, the controller 17 stops the dump truck 2. Once the dump truck 2 has stopped, the controller 17 sends a stop signal to the control server 12 indicating that the dump truck 2 has stopped. The controller 17 may also send stop position data indicating the stopping position of the dump truck 2 to the control server 12 based on the detection data from the position sensor 31 when the dump truck 2 has stopped.

[0034] Camera 35 captures images of objects around the dump truck 2. Camera 35 acquires video footage. As shown in Figure 2, camera 35 is positioned at the front of the dump truck 2's body 14. Camera 35 may also be positioned at the rear of the body 14 or at the side of the body 14. Multiple cameras 35 may be positioned on the body 14. An RGB camera is given as an example of camera 35. Camera 35 may also be an RGB-D camera or a hyperspectral camera. Camera 35 captures images of at least the road surface of the work site 1 where the dump truck 2 is traveling.

[0035] The detection area of ​​the obstacle sensor 34 and at least a portion of the imaging area of ​​the camera 35 overlap. The object detected by the obstacle sensor 34 and the object captured by the camera 35 are the same.

[0036] The processor 30A of the controller 17 includes a driving control unit 17A and a video transmission unit 17B.

[0037] The driving control unit 17A outputs control commands to control the driving equipment 15 of the dump truck 2. As will be described later, driving data for the dump truck 2 is generated in the control server 12. The driving control unit 17A controls the driving equipment 15 so that the dump truck 2 drives according to the driving data.

[0038] The video transmission unit 17B transmits video images captured by a camera 35 installed on a dump truck 2 traveling through the work site 1 to the control server 12. The video transmission unit 17B transmits real-time video images captured by the camera 35 to the control server 12.

[0039] Figure 5 is a block diagram showing a control server 12 according to an embodiment. A display device 68 and an input device 69 are connected to the control server 12. The display device 68 and the input device 69 are located in the control facility 120. The display device 68 provides display data to an administrator located in the control facility 120. The display device 68 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED). The input device 69 generates input data when operated by an administrator located in the control facility 120. Examples of input devices 69 include a touch panel, buttons, and a computer keyboard.

[0040] The processor 30A of the control server 12 includes a driving data generation unit 12A, a video acquisition unit 12B, a detection unit 12C, a display control unit 12D, an input data acquisition unit 12E, and a maintenance command unit 12F. The storage 30C of the control server 12 includes a video storage unit 12G.

[0041] The driving data generation unit 12A generates driving data indicating the driving conditions of the dump truck 2. The driving conditions of the dump truck 2 include the target position of the dump truck 2, the target direction of the dump truck 2, and the target driving speed of the dump truck 2. The driving data generation unit 12A transmits the generated driving data to the controller 17 of the dump truck 2. The dump truck 2 drives around the work site 1 based on the driving data generated by the driving data generation unit 12A.

[0042] The video acquisition unit 12B acquires video (real-time video) captured by the camera 35 installed on the dump truck 2. The video acquisition unit 12B receives video (real-time video) transmitted from the video transmission unit 17B.

[0043] The detection unit 12C detects a maintenance-required part that requires maintenance at the work site 1 based on the video acquired by the video acquisition unit 12B. The maintenance-required part includes an abnormal portion 72 of the road surface. The detection unit 12C can detect the maintenance-required part by performing image processing on the video captured by the camera 35. The detection unit 12C can calculate the abnormality level of the abnormal portion 72 by performing image processing on the video. When the abnormal portion 72 is a convex portion on the road surface, the detection unit 12C can calculate the height of the convex portion as the abnormality level by performing image processing on the video. When the abnormal portion 72 is a concave portion on the road surface, the detection unit 12C can calculate the depth of the concave portion as the abnormality level by performing image processing on the video. When the abnormality level of the abnormal portion 72 is equal to or higher than a predetermined threshold, the detection unit 12C determines that the abnormal portion 72 is a maintenance-required part. When the abnormality level of the abnormal portion 72 is less than the threshold, the detection unit 12C determines that the abnormal portion 72 is not a maintenance-required part.

[0044] The display control unit 12D generates display data to be displayed on the display device 68. The display control unit 12D causes the display device 68 to display the generated display data. The display control unit 12D causes the display device 68 to display the video acquired by the video acquisition unit 12B. The display control unit 12D causes the display device 68 to display a video including a maintenance-required part that requires maintenance at the work site 1. When the detection unit 12C detects a maintenance-required part, the display control unit 12D causes the display device 68 to display a video including the maintenance-required part.

[0045] The input data acquisition unit 12E acquires, from the input device 69, input data generated by operation of the input device 69.

[0046] When a maintenance-required part that requires maintenance exists on the road surface of the work site 1 where the dump truck 2 travels, the maintenance command unit 12F outputs a maintenance command that causes a work machine capable of performing maintenance to perform maintenance on the maintenance-required part. Examples of the work machine capable of performing maintenance include a bulldozer 4 and a motor grader 5. The maintenance command unit 12F outputs a maintenance command for causing at least one of the bulldozer 4 and the motor grader 5 to perform maintenance on the maintenance-required part.

[0047] The video storage unit 12G stores the video acquired by the video acquisition unit 12B. The video storage unit 12G stores video including a site requiring maintenance. The video storage unit 12G stores video of a part of the road surface including the site requiring maintenance. The video storage unit 12G stores the position of the site requiring maintenance.

[0048] [Travel Data] FIG. 6 is a diagram for explaining travel data of the dump truck 2 according to the embodiment. In the embodiment, the dump truck 2 is an unmanned dump truck that travels on the road surface of the work site 1 in accordance with the travel data. The travel data of the dump truck 2 defines travel conditions of the dump truck 2. The travel data of the dump truck 2 includes travel points 70, a travel path 71, a target position of the dump truck 2, a target azimuth of the dump truck 2, a target travel speed of the dump truck 2, and a target turning radius of the dump truck 2.

[0049] A plurality of the travel points 70 are set on the road surface of the work site 1. The travel points 70 define the target position of the dump truck 2. A target azimuth of the dump truck 2 and a target travel speed of the dump truck 2 are set for each of the plurality of travel points 70. The plurality of travel points 70 are set at intervals. The intervals of the travel points 70 may be uniform or non-uniform.

[0050] The target position of the dump truck 2 refers to the target position of the dump truck 2 when passing through the travel point 70. The target position of the dump truck 2 may be defined in the local coordinate system of the dump truck 2, or may be defined in the global coordinate system. The target azimuth of the dump truck 2 refers to the target azimuth of the dump truck 2 when passing through the travel point 70. The target travel speed of the dump truck 2 refers to the target travel speed of the dump truck 2 when passing through the travel point 70.

[0051] The travel path 71 refers to a virtual line indicating a target travel route of the dump truck 2. The travel path 71 is defined by a trajectory passing through the plurality of travel points 70. The dump truck 2 travels on the work site 1 in accordance with the travel path 71. The dump truck 2 travels such that the center of the dump truck 2 coincides with the travel path 71 in the vehicle width direction of the dump truck 2.

[0052] The travel control unit 17A of the controller 17 controls the travel device 15 so that the dump truck 2 travels according to the travel path 71, based on the detection data from the position sensor 31 and the detection data from the orientation sensor 32. Specifically, the travel control unit 17A controls the travel device 15 so that the deviation between the detected position of the dump truck 2 detected by the position sensor 31 and the target position of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70. The travel control unit 17A controls the travel device 15 so that the deviation between the detected orientation of the dump truck 2 detected by the orientation sensor 32 and the target orientation of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70. Specifically, the travel control unit 17A controls the travel device 15 so that the dump truck 2 travels at the target travel speed, based on the detection data from the speed sensor 33. Specifically, the travel control unit 17A controls the travel device 15 so that the deviation between the detected travel speed of the dump truck 2 detected by the speed sensor 33 and the target travel speed of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70.

[0053] In this embodiment, the video acquired by the video acquisition unit 12B is the video captured by the camera 35 when the dump truck 2, which is driving according to the driving data, is in motion.

[0054] [Parts requiring maintenance] Figure 7 is a diagram illustrating parts requiring maintenance according to the embodiment. In this embodiment, parts requiring maintenance include abnormal parts 72 that obstruct the movement of the dump truck 2. As shown in Figure 7, abnormal parts 72, such as protrusions or depressions, may be formed on the road surface. These abnormal parts 72 may make it difficult for the dump truck 2 to continue moving on the road. The camera 35 captures an image of a portion of the road surface including the abnormal parts 72, which are parts requiring maintenance. The video transmission unit 17B transmits the video (real-time video) including the abnormal parts 72, which are parts requiring maintenance, captured by the camera 35, to the control server 12.

[0055] Furthermore, if the obstacle sensor 34 detects an abnormal part 72 while the dump truck 2 is in motion, the driving control unit 17A of the controller 17 will stop the dump truck 2. If the dump truck 2 is stopped, the driving control unit 17A may send a stop signal to the control server 12 indicating that the dump truck 2 has stopped.

[0056] The video transmission unit 17B can calculate the position of the abnormal part 72 in the global coordinate system based on the position of the dump truck 2 detected by the position sensor 31 and the relative position between the dump truck 2 and the abnormal part 72 detected by the obstacle sensor 34. If the abnormal part 72 is a part that requires maintenance, the position of the part that requires maintenance can be calculated from the position of the abnormal part 72. The video transmission unit 17B transmits position data indicating the position of the part that requires maintenance, along with the video including the part that requires maintenance, to the control server 12. The video including the part that requires maintenance and the position of the part that requires maintenance are stored in the video storage unit 12G.

[0057] The video transmission unit 17B may also transmit video to the control server 12 if the administrator of the control facility 120 requests video transmission. When the administrator requests the transmission of video captured by the camera 35, they operate the input device 69. The input data acquisition unit 12E transmits a video request signal to the controller 17 based on the input data from the input device 69. The video transmission unit 17B may transmit video (real-time video) to the control server 12 based on the video request signal.

[0058] Furthermore, if the abnormal part 72 is detected by the obstacle sensor 34 and a stop signal indicating that the dump truck 2 has stopped is transmitted from the controller 17 to the control server 12, the display control unit 12D may display stop notification data on the display device 68 indicating that the abnormal part 72 has been detected and the dump truck 2 has stopped. The administrator of the control facility 120 can recognize that the dump truck 2 has stopped based on the stop notification data displayed on the display device 68. After recognizing that the dump truck 2 has stopped, the administrator may operate the input device 69 so that a video request signal is transmitted to the controller 17.

[0059] The video acquisition unit 12B receives video (real-time video) transmitted from the video transmission unit 17B. The video includes parts that require maintenance. The video includes images of the area around the dump truck 2, which has stopped because an abnormal part 72 was detected by the obstacle sensor 34. The display control unit 12D displays the video (real-time video) including the parts that require maintenance on the display device 68.

[0060] [Display Device] Figure 8 is a diagram showing an example of a display device 68 that displays an image including the part requiring maintenance according to the embodiment. As shown in Figure 8, an image (real-time image) including the abnormal part 72 captured by the camera 35 is displayed on the display device 68. The abnormal part 72 is the part requiring maintenance. The administrator can recognize the existence of the part requiring maintenance by checking the image displayed on the display device 68.

[0061] As described above, the detection unit 12C can detect the part requiring maintenance (abnormal part 72) by processing the image captured by the camera 35. The display control unit 12D can display a symbol image 90 around at least a portion of the part requiring maintenance detected by the detection unit 12C. The display control unit 12D uses the symbol image 90 to highlight the part requiring maintenance (abnormal part 72). In the example shown in Figure 8, the symbol image 90 is a frame image surrounding the part requiring maintenance (abnormal part 72). At least a portion of the symbol image 90 may be superimposed on the part requiring maintenance (abnormal part 72).

[0062] As described above, the position of the part requiring maintenance (abnormal part 72) in the global coordinate system is calculated based on the detection data from the position sensor 31 and the obstacle sensor 34 of the dump truck 2. The display control unit 12D can display the symbol image 90 on at least a part of the area around the part requiring maintenance (abnormal part 72) or superimpose the symbol image 90 on the part requiring maintenance (abnormal part 72) based on the position of the part requiring maintenance (abnormal part 72).

[0063] The detection unit 12C may calculate the size of the part requiring maintenance (abnormal part 72) based on the detection data of the obstacle sensor 34 of the dump truck 2. The obstacle sensor 34 can detect the three-dimensional shape of the abnormal part 72. The detection unit 12C can calculate the size of the abnormal part 72 based on the detection data of the obstacle sensor 34. The size of the abnormal part 72 includes the dimensions of the abnormal part 72 in three-dimensional space. If the abnormal part 72 is a protrusion on the road surface, the detection unit 12C can calculate the height of the protrusion as the dimension of the abnormal part 72 based on the detection data of the obstacle sensor 34. If the abnormal part 72 is a recess on the road surface, the detection unit 12C can calculate the depth of the recess as the dimension of the abnormal part 72 based on the detection data of the obstacle sensor 34. The display control unit 12D may display dimensional data indicating the size of the abnormal part 72 on the display device 68. The display control unit 12D may display character data indicating the size of the part requiring maintenance (abnormal part 72) as dimensional data around the part requiring maintenance (abnormal part 72) on the display screen of the display device 68.

[0064] When the administrator confirms the video footage including the part requiring maintenance (abnormal part 72), they operate the input device 69 to have the bulldozer 4 or motor grader 5 perform maintenance on the part requiring maintenance. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. The maintenance command unit 12F outputs a maintenance command to the bulldozer 4 or motor grader 5 to perform maintenance on the part requiring maintenance, based on the input data. If the bulldozer 4 and motor grader 5 are manned machines, the maintenance command is displayed on a display device installed in the operator's cab of the manned machine. The operator of the manned machine operates the machine to perform maintenance on the abnormal part 72 based on the maintenance command displayed on the display device. If the bulldozer 4 or motor grader 5 are unmanned machines, the controller of the unmanned machine outputs a control command to operate the unmanned machine to perform maintenance on the abnormal part 72, based on the maintenance command.

[0065] Repairing the abnormal part 72 (part requiring maintenance) includes removing the abnormal part 72. Repairing the abnormal part 72 includes leveling the road surface on which the abnormal part 72 is located. If the abnormal part 72 is a protrusion on the road surface, repairing the abnormal part 72 includes removing the protrusion and leveling the road surface. If the abnormal part 72 is a recess on the road surface, repairing the abnormal part 72 includes filling the recess with soil and leveling the road surface.

[0066] After maintenance by the bulldozer 4 or motor grader 5 is completed, the manager can check whether the abnormal part 72 has been removed by looking at the image displayed on the display device 68.

[0067] If the dump truck 2 stops due to the presence of the abnormal part 72, the administrator may operate the input device 69 to restart the dump truck 2. The dump truck 2 may also restart based on the input data from the input device 69.

[0068] The control server 12 may also determine whether or not to perform maintenance on the part requiring maintenance (abnormal part 72). As described above, the detection unit 12C can calculate the abnormality level of the abnormal part 72 by processing the video captured by the camera 35. The detection unit 12C can determine that maintenance is necessary if the abnormality level of the abnormal part 72 is above a predetermined threshold. If the abnormality level of the abnormal part 72 is below the threshold, it can determine that maintenance is not necessary.

[0069] The detection unit 12C may calculate the abnormality level of the abnormal portion 72 based on the detection data from the obstacle sensor 34. The obstacle sensor 34 can detect the three-dimensional shape of the abnormal portion 72. Based on the detection data from the obstacle sensor 34, the detection unit 12C can calculate the height of the protrusion or the depth of the recess as an abnormality level.

[0070] If the detection unit 12C determines that maintenance is necessary, the maintenance command unit 12F can output a maintenance command to at least one of the bulldozer 4 and the motor grader 5. After maintenance by the bulldozer 4 or the motor grader 5 is completed, the detection unit 12C can determine whether or not the abnormal part 72 has disappeared by processing the video captured by the camera 35. Alternatively, after maintenance is completed, the detection unit 12C may determine whether or not the abnormal part 72 has disappeared based on the detection data from the obstacle sensor 34.

[0071] [Management Method] Figure 9 is a flowchart showing the management method for the work site 1 according to the embodiment. In the following description, an example will be described in which the control server 12 determines whether or not there are parts that require maintenance.

[0072] The administrator of the control facility 120 operates the input device 69 to request the controller 17 to transmit the video captured by the camera 35. Based on the input data from the input device 69, the input data acquisition unit 12E sends a video request signal to the controller 17 requesting the transmission of the video (step S1).

[0073] The video transmission unit 17B transmits the video (real-time video) captured by the camera 35 to the control server 12 based on the video request signal. The video acquisition unit 12B receives the video transmitted from the video transmission unit 17B. The display control unit 12D displays the video acquired by the video acquisition unit 12B on the display device 68 (step S2).

[0074] The detection unit 12C calculates the abnormality level of the abnormal portion 72 based on the video acquired by the video acquisition unit 12B. Based on the abnormality level of the abnormal portion 72, the detection unit 12C determines whether or not there are parts that require maintenance (step S3).

[0075] If it is determined in step S3 that there are parts that require maintenance (step S3: Yes), the maintenance command unit 12F outputs a maintenance command to at least one of the bulldozer 4 and the motor grader 5 to perform maintenance on the parts that require maintenance (step S4).

[0076] If it is determined in step S3 that there are no parts requiring maintenance (step S3: No), no maintenance command is issued.

[0077] The video storage unit 12G stores the video (step S5). If there is a part that needs maintenance, the video storage unit 12G stores the video of the part that needs maintenance and the area around it. If there is a part that needs maintenance, the video storage unit 12G stores the location of the part that needs maintenance.

[0078] [Effects] As described above, the processor 30A of the control server 12 includes an image acquisition unit 12B that acquires images captured by a camera 35 installed on a dump truck 2 traveling through the work site 1, and a display control unit 12D that displays images including the parts requiring maintenance at the work site 1 on a display device 68.

[0079] According to this embodiment, if there are parts that require maintenance, an image including those parts is displayed on the display device 68. This allows for prompt maintenance of the parts that require maintenance. However, since the parts that require maintenance are not left unattended for a long period of time, the productivity of the work site 1 may decrease.

[0080] [Other Embodiments] In the above embodiment, when a part requiring maintenance (abnormal part 72) is identified, a maintenance command is output from the maintenance command unit 12F to have the bulldozer 4 or motor grader 5 perform maintenance on the part requiring maintenance. The maintenance command unit 12F may also output a confirmation command to have the driver of the light vehicle 6 confirm the condition of the area around the parked dump truck 2. The administrator operates the input device 69 to have the driver of the light vehicle 6 confirm the part requiring maintenance. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the maintenance command unit 12F outputs a confirmation command to have the driver of the light vehicle 6 confirm the part requiring maintenance. The confirmation command is displayed on the display device provided in the driver's cab of the light vehicle 6. Based on the confirmation command displayed on the display device, the driver of the light vehicle 6 can operate the light vehicle 6 to drive to the part requiring maintenance.

[0081] In the above-described embodiment, the display device 68 for displaying images may be provided on at least one of the bulldozer 4, motor grader 5, and light vehicle 6. The operator of the bulldozer 4 or motor grader 5 can start maintenance on the abnormal part without waiting for a maintenance command from the maintenance command unit 12F by checking the image displayed on the display device 68. The driver of the light vehicle 6 can drive the light vehicle 6 towards the part requiring maintenance in order to check the part requiring maintenance by checking the image displayed on the display device 68.

[0082] In the above embodiment, at least a portion of the functional parts of the control server 12 may be provided in the controller 17. At least a portion of the functional parts of the controller 17 may be provided in the control server 12.

[0083] In the above-described embodiment, each of the multiple functional units of the control server 12 may be configured by a separate computer (hardware). Each of the multiple functional units of the controller 17 may be configured by a separate computer (hardware).

[0084] 1...Work site, 2...Dump truck (work vehicle), 3...Excavator (work machine), 4...Bulldozer (work machine), 5...Motor grader (work machine), 6...Light vehicle (vehicle), 7...Loading area, 8...Soil removal area, 9...Transport route, 10...Management system, 12...Control server, 12A...Driving data generation unit, 12B...Video acquisition unit, 12C...Detection unit, 12D...Display control unit, 12E...Input data acquisition unit, 12F...Maintenance command unit, 12G...Video storage unit, 120...Control facility, 13...Communication system, 14...Vehicle body, 15... 16...Dump body, 17...Controller, 17A...Travel control unit, 17B...Video transmission unit, 30...Computer, 30A...Processor, 30B...Main memory, 30C...Storage, 30D...Input / output interface, 30E...Communication interface, 30F...Computer program, 31...Position sensor, 32...Directional sensor, 33...Speed ​​sensor, 34...Obstacle sensor, 35...Camera, 68...Display device, 69...Input device, 70...Travel point, 71...Travel path, 72...Abnormal area, 90...Symbol image.

Claims

1. A work site management system comprising a processor, the processor acquiring images captured by a camera installed on a work vehicle traveling at the work site, and displaying the images, including parts requiring maintenance at the work site, on a display device.

2. The work site management system according to claim 1, wherein the processor detects the part requiring maintenance based on the video, and when the part requiring maintenance is detected, the processor causes the video including the part requiring maintenance to be displayed on the display device.

3. The work site management system according to claim 2, wherein the processor causes a symbolic image to be displayed around at least a portion of the part requiring maintenance.

4. The work site management system according to claim 1, wherein the processor outputs a maintenance command to perform maintenance on the parts that require maintenance.

5. The work site management system according to claim 1, wherein the processor generates driving data of the work vehicle, and the video is captured by the camera when the work vehicle is driving according to the driving data.

6. The work vehicle is an unmanned dump truck that operates according to the driving data, as described in claim 5.

7. A method for managing a work site, comprising: acquiring images captured by a camera installed on a work vehicle traveling through the work site; and displaying the images, including parts requiring maintenance at the work site, on a display device.

8. A method for managing a work site according to claim 7, comprising detecting the part requiring maintenance based on the video, and if the part requiring maintenance is detected, displaying the video including the part requiring maintenance on the display device.

9. The work site management method according to claim 8, wherein a symbolic image is displayed around at least a portion of the area requiring maintenance.

10. A method for managing a work site according to claim 7, comprising outputting a maintenance command to perform maintenance on the aforementioned parts requiring maintenance.

11. A method for managing a work site according to claim 7, comprising generating driving data of the work vehicle, wherein the video is captured by the camera when the work vehicle is driving according to the driving data.

12. The work vehicle is an unmanned dump truck that travels according to the travel data, the method for managing a work site according to claim 11.