Work site management system and work site management method
The work site management system addresses the issue of decreased productivity by displaying real-time and stored vehicle footage to facilitate quick decision-making on resuming travel or maintenance, thereby minimizing downtime.
Patent Information
- Application Number
- PCT/JP2026/006575
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2026-02-24
- Publication Date
- 2026-11-12
AI Technical Summary
When a work vehicle encounters an event that prevents it from continuing to travel at a work site, such as an obstacle on the road, it often stops, leading to a decrease in productivity due to the vehicle being left unattended.
A work site management system that includes a processor to acquire real-time and stored video from a camera on the vehicle, allowing for the display of this information on a display device to facilitate prompt decision-making for resuming travel or initiating maintenance on the road surface.
This system enables quick action to address the obstruction, minimizing downtime and maintaining productivity by allowing for immediate restart of the vehicle or maintenance of the road surface.
Smart Images

Figure JP2026006575_12112026_PF_FP_ABST
Abstract
Description
Work site management system and work site management method
[0001] The present disclosure relates to a work site management system and a work site management method.
[0002] In the technical field related to work site management systems, a work site management system as disclosed in Patent Document 1 is known.
[0003] Japanese Unexamined Patent Application Publication No. 2019-116890
[0004] When an event that makes it difficult for a work vehicle traveling at a work site to continue traveling occurs, the work vehicle stops. If the stopped state of the work vehicle is left unattended, the productivity of the work site may decrease.
[0005] The present disclosure aims to suppress a decrease in the productivity of a work site.
[0006] According to the present disclosure, a work site management system is provided. The management system includes a processor. The processor acquires real-time video captured by a camera provided on a work vehicle traveling at a work site, acquires stored video captured by the camera during a retrospective time period traced back from the point in time when it is determined that an event making it difficult for the work vehicle to continue traveling has occurred, and causes a display device to display the real-time video and the stored video.
[0007] According to the present disclosure, a decrease in the productivity of a work site is suppressed.
[0008] FIG. 1 is a diagram schematically illustrating a work site according to an embodiment. FIG. 2 is a diagram schematically illustrating the management system according to the embodiment. FIG. 3 is a hardware configuration diagram illustrating a control server according to the embodiment. FIG. 4 is a block diagram illustrating a dump truck according to the embodiment. FIG. 5 is a block diagram illustrating the control server according to the embodiment. FIG. 6 is a diagram for explaining travel data of the dump truck according to the embodiment. FIG. 7 is a diagram for explaining an event that makes it difficult for the dump truck according to the embodiment to continue traveling. FIG. 8 is a diagram illustrating an example of a display device that displays real-time video and stored video according to the embodiment. FIG. 9 is a flowchart illustrating the work site management method according to the embodiment.
[0009] The embodiments of this disclosure will be described below with reference to the drawings, but this disclosure is not limited to these embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.
[0010] [Work Site] Figure 1 is a schematic diagram showing a work site 1 according to an embodiment. A mine or quarry is an example of a work site 1. A mine is a place or business establishment where minerals are extracted. A quarry is a place or business establishment where stone materials are extracted. Examples of mines include metal mines where metals are extracted, non-metallic mines where limestone is extracted, and coal mines where coal is extracted.
[0011] At work site 1, a dump truck 2, a type of work vehicle, performs the work. The dump truck 2 may be a manned or unmanned dump truck. A manned dump truck is a dump truck that performs work based on the driver's operation. An unmanned dump truck is a dump truck that performs work without the driver's operation. In this embodiment, the dump truck 2 is an unmanned dump truck. The dump truck 2 has a dump body. The work performed by the dump truck 2 includes transporting the cargo loaded in the dump body and soil removal work to discharge the cargo from the dump body.
[0012] At work site 1, a type of work machine, an excavator 3, a bulldozer 4, and a motor grader 5, will perform their duties. Each of the excavator 3, bulldozer 4, and motor grader 5 has a work implement. The work performed by the excavator 3 includes excavation work, where the work implement is used to excavate the work target, and loading work, where the work implement is used to load cargo onto the dump truck 2. The work performed by the bulldozer 4 includes excavation work, where the work implement is used to excavate the work target, leveling work, where the work implement is used to prepare the terrain of work site 1, and embankment formation work, where the work implement is used to form an embankment. The work performed by the motor grader 5 includes excavation work, where the work implement is used to excavate the work target, and leveling work, where the work implement is used to prepare the terrain of work site 1. The leveling work of the bulldozer 4 and the leveling work of the motor grader 5 include preparation work, where the road surface of work site 1 is prepared. Preparing the road surface of work site 1 includes leveling the road surface.
[0013] Excavator 3 may be a manned or unmanned work machine. A manned work machine is a work machine that performs work based on the operation of an operator. An unmanned work machine is a work machine that performs work without the operation of an operator. A manned work machine may perform work through the operation of an operator who is seated in the driver's cab of the work machine, or it may perform work through remote operation by an operator who is located outside the work machine. Bulldozer 4 and motor grader 5 may each be a manned or unmanned work machine. In this embodiment, excavator 3, bulldozer 4, and motor grader 5 are each manned work machines.
[0014] A light vehicle 6, a type of vehicle, travels within the work site 1. A light vehicle 6 is a lightweight vehicle that travels within the work site 1. The light vehicle 6 travels to inspect the work site 1 and to transport workers. The light vehicle 6 may be a manned vehicle or an unmanned vehicle. A manned vehicle is a vehicle that travels based on the driver's operation. An unmanned vehicle is a vehicle that travels without a driver's operation. In this embodiment, the light vehicle 6 is assumed to be a manned vehicle.
[0015] A loading area 7, a soil removal area 8, and a transport route 9 are provided at the work site 1. The loading area 7, soil removal area 8, and transport route 9 are work areas where a dump truck 2, an excavator 3, a bulldozer 4, a motor grader 5, and a light vehicle 6 can operate, respectively.
[0016] Loading area 7 refers to the work area where loading operations are carried out to load cargo onto dump truck 2. An example of cargo is excavated material excavated in loading area 7. Excavator 3 performs excavation and loading operations in loading area 7.
[0017] The soil removal area 8 is the work area where soil removal operations are carried out, in which the dump truck 2 unloads its cargo. The bulldozer 4 performs excavation, leveling, and embankment formation in the soil removal area 8. The motor grader 5 may also perform excavation and leveling in the soil removal area 8.
[0018] The transport path 9 refers to the road on which the dump truck 2 travels. The transport path 9 leads to the loading area 7 and the soil removal area 8, respectively. The transport path 9 is provided to connect at least the loading area 7 and the soil removal area 8. The dump truck 2 traveling towards at least one of the loading area 7 and the soil removal area 8 travels along the transport path 9. The dump truck 2 travels along the transport path 9, for example, by going back and forth between the loading area 7 and the soil removal area 8. At least one of the bulldozer 4 and the motor grader 5 performs excavation and leveling work on the transport path 9.
[0019] Furthermore, the bulldozer 4 may perform excavation, leveling, and embankment formation work in the loading area 7. The motor grader 5 may also perform excavation and leveling work in the loading area 7.
[0020] [Management System] Figure 2 is a schematic diagram showing the management system 10 according to the embodiment. The management system 10 manages the work site 1. The management system 10 manages the dump truck 2, shovel 3, bulldozer 4, motor grader 5, and light vehicle 6. In Figure 2, the dump truck 2 is shown, but the shovel 3, bulldozer 4, motor grader 5, and light vehicle 6 are not shown. In the following description, for the sake of simplicity, an example in which the management system 10 manages the dump truck 2 will be described.
[0021] The management system 10 comprises a control server 12 and a communication system 13. The control server 12 includes a computer. The control server 12 is located outside the dump truck 2. The control server 12 is installed in the control facility 120 at the work site 1. The control server 12 manages the work site 1. The control server 12 manages the dump truck 2. The communication system 13 includes at least one of the following: the internet, a mobile phone network, a satellite network, and a local area network (LAN).
[0022] The dump truck 2 comprises a body 14, a running gear 15, a dump body 16, and a controller 17. The body 14 is supported by the running gear 15. The running gear 15 supports the body 14 and travels on the road surface of the work site 1. The running gear 15 includes four wheels on which tires are mounted. The running gear 15 is driven by power generated by an engine (not shown). The running gear 15 is braked by the operation of a brake device (not shown). The running gear 15 turns by a steering device (not shown). The dump truck 2 travels on the work site 1 by rotating with the tires of the running gear 15 in contact with the road surface of the work site 1.
[0023] The dump body 16 is the component into which the cargo is loaded. The dump body 16 is supported by the vehicle body 14. The dump body 16 performs dumping and lowering operations. Dumping operation refers to the operation of moving the dump body 16 away from the vehicle body 14 and tilting it in the dumping direction. Lowering operation refers to the operation of moving the dump body 16 closer to the vehicle body 14. When loading operations are performed, the dump body 16 performs a lowering operation. When soil removal operations are performed, the dump body 16 performs a dumping operation.
[0024] The controller 17 includes a computer. The control server 12 and the controller 17 of the dump truck 2 communicate wirelessly via the communication system 13.
[0025] [Computer] Figure 3 is a hardware configuration diagram showing a control server 12 according to an embodiment. The control server 12 includes a computer 30. The computer 30 has a processor 30A such as a CPU (Central Processing Unit), a main memory 30B including non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory), a storage 30C such as a semiconductor disk, an input / output interface 30D including an input / output circuit, and a communication interface 30E including a communication circuit. The functions of the control server 12 are stored in the storage 30C as a computer program 30F. The processor 30A reads the computer program 30F from the storage 30C, loads it into the main memory 30B, and executes processing according to the computer program 30F. The computer program 30F may be distributed to the control server 12 via a network.
[0026] Similar to the control server 12, the controller 17 of the dump truck 2 also includes a computer 30. The controller 17 of the dump truck 2 also includes a processor 30A, main memory 30B, storage 30C for storing computer programs 30F, input / output interface 30D, and communication interface 30E.
[0027] Figure 4 is a block diagram showing a dump truck 2 according to an embodiment. The dump truck 2 includes a controller 17, a driving device 15, a dump body 16, a position sensor 31, a compass sensor 32, a speed sensor 33, an obstacle sensor 34, and a camera 35.
[0028] The position sensor 31 detects the position of the dump truck 2. The position sensor 31 is located on the body 14 of the dump truck 2. The position sensor 31 detects the position of the dump truck 2 using the Global Navigation Satellite System (GNSS). The Global Navigation Satellite System includes the Global Positioning System (GPS). The Global Navigation Satellite System detects the position in a global coordinate system defined by latitude, longitude, and altitude coordinate data. A global coordinate system refers to a coordinate system fixed to the Earth. The position sensor 31 includes a GNSS receiver located on the body 14. The position sensor 31 detects the position of the dump truck 2 in the global coordinate system.
[0029] The orientation sensor 32 detects the orientation of the dump truck 2. The orientation sensor 32 is positioned on the body 14 of the dump truck 2. The orientation of the dump truck 2 includes the orientation angle relative to a reference orientation. The reference orientation is, for example, north. An inertial measurement unit (IMU) is exemplified as the orientation sensor 32. The orientation sensor 32 may also include a calculator that calculates the orientation from position data detected by two GNSS antennas installed on the dump truck 2. The calculator can calculate the orientation from a vector connecting the two GNSS antennas. The orientation sensor 32 may also detect the tilt angle of the dump truck 2 with respect to the horizontal plane. If the orientation sensor 32 is an inertial sensor (IMU), it can detect the tilt angle of the dump truck 2 with respect to the horizontal plane.
[0030] The speed sensor 33 detects the travel speed of the dump truck 2. The speed sensor 33 detects the travel speed of the dump truck 2 by, for example, detecting the rotational speed of the drive shaft connected to the wheels of the running gear 15.
[0031] The obstacle sensor 34 detects objects around the dump truck 2 non-contact. As shown in Figure 2, the obstacle sensor 34 is positioned at the front of the dump truck 2's body 14. The obstacle sensor 34 may also be positioned at the rear of the body 14 or at the side of the body 14. Multiple obstacle sensors 34 may be positioned on the body 14. The obstacle sensor 34 detects objects by emitting energy waves. Examples of obstacle sensors 34 include a laser sensor (LiDAR: Light Detection and Ranging) that detects objects by emitting laser light, a radar sensor (RADAR: Radio Detection and Ranging) that detects objects by emitting radio waves, and an ultrasonic sensor that detects objects by emitting ultrasonic waves. The obstacle sensor 34 can detect the presence or absence of an object, its relative position (relative distance and direction), and its relative velocity. The obstacle sensor 34 may also include a three-dimensional sensor that detects the three-dimensional shape of an object.
[0032] The obstacle sensor 34 detects the condition of the road surface at least at the work site 1 where the dump truck 2 is traveling. The condition of the road surface includes the presence or absence of abnormal parts 72 on the road surface. Abnormal parts 72 on the road surface include ruts and potholes. Abnormal parts 72 on the road surface include protrusions or recesses on the road surface. Protrusions on the road surface include obstacles. Fallen rocks are an example of an obstacle. Abnormal parts 72 include parts of an embankment that has collapsed beside the road surface.
[0033] The obstacle sensor 34 detects abnormal sections 72 on the road surface in the path of the dump truck 2. Based on the detection data from the obstacle sensor 34, the controller 17 stops the dump truck 2 from moving. If the obstacle sensor 34 detects an abnormal section 72 on the road surface while the dump truck 2 is moving, the controller 17 outputs a control command to stop the dump truck 2. For example, if the obstacle sensor 34 detects an obstacle, the controller 17 stops the dump truck 2. Once the dump truck 2 has stopped, the controller 17 sends a stop signal to the control server 12 indicating that the dump truck 2 has stopped. The controller 17 may also send stop position data indicating the stopping position of the dump truck 2 to the control server 12 based on the detection data from the position sensor 31 when the dump truck 2 has stopped.
[0034] Camera 35 captures images of objects around the dump truck 2. Camera 35 acquires video footage. As shown in Figure 2, camera 35 is positioned at the front of the dump truck 2's body 14. Camera 35 may also be positioned at the rear of the body 14 or at the side of the body 14. Multiple cameras 35 may be positioned on the body 14. An RGB camera is given as an example of camera 35. Camera 35 may also be an RGB-D camera or a hyperspectral camera. Camera 35 captures images of at least the road surface of the work site 1 where the dump truck 2 is traveling.
[0035] The detection area of the obstacle sensor 34 and at least a portion of the imaging area of the camera 35 overlap. The object detected by the obstacle sensor 34 and the object captured by the camera 35 are the same.
[0036] The processor 30A of the controller 17 includes a driving control unit 17A, a video writing unit 17B, a real-time video transmission unit 17C, and a stored video transmission unit 17D. The storage 30C of the controller 17 includes a video storage unit 17E.
[0037] The driving control unit 17A outputs control commands to control the driving equipment 15 of the dump truck 2. As will be described later, driving data for the dump truck 2 is generated in the control server 12. The driving control unit 17A controls the driving equipment 15 so that the dump truck 2 drives according to the driving data.
[0038] The video writing unit 17B stores the video captured by the camera 35 in the video storage unit 17E. In this embodiment, the dump truck 2 has a drive recorder system. The controller 17 has a drive recorder function. The video writing unit 17B stores the saved video, which is the video captured by the camera 35 between a first time point t1 and a second time point t2 that is set back by a retrospective time Ta from the first time point t1, in the video storage unit 17E. The second time point t2 is a time point earlier than the first time point t1. The saved video is the video captured by the camera 35 at a retrospective time Ta set back from the first time point t1.
[0039] The real-time video transmission unit 17C transmits real-time video captured by a camera 35 installed on a dump truck 2 traveling through the work site 1 to the control server 12.
[0040] The saved video transmission unit 17D transmits the saved video stored in the video storage unit 17E to the control server 12.
[0041] Figure 5 is a block diagram showing a control server 12 according to an embodiment. A display device 68 and an input device 69 are connected to the control server 12. The display device 68 and the input device 69 are located in the control facility 120. The display device 68 provides display data to an administrator located in the control facility 120. The display device 68 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED). The input device 69 generates input data when operated by an administrator located in the control facility 120. Examples of input devices 69 include a touch panel, buttons, and a computer keyboard.
[0042] The processor 30A of the control server 12 includes a travel data generation unit 12A, a real-time video acquisition unit 12B, a stored video acquisition unit 12C, a display control unit 12D, an input data acquisition unit 12E, a determination unit 12F, a restart unit 12G, and a maintenance instruction unit 12H.
[0043] The travel data generation unit 12A generates travel data indicating the travel conditions of the dump truck 2. The travel conditions of the dump truck 2 include the target position of the dump truck 2, the target azimuth of the dump truck 2, and the target travel speed of the dump truck 2. The travel data generation unit 12A transmits the generated travel data to the controller 17 of the dump truck 2. The dump truck 2 travels on the work site 1 based on the travel data generated by the travel data generation unit 12A.
[0044] The real-time video acquisition unit 12B acquires real-time video captured by the camera 35 provided on the dump truck 2. The real-time video acquisition unit 12B receives the real-time video transmitted from the real-time video transmission unit 17C.
[0045] The stored video acquisition unit 12C acquires stored video captured by the camera 35 during the retroactive time Ta. The stored video acquisition unit 12C receives the stored video transmitted from the stored video transmission unit 17D.
[0046] The display control unit 12D generates display data to be displayed on the display device 68. The display control unit 12D causes the display device 68 to display the generated display data. In the embodiment, the display control unit 12D causes the display device 68 to display at least the real-time video and the stored video.
[0047] The input data acquisition unit 12E acquires input data generated by operating the input device 69 from the input device 69.
[0048] When the dump truck 2 is stopped, the determination unit 12F determines whether or not it is possible to resume the travel of the stopped dump truck 2 based on at least one of the real-time video and the stored video.
[0049] If the determination unit 12F determines that it is possible to resume driving, the restart unit 12G outputs a restart command to resume driving the dump truck 2. Alternatively, the restart unit 12G may output a restart command to resume driving the dump truck 2 based on the input data acquired by the input data acquisition unit 12E.
[0050] The maintenance command unit 12H outputs a maintenance command to a work machine capable of performing maintenance if there are parts requiring maintenance on the road surface of the work site 1 where the dump truck 2 is traveling. Examples of work machines capable of performing maintenance include a bulldozer 4 and a motor grader 5. The maintenance command unit 12H outputs a maintenance command to at least one of the bulldozer 4 and the motor grader 5 to perform maintenance on the parts requiring maintenance.
[0051] [Driving Data] Figure 6 is a diagram illustrating the driving data of a dump truck 2 according to an embodiment. In this embodiment, the dump truck 2 is an unmanned dump truck that travels on the road surface of the work site 1 according to the driving data. The driving data of the dump truck 2 defines the driving conditions of the dump truck 2. The driving data of the dump truck 2 includes the driving point 70, the driving path 71, the target position of the dump truck 2, the target direction of the dump truck 2, the target driving speed of the dump truck 2, and the target turning radius of the dump truck 2.
[0052] Multiple travel points 70 are set on the road surface of the work site 1. Each travel point 70 defines the target position of the dump truck 2. For each of the multiple travel points 70, the target direction and target travel speed of the dump truck 2 are set. The multiple travel points 70 are set at intervals. The intervals between the travel points 70 may be uniform or uneven.
[0053] The target position of dump truck 2 refers to the target position of dump truck 2 when it passes through point 70. The target position of dump truck 2 may be defined in the dump truck 2's local coordinate system or in the global coordinate system. The target bearing of dump truck 2 refers to the target bearing of dump truck 2 when it passes through point 70. The target driving speed of dump truck 2 refers to the target driving speed of dump truck 2 when it passes through point 70.
[0054] The driving path 71 refers to a virtual line indicating the target driving route of the dump truck 2. The driving path 71 is defined by a trajectory that passes through multiple driving points 70. The dump truck 2 travels around the work site 1 according to the driving path 71. The dump truck 2 travels such that the center of the dump truck 2 and the driving path 71 coincide in the width direction of the dump truck 2.
[0055] The travel control unit 17A of the controller 17 controls the travel device 15 so that the dump truck 2 travels according to the travel path 71, based on the detection data from the position sensor 31 and the detection data from the orientation sensor 32. Specifically, the travel control unit 17A controls the travel device 15 so that the deviation between the detected position of the dump truck 2 detected by the position sensor 31 and the target position of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70. The travel control unit 17A controls the travel device 15 so that the deviation between the detected orientation of the dump truck 2 detected by the orientation sensor 32 and the target orientation of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70. Specifically, the travel control unit 17A controls the travel device 15 so that the dump truck 2 travels at the target travel speed, based on the detection data from the speed sensor 33. Specifically, the travel control unit 17A controls the travel device 15 so that the deviation between the detected travel speed of the dump truck 2 detected by the speed sensor 33 and the target travel speed of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70.
[0056] [Events that make it difficult to continue driving] Figure 7 is a diagram illustrating events that make it difficult for the dump truck 2 to continue driving according to the embodiment. In the embodiment, if an event occurs that makes it difficult for the dump truck 2 to continue driving, the driving control unit 17A of the controller 17 stops the dump truck 2. In the following description, events that make it difficult for the dump truck 2 to continue driving will be appropriately referred to as abnormal events.
[0057] An abnormal event includes an abnormality in the driving condition of the dump truck 2. In this embodiment, an abnormal event includes the detection of an abnormal portion 72 of the road surface at the work site 1 where the dump truck 2 is driving by the obstacle sensor 34.
[0058] As shown in Figure 7, abnormal parts 72, such as protrusions or depressions, may be generated on the road surface. These abnormal parts 72 may make it difficult for the dump truck 2 to continue driving on the road. The obstacle sensor 34 of the dump truck 2 detects the abnormal parts 72. When the obstacle sensor 34 detects an abnormal part 72 while the dump truck 2 is driving, the driving control unit 17A of the controller 17 determines that an abnormal event has occurred. If the driving control unit 17A determines that an abnormal event has occurred, it stops the dump truck 2. When the dump truck 2 is stopped, the driving control unit 17A sends a stop signal to the control server 12 indicating that the dump truck 2 has stopped. Alternatively, the driving control unit 17A may send stop position data indicating the stopping position of the dump truck 2 to the control server 12 based on the detection data of the position sensor 31 when the dump truck 2 has stopped.
[0059] If the driving control unit 17A determines that an abnormal event has occurred, the video writing unit 17B stores the video captured by the camera 35 as saved video in the video storage unit 17E during a retrospective time Ta that goes back from the first time point t1 in which the abnormal event was determined to have occurred.
[0060] In this embodiment, the first time point t1 at which an abnormal event is determined to have occurred is the time when the dump truck 2 stopped due to the abnormal event. The video storage unit 17E stores the video footage of the retrospective time Ta, which is set back from the first time point t1 when the dump truck 2 stopped, as saved video footage.
[0061] The first time point t1 at which it is determined that an abnormal event has occurred may also be the time when braking of the running gear 15 of the dump truck 2 begins due to the abnormal event (the time when the brake device begins to operate). The first time point t1 at which it is determined that an abnormal event has occurred may also be the time when the obstacle sensor 34 detects the abnormal part 72.
[0062] The real-time video transmission unit 17C transmits real-time video to the control server 12. If it is determined that an abnormal event has occurred and saved video is stored in the video storage unit 17E, the saved video transmission unit 17D transmits the saved video stored in the video storage unit 17E to the control server 12.
[0063] Furthermore, the real-time video transmission unit 17C may transmit real-time video to the control server 12 if the administrator of the control facility 120 requests the transmission of real-time video. The stored video transmission unit 17D may transmit stored video to the control server 12 if the administrator of the control facility 120 requests the transmission of stored video. When the administrator requests the transmission of at least one of the real-time video and stored video, they operate the input device 69. The input data acquisition unit 12E transmits a video request signal to the controller 17 based on the input data from the input device 69. The real-time video transmission unit 17C may transmit real-time video to the control server 12 based on the video request signal. The stored video transmission unit 17D may transmit stored video to the control server 12 based on the video request signal.
[0064] Furthermore, if an abnormal event occurs and a stop signal indicating that dump truck 2 has stopped is transmitted from controller 17 to control server 12, display control unit 12D may display stop notification data indicating that dump truck 2 has stopped on display device 68. The administrator of control facility 120 can recognize that dump truck 2 has stopped based on the stop notification data displayed on display device 68. After recognizing that dump truck 2 has stopped, the administrator may operate input device 69 so that a video request signal is transmitted to controller 17.
[0065] The real-time video acquisition unit 12B receives real-time video transmitted from the real-time video transmission unit 17C. The saved video acquisition unit 12C receives saved video transmitted from the saved video transmission unit 17D. Both the real-time video and the saved video include images of the area around the dump truck 2 that has stopped due to an abnormal event. The display control unit 12D displays the real-time video and saved video of the area around the dump truck 2 that has stopped due to an abnormal event on the display device 68.
[0066] [Display Device] Figure 8 is a diagram showing an example of a display device 68 that displays real-time video 91 and saved video 92 according to the embodiment. As shown in Figure 8, the real-time video 91 and saved video 92 are displayed in parallel on the display screen of the display device 68. Both the real-time video 91 and the saved video 92 are moving images. The saved video 92 is played in parallel with the real-time video 91. By checking the real-time video 91 and saved video 92 displayed on the display device 68, the administrator can recognize the situation around the dump truck 2 that has stopped due to an abnormal event.
[0067] After the administrator recognizes the situation around the dump truck 2 that has stopped due to an abnormal event, they take measures to restart the dump truck 2. For example, if the administrator recognizes that the abnormal part 72 is an obstacle and it is difficult to restart the dump truck 2, the administrator operates the input device 69 to have the bulldozer 4 or motor grader 5 service the abnormal part 72. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the maintenance command unit 12H outputs a maintenance command to the bulldozer 4 or motor grader 5 to service the abnormal part 72. If the bulldozer 4 and motor grader 5 are manned work machines, the maintenance command is displayed on a display device installed in the operator's cab of the manned work machine. Based on the maintenance command displayed on the display device, the operator of the manned work machine operates the manned work machine so that the abnormal part 72 is serviced. If the bulldozer 4 or motor grader 5 is an unmanned work machine, the controller of the unmanned work machine outputs a control command to operate the unmanned work machine so that the abnormal part 72 is maintained, based on the maintenance command.
[0068] Repairing the abnormal portion 72 includes removing the abnormal portion 72. Repairing the abnormal portion 72 includes leveling the road surface on which the abnormal portion 72 is located. If the abnormal portion 72 is a protrusion on the road surface, repairing the abnormal portion 72 includes removing the protrusion and leveling the road surface. If the abnormal portion 72 is a recess on the road surface, repairing the abnormal portion 72 includes filling the recess with soil and leveling the road surface.
[0069] After maintenance by the bulldozer 4 or motor grader 5 is completed, the manager can check whether the abnormal part 72 has disappeared by looking at the real-time video 91 displayed on the display device 68. Once the manager has confirmed that the abnormal part 72 has disappeared, the manager operates the input device 69 to restart the dump truck 2. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the restart unit 12G transmits a restart command to the controller 17 of the dump truck 2 to restart the dump truck 2, which has stopped due to the abnormal event. The driving control unit 17A of the controller 17 restarts driving based on the restart command from the restart unit 12G. The dump truck 2 drives based on the driving data.
[0070] In addition, the dump truck 2 may stop due to a detection error by the obstacle sensor 34. The dump truck 2 may stop if there is no abnormal part 72 or if the abnormal part 72 is minor. The administrator, after checking the real-time video 91 and saved video 92 displayed on the display device 68, may immediately restart the stopped dump truck 2. If the administrator decides that it is OK to immediately restart the stopped dump truck 2, the administrator operates the input device 69 so that the dump truck 2 can start moving again. Based on the input data, the restart unit 12G transmits a restart command to the controller 17 of the dump truck 2 to restart the stopped dump truck 2.
[0071] The control server 12 may determine whether or not to perform maintenance on the abnormal part 72. The control server 12 may also determine whether or not to resume driving the dump truck 2. The determination unit 12F can calculate the abnormality level of the abnormal part 72 by image processing of at least one of the real-time video and the saved video. If the abnormal part 72 is a protrusion on the road surface, the determination unit 12F can calculate the height of the protrusion as the abnormality level by image processing of at least one of the real-time video and the saved video. If the abnormal part 72 is a recess on the road surface, the determination unit 12F can calculate the depth of the recess as the abnormality level by image processing of at least one of the real-time video and the saved video. If the abnormality level of the abnormal part 72 is above a predetermined threshold, the determination unit 12F can determine that maintenance is necessary. If the abnormality level of the abnormal part 72 is below the threshold, the determination unit 12F can determine that maintenance is not necessary.
[0072] The determination unit 12F may calculate the abnormality level of the abnormal portion 72 based on the detection data from the obstacle sensor 34. The obstacle sensor 34 can detect the three-dimensional shape of the abnormal portion 72. Based on the detection data from the obstacle sensor 34, the determination unit 12F can calculate the height of the protrusion or the depth of the recess as an abnormality level.
[0073] If the determination unit 12F determines that maintenance is necessary, the maintenance command unit 12H can output a maintenance command to at least one of the bulldozer 4 and the motor grader 5. After maintenance by the bulldozer 4 or the motor grader 5 is completed, the determination unit 12F can determine whether or not the abnormal part 72 has disappeared by processing the real-time video 91. Alternatively, after maintenance is completed, the determination unit 12F may determine whether or not the abnormal part 72 has disappeared based on the detection data from the obstacle sensor 34.
[0074] The determination unit 12F determines that the abnormality level of the abnormal part 72 is below a threshold, or that the abnormal part 72 has been eliminated by maintenance, and determines that the stationary dump truck 2 can be restarted. The restart unit 12G outputs a restart command to restart the dump truck 2 if it determines that it is possible to restart the dump truck 2.
[0075] [Management Method] Figure 9 is a flowchart showing the management method for the work site 1 according to the embodiment. In the following description, an example will be described in which the control server 12 determines whether or not to perform maintenance on the abnormal part 72 and whether or not to resume driving the dump truck 2.
[0076] When the obstacle sensor 34 of the dump truck 2 traveling on the road surface of the work site 1 detects an abnormal part 72, the driving control unit 17A determines that an abnormal event has occurred and stops the dump truck 2. The driving control unit 17A transmits a stop signal to the control server 12 indicating that the dump truck 2 has stopped. The display control unit 12D receives the stop signal indicating that the dump truck 2 has stopped (step S1).
[0077] When the display control unit 12D receives a stop signal, it displays stop notification data on the display device 68 indicating that the dump truck 2 has stopped. The administrator of the control facility 120 can recognize that the dump truck 2 has stopped based on the stop notification data displayed on the display device 68. After recognizing that the dump truck 2 has stopped, the administrator operates the input device 69 to request the controller 17 to transmit real-time video and saved video. The input data acquisition unit 12E transmits a video request signal to the controller 17 requesting the transmission of real-time video and saved video based on the input data from the input device 69 (step S2).
[0078] The real-time video transmission unit 17C transmits real-time video to the control server 12 based on the video request signal. The stored video transmission unit 17D transmits stored video to the control server 12 based on the video request signal. The real-time video acquisition unit 12B receives the real-time video transmitted from the real-time video transmission unit 17C. The stored video acquisition unit 12C receives the stored video transmitted from the stored video transmission unit 17D. Both the real-time video and the stored video include images of the area around the dump truck 2 that has stopped due to an abnormal event. The display control unit 12D displays the real-time video and stored video of the area around the dump truck 2 that has stopped due to an abnormal event on the display device 68 (step S3).
[0079] The determination unit 12F calculates the abnormality level of the abnormal portion 72 based on at least one of the real-time video and the saved video. Based on the abnormality level of the abnormal portion 72, the determination unit 12F determines whether or not it is possible to resume driving the dump truck 2 (step S4).
[0080] In step S4, if it is determined that it is possible to resume driving the dump truck 2 (step S4: Yes), the restart unit 12G outputs a restart command to resume driving the dump truck 2 (step S5).
[0081] If, in step S4, it is determined that it is not possible to resume driving the dump truck 2 (step S4: No), the maintenance command unit 12H outputs a maintenance command to at least one of the bulldozer 4 and the motor grader 5 to repair the abnormal part 72 (step S6).
[0082] [Effects] As described above, the processor 30A of the control server 12 includes a real-time video acquisition unit 12B that acquires real-time video images captured by a camera 35 installed on a dump truck 2 traveling through the work site 1, a saved video acquisition unit 12C that acquires saved video images captured by the camera 35 during a retrospective time Ta, starting from a first time point t1 when it is determined that an abnormal event has occurred that makes it difficult for the dump truck 2 to continue traveling, and a display control unit 12D that displays the real-time video and saved video on a display device 68.
[0083] According to the embodiment, if an abnormal event occurs and the dump truck 2 stops, real-time video and saved video of the area around the dump truck 2 that has stopped due to the abnormal event are displayed on the display device 68. This allows for prompt action to be taken to restart the dump truck 2. Since the dump truck 2 is not left stopped for a long period of time, the decrease in productivity at the work site 1 is suppressed.
[0084] [Other Embodiments] In the embodiments described above, the abnormal event may be an event that makes it difficult for the dump truck 2 to continue driving according to the driving data. The abnormal event may include the dump truck 2 deviating from the driving path 71. For example, if the dump truck 2 deviates from the driving path 71, the driving control unit 17A stops the dump truck 2. When the dump truck 2 deviates from the driving path 71 and stops, real-time video and saved video of the area around the stopped dump truck 2 may be displayed on the display device 68.
[0085] In the above embodiment, if the dump truck 2 stops due to an abnormal part 72, the maintenance command unit 12H outputs a maintenance command to have the bulldozer 4 or motor grader 5 service the abnormal part 72. The maintenance command unit 12H may also output a confirmation command to have the driver of the light vehicle 6 confirm the situation around the stopped dump truck 2. The administrator operates the input device 69 to have the driver of the light vehicle 6 confirm the abnormal part 72. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the maintenance command unit 12H outputs a confirmation command to have the driver of the light vehicle 6 confirm the abnormal part 72. The confirmation command is displayed on the display device provided in the driver's cab of the light vehicle 6. Based on the confirmation command displayed on the display device, the driver of the light vehicle 6 can operate the light vehicle 6 to drive to the abnormal part 72.
[0086] In the above-described embodiment, the display device 68 for displaying real-time and saved images may be provided on at least one of the bulldozer 4, motor grader 5, and light vehicle 6. The operator of the bulldozer 4 or motor grader 5 can start maintenance on the abnormal part 72 without waiting for a maintenance command from the maintenance command unit 12H by checking the real-time and saved images displayed on the display device 68. The driver of the light vehicle 6 can drive the light vehicle 6 towards the abnormal part 72 to check it by checking the real-time and saved images displayed on the display device 68.
[0087] In the above embodiment, at least a portion of the functional parts of the control server 12 may be provided in the controller 17. At least a portion of the functional parts of the controller 17 may be provided in the control server 12.
[0088] In the above-described embodiment, each of the multiple functional units of the control server 12 may be configured by a separate computer (hardware). Each of the multiple functional units of the controller 17 may be configured by a separate computer (hardware).
[0089] 1...Work site, 2...Dump truck (work vehicle), 3...Excavator (work machine), 4...Bulldozer (work machine), 5...Motor grader (work machine), 6...Light vehicle (driving vehicle), 7...Loading area, 8...Soil removal area, 9...Transport route, 10...Management system, 12...Control server, 12A...Driving data generation unit, 12B...Real-time video acquisition unit, 12C...Stored video acquisition unit, 12D...Display control unit, 12E...Input data acquisition unit, 12F...Determination unit, 12G...Restart unit, 12H...Maintenance command unit, 120...Control facility, 13...Communication system, 14...Vehicle body, 15...Driving gear, 16...Dump body 17...Controller, 17A...Driving control unit, 17B...Video writing unit, 17C...Real-time video transmission unit, 17D...Stored video transmission unit, 17E...Video storage unit, 30...Computer, 30A...Processor, 30B...Main memory, 30C...Storage, 30D...Input / output interface, 30E...Communication interface, 30F...Computer program, 31...Position sensor, 32...Direction sensor, 33...Speed sensor, 34...Obstacle sensor, 35...Camera, 68...Display device, 69...Input device, 70...Driving point, 71...Driving path, 72...Anomaly section, 91...Real-time video, 92...Stored video.
Claims
1. A work site management system comprising a processor, the processor acquires real-time video images captured by a camera installed on a work vehicle traveling at a work site, acquires saved video images captured by the camera during a retrospective period starting from the time when it is determined that an event has occurred that makes it difficult for the work vehicle to continue traveling, and displays the real-time video and the saved video on a display device.
2. The work site management system according to claim 1, wherein the aforementioned time is the time when the work vehicle stopped due to the aforementioned event.
3. The work site management system according to claim 1, wherein each of the real-time video and the stored video includes video of the area around the work vehicle that stopped due to the event.
4. The work site management system according to claim 1, wherein the work vehicle is provided with an obstacle sensor, and the event includes the detection of an abnormal portion of the road surface on which the work vehicle is traveling by the obstacle sensor.
5. The work site management system according to claim 1, wherein the processor generates driving data of the work vehicle, and the event is an event that makes it difficult to continue driving according to the driving data.
6. The work vehicle is an unmanned dump truck that operates according to the driving data, as described in claim 5.
7. The work site management system according to claim 1, wherein the processor acquires input data from an input device and outputs a restart command based on the input data to resume the movement of the work vehicle that has stopped due to the event.
8. The work site management system according to claim 1, wherein the processor determines, based on at least one of the real-time video and the stored video, whether or not it is possible to resume driving the work vehicle that has stopped due to the event, and if it is determined that it is possible to resume driving, outputs a restart command to resume driving the work vehicle.
9. A method for managing a work site, comprising: acquiring real-time video footage captured by a camera installed on a work vehicle traveling through the work site; acquiring saved video footage captured by the camera during a retrospective period starting from the time it is determined that an event has occurred that makes it difficult for the work vehicle to continue traveling; and displaying the real-time video footage and the saved video footage on a display device.
10. The method for managing a work site according to claim 9, wherein the aforementioned time is the time when the work vehicle stopped due to the aforementioned event.
11. The method for managing a work site according to claim 9, wherein each of the real-time video and the saved video includes video of the area around the work vehicle that stopped due to the event.
12. The method for managing a work site according to claim 9, wherein the work vehicle is provided with an obstacle sensor, and the event includes the detection of an abnormal portion of the road surface on which the work vehicle is traveling by the obstacle sensor.
13. A method for managing a work site according to claim 9, comprising generating driving data of the work vehicle, wherein the event is an event that makes it difficult to continue driving in accordance with the driving data.
14. The work vehicle is an unmanned dump truck that travels according to the travel data, the method for managing a work site according to claim 13.
15. A method for managing a work site according to claim 9, comprising: acquiring input data from an input device; and outputting a restart command based on the input data to resume the movement of the work vehicle that has stopped due to the event.
16. A method for managing a work site according to claim 9, comprising: determining whether it is possible to resume driving the work vehicle that has stopped due to the event, based on at least one of the real-time video and the stored video; and, if it is determined that it is possible to resume driving, outputting a restart command to resume driving the work vehicle.
