Work site management system and work site management method
The worksite management system addresses productivity loss by displaying real-time and stored vehicle footage to facilitate swift decision-making for resuming operations, thus reducing downtime.
Patent Information
- Application Number
- JP2025077572
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2026-11-19
AI Technical Summary
When a work vehicle encounters an event that makes it difficult to continue traveling at a worksite, the stopped state can lead to a decrease in productivity.
A worksite management system that includes a processor to acquire real-time and stored video from a camera on the work vehicle, allowing for the display of this information to facilitate prompt decision-making for resuming operations.
This approach helps suppress the decline in productivity by enabling quick assessment and action to restart the work vehicle, minimizing downtime.
Smart Images

Figure 2026181840000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a worksite management system and a worksite management method.
Background Art
[0002] In the technical field related to worksite management systems, a worksite management system as disclosed in Patent Document 1 is known.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] When an event that makes it difficult for a work vehicle traveling at a worksite to continue traveling occurs, the work vehicle stops. If the stopped state of the work vehicle is left unaddressed, the productivity of the worksite may decrease.
[0005] The present disclosure aims to suppress a decrease in the productivity of a worksite.
Means for Solving the Problems
[0006] According to the present disclosure, a worksite management system is provided. The management system includes a processor. The processor acquires real-time video captured by a camera provided on a work vehicle traveling at a worksite, acquires stored video captured by the camera during a retroactive time period traced back from the point in time when it is determined that an event making it difficult for the work vehicle to continue traveling has occurred, and causes a display device to display the real-time video and the stored video.
Effects of the Invention
[0007] According to this disclosure, the decline in productivity at the work site will be suppressed. [Brief explanation of the drawing]
[0008] [Figure 1] Figure 1 is a schematic diagram showing a work site according to the embodiment. [Figure 2] Figure 2 is a schematic diagram showing the management system according to the embodiment. [Figure 3] Figure 3 is a hardware configuration diagram showing a control server according to an embodiment. [Figure 4] Figure 4 is a block diagram of a dump truck according to an embodiment. [Figure 5] Figure 5 is a block diagram showing a control server according to an embodiment. [Figure 6] Figure 6 is a diagram illustrating the driving data of a dump truck according to the embodiment. [Figure 7] Figure 7 is a diagram illustrating a situation in which it becomes difficult for the dump truck according to the embodiment to continue driving. [Figure 8] Figure 8 shows an example of a display device that shows real-time video and stored video according to the embodiment. [Figure 9] Figure 9 is a flowchart showing a work site management method according to an embodiment. [Modes for carrying out the invention]
[0009] The embodiments of this disclosure will be described below with reference to the drawings, but this disclosure is not limited to these embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.
[0010] [Work site] Figure 1 is a schematic diagram showing a work site 1 according to an embodiment. Examples of work site 1 include mines or quarries. A mine refers to a place or business establishment where minerals are extracted. A quarry refers to a place or business establishment where stone materials are extracted. Examples of mines include metal mines where metals are extracted, non-metallic mines where limestone is extracted, and coal mines where coal is extracted.
[0011] At work site 1, a dump truck 2, a type of work vehicle, performs the work. Dump truck 2 may be a manned dump truck or an unmanned dump truck. A manned dump truck is a dump truck that performs work based on the driver's operation. An unmanned dump truck is a dump truck that performs work without the driver's operation. In this embodiment, dump truck 2 is an unmanned dump truck. Dump truck 2 has a dump body. The work performed by dump truck 2 includes transporting the cargo loaded in the dump body and soil removal work to discharge the cargo from the dump body.
[0012] At work site 1, a type of work machine, an excavator 3, a bulldozer 4, and a motor grader 5, will perform their duties. Each of the excavator 3, bulldozer 4, and motor grader 5 has an implement. The work performed by the excavator 3 includes excavation work, where the work machine is used to excavate the work target, and loading work, where the work machine is used to load cargo onto the dump truck 2. The work performed by the bulldozer 4 includes excavation work, where the work machine is used to excavate the work target, leveling work, where the work machine is used to prepare the terrain of work site 1, and embankment formation work, where the work machine is used to form an embankment. The work performed by the motor grader 5 includes excavation work, where the work machine is used to excavate the work target, and leveling work, where the work machine is used to prepare the terrain of work site 1. The leveling work by the bulldozer 4 and the leveling work by the motor grader 5 include preparation work, where the road surface of work site 1 is prepared. Preparing the road surface of work site 1 includes leveling the road surface.
[0013] Shovel 3 may be a manned work machine or an unmanned work machine. A manned work machine refers to a work machine that performs work based on the driving operation of an operator. An unmanned work machine refers to a work machine that performs work unmanned without relying on the driving operation of an operator. The manned work machine may perform work by the driving operation of an operator who is aboard the cab of the work machine, or may perform work by the remote operation of an operator located outside the work machine. Each of bulldozer 4 and motor grader 5 may also be a manned work machine or an unmanned work machine. In the embodiment, each of shovel 3, bulldozer 4, and motor grader 5 is assumed to be a manned work machine.
[0014] At work site 1, a light vehicle 6, which is a type of traveling vehicle, travels. The light vehicle 6 refers to a lightweight vehicle that travels on work site 1. The light vehicle 6 travels to patrol work site 1 or transport workers. The light vehicle 6 may be a manned vehicle or an unmanned vehicle. A manned vehicle refers to a vehicle that travels based on the driving operation of a driver. An unmanned vehicle refers to a vehicle that travels unmanned without relying on the driving operation of a driver. In the embodiment, the light vehicle 6 is assumed to be a manned vehicle.
[0015] Work site 1 is provided with a loading area 7, an earth dumping area 8, and a conveyance path 9. The loading area 7, the earth dumping area 8, and the conveyance path 9 are work areas in which each of dump truck 2, shovel 3, bulldozer 4, motor grader 5, and light vehicle 6 can perform work.
[0016] The loading area 7 refers to a work area where a loading operation for loading a load onto dump truck 2 is performed. As the load, an excavated material excavated in the loading area 7 is exemplified. Shovel 3 performs excavation work and loading work in loading area 7.
[0017] The earth dumping area 8 refers to a work area where the dumping operation in which the dump truck 2 discharges the load is performed. The bulldozer 4 performs excavation work, ground leveling work, and bank forming work in the earth dumping area 8. The motor grader 5 may perform excavation work and ground leveling work in the earth dumping area 8.
[0018] The transport path 9 refers to a traveling path on which the dump truck 2 travels. The transport path 9 leads to each of the loading area 7 and the earth dumping area 8. The transport path 9 is provided so as to connect at least the loading area 7 and the earth dumping area 8. The dump truck 2 heading toward at least one of the loading area 7 and the earth dumping area 8 travels on the transport path 9. The dump truck 2 travels on the transport path 9 so as to reciprocate between, for example, the loading area 7 and the earth dumping area 8. At least one of the bulldozer 4 and the motor grader 5 performs excavation work and ground leveling work on the transport path 9.
[0019] Note that the bulldozer 4 may perform excavation work, ground leveling work, and bank forming work in the loading area 7. The motor grader 5 may perform excavation work and ground leveling work in the loading area 7.
[0020] [Management System] Figure 2 is a diagram schematically illustrating the management system 10 according to the embodiment. The management system 10 manages the work site 1. The management system 10 manages each of the dump truck 2, the excavator 3, the bulldozer 4, the motor grader 5, and the light vehicle 6. In Figure 2, the dump truck 2 is illustrated, and illustration of the excavator 3, the bulldozer 4, the motor grader 5, and the light vehicle 6 is omitted. In the following description, for simplification of the description, an example in which the management system 10 manages the dump truck 2 will be described.
[0021] The management system 10 comprises a control server 12 and a communication system 13. The control server 12 includes a computer. The control server 12 is located outside the dump truck 2. The control server 12 is installed in the control facility 120 at the work site 1. The control server 12 manages the work site 1. The control server 12 manages the dump truck 2. The communication system 13 includes at least one of the following: the internet, a mobile phone network, a satellite network, and a local area network (LAN).
[0022] The dump truck 2 comprises a body 14, a running gear 15, a dump body 16, and a controller 17. The body 14 is supported by the running gear 15. The running gear 15 supports the body 14 and travels on the road surface of the work site 1. The running gear 15 includes four wheels on which tires are mounted. The running gear 15 is driven by power generated by an engine (not shown). The running gear 15 is braked by the operation of a brake device (not shown). The running gear 15 turns by a steering device (not shown). The dump truck 2 travels on the work site 1 by rotating with the tires of the running gear 15 in contact with the road surface of the work site 1.
[0023] The dump body 16 is the component into which the cargo is loaded. The dump body 16 is supported by the vehicle body 14. The dump body 16 performs dumping and lowering operations. Dumping operation refers to the operation of moving the dump body 16 away from the vehicle body 14 and tilting it in the dumping direction. Lowering operation refers to the operation of moving the dump body 16 closer to the vehicle body 14. When loading operations are performed, the dump body 16 performs a lowering operation. When soil removal operations are performed, the dump body 16 performs a dumping operation.
[0024] The controller 17 includes a computer. The control server 12 and the controller 17 of the dump truck 2 communicate wirelessly via the communication system 13.
[0025] [computer] Figure 3 is a hardware configuration diagram showing a control server 12 according to an embodiment. The control server 12 includes a computer 30. The computer 30 has a processor 30A such as a CPU (Central Processing Unit), a main memory 30B including non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory), a storage 30C such as a semiconductor disk, an input / output interface 30D including input / output circuits, and a communication interface 30E including communication circuits. The functions of the control server 12 are stored in the storage 30C as a computer program 30F. The processor 30A reads the computer program 30F from the storage 30C, loads it into the main memory 30B, and executes processing according to the computer program 30F. The computer program 30F may be distributed to the control server 12 via a network.
[0026] Similar to the control server 12, the controller 17 of the dump truck 2 also includes a computer 30. The controller 17 of the dump truck 2 also has a processor 30A, main memory 30B, storage 30C for storing computer programs 30F, an input / output interface 30D, and a communication interface 30E.
[0027] Figure 4 is a block diagram of a dump truck 2 according to an embodiment. The dump truck 2 includes a controller 17, a driving device 15, a dump body 16, a position sensor 31, a compass sensor 32, a speed sensor 33, an obstacle sensor 34, and a camera 35.
[0028] The position sensor 31 detects the position of the dump truck 2. The position sensor 31 is located on the body 14 of the dump truck 2. The position sensor 31 detects the position of the dump truck 2 using the Global Navigation Satellite System (GNSS). The Global Navigation Satellite System includes the Global Positioning System (GPS). The Global Navigation Satellite System detects the position in a global coordinate system defined by latitude, longitude, and altitude coordinate data. A global coordinate system refers to a coordinate system fixed to the Earth. The position sensor 31 includes a GNSS receiver located on the body 14. The position sensor 31 detects the position of the dump truck 2 in the global coordinate system.
[0029] The orientation sensor 32 detects the orientation of the dump truck 2. The orientation sensor 32 is located on the body 14 of the dump truck 2. The orientation of the dump truck 2 includes the orientation angle relative to a reference orientation. The reference orientation is, for example, north. An inertial measurement unit (IMU) is exemplified as the orientation sensor 32. The orientation sensor 32 may also include a calculator that calculates the orientation from position data detected by two GNSS antennas installed on the dump truck 2. The calculator can calculate the orientation from a vector connecting the two GNSS antennas. The orientation sensor 32 may also detect the tilt angle of the dump truck 2 with respect to the horizontal plane. If the orientation sensor 32 is an inertial measurement unit (IMU), it can detect the tilt angle of the dump truck 2 with respect to the horizontal plane.
[0030] The speed sensor 33 detects the travel speed of the dump truck 2. The speed sensor 33 detects the travel speed of the dump truck 2 by, for example, detecting the rotational speed of the drive shaft connected to the wheels of the running gear 15.
[0031] The obstacle sensor 34 detects objects around the dump truck 2 non-contact. As shown in Figure 2, the obstacle sensor 34 is positioned at the front of the dump truck 2's body 14. The obstacle sensor 34 may also be positioned at the rear of the body 14 or at the side of the body 14. Multiple obstacle sensors 34 may be positioned on the body 14. The obstacle sensor 34 detects objects by emitting energy waves. Examples of obstacle sensors 34 include laser sensors (LiDAR: Light Detection and Ranging) that detect objects by emitting laser light, radar sensors (RADAR: Radio Detection and Ranging) that detect objects by emitting radio waves, and ultrasonic sensors (Ultrasonic Sensor) that detect objects by emitting ultrasonic waves. The obstacle sensor 34 can detect the presence or absence of an object, its relative position (relative distance and direction), and its relative velocity. The obstacle sensor 34 may also include a 3D sensor that detects the three-dimensional shape of an object.
[0032] The obstacle sensor 34 detects the condition of the road surface at least at the work site 1 where the dump truck 2 is traveling. The condition of the road surface includes the presence or absence of abnormal parts 72 on the road surface. Abnormal parts 72 on the road surface include ruts and potholes. Abnormal parts 72 on the road surface include protrusions or depressions on the road surface. Protrusions on the road surface include obstacles. Fallen rocks are an example of an obstacle. Abnormal parts 72 include portions where an embankment next to the road surface has collapsed.
[0033] The obstacle sensor 34 detects abnormal sections 72 on the road surface in the path of the dump truck 2. Based on the detection data from the obstacle sensor 34, the controller 17 stops the dump truck 2 from moving. If the obstacle sensor 34 detects an abnormal section 72 on the road surface while the dump truck 2 is moving, the controller 17 outputs a control command to stop the dump truck 2. For example, if the obstacle sensor 34 detects an obstacle, the controller 17 stops the dump truck 2. Once the dump truck 2 has stopped, the controller 17 sends a stop signal to the control server 12 indicating that the dump truck 2 has stopped. The controller 17 may also send stop position data indicating the stopping position of the dump truck 2 to the control server 12 based on the detection data from the position sensor 31 when the dump truck 2 has stopped.
[0034] Camera 35 captures images of objects around the dump truck 2. Camera 35 acquires video footage. As shown in Figure 2, camera 35 is positioned at the front of the dump truck 2's body 14. Camera 35 may also be positioned at the rear of the body 14 or at the side of the body 14. Multiple cameras 35 may be positioned on the body 14. An RGB camera is given as an example of camera 35. Camera 35 may also be an RGB-D camera or a hyperspectral camera. Camera 35 captures images of at least the road surface of the work site 1 where the dump truck 2 is traveling.
[0035] The detection area of the obstacle sensor 34 and at least a portion of the imaging area of the camera 35 overlap. The objects detected by the obstacle sensor 34 and the objects captured by the camera 35 are the same.
[0036] The processor 30A of the controller 17 includes a driving control unit 17A, a video writing unit 17B, a real-time video transmission unit 17C, and a stored video transmission unit 17D. The storage 30C of the controller 17 includes a video storage unit 17E.
[0037] The driving control unit 17A outputs control commands to control the driving equipment 15 of the dump truck 2. As will be described later, driving data for the dump truck 2 is generated in the control server 12. The driving control unit 17A controls the driving equipment 15 so that the dump truck 2 drives according to the driving data.
[0038] The video writing unit 17B stores the video captured by the camera 35 in the video storage unit 17E. In this embodiment, the dump truck 2 has a drive recorder system. The controller 17 has a drive recorder function. The video writing unit 17B stores the saved video, which is the video captured by the camera 35 between a first time point t1 and a second time point t2 that is set back by a retrospective time Ta from the first time point t1, in the video storage unit 17E. The second time point t2 is a time point earlier than the first time point t1. The saved video is the video captured by the camera 35 at a retrospective time Ta that is set back from the first time point t1.
[0039] The real-time video transmission unit 17C transmits real-time video captured by a camera 35 installed on a dump truck 2 traveling through the work site 1 to the control server 12.
[0040] The saved video transmission unit 17D transmits the saved video stored in the video storage unit 17E to the control server 12.
[0041] Figure 5 is a block diagram showing a control server 12 according to an embodiment. A display device 68 and an input device 69 are connected to the control server 12. The display device 68 and the input device 69 are located in the control facility 120. The display device 68 provides display data to an administrator located in the control facility 120. The display device 68 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD). The input device 69 generates input data when operated by an administrator located in the control facility 120. Examples of input devices 69 include a touch panel, buttons, and a computer keyboard.
[0042] The processor 30A of the control server 12 includes a driving data generation unit 12A, a real-time video acquisition unit 12B, a saved video acquisition unit 12C, a display control unit 12D, an input data acquisition unit 12E, a determination unit 12F, a restart unit 12G, and a maintenance command unit 12H.
[0043] The driving data generation unit 12A generates driving data indicating the driving conditions of the dump truck 2. The driving conditions of the dump truck 2 include the target position of the dump truck 2, the target direction of the dump truck 2, and the target driving speed of the dump truck 2. The driving data generation unit 12A transmits the generated driving data to the controller 17 of the dump truck 2. The dump truck 2 drives around the work site 1 based on the driving data generated by the driving data generation unit 12A.
[0044] The real-time video acquisition unit 12B acquires real-time video images captured by the camera 35 installed on the dump truck 2. The real-time video acquisition unit 12B receives real-time video transmitted from the real-time video transmission unit 17C.
[0045] The video storage acquisition unit 12C acquires the video storage captured by the camera 35 at the retrospective time Ta. The video storage acquisition unit 12C receives the video storage transmitted from the video storage transmission unit 17D.
[0046] The display control unit 12D generates display data to be displayed on the display device 68. The display control unit 12D displays the generated display data on the display device 68. In this embodiment, the display control unit 12D displays at least real-time video and stored video on the display device 68.
[0047] The input data acquisition unit 12E acquires input data generated by the operation of the input device 69 from the input device 69.
[0048] The determination unit 12F determines, based on at least one of the real-time video and the saved video, whether or not it is possible to resume driving the stopped dump truck 2 if the dump truck 2 is stopped.
[0049] If the determination unit 12F determines that it is possible to resume driving, the restart unit 12G outputs a restart command to resume driving the dump truck 2. Alternatively, the restart unit 12G may output a restart command to resume driving the dump truck 2 based on the input data acquired by the input data acquisition unit 12E.
[0050] The maintenance command unit 12H outputs a maintenance command to a work machine capable of performing maintenance if there are parts requiring maintenance on the road surface of the work site 1 where the dump truck 2 is traveling. Examples of work machines capable of performing maintenance include a bulldozer 4 and a motor grader 5. The maintenance command unit 12H outputs a maintenance command to at least one of the bulldozer 4 and the motor grader 5 to perform maintenance on the parts requiring maintenance.
[0051] [Driving data] Figure 6 is a diagram illustrating the driving data of a dump truck 2 according to an embodiment. In this embodiment, the dump truck 2 is an unmanned dump truck that travels on the road surface of the work site 1 according to the driving data. The driving data of the dump truck 2 defines the driving conditions of the dump truck 2. The driving data of the dump truck 2 includes a driving point 70, a driving path 71, a target position of the dump truck 2, a target direction of the dump truck 2, a target driving speed of the dump truck 2, and a target turning radius of the dump truck 2.
[0052] Multiple travel points 70 are set on the road surface of the work site 1. Each travel point 70 defines the target position of the dump truck 2. For each of the multiple travel points 70, the target direction and target travel speed of the dump truck 2 are set. The multiple travel points 70 are set at intervals. The intervals between the travel points 70 may be uniform or uneven.
[0053] The target position of dump truck 2 refers to the target position of dump truck 2 when it passes through point 70. The target position of dump truck 2 may be defined in the local coordinate system of dump truck 2 or in the global coordinate system. The target bearing of dump truck 2 refers to the target bearing of dump truck 2 when it passes through point 70. The target driving speed of dump truck 2 refers to the target driving speed of dump truck 2 when it passes through point 70.
[0054] The driving path 71 is a virtual line indicating the target driving route of the dump truck 2. The driving path 71 is defined by a trajectory that passes through multiple driving points 70. The dump truck 2 travels around the work site 1 according to the driving path 71. The dump truck 2 travels such that its center coincides with the driving path 71 in the direction of the width of the dump truck 2.
[0055] The travel control unit 17A of the controller 17 controls the travel device 15 so that the dump truck 2 travels according to the travel path 71, based on the detection data from the position sensor 31 and the detection data from the orientation sensor 32. Specifically, the travel control unit 17A controls the travel device 15 so that the deviation between the detected position of the dump truck 2 detected by the position sensor 31 and the target position of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70. The travel control unit 17A controls the travel device 15 so that the deviation between the detected orientation of the dump truck 2 detected by the orientation sensor 32 and the target orientation of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70. The travel control unit 17A controls the travel device 15 so that the dump truck 2 travels at the target travel speed, based on the detection data from the speed sensor 33. Specifically, the travel control unit 17A controls the travel device 15 so that the deviation between the detected travel speed of the dump truck 2 detected by the speed sensor 33 and the target travel speed of the dump truck 2 set at the travel point 70 is small when passing through the travel point 70.
[0056] [Situations that make it difficult to continue driving] Figure 7 is a diagram illustrating an event in which it becomes difficult for the dump truck 2 to continue driving according to the embodiment. In this embodiment, if an event occurs that makes it difficult for the dump truck 2 to continue driving, the driving control unit 17A of the controller 17 stops the dump truck 2. In the following description, the event in which it becomes difficult for the dump truck 2 to continue driving will be appropriately referred to as an abnormal event.
[0057] An abnormal event includes an abnormality in the driving condition of the dump truck 2. In this embodiment, an abnormal event includes the detection of an abnormal portion 72 of the road surface at the work site 1 where the dump truck 2 is driving by the obstacle sensor 34.
[0058] As shown in Figure 7, abnormal parts 72, such as protrusions or depressions, may be generated on the road surface. These abnormal parts 72 may make it difficult for the dump truck 2 to continue driving on the road. The obstacle sensor 34 of the dump truck 2 detects the abnormal parts 72. When the obstacle sensor 34 detects an abnormal part 72 while the dump truck 2 is driving, the driving control unit 17A of the controller 17 determines that an abnormal event has occurred. If the driving control unit 17A determines that an abnormal event has occurred, it stops the dump truck 2. When the dump truck 2 is stopped, the driving control unit 17A sends a stop signal to the control server 12 indicating that the dump truck 2 has stopped. Alternatively, the driving control unit 17A may send stop position data indicating the stopping position of the dump truck 2 to the control server 12 based on the detection data from the position sensor 31 when the dump truck 2 has stopped.
[0059] If the driving control unit 17A determines that an abnormal event has occurred, the video writing unit 17B stores the video captured by the camera 35 as saved video in the video storage unit 17E during a retrospective time Ta, which is set back from the first time point t1 in which the abnormal event was determined to have occurred.
[0060] In this embodiment, the first time point t1 at which an abnormal event is determined to have occurred is the time when the dump truck 2 stopped due to the abnormal event. The video storage unit 17E stores the video footage from the time point t1 at which the dump truck 2 stopped, retrospectively, in the video storage unit 17E as saved video footage.
[0061] The first time point t1 at which an abnormal event is determined to have occurred may also be the time when braking of the running gear 15 of the dump truck 2 begins due to the abnormal event (the time when the brake device begins to operate). The first time point t1 at which an abnormal event is determined to have occurred may also be the time when the obstacle sensor 34 detects the abnormal part 72.
[0062] The real-time video transmission unit 17C transmits real-time video to the control server 12. If an abnormal event is detected and saved video is stored in the video storage unit 17E, the saved video transmission unit 17D transmits the saved video stored in the video storage unit 17E to the control server 12.
[0063] Furthermore, the real-time video transmission unit 17C may transmit real-time video to the control server 12 if the administrator of the control facility 120 requests the transmission of real-time video. The stored video transmission unit 17D may transmit stored video to the control server 12 if the administrator of the control facility 120 requests the transmission of stored video. The administrator operates the input device 69 when requesting the transmission of at least one of real-time video and stored video. The input data acquisition unit 12E transmits a video request signal to the controller 17 based on the input data from the input device 69. The real-time video transmission unit 17C may transmit real-time video to the control server 12 based on the video request signal. The stored video transmission unit 17D may transmit stored video to the control server 12 based on the video request signal.
[0064] Furthermore, if an abnormal event occurs and a stop signal indicating that dump truck 2 has stopped is sent from controller 17 to control server 12, display control unit 12D may display stop notification data indicating that dump truck 2 has stopped on display device 68. The administrator of control facility 120 can recognize that dump truck 2 has stopped based on the stop notification data displayed on display device 68. After recognizing that dump truck 2 has stopped, the administrator may operate input device 69 so that a video request signal is sent to controller 17.
[0065] The real-time video acquisition unit 12B receives real-time video transmitted from the real-time video transmission unit 17C. The saved video acquisition unit 12C receives saved video transmitted from the saved video transmission unit 17D. Both the real-time video and the saved video include images of the area around the dump truck 2 that has stopped due to an abnormal event. The display control unit 12D displays the real-time video and saved video of the area around the dump truck 2 that has stopped due to an abnormal event on the display device 68.
[0066] [Display device] Figure 8 shows an example of a display device 68 that displays real-time video 91 and saved video 92 according to the embodiment. As shown in Figure 8, the real-time video 91 and saved video 92 are displayed side by side on the display screen of the display device 68. Both the real-time video 91 and saved video 92 are moving images. The saved video 92 is played in parallel with the real-time video 91. By checking the real-time video 91 and saved video 92 displayed on the display device 68, the administrator can recognize the situation around the dump truck 2 that has stopped due to an abnormal event.
[0067] After the administrator recognizes the situation around the dump truck 2 that has stopped due to an abnormal event, they take measures to restart the dump truck 2. For example, if the administrator recognizes that the abnormal part 72 is an obstacle and it is difficult to restart the dump truck 2, the administrator operates the input device 69 to have the bulldozer 4 or motor grader 5 service the abnormal part 72. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the maintenance command unit 12H outputs a maintenance command to the bulldozer 4 or motor grader 5 to service the abnormal part 72. If the bulldozer 4 and motor grader 5 are manned work machines, the maintenance command is displayed on a display device installed in the operator's cab of the manned work machine. Based on the maintenance command displayed on the display device, the operator of the manned work machine operates the manned work machine so that the abnormal part 72 is serviced. If the bulldozer 4 or motor grader 5 is an unmanned work machine, the controller of the unmanned work machine outputs a control command to operate the unmanned work machine so that the faulty part 72 is serviced, based on the maintenance command.
[0068] Repairing the abnormal portion 72 includes removing the abnormal portion 72. Repairing the abnormal portion 72 includes leveling the road surface on which the abnormal portion 72 is located. If the abnormal portion 72 is a protrusion on the road surface, repairing the abnormal portion 72 includes removing the protrusion and leveling the road surface. If the abnormal portion 72 is a recess on the road surface, repairing the abnormal portion 72 includes filling the recess with soil and leveling the road surface.
[0069] After maintenance by the bulldozer 4 or motor grader 5 is completed, the manager can check whether the abnormal part 72 has disappeared by looking at the real-time video 91 displayed on the display device 68. After confirming that the abnormal part 72 has disappeared, the manager operates the input device 69 to restart the dump truck 2. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the restart unit 12G sends a restart command to the controller 17 of the dump truck 2 to restart the dump truck 2, which has stopped due to the abnormal event. The driving control unit 17A of the controller 17 restarts driving based on the restart command from the restart unit 12G. The dump truck 2 drives based on the driving data.
[0070] In addition, the dump truck 2 may stop due to a detection error by the obstacle sensor 34. The dump truck 2 may stop if there is no abnormal part 72 or if the abnormal part 72 is minor. After checking the real-time video 91 and saved video 92 displayed on the display device 68, the administrator may immediately restart the stopped dump truck 2. If the administrator decides that it is OK to immediately restart the stopped dump truck 2, the administrator operates the input device 69 to restart the dump truck 2. Based on the input data, the restart unit 12G transmits a restart command to the controller 17 of the dump truck 2 to restart the stopped dump truck 2.
[0071] The control server 12 may determine whether or not to perform maintenance on the abnormal part 72. The control server 12 may also determine whether or not to resume driving the dump truck 2. The determination unit 12F can calculate the abnormality level of the abnormal part 72 by image processing of at least one of the real-time video and the saved video. If the abnormal part 72 is a protrusion on the road surface, the determination unit 12F can calculate the height of the protrusion as the abnormality level by image processing of at least one of the real-time video and the saved video. If the abnormal part 72 is a recess on the road surface, the determination unit 12F can calculate the depth of the recess as the abnormality level by image processing of at least one of the real-time video and the saved video. If the abnormality level of the abnormal part 72 is above a predetermined threshold, the determination unit 12F can determine that maintenance is necessary. If the abnormality level of the abnormal part 72 is below the threshold, the determination unit 12F can determine that maintenance is not necessary.
[0072] The determination unit 12F may also calculate the abnormality level of the abnormal portion 72 based on the detection data from the obstacle sensor 34. The obstacle sensor 34 can detect the three-dimensional shape of the abnormal portion 72. Based on the detection data from the obstacle sensor 34, the determination unit 12F can calculate the height of the protrusion or the depth of the recess as an abnormality level.
[0073] If the determination unit 12F determines that maintenance is necessary, the maintenance command unit 12H can output a maintenance command to at least one of the bulldozer 4 and the motor grader 5. After maintenance by the bulldozer 4 or the motor grader 5 is completed, the determination unit 12F can determine whether or not the abnormal part 72 has disappeared by processing the real-time video 91. Alternatively, after maintenance is completed, the determination unit 12F may determine whether or not the abnormal part 72 has disappeared based on the detection data from the obstacle sensor 34.
[0074] The determination unit 12F determines that the abnormality level of the abnormal part 72 is below a threshold, or that the abnormal part 72 has been eliminated by maintenance, and determines that the stationary dump truck 2 can be restarted. The restart unit 12G outputs a restart command to restart the dump truck 2 if it determines that it is possible to restart the dump truck 2.
[0075] [Management method] Figure 9 is a flowchart showing the management method for the work site 1 according to the embodiment. In the following description, an example will be described in which the control server 12 determines whether or not to perform maintenance on the abnormal part 72 and whether or not to resume driving the dump truck 2.
[0076] When the obstacle sensor 34 of the dump truck 2 traveling on the road surface of the work site 1 detects an abnormal part 72, the driving control unit 17A determines that an abnormal event has occurred and stops the dump truck 2. The driving control unit 17A transmits a stop signal to the control server 12 indicating that the dump truck 2 has stopped. The display control unit 12D receives the stop signal indicating that the dump truck 2 has stopped (step S1).
[0077] When the display control unit 12D receives a stop signal, it displays stop notification data on the display device 68 indicating that the dump truck 2 has stopped. The administrator of the control facility 120 can recognize that the dump truck 2 has stopped based on the stop notification data displayed on the display device 68. After recognizing that the dump truck 2 has stopped, the administrator operates the input device 69 to request the controller 17 to transmit real-time video and saved video. The input data acquisition unit 12E transmits a video request signal to the controller 17 requesting the transmission of real-time video and saved video based on the input data from the input device 69 (step S2).
[0078] The real-time video transmission unit 17C transmits real-time video to the control server 12 based on the video request signal. The stored video transmission unit 17D transmits stored video to the control server 12 based on the video request signal. The real-time video acquisition unit 12B receives the real-time video transmitted from the real-time video transmission unit 17C. The stored video acquisition unit 12C receives the stored video transmitted from the stored video transmission unit 17D. Both the real-time video and the stored video include images of the area around the dump truck 2 that has stopped due to an abnormal event. The display control unit 12D displays the real-time video and stored video of the area around the dump truck 2 that has stopped due to an abnormal event on the display device 68 (step S3).
[0079] The determination unit 12F calculates the abnormality level of the abnormal portion 72 based on at least one of the real-time video and the saved video. Based on the abnormality level of the abnormal portion 72, the determination unit 12F determines whether or not it is possible to resume driving the dump truck 2 (step S4).
[0080] If it is determined in step S4 that it is possible to resume driving the dump truck 2 (step S4: Yes), the restart unit 12G outputs a restart command to resume driving the dump truck 2 (step S5).
[0081] If it is determined in step S4 that it is not possible to resume driving the dump truck 2 (step S4: No), the maintenance command unit 12H outputs a maintenance command to at least one of the bulldozer 4 and the motor grader 5 to repair the abnormal part 72 (step S6).
[0082] [effect] As described above, the processor 30A of the control server 12 includes a real-time video acquisition unit 12B that acquires real-time video images captured by a camera 35 installed on a dump truck 2 traveling through the work site 1, a saved video acquisition unit 12C that acquires saved video images captured by the camera 35 during a retrospective time Ta, starting from a first time point t1 when it is determined that an abnormal event has occurred that makes it difficult for the dump truck 2 to continue traveling, and a display control unit 12D that displays the real-time video and saved video on a display device 68.
[0083] According to this embodiment, if an abnormal event occurs and the dump truck 2 stops, real-time video and saved video of the area around the dump truck 2 that has stopped due to the abnormal event are displayed on the display device 68. This allows for prompt action to be taken to restart the dump truck 2. Since the dump truck 2 is not left stopped for a long period of time, the decrease in productivity at the work site 1 is suppressed.
[0084] [Other embodiments] In the above-described embodiment, the abnormal event may be an event that makes it difficult for the dump truck 2 to continue driving according to the driving data. The abnormal event may include the dump truck 2 deviating from the driving path 71. For example, if the dump truck 2 deviates from the driving path 71, the driving control unit 17A will stop the dump truck 2. When the dump truck 2 deviates from the driving path 71 and stops, real-time video and saved video of the area around the stopped dump truck 2 may be displayed on the display device 68.
[0085] In the above embodiment, if the dump truck 2 stops due to an abnormal part 72, the maintenance command unit 12H outputs a maintenance command to have the bulldozer 4 or motor grader 5 repair the abnormal part 72. The maintenance command unit 12H may also output a confirmation command to have the driver of the light vehicle 6 check the condition around the stopped dump truck 2. The administrator operates the input device 69 to have the driver of the light vehicle 6 check the abnormal part 72. The input data acquisition unit 12E acquires the input data generated by the operation of the input device 69. Based on the input data, the maintenance command unit 12H outputs a confirmation command to have the driver of the light vehicle 6 check the abnormal part 72. The confirmation command is displayed on the display device provided in the driver's cab of the light vehicle 6. Based on the confirmation command displayed on the display device, the driver of the light vehicle 6 can operate the light vehicle 6 to drive to the abnormal part 72.
[0086] In the above-described embodiment, the display device 68 for displaying real-time and saved images may be provided on at least one of the bulldozer 4, motor grader 5, and light vehicle 6. The operator of the bulldozer 4 or motor grader 5 can start maintenance on the abnormal part 72 without waiting for a maintenance command from the maintenance command unit 12H by checking the real-time and saved images displayed on the display device 68. The driver of the light vehicle 6 can drive the light vehicle 6 towards the abnormal part 72 to check it by checking the real-time and saved images displayed on the display device 68.
[0087] In the above-described embodiment, at least a portion of the functional components of the control server 12 may be provided in the controller 17. At least a portion of the functional components of the controller 17 may be provided in the control server 12.
[0088] In the above-described embodiment, each of the multiple functional units of the control server 12 may be configured by a separate computer (hardware). Each of the multiple functional units of the controller 17 may be configured by a separate computer (hardware). [Explanation of symbols]
[0089] 1...Work site, 2...Dump truck (work vehicle), 3...Excavator (work machine), 4...Bulldozer (work machine), 5...Motor grader (work machine), 6...Light vehicle (driving vehicle), 7...Loading area, 8...Soil removal area, 9...Transport route, 10...Management system, 12...Control server, 12A...Driving data generation unit, 12B...Real-time video acquisition unit, 12C...Stored video acquisition unit, 12D...Display control unit, 12E...Input data acquisition unit, 12F...Determination unit, 12G...Restart unit, 12H...Maintenance command unit, 120...Control facility, 13...Communication system, 14...Vehicle body, 15...Driving gear, 16...Dump body 17...Controller, 17A...Travel control unit, 17B...Video writing unit, 17C...Real-time video transmission unit, 17D...Stored video transmission unit, 17E...Video storage unit, 30...Computer, 30A...Processor, 30B...Main memory, 30C...Storage, 30D...Input / output interface, 30E...Communication interface, 30F...Computer program, 31...Position sensor, 32...Direction sensor, 33...Speed sensor, 34...Obstacle sensor, 35...Camera, 68...Display device, 69...Input device, 70...Travel point, 71...Travel path, 72...Abnormal area, 91...Real-time video, 92...Stored video.
Claims
1. Equipped with a processor, The aforementioned processor, Real-time video footage is captured by cameras installed on work vehicles traveling around the work site. The camera captures and stores saved video footage from the time period preceding the point in time when it is determined that an event has occurred that makes it difficult for the aforementioned work vehicle to continue driving. The real-time video and the saved video are displayed on a display device. A management system for work sites.
2. The aforementioned time is the point in time when the work vehicle stopped as a result of the aforementioned event. A work site management system according to claim 1.
3. Each of the aforementioned real-time video and the aforementioned saved video includes video of the area around the work vehicle that stopped due to the aforementioned event. A work site management system according to claim 1.
4. The aforementioned work vehicle is equipped with an obstacle sensor. The aforementioned event includes the detection of an abnormal portion of the road surface on which the work vehicle is traveling by the obstacle sensor. A work site management system according to claim 1.
5. The aforementioned processor, The driving data of the aforementioned work vehicle is generated, The aforementioned event is one in which it is difficult to continue driving according to the aforementioned driving data. A work site management system according to claim 1.
6. The aforementioned work vehicle is an unmanned dump truck that operates according to the aforementioned driving data. A work site management system according to claim 5.
7. The aforementioned processor, Acquire input data from the input device, Based on the input data, a restart command is output to resume the movement of the work vehicle that has stopped due to the aforementioned event. A work site management system according to claim 1.
8. The aforementioned processor, Based on at least one of the real-time video and the saved video, it is determined whether or not it is possible to resume the operation of the work vehicle that has stopped due to the event. If it is determined that it is possible to resume the aforementioned driving, a restart command is output to resume the driving of the work vehicle. A work site management system according to claim 1.
9. This involves acquiring real-time video footage captured by cameras mounted on work vehicles traveling around the work site, and To acquire saved video footage captured by the camera during the retrospective time period starting from the point in time when it was determined that an event occurred that made it difficult for the aforementioned work vehicle to continue driving, This includes displaying the real-time video and the stored video on a display device, Methods for managing the work site.
10. The aforementioned time is the point in time when the work vehicle stopped as a result of the aforementioned event. The method for managing a work site according to claim 9.
11. Each of the aforementioned real-time video and the aforementioned saved video includes video of the area around the work vehicle that stopped due to the aforementioned event. The method for managing a work site according to claim 9.
12. The aforementioned work vehicle is equipped with an obstacle sensor. The aforementioned event includes the detection of an abnormal portion of the road surface on which the work vehicle is traveling by the obstacle sensor. The method for managing a work site according to claim 9.
13. This includes generating driving data for the aforementioned work vehicle, The aforementioned event is one in which it is difficult to continue driving according to the aforementioned driving data. The method for managing a work site according to claim 9.
14. The aforementioned work vehicle is an unmanned dump truck that operates according to the aforementioned driving data. The method for managing a work site according to claim 13.
15. Acquiring input data from an input device, This includes outputting a restart command based on the input data to resume the operation of the work vehicle that has stopped due to the aforementioned event, The method for managing a work site according to claim 9.
16. Based on at least one of the real-time video and the saved video, it is determined whether or not it is possible to resume the operation of the work vehicle that has stopped due to the event. If it is determined that it is possible to resume the aforementioned driving, the system outputs a restart command to resume the driving of the work vehicle, including: The method for managing a work site according to claim 9.
Citation Information
Patent Citations
System for controlling work site and method for controlling work site
JP2019116890A
