Remote operation system for work machine and remote operation method for work machine

Abstract

This remote operation system for a work machine comprises a processor. The processor acquires a captured image showing an image of a work site for a remotely-operated work machine photographed by an imaging device, acquires work data related to an unmanned vehicle working at the work site, generates a work image representing the work data, and causes a display device disposed outside of the work machine to display the work image together with the captured image.

Description

Remote operation system for construction machinery and method for remotely operating construction machinery

[0001] The present disclosure relates to a remote operation system for construction machinery and a method for remotely operating construction machinery.

[0002] In the technical field related to remote operation systems for construction machinery, a remote operation system as disclosed in Patent Document 1 is known. In the remote operation system, the work site of the construction machinery is imaged by an imaging device. The captured image of the work site captured by the imaging device is displayed on a display device arranged in the remote operation room. The operator in the remote operation room remotely operates the construction machinery while checking the captured image of the work site displayed on the display device.

[0003] Japanese Patent Application Laid-Open No. 2023-174148

[0004] There are cases where an unmanned vehicle works at the work site. In the remote operation of construction machinery, if it becomes difficult for the operator to recognize the working status of the unmanned vehicle, the working efficiency of the construction machinery may decrease.

[0005] An object of the present disclosure is to suppress a decrease in working efficiency.

[0006] According to the present disclosure, a remote operation system for construction machinery is provided. The remote operation system includes a processor. The processor acquires a captured image showing an image of the work site of the construction machinery to be remotely operated captured by the imaging device, acquires work data related to an unmanned vehicle working at the work site, generates a work image showing the work data, and causes the work image and the captured image to be displayed on a display device arranged outside the construction machinery.

[0007] According to the present disclosure, a decrease in working efficiency is suppressed.

[0008] Figure 1 is a schematic diagram showing a work site according to the first embodiment. Figure 2 is a schematic diagram showing the management system and remote control system according to the first embodiment. Figure 3 is a schematic side view showing a bulldozer according to the first embodiment. Figure 4 is a schematic diagram showing the soil removal area according to the first embodiment. Figure 5 is a hardware configuration diagram showing the remote controller according to the first embodiment. Figure 6 is a block diagram showing the management system and remote control system according to the first embodiment. Figure 7 is a diagram for explaining the driving data according to the first embodiment. Figure 8 is a diagram showing an captured image displayed on the display device according to the first embodiment. Figure 9 is a flowchart showing the image display method according to the first embodiment. Figure 10 is a diagram showing an captured image displayed on the display device according to the second embodiment. Figure 11 is an enlarged view of the window image according to the second embodiment. Figure 12 is a diagram showing the captured image and window image according to the third embodiment. Figure 13 is a diagram showing the captured image and window image according to the fourth embodiment. Figure 14 is a schematic diagram showing the soil removal area according to the fifth embodiment.

[0009] The embodiments relating to this disclosure will be described below with reference to the drawings, but this disclosure is not limited to these embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

[0010] [First Embodiment] The first embodiment will now be described.

[0011] <Work Site> Figure 1 is a schematic diagram showing a work site 1 according to the first embodiment. An example of a work site 1 is a mine or a quarry. A mine is a place or business establishment where minerals are extracted. A quarry is a place or business establishment where stone materials are extracted. Examples of mines include metal mines where metals are extracted, non-metallic mines where limestone is extracted, and coal mines where coal is extracted.

[0012] At work site 1, an unmanned vehicle performs work. An unmanned vehicle is a work vehicle that performs work without operator operation. In this embodiment, the unmanned vehicle is a transport vehicle that performs the transport work of transporting cargo. In this embodiment, a dump truck 2, which is an example of a transport vehicle, performs work at work site 1. The dump truck 2 has a dump body. The dump truck 2 is capable of loading cargo onto its dump body and performing transport work. The dump truck 2 is capable of performing soil removal work by unloading the cargo from its dump body.

[0013] Multiple work machines operate at the work site 1. In this embodiment, the work machines are remotely controlled by an operator. Examples of work machines include a loading machine that loads cargo onto a dump truck 2, and an excavating machine that excavates at least a portion of the work site 1. The excavating machine is capable of performing either or both of the following: embankment formation work to form an embankment at the work site 1 and ground leveling work to level the ground at the work site 1.

[0014] In this embodiment, a shovel 3, which is an example of a loading machine, and a bulldozer 4 and a motor grader 5, which are examples of excavating machines, work at the work site 1. The shovel 3 has an implement. The shovel 3 can perform loading and excavating work using the implement. The bulldozer 4 and the motor grader 5 each have an implement. The bulldozer 4 and the motor grader 5 each can perform leveling work using the implement.

[0015] A loading area 6, a soil removal area 7, and a transport route 8 are provided at the work site 1. Each of the loading area 6, soil removal area 7, and transport route 8 is a work area where unmanned vehicles and work machinery can operate, respectively.

[0016] Loading area 6 refers to the work area where loading operations are carried out to load cargo onto dump truck 2. An example of cargo is excavated material excavated in loading area 6. Excavator 3 performs excavation and loading operations in loading area 6.

[0017] The soil removal area 7 is the work area where the soil removal operation is carried out, in which the dump truck 2 unloads its cargo. The bulldozer 4 performs embankment formation and leveling work in the soil removal area 7. The bulldozer 4 may also perform embankment formation and leveling work in the loading area 6.

[0018] The transport path 8 refers to the road on which the dump truck 2 travels. The transport path 8 leads to the loading area 6 and the soil removal area 7, respectively. The transport path 8 is provided to connect at least the loading area 6 and the soil removal area 7. The dump truck 2 traveling towards at least one of the loading area 6 and the soil removal area 7 travels along the transport path 8. The dump truck 2 travels along the transport path 8, for example, by going back and forth between the loading area 6 and the soil removal area 7. The motor grader 5 performs leveling work on the transport path 8.

[0019] A light vehicle 9 travels within the work site 1. A light vehicle 9 is a lightweight vehicle that travels within the work site 1. The external dimensions of the light vehicle 9 are smaller than those of the dump truck 2. The weight of the light vehicle 9 is lighter than that of the dump truck 2. The light vehicle 9 can travel within the loading area 6, the soil removal area 7, and the transport route 8. The light vehicle 9 travels to inspect the work site 1 and to transport workers. The light vehicle 9 may be an unmanned vehicle or a manned vehicle. A manned vehicle is a vehicle that travels based on driving operations by a driver. In this embodiment, the light vehicle 9 is a manned vehicle.

[0020] Each of the excavator 3, bulldozer 4, and motor grader 5 may be an unmanned or manned work machine. In this embodiment, each of the excavator 3, bulldozer 4, and motor grader 5 is a manned work machine. A manned work machine is a work machine that performs work based on the operation of an operator.

[0021] <Management System and Remote Control System> Figure 2 is a schematic diagram showing the management system 10 and remote control system 19 according to the first embodiment. The management system 10 manages the work site 1. The management system 10 manages each of the dump truck 2, shovel 3, bulldozer 4, motor grader 5, and light vehicle 9. In Figure 2, the bulldozer 4 is shown as a work machine. In the following description, for the sake of simplicity, an example will be given in which the management system 10 manages the dump truck 2 and the bulldozer 4.

[0022] The management system 10 comprises a management device 11 and a communication system 12. The management device 11 includes a computer. The management device 11 is located outside the dump truck 2 and the bulldozer 4. The management device 11 is installed in the control facility 13 of the work site 1. The management device 11 manages the work site 1. The management device 11 manages at least the dump truck 2 and the bulldozer 4. The communication system 12 includes at least one of the following: the internet, a mobile phone network, a satellite network, and a local area network (LAN).

[0023] The dump truck 2 comprises a body 14, a running gear 15, a dump body 16, a controller 17, and a wireless communication device 12A. The body 14 includes a body frame. The body 14 is supported by the running gear 15. The running gear 15 supports the body 14 and drives. The running gear 15 includes wheels, tires mounted on the wheels, an engine, a brake system, and a steering system. The dump body 16 is the component into which the cargo is loaded. The dump body 16 is supported by the body 14. The dump body 16 performs dumping and lowering operations. Dumping operation refers to the operation of moving the dump body 16 away from the body 14 and tilting it in the dumping direction. Lowering operation refers to the operation of moving the dump body 16 closer to the body 14. When loading work is performed, the dump body 16 performs a lowering operation. When soil removal work is performed, the dump body 16 performs a dumping operation. The controller 17 includes a computer. The wireless communication device 12A is connected to the controller 17.

[0024] The bulldozer 4 has a controller 18 and a wireless communication device 12B. The controller 18 includes a computer. The wireless communication device 12B is connected to the controller 18.

[0025] The communication system 12 includes a wireless communication device 12A connected to the controller 17 of the dump truck 2, a wireless communication device 12B connected to the controller 18 of the bulldozer 4, and a wireless communication device 12C connected to the management device 11. The management device 11 and the controller 17 of the dump truck 2 communicate wirelessly via the communication system 12. The management device 11 and the controller 18 of the bulldozer 4 communicate wirelessly via the communication system 12.

[0026] In this embodiment, the bulldozer 4 is remotely controlled by a remote control system 19. At least a portion of the remote control system 19 is located in a remote control room 20. The remote control room 20 is located outside the bulldozer 4. The remote control room 20 is installed in a remote location away from the work site 1. The remote control system 19 comprises a remote control device 21, a display device 22, and a remote controller 23.

[0027] The remote control device 21 is located in the remote control room 20. The remote control device 21 is operated by an operator in the remote control room 20. When the remote control device 21 is operated, an operation signal is generated to operate the bulldozer 4. The operator can operate the remote control device 21 while seated in the driver's seat 24.

[0028] The display device 22 is located in the remote control room 20. The display device 22 displays captured images of the work site 1. The display device 22 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED). The operator operates the remote control device 21 while checking the captured images of the work site 1 displayed on the display device 22. The bulldozer 4 is remotely controlled by the remote control device 21.

[0029] The remote controller 23 is located in the remote control room 20. The remote controller 23 includes a computer. The remote controller 23 and the controller 18 of the bulldozer 4 communicate via a communication system 25. The communication system 25 includes at least one of the following: the Internet, a mobile phone network, a satellite network, and a local area network (LAN).

[0030] The management device 11 and the remote controller 23 communicate via the communication system 25. Note that the communication system 12 and the communication system 25 may be separate communication systems. At least a portion of the communication system 12 and the communication system 25 may be the same communication system.

[0031] <Bulldozer> Figure 3 is a schematic side view showing a bulldozer 4 according to the first embodiment. As shown in Figure 3, the bulldozer 4 comprises a vehicle body 26, a traveling device 27, an excavating work machine 28, a ripper work machine 29, a position sensor 30, an attitude sensor 31, and an imaging device 32.

[0032] The vehicle body 26 supports an engine (not shown). The engine is the power source for the bulldozer 4. The running gear 27 supports the vehicle body 26 and moves. The running gear 27 has a pair of tracks 33. The bulldozer 4 moves as the tracks 33 rotate.

[0033] The excavation machine 28 performs excavation work, embankment work, soil pushing work, embankment formation work, and land leveling work. The excavation machine 28 is connected to the vehicle body 26. At least a part of the excavation machine 28 is positioned in front of the vehicle body 26. The excavation machine 28 has an excavation blade 34, a lift frame 35, a tilt cylinder 36, and a lift cylinder 37.

[0034] The drilling blade 34 is positioned in front of the vehicle body 26. The drilling blade 34 has a cutting edge 34A. The lift frame 35 supports the drilling blade 34. One end of the lift frame 35 is connected to the back of the drilling blade 34 via a pivot mechanism. The other end of the lift frame 35 is connected to the side of the traveling device 27 via a pivot mechanism. The lift frame 35 includes a first lift frame 35 connected to the left side of the back of the drilling blade 34 and a second lift frame 35 connected to the right side of the back of the drilling blade 34.

[0035] The tilt cylinder 36 and the lift cylinder 37 each operate the drilling blade 34. The tilt cylinder 36 is driven to tilt the drilling blade 34. The lift cylinder 37 is driven to move the drilling blade 34 up and down. One end of the tilt cylinder 36 is connected to the back of the drilling blade 34 via a pivot mechanism. The other end of the tilt cylinder 36 is connected to the upper surface of the lift frame 35. As the tilt cylinder 36 extends and retracts, the tilt angle of the drilling blade 34 changes. One end of the lift cylinder 37 is connected to the back of the drilling blade 34 via a pivot mechanism. The middle part of the lift cylinder 37 is connected to the side of the vehicle body 26. As the lift cylinder 37 extends and retracts, the drilling blade 34 moves in the vertical direction.

[0036] The ripper implement 29 performs ripping operations, such as cutting or crushing the workpiece. For example, it can be used for crushing rocks. The ripper implement 29 is connected to the vehicle body 26. At least a portion of the ripper implement 29 is positioned at the rear of the vehicle body 26. The ripper implement 29 includes a shank 38, a ripper arm 39, a tilt cylinder 40, a lift cylinder 41, and a beam 42.

[0037] The shank 38 is located at the rear of the vehicle body 26. The shank 38 has a ripper point 38A. The ripper point 38A is provided at the tip (lower end) of the shank 38, including the cutting edge position of the ripper. The ripper arm 39 supports the shank 38. The ripper arm 39 connects the vehicle body 26 and the shank 38. One end of the ripper arm 39 is connected to the rear of the vehicle body 26 via a pivot mechanism. The other end of the ripper arm 39 is connected to a beam 42. The beam 42 is rotatably connected to the ripper arm 39. The shank 38 is connected to the ripper arm 39 via the beam 42.

[0038] The tilt cylinder 40 and the lift cylinder 41 each operate the shank 38. The tilt cylinder 40 and the lift cylinder 41 are each connected to the vehicle body 26. The tilt cylinder 40 drives the shank 38 to tilt. The lift cylinder 37 drives the shank 38 to move up and down. One end of the tilt cylinder 40 is connected to the beam 42 via a pivot mechanism. The other end of the tilt cylinder 36 is connected to the rear of the vehicle body 26. As the tilt cylinder 40 extends and retracts, the tilt angle of the shank 38 changes. The tilt cylinder 40 moves the ripper point 38A in the front-rear direction. One end of the lift cylinder 41 is connected to the beam 42 via a pivot mechanism. The other end of the lift cylinder 41 is connected to the rear of the vehicle body 26. As the lift cylinder 41 extends and retracts, the shank 38 moves in the up-and-down direction. The lift cylinder 41 moves the shank 38, which includes the ripper point 38A, in the vertical direction.

[0039] The ripper work machine 29 pierces the workpiece with its ripper point 38A. While the traveling device 27 is stopped, the ripper point 38A is brought into contact with the workpiece, and as the traveling device 27 moves with the ripper point 38A embedded in the workpiece, the workpiece is cut or crushed. While the traveling device 27 is moving, the ripper work machine 29 is moved in the up, down, forward, and backward directions, bringing it into contact with the workpiece, and cutting or crushing occurs as the device moves with the workpiece embedded in it.

[0040] The position sensor 30 detects the position of the bulldozer 4. The position sensor 30 is located on the vehicle body 26. The position of the bulldozer 4 is detected using the Global Navigation Satellite System (GNSS). The Global Navigation Satellite System includes the Global Positioning System (GPS). The Global Navigation Satellite System detects the position in a global coordinate system defined by latitude, longitude, and altitude coordinate data. A global coordinate system refers to a coordinate system fixed to the Earth. The position sensor 30 includes a GNSS receiver located on the vehicle body 26. The position sensor 30 detects the position of the bulldozer 4 in the global coordinate system.

[0041] The attitude sensor 31 detects the attitude of the bulldozer 4. The attitude sensor 31 detects the attitude of the bulldozer 4 at least when the position sensor 30 has detected the position of the bulldozer 4. The attitude sensor 31 is positioned on the vehicle body 26. The attitude of the bulldozer 4 includes the tilt of the vehicle body 26. The attitude of the bulldozer 4 includes the tilt angle of the vehicle body 26 with respect to the horizontal plane. An inertial measurement unit (IMU) is exemplified as the attitude sensor 31. The attitude sensor 31 is capable of detecting the tilt angle of the vehicle body 26 with respect to the horizontal plane.

[0042] The imaging device 32 captures images of the object to be imaged. The imaging device 32 is positioned on the vehicle body 26. The object to be imaged by the imaging device 32 includes the work site 1 of the bulldozer 4. The object to be imaged by the imaging device 32 includes the ground of the work site 1 surrounding the bulldozer 4 and objects surrounding the bulldozer 4. The captured images showing the images of the object to be imaged captured by the imaging device 32 are displayed on the display device 22 in the remote control room 20. An RGB camera is given as an example of the imaging device 32. However, the imaging device 32 may also be an RGB-D camera or a hyperspectral camera.

[0043] <Dumping Area> FIG. 4 is a diagram schematically showing a dumping area 7 according to the first embodiment. The dumping area 7 is an example of a work area where the dump truck 2 and the bulldozer 4 can each operate at the work site 1. The dump truck 2 can travel in the dumping area 7. The ground of the dumping area 7 is substantially flat. In the dumping area 7, the dump truck 2 performs a dumping operation. There is a cliff 43 around the dumping area 7. The cliff 43 is located outside the edge 44 of the dumping area 7. The cliff 43 is a prohibited area where the operation of the dump truck 2 and the bulldozer 4 is prohibited at the work site 1. The dump truck 2 and the bulldozer 4 cannot travel on the cliff 43. The dumping area 7 and the cliff 43 are demarcated by the edge 44 of the dumping area 7. In the embodiment, an embankment 45 is formed at the edge 44. The embankment 45 protrudes upward from the ground of the dumping area 7 near the edge 44.

[0044] In the dumping operation, the dump truck 2 approaches the edge 44 of the dumping area 7 while reversing. After the controller 17 of the dump truck 2 determines that the dump truck 2 has approached the edge 44 of the dumping area 7, the dump body 16 is operated to perform a dumping action. When the dump body 16 performs a dumping action while the dump truck 2 is approaching the edge 44 while reversing, the load loaded on the dump body 16 is dumped onto the cliff 43.

[0045] The height of the embankment 45 is set to a height that does not interfere with the dumping operation of the dump truck 2 and is difficult for the dump truck 2 to climb over. The height of the embankment 45 is, for example, lower than the radius of the tire of the dump truck 2. Since the embankment 45 is formed at the edge 44, the dump truck 2 performing the dumping operation is suppressed from going outside the dumping area 7.

[0046] The bulldozer 4 performs a bank formation operation for forming a bank 45 in the dump area 7. The bulldozer 4 performs a leveling operation for leveling the ground in the dump area 7. For example, unevenness may be formed on the ground in the dump area 7 due to the running of the dump truck 2 or the fall of the load from the dump body 16. The bulldozer 4 performs a leveling operation so that the ground in the dump area 7 becomes flat.

[0047] <Computer> FIG. 5 is a hardware configuration diagram showing the remote controller 23 according to the first embodiment. The remote controller 23 includes a computer. The remote controller 23 has a processor 47 such as a CPU (Central Processing Unit), a main memory 48 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 49, an input / output interface 50 including an input / output circuit, and a communication interface 51 including a communication circuit. The functions of the remote controller 23 are stored in the storage 49 as a computer program 52. The processor 47 reads the computer program 52 from the storage 49 and expands it in the main memory 48, and executes processing according to the computer program 52. Note that the computer program 52 may be distributed to the remote controller 23 via a network.

[0048] Similar to the remote controller 23, each of the management device 11, the controller 17 of the dump truck 2, and the controller 18 of the bulldozer 4 also includes a computer. Each of the management device 11, the controller 17 of the dump truck 2, and the controller 18 of the bulldozer 4 also has a processor, a main memory, a storage for storing a computer program, an input / output interface, and a communication interface.

[0049] FIG. 6 is a block diagram showing the management system 10 and the remote operation system 19 according to the first embodiment.

[0050] The light vehicle 9 includes a position sensor 53 and a controller 54. The position sensor 53 detects the position of the light vehicle 9. The position sensor 53 includes a GNSS receiver located on the light vehicle 9. The position sensor 53 detects the position of the light vehicle 9 in a global coordinate system. The controller 54 transmits the detection data from the position sensor 53 to the management device 11 via the communication system 12.

[0051] The dump truck 2 includes a running gear 15, a dump body 16, a position sensor 55, a compass sensor 56, a speed sensor 57, and a controller 17.

[0052] The position sensor 55 detects the position of the dump truck 2. The position sensor 55 includes a GNSS receiver located on the dump truck 2. The position sensor 55 detects the position of the dump truck 2 in the global coordinate system.

[0053] The orientation sensor 56 detects the orientation of the dump truck 2. The orientation of the dump truck 2 includes the azimuth angle relative to a reference azimuth. The reference azimuth is, for example, north. An inertial measurement unit (IMU) is exemplified as the orientation sensor 56. The orientation sensor 56 may also include a calculator that calculates the orientation from position data detected by two GNSS antennas installed on the dump truck 2. The calculator can calculate the orientation from a vector connecting the two GNSS antennas. The orientation sensor 56 may also detect the tilt angle of the dump truck 2 with respect to the horizontal plane. If the orientation sensor 56 is an inertial sensor (IMU), it can detect the tilt angle of the dump truck 2 with respect to the horizontal plane.

[0054] The speed sensor 57 detects the travel speed of the dump truck 2. The speed sensor 57 detects the travel speed of the dump truck 2 by, for example, detecting the rotational speed of the drive shaft connected to the wheels of the running gear 15.

[0055] The processor 47 of the management device 11 has multiple functional units. The functional units of the processor 47 of the management device 11 include an edge data generation unit 58, a driving data generation unit 59, and a work data transmission unit 60.

[0056] The edge data generation unit 58 generates edge data for the work area. The edge data includes the shape and position of the edges of the work area. The position of the edges is defined in a global coordinate system. As described above, the work area includes the loading area 6, the soil removal area 7, and the transport path 8. In the embodiment, the edge data includes at least the shape and position of the edges 44 of the soil removal area 7. The edge data generation unit 58 generates edge data based on the detection data from the position sensor 53 of the light vehicle 9.

[0057] For surveying work site 1, the light vehicle 9 travels along the edge of the work area. The position of the light vehicle 9 as it travels along the edge of the work area is detected by the position sensor 53. The detection data from the position sensor 53 is transmitted from the controller 54 of the light vehicle 9 to the management device 11. The edge data generation unit 58 can generate edge data based on the detection data from the position sensor 53.

[0058] As explained with reference to Figure 4, a bank 45 is formed on the edge 44. When generating edge data for the soil removal area 7, the light vehicle 9 travels along the bank 45 through the soil removal area 7. The position of the edge 44 may also be the lower end of the bank 45 on the soil removal area 7 side. The position of the edge 44 may also be the slope of the bank 45 on the soil removal area 7 side. In the following description, the virtual line generated along the edge 44 will be appropriately referred to as the survey line 68. The edge data includes the survey line 68.

[0059] The driving data generation unit 59 generates driving data indicating the driving conditions of the dump truck 2 in the work area. The driving data generation unit 59 generates driving data of the dump truck 2 in at least the soil discharge area 7. The driving data generation unit 59 generates driving data of the dump truck 2 based on edge data. The driving conditions of the dump truck 2 include the target position of the dump truck 2, the target direction of the dump truck 2, the target driving speed of the dump truck 2, and the target turning radius of the dump truck 2.

[0060] Figure 7 is a diagram illustrating the driving data according to the first embodiment. Figure 7 shows the driving data set in the soil removal area 7. The driving data defines the driving conditions of the dump truck 2. The driving data for the dump truck 2 includes the driving point 70, the driving path 69, the target position of the dump truck 2, the target direction of the dump truck 2, and the target driving speed of the dump truck 2.

[0061] Multiple travel points 70 are set in the soil removal area 7. The travel points 70 define the target position of the dump truck 2. The target position of the dump truck 2 is defined in the global coordinate system. For each of the multiple travel points 70, the target direction and target travel speed of the dump truck 2 are set. The multiple travel points 70 are set at intervals. The intervals between the travel points 70 may be uniform or uneven.

[0062] The target position of dump truck 2 refers to the target position of dump truck 2 when it passes through point 70. The target direction of dump truck 2 refers to the target direction of dump truck 2 when it passes through point 70. The target travel speed of dump truck 2 refers to the target travel speed of dump truck 2 when it passes through point 70. The travel path 69 refers to a virtual line indicating the target travel route of dump truck 2. The travel path 69 is defined by a track that passes through multiple points 70.

[0063] As shown in Figure 7, of the multiple travel points 70, one travel point 70 is set as an entry point 70S, and one travel point 70 is set as an exit point 70E. In addition, of the multiple travel points 70, at least one travel point 70 is set as a switchback point 71, and at least one travel point 70 is set as an earth removal point 72. The switchback point 71 is the position where the dump truck 2 performs a switchback. A switchback is the action in which the forward-moving dump truck 2 makes a sharp change in direction and moves in reverse. The earth removal point 72 is the position where the dump truck 2 performs the earth removal operation to discharge its cargo.

[0064] The soil removal point 72 is set based on edge data (survey line 68). In this embodiment, the soil removal point 72 is set to overlap with the survey line 68.

[0065] The location of the travel point 70, which includes the switchback point 71 and the soil removal point 72, the location of the travel path 69 indicating the travel route of the dump truck 2, and the location of the survey line 68 are defined in a global coordinate system.

[0066] The driving data generated in the driving data generation unit 59 is transmitted to the dump truck 2. The controller 17 of the dump truck 2 controls the driving device 15 based on the driving data. The controller 17 controls the driving device 15 so that the dump truck 2 travels according to the driving path 69, based on the detection data from the position sensor 55 and the detection data from the orientation sensor 56. Specifically, the controller 17 controls the driving device 15 so that the deviation between the detected position of the dump truck 2 detected by the position sensor 55 and the target position of the dump truck 2 set at the driving point 70 is reduced when passing through the driving point 70. The controller 17 controls the driving device 15 so that the deviation between the detected orientation of the dump truck 2 detected by the orientation sensor 56 and the target orientation of the dump truck 2 set at the driving point 70 is reduced when passing through the driving point 70. The controller 17 controls the driving device 15 so that the dump truck 2 travels at the target driving speed, based on the detection data from the speed sensor 57. In other words, the controller 17 controls the driving device 15 so that the deviation between the detected driving speed of the dump truck 2 detected by the speed sensor 57 when passing the driving point 70 and the target driving speed of the dump truck 2 set at the driving point 70 becomes small.

[0067] In the example shown in Figure 7, the dump truck 2 enters the soil removal area 7 from the transport path 8 while moving forward. After entering the soil removal area 7 from the transport path 8, the dump truck 2 passes the entry point 70S and then enters the switchback point 71 while moving forward. After entering the switchback point 71, the dump truck 2 performs a switchback and then enters the soil removal point 72 while moving backward. The dump truck 2 that has entered the soil removal point 72 performs soil removal work at the soil removal point 72. With the dump truck 2 positioned at the soil removal point 72, the dump body 16 performs a dumping operation, and the load is removed from the dump truck 2 at the soil removal point 72. Having completed the soil removal work, the dump truck 2 moves forward towards the exit point 70E to leave the soil removal point 72. After passing the exit point 70E, the dump truck 2 leaves the soil removal area 7 and returns to the transport path 8.

[0068] The work data transmission unit 60 transmits work data related to the dump truck 2 to the remote controller 23. The work data of the dump truck 2 includes edge data generated by the edge data generation unit 58 and driving data of the dump truck 2 generated by the driving data generation unit 59. The work data of the dump truck 2 includes a driving path 69 indicating the driving route that the dump truck 2 will travel, a soil discharge point 72 indicating the soil discharge location where the dump truck 2 will perform soil discharge work, an edge 44 of the soil discharge area 7 where the dump truck 2 can work, a survey line 68 indicating the outer shape of the soil discharge area 7, and the location of any obstacles 73 that may obstruct the work of the dump truck 2.

[0069] The processor 47 of the remote controller 23 has multiple functional units. The functional units of the remote controller 23 include an operation signal transmission unit 61, an image acquisition unit 62, a work data acquisition unit 63, a work image generation unit 64, a reference image generation unit 65, and a display control unit 66.

[0070] The operation signal transmission unit 61 transmits an operation signal generated by the operation of the remote control device 21 to the controller 18 of the bulldozer 4. Based on the operation signal transmitted from the operation signal transmission unit 61, the controller 18 controls at least one of the traveling device 27, the excavating work machine 28, and the ripper work machine 29.

[0071] The image acquisition unit 62 acquires an image 80 showing the image of the work site 1 captured by the imaging device 32. The image acquisition unit 62 also acquires detection data from the position sensor 30 and the attitude sensor 31 when the image 80 is captured by the imaging device 32. The position of the image 80 is defined in the camera coordinate system set in the imaging device 32. Based on the detection data from the position sensor 30 and the attitude sensor 31, the image acquisition unit 62 can convert the position in the global coordinate system to the position on the image in the camera coordinate system.

[0072] The work data acquisition unit 63 acquires work data related to the dump truck 2. The work data acquisition unit 63 also acquires work data transmitted from the work data transmission unit 60.

[0073] The work image generation unit 64 generates a work image 81 that shows work data. The work image generation unit 64 generates a work image 81 for display on the display device 22.

[0074] The reference image generation unit 65 generates a reference image 82 showing at least one of the dimensions of the bulldozer 4 and the distance from the bulldozer 4. In one embodiment, the reference image generation unit 65 generates a reference image 82 showing the width of the excavation blade 34 and the distance from the bulldozer 4. The reference image generation unit 65 can generate a reference image 82 based, for example, on the specifications data of the bulldozer 4. The specifications data of the bulldozer 4 is known data.

[0075] The display control unit 66 causes the captured image 80 captured by the imaging device 32 to be displayed on the display device 22. The display control unit 66 causes the work image 81 to be displayed on the display device 22 together with the captured image 80. In one embodiment, the display control unit 66 superimposes the work image 81 onto the captured image 80. The display control unit 66 superimposes the reference image 82 onto the captured image 80.

[0076] Figure 8 shows the captured image 80 displayed on the display device 22 according to the first embodiment. In the example shown in Figure 8, the captured image 80 is an image of the soil removal area 7 captured by the imaging device 32. The captured image 80 shows an image of the soil removal area 7 in front of the bulldozer 4.

[0077] In the example shown in Figure 8, the captured image 80 includes an image of the edge 44 of the soil removal area 7. The captured image 80 includes an image of the front of the body 26 of the bulldozer 4. The captured image 80 includes an image of the dump truck 2 entering the soil removal area 7.

[0078] In the example shown in Figure 8, the captured image 80 includes an image of an obstacle 73 that obstructs the movement of the dump truck 2. In the example shown in Figure 8, the obstacle 73 is a rock present on the ground in the soil removal area 7.

[0079] A work image 81 showing work data related to dump truck 2 is superimposed on the captured image 80. The work image 81 is displayed on the display device 22 in a manner that highlights the work data. The work image 81 includes at least one of a graphic image, a line image, a symbol image, and an icon image. The work image 81 may also include at least one of a character image and a numerical image.

[0080] In the example shown in Figure 8, the work image 81 includes a travel path image 811 showing the travel path 69 on which the dump truck 2 travels. The travel path image 811 is a line-shaped image (band-shaped image) displayed on the display screen of the display device 22 so as to follow the travel path 69.

[0081] The work image 81 includes a switchback point image 812 showing the switchback point 71 of the dump truck 2. The switchback point image 812 is a circular image (point-like image) that is displayed on the display screen of the display device 22 so as to overlap the switchback point 71.

[0082] The work image 81 includes an image 813 showing the soil removal points 72 of the dump truck 2. Multiple soil removal points 72 are provided in the soil removal area 7. The soil removal point image 813 is a circular image displayed on the display screen of the display device 22 so as to overlap each of the multiple soil removal points 72. Note that the soil removal point image 813 is not limited to a circular image. The soil removal point image 813 may be, for example, a polygonal image or an icon image. The soil removal points 72 include unremoved points that indicate the unremoved position before soil removal work is performed, and removed points that indicate the removed position after soil removal work is performed. In the embodiment, the soil removal point image 813 includes an unremoved point image 813A showing the unremoved points and a removed point image 813B showing the removed points. The unremoved point image 813A is displayed on the display device 22 in the first display mode. The image 813B of the area where soil has been removed is displayed on the display device 22 in a second display mode that is different from the first display mode. The image 813A of the area where soil has not yet been removed is a ring image. The image 813B of the area where soil has been removed is a disk image obtained by filling in the ring image.

[0083] The work image 81 includes an edge image 814 showing the edge 44 of the soil removal area 7. The edge image 814 is a line-shaped image displayed on the display screen of the display device 22 so as to follow the edge 44 of the captured image 80. The edge image 814 is displayed so as to overlap the edge 44.

[0084] The work image 81 includes an obstacle image 815 that indicates the location of the obstacle 73. The obstacle image 815 is a triangular graphic image displayed on the display screen of the display device 22 so as to overlap the obstacle 73 in the captured image 80. The obstacle image 815 may be, for example, a circular graphic image or a graphic image of any polygonal shape. If the obstacle 73 is, for example, a building, the obstacle image 815 may be displayed so as to surround the building. The obstacle image 815 may be, for example, a frame-like image surrounding the building. Furthermore, if there is a no-entry area in the work site 1 where the dump truck 2 is prohibited from entering, the obstacle image 815 may be displayed so as to surround the no-entry area. The obstacle image 815 may be, for example, a frame-like image surrounding the no-entry area.

[0085] The work image 81 includes the names of the work machines and vehicles managed by the management system 10. In the example shown in Figure 8, character data 816 indicating the name of the dump truck 2 is displayed on the display device 22 as the work image 81. The character data 816 may be displayed so as to overlap the dump truck 2, or it may be displayed around the dump truck 2. If the captured image 80 includes the shovel 3 or motor grader 5, character data indicating the name of the shovel 3 or motor grader 5 may be displayed on the display device 22. If the captured image 80 includes the light vehicle 9, character data indicating the name of the light vehicle 9 may be displayed on the display device 22.

[0086] The work image 81 is superimposed on the captured image 80 using augmented reality (AR). When the position of the travel path 69 changes, the position of the travel path image 811 on the display screen of the display device 22 changes in accordance with the change in the position of the travel path 69. When the shape of the travel path 69 changes, the shape of the travel path image 811 on the display screen of the display device 22 changes in accordance with the change in the shape of the travel path 69. When the position of the switchback point 71 changes, the position of the switchback point image 812 on the display screen of the display device 22 changes in accordance with the change in the position of the switchback point 71. When the position of the soil removal point 72 changes, the position of the soil removal point image 813 on the display screen of the display device 22 changes in accordance with the change in the position of the soil removal point 72. The same applies to the edge image 814 and the obstacle image 815.

[0087] When the dump truck 2 performs soil removal work at an unremoved soil point, and the unremoved soil point changes to a soil-removed soil point, the soil-removed soil point image 813 on the display screen of the display device 22 changes from the unremoved soil point image 813A to the soil-removed soil point image 813B in accordance with the change in the state of the soil-removed soil point 72. In other words, the display form of the work image 81 changes in accordance with the change in the state of the work data. When the dump truck 2 performs soil removal work at soil-removed soil point 72, a soil removal completion signal indicating that the soil removal work has been completed is transmitted from the controller 17 of the dump truck 2 to the management device 11. The work data transmission unit 60 transmits the soil removal completion signal to the remote controller 23. The work image generation unit 64 can change the soil removal point image 813 from the unremoved soil point image 813A to the soil-removed soil point image 813B based on the soil removal completion signal.

[0088] A reference image 82 is superimposed on the captured image 80. The reference image 82 shows the width of the excavation blade 34 and the distance from the bulldozer 4. The reference image 82 includes a blade reference line 821 showing the width of the excavation blade 34 and a distance reference line 822 showing the distance from the bulldozer 4. The blade reference line 821 is displayed so as to extend forward from the left end and the right end of the excavation blade 34, respectively. The left blade reference line 821 and the right blade reference line 821 are parallel. The distance reference line 822 is displayed so as to connect the front end of the left blade reference line 821 and the front end of the right blade reference line 821. The distance reference line 822 is displayed at a position a specified distance (for example, 20 m) forward from the front end of the vehicle body 26.

[0089] <Image Display Method> Figure 9 is a flowchart illustrating the image display method according to the first embodiment. An image of the work site 1 is captured by the imaging device 32. The image of the work site 1 captured by the imaging device 32 is transmitted from the controller 18 of the bulldozer 4 to the remote controller 23. The image acquisition unit 62 acquires an image 80 showing the image of the work site 1 transmitted from the controller 18 of the bulldozer 4 (step S1).

[0090] The work data transmission unit 60 of the management device 11 transmits work data related to the dump truck 2 to the remote controller 23. The work data acquisition unit 63 acquires the work data related to the dump truck 2 transmitted from the management device 11 (step S2).

[0091] The work image generation unit 64 generates a work image 81 based on the work data acquired by the work data acquisition unit 63 (step S3). The reference image generation unit 65 generates a reference image 82 based on the specifications data of the bulldozer 4 (step S4).

[0092] As explained with reference to Figure 8, the display control unit 66 overlays the work image 81 and the reference image 82 onto the captured image 80 (step S5).

[0093] The display control unit 66 determines whether or not to terminate the image display (step S6). If it is determined in step S6 to continue the image display (step S6: No), the process returns to step S1. If it is determined in step S6 to terminate the image display (step S6: Yes), the image display terminates.

[0094] <Effects> As described above, in this embodiment, the processor 47 of the remote controller 23 includes an image acquisition unit 62 that acquires an image 80 showing an image of the work site 1 of the remotely operated bulldozer 4 captured by the imaging device 32, a work data acquisition unit 63 that acquires work data relating to the dump truck 2 working at the work site 1, a work image generation unit 64 that generates a work image 81 showing the work data, and a display control unit 66 that displays the work image 81 together with the image 80 on a display device 22 located outside the bulldozer 4.

[0095] According to this embodiment, a work image 81 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0096] The work image 81 is superimposed on the captured image 80 using augmented reality (AR). This allows the operator to smoothly recognize both the captured image 80 and the work image 81.

[0097] The work image 81 includes a travel path image 811 showing the travel path 69 of the dump truck 2. By displaying the travel path image 811 on the display device 22, the operator can recognize the travel route of the dump truck 2. The operator can recognize the location (route) where the dump truck 2 is scheduled to travel from the travel path image 811.

[0098] The work image 81 includes a switchback point image 812 showing the switchback point 71 of the dump truck 2. By displaying the switchback point image 812 on the display device 22, the operator can recognize the location where the switchback of the dump truck 2 is scheduled.

[0099] The work image 81 includes an image 813 showing the soil discharge point 72 of the dump truck 2. When the soil discharge point image 813 is displayed on the display device 22, the operator can recognize the location of the soil discharge work performed by the dump truck 2.

[0100] The soil removal point image 813 includes an unremoved soil removal point image 813A in a first display mode showing unremoved soil points, and a completed soil removal point image 813B in a second display mode showing completed soil removal points. By displaying the unremoved soil removal point image 813A and the completed soil removal point image 813B on the display device 22, the operator can recognize the location where soil removal work by the dump truck 2 is scheduled and the location where soil removal work by the dump truck 2 has been completed.

[0101] The work image 81 includes an edge image 814 showing the edge 44 of the soil removal area 7. By displaying the edge image 814 on the display device 22, the operator can recognize the position and shape of the edge 44. By displaying the edge image 814 on the display device 22, the operator can recognize the position of the descending cliff 43.

[0102] The work image 81 includes an obstacle image 815 showing the location of an obstacle 73 that obstructs the movement of the dump truck 2. The display of the obstacle 73 on the display device 22 allows the operator to recognize its location. The display of the obstacle image 815 on the display device 22 allows the operator to remotely control the bulldozer 4 to prevent it from contacting the obstacle 73.

[0103] In this embodiment, the reference image 82 is superimposed on the captured image 80. By superimposing the reference image 82 on the captured image 80, the operator can, for example, recognize the relative position between the excavation blade 34 and an object in the soil discharge area 7, or recognize the relative distance between the vehicle body 26 and an object in the soil discharge area 7.

[0104] [Second Embodiment] A second embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0105] Figure 10 shows the captured image 80 displayed on the display device 22 according to the second embodiment. In this embodiment, the work image generation unit 64 generates a window image 90 by superimposing the work image 91 onto the overhead view image 85 of the work site 1. The display control unit 66 superimposes the window image 90 onto the captured image 80. Superimposition display refers to a display method in which a window image 90, which is a secondary screen, is superimposed on a portion of the captured image 80, which is the main screen.

[0106] Figure 11 is an enlarged view of the window image 90 according to the second embodiment. In this embodiment, the work site 1 is surveyed in advance. Based on the survey data, two-dimensional data showing the two-dimensional shape of the terrain of the work site 1 is calculated. The two-dimensional data shows the two-dimensional shape of the terrain on a predetermined plane substantially parallel to the ground of the work site 1. The two-dimensional data may be defined by latitude and longitude coordinate data. The survey includes detecting the two-dimensional shape of the work site using a position sensor including a GNSS receiver. For example, by having a survey vehicle equipped with a position sensor travel along the edge of the work area, the work image generation unit 64 can calculate the two-dimensional data of the work site 1 based on the detection data from the position sensor.

[0107] Furthermore, three-dimensional data showing the three-dimensional shape of the terrain of the work site 1 may be calculated based on the survey data. The survey may include detecting the three-dimensional shape of the work site using a three-dimensional sensor. A three-dimensional sensor is a sensor capable of detecting the three-dimensional shape of an object to be detected. Examples of three-dimensional sensors include a laser sensor (LiDAR: Light Detection and Ranging) that detects the three-dimensional shape of an object by emitting laser light, and a stereo camera. For example, if an aircraft such as a drone equipped with a three-dimensional sensor flies over the work site 1, and the three-dimensional sensor mounted on the drone detects the work site 1, the work image generation unit 64 can calculate three-dimensional data of the work site 1 based on the detection data of the three-dimensional sensor. Based on the three-dimensional data of the work site 1, the work image generation unit 64 can generate an overhead view image 85 of the work site 1.

[0108] Window image 90 includes an overhead view image 85 of the soil removal area 7, a model image 2V of the dump truck 2, a model image 4V of the bulldozer 4, and a work image 91. Model image 4V of the bulldozer 4 includes a model image 34V of the excavation blade 34. Model image 2V is an image modeling the dump truck 2 as seen from above. Model image 4V is an image modeling the bulldozer 4 as seen from above.

[0109] When dump truck 2 travels in the soil removal area 7, the model image 2V moves in the window image 90 in accordance with the movement of dump truck 2. The work image generation unit 64 can move the model image 2V in the window image 90 based on the detection data of the position sensor 55 and orientation sensor 56 of dump truck 2. When bulldozer 4 travels in the soil removal area 7, the model image 4V moves in the window image 90 in accordance with the movement of bulldozer 4. The work image generation unit 64 can move the model image 4V in the window image 90 based on the detection data of the position sensor 30 and attitude sensor 31 of bulldozer 4.

[0110] The work image 91 is superimposed on the overhead image 85. The work image 91 includes a travel path image 911 showing the travel path 69 on which the dump truck 2 travels, a switchback point image 912 showing the switchback point 71 of the dump truck 2, a soil discharge point image 913 showing the soil discharge point 72 of the dump truck 2, and an edge image 914 showing the edge 44 of the soil discharge area 7. The soil discharge point image 913 includes an undischarged point image 913A showing the undischarged point and a completed soil discharge point image 913B showing the completed soil discharge point.

[0111] As described above, in this embodiment as well, the work image 91 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0112] [Third Embodiment] A third embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0113] Figure 12 shows an image capture 80 and a window image 90 according to the third embodiment. In this embodiment, a plurality of display devices 22 are arranged side by side in the remote control room 20. In the example shown in Figure 12, the display devices 22 include a first display device 22A and a second display device 22B arranged next to the first display device 22A.

[0114] Similar to the second embodiment described above, the work image generation unit 64 generates a window image 90 by superimposing the work image 91 onto the overhead view image 85 of the work site 1. As shown in Figure 12, the display control unit 66 may display the captured image 80 on the first display device 22A and the window image 90 on the second display device 22B. In the example shown in Figure 12, the reference image 82 is displayed superimposed on the captured image 80.

[0115] As described above, in this embodiment as well, the work image 91 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0116] [Fourth Embodiment] A fourth embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0117] Figure 13 shows an image capture 80 and a window image 90 according to the fourth embodiment. In this embodiment, a plurality of display devices 22 are arranged side by side in the remote control room 20. In the example shown in Figure 13, the display devices 22 include a first display device 22A and a second display device 22B arranged next to the first display device 22A.

[0118] As shown in Figure 13, the display control unit 66 causes the captured image 80, on which the work image 81 is superimposed, to be displayed on the first display device 22A. The work image generation unit 64 generates a window image 90 on which the work image 91 is superimposed on the overhead view image 85 of the work site 1, and the display control unit 66 causes the window image 90 to be displayed on the second display device 22B. In the example shown in Figure 13, the reference image 82 is superimposed on the captured image 80.

[0119] As described above, in this embodiment as well, the work image 91 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0120] [Fifth Embodiment] The fifth embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0121] Figure 14 is a schematic diagram showing the soil removal area 7 according to the fifth embodiment. As shown in Figure 14, the dump truck 2 may perform soil removal work so that multiple piles 46 are formed on the ground of the soil removal area 7. The multiple piles 46 are formed regularly at intervals in the soil removal area 7. The bulldozer 4 can perform leveling work to smooth out the multiple piles 46 formed on the ground of the soil removal area 7. Even when multiple piles 46 are formed on the ground of the soil removal area 7, the operator in the remote control room 20 can recognize the work status of the dump truck 2 by displaying the work image 81 showing the work data of the dump truck 2 together with the captured image 80 on the display device 22. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4 remotely controlled by the operator is suppressed.

[0122] [Other Embodiments] In the embodiments described above, the shovel 3, bulldozer 4, and motor grader 5 are all manned work machines. At least one of the shovel 3, bulldozer 4, and motor grader 5 may be an unmanned work machine. An unmanned work machine is a work machine that performs work without operator operation. When at least one of the shovel 3, bulldozer 4, and motor grader 5 travels according to a travel path, a travel path image showing the travel path of at least one of the shovel 3, bulldozer 4, and motor grader 5 may be displayed on the display device 22.

[0123] In the above-described embodiment, the light vehicle 9 is assumed to be a manned vehicle. The light vehicle 9 may also be an unmanned vehicle. When the light vehicle 9 travels according to the travel path, a travel path image showing the travel path of the light vehicle 9 may be displayed on the display device 22.

[0124] In the above embodiment, the reference image 82 does not need to be displayed on the display device 22.

[0125] In the above-described embodiment, the bulldozer 4 working in the soil removal area 7 is a bulldozer. However, the bulldozer 4 working in the soil removal area 7 may be other work machines such as shovels, wheel loaders, and motor graders.

[0126] In the above-described embodiment, at least a portion of the functional parts of the remote controller 23 may be provided in the management device 11. At least a portion of the functional parts of the management device 11 may be provided in the remote controller 23. For example, the work image generation unit 64 and the reference image generation unit 65 may be provided in the management device 11. The edge data generation unit 58 and the driving data generation unit 59 may be provided in the remote controller 23.

[0127] In the embodiments described above, each of the multiple functional units of the remote controller 23 may be configured by a separate computer (hardware). Each of the functional units of the management device 11 may be configured by a separate computer (hardware). The multiple functional units of the remote controller 23 and the multiple functional units of the management device 11 may be configured by a single computer (hardware).

[0128] 1...Work site, 2...Dump truck (unmanned vehicle), 2V...Model image, 3...Excavator (working machine), 4...Bulldozer (working machine), 4V...Model image, 5...Motor grader (working machine), 6...Loading area, 7...Soil removal area, 8...Transportation route, 9...Light vehicle, 10...Management system, 11...Management device, 12...Communication system, 12A...Wireless communication device, 12B...Wireless communication device, 12C...Wireless communication device, 13...Control facility, 14...Vehicle body, 15...Running gear, 16...Dump body, 17...Controller, 18...Controller, 19...Remote control system, 20...Remote control room, 2 1...Remote control device, 22...Display device, 22A...First display device, 22B...Second display device, 23...Remote controller, 24...Driver's seat, 25...Communication system, 26...Vehicle body, 27...Running gear, 28...Excavating work machine, 29...Ripper work machine, 30...Position sensor, 31...Attitude sensor, 32...Imaging device, 33...Track, 34...Excavating blade, 34A...Cutting edge, 34V...Model image, 35...Lift frame, 36...Tilt cylinder, 37...Lift cylinder, 38...Shank, 38A...Ripper point, 39...Ripper arm, 40...Tilt cylinder, 41...Lift cylinder, 4 2...Beam, 43...Downward cliff, 44...Edge, 45...Embankment, 46...Ground, 47...Processor, 48...Main memory, 49...Storage, 50...Input / Output interface, 51...Communication interface, 52...Computer program, 53...Position sensor, 54...Controller, 55...Position sensor, 56...Direction sensor, 57...Speed ​​sensor, 58...Edge data generation unit, 59...Driving data generation unit, 60...Work data transmission unit, 61...Operation signal transmission unit, 62...Image acquisition unit, 63...Work data acquisition unit, 64...Work image generation unit, 65...Reference image generation unit, 66...Display Control unit, 68...Survey line, 69...Travel path, 70...Travel point, 70E...Departure point, 70S...Entry point, 71...Switchback point, 72...Earth removal point, 73...Obstacle, 80...Acquired image, 81...Work image, 811...Travel path image, 812...Switchback point image, 813...Earth removal point image, 813A...Unremoved earth point image, 813B...Earth removal completed point image, 814...Edge image, 815...Obstacle image, 816...Text data, 82...Reference image, 821...Blade reference line, 822...Distance reference line, 85...Overhead view image, 90...Window image, 91...Work image, 911...Travel path image,912... Switchback point image, 913... Soil removal point image, 913A... Unremoved soil point image, 913B... Soil removal completed point image, 914... Edge image.

Claims

1. A remote control system for a work machine, comprising a processor, the processor acquiring an image showing a work site of a remotely controlled work machine captured by an imaging device, acquiring work data relating to an unmanned vehicle working at the work site, generating a work image showing the work data, and displaying the work image together with the image on a display device located outside the work machine.

2. The remote control system for a work machine according to claim 1, wherein the work data includes the travel route on which the unmanned vehicle travels.

3. The remote control system for a work machine according to claim 1, wherein the unmanned vehicle is a transport vehicle for transporting cargo, and the work data includes a soil removal position in which the unmanned vehicle performs soil removal work on the cargo.

4. The remote control system for a work machine according to claim 3, wherein the soil removal position includes an un-soiled position before the soil removal work is performed and a soil-removed position after the soil removal work is performed, and the work image includes an un-soiled point image in a first display mode showing the un-soiled position and a soil-removed point image in a second display mode showing the soil-removed position.

5. The remote control system for a work machine according to claim 1, wherein the work data includes the edges of the work area in which the unmanned vehicle can perform work.

6. The remote control system for a work machine according to claim 1, wherein the work data includes the location of obstacles that obstruct the work of the unmanned vehicle.

7. The remote control system for a work machine according to claim 1, wherein the processor overlays the work image onto the captured image.

8. The remote control system for a work machine according to claim 7, wherein the work machine has a drilling blade, the processor generates a reference image indicating the width of the drilling blade and the distance from the work machine, and displays the reference image superimposed on the captured image.

9. The remote control system for a work machine according to claim 1, wherein the processor generates a window image by superimposing the work image onto an overhead view image of the work site, and displays the window image superimposed on the captured image.

10. A remote control system for a work machine according to claim 1, wherein a remote control device for remotely operating the work machine is located in a remote control room, a plurality of the display devices are arranged side by side, and the processor generates a window image by superimposing the work image onto an overhead view image of the work site, displays the captured image on the first display device, and displays the window image on the second display device.

11. A method for remotely operating a work machine, comprising: acquiring an image showing a work site of a remotely operated work machine captured by an imaging device; acquiring work data relating to an unmanned vehicle working at the work site; generating a work image showing the work data; and displaying the work image together with the image on a display device located outside the work machine.

12. The remote operation method for a work machine according to claim 11, wherein the work data includes the travel route on which the unmanned vehicle travels.

13. The remote operation method for a work machine according to claim 11, wherein the unmanned vehicle is a transport vehicle for transporting cargo, and the work data includes a soil removal position in which the unmanned vehicle performs soil removal work on the cargo.

14. The remote operation method for a work machine according to claim 13, wherein the soil removal position includes an un-soiled position before the soil removal work is performed and a soil-removed position after the soil removal work is performed, and the work image includes an un-soiled point image in a first display mode showing the un-soiled position and a soil-removed point image in a second display mode showing the soil-removed position.

15. The remote control method for a work machine according to claim 11, wherein the work data includes the edges of the work area in which the unmanned vehicle can perform work.

16. The remote operation method for a work machine according to claim 11, wherein the work data includes the location of obstacles that obstruct the work of the unmanned vehicle.

17. A method for remotely operating a work machine according to claim 11, comprising superimposing the work image onto the captured image.

18. A remote operation method for a work machine according to claim 17, comprising: generating a reference image indicating the width of the drilling blade and the distance from the work machine; and superimposing the reference image onto the captured image.

19. A method for remotely operating a work machine according to claim 11, comprising: generating a window image by superimposing the work image onto an overhead view image of the work site; and displaying the window image superimposed on the captured image.

20. A method for remotely operating a work machine according to claim 11, wherein a remote control device for remotely operating the work machine is located in a remote control room where a plurality of the display devices are arranged side by side, and the method includes generating a window image by superimposing the work image onto an overhead view image of the work site, and displaying the captured image on the first display device and displaying the window image on the second display device.

Images