Remote control system for industrial machinery and method for remote control of industrial machinery

Abstract

To prevent a decline in work efficiency. [Solution] The remote control system for the work machine includes a processor. The processor acquires captured images showing the work site of the remotely controlled work machine captured by an imaging device, acquires work data related to the unmanned vehicle working at the work site, generates a work image showing the work data, and displays the work image together with the captured image on a display device located outside the work machine.

Description

Technical Field

[0001] The present disclosure relates to a remote operation system for a working machine and a method for remotely operating a working machine.

Background Art

[0002] In the technical field related to remote operation systems for working machines, a remote operation system as disclosed in Patent Document 1 is known. In the remote operation system, the work site of the working machine is imaged by an imaging device. The captured image of the work site captured by the imaging device is displayed on a display device arranged in the remote operation room. An operator in the remote operation room remotely operates the working machine while checking the captured image of the work site displayed on the display device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There are cases where unmanned vehicles work at the work site. In the remote operation of the working machine, if it becomes difficult for the operator to recognize the working status of the unmanned vehicle, the working efficiency of the working machine may decrease.

[0005] The present disclosure aims to suppress a decrease in working efficiency.

Means for Solving the Problems

[0006] This disclosure provides a remote control system for a work machine. The remote control system includes a processor. The processor acquires captured images showing the work site of the remotely controlled work machine captured by an imaging device, acquires work data relating to an unmanned vehicle working at the work site, generates a work image showing the work data, and displays the work image together with the captured image on a display device located outside the work machine. [Effects of the Invention]

[0007] According to this disclosure, the decline in work efficiency is suppressed. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a schematic diagram showing a work site according to the first embodiment. [Figure 2] Figure 2 is a schematic diagram showing the management system and remote control system according to the first embodiment. [Figure 3] Figure 3 is a schematic side view showing a bulldozer according to the first embodiment. [Figure 4] Figure 4 is a schematic diagram showing the soil removal area according to the first embodiment. [Figure 5] Figure 5 is a hardware configuration diagram showing a remote controller according to the first embodiment. [Figure 6] Figure 6 is a block diagram showing the management system and remote control system according to the first embodiment. [Figure 7] Figure 7 is a diagram illustrating the driving data according to the first embodiment. [Figure 8] Figure 8 shows an image captured and displayed on the display device according to the first embodiment. [Figure 9] Figure 9 is a flowchart showing the image display method according to the first embodiment. [Figure 10] Figure 10 shows an image captured and displayed on the display device according to the second embodiment. [Figure 11] Figure 11 is an enlarged view of the window image according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing a captured image and a window image according to the third embodiment. [Figure 13] FIG. 13 is a diagram showing a captured image and a window image according to the fourth embodiment. [Figure 14] FIG. 14 is a diagram schematically showing a waste dump area according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be appropriately combined. Also, some components may not be used.

[0010] [First Embodiment] The first embodiment will be described.

[0011] [Work Site] FIG. 1 is a diagram schematically showing a work site 1 according to the first embodiment. As the work site 1, a mine or a quarry is exemplified. A mine refers to a place or establishment for mining minerals. A quarry refers to a place or establishment for mining stone materials. Examples of mines include metal mines for mining metals, non-metal mines for mining limestone, and coal mines for mining coal.

[0012] In the work site 1, an unmanned vehicle works. An unmanned vehicle refers to a work vehicle that performs work without a driver's operation. In the embodiment, the unmanned vehicle is a transport vehicle that performs a transport operation of transporting a load. In the embodiment, a dump truck 2, which is an example of a transport vehicle, works at the work site 1. The dump truck 2 has a dump body. The dump truck 2 can perform a transport operation by loading a load on the dump body. The dump truck 2 can perform a waste dump operation of discharging the load from the dump body.

[0013] At work site 1, a plurality of work machines operate. In an embodiment, the work machines are remotely operated by an operator. Examples of work machines include a loading machine that performs a loading operation of loading a load onto a dump truck 2, and an excavating machine that performs an excavating operation of excavating at least a part of work site 1. The excavating machine can perform one or both of a bank formation operation of forming a bank at work site 1 and a leveling operation of leveling the ground at work site 1.

[0014] In an embodiment, a shovel 3, which is an example of a loading machine, a bulldozer 4, and a motor grader 5, which are examples of an excavating machine, operate at work site 1. The shovel 3 has a working machine. The shovel 3 can perform a loading operation and an excavating operation using the working machine. Each of the bulldozer 4 and the motor grader 5 has a working machine. Each of the bulldozer 4 and the motor grader 5 can perform a leveling operation using the working machine.

[0015] A loading area 6, a spoil area 7, and a conveyance path 8 are provided at work site 1. Each of the loading area 6, the spoil area 7, and the conveyance path 8 is a work area where each of the driverless vehicle and the work machine can operate.

[0016] The loading area 6 refers to a work area where a loading operation of loading a load onto the dump truck 2 is performed. Examples of the load include excavated material excavated in the loading area 6. The shovel 3 performs an excavating operation and a loading operation in the loading area 6.

[0017] The spoil area 7 refers to a work area where a spoil discharging operation in which the dump truck 2 discharges a load is performed. The bulldozer 4 performs a bank formation operation and a leveling operation in the spoil area 7. The bulldozer 4 may perform a bank formation operation and a leveling operation in the loading area 6.

[0018] The transport path 8 refers to the road on which the dump truck 2 travels. The transport path 8 leads to the loading area 6 and the soil removal area 7, respectively. The transport path 8 is provided to connect at least the loading area 6 and the soil removal area 7. The dump truck 2 traveling towards at least one of the loading area 6 and the soil removal area 7 travels along the transport path 8. The dump truck 2 travels along the transport path 8, for example, by going back and forth between the loading area 6 and the soil removal area 7. The motor grader 5 performs leveling work on the transport path 8.

[0019] A light vehicle 9 travels within the work site 1. A light vehicle 9 is a lightweight vehicle that travels within the work site 1. The external dimensions of the light vehicle 9 are smaller than those of the dump truck 2. The weight of the light vehicle 9 is lighter than that of the dump truck 2. The light vehicle 9 can travel within the loading area 6, the soil removal area 7, and the transport route 8. The light vehicle 9 travels to inspect the work site 1 and to transport workers. The light vehicle 9 may be an unmanned vehicle or a manned vehicle. A manned vehicle is a vehicle that travels based on driving operations by a driver. In this embodiment, the light vehicle 9 is a manned vehicle.

[0020] Each of the excavator 3, bulldozer 4, and motor grader 5 may be an unmanned or manned work machine. In this embodiment, each of the excavator 3, bulldozer 4, and motor grader 5 is a manned work machine. A manned work machine is a work machine that performs work based on the operation of an operator.

[0021] <Management system and remote control system> Figure 2 is a schematic diagram showing the management system 10 and remote control system 19 according to the first embodiment. The management system 10 manages the work site 1. The management system 10 manages each of the dump truck 2, excavator 3, bulldozer 4, motor grader 5, and light vehicle 9. In Figure 2, the bulldozer 4 is shown as a work machine. In the following description, for the sake of simplicity, an example will be given in which the management system 10 manages the dump truck 2 and the bulldozer 4.

[0022] The management system 10 comprises a management device 11 and a communication system 12. The management device 11 includes a computer. The management device 11 is located outside the dump truck 2 and the bulldozer 4. The management device 11 is installed in the control facility 13 of the work site 1. The management device 11 manages the work site 1. The management device 11 manages at least the dump truck 2 and the bulldozer 4. The communication system 12 includes at least one of the following: the internet, a mobile phone network, a satellite network, and a local area network (LAN).

[0023] The dump truck 2 comprises a body 14, a running gear 15, a dump body 16, a controller 17, and a wireless communication device 12A. The body 14 includes a body frame. The body 14 is supported by the running gear 15. The running gear 15 supports the body 14 and drives. The running gear 15 includes wheels, tires mounted on the wheels, an engine, a brake system, and a steering system. The dump body 16 is the component into which the cargo is loaded. The dump body 16 is supported by the body 14. The dump body 16 performs dumping and lowering operations. Dumping operation refers to the operation of moving the dump body 16 away from the body 14 and tilting it in the dumping direction. Lowering operation refers to the operation of moving the dump body 16 closer to the body 14. When loading work is performed, the dump body 16 performs a lowering operation. When soil removal work is performed, the dump body 16 performs a dumping operation. The controller 17 includes a computer. The wireless communication device 12A is connected to the controller 17.

[0024] The bulldozer 4 has a controller 18 and a wireless communication device 12B. The controller 18 includes a computer. The wireless communication device 12B is connected to the controller 18.

[0025] The communication system 12 includes a wireless communication device 12A connected to the controller 17 of the dump truck 2, a wireless communication device 12B connected to the controller 18 of the bulldozer 4, and a wireless communication device 12C connected to the management device 11. The management device 11 and the controller 17 of the dump truck 2 communicate wirelessly via the communication system 12. The management device 11 and the controller 18 of the bulldozer 4 communicate wirelessly via the communication system 12.

[0026] In this embodiment, the bulldozer 4 is remotely controlled by a remote control system 19. At least a portion of the remote control system 19 is located in a remote control room 20. The remote control room 20 is located outside the bulldozer 4. The remote control room 20 is installed in a remote location away from the work site 1. The remote control system 19 comprises a remote control device 21, a display device 22, and a remote controller 23.

[0027] The remote control device 21 is located in the remote control room 20. The remote control device 21 is operated by an operator in the remote control room 20. When the remote control device 21 is operated, an operation signal is generated to operate the bulldozer 4. The operator can operate the remote control device 21 while seated in the driver's seat 24.

[0028] The display device 22 is located in the remote control room 20. The display device 22 displays captured images of the work site 1. The display device 22 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD). The operator operates the remote control device 21 while checking the captured images of the work site 1 displayed on the display device 22. The bulldozer 4 is remotely controlled by the remote control device 21.

[0029] The remote controller 23 is located in the remote control room 20. The remote controller 23 includes a computer. The remote controller 23 and the controller 18 of the bulldozer 4 communicate via a communication system 25. The communication system 25 includes at least one of the following: the internet, a mobile phone network, a satellite network, and a local area network (LAN).

[0030] The management device 11 and the remote controller 23 communicate via the communication system 25. Note that communication system 12 and communication system 25 may be separate communication systems. At least a portion of communication system 12 and communication system 25 may be the same communication system.

[0031] <Bulldozer> Figure 3 is a schematic side view showing a bulldozer 4 according to the first embodiment. As shown in Figure 3, the bulldozer 4 comprises a vehicle body 26, a traveling device 27, an excavating workpiece 28, a ripper workpiece 29, a position sensor 30, an attitude sensor 31, and an imaging device 32.

[0032] The vehicle body 26 supports an engine (not shown). The engine is the power source for the bulldozer 4. The running gear 27 supports the vehicle body 26 and moves. The running gear 27 has a pair of tracks 33. The bulldozer 4 moves as the tracks 33 rotate.

[0033] The excavation work machine 28 performs excavation work, embankment work, soil pushing work, embankment formation work, and land leveling work. The excavation work machine 28 is connected to the vehicle body 26. At least a portion of the excavation work machine 28 is positioned in front of the vehicle body 26. The excavation work machine 28 has an excavation blade 34, a lift frame 35, a tilt cylinder 36, and a lift cylinder 37.

[0034] The drilling blade 34 is positioned in front of the vehicle body 26. The drilling blade 34 has a cutting edge 34A. The lift frame 35 supports the drilling blade 34. One end of the lift frame 35 is connected to the rear of the drilling blade 34 via a pivot mechanism. The other end of the lift frame 35 is connected to the side of the traveling device 27 via a pivot mechanism. The lift frame 35 includes a first lift frame 35 connected to the left side of the rear of the drilling blade 34 and a second lift frame 35 connected to the right side of the rear of the drilling blade 34.

[0035] The tilt cylinder 36 and the lift cylinder 37 each operate the drilling blade 34. The tilt cylinder 36 is driven to tilt the drilling blade 34. The lift cylinder 37 is driven to move the drilling blade 34 up and down. One end of the tilt cylinder 36 is connected to the back of the drilling blade 34 via a pivot mechanism. The other end of the tilt cylinder 36 is connected to the upper surface of the lift frame 35. The tilt angle of the drilling blade 34 changes as the tilt cylinder 36 extends and retracts. One end of the lift cylinder 37 is connected to the back of the drilling blade 34 via a pivot mechanism. The middle part of the lift cylinder 37 is connected to the side of the vehicle body 26. The drilling blade 34 moves up and down as the lift cylinder 37 extends and retracts.

[0036] The ripper implement 29 performs ripping operations, such as cutting or crushing the workpiece. For example, it can be used for crushing rocks. The ripper implement 29 is connected to the vehicle body 26. At least a portion of the ripper implement 29 is positioned at the rear of the vehicle body 26. The ripper implement 29 has a shank 38, a ripper arm 39, a tilt cylinder 40, a lift cylinder 41, and a beam 42.

[0037] The shank 38 is located at the rear of the vehicle body 26. The shank 38 has a ripper point 38A. The ripper point 38A is located at the tip (lower end) of the shank 38, including the cutting edge position of the ripper. The ripper arm 39 supports the shank 38. The ripper arm 39 connects the vehicle body 26 and the shank 38. One end of the ripper arm 39 is connected to the rear of the vehicle body 26 via a pivot mechanism. The other end of the ripper arm 39 is connected to a beam 42. The beam 42 is rotatably connected to the ripper arm 39. The shank 38 is connected to the ripper arm 39 via the beam 42.

[0038] The tilt cylinder 40 and the lift cylinder 41 each operate the shank 38. The tilt cylinder 40 and the lift cylinder 41 are each connected to the vehicle body 26. The tilt cylinder 40 drives the shank 38 to tilt. The lift cylinder 37 drives the shank 38 to move up and down. One end of the tilt cylinder 40 is connected to the beam 42 via a pivot mechanism. The other end of the tilt cylinder 36 is connected to the rear of the vehicle body 26. As the tilt cylinder 40 extends and retracts, the tilt angle of the shank 38 changes. The tilt cylinder 40 moves the ripper point 38A in the front-rear direction. One end of the lift cylinder 41 is connected to the beam 42 via a pivot mechanism. The other end of the lift cylinder 41 is connected to the rear of the vehicle body 26. As the lift cylinder 41 extends and retracts, the shank 38 moves in the up-and-down direction. The lift cylinder 41 moves the shank 38, which includes the ripper point 38A, in the vertical direction.

[0039] The ripper implement 29 pierces the workpiece with its ripper point 38A. While the traveling device 27 is stopped, the ripper point 38A is brought into contact with the workpiece, and as the traveling device 27 moves with the ripper point 38A embedded in the workpiece, the workpiece is cut or crushed. While the traveling device 27 is moving, the ripper implement 29 is moved in the up, down, forward, and backward directions, bringing it into contact with the workpiece, and cutting or crushing occurs as the device moves with the workpiece embedded in it.

[0040] The position sensor 30 detects the position of the bulldozer 4. The position sensor 30 is located on the vehicle body 26. The position of the bulldozer 4 is detected using the Global Navigation Satellite System (GNSS). The Global Navigation Satellite System includes the Global Positioning System (GPS). The Global Navigation Satellite System detects the position in a global coordinate system defined by latitude, longitude, and altitude coordinate data. A global coordinate system is a coordinate system fixed to the Earth. The position sensor 30 includes a GNSS receiver located on the vehicle body 26. The position sensor 30 detects the position of the bulldozer 4 in the global coordinate system.

[0041] The attitude sensor 31 detects the attitude of the bulldozer 4. The attitude sensor 31 detects the attitude of the bulldozer 4 at least when the position sensor 30 has detected the position of the bulldozer 4. The attitude sensor 31 is positioned on the vehicle body 26. The attitude of the bulldozer 4 includes the tilt of the vehicle body 26. The attitude of the bulldozer 4 includes the tilt angle of the vehicle body 26 with respect to the horizontal plane. An inertial measurement unit (IMU) is exemplified as the attitude sensor 31. The attitude sensor 31 is capable of detecting the tilt angle of the vehicle body 26 with respect to the horizontal plane.

[0042] The imaging device 32 captures images of the target to be imaged. The imaging device 32 is positioned on the vehicle body 26. The target to be imaged by the imaging device 32 includes the work site 1 of the bulldozer 4. The target to be imaged by the imaging device 32 includes the ground of the work site 1 surrounding the bulldozer 4 and objects surrounding the bulldozer 4. The captured images showing the images of the target to be imaged captured by the imaging device 32 are displayed on the display device 22 in the remote control room 20. An RGB camera is given as an example of the imaging device 32. However, the imaging device 32 may also be an RGB-D camera or a hyperspectral camera.

[0043] <Soil Disposal Area> Figure 4 is a schematic diagram showing the soil removal area 7 according to the first embodiment. The soil removal area 7 is an example of a work area where a dump truck 2 and a bulldozer 4 can operate at the work site 1. The dump truck 2 can travel through the soil removal area 7. The ground in the soil removal area 7 is substantially flat. The dump truck 2 performs soil removal work in the soil removal area 7. A sloping cliff 43 exists around the soil removal area 7. The sloping cliff 43 is located outside the edge 44 of the soil removal area 7. The sloping cliff 43 is a prohibited area where the dump truck 2 and the bulldozer 4 are prohibited from working at the work site 1. The dump truck 2 and the bulldozer 4 cannot travel on the sloping cliff 43. The soil removal area 7 and the sloping cliff 43 are separated by the edge 44 of the soil removal area 7. In this embodiment, a bank 45 is formed on the edge 44. The bank 45 protrudes upward from the ground of the soil removal area 7 near the edge 44.

[0044] During the soil removal operation, the dump truck 2 approaches the edge 44 of the soil removal area 7 while reversing. The controller 17 of the dump truck 2 starts the dumping operation of the dump body 16 after the dump truck 2 has approached the edge 44 of the soil removal area 7. As the dump truck 2 approaches the edge 44 while reversing, the dump body 16 starts the dumping operation, causing the load loaded on the dump body 16 to be removed down the cliff 43.

[0045] The height of the embankment 45 is set so as not to obstruct the soil removal work of the dump truck 2, and so as not to make it difficult for the dump truck 2 to drive over the embankment 45. The height of the embankment 45 is, for example, lower than the radius of the tires of the dump truck 2. Since the embankment 45 is formed on the edge 44, the dump truck 2 performing soil removal work is prevented from going outside the soil removal area 7.

[0046] Bulldozer 4 performs embankment formation work to create an embankment 45 in the soil removal area 7. Bulldozer 4 also performs leveling work to flatten the ground in the soil removal area 7. For example, unevenness may be formed on the ground in the soil removal area 7 due to the passage of dump truck 2 or the dropping of cargo from the dump body 16. Bulldozer 4 performs leveling work to make the ground in the soil removal area 7 flat.

[0047] <Computer> Figure 5 is a hardware diagram showing a remote controller 23 according to the first embodiment. The remote controller 23 includes a computer. The remote controller 23 has a processor 47 such as a CPU (Central Processing Unit), a main memory 48 including non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory), a storage 49, an input / output interface 50 including input / output circuits, and a communication interface 51 including communication circuits. The functions of the remote controller 23 are stored in the storage 49 as a computer program 52. The processor 47 reads the computer program 52 from the storage 49, loads it into the main memory 48, and executes processing according to the computer program 52. The computer program 52 may be distributed to the remote controller 23 via a network.

[0048] Similar to the remote controller 23, the management device 11, the controller 17 of the dump truck 2, and the controller 18 of the bulldozer 4 each include a computer. The management device 11, the controller 17 of the dump truck 2, and the controller 18 of the bulldozer 4 each also have a processor, main memory, storage for storing computer programs, an input / output interface, and a communication interface.

[0049] Figure 6 is a block diagram showing the management system 10 and remote control system 19 according to the first embodiment.

[0050] The light vehicle 9 includes a position sensor 53 and a controller 54. The position sensor 53 detects the position of the light vehicle 9. The position sensor 53 includes a GNSS receiver located on the light vehicle 9. The position sensor 53 detects the position of the light vehicle 9 in a global coordinate system. The controller 54 transmits the detection data from the position sensor 53 to the management device 11 via the communication system 12.

[0051] The dump truck 2 includes a running gear 15, a dump body 16, a position sensor 55, a compass sensor 56, a speed sensor 57, and a controller 17.

[0052] The position sensor 55 detects the position of the dump truck 2. The position sensor 55 includes a GNSS receiver located on the dump truck 2. The position sensor 55 detects the position of the dump truck 2 in a global coordinate system.

[0053] The orientation sensor 56 detects the orientation of the dump truck 2. The orientation of the dump truck 2 includes the azimuth angle relative to a reference azimuth. The reference azimuth is, for example, north. An inertial measurement unit (IMU) is exemplified as the orientation sensor 56. The orientation sensor 56 may also include a calculator that calculates the orientation from position data detected by two GNSS antennas installed on the dump truck 2. The calculator can calculate the orientation from a vector connecting the two GNSS antennas. The orientation sensor 56 may also detect the tilt angle of the dump truck 2 with respect to the horizontal plane. If the orientation sensor 56 is an inertial measurement unit (IMU), it can detect the tilt angle of the dump truck 2 with respect to the horizontal plane.

[0054] The speed sensor 57 detects the travel speed of the dump truck 2. The speed sensor 57 detects the travel speed of the dump truck 2 by, for example, detecting the rotational speed of the drive shaft connected to the wheels of the running gear 15.

[0055] The processor 47 of the management device 11 has multiple functional units. The functional units of the processor 47 of the management device 11 include an edge data generation unit 58, a driving data generation unit 59, and a work data transmission unit 60.

[0056] The edge data generation unit 58 generates edge data for the work area. The edge data includes the shape and position of the edges of the work area. The position of the edges is defined in a global coordinate system. As described above, the work area includes the loading area 6, the soil removal area 7, and the transport path 8. In this embodiment, the edge data includes at least the shape and position of the edges 44 of the soil removal area 7. The edge data generation unit 58 generates edge data based on the detection data from the position sensor 53 of the light vehicle 9.

[0057] For surveying work site 1, the light vehicle 9 travels along the edge of the work area. The position of the light vehicle 9 as it travels along the edge of the work area is detected by the position sensor 53. The detection data from the position sensor 53 is transmitted from the controller 54 of the light vehicle 9 to the management device 11. The edge data generation unit 58 can generate edge data based on the detection data from the position sensor 53.

[0058] As explained with reference to Figure 4, a bank 45 is formed at edge 44. When generating edge data for the soil removal area 7, the light vehicle 9 travels along the bank 45 through the soil removal area 7. The position of edge 44 may also be the lower end of the bank 45 on the soil removal area 7 side. The position of edge 44 may also be the slope of the bank 45 on the soil removal area 7 side. In the following description, the virtual line generated along edge 44 will be referred to as the survey line 68 as appropriate. The edge data includes the survey line 68.

[0059] The driving data generation unit 59 generates driving data indicating the driving conditions of the dump truck 2 in the work area. The driving data generation unit 59 generates driving data of the dump truck 2 in at least the soil discharge area 7. The driving data generation unit 59 generates driving data of the dump truck 2 based on edge data. The driving conditions of the dump truck 2 include the target position of the dump truck 2, the target direction of the dump truck 2, the target driving speed of the dump truck 2, and the target turning radius of the dump truck 2.

[0060] Figure 7 is a diagram illustrating the driving data according to the first embodiment. Figure 7 shows the driving data set in the soil removal area 7. The driving data defines the driving conditions of the dump truck 2. The driving data for the dump truck 2 includes the driving point 70, the driving path 69, the target position of the dump truck 2, the target direction of the dump truck 2, and the target driving speed of the dump truck 2.

[0061] Multiple travel points 70 are set in the soil removal area 7. Travel points 70 define the target position of the dump truck 2. The target position of the dump truck 2 is defined in the global coordinate system. For each of the multiple travel points 70, the target bearing and target travel speed of the dump truck 2 are set. The multiple travel points 70 are set at intervals. The intervals between travel points 70 may be uniform or uneven.

[0062] The target position of dump truck 2 refers to the target position of dump truck 2 when it passes through point 70. The target heading of dump truck 2 refers to the target heading of dump truck 2 when it passes through point 70. The target speed of dump truck 2 refers to the target speed of dump truck 2 when it passes through point 70. The driving path 69 refers to a virtual line indicating the target driving route of dump truck 2. The driving path 69 is defined by a trajectory that passes through multiple points 70.

[0063] As shown in Figure 7, of the multiple travel points 70, one travel point 70 is set as an entry point 70S, and one travel point 70 is set as an exit point 70E. In addition, of the multiple travel points 70, at least one travel point 70 is set as a switchback point 71, and at least one travel point 70 is set as an earth removal point 72. The switchback point 71 is the position where the dump truck 2 performs a switchback. A switchback is the action in which the forward-moving dump truck 2 makes a sharp change in direction and moves in reverse. The earth removal point 72 is the position where the dump truck 2 performs the earth removal operation to discharge its cargo.

[0064] The soil removal point 72 is set based on edge data (survey line 68). In this embodiment, the soil removal point 72 is set to overlap with the survey line 68.

[0065] The location of the travel point 70, including the switchback point 71 and the soil removal point 72, the location of the travel path 69 indicating the travel route of the dump truck 2, and the location of the survey line 68 are defined in the global coordinate system.

[0066] The driving data generated in the driving data generation unit 59 is transmitted to the dump truck 2. The controller 17 of the dump truck 2 controls the driving device 15 based on the driving data. The controller 17 controls the driving device 15 so that the dump truck 2 travels according to the driving path 69, based on the detection data from the position sensor 55 and the detection data from the orientation sensor 56. Specifically, the controller 17 controls the driving device 15 so that the deviation between the detected position of the dump truck 2 detected by the position sensor 55 and the target position of the dump truck 2 set at the driving point 70 is small when passing through the driving point 70. The controller 17 controls the driving device 15 so that the deviation between the detected orientation of the dump truck 2 detected by the orientation sensor 56 and the target orientation of the dump truck 2 set at the driving point 70 is small when passing through the driving point 70. The controller 17 controls the driving device 15 so that the dump truck 2 travels at the target driving speed, based on the detection data from the speed sensor 57. In other words, the controller 17 controls the driving device 15 so that the deviation between the detected driving speed of the dump truck 2 detected by the speed sensor 57 when passing the driving point 70 and the target driving speed of the dump truck 2 set at the driving point 70 becomes small.

[0067] In the example shown in Figure 7, dump truck 2 enters the soil removal area 7 from the transport path 8 while moving forward. After entering the soil removal area 7 from the transport path 8, dump truck 2 passes the entry point 70S and then enters the switchback point 71 while moving forward. After entering the switchback point 71, dump truck 2 performs a switchback and then enters the soil removal point 72 while moving backward. After entering the soil removal point 72, dump truck 2 performs soil removal work at the soil removal point 72. With dump truck 2 positioned at the soil removal point 72, the dump body 16 performs a dumping operation, and the load is removed from dump truck 2 at the soil removal point 72. Having completed the soil removal work, dump truck 2 moves forward towards the exit point 70E to leave the soil removal point 72. After passing the exit point 70E, dump truck 2 leaves the soil removal area 7 and returns to the transport path 8.

[0068] The work data transmission unit 60 transmits work data related to the dump truck 2 to the remote controller 23. The work data for the dump truck 2 includes edge data generated by the edge data generation unit 58 and driving data for the dump truck 2 generated by the driving data generation unit 59. The work data for the dump truck 2 includes a driving path 69 indicating the driving route that the dump truck 2 will travel, a dowel point 72 indicating the dowel location where the dump truck 2 will perform dowel removal work, an edge 44 of the dowel area 7 where the dump truck 2 can work, a survey line 68 indicating the outline of the dowel area 7, and the location of any obstacles 73 that may obstruct the work of the dump truck 2.

[0069] The processor 47 of the remote controller 23 has multiple functional units. The functional units of the remote controller 23 include an operation signal transmission unit 61, an image acquisition unit 62, a work data acquisition unit 63, a work image generation unit 64, a reference image generation unit 65, and a display control unit 66.

[0070] The operation signal transmission unit 61 transmits an operation signal generated by the operation of the remote control device 21 to the controller 18 of the bulldozer 4. Based on the operation signal transmitted from the operation signal transmission unit 61, the controller 18 controls at least one of the traveling device 27, the excavating work machine 28, and the ripper work machine 29.

[0071] The image acquisition unit 62 acquires an image 80 showing the work site 1 captured by the imaging device 32. The image acquisition unit 62 also acquires detection data from the position sensor 30 and the attitude sensor 31 when the image 80 is captured by the imaging device 32. The position of the image 80 is defined in the camera coordinate system set in the imaging device 32. Based on the detection data from the position sensor 30 and the attitude sensor 31, the image acquisition unit 62 can convert the position in the global coordinate system to the position on the image in the camera coordinate system.

[0072] The work data acquisition unit 63 acquires work data related to the dump truck 2. The work data acquisition unit 63 also acquires work data transmitted from the work data transmission unit 60.

[0073] The work image generation unit 64 generates a work image 81 that shows the work data. The work image generation unit 64 generates a work image 81 for display on the display device 22.

[0074] The reference image generation unit 65 generates a reference image 82 showing at least one of the dimensions of the bulldozer 4 and the distance from the bulldozer 4. In one embodiment, the reference image generation unit 65 generates a reference image 82 showing the width of the excavation blade 34 and the distance from the bulldozer 4. The reference image generation unit 65 can generate a reference image 82 based, for example, on the specifications data of the bulldozer 4. The specifications data of the bulldozer 4 is known data.

[0075] The display control unit 66 causes the captured image 80 captured by the imaging device 32 to be displayed on the display device 22. The display control unit 66 causes the work image 81 to be displayed on the display device 22 together with the captured image 80. In this embodiment, the display control unit 66 superimposes the work image 81 onto the captured image 80. The display control unit 66 superimposes the reference image 82 onto the captured image 80.

[0076] Figure 8 shows the captured image 80 displayed on the display device 22 according to the first embodiment. In the example shown in Figure 8, the captured image 80 is an image of the soil removal area 7 captured by the imaging device 32. The captured image 80 shows an image of the soil removal area 7 in front of the bulldozer 4.

[0077] In the example shown in Figure 8, the captured image 80 includes an image of the edge 44 of the soil removal area 7. The captured image 80 includes an image of the front of the body 26 of the bulldozer 4. The captured image 80 includes an image of the dump truck 2 entering the soil removal area 7.

[0078] In the example shown in Figure 8, the captured image 80 includes an image of an obstacle 73 that obstructs the movement of the dump truck 2. In the example shown in Figure 8, the obstacle 73 is a rock present on the ground in the soil removal area 7.

[0079] A work image 81 showing work data related to dump truck 2 is superimposed on the captured image 80. The work image 81 is displayed on the display device 22 in a manner that highlights the work data. The work image 81 includes at least one of a graphic image, a line image, a symbol image, and an icon image. The work image 81 may also include at least one of a character image and a numerical image.

[0080] In the example shown in Figure 8, the work image 81 includes a travel path image 811 showing the travel path 69 on which the dump truck 2 travels. The travel path image 811 is a line-shaped image (band-shaped image) displayed on the display screen of the display device 22 so as to follow the travel path 69.

[0081] The working image 81 includes a switchback point image 812 showing the switchback point 71 of the dump truck 2. The switchback point image 812 is a circular image (point-like image) that is displayed on the display screen of the display device 22 so as to overlap the switchback point 71.

[0082] The work image 81 includes a soil removal point image 813 showing the soil removal points 72 of the dump truck 2. Multiple soil removal points 72 are provided in the soil removal area 7. The soil removal point image 813 is a circular image displayed on the display screen of the display device 22 so as to overlap each of the multiple soil removal points 72. Note that the soil removal point image 813 is not limited to a circular image. The soil removal point image 813 may be, for example, a polygonal image or an icon image. The soil removal points 72 include un-soiled points that show the un-soiled position before soil removal work is performed, and completed points that show the soil-removed position after soil removal work is performed. In this embodiment, the soil removal point image 813 includes an un-soiled point image 813A showing the un-soiled point and a completed point image 813B showing the completed point. The un-soiled point image 813A is displayed on the display device 22 in the first display mode. The image 813B of the area where soil has been removed is displayed on the display device 22 in a second display mode that is different from the first display mode. The image 813A of the area where soil has not yet been removed is a ring image. The image 813B of the area where soil has been removed is a disk image obtained by filling in the ring image.

[0083] The working image 81 includes an edge image 814 showing the edge 44 of the soil removal area 7. The edge image 814 is a line-shaped image displayed on the display screen of the display device 22 so as to follow the edge 44 of the captured image 80. The edge image 814 is displayed so as to overlap the edge 44.

[0084] The work image 81 includes an obstacle image 815 that indicates the location of the obstacle 73. The obstacle image 815 is a triangular graphic image that is displayed on the display screen of the display device 22 so as to overlap the obstacle 73 in the captured image 80. The obstacle image 815 may be, for example, a circular graphic image or a graphic image of any polygonal shape. Also, if the obstacle 73 is, for example, a building, the obstacle image 815 may be displayed so as to surround the building. The obstacle image 815 may be, for example, a frame-like image surrounding the building. Furthermore, if there is a no-entry area in the work site 1 where the dump truck 2 is prohibited from entering, the obstacle image 815 may be displayed so as to surround the no-entry area. The obstacle image 815 may be, for example, a frame-like image surrounding the no-entry area.

[0085] The work image 81 includes the names of the work machines and vehicles managed by the management system 10. In the example shown in Figure 8, character data 816 indicating the name of the dump truck 2 is displayed on the display device 22 as the work image 81. The character data 816 may be displayed overlapping the dump truck 2 or displayed around the dump truck 2. If the captured image 80 includes the shovel 3 or motor grader 5, character data indicating the name of the shovel 3 or motor grader 5 may be displayed on the display device 22. If the captured image 80 includes the light vehicle 9, character data indicating the name of the light vehicle 9 may be displayed on the display device 22.

[0086] The working image 81 is superimposed on the captured image 80 using augmented reality (AR). When the position of the travel path 69 changes, the position of the travel path image 811 on the display screen of the display device 22 changes in accordance with the change in the position of the travel path 69. When the shape of the travel path 69 changes, the shape of the travel path image 811 on the display screen of the display device 22 changes in accordance with the change in the shape of the travel path 69. When the position of the switchback point 71 changes, the position of the switchback point image 812 on the display screen of the display device 22 changes in accordance with the change in the position of the switchback point 71. When the position of the soil removal point 72 changes, the position of the soil removal point image 813 on the display screen of the display device 22 changes in accordance with the change in the position of the soil removal point 72. The same applies to the edge image 814 and the obstacle image 815.

[0087] When dump truck 2 performs soil removal work at an unremoved soil point, and the unremoved soil point changes to a soil-removed soil point, the soil-removed soil point image 813 on the display screen of the display device 22 changes from an unremoved soil point image 813A to a soil-removed soil point image 813B in accordance with the change in the state of the soil-removed soil point 72. In other words, the display form of the work image 81 changes in accordance with the change in the state of the work data. When dump truck 2 performs soil removal work at soil-removed soil point 72, a soil removal completion signal indicating that the soil removal work has been completed is sent from the controller 17 of dump truck 2 to the management device 11. The work data transmission unit 60 transmits the soil removal completion signal to the remote controller 23. Based on the soil removal completion signal, the work image generation unit 64 can change the soil removal point image 813 from an unremoved soil point image 813A to a soil-removed soil point image 813B.

[0088] A reference image 82 is superimposed on the captured image 80. The reference image 82 shows the width of the excavation blade 34 and the distance from the bulldozer 4. The reference image 82 includes a blade reference line 821 showing the width of the excavation blade 34 and a distance reference line 822 showing the distance from the bulldozer 4. The blade reference line 821 is displayed so as to extend forward from the left end and the right end of the excavation blade 34, respectively. The left blade reference line 821 and the right blade reference line 821 are parallel. The distance reference line 822 is displayed so as to connect the front end of the left blade reference line 821 and the front end of the right blade reference line 821. The distance reference line 822 is displayed at a position a specified distance (e.g., 20m) forward from the front end of the vehicle body 26.

[0089] <How to display images> Figure 9 is a flowchart illustrating the image display method according to the first embodiment. An image of the work site 1 is captured by the imaging device 32. The image of the work site 1 captured by the imaging device 32 is transmitted from the controller 18 of the bulldozer 4 to the remote controller 23. The image acquisition unit 62 acquires an image 80 showing the image of the work site 1 transmitted from the controller 18 of the bulldozer 4 (step S1).

[0090] The work data transmission unit 60 of the management device 11 transmits work data related to the dump truck 2 to the remote controller 23. The work data acquisition unit 63 acquires the work data related to the dump truck 2 transmitted from the management device 11 (step S2).

[0091] The work image generation unit 64 generates a work image 81 based on the work data acquired by the work data acquisition unit 63 (step S3). The reference image generation unit 65 generates a reference image 82 based on the specifications data of the bulldozer 4 (step S4).

[0092] As explained with reference to Figure 8, the display control unit 66 overlays the work image 81 and the reference image 82 onto the captured image 80 (step S5).

[0093] The display control unit 66 determines whether or not to terminate the image display (step S6). If it is determined in step S6 to continue the image display (step S6: No), the process returns to step S1. If it is determined in step S6 to terminate the image display (step S6: Yes), the image display terminates.

[0094] <Effects> As described above, in this embodiment, the processor 47 of the remote controller 23 includes an image acquisition unit 62 that acquires an image 80 showing an image of the work site 1 of the remotely operated bulldozer 4 captured by the imaging device 32, a work data acquisition unit 63 that acquires work data relating to the dump truck 2 working at the work site 1, a work image generation unit 64 that generates a work image 81 showing the work data, and a display control unit 66 that displays the work image 81 together with the image 80 on a display device 22 located outside the bulldozer 4.

[0095] According to this embodiment, a work image 81 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0096] The working image 81 is superimposed on the captured image 80 using augmented reality (AR). This allows the operator to smoothly recognize both the captured image 80 and the working image 81.

[0097] The work image 81 includes a travel path image 811 showing the travel path 69 of the dump truck 2. By displaying the travel path image 811 on the display device 22, the operator can recognize the travel route of the dump truck 2. The operator can recognize the location (route) where the dump truck 2 is scheduled to travel from the travel path image 811.

[0098] The work image 81 includes a switchback point image 812 showing the switchback point 71 of the dump truck 2. By displaying the switchback point image 812 on the display device 22, the operator can recognize the location where the switchback of the dump truck 2 is scheduled.

[0099] The work image 81 includes an image 813 showing the soil discharge point 72 of the dump truck 2. By displaying the soil discharge point image 813 on the display device 22, the operator can recognize the location of the soil discharge operation by the dump truck 2.

[0100] The soil removal point image 813 includes an unremoved soil point image 813A in a first display mode showing unremoved soil points, and a completed soil removal point image 813B in a second display mode showing completed soil removal points. By displaying the unremoved soil point image 813A and the completed soil removal point image 813B on the display device 22, the operator can recognize the location where soil removal work by the dump truck 2 is scheduled and the location where soil removal work by the dump truck 2 has been completed.

[0101] The work image 81 includes an edge image 814 showing the edge 44 of the soil removal area 7. By displaying the edge image 814 on the display device 22, the operator can recognize the position and shape of the edge 44. By displaying the edge image 814 on the display device 22, the operator can recognize the position of the descending cliff 43.

[0102] The work image 81 includes an obstacle image 815 showing the location of an obstacle 73 that obstructs the movement of the dump truck 2. The display of the obstacle 73 on the display device 22 allows the operator to recognize its location. The display of the obstacle image 815 on the display device 22 allows the operator to remotely control the bulldozer 4 to avoid contact with the obstacle 73.

[0103] In this embodiment, the reference image 82 is superimposed on the captured image 80. By superimposing the reference image 82 on the captured image 80, the operator can, for example, recognize the relative position between the excavation blade 34 and an object in the soil discharge area 7, or recognize the relative distance between the vehicle body 26 and an object in the soil discharge area 7.

[0104] [Second Embodiment] A second embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0105] Figure 10 shows the captured image 80 displayed on the display device 22 according to the second embodiment. In this embodiment, the work image generation unit 64 generates a window image 90 by superimposing the work image 91 onto an overhead view image 85 of the work site 1. The display control unit 66 superimposes the window image 90 onto the captured image 80. Superimposition display refers to a display method in which a window image 90, which is a secondary screen, is displayed over a portion of the captured image 80, which is the main screen.

[0106] Figure 11 is an enlarged view of the window image 90 according to the second embodiment. In this embodiment, the work site 1 is surveyed in advance. Based on the survey data, two-dimensional data representing the two-dimensional shape of the terrain of the work site 1 is calculated. The two-dimensional data represents the two-dimensional shape of the terrain on a predetermined plane substantially parallel to the ground of the work site 1. The two-dimensional data may be defined by latitude and longitude coordinate data. The survey includes detecting the two-dimensional shape of the work site using a position sensor including a GNSS receiver. For example, by having a survey vehicle equipped with a position sensor travel along the edge of the work area, the work image generation unit 64 can calculate the two-dimensional data of the work site 1 based on the detection data from the position sensor.

[0107] Furthermore, 3D data showing the 3D shape of the terrain of the work site 1 may be calculated based on the survey data. The survey may include detecting the 3D shape of the work site using a 3D sensor. A 3D sensor is a sensor capable of detecting the 3D shape of an object to be detected. Examples of 3D sensors include laser sensors (LiDAR: Light Detection and Ranging) that detect the 3D shape of an object by emitting laser light, and stereo cameras. For example, if an aircraft such as a drone equipped with a 3D sensor flies over the work site 1, and the 3D sensor mounted on the drone detects the work site 1, the work image generation unit 64 can calculate 3D data of the work site 1 based on the detection data of the 3D sensor. Based on the 3D data of the work site 1, the work image generation unit 64 can generate an overhead image 85 of the work site 1.

[0108] Window image 90 includes an overhead view image 85 of the soil removal area 7, a model image 2V of dump truck 2, a model image 4V of bulldozer 4, and a work image 91. Model image 4V of bulldozer 4 includes a model image 34V of the excavation blade 34. Model image 2V is an image modeling dump truck 2 viewed from above. Model image 4V is an image modeling bulldozer 4 viewed from above.

[0109] When dump truck 2 travels in the soil removal area 7, the model image 2V moves in the window image 90 in accordance with the movement of dump truck 2. The work image generation unit 64 can move the model image 2V in the window image 90 based on the detection data of the position sensor 55 and orientation sensor 56 of dump truck 2. When bulldozer 4 travels in the soil removal area 7, the model image 4V moves in the window image 90 in accordance with the movement of bulldozer 4. The work image generation unit 64 can move the model image 4V in the window image 90 based on the detection data of the position sensor 30 and attitude sensor 31 of bulldozer 4.

[0110] The work image 91 is superimposed on the overhead image 85. The work image 91 includes a travel path image 911 showing the travel path 69 on which the dump truck 2 travels, a switchback point image 912 showing the switchback point 71 of the dump truck 2, a soil discharge point image 913 showing the soil discharge point 72 of the dump truck 2, and an edge image 914 showing the edge 44 of the soil discharge area 7. The soil discharge point image 913 includes an undischarged point image 913A showing the undischarged point and a completed soil discharge point image 913B showing the completed point.

[0111] As described above, in this embodiment as well, the work image 91 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0112] [Third Embodiment] A third embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0113] Figure 12 shows the captured image 80 and window image 90 according to the third embodiment. In this embodiment, a plurality of display devices 22 are arranged side by side in the remote control room 20. In the example shown in Figure 12, the display devices 22 include a first display device 22A and a second display device 22B arranged next to the first display device 22A.

[0114] Similar to the second embodiment described above, the work image generation unit 64 generates a window image 90 by superimposing the work image 91 onto the overhead view image 85 of the work site 1. As shown in Figure 12, the display control unit 66 may display the captured image 80 on the first display device 22A and the window image 90 on the second display device 22B. In the example shown in Figure 12, the reference image 82 is displayed superimposed on the captured image 80.

[0115] As described above, in this embodiment as well, the work image 91 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0116] [Fourth Embodiment] A fourth embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0117] Figure 13 shows the captured image 80 and window image 90 according to the fourth embodiment. In this embodiment, a plurality of display devices 22 are arranged side by side in the remote control room 20. In the example shown in Figure 13, the display devices 22 include a first display device 22A and a second display device 22B arranged next to the first display device 22A.

[0118] As shown in Figure 13, the display control unit 66 causes the captured image 80, on which the work image 81 is superimposed, to be displayed on the first display device 22A. The work image generation unit 64 generates a window image 90 on which the work image 91 is superimposed on the overhead view image 85 of the work site 1, and the display control unit 66 causes the window image 90 to be displayed on the second display device 22B. In the example shown in Figure 13, the reference image 82 is superimposed on the captured image 80.

[0119] As described above, in this embodiment as well, the work image 91 showing the work data of the dump truck 2 is displayed on the display device 22 together with the captured image 80, so that the operator in the remote control room 20 can recognize the work status of the dump truck 2. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4, which is remotely controlled by the operator, is suppressed.

[0120] [Fifth Embodiment] A fifth embodiment will now be described. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and the descriptions of those components will be simplified or omitted.

[0121] Figure 14 is a schematic diagram showing the soil removal area 7 according to the fifth embodiment. As shown in Figure 14, the dump truck 2 may perform soil removal work so that multiple piles 46 are formed on the ground of the soil removal area 7. The multiple piles 46 are formed regularly at intervals in the soil removal area 7. The bulldozer 4 can perform leveling work to smooth out the multiple piles 46 formed on the ground of the soil removal area 7. Even when multiple piles 46 are formed on the ground of the soil removal area 7, the operator in the remote control room 20 can recognize the work status of the dump truck 2 by displaying a work image 81 showing the work data of the dump truck 2 together with the captured image 80 on the display device 22. Since the operator can recognize the work status of the dump truck 2, a decrease in the work efficiency of the bulldozer 4 remotely controlled by the operator is suppressed.

[0122] [Other embodiments] In the above-described embodiment, the shovel 3, bulldozer 4, and motor grader 5 are all manned work machines. At least one of the shovel 3, bulldozer 4, and motor grader 5 may be an unmanned work machine. An unmanned work machine is a work machine that performs work without operator operation. When at least one of the shovel 3, bulldozer 4, and motor grader 5 is traveling according to a travel path, a travel path image showing the travel path of at least one of the shovel 3, bulldozer 4, and motor grader 5 may be displayed on the display device 22.

[0123] In the above-described embodiment, the light vehicle 9 is assumed to be a manned vehicle. The light vehicle 9 may also be an unmanned vehicle. When the light vehicle 9 travels according to a travel path, a travel path image showing the travel path of the light vehicle 9 may be displayed on the display device 22.

[0124] In the above embodiment, the reference image 82 does not need to be displayed on the display device 22.

[0125] In the above-described embodiment, the bulldozer 4 working in the soil removal area 7 is a bulldozer. However, the bulldozer 4 working in the soil removal area 7 may be other work machines such as an shovel, wheel loader, and motor grader.

[0126] In the above-described embodiment, at least a portion of the functional parts of the remote controller 23 may be provided in the management device 11. At least a portion of the functional parts of the management device 11 may be provided in the remote controller 23. For example, the work image generation unit 64 and the reference image generation unit 65 may be provided in the management device 11. The edge data generation unit 58 and the driving data generation unit 59 may be provided in the remote controller 23.

[0127] In the embodiments described above, each of the multiple functional units of the remote controller 23 may be configured by a separate computer (hardware). Each of the functional units of the management device 11 may be configured by a separate computer (hardware). The multiple functional units of the remote controller 23 and the multiple functional units of the management device 11 may be configured by a single computer (hardware). [Explanation of symbols]

[0128] 1...Work site, 2...Dump truck (unmanned vehicle), 2V...Model image, 3...Excavator (working machine), 4...Bulldozer (working machine), 4V...Model image, 5...Motor grader (working machine), 6...Loading area, 7...Soil removal area, 8...Transportation route, 9...Light vehicle, 10...Management system, 11...Management device, 12...Communication system, 12A...Wireless communication device, 12B...Wireless communication device, 12C...Wireless communication device, 13...Control facility, 14...Vehicle body, 15...Running gear, 16...Dump body, 17...Controller, 18...Controller, 19...Remote control system, 20...Remote control room, 2 1...Remote control device, 22...Display device, 22A...First display device, 22B...Second display device, 23...Remote controller, 24...Driver's seat, 25...Communication system, 26...Vehicle body, 27...Running gear, 28...Excavating work equipment, 29...Ripper work equipment, 30...Position sensor, 31...Attitude sensor, 32...Imaging device, 33...Track, 34...Excavating blade, 34A...Cutting edge, 34V...Model image, 35...Lift frame, 36...Tilt cylinder, 37...Lift cylinder, 38...Shank, 38A...Ripper point, 39...Ripper arm, 40...Tilt cylinder, 41...Lift cylinder, 4 2...Beam, 43...Downward cliff, 44...Edge, 45...Embankment, 46...Ground, 47...Processor, 48...Main memory, 49...Storage, 50...Input / Output interface, 51...Communication interface, 52...Computer program, 53...Position sensor, 54...Controller, 55...Position sensor, 56...Direction sensor, 57...Speed ​​sensor, 58...Edge data generation unit, 59...Driving data generation unit, 60...Work data transmission unit, 61...Operation signal transmission unit, 62...Image acquisition unit, 63...Work data acquisition unit, 64...Work image generation unit, 65...Reference image generation unit, 66...Display Control unit, 68...Survey line, 69...Travel path, 70...Travel point, 70E...Departure point, 70S...Entry point, 71...Switchback point, 72...Earth removal point, 73...Obstacle, 80...Acquired image, 81...Work image, 811...Travel path image, 812...Switchback point image, 813...Earth removal point image, 813A...Unremoved earth point image, 813B...Earth removal completed point image, 814...Edge image, 815...Obstacle image, 816...Text data, 82...Reference image, 821...Blade reference line, 822...Distance reference line, 85...Overhead view image, 90...Window image, 91...Work image, 911...Travel path image,912... Switchback point image, 913... Soil removal point image, 913A... Unremoved soil point image, 913B... Soil removal completed point image, 914... Edge image.

Claims

1. Equipped with a processor, The aforementioned processor, The imaging device acquires images showing the work site of a remotely operated work machine, Work data related to unmanned vehicles working at the aforementioned work site is acquired. A work image showing the aforementioned work data is generated, The work image is displayed together with the captured image on a display device located outside the work machine. Remote control system for industrial machinery.

2. The aforementioned work data includes the route traveled by the unmanned vehicle. A remote control system for a work machine according to claim 1.

3. The aforementioned unmanned vehicle is a transport vehicle for carrying cargo, The aforementioned work data includes the soil removal position where the unmanned vehicle performs the soil removal work on the load. A remote control system for a work machine according to claim 1.

4. The aforementioned soil removal location includes the unremoved soil location before the soil removal work is carried out and the removed soil location after the soil removal work is carried out. The aforementioned work image includes an image of the un-excavated point in a first display mode indicating the unexcavated position, and an image of the excavated point in a second display mode indicating the excavated position. A remote control system for a work machine according to claim 3.

5. The aforementioned work data includes the edges of the work area in which the unmanned vehicle can perform work. A remote control system for a work machine according to claim 1.

6. The aforementioned work data includes the location of obstacles that obstruct the work of the unmanned vehicle. A remote control system for a work machine according to claim 1.

7. The aforementioned processor, The aforementioned work image is superimposed onto the aforementioned captured image. A remote control system for a work machine according to claim 1.

8. The CR work machine has an excavation blade, The aforementioned processor, A reference image is generated showing the width of the drilling blade and the distance from the work machine. The aforementioned reference image is superimposed onto the captured image. A remote control system for a work machine according to claim 7.

9. The aforementioned processor, A window image is generated by superimposing the aforementioned work image onto an overhead view image of the work site. The aforementioned window image is superimposed onto the captured image. A remote control system for a work machine according to claim 1.

10. In a remote control room where a remote control device for remotely operating the aforementioned work machine is located, a plurality of the aforementioned display devices are arranged side by side. The aforementioned processor, A window image is generated by superimposing the aforementioned work image onto an overhead view image of the work site. The captured image is displayed on the first display device, and the window image is displayed on the second display device. A remote control system for a work machine according to claim 1.

11. To acquire images showing the work site of a remotely operated work machine captured by an imaging device, To acquire work data related to unmanned vehicles working at the aforementioned work site, To generate a work image showing the aforementioned work data, This includes displaying the aforementioned work image together with the captured image on a display device located outside the work machine, A method for remotely controlling industrial machinery.

12. The aforementioned work data includes the route traveled by the unmanned vehicle. A method for remotely operating a work machine according to claim 11.

13. The aforementioned unmanned vehicle is a transport vehicle for carrying cargo, The aforementioned work data includes the soil removal position where the unmanned vehicle performs the soil removal work on the load. A method for remotely operating a work machine according to claim 11.

14. The aforementioned soil removal location includes the unremoved soil location before the soil removal work is carried out and the removed soil location after the soil removal work is carried out. The aforementioned work image includes an image of the un-excavated point in a first display mode indicating the unexcavated position, and an image of the excavated point in a second display mode indicating the excavated position. A method for remotely operating a work machine according to claim 13.

15. The aforementioned work data includes the edges of the work area in which the unmanned vehicle can perform work. A method for remotely operating a work machine according to claim 11.

16. The aforementioned work data includes the location of obstacles that obstruct the work of the unmanned vehicle. A method for remotely operating a work machine according to claim 11.

17. This includes superimposing the aforementioned work image onto the aforementioned captured image. A method for remotely operating a work machine according to claim 11.

18. The CR work machine has an excavation blade, To generate a reference image showing the width of the drilling blade and the distance from the work machine, This includes superimposing the aforementioned reference image onto the captured image, A method for remotely operating a work machine according to claim 17.

19. The process involves generating a window image by superimposing the aforementioned work image onto an overhead view image of the work site, This includes superimposing the aforementioned window image onto the captured image, A method for remotely operating a work machine according to claim 11.

20. In a remote control room where a remote control device for remotely operating the aforementioned work machine is located, a plurality of the aforementioned display devices are arranged side by side. The process involves generating a window image by superimposing the aforementioned work image onto an overhead view image of the work site, This includes displaying the captured image on the first display device and displaying the window image on the second display device. A method for remotely operating a work machine according to claim 11.

Images