Remote apparatus and remote operation system

Abstract

The present invention provides assistance in remote operation by a remote operator by displaying a predicted travel route.　A remote apparatus (30) comprises: a first communication device (33) capable of communicating with a remote work machine (1); a remote operation device (35) that receives a maneuver and remotely operates the remote work machine (1) through the first communication device (33); and a display device (34) that displays a route object (90) indicating a predicted travel route of the remote work machine (1) in a manner superimposed on a captured image (70a, 70b) obtained by imaging the surroundings of the remote work machine (1). The display device (34) switches between display and non-display of the route object (90) depending on the maneuver received by the remote operation device (35).

Description

Remote device and remote control system

[0001] The present invention relates to a remote device and a remote control system for remotely controlling a work machine.

[0002] BACKGROUND ART Conventionally, there is known a remote control system that generates a monitoring screen for monitoring the traveling state and working state of a work vehicle and displays the screen on a display (for example, Patent Document 1).

[0003] Japanese Patent Publication No. 2021-36796

[0004] The remote control system of Patent Document 1 displays multiple screens showing the surroundings of the work vehicle (for example, the front, back, left, and right) during remote control, but is unable to display the route along which the work vehicle is expected to travel (predicted driving route).As a result, the remote control system of Patent Document 1 is unable to assist the remote operator in remote control by displaying the predicted driving route.

[0005] The present invention has been made to solve the problems of the prior art, and aims to provide a remote device and a remote control system that can assist a remote operator in remote control by displaying a predicted driving route.

[0006] A remote control device according to one aspect of the present invention comprises a first communication device capable of communicating with a remote work machine, a remote control device that receives operations and remotely controls the remote work machine via the first communication device, and a display device that displays a route object indicating the predicted travel route of the remote work machine superimposed on an image captured of the area around the remote work machine, and the display device switches between displaying and hiding the route object depending on the operation received by the remote control device.

[0007] The path object includes a traveling object indicating the predicted movement path of a traveling device possessed by the remote work machine, and a work object indicating the predicted movement path of a work device possessed by the remote work machine, and the display device may switch between displaying and hiding the traveling object and the work object depending on the operation received by the remote control device.

[0008] The running objects include a first running object indicating an expected movement path of the front wheels of the running device, and a second running object indicating an expected movement path of the rear wheels of the running device, and the remote control device is capable of a first operation to switch the running device to forward movement and a second operation to switch the running device to reverse movement, and the display device may, when the remote control device is operated in the first manner, display a first captured image of an area in front of the remote work machine and the first running object, and, when the remote control device is operated in the second manner, display a second captured image of an area behind the remote work machine and the second running object.

[0009] The remote device may be provided with a switching operation device that accepts switching operations for the display of the display device, and when the switching operation device is operated, the display device may switch between the first captured image and the second captured image in accordance with the operation of the switching operation device, regardless of the first operation and the second operation of the remote control device.

[0010] The display device may display the running object and the work object when the direction of travel of the running device due to the first operation and the second operation matches the imaging direction of the captured image, and may hide the running object and the work object when they do not match.

[0011] The display device may display the first running object and the work object and hide the second running object when the remote control device is operated in the first manner and the first captured image is displayed, and may display the second running object and the work object and hide the first running object when the remote control device is operated in the second manner and the second captured image is displayed.

[0012] The remote control device may be capable of a third operation that switches the traveling device to neutral, separate from the first operation and the second operation, and the display device may hide the traveling object and the work object when the remote control device is operated in the third manner.

[0013] A remote control system according to one embodiment of the present invention comprises a remote device and a remote work machine that is remotely controlled by the remote device, and the remote work machine has a second communication device that communicates with the first communication device.

[0014] According to the present invention, by displaying a predicted driving route, it is possible to assist the remote operator in remotely controlling the vehicle.

[0015] 1 is a configuration diagram of a remote control system according to an embodiment of the present invention; FIG. 2 is a side view of a tractor, which is an example of a remote work machine; FIG. 3 is a diagram showing an example of correspondence data; FIG. 4 is a diagram showing an example of a display screen when a remote control device is operated in a first manner; FIG. 5 is a diagram showing an example of a display screen when a remote control device is operated in a second manner; FIG. 6 is a diagram showing an example of a table; FIG. 7 is a diagram showing an example of a table in a modified example; FIG. 8 is a flowchart showing the display procedure of the display screen; FIG. 9 is a diagram showing an example of a display screen in a modified example; FIG. 10 is a diagram showing an example of a display screen when a remote control device is operated in a first manner in a modified example; FIG. 11 is a diagram showing another example of a display screen when a remote control device is operated in a first manner in a modified example;

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0017] FIG. 1 is a configuration diagram of a remotely controlled system 100 according to an embodiment of the present invention. The remotely controlled system 100 includes a remotely controlled work machine 1 and a remote device 30. The remotely controlled work machine 1 and the remote device 30 enable remote control (or remote operation) and remote monitoring of the remotely controlled work machine 1. The remotely controlled work machine 1 is a farm machine (also referred to as a "remotely controlled agricultural machine") that can be remotely driven (e.g., remotely driven and remotely operated) using the remote device 30. The remotely controlled work machine 1 includes agricultural machines such as tractors, rice transplanters, and harvesting combines, each equipped with (supporting) a work device 2 such as an implement. For convenience of explanation, the following description will use a tractor as an example of the remotely controlled work machine 1, and will omit descriptions of other agricultural machines. In the following description, the remotely controlled work machine 1 will be described as a tractor having a driver's seat 10 in which a worker (operator) sits. However, the remotely controlled work machine 1 may also be an agricultural machine dedicated to remote control that does not have a driver's seat 10.

[0018] FIG. 2 is a side view of a tractor, which is an example of the remote-controlled work machine 1. In the following description, the front side (in the direction of arrow A1 in FIG. 2 ) of an operator seated in the driver's seat 10 of the remote-controlled work machine 1 will be referred to as the front, the rear side (in the direction of arrow A2 in FIG. 2 ) of the operator will be referred to as the rear, the left side of the operator will be referred to as the left (the near side in FIG. 2 ), and the right side of the operator will be referred to as the right (the far side in FIG. 2 ). The horizontal direction, which is perpendicular to the fore-and-aft direction of the remote-controlled work machine 1, will be referred to as the width direction. The remote-controlled work machine 1 includes a body 3. The body 3 is provided with a traveling device 7. The traveling device 7 has front wheels 7F and rear wheels 7R on both the left and right sides of the body 3, supporting the body 3 so that it can travel. The traveling device 7 may be a crawler-type device.

[0019] As shown in FIGS. 1 and 2 , the vehicle body 3 is equipped with a prime mover 4, a transmission 5, a braking device 13, and a steering device 14. The prime mover 4 is composed of an engine (diesel engine, gasoline engine) or an electric motor. The transmission 5, for example, performs a gear change operation to vary the propulsive force of the traveling device 7 and to switch the traveling device 7 between forward and reverse. The transmission 5 has multiple gears that transmit power, a shifter that changes the connection of the gears, and a clutch that switches between transmitting and disconnecting power, and performs the gear change operation using these. The transmission 5 can also switch the traveling mode between four-wheel drive (4WD) in which the front wheels 7F and the rear wheels 7R are driven by power and the rotational speeds of the front wheels 7F and the rear wheels 7R are approximately the same (4WD constant speed state), four-wheel drive in which the rotational speed of the front wheels 7F is faster than the rotational speed of the rear wheels 7R (4WD accelerated speed state), and two-wheel drive (2WD) in which only the rear wheels 7R are driven. In this embodiment, the transmission 5 can perform separate speed change operations for the main speed change section (for example, continuously variable speed) and the sub speed change section (for example, stepped speed).

[0020] The braking device 13 controls the movement of the airframe 3. The steering device 14 operates the airframe 3 so that it faces in a predetermined direction.

[0021] A control device 11 for manually operating the remote-controlled work machine 1 is provided around the driver's seat 10. The remote-controlled work machine 1 can travel and work unmanned, but an operator seated in the driver's seat 10 can operate the control device 11 to travel the remote-controlled work machine 1 and perform work using the work device 2. The control device 11 includes a steering wheel 11a, an accelerator pedal 11b, a brake pedal 11c, a gear shift lever 11d, and the like.

[0022] A coupling device 8 is provided at the rear of the machine body 3. The coupling device 8 is configured with a three-point link mechanism or the like. The coupling device 8 can detachably couple the working device 2 to the machine body 3. The remote-controlled work machine 1 (machine body 3) can tow the working device 2 by coupling the working device 2 to the coupling device 8 and causing the traveling device 7 to travel. In addition, the coupling device 8 can raise and lower the working device 2 and change the attitude of the working device 2 by driving an actuator (e.g., a hydraulic cylinder).

[0023] The work devices 2 include, for example, a tilling device for tilling the soil, a fertilizer spreading device for spreading fertilizer, a pesticide spreading device for spreading pesticides, a harvesting device for harvesting, a reaping device for reaping grass etc., a spreading device for spreading grass etc., a grass collecting device for collecting grass etc., a shaping device for shaping grass etc. The remote-controlled work machine 1 performs agricultural work in the field using these work devices 2.

[0024] 1 , the remote-operated work machine 1 includes an on-board control device 21, a storage device 22, a second communication device (on-board communication device) 23, a position detection device 24, a first detection device (inertial measurement unit) 25, a second detection device (sensing device) 26, a state detection device 27, and a third detection device 28. In addition, an on-board network such as CAN, LIN, or FlexRay is configured in the remote-operated work machine 1, and the on-board control device 21 is electrically connected to the storage device 22, the second communication device 23, the position detection device 24, etc.

[0025] The on-vehicle control device 21 is composed of an ECU (electronic control unit) including a processor 21a and a memory 21b. The on-vehicle control device 21 is a controller that controls the operation of each part of the remote-controlled work machine 1. The memory 21b is composed of volatile or non-volatile memory, etc. The on-vehicle control device 21 controls a group of actuators including electric or hydraulic motors, cylinders, control valves, etc. for operating each part of the remote-controlled work machine 1, such as the prime mover 4, transmission 5, travel device 7, braking device 13, and coupling device 8. The memory 21b of the on-vehicle control device 21 stores various information and data used by the on-vehicle control device 21 to control the operation of each part of the remote-controlled work machine 1 in a readable and writable manner. The storage device 22 is, for example, an SSD (solid state drive), HDD (hard disk drive), etc.

[0026] The second communication device 23 is composed of an antenna, an IC (integrated circuit), an electric circuit, etc. for wireless communication via a mobile phone communication network, the Internet, or a wireless LAN. The on-board control device 21 communicates wirelessly with the remote device 30 via the second communication device 23. The on-board control device 21 also has an RTC (real-time clock) and can measure the date and time.

[0027] Although the present embodiment illustrates an example in which the remote work machine 1 and the remote device 30 communicate via a mobile phone communication network or the like, the remote work machine 1 and the remote device 30 may alternatively communicate with a mobile phone communication network or the like via an external device such as a server or a repeater. Alternatively, the remote work machine 1 and the remote device 30 may communicate directly using short-range wireless signals such as Bluetooth (registered trademark) Low Energy (BLU) signals or Ultra High Frequency (UHF) signals. In this case, it is sufficient that the second communication device 23 and the remote device 30 each have an interface for short-range wireless communication.

[0028] The position detection device 24 detects its own position (positioning information including latitude and longitude) using a satellite positioning system. That is, the position detection device 24 receives signals transmitted from positioning satellites (such as the position of the positioning satellite, the transmission time, and correction information) and detects its own position based on the signals. The position detection device 24 may detect its own position as a position corrected based on signals such as corrections from a base station (reference station) that can receive signals from the positioning satellites. The position detection device 24 may also calculate the position of the remote-controlled work machine 1 based on its detected position and pre-stored external shape information of the remote-controlled work machine 1. The position detection device 24 may also calculate the position of the work machine 2 based on its detected position, pre-stored external shape information of the work machine 2, and the mounting position of the work machine 2 relative to the machine body 3.

[0029] The first detection device 25 includes a gyro sensor or an acceleration sensor, and detects the roll angle, pitch angle, yaw angle, etc. of the remote-controlled work machine 1 (machine body 3). That is, the first detection device 25 detects the tilt attitude of the remote-controlled work machine 1 to the left and right and forward and backward relative to the horizontal state.

[0030] The second detection device 26 is a sensing device that senses (monitors) the surroundings of the remote-controlled work machine 1. Specifically, the second detection device 26 includes a laser sensor 26a, an ultrasonic sensor 26b, a camera 26c, and an object detection unit 26d. In this embodiment, the second detection device 26 includes a plurality of laser sensors 26a and ultrasonic sensors 26b. The plurality of laser sensors 26a and the plurality of ultrasonic sensors 26b are installed at predetermined locations, such as the front, rear, and left and right sides of the remote-controlled work machine 1, and detect surrounding conditions, such as the front, rear, left and right sides of the remote-controlled work machine 1, and objects in the surroundings. For example, the laser sensor 26a and the ultrasonic sensor 26b are installed at predetermined positions on the machine body 3 that can detect objects within a predetermined target detection distance from the remote-controlled work machine 1 and that are located lower than the machine body 3.

[0031] The laser sensor 26a and the ultrasonic sensor 26b are examples of object sensors. Note that at least one of the laser sensor 26a and the ultrasonic sensor 26b may be provided in plurality as an object sensor in the second detection device 26. Furthermore, the second detection device 26 may be provided with a plurality of other object sensors.

[0032] The laser sensor 26a is composed of an optical sensor such as a LiDAR (Light Detection and Ranging) sensor. The laser sensor 26a emits pulsed measurement light (laser light) millions of times per second from a light source such as a laser diode, and reflects the measurement light with a rotating mirror to scan the measurement light in the horizontal or vertical direction and project it onto a predetermined detection range (sensing range). The laser sensor 26a then receives the measurement light reflected by the target object with a light-receiving element.

[0033] The object detection unit 26d is composed of an electric circuit or an IC that detects the presence, position, and type of an object based on the light-receiving signal output from the light-receiving element of the laser sensor 26a. The object detection unit 26d detects the distance to the object based on the time between when the laser sensor 26a emits measurement light and when it receives the reflected light (TOF (Time of Flight) method). Objects detected by the object detection unit 26d include the field in which the remote-controlled work machine 1 travels and works, crops in the field, the ground, the slope of the ground, other objects, people, etc.

[0034] The ultrasonic sensor 26b is composed of an airborne ultrasonic sensor such as a sonar. The ultrasonic sensor 26b transmits measurement waves (ultrasonic waves) within a predetermined detection range using a transmitter, and receives the reflected waves from an object using a receiver. The object detection unit 26d detects the presence or absence, position, and type of object based on the signal output from the receiver of the ultrasonic sensor 26b. The object detection unit 26d also detects the distance to the object based on the time between when the ultrasonic sensor 26b transmits the measurement wave and when it receives the reflected wave (TOF method).

[0035] The camera 26c is composed of a CCD camera equipped with a CCD (Charge Coupled Device) image sensor, or a CMOS camera equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor. As shown in Fig. 2, the cameras 26c are installed at predetermined locations such as the front, rear, left and right sides of the remote-controlled work machine 1 and inside the cabin 9, and capture images of the surroundings such as the front, rear, left and right sides of the remote-controlled work machine 1 and output data of the captured images. The camera 26c is an example of an imaging device.

[0036] 2, the camera 26c1 installed inside the cabin 9 captures an image of the area in front of the remote-controlled work machine 1 from the driver's seat 10. More specifically, the camera 26c1 captures an image of the area in front of the remote-controlled work machine 1 (in the direction of travel, A1) from a field of view that is substantially the same as that of an operator seated in the driver's seat 10. In other words, the camera 26c1 can obtain an image of the area in the direction of travel of the remote-controlled work machine 1.

[0037] 2, the camera 26c2 installed at the rear of the body 3 of the remote-controlled work machine 1 captures images behind the remote-controlled work machine 1. More specifically, the camera 26c2 captures images in the direction opposite to the field of view of the operator seated in the driver's seat 10 (the direction of arrow A2). In other words, the camera 26c2 can obtain captured images in the direction opposite to the traveling direction of the remote-controlled work machine 1.

[0038] The object detection unit 26d can also detect the presence or absence of an object, the position of the object, the type of the object, and the like, based on the captured image data output from the camera 26c.

[0039] The second detection device 26 senses (monitors) the surrounding conditions of the remote-controlled work machine 1 and the work device 2 using the laser sensor 26a, ultrasonic sensor 26b, camera 26c, and object detection unit 26d, and outputs sensing information indicating the results to the on-board control device 21. The sensing information includes at least the detection information of the object detection unit 26d, the image data captured by the camera 26c, and the surrounding gradient, but the sensing information may also include detection information from the laser sensor 26a and ultrasonic sensor 26b.

[0040] The state detection device 27 detects the operating state of the remote-controlled work machine 1. Specifically, the state detection device 27 includes various sensors installed in each part of the remote-controlled work machine 1 (machine body 3) and a computing unit, and the computing unit detects (calculates) the operating state of the remote-controlled work machine 1 based on output signals from the various sensors. The state of the remote-controlled work machine 1 detected by the state detection device 27 includes the drive and stop states of each part of the remote-controlled work machine 1, the traveling direction, traveling speed, acceleration, steering angle, etc. of the remote-controlled work machine 1.

[0041] The state detection device 27 may acquire the position of the machine body 3 (position of the remote-controlled work machine 1) detected by the position detection device 24 at a predetermined cycle, and detect (calculate) the position of the work device 2 from the position of the machine body 3, or detect changes (transitions) in the position of the machine body 3. The state detection device 27 may also detect the traveling speed of the machine body 3 from changes in the position of the machine body 3. As another example, a rotation speed sensor may be provided to detect the rotation speed of the front wheels 7F and rear wheels 7R of the traveling device 7 or the rotation speed of a traveling motor that rotates the front wheels 7F and rear wheels 7R, and the state detection device 27 may detect the traveling speed of the machine body 3 based on the output signal of the rotation speed sensor. Alternatively, a sensor may be provided in the steering device 14, and the state detection device 27 may detect the steering angle of the machine body 3 based on the output signal of the sensor.

[0042] The state detection device 27 generates detection information indicating the detected operating state of the remote-controlled work machine 1 and outputs the detection information to the on-board control device 21. For example, the detection information of the state detection device 27 includes operation information of the remote-controlled work machine 1 and the work device 2. This operation information includes, for example, at least one piece of information among the speed (or acceleration) of the remote-controlled work machine 1, the gear shift position of the transmission 5, the braking position of the braking device 13, the operating position of the work device 2, and the steering angle of the machine body 3.

[0043] The third detection device 28 detects the state of the working device 2. The third detection device 28 includes various sensors installed in each part of the coupling device 8 and the working device 2, and a computing unit, and the computing unit detects (calculates) the operating state of the working device 2 based on output signals from the various sensors. The states of the working device 2 detected by the third detection device 28 include the type of working device 2, the attachment / detachment state of the working device 2, the drive / stop state of each part of the working device 2, the inclination of the working device 2, its height above ground, etc.

[0044] If the type of remote-controlled working machine 1 is a rice transplanter, the third detection device 28 may detect the drive / stop status of each mechanism related to seedling planting, the remaining amount of materials (seedlings, fertilizer), etc., based on output signals from various sensors arranged in each part of the seedling planting device, etc. If the type of remote-controlled working machine 1 is a combine harvester, the third detection device 38 may detect the drive / stop status of each mechanism related to harvesting, the characteristics of the harvested grain, etc., based on output signals from various sensors arranged in each part of the reaping device, grain tank, etc.

[0045] The position detection device 24, the first detection device 25, the state detection device 27, and the third detection device 28 output detection information indicating the results of detection at a predetermined cycle or at a predetermined timing to the on-board control device 21 as needed. The second detection device 26 also outputs sensing information indicating the results of sensing at a predetermined cycle or at a predetermined timing to the on-board control device 21 as needed. The on-board control device 21 stores the detection information and sensing information input from the position detection device 24, the state detection device 27, and the second detection device 26 in the internal memory 21b.

[0046] In addition, the on-board control device 21 sequentially transmits the detection information of the position detection device 24, the first detection device 25, the status detection device 27, and the third detection device 28A stored in the internal memory 21b, and the sensing information of the second detection device 26 to the remote device 30 via the second communication device 23 at a predetermined period or at a predetermined timing in the case of remote driving.

[0047] The detection information and sensing information transmitted from the remote-controlled work machine 1 in this manner includes correspondence data (see FIG. 3 ) that associates the position information of the remote-controlled work machine 1 with captured images of the direction of travel of the remote-controlled work machine 1. FIG. 3 is a diagram showing an example of the correspondence data. That is, correspondence data that associates the detection information of the position detection device 24 (i.e., the position information of the remote-controlled work machine 1) with the sensing information of the second detection device 26 (e.g., images captured by the camera 26c1) is sequentially transmitted to the remote device 30. Regarding the captured images, as shown in FIG. 3 , correspondence data that associates the position information of the remote-controlled work machine 1 detected by the position detection device 24 (e.g., positions PU1, PV1, PW1, PX1) with the images captured by the camera 26c1 (e.g., captured images GPU1, GPV1, GPW1, GPX1) is sequentially transmitted to the remote device 30.

[0048] The on-vehicle control device 21 also communicates with a control unit 2a provided in the work device 2, and causes the control unit 2a to control the operation of the work device 2. That is, the on-vehicle control device 21 performs work on the field by controlling the operation of the work device 2 via the control unit 2a. For example, the control unit 2a is made up of a CPU, memory, etc. Note that some work devices 2 do not have a control unit 2a. In this case, the on-vehicle control device 21 controls the attitude of the work device 2 using the coupling device 8, and the work device 2 performs work on the field.

[0049] The on-board control device 21 controls the travel of the remotely operated work machine 1, the work performed by the work device 2, and other operations of the remotely operated work machine 1 based on detection information acquired from the first detection device 25, the second detection device 26, the state detection device 27, the third detection device 28A, etc. Furthermore, when the on-board control device 21 receives a remote control signal transmitted from the remote device 30 via the second communication device 23, it controls the travel of the remotely operated work machine 1, the work performed by the work device 2, and other operations of the remotely operated work machine 1 based on the remote control signal in addition to the above information.

[0050] Furthermore, when controlling the travel of the remote-controlled work machine 1 or the work performed by the work device 2, the on-board control device 21 determines from the detection information of the object detection unit 26d whether an object is approaching the remote-controlled work machine 1 or the work device 2 within a predetermined distance and posing a risk of contact. If the on-board control device 21 determines that an object is approaching the remote-controlled work machine 1 or the work device 2 within a predetermined distance and posing a risk of contact, it controls the travel device 7 or the work device 2, etc., to automatically stop the travel or work of the remote-controlled work machine 1 and avoid contact with the object.

[0051] Next, the remote device 30 will be described. As shown in Fig. 1 , the remote device 30 is located at a distance from the remote-controlled work machine 1. The remote device 30 allows a remote operator to remotely control the remote-controlled work machine 1 and monitor the state of the remote-controlled work machine 1 and the surrounding conditions of the remote-controlled work machine 1. The remote device 30 includes a control device 31, a storage device 32, a first communication device 33, a display device 34, a remote control device 35, and an input device 36.

[0052] The control device 31 is a processor that controls the operation of each part of the remote device 30. This processor functions as the control device 31, controlling the operation of each part of the remote device 30, for example, by executing a remote control program or application stored in the storage device 32. The control device 31 may be implemented by hardware such as an integrated circuit (IC chip), or by software using a computer. In the latter case, the computer includes a recording medium on which a program, which is software that implements each function of the control device 31, and various data related to the remote work machine 1 are recorded in a computer-readable manner, an arithmetic circuit such as a CPU (Central Processing Unit) that executes the instructions of the program, and a RAM (Random Access Memory) that expands the program and various data. The arithmetic circuit then reads and executes the program from the recording medium, thereby realizing the functions of the control device 31.

[0053] The internal memory 31 a provided in the control device 31 is a volatile or non-volatile memory. Various information and data used by the control device 31 to control the operation of each part of the remote device 30 are stored in the internal memory 31 a in a readable and writable manner.

[0054] The control device 31 includes a display control unit 31b. For example, the processor executes a display control program stored in the storage device 32 or the internal memory 31a, thereby functioning as the display control unit 31b. The display control unit 31b may also be configured as hardware formed on an integrated circuit (IC chip) or the like. The display control unit 31b has a function of controlling a display screen G displayed by the display device 34. For example, the display control unit 31b causes the display device 34 to display a display screen G (see FIG. 4A, etc., described later) showing information related to remote driving.

[0055] The storage device 32 pre-stores applications such as a remote control program for remotely operating the remote work machine 1 and a remote monitoring program for remotely monitoring the remote work machine 1, as well as various data. The storage device 32 is, for example, an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

[0056] The first communication device 33 is composed of an antenna, an IC, an electric circuit, etc. for wireless communication via a mobile phone communication network, the Internet, or a wireless LAN. The first communication device 33 is capable of wireless communication with the remote-controlled work machine 1 under the control of the control device 31. The first communication device 33 receives various data transmitted from the second communication device 23 (detection information from the position detection device 24, the first detection device 25, the state detection device 27, and the third detection device 28, sensing information from the second detection device 26, etc.). For example, the first communication device 33 receives correspondence data that associates the position information of the remote-controlled work machine 1 with an image captured of the direction in which the remote-controlled work machine 1 is traveling.

[0057] The display device 34 is, for example, a liquid crystal display, an organic EL display, etc. The display device 34 displays various information for remotely operating the remote-controlled work machine 1. For example, the display device 34 displays a display screen G showing information related to remote operation based on an instruction signal from the display control unit 31b. Details of the display screen G will be described later.

[0058] The input device 36 is an interface for operating the remote device 30. The remote operator can input predetermined instructions to the remote device 30 by operating the input device 36. The input device 36 includes, for example, a switching operation tool 36a that outputs a switching instruction when an operation to switch the display on the display device 34 is performed. Note that the input device 36 may be a touch panel provided on the display device 34 that can detect touch operations on the display device 34, or a hardware switch or the like.

[0059] The remote control device 35 is a device for remotely controlling the remote-controlled work machine 1. The remote control device 35 includes a steering wheel 35a, an accelerator pedal 35b, a brake pedal 35c, and a gear shift lever 35d, which are arranged around the remote control seat. The remote control device 35 accepts operation by a remote operator. The remote operator sits in the remote control seat and operates the remote control device 35 to remotely control the travel of the remote-controlled work machine 1 or the work performed by the work device 2. The remote operator also monitors the remote-controlled work machine 1 and the surrounding conditions using the display device 34. The remote operator can also input predetermined information or instructions to the remote device 30 by operating the remote control device 35. The remote control device 35 may be a touchpad or a hardware switch, etc.

[0060] 1, the remote device 30 may be composed of a display terminal 70 and a remote control device 35. That is, the display terminal 70 may be a terminal device including a control device 31, a storage device 32, a first communication device 33, a display device 34, and an input device 36. The display terminal 70 may be, for example, a portable terminal device such as a tablet device or a smartphone, or a stationary computer installed in a base station.

[0061] When a remote operator operates the remote control device 35 to input an operation instruction for the remote-controlled work machine 1, the control device 31 generates a remote control signal corresponding to the operation instruction, and transmits the remote control signal to the remote-controlled work machine 1 via the first communication device 33. In other words, a remote control signal corresponding to operation of the steering wheel 35a, accelerator pedal 35b, brake pedal 35c, and gear shift lever 35d is transmitted to the remote-controlled work machine 1. When the on-board control device 21 of the remote-controlled work machine 1 receives the remote control signal from the remote device 30 via the second communication device 23, it operates each part of the remote-controlled work machine 1 based on the remote control signal, detection information from the position detection device 24, sensing information from the second detection device 26, and detection information from the state detection device 27, to control the traveling and steering of the remote-controlled work machine 1 and to control the work operation of the work device 2.

[0062] Furthermore, the on-board control device 21 periodically transmits detection information from the position detection device 24, the first detection device 25, the state detection device 27, and the third detection device 28, and sensing information from the second detection device 26, to the remote device 30 via the second communication device 23. When the control device 31 of the remote device 30 receives the detection information and sensing information via the first communication device 33, it stores the information in the internal memory 31a and displays it on the display device 34.

[0063] The control device 31 can cause the display device 34 to display a display screen G showing information related to remote driving. The display device 34 displays the display screen G based on an instruction signal from the control device 31 (display control unit 31b). Specifically, the display control unit 31b converts the information related to remote driving into a data format (image) that can be displayed on the display device 34. The display control unit 31b inputs the converted image and the display position of the image on the display device 34 as an instruction signal to the display device 34, causing the display screen G to be displayed.

[0064] 4A to 4C are diagrams showing an example of the display screen G. As shown on display screens G1 and G3 in FIGS. 4A and 4C , the display control unit 31b causes the display device 34 to display a first captured image 70a, which is an image of the area in front of the remote-controlled work machine 1 taken from the driver's perspective by camera 26c1 ( FIG. 2 ) installed inside the cabin 9, out of the multiple cameras 26c provided on the remote-controlled work machine 1. As shown in FIG. 4B , the display control unit 31b causes the display screen G2 of the display device 34 to display a second captured image 70b, which is an image of the area behind the remote-controlled work machine 1 taken by camera 26c2 ( FIG. 2 ) installed at the rear of the machine body 3. In the following description, the first captured image 70a and the second captured image 70b may be collectively referred to as the captured images 70a and 70b.

[0065] The display device 34, based on control by the display control unit 31b (for example, a display control signal), displays a path object 90 indicating a predicted travel path of the remote-controlled work machine 1 superimposed on captured images 70a, 70b of the surroundings of the remote-controlled work machine 1. The path object 90 includes a traveling object 91 indicating a predicted movement path of the traveling device 7 of the remote-controlled work machine 1, and a working object 92 indicating a predicted movement path of the working device 2 of the remote-controlled work machine 1. The traveling object 91 includes a first traveling object 91a indicating a predicted movement path of the front wheel 7F of the traveling device 7, and a second traveling object 91b indicating a predicted movement path of the rear wheel 7R of the traveling device 7.

[0066] The first running object 91a is an image object (image data) that shows the path along which the front wheels 7F of the running device 7 will move (pass) if the running device 7 continues to run at the current steering angle of the vehicle body 3, i.e., the predicted path of the front wheels 7F of the running device 7. The second running object 91b is an image object (image data) that shows the path along which the rear wheels 7R of the running device 7 will move (pass) if the running device 7 continues to run at the current steering angle of the vehicle body 3, i.e., the predicted path of the rear wheels 7R of the running device 7. The work object 92 is an image object (image data) that shows the path along which both ends of the work device 2 that form the maximum width will move (pass) if the running device 7 continues to run at the current steering angle of the vehicle body 3, i.e., the predicted path of the maximum width of the work device 2.

[0067] The control device 31 acquires the steering angle (detection information) of the machine body 3 from the remote-controlled work machine 1 via the first communication device 33, and dimensional information of the traveling device 7 and the work device 2 of the remote-controlled work machine 1. The dimensional information of the traveling device 7 and the work device 2 may be stored in advance in the storage device 32 of the remote device 30.

[0068] The control device 31 defines (calculates) predicted movement paths of the front wheels 7F, the rear wheels 7R, and both ends of the work device 2 based on the current steering angle of the vehicle body 3 and dimensional information of the traveling device 7 and the work device 2. The display control unit 31b converts the calculation result of the predicted movement path into a data format (object) displayable on the display device 34 and displays it as a path object 90 superimposed on the captured images 70a, 70b. As shown in FIGS. 4A and 4B , a first running object 91a, a second running object 91b, and a work object 92 are displayed superimposed on the captured images 70a, 70b. For example, the display control unit 31b identifies, by image recognition, the positions of the front wheels 7F, the rear wheels 7R, and both ends of the work device 2 that are at their widest points on the display screen G, as base point positions, and converts the calculation result of the predicted movement path into an image object extending from the base point position on the display screen G to generate a path object 90, which is displayed superimposed on the captured images 70a, 70b.

[0069] The remote control device 35 (speed change lever 35d) accepts a first operation to switch the traveling direction of the traveling device 7 to forward (the direction of arrow A1 in Figure 2), a second operation to switch the traveling direction of the traveling device 7 to reverse (the direction of arrow A2 in Figure 2), and a third operation to switch the traveling device 7 to neutral, separate from the first and second operations.

[0070] The display device 34 switches between displaying and hiding the path object 90 (the traveling object 91 and the work object 92) in accordance with an operation received by the remote control device 35. The display device 34 switches between displaying and hiding the path object 90 in accordance with an operation received by the remote control device 35 to switch the traveling direction of the traveling device 7. Specifically, when the remote control device 35 is operated in a first manner, the display device 34 displays the first captured image 70a and the first traveling object 91a, and when the remote control device 35 is operated in a second manner, the display device 34 displays the second captured image 70b and the second traveling object 91b. When the remote control device 35 is operated in a third manner, the display device 34 hides the traveling object 91 and the work object 92.

[0071] The storage device 32 stores in advance a table T1 that associates operations accepted by the remote control device 35 with objects to be displayed on the display device 34. Fig. 5A is a diagram showing an example of the table T1. In Fig. 5A, for convenience, objects are represented as "Obj". This also applies to Figs. 5B and 6, which will be described later. In table T1, "0" indicates that the object is not to be displayed, and "1" indicates that the object is to be displayed. This also applies to Fig. 5B, which will be described later. In table T1, "-" indicates that the display or non-display state is not changed (the state before the operation is maintained).

[0072] When the remote control device 35 (gear shift lever 35d) receives any one of the first, second, or third operations, the control device 31 transmits a corresponding remote control instruction signal to the remote work machine 1, and the display control unit 31b refers to table T1 and controls switching between displaying and hiding the path object 90 and the captured images 70a and 70b.

[0073] Fig. 6 is a flowchart showing the display procedure of the display screen G. Each step of the flowchart shown on the left side of Fig. 6 is executed by the on-board control device 21 based on a software program stored in the storage device 22. Each step of the flowchart shown on the right side of Fig. 6 is executed by the control device 31 of the remote device 30 based on a software program stored in the storage device 32.

[0074] When the remote device 30 starts up and receives an instruction input to remotely control the remote work machine 1 via the input device 36, the control device 31 starts controlling the remote operation of the remote work machine 1. As shown in Fig. 6, when the control device 31 of the remote device 30 starts controlling the remote operation of the remote work machine 1, the display control unit 31b of the remote device 30 starts processing to display the display screen G on the display device 34 (S100).

[0075] The on-board control device 21 of the remote-operated work machine 1 transmits the captured images 70a and 70b captured by the camera 26c from the remote-operated work machine 1 to the remote device 30 via the second communication device 23 (S101).

[0076] When the first communication device 33 of the remote device 30 receives the captured images 70a and 70b, the display control unit 31b displays the first captured image 70a or the second captured image 70b on the display screen G of the display device 34 based on the definition of table T1 (S102).

[0077] Furthermore, the on-board control device 21 transmits the steering angle of the remote-controlled work machine 1 to the remote device 30 via the second communication device 23 (S103).

[0078] When the first communication device 33 of the remote device 30 receives the steering angle, the display control unit 31b defines (calculates) a predicted movement path. The display control unit 31b refers to the table T1 and displays the calculation result of the predicted movement path on the display screen G (S104).

[0079] The control device 31 also determines whether the remote control device 35 has accepted an operation (S105). If the control device 31 determines that the remote control device 35 has accepted an operation (S105: YES), it transmits a corresponding remote control instruction signal to the remote-controlled work machine 1 (S106). The control device 31 refers to table T1 and performs an operation to switch between displaying and hiding the path object 90 (S107).

[0080] The remote-controlled work machine 1 sequentially executes the processes of steps S101 and S103, which are the transmission of the captured images 70a and 70b and the steering angle to the remote device 30. The display control unit 31b sequentially executes the processes of steps S102 and S104 to S107 upon reception by the remote-controlled work machine 1 and upon acceptance of an operation by the remote control device 35. The display control unit 31b may execute the processes of steps S102 and S104 to S107 at a predetermined cycle.

[0081] Next, transitions of the display screen G in response to operations accepted by the remote control device 35 will be described with reference to FIGS. 4A to 4C.

[0082] When the remote operator performs a first operation on the gear shift lever 35d, the display control unit 31b refers to the table T1 and displays on the display screen G1, as shown in Fig. 4A, a first captured image 70a captured in front of the remote-controlled work machine 1, a first traveling object 91a indicating the predicted movement path of the front wheel 7F, and a work object 92 indicating the predicted movement path of the work implement 2. The display screen G1 shown in Fig. 4A is an example of the display screen G when the remote operator performs the first operation on the remote control device 35. The display screen G1 displays the first traveling object 91a and the work object 92 superimposed on the first captured image 70a.

[0083] When the remote operator performs the second operation on the gear shift lever 35d, the display control unit 31b refers to the table T1 and displays on the display screen G1, as shown in Fig. 4B, a second captured image 70b captured behind the remote-controlled work machine 1, a second traveling object 91b indicating the predicted movement path of the rear wheel 7R, and a work object 92 indicating the predicted movement path of the work implement 2. The display screen G2 shown in Fig. 4B is an example of the display screen G when the remote operator performs the second operation on the remote control device 35. The display screen G2 displays the second traveling object 91b and the work object 92 superimposed on the second captured image 70b.

[0084] When the remote operator performs the third operation on the gearshift lever 35d, the display control unit 31b refers to the table T1 and, as shown in FIG. 4C , hides the traveling objects 91 (the first traveling object 91a and the second traveling object 91b) and the work object 92 indicating the predicted movement path of the work device 2 on the display screen G1. The display screen G3 shown in FIG. 4C is an example of the display screen G when the remote operator performs the third operation on the gearshift lever 35d while the display screen G1 (the first captured image 70a) is displayed on the display device 34. The display screen G3 maintains the display of the first captured image 70a that was displayed before the third operation was performed, and the traveling objects 91 and the work object 92 are not displayed.

[0085] <Modification> In the embodiment described above, the display control unit 31b controls switching between displaying and hiding the captured images 70a, 70b and the path object 90 in accordance with an operation received by the remote control device 35. In contrast, in the modification, the display control unit 31b mainly switches between displaying and hiding the path object 90 in accordance with the imaging direction of the captured images 70a, 70b displayed on the display device 34 and the traveling direction of the traveling device 7 (remotely operated work machine 1).

[0086] 7 is a diagram showing an example of a display screen G in a modified example. For example, as shown in display screens G4 and G5 in FIG. 7 , the display control unit 31b causes the display device 34 to display a first status display section 80a and a second status display section 80b that indicate the status of the remote-controlled work machine 1 detected by the first detection device 25 and the on-board control device 21 of the remote-controlled work machine 1. The first status display section 80a displays whether the remote-controlled work machine 1 is traveling by remote control, the traveling speed of the remote-controlled work machine 1 (machine body 3), and the rotation speed of the prime mover 4. The second status display section 80b displays the traveling direction (forward, reverse, neutral) of the traveling device 7, the auxiliary transmission status (e.g., gear position) and main transmission status (e.g., gear position) of the transmission 5, the traveling mode (e.g., 2WD) of the remote-controlled work machine 1, and the accelerator pedal operation amount.

[0087] The display device 34 has a first image display unit 81 and a second image display unit 82. For example, the first image display unit 81 is larger than the second image display unit 82. The first image display unit 81 is a main display screen, and the second image display unit 82 is a sub-display screen. Therefore, the first image display unit 81 displays an image that the remote operator is desired to focus on more than the second image display unit 82. The display control unit 31b displays one of two selected images captured by the multiple cameras 26c provided on the remote-operated work machine 1 on the first image display unit 81, and displays the other image on the second image display unit 82.

[0088] 7, on the display screen G4, the first image display section 81 displays a first captured image 70a showing the front of the remote-controlled work machine 1, and the second image display section 82 displays a second captured image 70b showing the rear of the remote-controlled work machine 1. On the display screen G5, the first image display section 81 displays the second captured image 70b, and the second image display section 82 displays the first captured image 70a. Note that the display screens G4 and G5 shown in FIG. 7 are merely examples, and the type of information to be displayed, the position of the information, etc. are not limited.

[0089] As described above, the remote device 30 has the switching operation device 36a. When a display switching operation is performed on the display device 34, the switching operation device 36a outputs a switching instruction to swap the display of, for example, the first image display unit 81 and the second image display unit 82 of the display device 34. When the switching operation device 36a is operated, the display device 34 switches from one of the first captured image 70a and the second captured image 70b to the other in accordance with the operation of the switching operation device 36a, regardless of the first operation and the second operation of the remote control device 35. When the remote operator operates the switching operation device 36a, the display control unit 31b inputs an instruction signal to the display device 34 to switch from one of the first captured image 70a and the second captured image 70b displayed on the first image display unit 81 and the second image display unit 82 to the other.

[0090] For example, when the display screen G4 is displayed on the display device 34, the display control unit 31b transitions to the display screen G5 upon receiving a display switching instruction from the switching operation device 36a. More specifically, the display control unit 31b causes the first image display unit 81 to display the second captured image 70b instead of the first captured image 70a, and causes the second image display unit 82 to display the first captured image 70a instead of the second captured image 70b.

[0091] Furthermore, when the display screen G5 is displayed on the display device 34, the display control unit 31b transitions to the display screen G4 upon receiving a display switching instruction from the switching operation device 36a. Specifically, the display control unit 31b causes the first image display unit 81 to display the first captured image 70a instead of the second captured image 70b, and causes the second image display unit 82 to display the second captured image 70b instead of the first captured image 70a.

[0092] When the traveling direction of the traveling device 7 due to the first operation and the second operation matches the imaging direction of the captured images 70a, 70b, the display device 34 displays the traveling object 91 and the work object 92, and when they do not match, the display device 34 hides the traveling object 91 and the work object 92. In more detail, when the remote control device 35 is operated in a first manner, the display device 34 displays the first traveling object 91a and the work object 92 in the first captured image 70a and hides the second traveling object 91b, and when the remote control device 35 is operated in a second manner, the display device 34 displays the second traveling object 91b and the work object 92 in the second captured image 70b and hides the first traveling object 91a.

[0093] 5B is a diagram showing an example of table T2 in Modification 1. Table T2 defines whether to display or hide the path object 90 in response to the operation of the remote control device 35, corresponding to the first captured image 70a and the second captured image 70b. As described above, the first captured image 70a is an image captured in front of the remotely controlled work machine 1. Table T2 holds a definition for displaying a first traveling object 91a and a work object 92 superimposed on the first captured image 70a in a first operation, which is an operation for moving the remotely controlled work machine 1 forward. The second captured image 70b is an image captured behind the remotely controlled work machine 1. Table T2 holds a definition for displaying a second traveling object 91b and a work object 92 superimposed on the second captured image 70b in a second operation, which is an operation for moving the remotely controlled work machine 1 backward.

[0094] When the remote control device 35 (gear shift lever 35d) receives any one of the first, second, or third operations, the control device 31 transmits a corresponding remote control instruction signal to the remote work machine 1, and the control device 31 (display control unit 31b) refers to table T2 and controls the display of a path object 90 on the captured images 70a, 70b that match the direction of travel of the traveling device 7.

[0095] 8A and 8B are diagrams showing an example of the display screen G when a first operation is performed on the remote control device 35 in the modified example. Figures 8C and 8D are diagrams showing an example of the display screen G when a second operation is performed on the remote control device 35 in the modified example. The transition of the display screen G will be described below using Figures 8A to 8D.

[0096] 8A and 8B , when the remote operator performs a first operation on the gear shift lever 35d, the display control unit 31b refers to table T2 and displays the first traveling object 91a and the work object 92 in the first captured image 70a, and hides the second traveling object 91b. For example, in the display screen G6 ( FIG. 8A ), the second status display unit 80b displays an indication 80b1 ("Shuttle: F," i.e., the shuttle lever is in the forward position) indicating that the traveling direction of the traveling device 7 has been switched to the forward direction. In the display screen G6, the first traveling object 91a and the work object 92 are superimposed on the first captured image 70a displayed in the first image display unit 81.

[0097] 8A and 8B , when the remote operator operates the switching operation tool 36a while the display screen G6 is displayed on the display device 34, the display control unit 31b controls switching between the first captured image 70a displayed on the first image display unit 81 and the second captured image 70b displayed on the second image display unit 82. After switching the display of the captured images 70a and 70b, the display control unit 31b refers to table T2 and causes the second image display unit 82, which is displaying the first captured image 70a, to display the first running object 91a and the work object 92, while hiding the second running object 91b. On the display screen G7, the first running object 91a and the work object 92 are superimposed on the first captured image 70a displayed on the second image display unit 82.

[0098] 8B and 8C , when the remote operator performs a second operation on the gear shift lever 35d while the display screen G7 is displayed on the display device 34, the table T2 is referenced, and the second traveling object 91b and the work object 92 are displayed in the second captured image 70b, and the first traveling object 91a is hidden. For example, in the display screen G8 ( FIG. 8C ), the second status display section 80b shows an indication 80b2 ("Shuttle: R," i.e., the shuttle lever is in the reverse position) indicating that the traveling direction of the traveling device 7 has been switched to the reverse direction. In the display screen G8, the second traveling object 91b and the work object 92 are superimposed on the second captured image 70b displayed in the first image display section 81.

[0099] 8C and 8D , when the remote operator operates the switching operation tool 36a while the display screen G8 is displayed on the display device 34, the display control unit 31b controls switching between the second captured image 70b displayed on the first image display unit 81 and the first captured image 70a displayed on the second image display unit 82. Furthermore, when the display control unit 31b switches the display of the captured images 70a and 70b, it refers to table T2 and causes the second image display unit 82, which is displaying the second captured image 70b, to display the second running object 91b and the work object 92, while hiding the first running object 91a. On the display screen G9, the second running object 91b and the work object 92 are superimposed on the second captured image 70b displayed on the second image display unit 82.

[0100] The display forms of the first running object 91a, the second running object 91b, and the work object 92 are not limited to those described above, and the line type (e.g., straight line, dashed line), line width, and color may be different. The path object 90 may also be an animation that moves along the traveling direction of the remote-controlled work machine 1. The display control unit 31b may also change the display form of the path object 90 depending on the captured images 70a, 70b to be superimposed.

[0101] The main characteristic features and effects of the remote device 30 and remote control system 100 in the embodiment described above are as follows.

[0102] (Item A1) A remote device 30 comprising a first communication device 33 capable of communicating with a remote work machine 1, a remote control device 35 that receives operations and remotely controls the remote work machine 1 via the first communication device 33, and a display device 34 that displays a route object 90 indicating a predicted travel route of the remote work machine 1 superimposed on captured images 70a, 70b of the surroundings of the remote work machine 1, wherein the display device 34 switches between displaying and hiding the route object 90 depending on the operations received by the remote control device 35.

[0103] The remote device 30 according to item A1 displays a route object 90 indicating the predicted driving route of the remote-controlled work machine 1, allowing the remote operator to grasp the predicted driving route and perform remote control based on the predicted driving route. Therefore, the remote device 30 can support the remote operator in remote control by displaying the predicted driving route. Furthermore, because the display and non-display of the route object 90 can be changed in response to an operation, the predicted driving route can be displayed appropriately in response to remote control.

[0104] (Item A2) The path object 90 includes a traveling object 91 indicating the predicted movement path of the traveling device 7 possessed by the remote work machine 1, and a work object 92 indicating the predicted movement path of the work device 2 possessed by the remote work machine 1, and the display device 34 switches between displaying and hiding the traveling object 91 and the work object 92 depending on the operation received by the remote control device 35. The remote device 30 described in Item A1.

[0105] The remote device 30 according to item A2 allows the remote operator to remotely control the remote-controlled work machine 1 while understanding the predicted movement paths of the traveling device 7 and the work device 2, thereby improving the operability of remote control. Therefore, by displaying the predicted movement paths of the traveling device 7 and the work device 2, the remote device 30 can more specifically support the remote operator in remote control.

[0106] (Item A3) The running object 91 includes a first running object 91a that indicates the predicted movement path of the front wheel 7F of the running device 7, and a second running object 91b that indicates the predicted movement path of the rear wheel 7R of the running device 7, the remote control device 35 is capable of a first operation to switch the running device 7 to forward movement and a second operation to switch the running device 7 to reverse movement, and the display device 34 displays a first captured image 70a that is an image of the area in front of the remote work machine 1 and the first running object 91a when the remote control device 35 is operated in the first manner, and displays a second captured image 70b that is an image of the area behind the remote work machine 1 and the second running object 91b when the remote control device 35 is operated in the second manner.

[0107] According to the remote device 30 relating to item A3, when the traveling device 7 is moving forward, the predicted movement path of the front wheel 7F is displayed, so the remote operator can grasp the predicted movement path of the front wheel 7F without performing any display operations, and when the traveling device 7 is moving backward, the predicted movement path of the rear wheel 7R is displayed, so the remote operator can grasp the predicted movement path of the rear wheel 7R without performing any display operations.

[0108] (Item A4) A remote device 30 described in Item A3, which is equipped with a switching operation device 36a that accepts an operation to switch the display of the display device 34, and when the switching operation device 36a is operated, the display device 34 switches between the first captured image 70a and the second captured image 70b in accordance with the operation of the switching operation device 36a, regardless of the first operation and the second operation of the remote control device 35.

[0109] According to the remote device 30 relating to item A4, the first captured image 70a and the second captured image 70b displayed on the display device 34 can be switched as appropriate, allowing the remote operator to properly grasp the surroundings of the work machine.

[0110] (Item A5) The display device 34 displays the running object 91 and the work object 92 when the direction of travel of the running device 7 due to the first operation and the second operation matches the imaging direction of the captured image, and hides the running object 91 and the work object 92 when they do not match. This is the remote device 30 described in Item A4.

[0111] According to the remote device 30 of item A5, the path object 90 is displayed in captured images in the traveling direction of the remote-controlled work machine 1, and the path object 90 is not displayed in captured images that are not in the traveling direction of the remote-controlled work machine 1. This allows the remote operator to understand that the captured image in which the path object 90 is displayed is the traveling direction of the remote-controlled work machine 1. In other words, the remote operator can be prevented from mistaking the traveling direction of the remote-controlled work machine 1 for the path object 90 on the display device 34.

[0112] (Item A6) The remote device 30 described in Item A5, wherein the display device 34 displays the first running object 91a and the work object 92 and does not display the second running object 91b when the remote control device 35 is operated in the first manner and the first captured image 70a is displayed, and displays the second running object 91b and the work object 92 and does not display the first running object 91a when the remote control device 35 is operated in the second manner and the second captured image 70b is displayed.

[0113] According to the remote device 30 relating to item A6, when the traveling device 7 is moving forward, the movement path of the front wheels 7F and the work device 2 is displayed, so the remote operator can grasp the predicted movement path of the front wheels 7F and the work device 2 without performing any display operations, and when the traveling device 7 is moving backward, the movement path of the rear wheels 7R and the work device 2 is displayed, so the remote operator can grasp the predicted movement path of the rear wheels 7R and the work device 2 without performing any display operations.

[0114] (Item A7) The remote control device 35 is capable of a third operation of switching the traveling device 7 to neutral, separate from the first operation and the second operation, and the display device 34 hides the traveling object 91 and the work object 92 when the remote control device 35 is operated in the third manner. The remote control device 30 is described in any one of items A3 to A6.

[0115] According to the remote device 30 relating to item A7, when the running device 7 is in neutral and the remote work machine 1 is stopped, the running object 91 and the work object 92 are hidden in the captured images 70a and 70b, making it easier to see the information contained in the first captured image 70a and the second captured image 70b.

[0116] (Item A8) A remote control system 100 comprising a remote device 30 according to any one of items A1 to A7 and the remote work machine 1 remotely controlled by the remote device 30, wherein the remote work machine 1 has a second communication device 23 that communicates with the first communication device 33.

[0117] According to the remote control system 100 according to item A8, it is possible to realize a remote control system that provides the advantageous effects described above.

[0118] Although the present invention has been described above, the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims.

[0119] 1: Remotely controlled work machine 2: Work device 7: Traveling device 7F: Front wheel 7R: Rear wheel 23: Second communication device 30: Remote device 33: First communication device 34: Display device 35: Remote control device 36a: Switching operation tool 70a: First captured image 70b: Second captured image 90: Path object 91: Traveling object 91a: First traveling object 91b: Second traveling object 92: Work object 100: Remote control system

Claims

1. A first communication device capable of communicating with a remote working machine, a remote control device that receives an operation and remotely controls the remote working machine via the first communication device, and a display device that superimposes and displays a path object indicating a predicted travel path of the remote working machine on a captured image of the surroundings of the remote working machine, the display device being a remote device that switches between displaying and not displaying the path object according to the operation received by the remote control device.

2. The path object includes a travel object indicating a predicted movement path of a travel device of the remote working machine and a work object indicating a predicted movement path of a work device of the remote working machine, and the display device is the remote device according to claim 1 that switches between displaying and not displaying the travel object and the work object according to the operation received by the remote control device.

3. The travel object includes a first travel object indicating a predicted movement path of a front wheel among the travel devices and a second travel object indicating a predicted movement path of a rear wheel among the travel devices, the remote control device is capable of a first operation of switching the travel device to forward and a second operation of switching the travel device to reverse, and the display device displays a first captured image of the front of the remote working machine and the first travel object when the remote control device is subjected to the first operation, and displays a second captured image of the rear of the remote working machine and the second travel object when the remote control device is subjected to the second operation. The remote device according to claim 2.

4. A switching operation tool for receiving a switching operation of the display of the display device is provided, and when the switching operation tool is operated, the display device switches the first captured image and the second captured image according to the operation of the switching operation tool regardless of the first operation and the second operation of the remote control device. The remote device according to claim 3.

5. The display device displays the travel object and the work object when the traveling direction of the travel device by the first operation and the second operation coincides with the imaging direction of the captured image, and does not display the travel object and the work object when they do not coincide. The remote device according to claim 4.

6. The display device displays the first traveling object and the work object and does not display the second traveling object when the remote control device is under the first operation and displaying the first captured image; and displays the second traveling object and the work object and does not display the first traveling object when the remote control device is under the second operation and displaying the second captured image. The remote device according to claim 5.

7. The remote control device is capable of a third operation of switching the traveling device to neutral, separate from the first operation and the second operation; and the display device does not display the traveling object and the work object when the remote control device is under the third operation. The remote device according to claim 3.

8. A remote control system comprising the remote device according to any one of claims 1 to 7, and the remote working machine remotely controlled by the remote device, wherein the remote working machine has a second communication device that communicates with the first communication device.

Images