Remote device and remote control system

Abstract

The present invention suppresses interference with remote control performed by a remote operator.　A remote device (30) is provided with: a first communication device (33) that can communicate with a remote work machine (1); a remote control device (35) that receives an operation ; a control device (31) that executes, in accordance with the operation of the remote control device (35), a first application (32a) that performs processing relating to remote control of the remote work machine (1) via the first communication device (33), and a second application (32b) that performs processing different from the remote control; and a display device (34) that displays a first application screen (W1) indicating information about the first application (32a), and a second application screen (W2) indicating information about the second application (32b). When the control device (31) is executing the remote control by means of the first application (32a), the display device (34) displays the first application screen (W1) in preference to the second application screen (W2).

Description

Remote device and remote control system

[0001] The present invention relates to a remote device and a remote control system for remotely controlling a work machine.

[0002] BACKGROUND ART Conventionally, there is known a remote control system that generates a monitoring screen for monitoring the traveling state and working state of a work vehicle and displays the screen on a display (for example, Patent Document 1).

[0003] Japanese Patent Publication No. 2021-36796

[0004] The remote operator must check the monitoring screen while remotely controlling the vehicle. The remote control system of Patent Document 1 does not take into consideration the possibility that information other than information related to remote control may be displayed on the display, thereby interfering with the remote operator's remote control.

[0005] The present invention has been made to solve the problems of the conventional technology, and has an object to provide a remote device and a remote control system that can prevent the remote control of the remote operator from being hindered.

[0006] A remote device according to one aspect of the present invention includes a first communication device capable of communicating with a remote work machine, a remote control device that accepts operations, a control device that executes a first application that performs processing related to remote control of the remote work machine via the first communication device in response to operation of the remote control device, and a second application that performs processing separate from the remote control, and a display device that displays a first application screen that shows information related to the first application and a second application screen that shows information related to the second application, and when the control device is performing the remote control using the first application, the display device displays the first application screen in priority over the second application screen.

[0007] When the display device receives an instruction to display the second application screen while the control device is performing the remote control using the first application, the display device may display the first application screen instead of the second application screen as a priority process.

[0008] The first application may be switchable between a first mode in which the remote control is performed and a second mode in which the remote work machine is monitored without performing the remote control, and when the first application is in the first mode, the display device may perform the priority processing when it receives an instruction to display the second application screen, and when the first application is in the second mode, the display device may display the second application screen without performing the priority processing when it receives an instruction to display the second application screen.

[0009] When the control device stops the remote control using the first application while the priority processing is being performed, the display device may display the second application screen in place of the first application screen.

[0010] When the display device receives an instruction to display the second application screen while the control device is performing the remote control using the first application, the display device may perform the priority processing of displaying the second application screen and then returning the display to the first application screen.

[0011] When the display device receives an instruction to display the second application screen while the control device is performing the remote control using the first application, the display device may display the second application screen and then perform the priority processing of displaying the first application screen in front of the second application screen.

[0012] When the display device receives an instruction to display the second application screen while the control device is performing the remote control using the first application, the display device may perform the priority processing of displaying the second application screen in an area that does not overlap with the first application screen.

[0013] When the control device is performing the remote control using the first application, the display device may display the second application screen upon receiving an instruction to display the second application screen, and the control device may stop the remote control of the first application when the display device displays the second application screen.

[0014] The remote device may include an input device that accepts input, the display device may display a first object that accepts a display instruction for the first application and a second object that accepts a display instruction for the second application screen, and the input device may accept operations in an area excluding the second object when the control device is performing the remote control using the first application.

[0015] The display device may display a third object indicating an indication position corresponding to operation of the input device, and when the control device is performing the remote control using the first application, the display range of the third object may be limited to an area excluding the second object.

[0016] The remote device may include an input device that accepts input, and the display device may display a first object that is operated by the input device and accepts an instruction to display the first application, and a second object that is operated by the input device and accepts an instruction to display the second application screen, and the input device may not accept operations on the first object and the second object while the display device is displaying the first application screen.

[0017] A remote control system according to one embodiment of the present invention comprises the above-mentioned remote device and the remote work machine that is remotely controlled by the remote device, and the remote work machine has a second communication device that communicates with the first communication device.

[0018] According to the present invention, it is possible to prevent the remote operator from being hindered from remotely operating the vehicle.

[0019] 1 is a configuration diagram of a remote control system according to an embodiment of the present invention. FIG. 1 is a side view of a tractor, which is an example of a remote work machine. FIG. 2 is a diagram showing an example of correspondence data. FIG. 3 is a diagram showing an example of a display screen (first application screen). FIG. 4 is a diagram showing an example of a display screen (second application screen). FIG. 5 is a diagram for explaining an example of a second application. FIG. 6 is a first diagram for explaining priority processing. FIG. 7 is a second diagram for explaining priority processing. FIG. 8 is a third diagram for explaining priority processing. FIG. 9 is a diagram showing another example of a display screen of a display device in a second modified example. FIG. 10 is a diagram for explaining priority processing in a third modified example. FIG. 11 is a flowchart showing the operation of a display control unit. FIG. 12 is a sequence diagram showing the operation of an input device, a display control unit, and a control device. FIG. 13 is a sequence diagram showing the operation of an input device, a display control unit, and a control device in a first modified example. FIG. 14 is a flowchart showing the operation of a control device in a second modified example. FIG. 15 is a diagram showing an example of a display screen of a display device in a third modified example.

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0021] FIG. 1 is a configuration diagram of a remotely controlled system 100 according to an embodiment of the present invention. The remotely controlled system 100 includes a remotely controlled work machine 1 and a remote device 30. The remotely controlled work machine 1 and the remote device 30 enable remote control (or remote operation) and remote monitoring of the remotely controlled work machine 1. The remotely controlled work machine 1 is a farm machine (also referred to as a "remotely controlled agricultural machine") that can be remotely driven (e.g., remotely driven and remotely operated) using the remote device 30. The remotely controlled work machine 1 includes agricultural machines such as tractors, rice transplanters, and combine harvesters, each equipped with a work device 2 such as an implement. For convenience of explanation, the following description will use a tractor as an example of the remotely controlled work machine 1, and will omit descriptions of other agricultural machines. In the following description, the remotely controlled work machine 1 will be described as a tractor having a driver's seat 10 in which a worker (operator) sits. However, the remotely controlled work machine 1 may also be an agricultural machine dedicated to remote control that does not have a driver's seat 10.

[0022] FIG. 2 is a side view of a tractor, which is an example of the remote-controlled work machine 1. In the following description, the front side (in the direction of arrow A1 in FIG. 2 ) of an operator seated in the driver's seat 10 of the remote-controlled work machine 1 will be referred to as the front, the rear side (in the direction of arrow A2 in FIG. 2 ) of the operator will be referred to as the rear, the left side of the operator will be referred to as the left (the near side in FIG. 2 ), and the right side of the operator will be referred to as the right (the far side in FIG. 2 ). The horizontal direction, which is perpendicular to the fore-and-aft direction of the remote-controlled work machine 1, will be referred to as the width direction. The remote-controlled work machine 1 includes a body 3. The body 3 is provided with a traveling device 7. The traveling device 7 has front wheels 7F and rear wheels 7R on both the left and right sides of the body 3, supporting the body 3 so that it can travel. The traveling device 7 may be a crawler-type device.

[0023] 1 and 2, the aircraft body 3 is equipped with a prime mover 4, a transmission 5, a braking device 13, and a steering device 14. The prime mover 4 is formed of an engine (diesel engine, gasoline engine), an electric motor, or the like.

[0024] The transmission 5, for example, performs a gear change operation to vary the propulsive force of the traveling device 7 and switch between forward and reverse traveling of the traveling device 7. The transmission 5 has multiple gears that transmit power, a shifter that changes the connection of the gears, and a clutch that switches between transmitting and disconnecting power, and performs the gear change operation using these. The transmission 5 can also switch between driving modes: four-wheel drive (4WD) in which the front wheels 7F and the rear wheels 7R are driven by power and the rotational speeds of the front wheels 7F and the rear wheels 7R are substantially the same (4WD constant speed state); four-wheel drive in which the rotational speed of the front wheels 7F is faster than the rotational speed of the rear wheels 7R (4WD speed increase state); and two-wheel drive (2WD) in which only the rear wheels 7R are driven. Note that in this embodiment, the transmission 5 can perform separate gear changes for the main transmission unit (e.g., continuously variable transmission) and the sub-transmission unit (e.g., stepped transmission).

[0025] The braking device 13 brakes the airframe 3. The steering device 14 steers the airframe 3.

[0026] A control device 11 for manually operating the remote-controlled work machine 1 is provided around the driver's seat 10. The remote-controlled work machine 1 can travel and work unmanned, but an operator seated in the driver's seat 10 can also operate the control device 11 to travel the remote-controlled work machine 1 and perform work using the work device 2. The control device 11 includes a steering wheel 11a, an accelerator pedal 11b, a brake pedal, a gear shift lever, and the like.

[0027] A coupling device 8 is provided at the rear of the machine body 3. The coupling device 8 is configured with a three-point link mechanism or the like. A working device 2 (implement) can be attached and detached to the coupling device 8. The remote-controlled work machine 1 (machine body 3) can tow the working device 2 by coupling the working device 2 to the coupling device 8 and traveling using the traveling device 7. In addition, the coupling device 8 can raise and lower the working device 2 and change its posture by driving an actuator (e.g., a hydraulic cylinder).

[0028] Examples of the working device 2 include a tilling device for tilling, a fertilizer spreading device for spreading fertilizer, a pesticide spreading device for spreading pesticide, a harvesting device for harvesting, a reaping device for reaping grass etc., a spreading device for spreading grass etc., a grass collecting device for collecting grass etc., a shaping device for shaping grass etc. The remote-controlled working machine 1 performs agricultural work in the field using these working devices 2.

[0029] 1 , the remote-operated work machine 1 includes an on-board control device 21, a storage device 22, a second communication device (on-board communication device) 23, a position detection device 24, a first detection device (inertial measurement unit) 25, a second detection device (sensing device) 26, a state detection device 27, and a third detection device 28. In addition, an on-board network such as CAN, LIN, or FlexRay is configured in the remote-operated work machine 1, and the on-board control device 21 is electrically connected to the storage device 22, the second communication device 23, the position detection device 24, etc.

[0030] The on-vehicle control device 21 is composed of an ECU (electronic control unit) including a processor 21a and a memory 21b. The on-vehicle control device 21 is a controller that controls the operation of each part of the remote-controlled work machine 1. The on-vehicle control device 21 controls a group of actuators including electric or hydraulic motors, cylinders, control valves, etc. for operating each part of the remote-controlled work machine 1, such as the prime mover 4, transmission 5, traveling device 7, braking device 13, and coupling device 8. The memory 21b is composed of volatile or non-volatile memory, etc. Various information and data used by the on-vehicle control device 21 to control the operation of each part of the remote-controlled work machine 1 are stored in a readable and writable manner in the memory 21b of the on-vehicle control device 21.

[0031] The storage device 22 is, for example, an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

[0032] The second communication device 23 is composed of an antenna, an IC (integrated circuit), an electric circuit, etc. for wireless communication via a mobile phone communication network, the Internet, or a wireless LAN. The on-board control device 21 communicates wirelessly with the remote device 30 via the second communication device 23. The on-board control device 21 also has an RTC (real-time clock) and can measure the date and time.

[0033] Although the present embodiment illustrates an example in which the remote work machine 1 and the remote device 30 communicate via a mobile phone communication network or the like, the remote work machine 1 and the remote device 30 may alternatively communicate via a mobile phone communication network or an external device such as a server or a repeater. Alternatively, the remote work machine 1 and the remote device 30 may communicate directly using short-range wireless signals such as Bluetooth (registered trademark) Low Energy (BLE) signals or Ultra High Frequency (UHF) signals. In this case, it is sufficient that the second communication device 23 and the remote device 30 each have an interface for short-range wireless communication.

[0034] The position detection device 24 detects its own position (positioning information including latitude and longitude) using a satellite positioning system. That is, the position detection device 24 receives signals transmitted from positioning satellites (such as the position of the positioning satellite, the transmission time, and correction information) and detects its own position based on the signals. The position detection device 24 may detect its own position as a position corrected based on correction information signals from a base station (reference station) that can receive signals from the positioning satellites. The position detection device 24 may also calculate the position of the remote-controlled work machine 1 based on its detected position and pre-stored external shape information of the remote-controlled work machine 1. The position detection device 24 may also calculate the position of the work machine 2 based on its detected position, pre-stored external shape information of the work machine 2, and the attachment position of the work machine 2 relative to the machine body 3.

[0035] The first detection device 25 includes a gyro sensor or an acceleration sensor, and detects the roll angle, pitch angle, yaw angle, etc. of the remote-controlled work machine 1 (machine body 3). That is, the first detection device 25 detects the left / right and front / rear tilt attitude of the remote-controlled work machine 1 relative to the horizontal state, and the steering state of the remote-controlled work machine 1 (steering angle, steering direction, etc.).

[0036] The second detection device 26 senses (monitors) the surroundings of the remote-controlled work machine 1. For example, the second detection device 26 includes a laser sensor 26a, an ultrasonic sensor 26b, a camera 26c, and an object detection unit 26d. In this embodiment, the second detection device 26 includes a plurality of laser sensors 26a and ultrasonic sensors 26b. The plurality of laser sensors 26a and the plurality of ultrasonic sensors 26b are installed at predetermined locations, such as the front, rear, left and right sides, of the remote-controlled work machine 1, and detect surrounding conditions, such as the front, rear, left and right sides of the remote-controlled work machine 1, and objects in the surroundings. For example, the laser sensor 26a and the ultrasonic sensor 26b are installed at predetermined positions on the machine body 3 that can detect objects within a predetermined target detection distance from the remote-controlled work machine 1 and that are located lower than the machine body 3.

[0037] The laser sensor 26a and the ultrasonic sensor 26b are examples of object sensors. Note that at least one of the laser sensor 26a and the ultrasonic sensor 26b may be provided in plurality as an object sensor in the second detection device 26. Furthermore, the second detection device 26 may be provided with a plurality of other object sensors.

[0038] The laser sensor 26a is composed of an optical sensor such as a LiDAR (Light Detection and Ranging) sensor. The laser sensor 26a emits pulsed measurement light (laser light) millions of times per second from a light source such as a laser diode, and reflects the measurement light with a rotating mirror to scan the measurement light in the horizontal or vertical direction and project it onto a predetermined detection range (sensing range). The laser sensor 26a then receives the measurement light reflected by the target object with a light-receiving element.

[0039] The object detection unit 26d is composed of an electric circuit or an IC that detects the presence, position, and type of an object based on the light-receiving signal output from the light-receiving element of the laser sensor 26a. The object detection unit 26d detects the distance to the object based on the time between when the laser sensor 26a emits measurement light and when it receives the reflected light (TOF (Time of Flight) method). Objects detected by the object detection unit 26d include the work site where the remote-controlled work machine 1 travels and works, the field, crops in the field, the ground, the road surface, other objects, and people.

[0040] The ultrasonic sensor 26b is composed of an airborne ultrasonic sensor such as a sonar. The ultrasonic sensor 26b transmits measurement waves (ultrasonic waves) within a predetermined detection range using a transmitter, and receives the reflected waves from an object using a receiver. The object detection unit 26d detects the presence or absence, position, and type of object based on the signal output from the receiver of the ultrasonic sensor 26b. The object detection unit 26d also detects the distance to the object based on the time between when the ultrasonic sensor 26b transmits the measurement wave and when it receives the reflected wave (TOF method).

[0041] The camera 26c is composed of a CCD camera equipped with a CCD (Charge Coupled Device) image sensor, or a CMOS camera equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor. As shown in Fig. 2, the cameras 26c are installed at predetermined locations such as the front, rear, left and right sides of the remote-controlled work machine 1 and inside the cabin 9, and capture images of the surroundings such as the front, rear, left and right sides of the remote-controlled work machine 1 and output data of the captured images. The camera 26c is an example of an imaging device.

[0042] 2, the camera 26c1 installed inside the cabin 9 captures an image of the area in front of the remote-controlled work machine 1 from the driver's seat 10. More specifically, the camera 26c1 captures an image of the area in front of the remote-controlled work machine 1 (in the direction of travel, A1) from a field of view that is substantially the same as that of an operator seated in the driver's seat 10. In other words, the camera 26c1 can obtain an image of the area in the direction of travel of the remote-controlled work machine 1.

[0043] 2, the camera 26c2 installed at the rear of the body 3 of the remote-controlled work machine 1 captures images behind the remote-controlled work machine 1. More specifically, the camera 26c2 captures images in the direction opposite to the field of view of the operator seated in the driver's seat 10 (direction A2). In other words, the camera 26c2 can obtain captured images of the direction opposite to the traveling direction of the remote-controlled work machine 1.

[0044] The object detection unit 26d can also detect the presence or absence of an object, the position of the object, the type of the object, and the like, based on the captured image data output from the camera 26c.

[0045] The second detection device 26 senses the surrounding conditions of the remote-controlled work machine 1 and the work device 2 using the laser sensor 26a, the ultrasonic sensor 26b, the camera 26c, and the object detection unit 26d, and outputs sensing information indicating the results to the on-board control device 21. The sensing information includes at least the detection information of the object detection unit 26d and the image data captured by the camera 26c, but the sensing information may also include detection information from the laser sensor 26a and the ultrasonic sensor 26b.

[0046] The state detection device 27 detects the operating state of the remote-controlled work machine 1. Specifically, the state detection device 27 includes various sensors installed in each part of the remote-controlled work machine 1 (machine body 3) and a computing unit, and the computing unit detects (calculates) the operating state of the remote-controlled work machine 1 based on output signals from the various sensors. The state of the remote-controlled work machine 1 detected by the state detection device 27 includes the drive and stop states of each part of the remote-controlled work machine 1, the traveling direction, traveling speed, acceleration, etc. of the remote-controlled work machine 1.

[0047] The state detection device 27 generates detection information indicating the detected operating state of the remote-controlled work machine 1 and outputs the detection information to the on-board control device 21. For example, the detection information of the state detection device 27 includes operation information of the remote-controlled work machine 1 and the work device 2. This operation information includes, for example, at least one piece of information among the speed (or acceleration) of the remote-controlled work machine 1, the gear shift position of the transmission 5, the braking position of the brake device 13, and the operating position of the work device 2.

[0048] The third detection device 28 detects the state of the working device 2. The third detection device 28 includes various sensors installed in each part of the coupling device 8 and the working device 2, and a computing unit, and the computing unit detects (calculates) the operating state of the working device 2 based on output signals from the various sensors. The states of the working device 2 detected by the third detection device 28 include the drive and stop states of each part of the working device 2, the inclination of the working device 2, its height above ground, etc.

[0049] If the type of remote-controlled working machine 1 is a rice transplanter, the third detection device 28 may detect the drive and stop states of each mechanism related to seedling planting, the remaining amount of materials (seedlings, fertilizer), etc., based on output signals from various sensors arranged in each part of the seedling planting device, etc. If the type of remote-controlled working machine 1 is a combine harvester, the third detection device 28 may detect the drive and stop states of each mechanism related to harvesting, the characteristics of the harvested grain, etc., based on output signals from various sensors arranged in each part of the reaping device, grain tank, etc.

[0050] The position detection device 24, the first detection device 25, the state detection device 27, and the third detection device 28 output detection information indicating the results of detection at a predetermined cycle or at a predetermined timing to the on-board control device 21 as needed. The second detection device 26 also outputs sensing information indicating the results of sensing at a predetermined cycle or at a predetermined timing to the on-board control device 21 as needed. The on-board control device 21 stores this detection information and sensing information in its internal memory 21b.

[0051] In addition, the on-board control device 21 sequentially transmits the detection information and sensing information stored in the internal memory 21b to the remote device 30 via the second communication device 23 at a predetermined period or at a predetermined timing in the case of remote driving.

[0052] The detection information and sensing information transmitted from the remote-controlled work machine 1 in this manner includes correspondence data (see FIG. 3 ) that associates the position information of the remote-controlled work machine 1 with captured images of the direction of travel of the remote-controlled work machine 1. FIG. 3 is a diagram showing an example of the correspondence data. That is, correspondence data that associates the detection information of the position detection device 24 (i.e., the position information of the remote-controlled work machine 1) with the sensing information of the second detection device 26 (e.g., images captured by the camera 26c1) is sequentially transmitted to the remote device 30. Regarding the captured images, as shown in FIG. 3 , correspondence data that associates the position information of the remote-controlled work machine 1 detected by the position detection device 24 (e.g., positions PU1, PV1, PW1, PX1) with the images captured by the camera 26c1 (e.g., captured images GPU1, GPV1, GPW1, GPX1) is sequentially transmitted to the remote device 30.

[0053] The on-vehicle control device 21 also communicates with a control unit 2a provided in the work device 2, and causes the control unit 2a to control the operation of the work device 2. That is, the on-vehicle control device 21 performs work on the field by controlling the operation of the work device 2 via the control unit 2a. For example, the control unit 2a is made up of a CPU, memory, etc. Note that some work devices 2 do not have a control unit 2a. In this case, the on-vehicle control device 21 controls the attitude of the work device 2 using the coupling device 8, and the work device 2 performs work on the field.

[0054] The on-board control device 21 controls the travel of the remotely operated work machine 1, the work performed by the work device 2, and other operations of the remotely operated work machine 1 based on detection information obtained from the first detection device 25, the second detection device 26, the state detection device 27, the third detection device 28, etc. Furthermore, when the on-board control device 21 receives a remote control signal transmitted from the remote device 30 via the second communication device 23, it controls the travel of the remotely operated work machine 1, the work performed by the work device 2, and other operations of the remotely operated work machine 1 based on the remote control signal in addition to the above information.

[0055] Furthermore, when controlling the travel of the remote-controlled work machine 1 or the work performed by the work device 2, the on-board control device 21 determines from the detection information of the object detection unit 26d whether or not an object is approaching the remote-controlled work machine 1 or the work device 2 within a predetermined distance and is likely to come into contact with the remote-controlled work machine 1 or the work device 2. If the on-board control device 21 determines that an object is approaching the remote-controlled work machine 1 or the work device 2 within a predetermined distance and is likely to come into contact with the remote-controlled work machine 1 or the work device 2, the on-board control device 21 controls the travel device 7 or the work device 2, etc., to automatically stop the travel or work of the remote-controlled work machine 1 and avoid contact with the object.

[0056] Next, the remote device 30 will be described. As shown in Fig. 1 , the remote device 30 is located at a distance from the remote-controlled work machine 1. The remote device 30 allows a remote operator to remotely control the remote-controlled work machine 1 and monitor the state of the remote-controlled work machine 1 and the surrounding conditions of the remote-controlled work machine 1. The remote device 30 includes a control device 31, a storage device 32, a first communication device 33, a display device 34, a remote control device 35, and an input device 36.

[0057] The control device 31 is a processor that controls the operation of each part of the remote device 30. This processor functions as the control device 31, controlling the operation of each part of the remote device 30, for example, by executing a remote control program or application stored in the storage device 32. The control device 31 may be implemented by hardware such as an integrated circuit (IC chip), or by software using a computer. In the latter case, the computer includes a recording medium on which a program, which is software that implements each function of the control device 31, and various data related to the remote work machine 1 are recorded in a computer-readable manner, an arithmetic circuit such as a CPU (Central Processing Unit) that executes the instructions of the program, and a RAM (Random Access Memory) that expands the program and various data. The arithmetic circuit then reads and executes the program from the recording medium, thereby realizing the functions of the control device 31.

[0058] The internal memory 31 a provided in the control device 31 is a volatile or non-volatile memory. Various information and data used by the control device 31 to control the operation of each part of the remote device 30 are stored in the internal memory 31 a in a readable and writable manner.

[0059] The display control unit 31b provided in the control device 31 has a function of controlling the display screen G displayed by the display device 34. For example, the display control unit 31b causes the display device 34 to display a first application screen W1 ( FIG. 4 ) showing information related to remote driving.

[0060] The storage device 32 pre-stores applications such as a remote control program for remotely operating the remote work machine 1 and a remote monitoring program for remotely monitoring the remote work machine 1, as well as various data. The storage device 32 is, for example, an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

[0061] The first communication device 33 is composed of an antenna, an IC, an electric circuit, etc. for wireless communication via a mobile phone communication network, the Internet, or a wireless LAN. The first communication device 33 is capable of wireless communication with the remote-controlled work machine 1 under the control of the control device 31. The first communication device 33 receives various data transmitted from the second communication device 23 (detection information from the position detection device 24, the first detection device 25, the state detection device 27, and the third detection device 28, sensing information from the second detection device 26, etc.). For example, the first communication device 33 receives correspondence data that associates the position information of the remote-controlled work machine 1 with an image captured of the direction in which the remote-controlled work machine 1 is traveling.

[0062] The display device 34 is, for example, a liquid crystal display, an organic EL display, etc. The display device 34 displays various information for remotely operating the remote-operated work machine 1. For example, the display device 34 displays a first application screen W1 (see FIG. 4 described later) showing information related to remote operation based on an instruction signal from the display control unit 31b. Images captured by the multiple cameras 26c are displayed on the first application screen W1.

[0063] The input device 36 is an interface for operating the remote device 30. The remote operator can input predetermined instructions to the remote device 30 by operating the input device 36. The input device 36 includes, for example, a pointing device 36a such as a mouse. The input device 36 has sensors that detect the amount of operation, the speed of operation, the direction of operation, etc., and a driver that calculates (defines) instruction signals for the remote device 30 based on the sensor detection results. The driver is a program stored in the memory device 32. When the remote operator operates the input device 36, the sensor of the input device 36 detects the amount of operation, etc., and inputs the detected information to the driver of the input device 36. The driver of the input device 36 defines the instruction signal based on the sensor detection results. Once the input device 36 defines the instruction signal, it inputs the instruction signal to the control device 31. The control device 31 controls the remote device 30 based on the instruction signal input by the input device 36.

[0064] The input device 36 may be a touch panel provided on the display device 34 that can detect touch operations on the display device 34, or may be a hardware switch or the like.

[0065] The remote control device 35 is a device for remotely controlling the remote-controlled work machine 1. The remote control device 35 includes a steering wheel 35a, an accelerator pedal 35b, a brake pedal 35c, and a gear shift lever 35d, which are arranged around the remote control seat. The remote control device 35 accepts operation by a remote operator. The remote operator sits in the remote control seat and operates the remote control device 35 to remotely control the travel of the remote-controlled work machine 1 or the work performed by the work device 2. The remote operator also monitors the remote-controlled work machine 1 and the surrounding conditions using the display device 34. The remote operator can also input predetermined information or instructions to the remote device 30 by operating the remote control device 35. The remote control device 35 may be a touchpad or a hardware switch, etc.

[0066] When a remote operator operates the remote control device 35 to input an operation instruction for the remote-controlled work machine 1, the control device 31 generates a remote control signal corresponding to the operation instruction and causes the first communication device 33 to transmit the remote control signal to the remote-controlled work machine 1. In other words, a remote control signal corresponding to the operation of the steering wheel 35a, accelerator pedal 35b, brake pedal 35c, and gear shift lever 35d is transmitted to the remote-controlled work machine 1. When the on-board control device 21 of the remote-controlled work machine 1 receives a remote control signal from the remote device 30 via the second communication device 23, it operates each part of the remote-controlled work machine 1 based on the remote control signal, detection information from the position detection device 24, sensing information from the second detection device 26, and detection information from the state detection device 27, to control the traveling and steering of the remote-controlled work machine 1 and to control the work operation of the work device 2.

[0067] Furthermore, the on-board control device 21 transmits the detection information of the position detection device 24, the first detection device 25, the state detection device 27, and the third detection device 28, and the sensing information of the second detection device 26 to the remote device 30 via the second communication device 23. When the control device 31 of the remote device 30 receives the detection information and the sensing information via the first communication device 33, the control device 31 stores the information in the internal memory 31 a and displays it on the display device 34.

[0068] 1, the remote device 30 may be composed of a display terminal 70 and a remote control device 35. That is, the display terminal 70 may be a terminal device including a control device 31, a storage device 32, a first communication device 33, a display device 34, and an input device 36. The display terminal 70 may be, for example, a portable terminal device such as a tablet device or a smartphone, or a stationary computer installed in a base station.

[0069] The remote device 30 can perform other processes in addition to processes related to remote control of the remote-controlled work machine 1. As shown in FIG. 1, the storage device 32 stores a first application 32a that performs processes related to remote control in response to the operation of the remote control device 35, and a second application 32b that performs processes other than remote control. For convenience, applications are denoted as "APP" in FIG. 1. This also applies to FIGS. 11 to 14, which will be described later. Note that while FIG. 1 shows one second application 32b, the number of second applications 32b may be multiple and is not limited thereto.

[0070] When the remote operator performs a selection operation on the first application 32 a, the control device 31 can execute the first application 32 a. When the remote operator performs a selection operation on the second application 32 b, the control device 31 can execute the second application 32 b.

[0071] The control device 31 can display a first application screen W1 ( FIG. 4 ) showing information about the first application 32 a and a second application screen W2 ( FIG. 5A ) showing information about the second application 32 b on the display device 34. The display device 34 displays the first application screen W1 and the second application screen W2 based on an instruction signal from the display control unit 31 b.

[0072] 4 to 10 are diagrams illustrating examples of the display screen G of the display device 34. For example, as shown in FIG. 4, the display control unit 31b causes the display device 34 to display a first application screen W1. The display control unit 31b causes a first image display unit 70a and a second image display unit 70b to display images captured by the multiple cameras 26c provided on the remote-controlled work machine 1 as the first application screen W1. The first image display unit 70a displays a forward-captured image of the area in front of the remote-controlled work machine 1 captured by the camera 26c1 from the driver's eye level (the eye level of the worker seated in the driver's seat 10). The second image display unit 70b displays a rear-captured image of the area behind the remote-controlled work machine 1 captured by the camera 26c2.

[0073] The display control unit 31b displays, as the first application screen W1, a first status display section 80a and a second status display section 80b that indicate the status of the remote-controlled work machine 1 detected by the first detection device 25 and the on-board control device 21 of the remote-controlled work machine 1. The first status display section 80a displays whether the remote-controlled work machine 1 is traveling by remote control (mode display section 80a1 in FIG. 4 ), the traveling speed of the remote-controlled work machine 1 (machine body 3), and the rotation speed of the prime mover 4. The second status display section 80b displays the traveling direction (forward or reverse) of the traveling device 7, the auxiliary transmission state (e.g., gear position) of the transmission 5, the main transmission state (e.g., gear position), the traveling mode of the remote-controlled work machine 1 (e.g., 2WD, etc.), and the operation amount of the accelerator pedal 35b. Note that the first application screen W1 shown in FIG. 4 is an example, and the captured image, the type of information to be displayed, and the arrangement of the captured image and information are not limited.

[0074] As shown in display screen G1b in FIG. 5A , display control unit 31b causes display device 34 to display second application screen W2 (W2a), which displays predetermined information related to second application 32b (32b1). Second application 32b may have a function separate from the remote control processing, such as a program that notifies weather information on second application screen W2b, as shown in FIG. 5B . Second application 32b2 may be a program for configuring various settings for remote device 30, or a program (email software, chat software) for communication between the remote operator and other workers, and is not limited to the example shown in FIG. 5B .

[0075] As shown in FIGS. 4, 5A, 6 to 10, and 15, the display screen G displays a first object 91 that accepts a display instruction for the first application 32a, a second object 92 that accepts a display instruction for the second application 32b, and a third object 93 that indicates a designated position corresponding to an operation of the input device 36 (pointing device 36a). The display control unit 31b can change (move) the display position of the third object 93 in response to an operation of the input device 36 by the remote operator. When the remote operator moves the third object 93 to the position of the first object 91 or the second object 92 and performs an operation to select these objects, the input device 36 inputs a display instruction signal for the first application 32a or the second application 32b corresponding to the operation to the control device 31. The display control unit 31b displays the first application screen W1 and / or the second application screen W2 on the display device 34 based on the display instruction signal from the input device 36. Screen transitions on the display screen G of the display device 34 will be described in detail below.

[0076] The control device 31 executes remote control using the first application 32a in response to the remote operator's operation of the remote control device 35. The control device 31 executes remote control using the first application 32a, and causes the display control unit 31b to display the first application screen W1 as the display screen G on the display device 34.

[0077] When the control device 31 is executing remote control using the first application 32a, the display device 34 can display the first application screen W1 with priority over the second application screen W2a. Specifically, when executing remote control, the control device 31 turns on a priority processing flag in the internal memory 31a. When stopping remote control, the control device 31 turns off the priority processing flag in the internal memory 31a. When receiving a display instruction signal for the second application screen W2a, the display control unit 31b refers to the priority processing flag in the internal memory 31a, and if the priority processing flag is on, causes the display device 34 to display the first application screen W1 with priority. The display device 34 displays the first application screen W1 with priority.

[0078] Specifically, when the display control unit 31b receives an instruction to display the second application screen W2a while remote control is being performed (priority processing flag is ON), it causes the display device 34 to display the first application screen W1 instead of the second application screen W2a as priority processing. For example, the display control unit 31b causes the display device 34 to display the second application screen W2a as display screen G2 ( FIG. 6 ), and then returns the display to display screen G1a ( FIG. 4 ). Note that the display control unit 31b may cause the display device 34 to display the second application screen W2a as display screen G2 ( FIG. 6 ), and then display the first application screen W1 in front of the second application screen W2a as display screen G3 ( FIG. 7 ).

[0079] When the display control unit 31b executes the priority processing of the first application screen W1, the display control unit 31b turns on a standby flag in the internal memory 31a. The standby flag indicates that an instruction to display the second application screen W2a has been input and that the priority processing has been executed.

[0080] If the remote operator performs an operation to stop remote control while the above-described priority processing is being performed, the control device 31 stops remote control by the first application 32a and sets the priority processing flag to OFF.

[0081] The display control unit 31b periodically references the priority processing flag and the standby flag stored in the internal memory 31a. When the priority processing flag is OFF and the standby flag is ON, the display control unit 31b causes the display device 34 to display the second application screen W2a instead of the first application screen W1 as the display screen G4 (FIG. 8).

[0082] 11 is a flowchart showing the operation of the display control unit 31b. The display control unit 31b executes steps S8 to S11 and S17 shown in FIG. 11 as one subroutine at a predetermined interval.

[0083] The display control unit 31b refers to the priority processing flag in the internal memory 31a (S9), and if the priority processing flag is ON (S9: YES), performs priority processing of the first application screen W1 (S10) and sets the standby flag ON (S11).

[0084] If the priority processing flag is OFF (S9: NO), the display control unit 31b refers to the standby flag in the internal memory 31a (S15), and if the standby flag is ON (S15: YES), it displays the second application screen W2a on the display device 34 (S8) and turns the standby flag OFF (S16).

[0085] 12 is a sequence diagram showing the operations of the input device 36, the display control unit 31b, and the control device 31. When the remote operator performs a selection operation for the first application 32a, the input device 36 inputs a display instruction signal and a start instruction signal for the first application 32a to the control device 31 (S1a). The control device 31 turns on the priority processing flag (S2) and inputs a display instruction signal for the first application 32a to the display control unit 31b (S3). The display control unit 31b displays the first application screen W1 as the display screen G1a (FIG. 4) on the display device 34 (S4). The control device 31 executes remote control using the first application 32a (S5).

[0086] When the remote operator performs a selection operation for the second application 32b, the input device 36 inputs a display instruction signal for the second application 32b to the control device 31 (S6). The control device 31 inputs a display instruction signal for the second application 32b to the display control unit 31b (S7). The display control unit 31b displays the second application screen W2a as the display screen G2 (FIG. 6) on the display device 34 (S8).

[0087] The display control unit 31b references the priority processing flag and detects that the priority processing flag is ON (S9: YES), and performs priority processing of the first application screen W1 (S10). The display control unit 31b turns on the standby flag (S11).

[0088] When the remote operator performs an operation to stop remote operation, the input device 36 inputs an instruction signal to stop remote operation of the first application 32a to the control device 31 (S12). The control device 31 turns off the priority processing flag (S13) and stops remote operation by the first application 32a (S14). The display control unit 31b references the priority flag and detects that the priority processing flag is off (S9: NO), and references the standby flag and detects that the standby flag is on (S15: NO). The display control unit 31b displays the second application screen W2a on the display device 34 as the display screen G2 (S8) and turns off the standby flag (S16).

[0089] FIG. 12 illustrates a case where the remote operator performs a selection operation on the second application 32b, i.e., where a display instruction signal for the second application screen W2 is input to the control device 31 by the input device 36. However, the display instruction signal for the second application 32b may be input to the control device 31 by the second application 32b. Specifically, for example, the second application 32b2 shown in FIG. 5B has a function for displaying the second application screen W2b as a pop-up screen and displays the second application screen W2b on the display device 34 in accordance with a predetermined condition. Examples of the predetermined condition include a case where a predetermined time has elapsed since the previous display or a predetermined notification signal has been input to the second application 32b, which is not based on an operation instruction from the remote operator. The display control unit 31b monitors whether an event for displaying the second application screen W2b has occurred. When an event for displaying the second application screen W2b has occurred, the display control unit 31b references the priority processing flag and determines whether to prioritize processing of the first application screen W1.

[0090] <First Modification> In the remote device 30 of the above embodiment, the first application 32a simultaneously receives an instruction from the remote operator to display the first application screen W1 and to start remote operation (FIG. 12: S1a). However, in the remote device 30 of the first modification, as shown in FIG. 13, the first application 32a is switchable between a first mode in which remote operation is performed and a second mode in which the remote work machine 1 is monitored without performing remote operation, which is different from the remote device 30 of the above embodiment.

[0091] The first mode is a function for remotely controlling the remote work machine 1 by the remote control device 35 and for sensing the surroundings of the remote work machine 1. The second mode is a function for not performing the remote control but for sensing the surroundings of the remote work machine 1. For example, when the first application 32a switches to the second mode, it stops remote control and does not accept operation instructions from the remote control device 35.

[0092] For example, when the remote operator selects and operates the mode display section 80a1 of the first application screen W1 ( FIG. 4 ), the input device 36 inputs an instruction signal for switching from one of the first mode and the second mode to the other to the control device 31. The control device 31 switches the first application 32a to the first mode or the second mode based on the instruction signal input by the input device 36. Furthermore, the display control section 31b switches the display of the mode display section 80a1 to match the mode in which the first application 32a is operating.

[0093] When the first application 32a is in the first mode, the display control unit 31b performs priority processing upon receiving an instruction to display the second application screen W2a, and when the first application 32a is in the second mode, the display control unit 31b displays the second application screen W2a without performing priority processing upon receiving an instruction to display the second application screen W2a. Specifically, when the remote operator performs an operation to select the first mode, the control device 31 turns on a priority processing flag in the internal memory 31a. When the remote operator performs an operation to select the second mode, the control device 31 turns off the priority processing flag in the internal memory 31a. When the display control unit 31b receives an instruction to display the second application screen W2a, it refers to the priority processing flag and switches whether or not to perform priority processing of the first application screen W1.

[0094] 13 is a sequence diagram showing the operations of the input device 36, the display control unit 31b, and the control device 31 in the first modified example. Note that the sequence triggered by step S6 is the same as that in FIG. 12, and therefore description thereof will be omitted.

[0095] When the remote operator performs a selection operation for the first application 32a, the input device 36 inputs a display instruction signal for the first application 32a to the control device 31 (S1b). The control device 31 inputs a display instruction signal for the first application 32a to the display control unit 31b (S3). The display control unit 31b displays the first application screen W1 on the display device 34 (S4).

[0096] When the remote operator selects the first mode, the input device 36 inputs a first mode instruction signal to the control device 31 (S20). The control device 31 turns on the priority processing flag (S2) and executes remote control (S5).

[0097] When the remote operator selects the second mode, the input device 36 inputs a second mode instruction signal to the control device 31 (S21). The control device 31 turns off the priority processing flag (S12) and stops remote control (S23). The display control unit 31b references the priority flag and detects that the priority processing flag is off (S9: NO), and references the standby flag and detects that the standby flag is on (S15: NO). The display control unit 31b causes the display device 34 to display the second application screen W2a as the display screen G4 (S8).

[0098] <Second Modification> In the remote device 30 of the above embodiment, when remote control is being performed, the first application screen W1 is displayed preferentially (FIGS. 11 and 12: S10). However, in the remote device 30 of the second modification, as shown in FIG. 14, when the display device 34 displays the second application screen W2a, remote control by the first application 32a is stopped, which is different from the remote device 30 of the above embodiment and the first modification.

[0099] In detail, when a display instruction signal for the first application screen W1 or the second application screen W2a is input from the control device 31, the display control unit 31b causes the display device 34 to display the first application screen W1 or the second application screen W2a as the display screen G, and notifies the control device 31 whether or not the control based on the instruction signal was successful. The control device 31 determines whether or not to stop remote control based on the notification signal from the display control unit 31b.

[0100] 14 is a flowchart showing the operation of the control device 31 in the second modified example. Steps S30 to S32 shown in FIG.

[0101] The control device 31 receives a notification signal from the display control unit 31b notifying the display device 34 (display screen G) of successful display (S30). The notification signal includes information indicating whether the application screen displayed in the process is the first application screen W1 or the second application screen W2. The control device 31 determines whether the notification signal input by the display control unit 31b is a notification signal indicating successful display of the first application screen W1 (S31). If the control device 31 determines that the notification signal is a notification signal indicating successful display of the first application screen W1 (YES in S31), the control device 31 returns to step S30, and the control device 31 waits for reception of the notification signal. If the control device 31 determines that the notification signal is not a notification signal indicating successful display of the first application screen W1, i.e., a notification signal indicating successful display of the second application screen W2a (NO in S31), the control device 31 stops remote control by the first application 32a (S32).

[0102] In step S32, the display control unit 31b may, as one of the processes related to the remote control by the first application 32a, display information indicating that the remote control has been stopped in an emergency on the display device 34. For example, as shown in Fig. 9, the display control unit 31b displays a notification unit 81 indicating that the remote-controlled work machine 1 has been stopped in an emergency ("emergency stop") on the first application screen W1 as the display screen G6.

[0103] <Third Modification> In the remote device 30 of the above embodiment, the second application screen W2a is displayed superimposed on the first application screen W1, and then priority processing of the first application screen W1 is performed ( FIG. 6 : G2, FIG. 7 : G3). The remote device 30 of the third modification differs from the remote devices 30 of the above embodiment, the first modification, and the second modification in that, when an instruction to display the second application screen W2 is received, priority processing is performed to display the second application screen W2 in an area that does not overlap with the first application screen W1, as shown in FIG.

[0104] Specifically, when the remote operator performs a selection operation for the first application 32a, the display control unit 31b displays the first application screen W1 on the display device 34 and stores coordinate information indicating the display position of the first application screen W1 in the internal memory 31a. When the remote operator performs a selection operation for the second application 32b1, the display control unit 31b refers to the internal memory 31a and displays the second application screen W2a in an area that does not overlap with the first application screen W1 (FIG. 10: display screen G5).

[0105] 15 , when the display control unit 31b determines that there is an area S on the first application screen W1 that can be superimposed and that the second application screen W2 can be displayed, the display control unit 31b may, as a priority process, superimpose and display the second application screen W2 on the area S of the first application screen W1. The area S that can be superimposed is, for example, an area in which information related to remote control is not displayed.

[0106] When the display control unit 31b causes the display device 34 to display the first application screen W1, it stores information (coordinate information, area) related to the area S that can be superimposed in the internal memory 31a. When the display control unit 31b receives an instruction to display the second application screen W2b, it refers to the internal memory 31a and determines whether the display area of ​​the second application screen W2b for which the display instruction was received is smaller than the area S. When the display control unit 31b determines that the display area of ​​the second application screen W2b is smaller than the area S, it displays the second application screen W2b at a position superimposed on the area S ( FIG. 15 : G7).

[0107] <Fourth Modification> In the remote device 30 of the above embodiment, even when remote control is being performed, the remote device 30 accepts operation instruction signals for the second application 32b via the input device 36. The remote device 30 of the fourth modification differs from the remote device 30 of the above embodiment and the first to third modifications in that, when remote control is being performed (when the priority processing flag is ON), the remote device 30 limits the operation of the input device 36 by the remote operator and does not accept display operations for the second application 32b.

[0108] As described above, the display control unit 31b causes the display device 34 to display a first object 91 that accepts a display instruction from the first application 32a, a second object 92 that accepts a display instruction from the second application 32b, and a third object 93 that indicates an indication position corresponding to the operation of the input device 36 (pointing device 36a).

[0109] When the control device 31 is executing remote control using the first application 32 a, the input device 36 accepts operations in an area excluding the second object 92.

[0110] The internal memory 31a stores a list of objects (areas) that do not accept operation instructions from the remote operator when the priority processing flag is ON. For example, the list stores the area where the second object 92 is located as an area that does not accept operation instructions. When the remote operator operates the input device 36, the input device 36 calculates a pointed position on the display screen G from the detection results of the sensor of the input device 36. After calculating the pointed position, the input device 36 refers to the list and determines whether an instruction signal corresponding to the pointed position is input to the control device 31.

[0111] For example, as shown in FIG. 4 , on the display screen G1a where remote control is being performed by the first application 32a, the remote operator operates the input device 36 to select the mode display section 80a1. As described above, the mode display section 80a1 is associated with a process for switching between the first mode and the second mode. The input device 36 determines whether to accept an operation instruction based on the detected pointing position and the list. If the input device 36 determines that the operation instruction is accepted, it inputs an instruction signal to switch between the first mode and the second mode to the control device 31.

[0112] On the display screen G1a where remote control is being performed by the first application 32a, the remote operator operates the input device 36 to select the second object 92. The input device 36 determines whether or not to accept the operation instruction based on the detected pointed position and the list. The input device 36 determines not to accept the operation instruction and does not input an instruction signal to the control device 31.

[0113] The display control unit 31 b limits the display range of the third object 93 to an area excluding the second object 92 .

[0114] 16 is a diagram showing the display range of the third object 93 in the fourth modified example. A trajectory 95 connecting the start point Ps and the end point Pg is a designated position calculated based on the detection results of the sensor of the input device 36. The display control unit 31b updates the display position of the third object 93 displayed on the display device 34 along the trajectory 95 in the section from the start point Ps to the position Px. As a result, the third object 93 moves from the position Ps to the position Px. The display control unit 31b determines that the section from the position Px to the end point Pg is the area of ​​the second object 92, and does not update the display position of the third object 93. As a result, the display range of the third object 93 is limited to the area excluding the second object 92.

[0115] While the display device 34 is displaying the first application screen W1, the input device 36 does not accept operations of the first object 91 and the second object 92. Specifically, while the input device 36 is displaying the first application screen W1, the input device 36 disables the function of the driver that calculates (defines) an instruction signal corresponding to the operation. As a result, even if a sensor of the input device 36 detects an operation, the input device 36 does not input the corresponding instruction signal to the control device 31 (does not accept the operation).

[0116] The control device 31 may be in a state where it does not accept operations from the input device 36. Specifically, while the display device 34 is displaying the first application screen W1, the control device 31 ignores the display instruction signal for the first application 32a and the display instruction signal for the second application 32b sent from the input device 36.

[0117] A preferred embodiment of the present invention provides a remote device 30 and a remote control system 100 as described in the following sections.

[0118] (Item A1) A remote device 30 comprising: a first communication device 33 capable of communicating with a remote work machine 1; a remote control device 35 that accepts operations; a control device 31 that executes a first application 32a that performs processing related to remote control of the remote work machine 1 via the first communication device 33 in accordance with operation of the remote control device 35, and a second application 32b that performs processing separate from the remote control; and a display device 34 that displays a first application screen W1 that shows information related to the first application 32a and a second application screen W2 that shows information related to the second application 32b, wherein the display device 34 displays the first application screen W1 in priority to the second application screen W2 when the control device 31 is performing the remote control using the first application 32a.

[0119] According to the remote device 30 relating to item A1, when a remote operator is performing remote control while checking information related to remote control, the remote control is prevented from being hindered by information other than the information related to remote control being displayed on the display device 34.

[0120] (Item A2) When the control device 31 is performing the remote control using the first application 32a, the display device 34 receives an instruction to display the second application screen W2, and as a priority process, displays the first application screen W1 instead of the second application screen W2. This is the remote device 30 described in Item A1.

[0121] According to the remote device 30 relating to item A2, when the display device 34 displays the first application screen W1, the second application screen W2 is not displayed, so the remote operator can continue remote operation without being hindered by the second application screen W2.

[0122] (Item A3) The remote device 30 described in item A2, wherein the first application 32a is switchable between a first mode in which the remote control is performed and a second mode in which the remote work machine 1 is monitored without performing the remote control, and when the first application 32a is in the first mode, the display device 34 performs the priority processing when it receives an instruction to display the second application screen W2, and when the first application 32a is in the second mode, the display device 34 displays the second application screen W2 without performing the priority processing when it receives an instruction to display the second application screen W2.

[0123] According to the remote device 30 according to item A3, when the remote operator is not remotely operating the device, the second application screen W2 can be displayed, thereby improving the versatility of the remote device 30.

[0124] (Item A4) The remote device 30 described in Item A2, wherein when the priority processing is being performed and the control device 31 stops the remote control using the first application 32a, the display device 34 displays the second application screen W2 instead of the first application screen W1.

[0125] According to the remote device 30 according to item A4, when remote control is stopped, the second application screen W2 can be displayed, thereby improving the versatility of the remote device 30.

[0126] (Item A5) When the control device 31 is performing the remote control using the first application 32a, the display device 34, upon receiving an instruction to display the second application screen W2, performs the priority processing of displaying the second application screen W2 and then returning the display to the first application screen W1.

[0127] According to the remote device 30 relating to item A5, the remote operator can continue to identify the first application 32a while being aware that an instruction to display the second application screen W2 has been issued during remote control, thereby enabling stable remote control.

[0128] (Item A6) When the control device 31 is performing the remote control using the first application 32a, when the display device 34 receives an instruction to display the second application screen W2, the display device 34 performs the priority processing of displaying the second application screen W2 and then displaying the first application screen W1 in front of the second application screen W2, as described in item A5 of the remote device 30.

[0129] According to the remote device 30 relating to item A6, the remote operator can continue to identify the first application 32a while being aware that an instruction to display the second application screen W2 has been issued during remote control, thereby enabling stable remote control.

[0130] (Item A7) When the control device 31 is performing the remote control using the first application 32a, the display device 34 performs the priority processing of displaying the second application screen W2 in an area that does not overlap with the first application screen W1 when it receives an instruction to display the second application screen W2.

[0131] According to the remote device 30 in item A7, the remote operator can properly view the first application 32a, which requires more attention than the second application 32b, thereby improving work efficiency.

[0132] (Item A8) When the control device 31 is performing the remote control using the first application 32a, the display device 34 displays the second application screen W2 upon receiving an instruction to display the second application screen W2, and the control device 31 stops the remote control of the first application 32a when the display device 34 displays the second application screen W2. A remote device 30 described in any one of items A1 to A7.

[0133] According to the remote device 30 in item A8, the remote operator can avoid performing remote control while looking at the second application screen W2.

[0134] (Item A9) A remote device 30 described in any one of items A1 to A8, which is provided with an input device 36 that accepts input, wherein the display device 34 displays a first object 91 that accepts display instructions for the first application 32a and a second object 92 that accepts display instructions for the second application screen W2, and wherein the input device 36 accepts operations in an area excluding the second object 92 when the control device 31 is performing the remote control using the first application 32a.

[0135] According to the remote device 30 in item A9, when the first application 32a is performing remote control, the display device 34 does not display the second application screen W2, which prevents the remote operator from performing remote control while looking at the second application screen W2.

[0136] (Item A10) The display device 34 displays a third object 93 indicating an indicated position corresponding to the operation of the input device 36, and when the control device 31 is performing the remote control using the first application 32a, the remote device 30 described in Item A9 limits the display range of the third object 93 to an area excluding the second object 92.

[0137] According to the remote device 30 in this item A10, when the first application 32a is performing remote control, the display device 34 can be easily prevented from displaying the second application screen W2.

[0138] (Item A11) A remote device 30 described in any one of items A1 to A8, comprising an input device 36 that accepts input, wherein the display device 34 displays a first object 91 that is operated by the input device 36 and accepts display instructions for the first application 32a, and a second object 92 that is operated by the input device 36 and accepts display instructions for the second application screen W2, and wherein the input device 36 does not accept operations on the first object 91 and the second object 92 while the display device 34 is displaying the first application screen W1.

[0139] According to the remote device 30 in item A11, when the first application 32a is performing remote control, the display device 34 does not display the second application screen W2, which prevents the remote operator from performing remote control while looking at the second application screen W2.

[0140] (Item A12) A remote control system 100 comprising a remote device 30 according to any one of items A1 to A11, and the remote work machine 1 that is remotely controlled by the remote device 30, wherein the remote work machine 1 has a second communication device 23 that communicates with the first communication device 33.

[0141] According to the remote control system 100 in this item A12, it is possible to realize the remote control system 100 that provides the advantageous effects described above.

[0142] Although the present invention has been described above, the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims.

[0143] 1: Remotely controlled work machine 23: Second communication device 30: Remote device 31: Control device 32a: First application 32b (32b1, 32b2): Second application 33: First communication device 34: Display device 35: Remote control device 36: Input device 91: First object 92: Second object 93: Third object 100: Remote control system W1: First application screen W2 (W2a, W2b): Second application screen

Claims

1. A first communication device capable of communicating with a remote working machine, a remote control device for receiving operations, a first application that performs processing related to remote control of the remote working machine via the first communication device according to the operations of the remote control device, a second application that performs processing different from the remote control, a control device that executes them, a first application screen that shows information about the first application, and a display device that displays a second application screen that shows information about the second application, wherein when the control device is executing the remote control by the first application, the display device preferentially displays the first application screen over the second application screen.

2. The remote device according to claim 1, wherein when the display device receives an instruction to display the second application screen while the control device is executing the remote control by the first application, as a priority process, the display device displays the first application screen instead of the second application screen.

3. The first application is switchable between a first mode of executing the remote control and a second mode of monitoring the remote working machine without executing the remote control. When the first application is in the first mode, when the display device receives an instruction to display the second application screen, the display device performs the priority process. When the first application is in the second mode, when the display device receives an instruction to display the second application screen, the display device displays the second application screen without performing the priority process. The remote device according to claim 2.

4. The remote device according to claim 2, wherein when the control device stops the remote control by the first application while the display device is performing the priority process, the display device displays the second application screen instead of the first application screen.

5. The remote device according to claim 2, wherein when the display device receives an instruction to display the second application screen while the control device is executing the remote control by the first application, the display device performs the priority process of displaying the second application screen and then returning the display to the first application screen.

6. The remote device according to claim 5, wherein when the control device is executing the remote control by the first application and the display device receives an instruction to display the second application screen, the display device displays the second application screen and then performs the priority process of displaying the first application screen in front of the second application screen.

7. The remote device according to claim 2, wherein when the control device is executing the remote control by the first application and the display device receives an instruction to display the second application screen, the display device performs the priority process of displaying the second application screen in an area that does not overlap with the first application screen.

8. The remote device according to claim 1, wherein when the control device is executing the remote control by the first application and the display device receives an instruction to display the second application screen, the display device displays the second application screen, and when the display device displays the second application screen, the control device stops the remote control of the first application.

9. The remote device according to claim 1, comprising an input device that receives an input, wherein the display device displays a first object that receives an instruction to display the first application and a second object that receives an instruction to display the second application screen, and when the control device is executing the remote control by the first application, the input device receives an operation in an area excluding the second object.

10. The remote device according to claim 9, wherein the display device displays a third object indicating an instruction position corresponding to an operation of the input device, and when the control device is executing the remote control by the first application, the display range of the third object is restricted to an area excluding the second object.

11. A remote device according to claim 1, comprising an input device that receives an input, wherein the display device displays a first object that is operated by the input device and receives a display instruction for the first application, and a second object that is operated by the input device and receives a display instruction for the second application screen, and the input device does not receive operations of the first object and the second object while the display device is displaying the first application screen.

12. A remote control system comprising the remote device according to any one of claims 1 to 11, and the remote working machine remotely controlled by the remote device, wherein the remote working machine has a second communication device that communicates with the first communication device.

Images