Operation device, and operation system

Abstract

To provide an operation device capable of properly acquiring an operation history of an operator.SOLUTION: An operation device 20 for operating an operation object includes an operation mechanism 210 that accepts operations by an operator, and a control device 200 that acquires an operation history which is a history of operating the operation object through the operation mechanism in association with an operator identifier for identifying the operator operating the operation mechanism.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an operation device and an operation system for operating an operation object. [Background technology]

[0002] A remote operation system configured to be capable of selectively remotely operating a plurality of work machines to be operated (to be able to switch the work machine to be operated from a first work machine to another work machine) has been disclosed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-198158 A Summary of the Invention [Problem to be solved by the invention]

[0004] In an operation system such as that exemplified in Patent Document 1, there are cases where the operation target (work machine) of the operation device (remote operation device) is switched. It is also assumed that the operator who operates the operation target via the operation device will be replaced by another operator. However, the operation system of Patent Document 1 does not take into consideration the case where the operator is replaced by another operator. For this reason, there is a problem in that it is difficult to appropriately acquire the operation history of the operator when the operation target is switched or the operator is replaced.

[0005] An object of the present invention is to provide an operation device and an operation system that can appropriately acquire an operation history of an operator. [Means for solving the problem]

[0006] An operating device according to a first aspect of the present invention is an operating device for operating an operating object, and includes an operating mechanism that accepts operations by an operator, and a control device that acquires, in association with each other, an operation history, which is a history of operating the operating object through the operating mechanism, and an operator identifier for identifying the operator operating the operating mechanism.

[0007] According to the operation device having the above configuration, the operation history of the operation target can be acquired separately for each operator identifier, and further for each operator identified by the operator identifier.

[0008] An operating device according to a second aspect of the present invention is the operating device according to the first aspect, further comprising a communication device capable of communicating with the operating target, and it is preferable that the control device communicates with a predetermined operating target selected from a plurality of operating targets via the communication device, thereby enabling the predetermined operating target to be operated by the operating mechanism.

[0009] According to the operation device having the above configuration, even if a specific operation target can be selected and operated from among a plurality of operation targets, the operation history of the operation target can be obtained separately for each operator identifier, and ultimately for each operator identified by the operator identifier.

[0010] An operating device according to a third aspect of the present invention is the operating device according to the first or second aspect, wherein the control device is capable of switching the operation object from one object to another object in response to a switching instruction for switching the operation object, and when the operation object is switched from the one object to the other object, it is preferable to obtain the operation history of the other object in association with the operator identifier.

[0011] According to the operation device having the above configuration, when an operation target is switched from one operation target to another operation target, the operation history of the other operation target is acquired together with the operator identifier, whereby the operation history of each different operation target by the operator can be acquired separately.

[0012] In an operating device according to a fourth aspect of the present invention, in the operating device according to the third aspect, it is preferable that the control device determines whether or not there is an instruction to switch the operating object when a condition is satisfied that no work is being performed using the operating object.

[0013] According to the operation device having the above configuration, the presence or absence of an instruction to switch the operation object is determined on the condition that no work using the operation object is being performed. When work using one operation object is being performed, it is possible to avoid a situation in which the one operation object is switched to another operation object and the work is stopped against the operator's intention.

[0014] In an operating device according to a fifth aspect of the present invention, in the operating device according to any one of the first to fourth aspects, when the operator identifier changes from one operator identifier corresponding to one operator to another operator identifier corresponding to another operator, it is preferable that the control device acquires the operation history of the operation target in association with the other operator identifier.

[0015] According to the operation device having the above configuration, when an operator is replaced by another operator, the operation history of the operation target is acquired together with an operator identifier for identifying the other operator. This makes it possible to distinguish and acquire the operation history of the operation target for each different operator.

[0016] It is preferable that the operating device according to a sixth aspect of the present invention, in the operating device according to any one of the first to fifth aspects, further comprises an input unit capable of receiving at least one of an operator change notification from one operator to another operator and an instruction to switch the operating object from one object to another object.

[0017] According to the operation device having this configuration, when a notification of a change from one operator to another operator is received through the input unit (and when the operator is actually changed), the operation history of the operation object is acquired together with the operator identifier of the other operator. This allows the operation history of the operation object to be distinguished and acquired for each different operator. In addition to or instead of this, when an instruction to switch from one operation object to another operation object is received through the input unit (and when the operation object is actually switched), the operation history of the other operation object is acquired together with the operator identifier. This allows the operation history of each different operation object by the operator to be distinguished and acquired.

[0018] An operating device according to a seventh aspect of the present invention is preferably the operating device according to the sixth aspect, wherein the input unit has an operator switching input interface capable of accepting a change of operator notification, and an operation object switching input interface capable of accepting an instruction to switch the operation object.

[0019] According to the operation device having the above configuration, an operator change notification and an instruction to switch the operation target are received through the input unit. Therefore, the operation history of the operation target for each different operator can be distinguished and acquired, and / or the operation history of each different operation target by the operator can be distinguished and acquired.

[0020] An operating device according to an eighth aspect of the present invention is the operating device according to the sixth aspect, further comprising a communication device capable of communicating with the operating target, and the control device is capable of executing a communication establishment process for establishing communication with the operating target through the communication device and a communication disconnection process for disconnecting the communication established with the operating target, and when a condition that the operator has not reported a change of operator through the input unit is satisfied, it is preferable that the control device determines whether or not there is an instruction to switch the operating target through the input unit while maintaining the communication established with the one operating target.

[0021] According to the operation device having this configuration, on the condition that there is no change notification of the operator through the input unit, the presence or absence of an instruction to switch the operation target through the input unit is determined while maintaining the communication established with one operation target. Therefore, when the operation target designated as a new operation target is the same as the previous operation target, the communication already established between the operation device and the operation target is continuously used, and it is not necessary to execute the communication disconnection process and the communication establishment process again for the operation target, and the operation of the operation target can be smoothly resumed.

[0022] An operating device according to a ninth aspect of the present invention is the operating device according to the sixth aspect, further comprising a communication device capable of communicating with the operating target, and the control device is capable of executing a communication establishment process for establishing communication with the operating target through the communication device, and a communication disconnection process for disconnecting the communication established with the operating target, and it is preferable that the control device determines whether or not there is an instruction to switch the operating target through the input unit when a condition that a change of operator has been reported through the input unit and a condition that the acquired operator identifier has changed from one operator identifier to another operator identifier are each satisfied.

[0023] According to the operation device having this configuration, when an operator is switched from one operator to another operator, the operation target is prevented from being switched by the one operator, and the operation target can be switched according to the intention of the other operator.

[0024] An operating device according to a tenth aspect of the present invention is the operating device according to the ninth aspect, wherein the control device executes the communication disconnection process to disconnect the communication established with the operation target when a condition is satisfied that a change of operator has been notified through the input unit.

[0025] According to the operation device having the above configuration, when an operator is replaced by another operator, the communication established with the operation target is cut off. This realizes a state in which the operation target designated by the other operator does not exist, and prevents the other operator from feeling uncomfortable due to the communication establishment state remaining between the operation device and the operation target designated by the one operator.

[0026] An eleventh aspect of the present invention is an operating device according to the tenth aspect, wherein the control device executes the communication establishment process for establishing communication with the other operation target when a condition is satisfied that an instruction to switch the operation target is given via the input unit after communication established with the one operation target is disconnected.

[0027] According to the operation device having the above configuration, a switching instruction is given through the input unit according to the operator's intention, and the operation target after the actual switching can be operated by the operator. This makes it possible to obtain both the operator identifier of the operator and the operation history of the operation target after the switching.

[0028] An operating device according to a twelfth aspect of the present invention is the operating device according to any one of the first to eleventh aspects, and when a condition is satisfied that an operator change notification has been made from one operator to another operator via an input unit while work is being performed using the operation object, the control device preferably acquires an operation history of the operation object in association with the operator identifier identifying the other operator.

[0029] According to the operation device having the above configuration, when an operator is replaced while a task using an operation object is being performed, the operation history of the operation object is acquired together with the operator identifier of the new operator.

[0030] An operating device according to a thirteenth aspect of the present invention is an operating device according to any one of the first to twelfth aspects, wherein the control device executes an authentication process of the operator based on the acquired operator identifier, and if authentication of the operator fails, preferably prohibits or restricts the operator from transmitting the operation mode of the operating mechanism to the operation target.

[0031] According to the operation device having the above configuration, when an operator is replaced by another operator, if authentication of the other operator fails, the operation of the operation target by the other operator through the operation mechanism is prohibited or restricted, thereby obtaining the operation history of both the other operator and the operation target that was prohibited or restricted.

[0032] An operating device according to a 14th aspect of the present invention is an operating device according to any one of the first to 13th aspects, wherein the control device executes an authentication process of the operator based on the acquired operator identifier, and if the authentication of the operator fails, prohibits switching of the operation target, and if the authentication of the operator is successful, allows switching of the operation target.

[0033] According to the operation device having the above configuration, when an operator is replaced by another operator, if authentication of the other operator fails, switching of the operation target by a switching instruction from the other operator is prohibited, whereas if authentication of the other operator is successful, switching of the operation target by a switching instruction from the other operator is permitted.

[0034] What is provided by the present invention is an operation system, the operation system including an operation device according to any one of the first to fourteenth aspects and the operation target. [Brief description of the drawings]

[0035] [Figure 1] FIG. 1 is a configuration diagram of an operation system according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a perspective view of a remote control device according to one embodiment of the present invention. [Diagram 3] FIG. 3 is a perspective view of a work machine according to one embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart of an operation system according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a flowchart of an authentication processing function of the operation system according to one embodiment of the present invention. [Figure 6] FIG. 6 is a flowchart of an operation processing function of an operation system according to an embodiment of the present invention. [Figure 7A] FIG. 7A is a diagram showing a first authentication screen for an operator in the operation system according to one embodiment of the present invention. [Figure 7B] FIG. 7B is a diagram showing a second authentication screen for an operator in the operation system according to one embodiment of the present invention. [Figure 8A] FIG. 8A is a diagram showing a work content designation screen in the operation system according to one embodiment of the present invention. [Figure 8B] FIG. 8B is a diagram showing an operation target designation screen in the operation system according to one embodiment of the present invention. [Figure 8C] FIG. 8C is a diagram showing a confirmation screen for the work content and the operation target in the operation system according to one embodiment of the present invention. [Figure 9A] FIG. 9A is a diagram showing a first lever pattern setting screen in the operation system according to one embodiment of the present invention. [Figure 9B] FIG. 9B is a diagram showing a second lever pattern setting screen in the operation system according to one embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing a lever sensitivity setting screen in the operation system according to one embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing a screen for setting the seat vibration mode in the operation system according to one embodiment of the present invention. [Figure 12]FIG. 12 is a diagram showing a setting screen for operation assistance in the operation system according to one embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing a guidance setting screen in the operation system according to one embodiment of the present invention. [Figure 14] FIG. 14 is a diagram showing a depth sense image in the operation system according to one embodiment of the present invention. [Figure 15] FIG. 15 is a diagram showing a terrain image in an operation system according to one embodiment of the present invention. [Figure 16A] FIG. 16A is a diagram showing a first button function setting screen in the operation system according to one embodiment of the present invention. [Figure 16B] FIG. 16B is a diagram showing a second button function setting screen in the operation system according to one embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an operation selection screen in an operation system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] (Operation System Configuration) FIG. 1 is a configuration diagram of an operation system according to an embodiment of the present invention. This operation system has an information processing terminal 10 and a remote control device 20 (operation device), and is configured to remotely operate a work machine 40 (operation target) using the remote control device 20. In another embodiment, the operation system may include the work machine 40. The information processing terminal 10 and the remote control device 20 are configured to be able to communicate with each other via a first communication network. The remote control device 20 and the work machine 40 are configured to be able to communicate with each other via a second communication network. The first communication network and the second communication network may have the same communication standard or different standards. The information processing terminal 10 may be omitted. The remote control device 20 may have the functions of the information processing terminal 10 built in. The "operation target" may be any device that has a function of performing a work task by remote control, such as a vehicle, a tiller, or an airplane, in addition to the work machine 40.

[0037] (Configuration of information processing terminal) As shown in FIG. 1, the information processing terminal 10 includes a terminal control device 100 (control device, control unit), a terminal input interface 110 (input interface, input unit), a terminal output interface 120 (output interface, output unit), and a terminal wireless communication device 150 (communication device) for communicating via a first communication network. The information processing terminal 10 is configured by a smartphone, tablet terminal, or notebook computer carried by an operator or mounted on a remote control device 20. The terminal control device 100 is configured by an arithmetic processing device (a single-core processor or a multi-core processor or a processor core constituting the same), reads necessary data and software from a storage device such as a memory, and executes arithmetic processing according to the software on the data. The terminal input interface 110 and the terminal output interface 120 are configured by, for example, a touch panel display.

[0038] As described above, the information processing terminal 10 may be a component of the remote control device 20. In this case, the information processing terminal 10 and the remote control device 20 may be configured to be capable of wired or wireless communication, the terminal input interface 110 may constitute a part of the remote input interface 210, and the terminal output interface 120 may constitute a part of the remote output interface 220.

[0039] (Configuration of remote control device) As shown in FIG. 1, the remote operation device 20 has a remote control device 200 (control device, control unit), a remote input interface 210 (input interface, input unit), a remote output interface 220 (output interface, output unit), and a remote wireless communication device 250 (communication device) capable of communicating with the information processing terminal 10 or the work machine 40 via a first communication network and a second communication network. The remote wireless communication device 250 is assigned an operation device identifier (IP address) for identifying the remote wireless communication device 250 in the second communication network and making it a transmission destination. The remote control device 200 is composed of an arithmetic processing device (a single-core processor or a multi-core processor or a processor core constituting the same), reads necessary data and software from a storage device such as a memory, and executes arithmetic processing according to the software on the data. The remote control device 200 communicates with a predetermined work machine 40 selected from a plurality of work machines 40 via the remote wireless communication device 250, thereby enabling the predetermined work machine 40 to be operated by the remote input interface 210.

[0040] The remote input interface 210 includes a remote operation mechanism 211 that accepts operations by an operator, and a face authentication sensor 212. The remote output interface 220 includes a remote image output device 221, a remote audio output device 222, and a seat drive device 224.

[0041] FIG. 2 is a perspective view of the remote control mechanism 211 according to this embodiment. The remote control mechanism 211 has the devices and members shown in FIG. 2, and includes a traveling operation device, a swivel operation device, a boom operation device, an arm operation device, and an end attachment operation device. Each operation device has an operation lever that receives a rotation operation. The operation lever (travel lever) of the traveling operation device is operated to drive a hydraulic traveling motor (actuator) that constitutes the traveling drive mechanism in the lower travel body 41 of the work machine 40. The travel lever may also serve as a traveling pedal. For example, a traveling pedal fixed to the base or lower end of the travel lever may be provided. The operation lever (swivel lever) of the swing operation device is operated to move a hydraulic swing motor (actuator) that constitutes the swing drive mechanism of the work machine 40. The operation lever (boom lever) of the boom operation device is operated to move the boom cylinder 442 of the work machine 40. The operating lever (arm lever) of the arm operating device is operated to move the arm cylinder 444 of the work machine 40. The operating lever (tip attachment lever) of the end attachment operating device is operated to move the end attachment cylinder 446 of the work machine 40. An optional pedal that is operated to move the end attachment of the work machine 40, etc. may be disposed on the side of the travel pedal.

[0042] The operation levers constituting the remote operation mechanism 211 are arranged around the seat 24 on which the operator sits, as shown in FIG. 2, for example. The seat 24 has a seat part 241 installed on the upper side of a seat base 240 and a backrest part 242, and is in the form of a high-back chair with armrests. The seat 24 may be in the form of a low-back chair without a headrest, or in the form of a chair without the backrest part 242, or in any other form on which the operator can sit. The seat 24 and / or the side of the seat 24 may be configured so that the information processing terminal 10 can be mounted. In this case, it is preferable that the touch screen of the information processing terminal 10 is mounted facing the seat 24 so that the display of the information processing terminal 10 can be easily operated and viewed by the operator seated on the seat 24. In addition, a cutoff lever (not shown) may be mounted on the seat 24 and / or the side of the seat 24. The cutoff lever has a function of prohibiting the operation of the work machine (stopping the actuator operation) when tilted from the unlocked position to the locked position.

[0043] Left and right travel levers 2110 corresponding to the left and right crawlers are arranged side by side in front of the seat 24. One operation lever may serve as multiple operation levers. For example, the left operation lever 2111 provided in front of the left frame of the seat 24 shown in FIG. 2 may function as an arm lever when operated in the front-rear direction, and as a rotation lever when operated in the left-right direction. Similarly, the right operation lever 2112 provided in front of the right frame of the seat St shown in FIG. 2 may function as a boom lever when operated in the front-rear direction, and as a tip attachment lever when operated in the left-right direction. Such a lever pattern may be arbitrarily changed by an operation instruction from the operator.

[0044] 2, the face authentication sensor 212 is configured with a camera and / or a distance measuring sensor arranged in front of the seat 24. An authentication process for the operator is executed based on an image of the operator's face captured by the camera and / or a three-dimensional shape of the operator's face captured by the distance measuring sensor.

[0045] The remote image output device 221 is composed of a single image output device or a plurality of image output devices. For example, as shown in FIG. 2, the remote image output device 221 is composed of a central remote image output device 2210, a left remote image output device 2211, and a right remote image output device 2212, each having a substantially rectangular screen, which are arranged in front of the seat 24, diagonally forward to the left, and diagonally forward to the right. The shapes and sizes of the screens (image display areas) of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 may be the same or different. The remote image output device 221 may be composed of a single curved or bendable image output device, or two or four or more image output devices arranged to surround the front of the seat 24.

[0046] 2, the right edge of the left remote image output device 2211 is adjacent to the left edge of the central remote image output device 2210 so that the screens of the central remote image output device 2210 and the left remote image output device 2211 form a first specified angle (e.g., 120° to 150°). As shown in FIG. 2, the left edge of the right remote image output device 2212 is adjacent to the right edge of the central remote image output device 2210 so that the screens of the central remote image output device 2210 and the right remote image output device 2212 form a second specified angle (e.g., 120° to 150°).

[0047] The screens of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 may be parallel to the vertical direction or may be inclined to the vertical direction. At least one of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 may be composed of a plurality of divided image output devices. For example, the central remote image output device 2210 may be composed of vertically adjacent image output devices each having a substantially rectangular screen.

[0048] The remote audio output device 222 is composed of one or more speakers, and is composed of audio output devices arranged, for example, behind the seat 24, at the rear of the left armrest, and at the rear of the right armrest. The specifications of the audio output devices may be the same or different. The remote audio output device 222 may be configured to output the sound of the work machine 40 collected by a microphone mounted on the work machine 40.

[0049] The seat drive device 224 constitutes a "seat device" together with the seat 24. The seat drive device 224 is configured to tilt or swing the seat 24 relative to the seat base 240 (or integrally with the seat base 240) about the pitch axis and / or the roll axis (about the roll axis as necessary). The seat drive device 224 may be configured to oscillate the seat 24 relative to the seat base 240 (or integrally with the seat base 240) in at least one direction of the pitch axis, the roll axis, and the yaw axis.

[0050] (Work machine configuration) 1, the work machine 40 has a real machine control device 400, a real machine input interface 410, a real machine output interface 420, and a real machine wireless communication device 450 for communicating via a second communication network. A real machine identifier (IP address) is assigned to the real machine wireless communication device 450 to identify the real machine wireless communication device 450 in the second communication network and set it as a transmission destination. Each of the components of the real machine control device 400 is configured by an arithmetic processing device (a single-core processor or a multi-core processor or a processor core constituting the same), reads necessary data and software from a storage device such as a memory, and executes arithmetic processing according to the software on the data.

[0051] Fig. 3 is a perspective view of a work machine 40 according to this embodiment. The work machine 40 is, for example, a hydraulic excavator, and as shown in Fig. 3, includes a crawler-type lower traveling body 41 constituting a traveling drive mechanism, and an upper rotating body 42 rotatably mounted on the lower traveling body 41 via a swing drive mechanism. A cab 43 (operator's cab) is provided on the front left side of the upper rotating body 42. A work mechanism 44 is provided on the front right side of the upper rotating body 42. The cab 43 may be configured to be driven to move up and down relative to the upper rotating body 42 by a lifting drive mechanism (not shown) having a link mechanism and an actuator, and may be configured to change the tilt angle relative to the upper rotating body 42 by a tilt drive mechanism (not shown) having an actuator.

[0052] The real machine input interface 410 has a real machine operation mechanism 411, a real machine image pickup device 412, and a real machine sensor group 416. The real machine operation mechanism 411 has a plurality of operation levers arranged around a seat arranged inside the cab 43 in the same manner as the remote operation mechanism 211. An actuation mechanism or a robot that receives a signal according to the operation mode of the remote operation lever and moves the real machine operation lever based on the received signal is provided in the cab 43. The real machine operation mechanism 411 may be configured by an actuation mechanism that moves a control valve that constitutes a hydraulic circuit mounted on the work machine 40. The real machine image pickup device 412 is installed, for example, inside the cab 43, and picks up an image of the environment in front of the cab 43 including at least a part of the working mechanism 44 through the front window and the left and right side windows. A part or all of the front window and the side windows may be omitted. The real machine image pickup device 412 may be mounted outside the cab 43 as long as it picks up an image of the environment in front of the cab 43 including at least a part of the working mechanism 44.

[0053] The actual machine sensor group 416 includes various sensors configured to output signals to the actual machine control device 400 according to parameter values ​​related to the state of the work machine 40. The state of the work machine 40 includes the actual machine attitude detected by the actual machine attitude sensor, that is, the tilt angle (pitch angle) about the pitch axis and / or the tilt angle (roll angle) about the roll axis with respect to the vertical direction of the upper rotating body 42. The actual machine attitude sensor is configured, for example, with a three-dimensional acceleration sensor. The state of the work machine 40 may include whether or not the prime mover (equipment that generates vibration, such as an engine or a work oil pump) is operating, as well as the hoisting angle of the boom 441 relative to the upper rotating body 42, the rotation angle of the arm 443 relative to the boom 441, the rotation angle of the tip attachment 445 (lift magnet) relative to the arm 443, the operating oil pressure, the remaining amount of fuel, and the like. The state of the work machine 40 may include the sound of the work machine 40 collected by a microphone mounted on the work machine 40.

[0054] 3, the working mechanism 44 (operating mechanism) has a boom 441 having a base end movably attached to the upper rotating body 42, an arm 443 having a base end rotatably connected to the tip of the boom 441, and a tip attachment 445 having a base end rotatably connected to the tip of the arm 443. The working mechanism 44 is equipped with a boom cylinder 442, an arm cylinder 444, and a tip attachment cylinder 446 as actuators each composed of an extendable hydraulic cylinder.

[0055] As the tip attachment 445, in addition to a bucket having a container for storing objects such as soil and sand, a lifting magnet, a grapple that grips an object by opening and closing multiple opposing gripping claws, a bucket with a hook that can hook an object to a hook attached to the back and lift it, and a breaker having a chisel that is supported on the breaker body so as to be capable of reciprocating and striking the work object may be used. The chisel of the breaker may be configured to be operable by a switch. In addition, a swing mechanism that swings the tip end left and right relative to each base end, and a slide arm that is provided with a slidable extension mechanism on the arm 443 and is configured to be extendable and retractable on the arm 443 may be provided at the base end of the boom 441 and the base end of the arm 443, and may be configured to be operable by a switch or an optional pedal. A dozer having a blade lifting drive mechanism that drives the blade to lift and lower by a dozer cylinder is provided at the front side of the lower traveling body 41, and may be configured to be operable by a switch or an optional pedal.

[0056] The boom cylinder 442 is interposed between the boom 441 and the upper rotating body 42 so as to extend and retract when supplied with hydraulic oil to rotate the boom 441 in the hoisting direction. The arm cylinder 444 is interposed between the arm 443 and the boom 441 so as to extend and retract when supplied with hydraulic oil to rotate the arm 443 about a horizontal axis relative to the boom 441. The tip attachment cylinder 446 is interposed between the tip attachment 445 and the arm 443 so as to extend and retract when supplied with hydraulic oil to rotate the tip attachment 445 about a horizontal axis relative to the arm 443.

[0057] The real machine control device 400 includes a switch that can switch between the presence and absence of power supply to the real machine control device 400. The switch switches from a state in which there is no power supply to the real machine control device 400 to a state in which there is power supply to the real machine control device 400, and the real machine control device 400 starts operating. When the real machine control device 400 starts operating, the real machine control device 400 starts the real machine wireless communication device 450, and the real machine wireless communication device 450 connects to the first communication network and the second communication network. In addition, the real machine control device 400 that has started operating starts the real machine imaging device 412 and the real machine sensor group 416. In a state in which there is no power supply to the real machine control device 400 due to the switch, the real machine control device 400 stops operating, the real machine wireless communication device 450 is stopped, and communication with the information processing terminal 10 or the work machine 40 via the first communication network and the second communication network is not possible. In addition, when the real machine control device 400 stops operating, the operations of the real machine imaging device 412 and the real machine sensor group 416 stop.

[0058] (function) Fig. 4 is a flowchart of an operation system according to an embodiment of the present invention. Fig. 5 is a flowchart of an authentication processing function of the operation system according to this embodiment. Fig. 6 is a flowchart of an operation processing function of the operation system according to this embodiment. Fig. 7A is a diagram showing a first authentication screen for an operator in the operation system according to this embodiment. Fig. 7B is a diagram showing a second authentication screen for an operator in the operation system according to this embodiment. The functions of the remote operation device 20 and the remote operation system having the above configuration will be described with reference to Figs. 4 to 7B.

[0059] When the processing of the operation system is started, first, the remote control device 200 (authentication unit) executes the authentication processing (FIG. 4 / STEP 200). Specifically, as shown in FIG. 7A, a message Q01 prompting the operator to sit in the seat 24 and an authentication button Q02 are output to the remote output interface 220 (remote image output device 221) (output, display by the remote output interface 220 (same hereinafter)). The various buttons output to the remote output interface 220 (remote image output device 221) constitute a touch panel type input interface.

[0060] When the authentication button Q02 is tapped, a face authentication button Q03 and an identifier authentication button Q04 are output to the remote output interface 220 as shown in FIG. 7B. When the face authentication button Q03 is tapped, for example, a camera and / or a distance measuring sensor mounted on the remote operation device 20 and / or the information processing terminal is used to perform face authentication of the operator (and further by querying a database). When the identifier authentication button Q04 is tapped, the operator authentication is performed using the operator identifier (or the operator identifier (user ID) and passcode) input through the remote input interface 210 (remote image output device 221). When the authentication is successful, an OK button Q05 is output to the remote output interface 220 (remote image output device 221) together with information indicating that the authentication is successful. When the authentication is unsuccessful, information indicating that the authentication is unsuccessful is output to the remote output interface 220 (remote image output device 221). If the authentication fails, the face authentication button Q03 and the identifier authentication button Q04 may be output again to the remote output interface 220 (remote image output device 221).

[0061] Details of the authentication process executed by the remote control device 200 will be described with reference to the flowchart of Fig. 5. The authentication process may be executed steadily or periodically as background processing. Specifically, first, an operator identifier (including captured image data and / or distance measurement data for face authentication) is acquired (Fig. 5 / STEP 202). If an operator identifier based on face authentication is not acquired for a certain period of time, an operation history indicating that the operator is away from his / her seat (not seated in the seat 24) is acquired together with the last acquired operator identifier.

[0062] It is determined whether or not an operator change notification has been accepted through the remote input interface 210 (or the change notification input interface (touch button) that constitutes the remote input interface) (FIG. 5 / STEP 240). If the determination result is positive (YES in FIG. 5 / STEP 240), a first flag f1 indicating the presence or absence of a change notification is set to "1" (FIG. 5 / STEP 241). On the other hand, if the determination result is negative (NO in FIG. 5 / STEP 240), the first flag f1 is set to "0" (FIG. 5 / STEP 242).

[0063] Next, it is determined whether the operator identifier has changed (whether the operator identifier over the most recent multiple times is different from the operator identifier from the past) (FIG. 5 / STEP 250). If the determination result is positive (FIG. 5 / YES in STEP 250), a second flag f2 indicating whether the operator identifier has changed is set to "1" (FIG. 5 / STEP 251). For example, if an operator leaves his / her seat and another operator sits in the seat 24, the newly acquired operator identifier is different from the last acquired operator identifier, so the determination result is positive. On the other hand, if the determination result is negative (FIG. 5 / NO in STEP 250), the second flag f2 is set to "0" (FIG. 5 / STEP 252).

[0064] Furthermore, it is determined whether the first flag f1 and the second flag f2 are equal (FIG. 5 / STEP 254). If the determination result is negative (FIG. 5 / NO in STEP 254), that is, if an operator change report has been made but the operator identifier has not changed, or if an operator change report has not been made but the operator identifier has changed, the authentication flag f indicating the success or failure of authentication is set to "0" (FIG. 5 / STEP 262).

[0065] On the other hand, if the determination result is positive (FIG. 5 / STEP 254: YES), that is, if an operator change report is made and the operator identifier has changed, or if an operator change report is not made and the operator identifier has not changed, the success or failure of authentication is determined (FIG. 5 / STEP 260). Specifically, the success or failure of authentication is determined depending on whether the latest operator identifier is registered in the storage device constituting the remote control device 200 and / or an external database. If the determination result is positive (FIG. 5 / STEP 260: YES), the authentication flag f is set to "1" (FIG. 5 / STEP 261). On the other hand, if the determination result is negative (FIG. 5 / STEP 260: NO), the authentication flag f is set to "0" (FIG. 5 / STEP 262). A series of authentication processes is terminated by setting the authentication flag f.

[0066] The order of the process of determining whether or not an operator change notification has been received and the associated process related to the first flag f1 (see FIG. 5 / STEP 240, STEP 241, and STEP 242) and the process of determining whether or not an operator identifier has changed and the associated process related to the second flag f2 (see FIG. 5 / STEP 250, STEP 251, and STEP 252) may be reversed, or may be parallel processes. The process of determining whether or not an operator change notification has been received through the remote input interface 210 and the associated process related to the first flag f1 may be omitted (see FIG. 5 / STEP 240, STEP 241, STEP 242, and STEP 254). The process of determining whether or not an operator identifier has changed and the associated process related to the second flag f2 may be omitted (see FIG. 5 / STEP 250, STEP 251, and STEP 252). The process of comparing the first flag f1 and the second flag f2 may be omitted (see FIG. 5 / STEP 254).

[0067] FIG. 8A is a diagram showing a work content designation screen in the operation system according to this embodiment. After the operator is authenticated by the authentication process in STEP 200 in FIG. 4, when the OK button Q05 is tapped, the operation target and work content (scheduled work content) are designated or selected (FIG. 4 / STEP 204). That is, each input interface accepts a designation command for designating the operation target. More specifically, when the OK button Q05 is tapped, as shown in FIG. 8A, work content selection windows Q11 and Q12 and a back button Q13 are output to the remote output interface 220 (remote image output device 221). In each of the multiple (for example, two) work content selection windows Q11 and Q12, for example, the work name, the name of the office involved in the work, the work location, the characteristics of the work target (materials, etc.), and the work period are displayed. When the back button Q13 is tapped, etc., the screen shown in FIG. 7A or FIG. 7B is output to the remote output interface 220 (remote image output device 221).

[0068] Fig. 8B is a diagram showing an operation target designation screen in the operation system according to this embodiment. When one of the above-mentioned work content selection windows Q11 and Q12 is tapped or the like, as shown in Fig. 8B, operation target candidate selection windows Q14 and Q15 and a back button Q16 are output to the remote output interface 220 (remote image output device 221). For example, the model number, name and specification information of the work machine 40, which is the operation target candidate, are displayed in each of the multiple (for example, two) operation target candidate selection windows Q14 and Q15. The model number, name and specification information of the operation target candidate are associated with an actual machine identifier for identifying the work machine 40 (operation target) and are registered in a storage device constituting the remote control device 200 and / or an external database. Of the work machines 40 to be operated, only operation target candidates that have been switched by the switch to a state in which power is being supplied to the actual machine control device 400 can be selected, and operation target candidates that have not been switched to a state in which power is being supplied are displayed in a grayed-out state, and windows Q14 and Q15 are output to the remote output interface 220 (remote image output device 221) so as not to accept selection even if they are tapped, etc. When the back button Q16 is tapped, etc., the screen shown in Fig. 8A is output to the remote output interface 220 (remote image output device 221).

[0069] Depending on the operator authenticated by the authentication process (see FIG. 4 / STEP 200), a plurality of operation target candidates may be searched for from the storage device constituting the remote control device 200 and / or an external database, and a plurality of operation target candidate selection windows showing the model numbers of the plurality of operation target candidates may be output to the remote output interface 220 (remote image output device 221). In other words, the remote control device 200 outputs at least one operation target candidate to the remote output interface 220 (remote image output device 221) depending on the operator authenticated (by the authentication unit). As a result, a new operation target may be designated by the operator through the remote input interface 210 according to his / her intention from among the operation target candidates output to the remote output interface 220 depending on the operator. Note that the operation target candidates to be used by each operator depending on the operator identified by the operator identifier are registered in advance in the storage device and / or the external database. Furthermore, the remote input interface 210 is capable of accepting the designation command for a new operation target selected from the at least one operation target candidate output to the remote output interface 220 (remote image output device 221).

[0070] Depending on the type of work content (scheduled work content) selected from the multiple work content selection windows, multiple operation target candidates (may be at least one operation target candidate) may be searched for from the storage device constituting the remote control device 200 and / or an external database, and multiple operation target candidate selection windows showing the model numbers and the like of each of the multiple operation target candidates may be output to the remote output interface 220 (remote image output device 221). As a result, a new operation target may be designated by the operator through the remote input interface 210 according to his / her intention from among the operation target candidates output to the remote output interface 220 (remote image output device 221) according to the type of the scheduled work content. Note that the storage device and / or the external database have previously registered the scheduled work content of each operator and the operation target candidates to be used when executing the scheduled work content according to the type of operator identified by the operator identifier.

[0071] Fig. 8C is a diagram showing a confirmation screen for work content and operation target in the operation system according to this embodiment. When one of the operation target selection windows Q14 and Q15 is tapped or the like, a selection result confirmation window Q17, a back button Q18, and an operation start button Q19 are output to the remote output interface 220 (remote image output device 221) as shown in Fig. 8C. For example, the selected work content and information on the work machine 40 that is the operation target are displayed in the selection result confirmation window Q17. When the back button Q18 is tapped or the like, the screen shown in Fig. 8B is output to the remote output interface 220 (remote image output device 221).

[0072] Next, an operation to start using the operation target is accepted (FIG. 4 / STEP 206). Specifically, the operation start button Q19 shown in FIG. 8C is tapped or the like.

[0073] In response to this, it is determined whether there is an existing operation target for the remote control device 200 (FIG. 4 / STEP 208). Specifically, it is determined whether there is a work machine 40 that is an operation target and in a state in which communication is established with the remote operation device 20. When communication with the operation target is established and / or when the established communication is released, the value of the flag may be changed, and the value of the flag may be used for this determination.

[0074] If it is determined that there is no existing operation target (FIG. 4 / STEP 208: NO), the remote control device 200 starts the execution of a communication establishment process as an "operation preparation process" (FIG. 4 / STEP 210). In this process, the remote control device 200 first acquires the actual machine identifier corresponding to the work machine 40 selected as the operation target by the designation command from the actual machine identifiers associated with each of a plurality of operation target candidates registered in the storage device and / or an external database.

[0075] Next, the remote control device 200 transmits a communication establishment request signal to a destination identified and indicated by the actual machine identifier corresponding to the actual wireless communication device 450 of the work machine 40 selected as the operation target, requesting the establishment of communication with the work machine 40 that is the operation target. The communication establishment request signal includes the operation device identifier corresponding to the remote wireless communication device 250 possessed by the remote operation device 20.

[0076] When the actual machine control device 400 mounted on the work machine 40 to be operated receives the communication establishment request signal through the actual machine wireless communication device 450, it transmits an approval signal indicating that the request for communication establishment has been approved to the remote wireless communication device 250 possessed by the remote operation device 20, which is the destination identified and indicated by the operation device identifier contained in the communication establishment request signal.

[0077] When the remote control device 200 receives the approval signal through the remote wireless communication device 250, it transmits an activation signal to activate the actual machine imaging device 412 and the actual machine sensor group 416 mounted on the work machine 40 to the destination identified and indicated by the actual machine identifier, and waits until it receives a signal corresponding to the actual machine operating status regarding the image acquired by the actual machine imaging device 412 through the remote wireless communication device 250 and the status of the work machine 40 acquired by the actual machine sensor group 416.

[0078] When the real machine control device 400 receives the activation signal through the real machine wireless communication device 450, it activates the real machine imaging device 412 and the real machine sensor group 416, starts transmitting signals corresponding to the real machine operation status relating to the captured images acquired by the real machine imaging device 412 and the state of the work machine 40 acquired by the real machine sensor group 416 to a destination identified and indicated by the operation device identifier, and starts receiving the signals corresponding to the operation mode of the remote control lever for controlling the operation of the actuation mechanism through the real machine wireless communication device 450. Note that the signals corresponding to the real machine operation status contain information related to the real machine identifier corresponding to the real machine wireless communication device 450.

[0079] The remote control device 200 outputs the captured image and the actual machine operation status received through the remote wireless communication device 250 to the remote output interface 220. For example, the remote control device 200 outputs the received captured image to a remote image output device 221, outputs the sound of the industrial machine 40 included in the received actual machine operation status to a remote sound output device 222, and controls the operation of the seat drive device 224 according to the tilt angle included in the received actual machine operation status.

[0080] As described above, the actual machine control device 400 executes a process for establishing the communication with the operation target through the communication device as an operation preparation process. By executing the communication establishment process, communication is established between the remote wireless communication device 250 of the work machine 40 selected as the operation target and the remote wireless communication device 250 for remotely operating the work machine 40 (operation target) by the remote operation device 20.

[0081] If there is a plan to perform communication establishment processing between the work machine 40 to be operated and the remote operation device 20, the remote control device 200 may transmit a communication establishment request signal before the time to execute the communication establishment processing. In this case, the communication establishment request signal includes the operation device identifier corresponding to the remote wireless communication device 250 possessed by the remote operation device 20 and information regarding the time to establish the communication establishment processing. The actual machine control device 400 mounted on the work machine 40 to be operated stores the operation device identifier contained in the communication establishment request signal received through the remote wireless communication device 250 in a storage device mounted on the work machine 40, reads out the operation device identifier from the storage device when it is time to establish the communication establishment processing, and transmits the approval signal indicating that the request for communication establishment has been approved to the remote wireless communication device 250 possessed by the remote operation device 20 that is the transmission destination identified and indicated by the read out operation device identifier.

[0082] Further, the operating device identifier and the actual machine identifier may be assigned something other than an IP address as long as they can identify the remote wireless communication device 250 and the actual machine wireless communication device 450 in the second communication network. For example, a wireless LAN access point or the like may be assigned to the remote wireless communication device 250 and the actual machine wireless communication device 450, such as an SSID (Service Set Identifier). The operating device identifier and the actual machine identifier may be configured to correspond to, for example, a plurality of letters, numbers, and / or symbols representing the serial numbers of components such as an engine and / or a battery mounted on the work machine 40 together with the actual machine wireless communication device 450, so that the remote wireless communication device 250 and the actual machine wireless communication device 450 can be identified in the second communication network. In communication, an authenticator is generated according to a designation method based on the operating device identifier and the actual machine identifier. An authenticator may be generated according to a different designation method depending on the difference in the data or letters, numbers, and symbols constituting the operating device identifier and the actual machine identifier. Then, the remote control device 200 uses the generated authenticator to establish wireless communication with the remote wireless communication device 250 and the actual wireless communication device 450 identified by the communication identifier.

[0083] On the other hand, if it is determined that there is an existing operation target for the remote control device 200 (YES in FIG. 4 / STEP 208), the remote control device 200 starts executing a communication disconnection process as an "operation preparation process" in addition to the communication establishment process (FIG. 4 / STEP 212).

[0084] In the communication establishment process in the operation preparation process, the remote control device 200 first acquires the actual machine identifier corresponding to the work machine 40 (other operation target) selected as the operation target by the designation command from the actual machine identifiers associated with each of the multiple operation target candidates registered in the storage device and / or external database.

[0085] Next, the remote control device 200 transmits a communication establishment request signal to a destination identified and indicated by the actual machine identifier corresponding to the actual machine wireless communication device 450 of the work machine 40 selected as the target to be operated, requesting the establishment of communication with the work machine 40 to be operated.

[0086] When the actual machine control device 400 mounted on the work machine 40 to be operated receives the communication establishment request signal through the actual machine wireless communication device 450, it transmits an approval signal indicating that the request for communication establishment has been approved to the remote wireless communication device 250 possessed by the remote operation device 20, which is the destination identified and indicated by the operation device identifier contained in the communication establishment request signal.

[0087] When the remote control device 200 receives the approval signal through the remote wireless communication device 250, it transmits an activation signal to activate the actual machine imaging device 412 and the actual machine sensor group 416 mounted on the work machine 40 to the destination identified and indicated by the actual machine identifier, and waits until it receives a signal corresponding to the actual machine operating status regarding the image acquired by the actual machine imaging device 412 through the remote wireless communication device 250 and the status of the work machine 40 acquired by the actual machine sensor group 416.

[0088] When the real machine control device 400 receives the activation signal through the real machine wireless communication device 450, it activates the real machine imaging device 412 and the real machine sensor group 416, starts transmitting signals corresponding to the real machine operation status relating to the captured images acquired by the real machine imaging device 412 and the state of the work machine 40 acquired by the real machine sensor group 416 to a destination identified and indicated by the operation device identifier, and starts receiving the signals corresponding to the operation mode of the remote control lever for controlling the operation of the actuation mechanism through the real machine wireless communication device 450. Note that the signals corresponding to the real machine operation status contain information related to the real machine identifier corresponding to the real machine wireless communication device 450.

[0089] The remote control device 200 outputs the captured images and the actual machine operation status received through the remote wireless communication device 250 to the remote output interface 220 .

[0090] As described above, as the communication establishment process in the operation preparation process, the remote control device 200 executes the process of establishing the communication with the operation target through the communication device.

[0091] In the communication disconnection processing in the operation preparation processing, the remote control device 200 first stops outputting to the remote output interface 220 captured images and actual operating status of the work machine 40 (one work machine) that is the existing operation target, which are received through the remote wireless communication device 250, and further stops transmitting to the work machine 40 (one work machine) a signal corresponding to the operation mode of the remote operation lever to the work machine 40 (one work machine).

[0092] Next, the remote control device 200 transmits a communication disconnection request signal to a destination identified and indicated by the actual machine identifier corresponding to the actual machine wireless communication device 450 of the work machine 40 that is the existing target of operation, requesting that the established communication be disconnected.

[0093] When the actual machine control device 400 mounted on the existing work machine 40 to be operated receives the communication disconnection request signal via the actual machine wireless communication device 450, it executes processing to turn off the power to the actual machine imaging device 412 and the actual machine sensor group 416 that have been activated. Then, the actual machine control device 400 executes processing to cut off the power supply being supplied to the actual machine control device 400.

[0094] As described above, the real machine control device 400 executes a communication disconnection process as an operation preparation process to cancel the established communication state between the remote operation device 20 and the work machine 40 in the established communication state. Note that, during this communication disconnection process, the energized state to the real machine control device 400 mounted on the work machine 40 may be maintained (processing to cut off the power supply to the real machine control device 400 may not be performed). This allows the real machine control device 400 to receive the signal according to the operation mode of the remote operation lever for controlling the operation of the operating mechanism through the real machine wireless communication device 450, and when the real machine imaging device 412 and the real machine sensor group 416 are started again, a state in which it is possible to transmit the captured image acquired by the real machine imaging device 412 and the signal according to the real machine operating status regarding the state of the work machine 40 acquired by the real machine sensor group 416 to the remote operation device 20 through the real machine wireless communication device 450 may be maintained.

[0095] The communication establishment process and the communication disconnection process may be processed in parallel or in parallel. When the communication establishment process and the communication disconnection process are processed in parallel or in parallel, the remote control device 200 performs the process of acquiring the actual machine identifier corresponding to the work machine 40 (another operation target), while simultaneously starting the process of stopping the output of the captured image and the actual machine operation status of the work machine 40 (one work machine) to the remote output interface 220 and the process of stopping the transmission to the work machine 40 (one work machine) of a signal corresponding to the operation mode of the remote operation lever to the work machine 40 (one work machine). Then, the remote control device 200 proceeds with the communication establishment process and the communication disconnection process by performing the respective processes in parallel or in parallel.

[0096] Here, either the process of turning off the power supply of the real machine imaging device 412 and the real machine sensor group 416 of the work machine 40 (one work machine) that is an existing operation target or the process of starting up the real machine imaging device 412 and the real machine sensor group 416 of the work machine 40 (another operation target) may be executed first. That is, the remote control device 200 may be temporarily in a state of receiving captured images and real machine operation status from both the work machine 40 (one work machine) that is an existing operation target and the work machine 40 (another operation target). In this case, the remote control device 200 selects the captured image and the real machine operation status of the work machine 40 (another operation target) by reading the real machine identifier of the transmission source attached to the signal indicating the received captured image and real machine operation status, and outputs it to the remote output interface 220. In addition, the start time of the communication establishment process and the start time of the communication disconnection process may be the same or different. The end time of the communication establishment process and the end time of the communication disconnection process may be the same or different. The communication establishment process and the communication disconnection process may be serially processed (executed sequentially).

[0097] After the operation preparation process starts, a designated item setting process is executed to set the contents of the designated items for supporting the operation of the work machine 40, which is the operation target by the remote operation device 20 (FIG. 4 / STEP 214). If it is determined that there is no existing operation target (NO in FIG. 4 / STEP 208), the designated item setting process is executed in parallel or in parallel with the execution of the communication establishment process (FIG. 4 / STEP 210) as the operation preparation process, and is started, for example, at the same time as the execution of the communication establishment process starts. If it is determined that there is an existing operation target (YES in FIG. 4 / STEP 208), the designated item setting process is executed in parallel or in parallel with the execution of the communication establishment process and the communication disconnection process (FIG. 4 / STEP 212) as the operation preparation process, and is started, for example, at the same time as the execution of the communication establishment process and the communication disconnection process starts. Next, the remote control device 200 determines whether the operation preparation process is completed (FIG. 4 / STEP 216). If it is determined that the operation preparation process has not been completed (NO in FIG. 4 / STEP 216), the designated item setting process continues to be executed (FIG. 4 / STEP 214).

[0098] The designated item setting process will be described in detail below. Fig. 9A is a diagram showing a first lever pattern setting screen in the operation system according to this embodiment. Fig. 9B is a diagram showing a second lever pattern setting screen in the operation system according to this embodiment. In the designated item setting process, first, for example, a menu screen for selecting multiple designated items is output to the remote output interface 220 (remote image output device 221). When "lever pattern" is selected as a designated item on the menu screen, the first lever pattern setting screen shown in Fig. 9A is output to the remote output interface 220 (remote image output device 221).

[0099] As shown in FIG. 9A, the first lever pattern setting screen includes a first operation guide window Q21, a lever pattern window Q22, an OK button Q23, and a change button Q24. The first operation guide window Q21 displays information guiding the operation method for setting the lever pattern. The lever pattern window Q22 displays a provisionally set lever pattern. When the OK button Q23 is tapped or the like by the operator, the lever pattern displayed in the lever pattern window Q22 is set for real. When the change button Q24 is tapped or the like by the operator, the second lever pattern setting screen shown in FIG. 9B is output to the remote output interface 220 (remote image output device 221).

[0100] The finally set lever pattern may be associated with an operator identifier (including image and distance measurement data for face authentication) and / or an actual machine identifier and stored in a storage device and / or a database constituting the remote control device 200. The operator identifier used in the authentication process and / or the actual machine identifier used in the designation of the operation target may be used to search or read out the last finally set lever pattern from the storage device, and then displayed as a default in the lever pattern window Q22 as a provisionally set lever pattern. The finally set lever pattern may simply be stored in a storage device and / or an external database constituting the remote control device 200, and then read out from the storage device or the like, and then displayed as a default in the lever pattern window Q22 as a provisionally set lever pattern.

[0101] As shown in FIG. 9B, the second lever pattern setting screen includes a second operation guide window Q25 and a plurality of (for example, four) lever pattern candidate windows Q26, Q27, Q28, and Q29. The second operation guide window Q21 displays information for guiding the selection of one of the lever pattern candidate windows Q26, Q27, Q28, and Q29 in order to set a lever pattern. Each of the lever pattern windows Q26, Q27, Q28, and Q29 displays a lever pattern candidate different from the provisionally set lever pattern. In response to a selective tap or the like on one of the lever pattern candidate windows Q26, Q27, Q28, and Q29, the first lever pattern setting screen is output to the remote output interface 220 (remote image output device 221) (see FIG. 9A). At this time, the selected lever pattern candidate is displayed in the lever pattern window Q22 as a provisionally set lever pattern.

[0102] The first lever pattern setting screen and the second lever pattern setting screen may be output to the remote output interface 220 (remote image output device 221) as a single screen, or as three or more transitional screens.

[0103] FIG. 10 is a diagram showing a screen for setting the lever sensitivity in the operation system according to this embodiment. When "lever sensitivity" is selected as a designated item on the menu screen, the lever sensitivity setting screen shown in FIG. 10 is output to the remote output interface 220 (remote image output device 221). As shown in FIG. 10, the lever sensitivity setting screen includes an operation guide window Q31, a lever sensitivity adjustment bar Q32, and an OK button Q33. The operation guide window Q31 displays information that guides the operation method for setting the lever sensitivity. The lever sensitivity adjustment bar Q32 includes a measure indicating the lever sensitivity (which determines the amount of displacement of the operation target according to the amount of lever operation or the speed of displacement) and, for example, a triangular slider, and the lever sensitivity is provisionally set by adjusting the position of the slider along the measure. When the OK button Q33 is tapped, the lever sensitivity displayed on the lever sensitivity adjustment bar Q32 is set for real.

[0104] The finally set lever sensitivity may be associated with the operator identifier and / or the actual machine identifier and stored in a storage device constituting the remote control device 200 and / or an external database. The operator identifier used in the authentication process and / or the actual machine identifier used in the designation of the operation target may be used to search or read out the last finally set lever sensitivity from the storage device as a provisionally set lever sensitivity, and the default position of the slider in the lever sensitivity adjustment bar Q32 may be adjusted. The finally set lever sensitivity may simply be stored in a storage device constituting the remote control device 200 and / or an external database, and the default position of the slider in the lever sensitivity adjustment bar Q32 may be adjusted after being read out from the storage device or the like.

[0105] FIG. 11 is a diagram showing a screen for setting the seat vibration mode in the operation system according to this embodiment. When "seat vibration" is selected as a designated item on the menu screen, the seat vibration setting screen shown in FIG. 11 is output to the remote output interface 220 (remote image output device 221). As shown in FIG. 11, the seat vibration setting screen includes an operation guide window Q41, a seat vibration strength selection window Q42, and an OK button Q43. The operation guide window Q41 displays information that guides an operation method for setting the strength of vibration (including the presence or absence of vibration) of the seat 24 (or the seat device). The seat vibration strength selection window Q42 includes a plurality of buttons, and the strength of the vibration of the seat device is provisionally set by selectively tapping one of the plurality of buttons. When the OK button Q43 is tapped, the strength of the vibration of the seat device corresponding to the one button is set.

[0106] The actually set vibration intensity may be associated with the operator identifier and / or the actual machine identifier and stored in a storage device constituting the remote control device 200 and / or an external database. The operator identifier used in the authentication process and / or the actual machine identifier used in the designation of the operation target may be used to search or read out the last actually set vibration intensity from the storage device, and then displayed as a provisional vibration intensity by default in the seat vibration intensity selection window Q42. The actually set vibration intensity may simply be stored in a storage device constituting the remote control device 200 and / or an external database, and then read out from the storage device or the like, and then displayed as a provisional vibration intensity by default in the seat vibration intensity selection window Q42.

[0107] FIG. 12 is a diagram showing a screen for setting the operation assist in the operation system according to this embodiment. When "operation assist" is selected as a designated item on the menu screen, the operation assist setting screen shown in FIG. 12 is output to the remote output interface 220 (remote image output device 221). As shown in FIG. 12, the operation assist setting screen includes an operation guide window Q51, an operation assist window Q52, and an OK / no button Q53. The operation guide window Q51 displays information that guides an operation method for setting the presence or absence of operation assist. The operation assist window Q52 includes an ON button and an OFF button for provisionally setting the presence or absence of operation assist or ON / OFF. When the OK button Q53 is tapped, the operation assist is set for real. When the operation assist is set for real to ON, the operation content assisted by the operation assist may be output to the remote output interface 220 (remote image output device 221). The operation content is, for example, a horizontal pulling operation in which the bucket, which is the tip attachment 445, is moved linearly along the ground on which the lower running body 41 of the work machine 40 is in contact, and an operation in which the bucket is moved from a first position away from the upper rotating body 42 in the forward direction to a second position closer to the upper rotating body 42 than the first position. There may be multiple types of operation content assisted by the operation assist, and the configuration may be such that the presence or absence of operation assist can be set for the multiple types of operation content.

[0108] Also, a simulation image showing an operating situation in which the work machine 40 is assisted by the operation assist may be output to the remote output interface 220 (remote image output device 221). The remote control device 200 executes a process of recognizing an operating mode of the remote input interface 210 (remote operation mechanism 211) by the operator, a process of correcting the recognized operating mode so as to approach a target operating mode, a process of recognizing a motion mode of the work machine 40 in the virtual space based on the corrected operating mode, and a process of outputting an image that simulates a state in which a change in the attitude of the work machine 40 is reflected according to the recognized motion mode to the remote output interface 220 (remote image output device 221) as the simulation image. For example, when the OK button Q53 is tapped, etc., a simulation image showing an operating situation in which the work machine 40, which is the operation target in the virtual space, is assisted by the operation assist is output to the remote output interface 220 (remote image output device 221). Then, when the operator operates the work machine 40 in the virtual space through the remote input interface 210 (remote operation mechanism 211), the work machine 40 moves in accordance with the operation mode of the remote input interface 210 (remote operation mechanism 211) in the simulation image based on the operation mode of the remote input interface 210 (remote operation mechanism 211), and the position and attitude of the tip attachment 445 and the like are shown to change. Therefore, the remote control device 200, the remote input interface 210 (remote operation mechanism 211) and the remote output interface 220 (remote image output device 221) constitute a "simulation device." Here, when the operator operates the bucket through the remote input interface 210 (remote operation mechanism 211) to move the bucket closer to a second position that is closer to the upper rotating body 42 than the first position in the virtual space, the simulation image shows the tip of the bucket moving linearly from the first position to the second position.Specifically, when the operator performs an operation of rotating the boom 441 upward while rotating the arm 443 downward and further rotating the bucket tip so that it faces downward through the remote input interface 210 (remote operation mechanism 211), the operation is corrected (the deviation amount of the bucket tip from the target trajectory is reduced) so that the bucket tip does not deviate from the target trajectory that linearly connects the first position to the second position by more than a predetermined distance, and is reflected in the simulation image. The first position, the second position, and the target trajectory may be visualized and shown in the simulation image, and the first position and the second position may be set to any position by the operator's operation (for example, by moving the bucket tip to a position to be the first position or the second position and pressing a decision button to set the position). In addition, after communication is established between the remote operation device 20 and the work machine 40 to be operated and the remote operation of the work machine 40 by the remote operation device 20 becomes possible, a remote operation simulation of the work machine 40 in the virtual space may be permitted in a state in which the remote operation of the work machine 40 is stopped or temporarily stopped.

[0109] FIG. 13 is a diagram showing a guidance setting screen in the operation system according to this embodiment. When "Guidance (operation support image)" is selected as a designated item on the menu screen, the operation support image setting screen shown in FIG. 13 is output to the remote output interface 220 (remote image output device 221). As shown in FIG. 13, the operation support image setting screen includes an operation guide window Q61, a depth sense image window Q62, a terrain image window Q63, and an OK button Q64. The operation guide window Q61 displays information that guides an operation method for setting the display mode of the operation support image in the remote output interface 220 (remote image output device 221). The depth sense image window Q62 includes an ON button and an OFF button for provisionally setting the presence or absence or ON / OFF of the display of the depth sense image in the remote output interface 220 (remote image output device 221). The terrain image window Q63 includes an ON button and an OFF button for provisionally setting the presence or absence or ON / OFF of the display of the terrain image in the remote output interface 220 (remote image output device 221). When the OK button Q64 is tapped or the like, the display or non-display of the depth sense image and the display or non-display of the terrain image are set.

[0110] The set information on whether or not to display the depth sense image and / or the terrain image may be stored in a storage device constituting the remote control device 200 and / or an external database in association with an operator identifier and / or an actual machine identifier. The operator identifier used in the authentication process and / or the actual machine identifier used in the designation of the operation target may be used to search or read out the last set information on whether or not to display (ON / OFF) from the storage device, and then displayed as a default in the depth sense image window Q62 and / or the terrain image window Q63. The set information on whether or not to display the depth sense image and / or the terrain image may simply be stored in a storage device constituting the remote control device 200 and / or an external database, and then read out from the storage device or the like, and then displayed as a default in the depth sense image window Q62 and / or the terrain image window Q63.

[0111] Also, a simulation image showing a depth sense image and / or a terrain image may be output to the remote output interface 220 (remote image output device 221). For example, when the display of the depth sense image is set to ON, a simulation image showing an operating situation assisted by the operation assist in the work machine 40, which is the operation target in the virtual space, is output to the remote output interface 220 (remote image output device 221). Then, when the operator operates the work machine 40 in the virtual space through the remote input interface 210 (remote operation mechanism 211), the simulation image shows how the work machine 40 operates according to the operation mode, and the position and attitude of the tip attachment 445 and the like change. For this reason, the remote control device 200, the remote input interface 210 (remote operation mechanism 211) and the remote output interface 220 (remote image output device 221) constitute a "simulation device." Here, in the simulation image showing the operating situation of the work machine 40, a depth sense image for showing the position or depth sense of one or more locations defined in the work mechanism 44 is displayed. Fig. 14 is a diagram showing a depth perception image in the operation system according to this embodiment. As a result, for example, as shown in Fig. 14, a mark M is displayed on the simulation image. A mark M is displayed indicating the position of a second specified point p2 as a result of vertically projecting a first specified point p1, which is the left-right center point of the base end of the tip attachment 445. The first specified point p1 may be specified as a point other than the left-right center point of the base end of the tip attachment 445 (such as the tip of the tip attachment 445). The second specified point p2 may be displayed at a predetermined distance from the first specified point p1 in the virtual space, or may be displayed at any height in the virtual space.

[0112] Fig. 15 is a diagram showing a terrain image in the operation system according to this embodiment. When the display of the terrain image is set to ON, a terrain image showing the terrain around (forward of) the work machine 40 is superimposed on a simulation image showing the operating status of the work machine 40. This causes, for example, as shown in Fig. 15, a mesh-like terrain image showing the terrain or unevenness of the ground surface to be displayed in the simulation image.

[0113] Fig. 16A is a diagram showing a first button function setting screen in the operation system according to this embodiment. When "button assignment" is selected as a specification item on the menu screen, the first button function setting screen shown in Fig. 16A is output to the remote output interface 220 (remote image output device 221). As shown in Fig. 16A, the first button function setting screen includes an operation guide window Q71, a button layout window Q72, a function assignment status window Q73, and an OK button Q74. The operation guide window Q71 displays information that guides the user on how to assign functions to multiple buttons that make up the remote operation mechanism 211. Functions that can be assigned to the buttons include a function for adjusting the angle of the real machine imaging device 412 in the left-right direction, a function for adjusting the angle of the real machine imaging device 412 in the up-down direction, a zoom-in function and a zoom-out function for the real machine imaging device 412, a function for switching on / off an audio output device (for example, a horn that issues an alarm sound to the surroundings) mounted on the work machine 40, a setting function such as calibration of the operation lever that constitutes the remote operation mechanism 211, a function for switching between increasing and decreasing the pressure of the hydraulic oil for the work mechanism 44, a function for switching between exciting and releasing the lifting magnet as the end attachment 445, and a function for canceling the temporary stop state of the work machine 40. The button layout window Q72 shows the layout of multiple buttons to which functions can be assigned. For example, the button layout window Q72 shows an arrangement of a total of ten buttons, with buttons L1, L2, and L3 arranged on the left travel lever 2110, buttons L4 and L5 arranged on the left operation lever 2111, buttons R1, R2, and R3 arranged on the right travel lever 2110, and buttons R4 and R5 arranged on the right operation lever 2112. The function assignment status window Q73 shows a chart showing the functions assigned to each of the multiple buttons. When the OK button Q74 is tapped, etc., the functions assigned to each button shown in the function assignment status window Q73 are finally set.

[0114] The functions assigned to each button in the actual setting may be associated with the operator identifier and / or the actual device identifier and stored in a storage device constituting the remote control device 200 and / or an external database. The operator identifier used in the authentication process and / or the actual device identifier used in the operation target designation may be used to search or read out the last function assigned to each button from the storage device, and then displayed as a default in the function assignment status window Q73. The functions assigned to each button in the actual setting may simply be stored in a storage device constituting the remote control device 200 and / or an external database, and then read out from the storage device or the like and displayed as a default in the function assignment status window Q73.

[0115] FIG. 16B is a diagram showing a second button function setting screen in the operation system according to this embodiment. When one of the buttons shown in the button layout window Q72 (or the function assignment status window Q73) is selectively tapped, the second button function setting screen shown in FIG. 16B is output to the remote output interface 220 (remote image output device 221). As shown in FIG. 16B, the second button function setting screen includes a button window Q75, a button function window Q76, a button function selection window Q77, and an OK button Q78. The button window Q75 displays (highlights as appropriate) the name and / or position of one button selected from the multiple buttons. The button function window Q76 displays the function assigned to the one button. The button function selection window Q77 displays a list of functions that can be assigned to the one button (in the form of a pull-down menu, etc.). When the OK button Q78 is tapped or the like, a first button function setting screen is output to the remote output interface 220 (remote image output device 221) (see FIG. 16A).

[0116] In addition, when the work machine 40 is equipped with multiple actual machine imaging devices 412 with different imaging ranges (which may partially overlap), "screen allocation" may be selected as a specification item on the menu screen, and a correspondence may be set between each of the multiple image output devices (displays or monitors) that make up the remote image output device 221 and the captured images obtained through each of the multiple actual machine imaging devices 412 displayed on each image output device.

[0117] As described above, various designated item setting processes are executed. If it is determined that the operation preparation process being executed has been completed (FIG. 4 / STEP 216: YES), the remote control device 200 in the remote operation device 20 executes remote operation process for remotely operating the work machine 40 (FIG. 4 / STEP 218). If the designated item setting process is in progress for a certain designated item when the operation preparation process is completed, the designated item setting process may be executed continuously (FIG. 4 / STEP 214).

[0118] On the condition (requirement) that communication for remotely operating a work machine 40 (operation target) by the remote operation device 20 has been established, the actual machine control device 400 in the work machine 40 transmits a captured image acquired via the actual machine imaging device 412 to the remote operation device 20. The transmitted captured image (or captured image data) may be not only the captured image itself, but also image data representing a simulated environmental image generated based on the captured image.

[0119] For example, in the remote operation device 20, the remote control device 200 may establish communication for remotely operating the work machine 40 (operation target) by the remote operation device 20 in response to transmitting a designation signal to the work machine 40 via the remote wireless communication device 250. When a plurality of work machines 40 are a plurality of operation target candidates, a designation signal may be transmitted to each of the plurality of work machines 40. The presence or absence of a designation operation via the remote input interface 210 may be determined by the operator, and a designation signal may be transmitted when the determination result is positive. The "designation operation" is, for example, an operation such as tapping on the remote input interface 210 for the operator to designate the work machine 40 that he or she intends to remotely operate.

[0120] When the remote operation device 20 receives a captured image through the remote wireless communication device 250, the captured image or the work environment image is output to the remote image output device 221 (display device) (FIG. 6 / STEP 280). As a result, for example, a captured image showing the ground spreading out in front of the cab 43, a bucket corresponding to the tip attachment 445 that is a part of the work mechanism 44, the ground, and the like is output to the remote image output device 221.

[0121] When the designated item "button assignment" is selected in the designated item setting process and the function assignment of each button is set, the assigned function is applied. As a result, the range and / or scale of the captured image can be adjusted by the operator operating the button to which the angle adjustment and / or zoom-in / zoom-out functions of the real imaging device 412 are assigned (see FIG. 4 / STEP 214, FIG. 16A, and FIG. 16B).

[0122] When the designated item "guidance" is selected in the designated item setting process and the display of the depth image is set to "Yes (ON)", a mark M may be superimposed on the captured image as a depth image (see FIG. 4 / STEP 214, FIG. 13, and FIG. 14). For example, the remote control device 200 calculates the position coordinates of a first designated point p1, which is the left-right center point of the base end of the tip attachment 445, from the hoisting angle of the boom 441, the rotation angle of the arm 443, and the rotation angle of the tip attachment 445, and displays the mark M superimposed on the position of a second designated point p2 obtained by projecting the position coordinates vertically downward. When the designated item "guidance" is selected in the designated item setting process and the display of the terrain image is set to "Yes (ON)", a mesh may be superimposed on the captured image as a terrain image (see FIG. 4 / STEP 214, FIG. 13, and FIG. 15). The mesh to be superimposed is generated, for example, based on the distance from the work machine 40 to the surrounding ground surface obtained by a sensing device mounted on the work machine 40 that measures the distance to surrounding objects.

[0123] Next, the remote control device 200 recognizes the operation mode of the remote operation mechanism 211 (FIG. 6 / STEP 281). Specifically, when the designated item "lever pattern" is selected in the designated item setting process and the lever pattern is set, the operation mode of the remote operation mechanism 211 or the operating levers 2110, 2111, and 2112 constituting the remote operation mechanism 211 is recognized according to the lever pattern (see FIG. 4 / STEP 214, FIG. 9A, and FIG. 9B). When the designated item "lever sensitivity" is selected in the designated item setting process and the lever sensitivity is set, the operation mode of the remote operation mechanism 211 or the operating levers 2110, 2111, and 2112 constituting the remote operation mechanism 211 is recognized according to the lever sensitivity (see FIG. 4 / STEP 214 and FIG. 10).

[0124] Next, the remote control device 200 judges whether the authentication flag f (see FIG. 5 / STEP 261 and STEP 262) is 1 or not (FIG. 6 / STEP 282). When the authentication flag f is set to "1" in response to the success of the authentication process of the operator (YES in FIG. 6 / STEP 282 (YES in FIG. 5 / STEP 260 → see STEP 261)), a remote operation command corresponding to the operation mode is transmitted to the work machine 40 through the remote wireless communication device 250 (FIG. 6 / STEP 284). When the authentication flag f is set to "0" in response to the failure of the authentication process of the operator (NO in FIG. 6 / STEP 282 (NO in FIG. 5 / STEP 254, NO in STEP 260 → see STEP 262)), the transmission (transmission) of the remote operation command corresponding to the operation mode to the work machine 40 through the remote wireless communication device 250 is suspended (prohibited) or only a part of the remote operation command is transmitted (limited) (FIG. 6 / STEP 286).

[0125] The remote control device 200 then acquires an operation history, which is a history of operation of the work machine 40 via the remote operation mechanism 211, in association with an operator identifier for identifying the operator operating the remote operation mechanism 211. In particular, in this embodiment, a time series of the operation modes of the remote operation mechanism 211 by the operator is acquired sequentially as the operation history of the work machine 40 in association with the operator identifier and / or the actual machine identifier, and stored and held in a storage device constituting the remote control device 200 and / or an external database (FIG. 6 / STEP 288). When the execution of remote operation processing for remotely operating the work machine 40 by the remote control device 200 (FIG. 4 / STEP 218) continues, the processing of the operation processing function (FIG. 6 / STEP 280 → . . . → FIG. 6 / STEP 288) is repeated.

[0126] In the work machine 40, when the actual machine control device 400 receives a remote operation command via the actual machine wireless communication device 450, the operation of the work mechanism 44 and the like is controlled in accordance with the remote operation command. For example, while holding an object such as soil or sand with a bucket as the tip attachment 445, a predetermined task is performed involving the operation of the work mechanism 44 and the like in order to displace the real space position of the bucket.

[0127] When the designated item "operation assist" is selected in the designated item setting process and the operation assist is set to "ON", specific operation content is assisted based on the setting. For example, when a horizontal pulling operation is assisted based on the setting, when the operator operates the bucket to approach a second position closer to the upper rotating body 42 of the work machine 40 (operation target) from a first position away from the upper rotating body 42 in the forward direction through the remote input interface 210 (remote operation mechanism 211), the tip of the bucket moves linearly from the first position to the second position in the work machine 40, and a captured image or a work environment image capturing this state is output to the remote image output device 221 (display device). Specifically, when an operation of rotating the boom 441 upward while rotating the arm 443 downward and further rotating the bucket tip so that it faces downward is performed by the operator through the remote input interface 210 (remote operation mechanism 211), the operation is corrected (the deviation of the bucket tip from the target trajectory is reduced) so that the bucket tip does not deviate by more than a predetermined distance from a target trajectory that linearly connects the first position to the second position, and is reflected in the operation of the work mechanism 44. The first position, the second position, and the target trajectory may be superimposed on a captured image or a work environment image and output to the remote image output device 221 (display device), and the first position and the second position may be set to any position by the operator's operation (for example, the position can be set by moving the bucket tip to a position desired to be the first position or the second position and pressing a decision button).

[0128] In the work machine 40, the actual machine control device 400 transmits the actual machine operation status acquired through the actual machine sensor group 416 to the remote operation device 20. The actual machine operation status includes the actual machine attitude (tilt angle around the pitch axis and / or roll axis with respect to the vertical direction of the upper rotating body 42), whether the prime mover is operating, the attitude of the work mechanism 44, etc. When the remote operation device 20 receives the actual machine operation status through the remote wireless communication device 250, the remote control device 200 controls the operation of the seat drive device 224 to adjust the attitude of the seat 24 so that the seat attitude is aligned or coincident with the actual machine attitude. At this time, among the frequency components representing the time-varying manner of the actual machine attitude, high-frequency components exceeding a threshold value may be cut, and the time-varying manner of the seat attitude may be adjusted according to only low-frequency components below the threshold value.

[0129] When the remote control device 200 determines, based on the actual machine operating status, that the prime mover of the work machine 40 is operating, the seat 24 may be vibrated by the seat drive device 224. The seat 24 vibrates around the pitch axis and / or the roll axis with a predetermined minute amplitude and at a frequency corresponding to the frequency of the prime mover, so that the attitude of the seat 24 changes slightly with respect to the designated attitude. Alternatively, the seat 24 vibrates in a direction parallel to at least one of the pitch axis, the roll axis, and the yaw axis with a predetermined minute amplitude and at a frequency corresponding to the frequency of the prime mover, so that the attitude of the seat 24 changes slightly in at least one of the forward / backward, left / right, and up / down directions while being maintained in the designated attitude.

[0130] When the designated item "seat vibration" is selected in the designated item setting process and the vibration intensity of the seat 24 (or seat device) is set, the operating speed and amplitude of the seat drive device 224, and ultimately the speed and amount of change in the posture of the seat 24, are adjusted according to the intensity (see Figure 4 / STEP 214 and Figure 11).

[0131] In the remote operation device 20, the remote control device 200 determines whether or not an operation end condition for the work machine 40 has been satisfied (FIG. 4 / STEP 220). The operation end condition is a condition for determining that work is not being performed using the work machine 40 that is the operation target.

[0132] The operation end condition includes, for example, a condition that the operation of the prime mover of the work machine 40 has been stopped (or an operation to stop the operation of the prime mover of the work machine 40 has been performed via the remote input interface 210). The operation end condition may also include a condition that the operation lever, travel pedal, and / or option pedal constituting the remote input interface 210 have been returned to the neutral position or the initial position (and that state has been maintained for a certain period of time) to cause the work machine 40 to end a predetermined operation of the operating mechanisms such as the work mechanism 44. In addition, the operation end condition may include conditions such as whether or not the cut-off lever has been operated to the lock position, whether or not an operation to stop the operation of the prime mover of the work machine 40 has been performed via the remote input interface 210, and whether or not the switch constituting the remote input interface 210 has been switched from the ON state to the OFF state to cause the work machine 40 to end a predetermined operation of the operating mechanisms such as the work mechanism 44, and may be detected depending on whether or not a plurality of conditions are satisfied.

[0133] If it is determined that the operation end condition is not satisfied (NO in FIG. 4 / STEP 220), the remote operation of the work machine 40 by the remote operation device 20 continues (FIG. 4 / STEP 218).

[0134] On the other hand, if it is determined that the operation end condition is satisfied (YES in FIG. 4 / STEP 220), the remote control device 200 determines the operation content by the operator via the remote input interface 210 (FIG. 4 / STEP 222).

[0135] FIG. 17 is a diagram showing an operation selection screen in the operation system according to this embodiment. Specifically, the operation selection screen shown in FIG. 17 is output to the remote output interface 220. The operation selection screen shows an operation restart button Q81, an operation switching button Q82, and an operation end button Q83 which constitute the remote input interface 210. The operation restart button Q81 constitutes an input interface for restarting remote operation, via the remote operation device 20, of the work machine 40 which is the operation target and in a state of communication establishment with the remote operation device 20. The operation switching button Q82 constitutes an input interface for starting remote operation, via the remote operation device 20, of a work machine 40 (second work machine) which is the operation target different from the work machine 40 (first work machine) which is the operation target and in a state of communication establishment with the remote operation device 20. The operation end button Q83 constitutes an input interface for ending remote operation, via the remote operation device 20, of the work machine 40 which is the operation target and in a state of communication establishment with the remote operation device 20.

[0136] When the operator taps the operation resume button Q81 (when the operation content is an operation to resume remote operation of the work machine 40, which is the operation target and in which communication has been established with the remote control device 20, via the remote control device 20) (FIG. 4 / STEP 222-1), remote operation of the work machine 40 by the remote control device 20 is continued (FIG. 4 / STEP 218).

[0137] When the operation switching button Q82 is tapped or the like (when the operation content is an operation for starting remote operation, via the remote operation device 20, of a work machine 40 (second work machine) that is an operation target other than the work machine 40 (first work machine) that is an operation target and has established communication with the remote operation device 20) (FIG. 4 / STEP 222-2), the authentication process (FIG. 4 / STEP 200) and subsequent processes are repeated. In this case, since there is a work machine 40 (first work machine) that is an operation target and has established communication with the remote operation device 20, communication disconnection process with the first work machine and communication establishment process with the new operation target work machine 40 (second work machine) are executed as "operation preparation process" (FIG. 4 / STEP 208: YES → see STEP 212).

[0138] If the authentication flag f is set to “0” during the authentication process (NO in STEP 254 in FIG. 5, NO in STEP 260 → see STEP 262), the operation resume button Q81 and / or the operation switch button Q82 may not be displayed, or operations such as tapping may be prohibited.

[0139] When the operation end button Q83 is tapped or the like (when the operation content is an operation to end remote operation of the work machine 40, which is the operation target and in which communication has been established with the remote control device 20, via the remote control device 20) (FIG. 4 / STEP 222-3), communication disconnection processing between the remote control device 20 and the work machine 40, which is the operation target, is executed (FIG. 4 / STEP 224), and remote operation of the work machine 40 by the remote control device 20 ends.

[0140] (Action and effect) According to the remote control device 20 that exerts the above-mentioned functions and the operation system configured by the remote control device 20, during the execution of the communication establishment process as the operation preparation process, it becomes possible to set the contents of the designated items for supporting the operation of the work machine 40, which is the object of operation by the remote control device 20 after the completion of the communication establishment process (see NO in FIG. 4 / STEP 208→STEP 210→STEP 214, FIG. 9A, FIG. 9B, FIG. 10, FIG. 11, FIG. 12, FIG. 13, FIG. 16A and FIG. 16B). Therefore, the waiting time during the execution of the communication establishment process between the remote control device 20 and the work machine 40, which is the object of operation, can be utilized. As a result, it is also possible to reduce the stress of the operator during the waiting time. Note that the setting of the contents of the designated items (designated item setting process) may be executed at least during the execution of the communication establishment process.

[0141] When the operation target is switched from one work machine 40 (first operation target / first work machine / first target) to another work machine 40 (second operation target / second work machine / other target) through the communication device, a communication disconnection process and a communication establishment process are executed as an operation preparation process (see Fig. 4 / STEP 208 ... YES → STEP 212). When the communication disconnection process and the communication establishment process are executed, it becomes possible to set the contents of the designated items through the remote input interface 210 (see Fig. 4 / STEP 214). Therefore, the waiting time during the communication disconnection process and the communication establishment process for switching the operation target is utilized.

[0142] It becomes possible to virtually experience, through the simulation device, the assistance mode and / or operation mode of the work machine 40 to be operated, which is predicted to occur according to the settings of the specified items through the remote input interface 210 (see Figs. 14 and 15). It becomes possible to reset or change the settings of the specified items through the remote input interface 210 based on the experience. As described above, the simulation device is composed of the remote control device 200, the remote input interface 210 (remote operation mechanism 211), and the remote output interface 220 (remote image output device 221).

[0143] The setting contents of the specified items are stored in a storage device or the like constituting the remote control device 200, and the setting contents of the specified items stored in the storage device or the like are output to the remote output interface 220 (remote image output device 221) (see Fig. 9A / Q22, Fig. 10 / Q32, Fig. 11 / Q42, Fig. 12 / Q52, Fig. 13 / Q62, Q63, Fig. 16A / Q73). The current setting contents of the specified items output to the remote output interface 220 can be confirmed by an operator or the like. Based on the output contents, it becomes possible to reset or change the setting contents of the specified items through the remote input interface 210 (see Figs. 9B / Q26 to Q29, Fig. 10 / Q32, Fig. 11 / Q42, Fig. 12 / Q52, Fig. 13 / Q62, Q63, Fig. 16B / Q75 to Q78).

[0144] According to the remote operation device 20 that performs the above functions and the operation system configured thereby, when the operation target is switched from a work machine 40 (first work machine) as one operation target to a work machine 40 (second work machine) as another operation target, communication disconnection processing and communication establishment processing are executed in parallel (see FIG. 4 / STEP 208: YES → STEP 210). The "communication disconnection processing" is processing for disconnecting communication that was established between the remote operation device 20 and the work machine 40 as the one operation target. The "communication establishment processing" is processing for establishing communication between the remote operation device 20 and the work machine 40 as the other operation target. For this reason, switching of the operation target can be executed smoothly.

[0145] The operator can, in accordance with his / her intention, designate a work machine 40 as a new operation target through the remote input interface 210. More specifically, the operator can, in accordance with his / her intention, cancel the designation of the work machine 40 that was the previous operation target through the remote input interface 210, and then designate a work machine 40 as a new operation target (see Fig. 4 / STEP 222-2 →STEP 204 and Fig. 17 / Q82 → ... → Fig. 8B). This can avoid a situation in which the operation target is erroneously switched when the operator does not intend to switch the operation target.

[0146] When a work machine 40 that is an operation target with established communication is specified as a new operation target through the remote input interface 210, the remote control device 200 suspends the execution of the communication disconnection process and the communication establishment process (FIG. 4 / STEP 222 1 →STEP 218 and FIG. 17 / Q81). When the work machine 40 specified as the new operation target is the same as the work machine 40 that was previously specified, the communication already established with that work machine 40 continues to be used. This makes it unnecessary to execute the communication disconnection process and the communication establishment process again when specifying (re-specifying) a work machine 40 that is a new operation target, and operation of the operation target can be smoothly resumed.

[0147] When the operator gives an instruction to end operation of the work machine 40 to be operated through the remote input interface 210 according to his / her intention (command to end operation), the remote control device 200 executes communication disconnection processing (see FIG. 4 / STEP 222-3 →STEP 224 and FIG. 17 / Q83). This can prevent a situation in which communication established between the remote operation device 20 and the work machine 40 to be operated is erroneously disconnected.

[0148] According to the remote operation device 20 that performs the above-mentioned functions and the operation system configured therewith, the operation history of the work machine 40 that is the operation target can be acquired separately for each operator identifier and, in turn, each operator identified by the operator identifier, and can be stored and retained in a database or the like (see FIG. 6 / STEP 288). When the operation target of the remote operation device 20 is switched from one work machine 40 (first work machine) to another work machine 40 (second work machine) (see FIG. 4 / STEP 222-2 → . . . →STEP 204 and FIG. 8B / Q14, Q15), the operation history of the other work machine 40 is acquired together with the operator identifier (see FIG. 6 / STEP 288). This allows the operation history of each different operation target (work machine 40) by the operator to be acquired separately.

[0149] Furthermore, when the condition that no work is being performed using the work machine 40 that is the operation target is satisfied, the remote control device 200 determines whether or not there is an instruction to switch the operation target (see FIG. 4 / STEP 220 -> STEP 222). This makes it possible to avoid a situation in which, when work is being performed using a work machine 40 that is an operation target, the operation target is switched to another operation target, and the work is stopped against the operator's intention.

[0150] When an operator is replaced by another operator, the operation history of the work machine 40 to be operated is acquired together with an operator identifier for identifying the new operator (FIG. 5 / STEP 202->...->YES in STEP 250, see FIG. 6 / STEP 288). This makes it possible to distinguish and acquire the operation history of the work machine 40 to be operated for each different operator.

[0151] Furthermore, when a notification of a change from one operator to another operator is received through the remote input interface 210 (and further, when an operator is actually changed and the operator identifier changes from one operator identifier corresponding to one operator to another operator identifier corresponding to another operator), the operation history of the operation target is acquired together with the operator identifier of the other operator (see FIG. 5 / STEP 202->...->YES in STEP 240, FIG. 6 / STEP 288). This allows the operation history of the work machine 40, which is the operation target, to be distinguished and acquired for each different operator. Furthermore, when a switching instruction (switching command) for switching the operation target from one work machine 40 (first work machine) to another work machine 40 (second work machine) is received through the remote input interface 210 (and further, when the operation target is actually switched), the operation history of the other work machine 40 is acquired together with (associated with) the operator identifier (see FIG. 5 / STEP 202->...->YES in STEP 240, FIG. 6 / STEP 288). This makes it possible to distinguish and acquire the operation history for each work machine 40 that is a different target of operation by the operator.

[0152] When an operator is replaced by another operator, if authentication of the other operator fails (see Fig. 5 / STEP 250: YES → STEP 251 → ... → NO in STEP 260 → STEP 262), the operation of the work machine 40 as the operation target by the other operator through the remote operation mechanism 211 is prohibited or restricted (see Fig. 6 / STEP 282: NO → STEP 286). As a result, the prohibited or restricted operation history of the other operator and the work machine 40 as the operation target are both acquired and stored in a database or the like.

[0153] When an operator is replaced by another operator, if authentication of the other operator fails (see FIG. 5 / STEP 250: YES → STEP 251 → ... → STEP 260: NO → STEP 262), switching of the work machine 40 as the operation target in response to a switching instruction from the other operator is prohibited. On the other hand, if authentication of the other operator is successful, switching of the work machine 40 as the operation target in response to a switching instruction from the other operator is permitted.

[0154] (Another embodiment of the present invention) When the operation target is switched from one operation target work machine 40 to another operation target work machine 40 having specifications different from the one operation target, the setting of the designated items corresponding to the differences in the specifications may be accepted by the remote control device 200 through the remote input interface 210. For example, when the types of the tip attachments 445 are different, it may be possible to set the assignment mode to each button so that a function specific to the operation of the tip attachment 445 (e.g., a lifting magnet) of the work machine 40 after the switch is assigned to the button. Specifically, when a function specific to the operation of the tip attachment 445, which is a difference in the specifications of the work machine 40 after the switch, is not assigned to the button, it is determined that the operation preparation process is not completed (NO in FIG. 4 / STEP 216), and the designated item setting process may be continuously executed (FIG. 4 / STEP 214). This ensures that a function specific to the operation corresponding to the differences in the specifications is assigned to the button, enabling smooth operation of the other work machine 40 after the switch of the operation target.

[0155] In the above embodiment, the "simulation device" presents a simulation image showing the operating status of the work machine 40 by displaying an operation assist, a depth sense image, and a terrain image specified on the menu screen, but the "simulation device" may also be made to allow the user to experience other designated item functions that can be specified on the menu screen. For example, when the designated item "lever pattern" is selected in the designated item setting process and the lever pattern is set, the operation mode of the remote control mechanism 211 or the operating levers 2110, 2111, and 2112 that configure the remote control mechanism 211 is recognized in accordance with the lever pattern, and the operation of the work mechanism 44, etc. of the work machine 40 in the virtual space is controlled in accordance with the recognized operation mode. When the designated item "lever sensitivity" is selected in the designated item setting process and the lever sensitivity is set, the operation of the work mechanism 44, etc. of the work machine 40 in the virtual space is controlled in accordance with the operation mode recognized according to the lever sensitivity. When the designated item "seat vibration" is selected in the designated item setting process and the strength of the seat vibration is finally set, the operating speed and amplitude of the seat drive device 224, and further the speed and amount of change in the attitude of the seat 24, are adjusted according to the operating status of the work machine 40 in the virtual space in response to the intensity. The operating status of the work machine 40 in the virtual space includes the tilt angle around the pitch axis and / or roll axis of the upper rotating body 42 of the work machine 40 relative to the vertical direction in the virtual space, whether or not the prime mover of the work machine 40 is operating in the virtual space, and the attitude of the work mechanism 44 of the work machine 40 in the virtual space. When the designated item "button assignment" is selected in the designated item setting process and the degree of button assignment is finally set, for example, a button to which an angle adjustment and / or a zoom-in function / zoom-out function of the actual machine imaging device 412 of the work machine 40 in the virtual space is assigned is operated by the operator, and a virtual captured image or work environment image with the range and / or scale of the captured image adjusted is output to the remote image output device 221 (display device).

[0156] Of the functions of the remote operation device 20, except for functions specific to the remote operation device 20, such as communication establishment processing (see FIG. 4 / STEP 210 and STEP 212), communication disconnection processing (see FIG. 4 / STEP 212), and remote operation processing (see FIG. 4 / STEP 218), authentication processing (see FIG. 4 / STEP 200), setting processing of the operation target and work content (see FIG. 4 / STEP 204), and designated item setting processing (see FIG. 4 / STEP 214), may be executed by the information processing terminal 10. The functions of the remote operation device 20, such as communication establishment processing (see FIG. 4 / STEP 210 and STEP 212), communication disconnection processing (see FIG. 4 / STEP 212), and remote operation processing (see FIG. 4 / STEP 218), may be executed by a server (management device) capable of communicating with the remote operation device 20 and the work machine 40 via the second communication network.

[0157] Even after a communication disconnection process is executed between the remote operation device 20 and a work machine 40 that is an operation target, the communication disconnection process may be executed so that the operating status (such as a powered state) of the real machine control device 400 and the real machine wireless communication device 450 of the work machine 40 is maintained as it was before the communication disconnection process. A command at the time of such disconnection may be transmitted by the remote control device 200. As a result, when communication is established again between the remote operation device 20 and the work machine 40 that is an operation target, a process for restoring the operating status of the real machine control device 400 and the real machine wireless communication device 450 to the status before the communication disconnection process (such as a process for energizing the real machine wireless communication device 450 to return it to a state where the communication establishment process can be executed) is not required, so that the communication establishment process can be executed smoothly and quickly. For example, when the communication disconnection process is executed, the power supply state to the real machine control device 400 mounted on the work machine 40 is maintained (the process of cutting off the power supply to the real machine control device 400 is not performed), and the power supply state to the real machine imaging device 412 and the real machine sensor group 416 is maintained (the process of cutting off the power supply to the real machine imaging device 412 and the real machine sensor group 416 is not performed).The real machine control device 400 then transmits signals corresponding to the real machine operating status relating to the captured images acquired by the real machine imaging device 412 and the status of the work machine 40 acquired by the real machine sensor group 416 to a destination identified and indicated by the operation device identifier, and continues the process of receiving the signals corresponding to the operation mode of the remote operation lever for controlling the operation of the operating mechanism through the real machine wireless communication device 450. Also, in this case, a signal according to the actual machine operating status is transmitted to a destination identified and indicated by the operation device identifier, and the destination remote operation device 20 can determine whether or not information related to the actual machine identifier contained in the received signal corresponds to that of the work machine 40 currently being operated by the remote operation device 20, thereby determining that the received signal is not that of the work machine 40 currently being operated by the remote operation device 20. Then, since the received signal is not that of the work machine 40 currently being operated by the remote operation device 20, a process of discarding the received signal is performed.In addition, when the control device in the present invention includes an actual machine control device 400 of the work machine 40, the actual machine control device 400 may perform control so that the operating status (such as power supply state) of the actual machine control device 400 and the actual machine wireless communication device 450, etc. are maintained as they were before the communication disconnection processing.

[0158] Execution of the communication disconnection process may disconnect a part of the communication established between the remote operation device 20 and one of the work machines 40 to be operated. For example, when the communication disconnection process is executed, a process is executed to turn off the power to the actual machine imaging device 412 and the actual machine sensor group 416 that have been activated, but the power supply to the actual machine control device 400 mounted on the work machine 40 is maintained (a process to cut off the power supply to the actual machine control device 400 is not executed). The actual machine control device 400 then transmits signals corresponding to the actual machine operating status relating to the captured images acquired by the actual machine imaging device 412 and the status of the work machine 40 acquired by the actual machine sensor group 416 to a destination identified and indicated by the operation device identifier, and continues the process of receiving the signals corresponding to the operation mode of the remote operation lever for controlling the operation of the operating mechanism through the actual machine wireless communication device 450. In this case, the signal corresponding to the actual machine operation status regarding the state of the work machine 40 acquired by the actual machine sensor group 416 becomes an empty signal with no information, and the signal corresponding to the operation mode of the remote control lever is not received until the work machine 40 becomes an operation target again. The empty signal may be replaced with a signal indicating information indicating that the signal is empty. The empty signal may be transmitted at a transmission cycle lower than the transmission cycle in which the signal corresponding to the actual machine operation status acquired by the activated actual machine imaging device 412 and the actual machine sensor group 416 is transmitted to the transmission destination. In this case, the signal corresponding to the actual machine operation status is transmitted to the transmission destination identified and indicated by the operation device identifier, and the remote operation device 20, which is the transmission destination, can determine whether or not the information related to the actual machine identifier included in the received signal corresponds to that of the work machine 40 currently operated by the remote operation device 20, thereby determining that the received signal is not the current operation target of the remote operation device 20. Then, since the received signal is not the current operation target of the remote operation device 20, a process of discarding the received signal is performed.As a result, the communication volume between the remote control device 20 and the work machine 40 is maintained, although it is low compared to the communication established state, so that when communication is established again between the remote control device 20 and the work machine 40 as the one operation target, the process of restoring the operating status of the real machine control device 400 and the real machine wireless communication device 450, etc. to the status before the communication disconnection process is unnecessary or can be simplified. For example, when the real machine control device 400 receives a communication establishment request signal from the remote control device 200, the real machine control device 400 can transmit a signal including an imaged image and the real machine operating status to the remote control device 200 simply by activating the machine imaging device 412 and the real machine sensor group 416, and can restore the status before the communication disconnection process. Also, for example, the real machine control device 400 may enable the power supply to the real machine imaging device 412 and the real machine sensor group 416 by not performing the process of turning off the power of the activated real machine imaging device 412 and the real machine sensor group 416 when the communication disconnection process is executed. In this case, the remote control device 200 receives a signal including a captured image and an actual machine operation status before transmitting a communication establishment request signal to the actual machine control device 400, and outputs the captured image and the actual machine operation status to the remote output interface 220 when the communication establishment process is executed. As described above, the communication establishment process is executed smoothly and quickly. Note that the communication disconnection process in the present invention includes a process in which all communications between the communication devices are disconnected, as well as a process in which some communications are maintained as described above. In this case, the communication between the communication devices may include a main communication that requires a large amount of communication and a sub-communication that requires a relatively small amount of communication, and the communication disconnection process may be a mode in which the main communication is disconnected while the sub-communication is maintained.

[0159] When the condition that there is no operator change report through the remote input interface 210 (first flag f1 is set to "0") is satisfied (see FIG. 5 / STEP 240: NO → STEP 242), the presence or absence of an instruction to switch the work machine 40 as an operation target through the remote input interface 210 may be determined (see FIG. 4 / STEP 222) while maintaining the communication established with the work machine 40 as one operation target. As a result, when the work machine 40 designated as a new operation target is the same as the previous operation target (see FIG. 4 / STEP 222-1), the communication already established between the remote operation device 20 and the work machine 40 continues to be used, eliminating the need to execute communication disconnection processing and communication establishment processing again, and operation of the work machine 40 can be smoothly resumed.

[0160] When the conditions that an operator change report has been made through the remote input interface 210 (the first flag f1 is set to "1") and that the acquired operator identifier has changed from one operator identifier to another operator identifier (the second flag f2 is set to "1") are respectively satisfied (see FIG. 5 / STEP 240: YES → STEP 241 and STEP 250: YES → STEP 251), the presence or absence of an instruction to switch the work machine 40 as the operation target through the remote input interface 210 may be determined (see FIG. 4 / STEP 222). As a result, when an operator is changed from one operator to another operator, the one operator is prevented from designating the work machine 40 as the operation target to be remotely operated by the other operator, and the other operator can designate the work machine 40 as the operation target to be remotely operated by himself / herself according to his / her will.

[0161] When the condition that an operator change report has been made through the remote input interface 210 (the first flag f1 is set to "1") is satisfied (see FIG. 5 / STEP 240 -> STEP 241), the communication disconnection process may be executed to disconnect the communication established between the remote operation device 20 and the work machine 40 as the operation target designated by one operator before the change. This prevents a situation in which the other operator feels uncomfortable after the change due to the communication establishment state between the remote operation device 20 and the work machine 40 as the operation target designated by the one operator remains. For example, a captured image or a work environment image received by the remote operation device 20 from the work machine 40 as the operation target designated by the one operator is output to the remote image output device 221. Since the captured image or the work environment image is unnecessary information that is not related to the remote operation by the other operator himself, if the captured image or the work environment image continues to be output to the remote image output device 221, the other operator will feel uncomfortable. Here, when a change of operator is notified, the communication established between the remote control device 20 and the work machine 40 to be operated is cut off, and the captured image or work environment image is no longer output to the remote image output device 221, thereby preventing the other operator from feeling uncomfortable.

[0162] In the above embodiment, when the condition that an instruction to switch the work machine 40 as the operation target has been given through the remote input interface 210 is satisfied (see FIG. 4 / STEP 222-2 and FIG. 17 / Q82) and it is determined that an existing operation target exists (YES in FIG. 4 / STEP 208), the remote control device 200 starts to execute a communication disconnection process as an "operation preparation process" (FIG. 4 / STEP 212) in addition to the communication establishment process, but this is not limited to this. For example, after communication established with one work machine 40 (first work machine) is disconnected in response to an operation by the operator through the remote input interface 210, when the condition that an instruction to switch the work machine 40 as the operation target has been given through the remote input interface 210 by the operator is satisfied (see FIG. 4 / STEP 222-2 and FIG. 17 / Q82), the communication establishment process may be executed to establish communication with another work machine 40 (second work machine). In other words, the input interface is capable of further receiving a switching command for switching the operation target from the one operation target, and after receiving the switching command, receives the designation command for designating the other operation target as the operation target. This makes it possible to execute a process for establishing communication with a work machine 40 (first work machine) and a process for disconnecting communication with a work machine 40 (second work machine) in accordance with the will of the operator.

[0163] When work is being carried out using the work machine 40 as the object of operation (FIG. 4 / STEP 218), if the condition that an operator change has been reported from one operator to another operator via the remote input interface 210 is satisfied (FIG. 5 / STEP 240: YES → see STEP 241), the operation history of the work machine 40 as the object of operation may be acquired together with (in association with) an operator identifier that identifies the other operator (see FIG. 6 / STEP 288).

[0164] In the above embodiment, the operator change notification and the operation target switching instruction are input through the remote input interface 210, but one of them may be input through the terminal input interface 110 and transmitted from the terminal processing device 10 to the remote operation device 20. That is, the input interface in the present invention may have an operator switching input interface capable of receiving the operator change notification and an operation target switching input interface capable of receiving the operation target switching instruction. Other individual input interfaces may be used.

[0165] An operating device according to a first aspect of the present invention is an operating device for operating an operating object, and includes an operating mechanism that accepts operations by an operator, and a control device that acquires, in association with each other, an operation history, which is a history of operating the operating object through the operating mechanism, and an operator identifier for identifying the operator operating the operating mechanism.

[0166] According to the operation device having the above configuration, the operation history of the operation target can be acquired separately for each operator identifier, and further for each operator identified by the operator identifier.

[0167] An operating device according to a second aspect of the present invention is the operating device according to the first aspect, further comprising a communication device capable of communicating with the operating target, and it is preferable that the control device communicates with a predetermined operating target selected from a plurality of operating targets via the communication device, thereby enabling the predetermined operating target to be operated by the operating mechanism.

[0168] According to the operation device having the above configuration, even if a specific operation target can be selected and operated from among a plurality of operation targets, the operation history of the operation target can be obtained separately for each operator identifier, and ultimately for each operator identified by the operator identifier.

[0169] An operating device according to a third aspect of the present invention is the operating device according to the first or second aspect, wherein the control device is capable of switching the operation object from one object to another object in response to a switching instruction for switching the operation object, and when the operation object is switched from the one object to the other object, it is preferable to obtain the operation history of the other object in association with the operator identifier.

[0170] According to the operation device having the above configuration, when an operation target is switched from one operation target to another operation target, the operation history of the other operation target is acquired together with the operator identifier, whereby the operation history of each different operation target by the operator can be acquired separately.

[0171] In an operating device according to a fourth aspect of the present invention, in the operating device according to the third aspect, it is preferable that the control device determines whether or not there is an instruction to switch the operating object when a condition is satisfied that no work is being performed using the operating object.

[0172] According to the operation device having the above configuration, the presence or absence of an instruction to switch the operation object is determined on the condition that no work using the operation object is being performed. When work using one operation object is being performed, it is possible to avoid a situation in which the one operation object is switched to another operation object and the work is stopped against the operator's intention.

[0173] In an operating device according to a fifth aspect of the present invention, in the operating device according to any one of the first to fourth aspects, when the operator identifier changes from one operator identifier corresponding to one operator to another operator identifier corresponding to another operator, it is preferable that the control device acquires the operation history of the operation target in association with the other operator identifier.

[0174] According to the operation device having the above configuration, when an operator is replaced by another operator, the operation history of the operation target is acquired together with an operator identifier for identifying the other operator. This makes it possible to distinguish and acquire the operation history of the operation target for each different operator.

[0175] It is preferable that the operating device according to a sixth aspect of the present invention, in the operating device according to any one of the first to fifth aspects, further comprises an input unit capable of receiving at least one of an operator change notification from one operator to another operator and an instruction to switch the operating object from one object to another object.

[0176] According to the operation device having this configuration, when a notification of a change from one operator to another operator is received through the input unit (and when the operator is actually changed), the operation history of the operation object is acquired together with the operator identifier of the other operator. This allows the operation history of the operation object to be distinguished and acquired for each different operator. In addition to or instead of this, when an instruction to switch from one operation object to another operation object is received through the input unit (and when the operation object is actually switched), the operation history of the other operation object is acquired together with the operator identifier. This allows the operation history of each different operation object by the operator to be distinguished and acquired.

[0177] An operating device according to a seventh aspect of the present invention is preferably the operating device according to the sixth aspect, wherein the input unit has an operator switching input interface capable of accepting a change of operator notification, and an operation object switching input interface capable of accepting an instruction to switch the operation object.

[0178] According to the operation device having the above configuration, an operator change notification and an instruction to switch the operation target are received through the input unit. Therefore, the operation history of the operation target for each different operator can be distinguished and acquired, and / or the operation history of each different operation target by the operator can be distinguished and acquired.

[0179] An operating device according to an eighth aspect of the present invention is the operating device according to the sixth aspect, further comprising a communication device capable of communicating with the operating target, and the control device is capable of executing a communication establishment process for establishing communication with the operating target through the communication device and a communication disconnection process for disconnecting the communication established with the operating target, and when a condition that the operator has not reported a change of operator through the input unit is satisfied, it is preferable that the control device determines whether or not there is an instruction to switch the operating target through the input unit while maintaining the communication established with the one operating target.

[0180] According to the operation device having this configuration, on the condition that there is no change notification of the operator through the input unit, the presence or absence of an instruction to switch the operation target through the input unit is determined while maintaining the communication established with one operation target. Therefore, when the operation target designated as a new operation target is the same as the previous operation target, the communication already established between the operation device and the operation target is continuously used, and it is not necessary to execute the communication disconnection process and the communication establishment process again for the operation target, and the operation of the operation target can be smoothly resumed.

[0181] An operating device according to a ninth aspect of the present invention is the operating device according to the sixth aspect, further comprising a communication device capable of communicating with the operating target, and the control device is capable of executing a communication establishment process for establishing communication with the operating target through the communication device, and a communication disconnection process for disconnecting the communication established with the operating target, and it is preferable that the control device determines whether or not there is an instruction to switch the operating target through the input unit when a condition that a change of operator has been reported through the input unit and a condition that the acquired operator identifier has changed from one operator identifier to another operator identifier are each satisfied.

[0182] According to the operation device having this configuration, when an operator is switched from one operator to another operator, the operation target is prevented from being switched by the one operator, and the operation target can be switched according to the intention of the other operator.

[0183] An operating device according to a tenth aspect of the present invention is the operating device according to the ninth aspect, wherein the control device executes the communication disconnection process to disconnect the communication established with the operation target when a condition is satisfied that a change of operator has been notified through the input unit.

[0184] According to the operation device having the above configuration, when an operator is replaced by another operator, the communication established with the operation target is cut off. This realizes a state in which the operation target designated by the other operator does not exist, and prevents the other operator from feeling uncomfortable due to the communication establishment state remaining between the operation device and the operation target designated by the one operator.

[0185] An eleventh aspect of the present invention is an operating device according to the tenth aspect, wherein the control device executes the communication establishment process for establishing communication with the other operation target when a condition is satisfied that an instruction to switch the operation target is given via the input unit after communication established with the one operation target is disconnected.

[0186] According to the operation device having the above configuration, a switching instruction is given through the input unit according to the operator's intention, and the operation target after the actual switching can be operated by the operator. This makes it possible to obtain both the operator identifier of the operator and the operation history of the operation target after the switching.

[0187] An operating device according to a twelfth aspect of the present invention is the operating device according to any one of the first to eleventh aspects, and when a condition is satisfied that an operator change notification has been made from one operator to another operator via an input unit while work is being performed using the operation object, the control device preferably acquires an operation history of the operation object in association with the operator identifier identifying the other operator.

[0188] According to the operation device having the above configuration, when an operator is replaced while a task using an operation object is being performed, the operation history of the operation object is acquired together with the operator identifier of the new operator.

[0189] An operating device according to a thirteenth aspect of the present invention is an operating device according to any one of the first to twelfth aspects, wherein the control device executes an authentication process of the operator based on the acquired operator identifier, and if authentication of the operator fails, preferably prohibits or restricts the operator from transmitting the operation mode of the operating mechanism to the operation target.

[0190] According to the operation device having the above configuration, when an operator is replaced by another operator, if authentication of the other operator fails, the operation of the operation target by the other operator through the operation mechanism is prohibited or restricted, thereby obtaining the operation history of both the other operator and the operation target that was prohibited or restricted.

[0191] An operating device according to a 14th aspect of the present invention is an operating device according to any one of the first to 13th aspects, wherein the control device executes an authentication process of the operator based on the acquired operator identifier, and if the authentication of the operator fails, prohibits switching of the operation target, and if the authentication of the operator is successful, allows switching of the operation target.

[0192] According to the operation device having the above configuration, when an operator is replaced by another operator, if authentication of the other operator fails, switching of the operation target by a switching instruction from the other operator is prohibited, whereas if authentication of the other operator is successful, switching of the operation target by a switching instruction from the other operator is permitted.

[0193] What is provided by the present invention is an operation system, the operation system including an operation device according to any one of the first to fourteenth aspects and the operation target. [Explanation of symbols]

[0194] 10. Information processing terminal 100. Terminal control device 110...Terminal input interface 120...Terminal output interface 150. Terminal wireless communication equipment 20. Remote control device (operation device) 24 seats 200. Remote control device 210 Remote input interface 211. Remote control mechanism 220 Remote output interface 221 Remote image output device 222. Remote audio output device 224. Seat drive mechanism 250 Remote wireless communication equipment 40. Work machinery 44‥Working mechanism 400... Actual machine control device 410... Actual machine input interface 420... Actual machine output interface

Claims

1. An operation device for operating an operation object, an operation mechanism for receiving an operation by an operator; a control device that acquires an operation history, which is a history of operations of the operation target through the operation mechanism, and an operator identifier for identifying an operator operating the operation mechanism, in association with each other; An operating device comprising:

2. 2. The operating device according to claim 1, Further comprising a communication device capable of communicating with the operation target, The operating device according to claim 1 , wherein the control device communicates with a predetermined operation target selected from a plurality of operation targets via the communication device, thereby enabling the predetermined operation target to be operated by the operating mechanism.

3. 3. The operating device according to claim 1, The control device includes: The operation object is capable of being switched from one object to another object in response to a switching instruction for switching the operation object, When the operation target is switched from the one target to the other target, the operation history of the other target is acquired in association with the operator identifier.

4. 4. The operating device according to claim 3, The control device determines whether or not there is an instruction to switch the operation object when a condition that an operation using the operation object is not being performed is satisfied.

5. 3. The operating device according to claim 1, When the operator identifier changes from one operator identifier corresponding to one operator to another operator identifier corresponding to another operator, the control device acquires the operation history of the operation target in association with the other operator identifier.

6. 3. The operating device according to claim 1, An operation device further comprising an input unit capable of receiving at least one of an operator change notification from one operator to another operator and an instruction to switch the operation target from one target to another target.

7. 7. The operating device according to claim 6, The input unit includes: an operator switching input interface capable of receiving a change of operator notification; an operation object switching input interface capable of receiving an instruction to switch the operation object; An operating device having

8. 7. The operating device according to claim 6, Further comprising a communication device capable of communicating with the operation target, The control device includes: It is possible to execute a communication establishment process for establishing communication with the operation target through the communication device, and a communication disconnection process for disconnecting the communication established with the operation target, When a condition that the operator has not reported a change of operator through the input unit is satisfied, the operation device determines whether or not an instruction to switch the operation object has been issued through the input unit while maintaining communication established with the one operation object.

9. 7. The operating device according to claim 6, Further comprising a communication device capable of communicating with the operation target, The control device includes: It is possible to execute a communication establishment process for establishing communication with the operation target through the communication device, and a communication disconnection process for disconnecting the communication established with the operation target, an operation device that determines whether or not an instruction to switch the operation target has been made through the input unit when a condition that a change of operator has been reported through the input unit and a condition that the acquired operator identifier has changed from one operator identifier to another operator identifier are each satisfied.

10. 10. The operating device according to claim 9, The control device executes the communication disconnection process to disconnect the communication established with the operation target when a condition that a change of operator has been notified via the input unit is satisfied.

11. The operating device according to claim 10, The control device executes the communication establishment process to establish communication with the other operation target when a condition is satisfied that an instruction to switch the operation target has been given through the input unit after communication established with the one operation target has been disconnected.

12. 3. The operating device according to claim 1, The control device, when a condition is satisfied that one operator has notified another operator of a change of operator through an input unit while work is being performed using the operation object, acquires an operation history of the operation object in association with the operator identifier that identifies the other operator.

13. 3. The operating device according to claim 1, The control device executes an authentication process for the operator based on the acquired operator identifier, and if authentication of the operator fails, prohibits or restricts the operator from transmitting the operation mode of the operation mechanism to the operation target.

14. 3. The operating device according to claim 1, the control device executes an authentication process for the operator based on the acquired operator identifier, and if authentication of the operator fails, prohibits switching of the operation target, and if authentication of the operator succeeds, allows switching of the operation target.

15. 1. An operating system, comprising: An operating device according to any one of claims 1 to 14; The operation target; An operating system comprising:

Images