Remote operation system, remote operation device, management device, and remote operation assistance method

Abstract

This remote operation system (400) comprises a controller (10) and a storage device (202). The storage device (202) is configured to store, in association with each other, each of a plurality of operation targets (300) and a communication specification by which the controller (10) communicates with the operation targets (300). The controller (10) is configured to: recognize a specific operation target (301), which is an operation target to communicate with, among the plurality of operation targets (300); recognize, as a specific communication specification, a communication specification stored in a storage device (21) in association with the specific operation target (301); and communicate with the specific operation target (301) by using the specific communication specification.

Description

Remote operation system, remote operation device, management device, and remote operation support method

[0001] The present disclosure relates to a technology for remotely operating an operation target such as a working machine.

[0002] Conventionally, technologies for remotely operating a working machine are known (for example, Patent Documents 1-4).

[0003] When a plurality of working machines are candidates for a remote operation target (operation target), the controller of the remote operation system needs to communicate with each of the plurality of working machines. However, the communication specifications of the plurality of working machines may differ from each other.

[0004] Japanese Patent Application Laid-Open No. 2022-121138, Patent No. 6602808, Japanese Patent Publication No. 8-3192, Patent No. 4248725

[0005] An object of the present disclosure is to provide a technology that enables a controller to communicate with each of a plurality of operation targets even when the communication specifications of the plurality of operation targets differ from each other.

[0006] The remote operation system according to the first aspect includes a controller and a storage device. The storage device is configured to store in association with each of a plurality of operation targets a communication specification for the controller to communicate with that operation target. The controller recognizes a specific operation target, which is an operation target to be communicated with among the plurality of operation targets, and recognizes the communication specification stored in the storage device in association with the specific operation target as a specific communication specification, and is configured to communicate with the specific operation target using the specific communication specification.

[0007] This is a block diagram of a remote control system according to an embodiment. This is a perspective view showing the remote control device of the remote control system. This is a perspective view showing a work machine, which is an example of an object to be controlled. This is a block diagram showing the remote control device, the work machine, and a server (an example of a management device). This is a diagram showing an example of data stored in the storage device of the remote control system. This is a flowchart showing the flow of registering an object to be controlled in a database. This is a flowchart showing an example of calculation processing performed by the controller of the remote control system. This is a diagram showing an example of data stored in the storage device of the remote control system. This is a diagram showing an example of data signal format conversion. This is a flowchart showing an example of communication establishment processing. This is a flowchart showing an example of startup processing. This is a flowchart showing another example of startup processing. This is a diagram showing an example of the correspondence between control amount instructions and operation signals.

[0008] A remote control system, remote control device, management device, and remote control support method according to the embodiments of this disclosure will be described with reference to the drawings.

[0009] [Remote Control System] Figure 1 is a block diagram showing a remote control system 400 according to an embodiment. This remote control system 400 comprises a remote control device 100 (cockpit), a server 200, and a plurality of controllable objects 300 (300A, 300B, 300C). In this embodiment, each of the plurality of controllable objects 300 is a work machine that performs various tasks at a work site.

[0010] The remote control device 100 is configured to communicate with each of the multiple control targets 300 via the network 500 by performing a communication establishment process described later. The remote control device 100 is also configured to communicate with the server 200 via the network 500. Furthermore, each of the multiple control targets 300 may be configured to communicate with the server 200 via the network 500.

[0011] Network 500 is an information communication network such as the Internet, telephone lines, mobile phone lines, satellite communication networks, WAN (Wide Area Network), LAN (Local Area Network), or dedicated lines. Network 500 may be any of these communication networks, or it may be composed of a combination of multiple types of communication networks. Network 500 may include a communication network that uses technology to construct a virtual dedicated line, such as VPN (Virtual Private Network). The LAN may be a wireless LAN or a wired LAN.

[0012] In the specific example shown in Figure 4, the remote control device 100 (master) is configured to communicate with each of the multiple control targets 300 (slaves) using a wireless LAN, and is also configured to communicate with the server 200 using a wireless LAN.

[0013] [Remote Control Device] Figure 2 is a perspective view showing the remote control device 100. The remote control device 100 is a device for remotely controlling a specific target 301, which is a target 300 selected from a plurality of target 300s. By performing a communication establishment process described later, the remote control device 100 becomes able to communicate with the specific target 301 included in the plurality of target 300s via the network 500. After communication between the remote control device 100 and the specific target 301 is established, the operator can remotely control the specific target 301 using the remote control device 100. In this embodiment, Figure 1 illustrates the case in which a target 300B (work machine) is selected as the specific target 301 from a plurality of target 300s.

[0014] The remote control device 100 may include a controller 10, an operation input device 11, a video decoder 12, an audio decoder 13, a video output device 14, a sound output device 15, a designated operation input device 16, a face recognition device 17, and a communication device 18 for the operation device.

[0015] The controller 10 performs communication establishment processing and startup processing, which will be described later. The controller 10 performs various calculation processes for remotely controlling the specific target 301. The controller 10 includes a computer equipped with an arithmetic processing unit and memory. The controller 10 has a communication establishment processing unit 101, a startup processing unit 102, and an operation command unit 103. The communication establishment processing unit 101 performs the communication establishment processing. The startup processing unit 102 performs the startup processing. The operation command unit 103 transmits operation commands to the specific target 301 in response to operation input to the operation input device 11. The respective functions of the communication establishment processing unit 101, the startup processing unit 102, and the operation command unit 103 are realized by the arithmetic processing unit executing a control program stored in the memory.

[0016] The operation input device 11 is a device that receives operation inputs made by the operator of the remote control device 100. The operation input device 11 includes a travel operation input device, a slewing operation input device, and an attachment operation input device. Each of these input devices may include an operation lever or an operation pedal. The travel operation input device may include at least one of an operation lever and an operation pedal that receives a travel operation input, which is an operation input for traveling the lower traveling body of the specific operating target 301. The slewing operation input device may include an operation lever that receives a slewing operation input, which is an operation input for slewing the upper slewing body of the specific operating target 301 relative to the lower traveling body. The attachment operation input device may include a plurality of operation levers that receive attachment operation inputs, which are operation inputs for operating an attachment (working device) including a boom, arm, and tip attachment of the specific operating target 301.

[0017] An electrical signal (voltage or current) corresponding to the magnitude of the operation input given to the operation input device 11 is input to the controller 10 (for example, the second controller 10B described later). Specifically, when the travel operation input device receives a travel operation input from an operator, an electrical signal corresponding to the travel operation input is input to the second controller 10B. When the turning operation input device receives a turning operation input from an operator, an electrical signal corresponding to the turning operation input is input to the second controller 10B. When the attachment operation input device receives an attachment operation input from an operator, an electrical signal corresponding to the attachment operation input is input to the second controller 10B. The electrical signals as travel operation inputs, the electrical signals as turning operation inputs, and the electrical signals as attachment operation inputs are input to different input ports on the second controller 10B, for example, so that the second controller 10B can distinguish and recognize the travel operation inputs, the turning operation inputs, and the attachment operation inputs.

[0018] The remote control device 100 has a seat 19 for the operator to sit on. The seat 19 includes a seat portion 19A and a backrest portion 19B. The operation input device 11 is located near the seat 19. An information processing terminal (not shown) may be located in front of or to the side of the seat 19. This information processing terminal may have a touch panel for receiving operation input from the operator. The remote control device 100 may have a shut-off lever 20. The shut-off lever 20 is displaceable between a locked position that prohibits the operation of the target device 300 and an unlocked position that allows the operation of the target device 300. The shut-off lever 20 is displaced between the locked position and the unlocked position by operation by the operator.

[0019] As shown in Figure 2, the operation input device 11 may include, for example, a travel lever and travel pedal positioned in front of the seat 19, a left operation lever positioned to the left of the seat 19, and a right operation lever positioned to the right of the seat 19. For example, the left operation lever may function as an arm operation input device for operating the arm of the specific operating target 301 when operated forward or backward, and as a slewing operation input device for slewing the upper slewing body of the specific operating target 301 relative to the lower traveling body when operated left or right. The right operation lever may function as a boom operation input device for operating the boom of the specific operating target 301 when operated forward or backward, and as a tip attachment operation input device for operating the tip attachment of the specific operating target 301 when operated left or right. This operation pattern may be changed by an operation pattern selection process described later.

[0020] The video decoder 12 decodes the video data encoded by the video encoder 32 (described later) in the specific target device 301, and inputs the decoded video data to the video output device 14. Specifically, the video data is, for example, video data captured by an imaging device (an example of an information acquisition device 34), such as a camera (described later), for photographing the work site where the specific target device 301 is located. The imaging device may be mounted on the specific target device 301, or it may be located at a distance from the specific target device 301 at the work site where the specific target device 301 is located. The video encoder 32 and the imaging device may be separate devices, or the imaging device may have the functions of the video encoder 32. Specifically, if the imaging device is located at a distance from the specific target device 301 at the work site, the video data may be encoded by the video encoder provided in the imaging device. The video data may be transmitted from the imaging device to the remote control device 100, or it may be input from the imaging device to the specific target device 301 and then transmitted from the specific target device 301 to the remote control device 100.

[0021] The audio decoder 13 decodes the sound data encoded by the audio encoder 33 (described later) in the specific operation target 301, and inputs the decoded sound data to the sound output device 15. Specifically, the sound data is, for example, sound data collected by a sound acquisition device (an example of an information acquisition device 34), such as a microphone, that collects sounds occurring at the work site where the specific operation target 301 is located. The sound acquisition device may be mounted on the specific operation target 301, or it may be located at a location away from the specific operation target 301 at the work site where the specific operation target 301 is located. The audio encoder 33 and the sound acquisition device may be separate devices, or the sound acquisition device may have the functions of the audio encoder 33. Specifically, if the sound acquisition device is located at a location away from the specific operation target 301 at the work site, the sound data may be encoded by the audio encoder provided in the sound acquisition device. The sound data may be transmitted from the sound acquisition device to the remote control device 100, or it may be input from the sound acquisition device to the specific target of operation 301 and transmitted from the specific target of operation 301 to the remote control device 100.

[0022] The video output device 14 is configured to output video of the work site captured by a camera (imaging device) mounted on the specific operation target 301 or a camera (imaging device) placed at the work site, and transmitted from the specific operation target 301 or the imaging device placed at the work site. In this embodiment, the video output device 14 displays video corresponding to video data input from the video decoder 12, specifically, video corresponding to video data transmitted from the specific operation target 301 or the imaging device and decoded by the video decoder 12. That is, the video output device 14 may be configured to display video corresponding to video data transmitted from the specific operation target 301, or video corresponding to video data transmitted from an imaging device placed at the work site. The video output device 14 may include one or more displays. In the specific example shown in Figure 2, the video output device 14 includes a central display positioned in front of the seat 19, a left display positioned diagonally to the left front of the seat 19, and a right display positioned diagonally to the right front of the seat 19.

[0023] The sound output device 15 is configured to output sounds from the work site that are collected by a microphone (sound acquisition device) mounted on the specific operating target 301 or a microphone (sound acquisition device) placed at the work site, and transmitted from the specific operating target 301 or the sound acquisition device placed at the work site. In this embodiment, the sound output device 15 outputs sounds corresponding to sound data input from the audio decoder 13, specifically, sounds corresponding to sound data transmitted from the specific operating target 301 or the sound acquisition device and decoded by the audio decoder 13. That is, the sound output device 15 may be configured to output sounds corresponding to sound data transmitted from the specific operating target 301, or sounds corresponding to sound data transmitted from a sound acquisition device placed at the work site. The sound output device 15 is composed of one or more speakers. The sound output device 15 may include, for example, a speaker located behind the seat 19, a speaker located behind the left armrest, and a speaker located behind the right armrest.

[0024] The remote control device 100 may have a seat drive device, which is not shown in the figure. The seat drive device may be configured to tilt or swing the seat 19 relative to the seat base 21 or integrally with the seat base 21.

[0025] The designated operation input device 16 is a device configured to receive designated operations from personnel involved in the work. Specifically, for example, the designated operation input device 16 may be composed of a touch panel terminal equipped with a display function and an input function, in which case it may be configured to display various information on the display surface of the designated operation input device 16 and to receive designated operations from personnel involved in the work via the display surface. The designated operation is an input operation for personnel involved in the work to specify a desired operation target 300 from among a plurality of operation targets 300, and is given to the designated operation input device 16 by the personnel involved in the work. Persons involved in the work may be an operator who operates the remote control device 100, a manager (supervisor) who manages the work performed by the operation target 300 by remote control using the remote control device 100, an assistant who assists in the work, or other personnel involved in the work.

[0026] The designated operation input device 16 is, for example, located on the right or left side of the sheet 19 and receives designated operations via a display surface from work personnel such as an operator operating the remote control device 100. The designated operation input device 16 displays various information on its display surface based on display signals input from the controller 10 (for example, the third controller 10C described later) and inputs designated operation signals representing the designated information related to the designated operation received via the display surface to the third controller 10C. The designated operation input device 16 may be configured as a terminal provided separately from the remote control device 100 and capable of communicating with the remote control device 100. In this case, the designated operation input device 16 can receive designated operations from work personnel such as a manager (supervisor) or assistant. The designated operation input device 16 includes a computer equipped with an arithmetic processing unit and memory. The functions of the designated operation input device 16 are realized by the arithmetic processing unit executing a control program stored in the memory.

[0027] The face recognition device 17 may include a face recognition sensor. The face recognition sensor may include, for example, a camera and / or a distance sensor positioned in front of the seat 19. The face recognition device 17 performs authentication processing of the operator based on the image data of the operator's face acquired by the camera and / or data on the three-dimensional shape of the operator's face acquired by the distance sensor. The remote control device 100 does not necessarily have to be equipped with the face recognition device 17.

[0028] The control device communication device 18 is a device for communicating with a specific target device 301 via the network 500. The control device communication device 18 is assigned destination information that identifies the control device communication device 18 on the network 500 and specifies the control device communication device 18 as the destination. The destination information may be, for example, an IP address.

[0029] The communication device 18 for the operating device may be, for example, a router. The communication device 18 for the operating device may be, for example, an edge router located at the boundary between the remote control device 100 and the external network 500. The communication device 18 for the operating device may be assigned an IP address (global IP address or local IP address), and the communication device 18 for the operating device may be configured to receive data signals sent from the server 200 or specific target 301 (described later) to the IP address (global IP address or local IP address) of the communication device 18 for the operating device.

[0030] The controller 10 may consist of a single controller. In the specific example shown in Figure 4, the controller 10 consists of a plurality of controllers that are connected to each other in a communicative manner via a concentrator (HUB). Specifically, the controller 10 shown in Figure 4 includes a first controller 10A (communication controller 10A), a second controller 10B (controller for operation function control 10B), and a third controller 10C (controller for non-operation function control 10C).

[0031] Furthermore, the communication device 18 for the operating device may include a fourth controller 10D (communication device controller 10D). In this case, the controller 10 may further include this fourth controller 10D. The communication device controller 10D controls the operation of the communication device 18 for the operating device.

[0032] The communication device 18 for the operating device controls communication between the remote control device 100 and the server 200, and controls communication between the remote control device 100 and the target device 300 (specific target device 301). The communication device controller 10D of the communication device 18 for the operating device includes a computer equipped with an arithmetic processing unit and memory. The functions of the communication device controller 10D are realized by the arithmetic processing unit executing a control program stored in the memory.

[0033] The second controller 10B generates an operation signal representing an operation amount corresponding to the magnitude of the electrical signal (voltage or current) input from the operation input device 11, and inputs the generated operation signal to the first controller 10A. Specifically, the second controller 10B recognizes, based on the electrical signal input to the second controller 10B, whether the electrical signal corresponds to the driving operation input, the turning operation input, or the attachment operation input. That is, the second controller 10B recognizes the type of operation input given to the operation input device 11 based on the electrical signal. The second controller 10B then generates an operation signal in which information regarding the type of operation input is added to information regarding the operation amount corresponding to the magnitude of the electrical signal, and inputs this operation signal to the first controller 10A. The second controller 10B includes a computer equipped with an arithmetic processing unit and memory. The functions of the second controller 10B are realized by the arithmetic processing unit executing a control program stored in the memory.

[0034] The first controller 10A generates an operation command based on an operation signal input from the second controller 10B and transmits the generated operation command to the specific target 301 via the communication device 18 for the operation device. Specifically, as will be described later, the first controller 10A converts the format of the data signal representing the operation command to be transmitted to the specific target 301 (second target 300B) into a specific format, and transmits the data signal converted to the specific format to the communication device 36 for the operation device, which will be described later and is located on the specific target 301. The first controller 10A includes a computer equipped with an arithmetic processing unit and memory. The functions of the first controller 10A are realized by the arithmetic processing unit executing a control program stored in the memory.

[0035] The third controller 10C generates a display signal to instruct the display content on the display surface of the designated operation input device 16, and inputs the generated display signal to the designated operation input device 16. Specifically, the third controller 10C generates a display signal to instruct the display content on the display surface of the designated operation input device 16 based on information such as information obtained from the first controller 10A, information obtained from the second controller 10B, information related to the operation device communication device 18, information related to the state of the specific operation target 301 obtained via the operation device communication device 18, and information provided by the server 200, and inputs the generated display signal to the designated operation input device 16.

[0036] Furthermore, the third controller 10C outputs a command signal to at least one of the first controller 10A, the second controller 10B, and the communication device for the operating device 18 based on the designated operation signal input from the designated operation input device 16, that is, the designated operation signal representing the designated information regarding the designated operation received by the designated operation input device 16.

[0037] Furthermore, the third controller 10C may acquire image data of the operator's face and / or data relating to the three-dimensional shape of the operator's face from the face recognition device 17 and perform an authentication process to determine whether the operator is a pre-registered legitimate operator.

[0038] Furthermore, the third controller 10C performs controls necessary for the remote control device 100, other than those performed by the first controller 10A and the second controller 10B. The third controller 10C includes a computer equipped with an arithmetic processing unit and memory. The functions of the third controller 10C are realized by the arithmetic processing unit executing a control program stored in the memory.

[0039] [Management Device] Server 200 (an example of a management device) comprises a server controller 201 and a storage device 202. The server controller 201 controls the operation of server 200.

[0040] The storage device 202 is configured to store each of the multiple target devices 300 in association with the communication specifications for which the controller 10 communicates with the target device 300. The server controller 201 receives a specification request signal transmitted from the remote control device 100 to the server 200 and transmits a response signal representing the communication specifications corresponding to the specification request signal to the remote control device 100.

[0041] The operating configuration of server 200 may be on-premises, cloud-based, or hybrid cloud-based. On-premises configurations may involve server 200 being operated and managed on equipment owned by construction-related businesses, such as businesses involved in the operation of the target system or businesses involved in construction work. Cloud-based configurations may involve server 200 being operated and managed on equipment owned by a cloud service provider. Hybrid cloud configurations may involve connecting on-premises servers and cloud servers via a communication network, such as a VPN. The cloud platform provided by the cloud service provider may be, for example, Microsoft's Azure®, Amazon's AWS®, Google Cloud®, or other platforms.

[0042] [Operation Targets] As shown in FIG. 1, the plurality of operation targets 300 include, for example, a first operation target 300A, a second operation target 300B, and a third operation target 300C. In the present embodiment, each of the plurality of operation targets 300 is a work machine such as an excavator, a bulldozer, or a crane.

[0043] FIG. 3 is a perspective view showing an example of the operation target 300. The operation target 300 shown in FIG. 3 is an excavator. The operation target 300 includes a lower traveling body 320, an upper revolving body 302 supported by the lower traveling body 320 so as to be rotatable about a vertical rotation axis, and a working device 303 supported by the upper revolving body 302. The lower traveling body 320 includes a lower frame and a traveling device that supports the lower frame. The traveling device includes, for example, a pair of left and right crawlers and a schematic traveling motor that drives these crawlers. The upper revolving body 302 includes a revolving frame supported by the lower frame and a cab 304 supported by the revolving frame.

[0044] The working device 303 includes a boom 305 rotatably supported by the upper revolving body 302, an arm 306 rotatably supported at the tip of the boom 305, and a tip attachment 307 rotatably supported at the tip of the arm 306. In the specific example shown in FIG. 3, the tip attachment 307 is a bucket, but the tip attachment 307 is not limited to a bucket and may be other tip attachments such as a crusher or a fork.

[0045] The operation target 300 further includes a plurality of actuators and a drive mechanism 311 that drives the plurality of actuators. The plurality of actuators include a boom cylinder 308 for rotating the boom 305 with respect to the upper revolving body 302, an arm cylinder 309 for rotating the arm 306 with respect to the boom 305, a tip attachment cylinder 310 for rotating the tip attachment 307 with respect to the arm 306, a schematic swing motor for swinging the upper revolving body 302 with respect to the lower traveling body 320, and the traveling motor.

[0046] Each of the plurality of actuators may be a hydraulic actuator or an electric actuator. When the plurality of actuators includes a hydraulic actuator, the drive mechanism 311 includes a hydraulic circuit including a hydraulic pump that is driven by a prime mover (drive source) such as an engine to discharge hydraulic oil, and a control valve. When the plurality of actuators includes an electric actuator, the drive mechanism 311 includes an electric circuit including a battery.

[0047] When the operator gives an operation input to the operation input device 11, the remote operation device 100 (operation command unit 103) transmits an operation command corresponding to the operation input to the operation target 300 (specific operation target 301). The specific operation target 301 receives the operation command in the operation target communication device 36 described later, and the drive mechanism 311 drives the actuator indicated in the operation command according to the operation command. As a result, the operation target 300 performs an operation corresponding to the operation input by the operator.

[0048] As shown in FIG. 1, each of the plurality of operation targets 300 includes an operation target controller 30, an operation input device 31, a video encoder 32, an audio encoder 33, an information acquisition device 34, a changeover switch 35, and an operation target communication device 36.

[0049] The operation target controller 30 controls the operation of the operation target 300. The operation target controller 30 includes a computer including an arithmetic processing unit and a memory. The operation target controller 30 performs, for example, control for transmitting the information acquired by the information acquisition device 34 to the remote operation device 100, control of the operation of the actuator based on the data signal received from the remote operation device 100, and the like. The function of the operation target controller 30 is realized by the arithmetic processing unit executing a control program stored in the memory.

[0050] The operation input device 31 is used when an operator is on board the object to be operated 300 and operates the object to be operated 300. The operation input device 31 includes a travel operation input device, a turning operation input device, and an attachment operation input device. Each of these input devices may include an operation lever or an operation pedal.

[0051] The information acquisition device 34 includes a detector that detects surrounding information, which is information about the area around the target device 300 at the work site. The surrounding information detected by the information acquisition device 34 is transmitted via the network 500 by the target device communication device 36, and the remote control device 100 receives the surrounding information.

[0052] As shown in Figure 3, the information acquisition device 34 includes an imaging device such as a camera that acquires video of the work site where the object to be operated 300 is located. The imaging device may be attached to a predetermined part of the object to be operated 300, for example. The predetermined part may be a position close to the eye level of the operator when the operator is seated in the cabin 304, as shown in Figure 3. The imaging device may be placed at a location away from the object to be operated 300 in the work site where the object to be operated 300 is located.

[0053] The information acquisition device 34 may include a sound acquisition device such as a microphone, which is a device that converts sounds from the work site where the object to be operated 300 is located into electrical signals. The sound acquisition device may be attached to the object to be operated 300, or it may be placed at a location away from the object to be operated 300 in the work site where the object to be operated 300 is located.

[0054] The information acquisition device 34 may include an object information detector (not shown) capable of acquiring information about objects present at the work site. The object information detector may be, for example, a three-dimensional information acquirer capable of acquiring three-dimensional information such as point cloud data of the work site, or an object detector such as an infrared sensor capable of detecting objects present at the work site. The three-dimensional information acquirer may be, for example, a LiDAR (Light Detection and Ranging). The object information detector may be placed at a predetermined location on the object to be operated 300 and configured to acquire information about objects present at the work site around the object to be operated 300, or it may be placed at a location away from the object to be operated 300 at the work site where the object to be operated 300 is located.

[0055] The video encoder 32 is a device that encodes video data, which is video data acquired by the imaging device of the information acquisition device 34. The controlled controller 30 transmits the encoded video data to the remote control device 100 via the network 500.

[0056] The audio encoder 33 is a device that encodes sound data, which is sound data acquired by the sound acquisition device of the information acquisition device 34. The controlled controller 30 transmits the encoded sound data to the remote control device 100 via the network 500.

[0057] The communication device 36 for the target device is a device for communicating with the remote control device 100 via the network 500. The communication device 36 for the target device is assigned destination information that identifies the communication device 36 on the network 500 and specifies the communication device 36 as the destination. The destination information may be, for example, an IP address.

[0058] The communication device 36 for the target of operation may be, for example, a router. The communication device 36 for the target of operation may be, for example, an edge router located at the boundary between the target of operation 300 and the external network 500. The communication device 36 for the target of operation may be assigned an IP address (global IP address or local IP address), and the communication device 36 for the target of operation may be configured to receive data signals sent from the remote control device 100 or server 200 to the global IP address or local IP address of the communication device 36 for the target of operation.

[0059] The operating target 300 may be equipped with a state detection device 37 (see Figure 4) which includes at least one sensor for detecting the state of the operating target 300. The state detection device 37 may include a posture detection device which includes at least one of the following: a boom posture sensor for detecting the posture of the boom 305, an arm posture sensor for detecting the posture of the arm 306, a tip attachment posture sensor for detecting the posture of the tip attachment, and a slewing body posture sensor for detecting the posture of the upper slewing body 302. The state detection device 37 may also include a tilt detection device which is positioned on the upper slewing body 302 of the operating target 300 (e.g., the cabin 304) for detecting the tilt of the upper slewing body 302. The state detection device 37 or the operating target 300 transmits information about the state of the operating target 300, including information such as the detected posture of the operating target 300 and the detected tilt of the upper slewing body 302, to the remote control device 100 via the network 500.

[0060] The changeover switch 35 is a switch for switching the control mode of the target device 300. The changeover switch 35 can switch between a remote control mode in which the target device 300 can be remotely controlled by the remote control device 100, and a live operation mode in which the target device 300 can be controlled by an operator who is in the cabin 304 of the target device 300. When the changeover switch 35 is set to live operation mode, an operator can board the target device 300 and operate it. When the changeover switch 35 is set to remote control mode, an operator can remotely control the target device 300 by providing an operation input to the operation input device 11 of the remote control device 100.

[0061] The controller to be operated 30 may consist of a single controller. In the specific example shown in Figure 4, the controller to be operated 30 consists of a plurality of controllers that are connected to each other in a communicative device (HUB). Specifically, the controller to be operated 30 shown in Figure 4 includes a first controller 30A (communication controller 30A), a second controller 30B, a third controller 30C (main controller 30C), and a fourth controller 30D (IT controller 30D).

[0062] The first controller 30A controls communication between the target device 300 and the remote control device 100. The first controller 30A recognizes the operation commands contained in the data signals transmitted from the remote control device 100 and received by the communication device 36 for the target device, based on the format applied to the target device 300, and outputs an operation signal corresponding to the recognized operation command to the second controller 30B. The first controller 30A also transmits a data signal representing information about the state of the target device 300, which is output from the second controller 30B and detected by the state detection device 37, to the communication device 18 for the control device. The first controller 30A includes a computer equipped with an arithmetic processing unit and memory. The functions of the first controller 30A are realized by the arithmetic processing unit executing a control program stored in the memory.

[0063] The second controller 30B controls the operation of the drive mechanism 311 so that it operates in response to an operation signal output from the first controller 30A. The second controller 30B performs control to operate the drive mechanism 311 in response to an operation signal when the control mode of the target device 300 is set to the remote control mode. The second controller 30B inputs information regarding the state of the target device 300 output from the state detection device 37 to the first controller 30A. If there is an abnormality in the information regarding the state of the target device 300 output from the state detection device 37, the second controller 30B interrupts the control to operate the drive mechanism 311 in response to the operation signal. The second controller 30B includes a computer equipped with an arithmetic processing unit and memory. The functions of the second controller 30B are realized by the arithmetic processing unit executing a control program stored in the memory.

[0064] The third controller 30C controls the operation of the operating object 300 regardless of the control mode when the operating object 300 is in operation. The third controller 30C performs various controls, such as controlling the prime mover, such as the engine, of the operating object 300, controlling the hydraulic and / or electrical circuits of the operating object 300, and controlling the display of a display device installed in the cabin 304 of the operating object 300. The third controller 30C includes a computer equipped with an arithmetic processing unit and memory. The functions of the third controller 30C are realized by the arithmetic processing unit executing a control program stored in the memory.

[0065] The fourth controller 30D controls communication between the target device 300 and the server 200. The fourth controller 30D includes a computer equipped with an arithmetic processing unit and memory. The functions of the fourth controller 30D are realized by the arithmetic processing unit executing a control program stored in the memory. The fourth controller 30D may further include a storage device.

[0066] The features of the remote control system 400 and the remote control support method according to this embodiment will be described in detail below.

[0067] In the remote control system 400 according to this embodiment, the storage device 202 is configured to store each of the multiple target devices 300 in association with the communication specifications for which the controller 10 communicates with the target device 300. The controller 10 of the remote control device 100 recognizes a specific target device 301 that should communicate with the multiple target devices 300, and sends a specification request signal to the server 200 requesting the communication specifications for the specific target device 301. The server 200 receives the specification request signal sent from the remote control device 100. The server controller 201 sends a response signal to the remote control device 100 corresponding to the specification request signal, that is, a response signal including data representing the communication specifications stored in the storage device 202 in association with the specific target device 301. The remote control device 100 receives the response signal, and the controller 10 of the remote control device 100 recognizes the communication specifications represented by the response signal as specific communication specifications, and is configured to communicate with the specific target device 301 using the specific communication specifications.

[0068] In this embodiment, the storage device 202 is configured to store the communication specifications associated with each of the multiple target devices 300. Therefore, the controller 10 can recognize the specific communication specifications, which are the communication specifications for a specific target device 301 that the controller 10 should communicate with, provided by the server 200, and can communicate with the specific target device 301 using the recognized specific communication specifications. Consequently, even if the communication specifications of the multiple target devices 300 are different from each other, the controller 10 can communicate with each target device 300 using the communication specifications that are appropriate for each of the multiple target devices 300.

[0069] Furthermore, in this embodiment, the communication specifications for each of the multiple target devices 300 include a communication establishment specification for the controller 10 to establish communication with that target device 300, and the controller 10 is configured to perform a communication establishment process to establish communication with a specific target device 301 using the communication establishment specification included in the specific communication specification. The controller 10 can establish communication with each of the multiple target devices 300 using a communication establishment specification that is suitable for each of the multiple target devices 300. In this embodiment, the communication specifications include a communication establishment specification and format information, and the communication establishment specification includes a communication method for the target device 300 and destination information for the target device 300. The communication method will be described later.

[0070] Furthermore, in this embodiment, the storage device 202 is configured to store in association with each of the multiple target devices 300 and the start specification for the controller 10 to start that target device 300. The server controller 201 of the server 200 transmits a response signal to the remote control device 100 in response to the specification request signal transmitted from the remote control device 100, that is, a response signal including data representing the start specification stored in the storage device 202 in association with a specific target device 301. In other words, the server controller 201 transmits a response signal to the remote control device 100 that includes not only data representing the communication specification, including the communication establishment specification, but also data representing the start specification for starting the specific target device 301. The remote control device 100 receives the response signal, and the controller 10 of the remote control device 100 recognizes the start specification represented by the response signal as a specific start specification. After communication between the controller 10 and the specific target device 301 is established, the controller 10 executes a start process, which is a process for starting the specific target device 301, using the specific start specification. In this embodiment, even if the starting specifications of multiple target devices 300 differ from one another, the controller 10 can start each of the multiple target devices 300 using a starting specification that is suitable for each of the multiple target devices 300 after establishing communication with a specific target device 301.

[0071] Furthermore, in this embodiment, as described above, the communication specifications of each of the multiple target devices 300 include format information. This format information is information regarding the format of the data signal for which the controller 10 transmits the data signal to the target device 300, and is stored in the storage device 202 in association with the target device 300. The controller 10 recognizes the format stored in the storage device 202 in association with a specific target device 301 as a specific format based on the response signal transmitted from the server 200. The controller 10 is configured to convert the format of the data signal to be transmitted to the specific target device 301 into the specific format and transmit the converted data signal to the specific target device 301. In this embodiment, even if the data signal formats of the multiple target devices 300 are different from each other, each of the multiple target devices 300 can receive the data signal that has been converted to the specific format by the controller 10 and transmitted from the controller 10. Therefore, since the specific target device 301 itself does not need to change the format of the data signal, the specific target device 301 can use the received data signal to control its operation without changing its format. This eliminates the need for each of the multiple targets 300 to add equipment or modify equipment to change the format, which is advantageous in terms of cost.

[0072] Figure 5 shows an example of data stored in the storage device 202. The storage device 202 pre-stores data related to specifications, such as those shown in Figure 5. That is, as shown in Figure 5, the storage device 202 stores the communication specifications for the controller 10 to communicate with each of the first target 300A, the second target 300B, and the third target 300C in association with each of them. In the specific example shown in Figure 5, the communication specifications include communication establishment specifications and format information related to the format of the data signal. The storage device 202 also stores the startup specifications for the controller 10 to start each of the first target 300A, the second target 300B, and the third target 300C in association with each of them.

[0073] Specifically, as shown in Figure 5, the storage device 202 stores the communication establishment specification A1, the data signal format A2, and the startup specification A3 in association with the first operation target 300A, the communication establishment specification B1, the data signal format B2, and the startup specification B3 in association with the second operation target 300B, and the communication establishment specification C1, the data signal format C2, and the startup specification C3 in association with the third operation target 300C.

[0074] In this embodiment, communication establishment specification A1 may include, for example, information relating to the communication method of the first target 300A and destination information of the first target 300A; communication establishment specification B1 may include, for example, information relating to the communication method of the second target 300B and destination information of the second target 300B; and communication establishment specification C1 may include, for example, information relating to the communication method of the third target 300C and destination information of the third target 300C.

[0075] Furthermore, in this embodiment, the startup specification A3 may include, for example, at least one of the following: equipment startup processing related information for the first operating target 300A, operation pattern selection processing related information for the first operating target 300A, calibration processing related information for the first operating target 300A, and initial operation information for the first operating target 300A. The startup specification B3 may include, for example, at least one of the following: equipment startup processing related information for the second operating target 300B, operation pattern selection processing related information for the second operating target 300B, calibration processing related information for the second operating target 300B, and initial operation information for the second operating target 300B. The startup specification C3 may include, for example, at least one of the following: equipment startup processing related information for the third operating target 300C, operation pattern selection processing related information for the third operating target 300C, calibration processing related information for the third operating target 300C, and initial operation information for the third operating target 300C. This information will be described later.

[0076] Figure 6 is a flowchart showing the process of registering the target device 300 in the database of the storage device 202. As shown in Figure 6, in step S31, for example, a service technician manually modifies the target device 300. Modification of the target device 300 may include, for example, the installation of equipment such as the first controller 30A, second controller 30B, fourth controller 30D, information acquisition device 34, changeover switch 35, communication device 36, and concentrator (HUB) as shown in Figure 4. Modification of the target device 300 may also include the installation of a control program for remotely operating the target device 300 using the remote control device 100. Some of the equipment may be the same equipment that was installed before the modification of the target device 300. For example, a camera and microphone as the information acquisition device 34 may be newly installed on the target device 300 by a service technician, while equipment other than the information acquisition device 34 may continue to be the same equipment that was installed before the modification of the target device 300.

[0077] In step S32, data of the modified target device 300 is provisionally registered in the database of the storage device 202. This provisional registration includes data related to the specifications shown in Figure 5. The provisional registration of data related to the specifications of the target device 300 is performed, for example, by a service technician. Specifically, the service technician may provisionally register the data related to the specifications of the target device 300 by operating an information terminal, such as a portable information terminal that can communicate with the server 200. If equipment that was installed before the modification of the target device 300 (existing equipment) is still in use after the modification of the target device 300, the data related to the specifications may be provisionally registered taking into account such existing equipment. For example, if the communication device 36 of the target device 300 was installed before the modification of the target device 300, communication specifications such as communication establishment specifications and data signal format are provisionally registered based on the specifications of the communication device 36. The specifications of equipment installed before the modification of the target device 300 vary depending on the target device 300 and the manufacturer of the equipment. However, based on the operation of the service technician, data regarding the specifications is provisionally registered, and the storage device 202 provisionally registers data regarding the specifications that differ for each target device 300. The data provisionally registered in the storage device 202 may be data listed in the equipment catalog, or it may be equipment data obtained through prior testing. Alternatively, data corresponding to the target equipment may be provisionally registered in the storage device 202 based on the operation of the service technician who confirmed that the target equipment is installed in the modified target device 300.

[0078] In step S33, the operation of the target device 300 is checked. If the operation check is successful, in step S34, data relating to the specifications of the target device 300 is officially registered in the database of the storage device 202. The operation check of the target device 300 may be performed, for example, by actually remotely controlling the target device 300 using the remote control device 100. In this case, the service technician confirms that the target device 300 is operating properly while the remote control is being performed. If the operation check is successful, the service technician officially registers the data of the target device 300, which has already been provisionally registered, by operating an information terminal, such as a portable information terminal that can communicate with the server 200.

[0079] Figure 7 is a flowchart showing an example of the calculation processing performed by the controller 10 of the remote control system 400.

[0080] In step S11, the controller 10 selects a desired work site from among a plurality of pre-registered work sites. The plurality of work sites may be registered in a database on the storage device 202 of the server 200, in a database on the remote control device 100, or in another database. An operator or other work-related person performs a work site specification operation to the specification operation input device 16 to specify a desired work site from among the plurality of work sites, and the specification operation input device 16 inputs information about the work site specification operation to the controller 10. The controller 10 recognizes the work site corresponding to the work site specification operation as a specific work site from among the plurality of work sites, and sets the recognized specific work site as the target for work performed using the remote control device 100.

[0081] In step S12, the controller 10 selects one or more operation targets 300 that are located at the specific work site selected in step S11 from among a plurality of pre-registered operation targets 300. The work sites where each of the plurality of operation targets 300 are located may be pre-registered in the database of the storage device 202, or in the database of the remote control device 100, or in any other database. An operator or other work-related person performs a designation operation (first designation operation) to the designation operation input device 16 to designate a desired operation target 300 from among the selected one or more operation targets 300. The designation operation input device 16 inputs information about the first designation operation to the controller 10. The controller 10 recognizes the operation target 300 designated by the first designation operation received by the designation operation input device 16 as a specific operation target 301, and sets the recognized specific operation target 301 as an object to be remotely controlled by the remote control device 100. Furthermore, the designated operation input device 16 may display each of the pre-registered operation targets 300 as selectable. For example, the controller 10 cannot communicate with operation targets 300 whose toggle switch 35 for enabling remote operation is not set to remote operation mode. In this case, the designated operation input device 16 may hide the operation target 300 or display it grayed out so that it cannot be selected.

[0082] In step S13, the controller 10 acquires information about the specific target 301. Specifically, if the specific target 301 is, for example, the second target 300B, the controller 10 acquires the communication specifications (specific communication specifications) and startup specifications (specific startup specifications) for the second target 300B, which is the specific target 301, from the server 200. The controller 10 has in advance stored destination information such as an IP address for communicating with the server 200. Therefore, the controller 10 can send a specification request signal to the server 200 requesting information about the communication specifications and startup specifications for the second target 300B. The specification request signal may include an identifier (identification information) that identifies the second target 300B. The specification request signal may also include a request for the communication establishment specifications of the second target 300B, a request for the format information of the second target 300B, and a request for the startup specifications of the second target 300B. In response to the specification request signal transmitted from the controller 10, the server 200 selects communication specification information and startup specification information for the second target device 300B from the data stored in the storage device 202, and sends a response signal containing data representing the selected communication specification and startup specification to the remote control device 100, which receives the response signal. As a result, the controller 10 of the remote control device 100 can recognize, for example, the communication establishment specification B1, data signal format B2 (specific format), and startup specification B3 of the second target device 300B shown in Figure 5.

[0083] Specifically, in step S13, the controller 10 may obtain the information shown in Figure 8 from the storage device 202 as information about the specific target of operation 301. Figure 8 is a diagram showing an example of data stored in the storage device 202. As shown in Figure 8, the storage device 202 may store information relating each of the multiple targets of operation 300 to information about the destinations of multiple devices in that target of operation 300. In this case, the specification request signal that the controller 10 requests from the server 200 may also include a request for information about the destinations of the multiple devices as shown in Figure 8. Therefore, the controller 10 can obtain information about the destinations of multiple devices in the second target of operation 300B shown in Figure 8. Specifically, the controller 10 can obtain the global IP address or local IP address of the communication device 36 for the target of operation, the local IP address and port number of the first controller 30A, the local IP address and port number of the second controller 30B, the local IP address and port number of the video encoder 32, and the local IP address and port number of the audio encoder 33. These IP addresses and port numbers are part of the specific communication specifications. The controller 10 may also acquire information regarding the encoding method of the video encoder 32 and the encoding method of the audio encoder 33, in addition to the destination information. Furthermore, the controller 10 may acquire the local IP address and port number of the third controller 30C and the local IP address and port number of the fourth controller 30D.

[0084] Next, in step S14, the controller 10 executes a communication establishment process using the specified communication specifications to establish communication with the second target of operation 300B (specific target of operation 301). Details of the communication establishment process will be described later.

[0085] Next, in step S15, after establishing communication with the second target device 300B (specific target device 301), the controller 10 executes a start-up process to start the second target device 300B using the specific start-up specification. Details of the start-up process will be described later.

[0086] Next, in step S16, the controller 10 starts controlling the specific operation target 301 at the specific work site to perform the actual work. Specifically, this is done as follows:

[0087] Figure 9 shows an example of data signal format conversion. The upper table in Figure 9 shows the default data signal format for the remote control device 100, which is the format before conversion. The lower table in Figure 9 shows the default data signal format B2 (specific format) for the second operating target 300B, which is the specific operating target 301.

[0088] As described above, the controller 10 recognizes a format B2 suitable for the second target 300B based on the data acquired from the storage device 202 (step S13). Therefore, in step S16, the controller 10 converts the format of the data signal to be transmitted to the second target 300B, which is designated as a specific target 301, to the specific format, and transmits the converted data signal to the specific target 301. As a result, even if the data signal formats of the multiple targets 300 are different from each other, the second target 300B, which is designated as a specific target 301, can receive the data signal that has been converted to the specific format by the controller 10 and transmitted from the controller 10. Therefore, the target controller 30 of the second target 300B can use the data signal to control the operation of the second target 300B without changing the format of the data signal.

[0089] Here, we will explain in detail the contents of the table in Figure 9. Items A1 to A8 in the upper table of Figure 9 and items A1 to A10 in the lower table are, for example, information (data) about the operation amount indication corresponding to the operation amount of the operation lever, while items S1 to S8 in the upper table and items S1 to S16 in the lower table are, for example, information (data) about on / off indications such as lever lock. In the default format of the remote control device 100 shown in the upper table of Figure 9, each of items A1 to A8 is 4 bits of information (data), whereas in the default format of the second operating target 300B shown in the lower table of Figure 9, each of items A1 to A16 is 8 bits of information (data). Therefore, in step S16, the controller 10 performs a data size conversion.

[0090] In the upper table, items A1 to A8 contain data on the control quantity instructions that are input from the operation input device 11 to the second controller 10B in the remote control device 100. Specifically, for example, the control quantity corresponding to the magnitude of the travel operation input given to the travel operation input device 11, the control quantity corresponding to the magnitude of the turn operation input given to the turn operation input device, and the control quantity corresponding to the magnitude of the attachment operation input given to the attachment operation input device are each distinguished from other control quantities and input as control quantity instructions in one of the items A1 to A8 in the upper table.

[0091] Furthermore, items S1 to S8 in the upper table contain data about on / off instructions input to the second controller 10B from a plurality of switches provided on the remote control device 100. Specifically, for example, the plurality of switches may include switches for switching the start and stop of the engine or other prime mover of the second control target 300B, switches for switching the lights of the second control target 300B on and off, switches for switching the wipers of the second control target 300B on and off, and switches for switching whether or not to operate the horn of the second control target 300B. Each of these on / off instructions is distinguished from other on / off instructions and input into one of items S1 to S8 in the upper table.

[0092] Furthermore, in the default format for the remote control device 100 shown in the upper table of Figure 9, there are 8 data items related to the operation quantity instruction (items A1 to A8 in the upper table) and 8 data items related to the on / off instruction (items S1 to S8 in the upper table). In contrast, in the default format for the second control target 300B shown in the lower table of Figure 9, there are 10 data items related to the operation quantity instruction (items A1 to A10 in the lower table) and 16 data items related to the on / off instruction (items S1 to S16 in the lower table). That is, in Figure 9, the operation quantity instructions entered in items A1 to A8 in the upper table correspond to items A1 to A8 in the lower table in numerical order, but the upper table does not have items corresponding to items A9 and A10 in the lower table. Furthermore, in Figure 9, the on / off instruction data entered in items S1 to S8 in the upper table corresponds in numerical order to items S1 to S8 in the lower table, but the upper table does not have items corresponding to items S9 to S16 in the lower table. Therefore, in step S16, the controller 10 sets default values ​​(for example, "0" or "1") for items that are not used in the converted format (items A9, A10, S9 to S16 in the lower table).

[0093] Furthermore, in the specific example of format conversion described above, in step S16, the controller 10 assigns the data of multiple items in the format before conversion (for example, the data of items A1 to A8) to multiple items in the format after conversion (for example, items A1 to A8) in numerical order, but the data assignment to each item may be changed. Also, if a "checksum" does not exist in a particular format, the controller 10 may set a default value for the checksum bits. As for the checksum, a known method such as CRC (Cyclic Redundancy Check) may be used.

[0094] The controller 10 converts the format of the data signal to be transmitted to the specific operation target 301 into the specified format as described above, and transmits the converted data signal to the specific operation target 301, thereby enabling the specific operation target 301 to perform the actual work at the specific work site.

[0095] Next, in step S17, the controller 10 determines whether to change the specific operation target 301, which is the target to be remotely controlled by the remote control device 100, from the second operation target 300B to another operation target 300. Specifically, for example, if the person in charge of the work changes the specific operation target 301, they perform a designation operation (second designation operation) to the designation operation input device 16 to designate the specific operation target 301 as the target of the next remote control by the remote control device 100. If the designation operation input device 16 has not received a designation operation, the controller 10 does not change the specific operation target 301 and proceeds to the process in step S18. On the other hand, if the designation operation input device 16 has received a designation operation, the controller 10 changes the specific operation target 301 to the operation target 300 indicated in the designation operation and repeats the process from step S11 onwards.

[0096] In step S17, if the designated operation input device 16 receives the second designated operation from a worker while communication between the controller 10 and the designated operation target 301 has been established as described above, the controller 10 recognizes the operation target 300 designated by the second designated operation as the second designated operation target 301 from among the multiple operation targets 300. In this case, even if communication between the controller 10 and the designated operation target 301 has been established, the controller 10 can select the operation target 300 designated by the second designated operation performed by the worker as the second designated operation target 301 from among the multiple operation targets 300. When the controller 10 recognizes the second designated operation target 301, it may change the designated operation target 301 to the second designated operation target 301 and execute a communication establishment process to establish communication with the second designated operation target 301. Furthermore, the controller 10 may perform a communication establishment process to establish communication with the second specific operation target 301 when the remote operation of the specific operation target 301 (first specific operation target 301) that is set at that time based on the first specified operation is completed. The controller 10 may maintain the state in which communication with the first specific operation target 301 is established until it performs the communication establishment process to establish communication with the second specific operation target 301, and may disconnect communication with the first specific operation target 301 when it performs the communication establishment process with the second specific operation target 301.

[0097] In step S18, the controller 10 determines whether or not to terminate control for remote operation by the remote control device 100. Specifically, for example, if a worker wants the controller 10 to terminate control of the remote operation, they input to terminate the remote operation to an input device such as the designated operation input device 16. If the input device does not receive the input, the controller 10 does not terminate control of the remote operation and repeats the process from step S16 onward. On the other hand, if the input device receives the input, the controller 10 terminates control of the remote operation.

[0098] The above describes the main features of the remote control system 400 according to this embodiment. The communication establishment process and the startup process will be described in detail below.

[0099] [Communication Establishment Process] Figure 10 is a flowchart showing an example of the communication establishment process in step S14 of Figure 7.

[0100] In step S103, the controller 10 (specifically, the communication establishment processing unit 101) stores (sets) destination information for multiple devices in the remote control device 100 based on the information including specific communication specifications obtained from the server 200 in step S13 of Figure 7.

[0101] Specifically, in step S103, the controller 10 sets, for example, the local IP address and port number of the first controller 30A of the specified operation target 301 as destination information, the local IP address and port number of the second controller 30B of the specified operation target 301 as destination information, the local IP address and port number of the video encoder 32 of the specified operation target 301 as destination information, and the local IP address and port number of the audio encoder 33 of the specified operation target 301 as destination information. This destination information is used to specify the destination of information when the multiple devices in the remote control device 100 communicate with the multiple devices in the specified operation target 301. The controller 10 may also set the IP address (global IP address or local IP address) of the operation target communication device 36 of the specified operation target 301 as destination information. In addition to the above destination information, the controller 10 may also set the encoding method (video encoding method) of the video encoder 32 and the encoding method (sound encoding method) of the audio encoder 33.

[0102] These destination information and encoding methods may be stored, for example, in the first controller 10A, or in the communication device for the operating device 18 (specifically, the fourth controller 10D of the communication device 18). Alternatively, the destination information for the first controller 30A may be stored in the first controller 10A, the destination information for the second controller 30B may be stored in the second controller 10B, the destination information and encoding method for the video encoder 32 may be stored in the video decoder 12, and the destination information and encoding method for the audio encoder 33 may be stored in the audio decoder 13.

[0103] Next, in step S104, the controller 10 sends a pairing request signal to the specific target device 301 to pair the remote control device 100 with the specific target device 301. Specifically, the controller 10 sends the pairing request signal to the IP address (global IP address or local IP address) of the target device communication device 36. The target device communication device 36 of the specific target device 301 receives the pairing request signal, and the target device controller 30 recognizes the pairing request signal (step S201, described later).

[0104] Specifically, the pairing request signal may include, for example, a first pairing request signal for pairing the first controller 10A of the remote control device 100 with the first controller 30A of the specific target device 301 ("Pair A" in Figure 4), a second pairing request signal for pairing the second controller 10B of the remote control device 100 with the second controller 30B of the specific target device 301 ("Pair B" in Figure 4), a third pairing request signal for pairing the video decoder 12 of the remote control device 100 with the video encoder 32 of the specific target device 301 ("Pair C" in Figure 4), and a fourth pairing request signal for pairing the audio decoder 13 of the remote control device 100 with the audio encoder 33 of the specific target device 301 ("Pair D" in Figure 4). In this case, the pairing request signal may include destination information for multiple devices of the remote control device 100 (operator-side information) and destination information for multiple devices of the specific target device 301 (target device-side information).

[0105] Specifically, the information on the operating device side may include destination information for the communication device 18 for the operating device (e.g., global IP address or local IP address), destination information for the first controller 10A (e.g., local IP address and port number), destination information for the second controller 10B (e.g., local IP address and port number), destination information for the video decoder 12 (e.g., local IP address and port number), and destination information for the audio decoder 13 (e.g., local IP address and port number). The information on the target device side may include, for example, the information shown in Figure 8.

[0106] In step S201, the target controller 30 of the specified target device 301 determines whether or not it has received the pairing request signal. If the target controller 30 has not received the pairing request signal (NO in step S201), it repeats the process in step S201. If the target controller 30 has received the pairing request signal (YES in step S201), it performs the process in step S202.

[0107] In step S202, the target controller 30 stores (sets) the destination information for each of the multiple devices 10A, 10B, 12, and 13 of the remote control device 100 based on the operating device side information included in the pairing request signal.

[0108] Specifically, in step S202, the target controller 30 sets, for example, the local IP address and port number of the first controller 10A of the remote control device 100 as destination information, the local IP address and port number of the second controller 10B of the remote control device 100 as destination information, the local IP address and port number of the video decoder 12 of the remote control device 100 as destination information, and the local IP address and port number of the audio decoder 13 of the remote control device 100 as destination information. This destination information is used to specify the destination of information when multiple devices in the specific target device 301 communicate with multiple devices in the remote control device 100. The target controller 30 may also set the IP address (global IP address or local IP address) of the communication device 18 for the remote control device 100 as destination information.

[0109] These destination information may be stored, for example, in the first controller 30A or in the communication device 36 for operation. Alternatively, the destination information for the first controller 10A may be stored in the first controller 30A, the destination information for the second controller 10B may be stored in the second controller 30B, the destination information for the video decoder 12 may be stored in the video encoder 32, and the destination information for the audio decoder 13 may be stored in the audio encoder 33.

[0110] In step S203, the controller to be operated 30 transmits a response signal to the remote control device 100 that represents a response to the pairing request signal.

[0111] The response signal transmitted by the target controller 30 may include content indicating whether pairing is possible or not. Specifically, for example, the response signal may include content indicating whether pairing is possible or impossible between the first controller 10A of the remote control device 100 and the first controller 30A of the specific target device 301; content indicating whether pairing is possible or impossible between the second controller 10B of the remote control device 100 and the second controller 30B of the specific target device 301; content indicating whether pairing is possible or impossible between the video decoder 12 of the remote control device 100 and the video encoder 32 of the specific target device 301; and content indicating whether pairing is possible or impossible between the audio decoder 13 of the remote control device 100 and the audio encoder 33 of the specific target device 301. Examples of situations where pairing is impossible include when the operating controller 30 cannot recognize the device to be paired that is installed on the specific operating target 301, when the device to be paired is malfunctioning, when the device to be paired is not installed on the specific operating target 301, or when the power to the device to be paired is turned off.

[0112] Next, in step S105, the controller 10 determines whether or not it has received the response signal within a predetermined time. If the controller 10 does not receive the response signal within the predetermined time (NO in step S105), it determines that there is an error and repeats the process from step S11 onwards in Figure 7. On the other hand, if the controller 10 has received the response signal within the predetermined time (YES in step S105), it performs the process in step S106.

[0113] In step S106, the controller 10 determines whether pairing is possible or not. Specifically, if the response signal received in step S105 contains at least one statement indicating that pairing is not possible, the controller 10 determines that pairing is not possible. In this case (NO in step S106), the controller repeats the process from step S11 onwards in Figure 7.

[0114] On the other hand, if the response signal received in step S105 indicates that pairing is possible for all of the target devices 30A, 30B, 32, and 33 in the specific operation target 301, the controller 10 determines that pairing is possible. In this case (YES in step S106), the controller 10 transmits a pairing completion signal (pairing start signal) to the specific operation target 301 indicating that pairing of each pair A to D has been completed (step S107). The communication device 36 of the specific operation target 301 receives the pairing completion signal (pairing start signal), and the operation target controller 30 recognizes that pairing has been completed (step S204). The controller 10 then terminates the communication establishment process. As a result, communication between the remote control device 100 and the specific operation target 301 is established. Specifically, information can be communicated between the first controller 10A of the remote control device 100 and the first controller 30A of the specific target device 301, between the second controller 10B of the remote control device 100 and the second controller 30B of the specific target device 301, between the video decoder 12 of the remote control device 100 and the video encoder 32 of the specific target device 301, and between the audio decoder 13 of the remote control device 100 and the audio encoder 33 of the specific target device 301.

[0115] Once the communication establishment process is complete, it becomes possible to send and receive information (data) such as the following:

[0116] The video data acquired by the imaging device of the information acquisition device 34 and encoded by the video encoder 32 is input to the video decoder 12 via the communication device for the target of operation 36 and the communication device for the operation device 18. This video data is decoded in the video decoder 12 based on the encoding method of the video encoder 32. This makes it possible to display the video corresponding to the video data on the video output device 14.

[0117] Furthermore, sound data acquired by the sound acquisition device of the information acquisition device 34 and encoded by the audio encoder 33 is input to the audio decoder 13 via the communication device for the target of operation 36 and the communication device for the operation device 18. This sound data is decoded in the audio decoder 13 based on the encoding method of the audio encoder 33. This makes it possible to output sound corresponding to the sound data from the sound output device 15.

[0118] Furthermore, data signals transmitted from the first controller 30A of the specific target 301, for example, data signals representing information about the state of the specific target 301 detected by the state detection device 37, are input to the first controller 10A via the communication device 36 for the target and the communication device 18 for the control device. These data signals are input from the first controller 10A to the third controller 10C in the remote control device 100, and the third controller 10C controls the operation of the specific target 301 based on these data signals. For example, the third controller 10C performs controls such as driving the seat drive device based on information about the tilt of the cabin 304 of the specific target 301 included in the data signals, and displaying information about the posture of the specific target 301 included in the data signals on the video output device 14.

[0119] Furthermore, the operation command generated and transmitted by the first controller 10A based on the operation signal input from the second controller 10B to the first controller 10A is input to the first controller 30A via the communication device for the operation device 18 and the communication device for the target of operation 36, and this operation command is input from the first controller 30A to the second controller 30B. The second controller 30B controls the operation of the drive mechanism 311 so that the drive mechanism 311 performs an action in accordance with the operation command, that is, an action in accordance with the operation signal. As a result, the specific target of operation 301 can perform an action in accordance with the operation input input by the operator to the operation input device 11 of the remote control device 100.

[0120] In this embodiment, the communication establishment specification includes the communication method of the specific target 301 and the destination information of the specific target 301. The remote control device 100 can perform the above-described communication establishment process by obtaining the communication method of the specific target 301 and the destination information of the specific target 301 (IP address, port number, etc.) from the server 200. The communication method is, for example, a communication control method defined in each target controller 30 of the multiple targets 300, and may specifically be an API (Application Programming Interface). Once the destination information and communication method (communication control method) suitable for the specific target 301 as the communication partner are determined, the controller 10 of the remote control device 100 can communicate with the specific target 301 as the communication partner for the communication establishment process. Specifically, for example, once it is determined what protocols and commands to use in communication with the specific target 301 as the communication partner, the controller 10 can communicate with the communication partner for the communication establishment process. In other words, once the communication control method is determined, the controller 10 can complete the pairing described above, and as a result, it can proceed to a phase where it can send and receive data signals such as video and sound.

[0121] The aforementioned communication method may include at least one of several communication methods (multiple communication lines), such as carrier lines, optical communication, satellite communication, and dedicated lines (dedicated communication). For example, information on which communication method the controller 10 of the remote control device 100 should use to establish communication with each of the multiple target devices 300 may be stored in the storage device 202 in association with the target device 300, and this information may be provided to the controller 10 from the server 200. This allows the controller 10 to obtain information regarding the communication method for a specific target device 301 from the storage device 202. In the communication establishment process for establishing communication with a specific target device 301, the controller 10 can execute the communication establishment process using a communication line connected to the specific target device 301, i.e., a communication method corresponding to the specific target device 301 from among various communication methods such as carrier lines, optical communication, satellite communication, and dedicated lines. This allows the controller 10 to, for example, in step S104 of Figure 10, transmit a pairing request signal to the specific target device 301 using the communication line, and the specific target device 301 can receive the pairing request signal.

[0122] [Startup Process] Figure 11 is a flowchart showing an example of the startup process in step S15 of Figure 7. In this embodiment, the startup process includes an equipment startup process, which is a startup process for starting a predetermined piece of equipment as shown in Figure 11.

[0123] The controller 10 (start processing unit 102) and the controller 30 to be operated may perform the processes shown in steps S111 and S211 in Figure 11 after the start of the processes in steps S107 and S204 in Figure 10. However, some of the processes shown in Figure 11 may be omitted.

[0124] In step S111, the controller 10 of the remote control device 100 transmits a start sequence start signal to the specified target 301 to initiate the startup of a predetermined device in the specified target 301. In step S211, the target controller 30 of the specified target 301, which has received the start sequence start signal, starts the start sequence. The predetermined device may include, for example, at least one of a first controller 30A, a second controller 30B, a video encoder 32, an audio encoder 33, and a prime mover.

[0125] The storage device 202 stores each of the multiple operating targets 300 in association with equipment start-up processing related information (an example of a start-up specification), which is information regarding the equipment start-up process for that operating target 300. Specifically, the storage device 202 may store, for example, each of the multiple operating targets 300 in association with information regarding the equipment to be started (start-up target) for that operating target 300 (an example of equipment start-up processing related information). The controller 10 may be configured to receive equipment start-up processing related information from the server 200. Specifically, for example, in step S13 of Figure 7, the controller 10 may acquire equipment start-up processing related information for a specific operating target 301 from the server 200 as part of the specific start-up specification. The equipment start-up processing related information may include, for example, information that the start-up target is a prime mover, information that the start-up target is a first controller 30A, information that the start-up target is a second controller 30B, or information that the start-up target is another electrical device.

[0126] In step S112, the controller 10 transmits a key-on instruction signal to the specified operation target 301 in response to an operation (key-on instruction operation) input by a worker, such as an operator, to the designated operation input device 16. The key-on instruction signal is an instruction signal to switch the state in which power is not supplied to the specified operation target 301 from a state in which power is supplied to the specified operation target 301 from a state in which power is supplied from a power supply device such as a battery mounted on the specified operation target 301. Specifically, the specified equipment may be an electrical device including a prime mover such as an engine, or it may be other electrical equipment.

[0127] In step S212, the controller 30 that receives the key ON instruction signal performs a process to switch to a state in which power is supplied from the power supply device to the predetermined device. Note that the controller 10 does not have to be triggered by an operation by a worker on the designated operation input device 16 (key ON instruction operation). For example, it may transmit the key ON instruction signal to the specific target device 301 together with the start sequence start signal in step S111, or it may transmit the key ON instruction signal to the specific target device 301 automatically after transmitting the start sequence start signal.

[0128] Next, in step S214, the controller to be operated 30 transmits a response signal to the remote control device 100. This response signal indicates that the controller to be operated 30 has executed a process to switch to a state in which power is supplied to a predetermined device based on the key ON instruction signal, and may include information such as that indicating that the startup sequence has been completed without any abnormalities.

[0129] Next, in step S113, the controller 10 determines whether or not it has received the response signal. If the controller 10 has not received the response signal (NO in step S113), it remains in standby mode. On the other hand, if the controller 10 has received the response signal (YES in step S113), it performs the process in step S114. If the controller 10 cannot receive the response signal within a predetermined time, it may determine that there is an error, interrupt the startup process, and restart the startup process from step S111.

[0130] In step S114, the controller 10 transmits a prime mover ON instruction signal (engine ON instruction signal) to the specified operation target 301 in response to an operation (prime mover ON instruction operation) input by a worker, such as an operator, to the designated operation input device 16. The prime mover ON instruction signal is an instruction signal for starting (operating) a prime mover such as an engine.

[0131] In step 215, the controller 30 of the specified operation target 301, which has received the engine ON instruction signal, performs processing to start (operate) the engine of the specified operation target 301. Note that the controller 10 does not need to be triggered by an operation by a worker to the designated operation input device 16 (engine ON instruction operation). For example, the controller 10 may automatically transmit the engine ON instruction signal to the specified operation target 301 when it receives the response signal (YES in step S113).

[0132] Next, in step S217, the controller to be operated 30 transmits a response signal to the remote control device 100. This response signal indicates that the controller to be operated 30 has performed the process to start (operate) the prime mover based on the prime mover ON instruction signal, and may include information indicating, for example, that the prime mover has started up without any problems. The controller to be operated 30 may also transmit a response signal to the remote control device 100 when, for example, a sensor that detects the rotational speed of the prime mover detects that the rotational speed of the prime mover has increased to a predetermined rotational speed.

[0133] Next, in step S115, the controller 10 determines whether or not it has received the response signal. If the controller 10 has not received the response signal (NO in step S115), it remains in standby mode. On the other hand, if the controller 10 has received the response signal (YES in step S115), it terminates the equipment start process (prime mover start process). The controller 10 then proceeds to the process shown in step S121 in Figure 12. In step S115, if the controller 10 cannot receive the response signal within a predetermined time, it may determine that there is an error, interrupt the start process, and restart the start process from step S111.

[0134] The startup process may include an operation pattern selection process, a calibration process, and an initial operation process. Figure 12 is a flowchart showing an example of the operation pattern selection process, calibration process, and initial operation process. After the processes of steps S115 and S217 in Figure 11, the controller 10 and the target controller 30 may perform the processes of steps S121 and S221 shown in Figure 12. However, some of the processes shown in Figure 12 may be omitted.

[0135] In Figure 12, steps S121-S124 and S221-S222 are processes that include operation pattern selection, step S125 is a calibration process, and steps S126 and S224 are initial operation processes.

[0136] In step S121, the controller 10 determines whether or not to change the operation pattern used in the operation input device 11. For example, the controller 10 displays the operation pattern of the operation input device 11 that is currently set (e.g., a standard operation pattern) on a predetermined display device. The predetermined display device may be at least one of the video output device 14 and the designated operation input device 16, or it may be any other display device.

[0137] Here, the "operation pattern" is a pattern of combinations between operations in the operation input device 11 and the operation of the actuator of the specific target of operation 301. Specifically, operations in the operation input device 11 include, for example, operations in the forward / backward direction given to the left operation lever, operations in the left / right direction given to the left operation lever, operations in the forward / backward direction given to the right operation lever, and operations in the left / right direction given to the right operation lever. The operation of the actuator includes, for example, the operation of the boom cylinder 308, the arm cylinder 309, the tip attachment cylinder 310, and the slewing motor of the specific target of operation 301.

[0138] The standard operation pattern is recognized, for example, based on items A1 to A8 related to operation quantity instructions shown in the upper table of Figure 9. The display screen of the predetermined display device displays items for selecting whether to use the standard operation pattern or change to a different operation pattern. If an operator or other work-related person inputs an operation to select the standard operation pattern on the display screen of the predetermined display device, the controller 10 determines that there is no change in the operation pattern used in the operation input device 11. In this case (NO in step S121), the controller 10 terminates the operation pattern selection process and performs the process in step S125. If an operator or other work-related person inputs an operation to change to a different operation pattern on the display screen of the predetermined display device, the controller 10 determines that there is a change in the operation pattern used in the operation input device 11. In this case (YES in step S121), the controller 10 executes the process in step S122, i.e., the process of selecting an operation pattern.

[0139] The storage device 202 stores each of the multiple target devices 300 in association with operation pattern selection processing related information (an example of a startup specification), which is information regarding the operation pattern selection process for that target device 300. Specifically, the storage device 202 may store, for example, each of the multiple target devices 300 in association with information regarding whether or not it is possible to select an operation pattern for that target device 300 (an example of operation pattern selection processing related information), and, if it is possible to select an operation pattern, information regarding the operation patterns that can be selected for that target device 300 (an example of operation pattern selection processing related information). The controller 10 may be configured to receive operation pattern selection processing related information from the server 200. Specifically, for example, in step S13 of Figure 7, the controller 10 may acquire operation pattern selection processing related information for a specific target device 301 from the server 200 as part of the specific startup specification.

[0140] In the operation pattern selection process of step S122, the controller 10 displays multiple options for operation patterns that can be selected on the specific operation target 301 on a display device such as the video output device 14 or another display. The worker inputs an operation to the designated operation input device 16 to select one of the operation patterns from the multiple options, and the controller 10 recognizes the operation pattern selected by the operation input and transmits the selection result of the recognized operation pattern to the specific operation target 301 (step S123).

[0141] The controller 30 of the specified target 301 sets the operation pattern in the specified target 301 to the operation pattern shown in the selection result (step S221). For example, if the set operation pattern is a standard operation pattern, the controller 30 recognizes the operation command transmitted from the remote control device 100 based on the standard operation pattern. If the set operation pattern is an operation pattern other than the standard operation pattern, the controller 30 recognizes the operation command transmitted from the remote control device 100 based on the other operation pattern. If the set operation pattern is one of the other operation patterns, the controller 30 may recognize the operation command transmitted from the remote control device 100 by performing a process to convert the standard operation pattern to the other operation pattern.

[0142] Then, the controller 30 to be operated sends a response signal to the remote control device 100 indicating that the setting of the operation pattern has been completed (step S222).

[0143] In step S124, the controller 10 determines whether or not it has received the response signal. If it has not received the response signal (NO in step S124), it remains in standby mode. On the other hand, if it has received the response signal (YES in step S124), the controller 30 starts the process of converting the format shown in the upper table of Figure 9 to the format shown in the lower table of Figure 9. Specifically, this is done as follows.

[0144] For example, the controller 10 inputs several control quantities, including control quantities corresponding to the magnitude of the travel operation input, control quantities corresponding to the magnitude of the turning operation input, and control quantities corresponding to the magnitude of the attachment operation input, as control quantity instructions, distinguishing them from other control quantities, into items (items A1 to A8) related to control quantity instructions corresponding to the standard operation patterns shown in the upper table of Figure 9. The controller 10 then performs a process to convert the data signal format shown in the upper table of Figure 9 to the format of the specific operation target 301 (specific format) shown in the lower table of Figure 9.

[0145] Meanwhile, the specific target device 301 receives a data signal transmitted from the remote control device 100, and the target device controller 30 performs a process to rearrange the arrangement of items (items A1 to A8) related to the operation amount instruction corresponding to the standard operation pattern in the data signal so as to correspond to the other operation pattern set. Then, the target device controller 30 performs a process to recognize the operation amount instruction based on the rearranged items A1 to A8. By having the controller 10 and the target device controller 30 perform the above-described process, the selected operation pattern can be used when operating the specific target device 301 with the operation input device 11.

[0146] If the specified target 301 does not support the selection (change) of operation patterns, or if the number of operation patterns that can be used on the specified target 301 is limited, the remote control system 400 may be configured to allow the selection of operation patterns used in the operation input device 11 through a process other than the process described above. For example, in the operation pattern selection process, the controller 10 displays multiple options for operation patterns that can be used on the operation input device 11 on a display device such as the video output device 14 or another display. The person in charge of the work makes an operation input to the designated operation input device 16 to select one of the operation patterns from the multiple options, and the controller 10 recognizes the operation pattern selected based on the operation input. Next, the controller 10 performs a process to rearrange the items (items A1 to A8) related to operation quantity instructions that correspond to the standard operation patterns shown in the upper table of Figure 9, based on the recognized operation pattern. The target controller 30 also starts a process to convert the items (items A1 to A8) related to operation quantity instructions stored in the upper table, whose arrangement has been rearranged by the rearrangement process, into the format of the specified target 301 shown in the lower table of Figure 9. Furthermore, the target controller 30 starts processing to recognize the control amount instruction based on the arrangement of items (items A1 to A8) related to the control amount instruction corresponding to the standard operation pattern in the data signal transmitted from the remote control device 100. By having the controller 10 and the target controller 30 perform the above processing, the selected operation pattern can be used when operating the specific target 301 with the operation input device 11.

[0147] Next, we will explain the calibration process.

[0148] In step S125, the controller 10 performs a calibration process on the specific target 301. The storage device 202 stores each of the multiple targets 300 in association with calibration process-related information (an example of a startup specification), which is information regarding the calibration process for that target 300. Specifically, the storage device 202 may store, for example, each of the multiple targets 300 in association with information on whether or not a calibration process is possible for that target 300 (an example of calibration process-related information) and information for performing the calibration process if a calibration process is possible (an example of calibration process-related information). The controller 10 may be configured to receive calibration process-related information from the server 200. Specifically, for example, in step S13 of Figure 7, the controller 10 may acquire calibration process-related information for the specific target 301 from the server 200 as part of the specific startup specification.

[0149] In the calibration process of step S125, the controller 10 may calibrate the correspondence between the maximum and minimum values ​​of the operation signals used by the target controller 30 of the specific target 301 to control the operation of the drive mechanism 311, and the maximum and minimum values ​​of the operation amount indication, based on the calibration process-related information provided from the storage device 202. Specifically, this is as follows.

[0150] As shown in Figure 13, the controller 10 may calibrate the correspondence between the control signal and the control amount indication so that the control signal has a predetermined linear relationship with the control amount indication in the range of the control amount indication from "0%" where the control amount indication is zero to "+100%" where the control amount indication is maximum, and in the range of the control amount indication from "0%" where the control amount indication is zero to "-100%" where the control amount indication is minimum.

[0151] In Figure 13, the dashed line shows an example of the correspondence between the operating signal and the manipulated variable indication before calibration, and the solid line shows an example of the correspondence between the operating signal and the manipulated variable indication after calibration. On the horizontal axis of Figure 13, "0%" is the command value (value of the manipulated variable) such that the drive mechanism 311 does not drive the actuator, "+100%" is the command value (value of the manipulated variable) such that the drive mechanism 311 drives the actuator to its maximum in one direction (positive direction), and "-100%" is the command value (value of the manipulated variable) such that the drive mechanism 311 drives the actuator to its maximum in the other direction (negative direction).

[0152] The controller 10 may, for example as shown in Figure 13, calibrate the correspondence between the control signal and the manipulated variable instruction so that when the manipulated variable instruction is "0%", the control signal becomes "α0", when the manipulated variable instruction is "+100%", the control signal becomes "α1", and when the manipulated variable instruction is "-100%", the control signal becomes "α2", and the control signal changes linearly with respect to the manipulated variable instruction within the range of the manipulated variable instruction from "0%" to "+100%", and within the range of the manipulated variable instruction from "0%" to "-100%".

[0153] The controller 10 corrects the manipulated amount instructions stored in items A1 to A8 in Figure 9, respectively, based on the correspondence between the calibrated operation signal and the manipulated amount instruction. This calibrates the degree of operation of the actuator of the specific target 301 in response to the operation input given to the operation input device 11.

[0154] Once the calibration process is complete, the controller 30 starts initial operation control (initial operation processing) before the actual operation by the specific target 301 at the specific work site (step S126).

[0155] The initial operation processing included in the startup process is preferably performed after pairing is complete and before the specific operation target 301 actually performs work at the specific work site. Until the startup process, including this initial operation processing, is completed, the transmission of an operation command corresponding to the operation amount instruction based on the magnitude of the operation input to the operation input device 11 to the specific operation target 301, and the operation of the specific operation target 301 in response to said operation command are withheld.

[0156] In steps S126 and S224, it is desirable that the initial operation of the specific operation target 301 does not interfere with any objects surrounding the specific operation target 301, and that the change in the operation of the specific operation target 301 is captured within the shooting range of the image acquired by the imaging device of the information acquisition device 34. Examples of such initial operations of the specific operation target 301 include the operation of the wipers and the turning on of the lights.

[0157] The storage device 202 may store each of the multiple target devices 300 in association with initial operation information (an example of a startup specification), which is information relating to an initial operation suitable for that target device 300. Specifically, the storage device 202 may store, for example, each of the multiple target devices 300 in association with initial operation information, which includes an initial operation that causes the change in the operation of the target device 300 to be reflected in the shooting range of the image acquired by the imaging device for that target device 300. The controller 10 may be configured to receive initial operation information from the server 200. Specifically, for example, in step S13 of Figure 7, the controller 10 may acquire initial operation information for a specific target device 301 from the server 200 as part of the specific startup specification.

[0158] In step S126, the controller 10 displays candidate initial operations for the specific target 301 on a predetermined display device based on the initial operation information, and causes the predetermined display device to display a message prompting the operator to start the initial operation. The predetermined display device may be at least one of the video output device 14 and the designated operation input device 16, or it may be any other display device. The operator, in accordance with the display, performs an input operation, for example, to the designated operation input device 16, to cause the specific target 301 to perform a wiper operation as the initial operation.

[0159] If the remote control device 100 is able to communicate correctly with the specific target device 301, the video output device 14 displays the video captured by the information acquisition device 34 (imaging device). The operator can confirm the operation of the wiper of the specific target device 301 in the video displayed on the video output device 14.

[0160] If the remote control device 100 is unable to communicate properly with the specific target device 301, the video output device 14 will not display the video captured by the information acquisition device 34 (imaging device). In this case, the operator will not be able to confirm the operation of the wiper of the specific target device 301 on the video output device 14.

[0161] The controller 10 causes the predetermined display device (e.g., designated operation input device 16) to output a display asking whether the specific operation target 301 has operated properly. The operator responds by indicating whether they have confirmed that the specific operation target 301 has operated properly on the designated operation input device 16. If the response is negative, the remote operation is interrupted. In this case, the controller 10 may restart the startup process from step S111. If the response is positive, the remote operation of the specific operation target 300 by the remote operation device 100 is permitted. That is, the controller 10 sequentially transmits operation commands corresponding to the operation amount instructions given to the operation input device 11 to the specific operation target 301, thereby enabling the specific operation target 301 to perform operations (such as driving, turning, and attachment operation) in accordance with the operation commands (steps S128 and S225 in Figure 12).

[0162] If the initial operation information includes information that the imaging device of the information acquisition device 34 is located inside the cabin 304 of the specific operating target 301, the controller 10 may select the operation of the wipers as the initial operation of the specific operating target 301. For example, in the modification of the operating target 300 (step S31 in Figure 6), if a service technician modifies the operating target 300 to install the imaging device inside the cabin 304, the service technician may perform an input operation in the controller 10 or storage device 202 such that the information is stored in association with the operating target 300. This makes it possible for the controller 10 to recognize the information during the startup process.

[0163] Furthermore, if the imaging device is located outside the cabin 304, the controller 10 may select turning on the lights as the initial operation of the specified operation target 301, or it may select sounding the horn as the initial operation of the specified operation target 301. In these cases as well, the controller 10 will output a message to the predetermined display device asking whether the specified operation target 301 has performed its initial operation appropriately. Note that the initial operation may also be the operation of an attachment.

[0164] As described above, the controller 10 may select an action such as operating the wipers or turning on the lights as the initial action of the specified operation target 301 in response to the operator's input, or the controller 10 may automatically select an action such as operating the wipers or turning on the lights as the initial action of the specified operation target 301 even without an operator's input. When the controller 10 automatically selects the initial action, the controller 10 may start the initial action processing, select the initial action, and execute the selected initial action, provided that the processing before the initial action processing (in the specific example in Figure 12, the calibration processing in step S125) is completed.

[0165] Furthermore, the controller 10 may also automatically determine whether the specific target 301 operated appropriately during the initial operation processing. Specifically, for example, the controller 10 may use image analysis to determine whether the image displayed on the video output device 14 contains changes corresponding to the initial operation of the specific target 301 (for example, the operation of a wiper), and if the image analysis identifies changes corresponding to the operation of the specific target 301, it may determine that the specific target 301 operated appropriately.

[0166] As described above, according to the remote control system 400 and the remote control support method according to this embodiment, even if the communication specifications of the multiple target devices 300 are different from each other, the controller 10 can communicate with each target device 300 using a communication specification that is suitable for each of the multiple target devices 300. Furthermore, in this embodiment, even if the starting specifications of the multiple target devices 300 are different from each other, the controller 10 can start a specific target device 301 using a starting specification that is suitable for that specific target device 301 after communication with that specific target device 301 has been established.

[0167] Furthermore, the remote control system 400, remote control device 100, server 200 (management device), and remote control support method according to this embodiment may have the following additional features.

[0168] [Modification 1] In Modification 1, the storage device 202 may be configured to store in association with each of the multiple objects to be operated 300 and the device communication specifications, which are the communication specifications of an information acquisition device 34 located at the work site where the objects to be operated 300 are located and which acquires information about the status of the work site. The controller 10 may perform a communication establishment process to establish communication with the information acquisition device 34 using the device communication specifications.

[0169] The device communication specifications may include, for example, the communication method of the information acquisition device 34 and the destination information of the information acquisition device 34. The communication method may be, for example, a communication control method defined in the controller of the information acquisition device 34, and may specifically be an API (Application Programming Interface). The destination information may be, for example, an IP address.

[0170] In this modified example 1, the information acquisition device 34 includes at least one of an imaging device that photographs the work site and a sound acquisition device that collects sounds from the work site. At least one of the video data acquired by the imaging device and the sound data acquired by the sound acquisition device is transmitted to the remote control device 100 with which communication has been established. The storage device 202 stores, for example, each of the multiple target devices 300 and the information acquisition device 34 located at the work site where the target device 300 exists, in association with each of them. The controller 10 may be configured to receive information about the device communication specifications of the information acquisition device 34 from the server 200. Specifically, the controller 10 may, for example, in step S13 of Figure 7, obtain information about the device communication specifications of the information acquisition device 34 from the server 200.

[0171] Each of the multiple operating targets 300 may be equipped with a positioning device for a satellite positioning system such as GNSS (Global Navigation Satellite System) to detect the location of the operating target 300. In this case, each of the multiple operating targets 300 may transmit location information detected by the positioning device to the server 200 at predetermined intervals, and the storage device 202 may store the location information of each of the multiple operating targets 300. The storage device 202 may also pre-store the location information of the information acquisition device 34 located at the work site.

[0172] In this modified example 1, the storage device 202 is configured to store the device communication specifications of the information acquisition device 34 in association with each of the multiple operation targets 300. Therefore, the controller 10 can establish communication with the information acquisition device 34 using the device communication specifications of the information acquisition device 34 stored in association with a specific operation target 301, that is, the device communication specifications that are compatible with the information acquisition device 34 located at the same work site as the specific operation target 301.

[0173] [Modification 2] In the above embodiment, as shown in Figure 4, the controller 10 of the remote control device 100 includes a first controller 10A, a second controller 10B, a third controller 10C, and a fourth controller 10D. However, two or more of these controllers 10A to 10D may be composed of a single controller. Specifically, for example, the controller 10 may be composed of a single controller equipped with the functions of the first to fourth controllers 10A to 10D. Alternatively, the controller 10 may include, for example, a single controller equipped with the functions of the first to third controllers 10A to 10C and a fourth controller 10D.

[0174] The communication establishment process in the above embodiment may be performed by the first controller 10A, by the fourth controller 10D (i.e., the communication device 18 for the operating device), or by the first controller 10A and the fourth controller 10D working together.

[0175] [Modification 3] In the remote control system 400, the controller 10 determines whether or not there is a malfunction in the startup process, and if there is a malfunction in the startup process, it may interrupt the establishment of communication between the controller 10 and the specific operation target 301. Also, if there is a malfunction in the startup process, the controller 10 may restart the startup process from the beginning (for example, from step S111). A specific example of a malfunction in the startup process is, for example, that the device targeted by the startup process does not operate. The device targeted by the startup process may be, for example, the wiper described above, a light, or another device.

[0176] [Modification 4] The remote operation support method according to this embodiment includes: the storage device 202 storing a communication specification for each of the multiple operation targets 300 in association with the controller 10 for communicating with the operation target 300; the controller 10 recognizing a specific operation target 301 among the multiple operation targets 300 that the controller 10 should communicate with; the controller 10 recognizing the communication specification associated with the specific operation target 301 stored in the storage device 202 as a specific communication specification; and the controller 10 communicating with the specific operation target 301 using the specific communication specification. In this method, since the storage device 202 stores a communication specification in association with each of the multiple operation targets 300, the controller 10 can recognize the specific communication specification, which is the communication specification for the specific operation target 301 that the controller 10 should communicate with, and can communicate with the specific operation target 301 using the recognized specific communication specification. Therefore, even if the communication specifications of the multiple target devices 300 differ from each other, the controller 10 can communicate with each target device 300 using a communication specification that is suitable for each of the multiple target devices 300.

[0177] [Other Modifications] Although embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above and further includes, for example, the following modifications.

[0178] (A) Regarding the object to be operated, in the specific example shown in Figure 3, the object to be operated 300 is a shovel, but the work machine in this disclosure may be other work machines such as a bulldozer or a crane. Also, in the above embodiment, each of the multiple objects to be operated 300 is a work machine, but the object to be operated in this disclosure may be a vehicle such as a transport vehicle, an unmanned aerial vehicle such as a drone, or any other object to be operated. The transport vehicle may be a dump truck, for example.

[0179] Furthermore, in the specific example shown in Figure 4, the target device 300 is equipped with devices such as the target device controller 30 and the information acquisition device 34, but it does not have to be equipped with any of the devices shown in Figure 4, and it may also be equipped with devices not shown in Figure 4. Even in this case, since the data registered in the storage device 202 based on the service operator's operation includes data of the devices equipped in the target device 300, the controller 10 of the remote control device 100 can transmit an operation command to the specific target device 301 that matches the configuration of the devices of the specific target device 301 based on the data registered in the storage device 202.

[0180] (B) Regarding the remote control system, the remote control system 400 according to the above embodiment comprises a remote control device 100, a server 200, and a plurality of control targets 300. However, the remote control system 400 does not necessarily have to include a server 200. In this case, the storage device 202 may be provided in the remote control device 100, for example. Alternatively, the storage device 202 may be configured as a storage device in the controller of the control targets 300 (for example, the fourth controller 30D). The remote control system 400 comprises at least one of the remote control device 100 and the server 200, and does not necessarily have to include the plurality of control targets 300. That is, the plurality of control targets 300 do not necessarily constitute a part of the remote control system 400.

[0181] (C) Controller In the above embodiment, the remote control device 100 includes a controller 10, but the controller in this disclosure may be provided in a management device such as a server, or in an information processing terminal other than a server. The information processing terminal may be, for example, a tablet computer, a smartphone, a laptop personal computer, a desktop personal computer, or any other type of information processing terminal.

[0182] (D) With respect to the storage device, in the above embodiment the server 200 is equipped with a storage device 202, but the storage device in this disclosure may be equipped in the remote control device 100 as described above, or in an information processing terminal other than the server. Examples of such information processing terminals include the various computers described above.

[0183] (E) Regarding the information acquisition device, in the above embodiment the object to be operated 300 is equipped with an imaging device such as a camera, which is an example of an information acquisition device. However, the information acquisition device does not necessarily have to be mounted on the object to be operated 300, and may be located at a location away from the object to be operated 300 in the same work site as the object to be operated 300.

[0184] (F) The remote control device of the present disclosure is configured to be used in the remote control system 400 according to the embodiment, and may include either or both of the controller 10 and the storage device 202.

[0185] (G) The management device of the present disclosure is configured to be used in the remote control system 400 according to the embodiment and may include one or both of the controller 10 and the storage device 202.

[0186] (H) In the above embodiment, the storage device 202 is configured to store in association with each of the multiple operation targets 300 and the device communication specification, which is the communication specification of the information acquisition device 34 that acquires information about the conditions of the work site where the operation target 300 is located. However, when the storage device 202 stores the operation target 300 in association with the device communication specification, the operation target 300 does not necessarily have to be located at the work site. That is, the storage device 202 may be configured to store in association with each of the multiple operation targets 300 and the device communication specification, which is the communication specification of the information acquisition device 34 that acquires information about the conditions of the work site where the operation target 300 is scheduled to be located.

[0187] As described above, the present disclosure provides a technology that enables the controller to communicate with each of the multiple targets using a communication specification that is suitable for each of the multiple targets, even when the communication specifications of the multiple targets differ from one another.

[0188] A remote control system according to the first embodiment comprises a controller and a storage device, the storage device being configured to store in association with each of a plurality of target devices and a communication specification for the controller to communicate with that target device, the controller being configured to recognize a specific target device among the plurality of target devices that is to be communicated with, recognize the communication specification stored in the storage device in association with the specific target device as a specific communication specification, and communicate with the specific target device using the specific communication specification.

[0189] In this first embodiment, the storage device is configured to store each of the plurality of targets and its communication specifications in association with each of them, so that the controller can recognize the specific communication specifications, which are the communication specifications of the specific target that the controller should communicate with, and can communicate with the specific target using the recognized specific communication specifications. Therefore, even if the communication specifications of the plurality of targets are different from each other, the controller can communicate with each of the plurality of targets using the communication specifications that are suitable for each of them.

[0190] A remote control system according to a second embodiment preferably comprises the following further configuration in addition to the remote control system according to the first embodiment. That is, in a remote control system according to a second embodiment, the communication specification for each of the plurality of targets to be operated preferably includes a communication establishment specification for the controller to establish communication with that target to be operated, and the controller is preferably configured to perform a communication establishment process for establishing communication with a specific target to be operated using the communication establishment specification included in the specific communication specification. In this second embodiment, the controller can establish communication with each of the plurality of targets to be operated using a communication establishment specification that is suitable for each of those targets to be operated.

[0191] A remote control system according to a third embodiment preferably further comprises the following configuration in addition to the remote control system according to the first or second embodiment. That is, in a remote control system according to a third embodiment, the storage device is configured to store each of the plurality of target devices in association with a start specification for the controller to start that target device, and the controller preferably recognizes the start specification stored in the storage device in association with a specific target device as a specific start specification, and after communication between the controller and the specific target device is established, executes a start process, which is a process for starting the specific target device, using the specific start specification. In this third embodiment, even if the start specifications for the plurality of target devices are different from each other, the controller can start the specific target device using a start specification that is suitable for that specific target device after communication with that specific target device is established.

[0192] The remote control system according to the fourth embodiment preferably further comprises the following configuration in the remote control system according to any one of the first to third embodiments. That is, in the remote control system according to the fourth embodiment, the communication specification for each of the plurality of targets includes format information relating to the format of the data signal for which the controller transmits a data signal to the target, and format information associated with the target and stored in the storage device, and the controller is preferably configured to recognize the format associated with a specific target and stored in the storage device as a specific format, convert the format of the data signal to be transmitted to the specific target to the specific format, and transmit the converted data signal to the specific target. In this fourth embodiment, even if the data signal formats of the plurality of targets are different from each other, each of the plurality of targets can receive the data signal that has been converted to the specific format by the controller and transmitted from the controller, so that each of the plurality of targets can use the data signal to control the operation of its target without changing the format of the data signal. This eliminates the need to add equipment or modify equipment to change the format for each of the multiple targets mentioned above, resulting in cost advantages.

[0193] A remote control system according to the fifth embodiment preferably further comprises the following configuration in a remote control system according to any one of the first to fourth embodiments. That is, in a remote control system according to the fifth embodiment, the storage device is configured to store each of the plurality of targets for operation in association with a device communication specification, which is the communication specification of an information acquisition device that acquires information about the conditions of the work site where the target for operation is located, and the controller preferably performs a communication establishment process for establishing communication with the information acquisition device using the device communication specification. In this fifth embodiment, since the storage device is configured to store each of the plurality of targets for operation in association with the device communication specification of the information acquisition device, the controller can establish communication with the information acquisition device using the device communication specification of the information acquisition device stored in association with the specific target for operation, that is, the device communication specification that is compatible with the information acquisition device located in the same work site as the work site where the specific target for operation is located.

[0194] A remote control system according to the sixth embodiment may further include the following configuration in a remote control system according to any one of the first to fifth embodiments. That is, in a remote control system according to the sixth embodiment, the controller preferably determines whether or not there is a malfunction in the startup process, and if there is a malfunction in the startup process, it is preferable to interrupt the establishment of communication between the controller and the specific target of operation.

[0195] A remote control system according to the seventh embodiment may further include the following configuration in a remote control system according to any one of the first to sixth embodiments. That is, the remote control system according to the seventh embodiment further includes a designated operation input device configured to receive designated operations by personnel involved in the work, and it is preferable that the controller recognizes the target of operation specified by the designated operation received by the designated operation input device as the specific target of operation from among the plurality of target of operation. In this seventh embodiment, the controller can select the target of operation specified by the designated operation performed by the personnel involved in the work as the specific target of operation from among the plurality of target of operation.

[0196] The remote control system according to the eighth embodiment may further include the following configuration in a remote control system according to any one of the first to seventh embodiments. That is, the remote control system according to the eighth embodiment further includes a designated operation input device configured to receive a designated operation (first designated operation) by a person in charge of work, and when the designated operation input device receives a second designated operation by a person in charge of work while communication between the controller and the specific operation target (first specific operation target) is established, it is preferable that the controller recognizes the operation target designated by the second designated operation as the second specific operation target from among the plurality of operation targets. In this eighth embodiment, even when communication between the controller and the first specific operation target is established, the controller can select the operation target designated by the second designated operation performed by the person in charge of work as the second specific operation target from among the plurality of operation targets. In this case, the controller may execute a communication establishment process to establish communication with the second specific operation target at the time it recognizes the second specific operation target. Furthermore, the controller may perform a communication establishment process to establish communication with the second specific target when the remote operation of the first specific target is completed.

[0197] The remote control device according to the ninth embodiment is configured to be used in a remote control system according to any one of the first to eighth embodiments, and comprises either or both of the controller and the storage device. This remote control device according to the ninth embodiment constitutes at least a part of a remote control system according to any one of the first to eighth embodiments.

[0198] The management device according to the tenth embodiment is configured to be used in a remote control system according to any one of the first to eighth embodiments, and comprises either or both of the controller and the storage device. This management device according to the tenth embodiment constitutes at least a part of a remote control system according to any one of the first to eighth embodiments.

[0199] The remote operation support method according to the eleventh embodiment includes: a storage device storing a communication specification for each of a plurality of operation targets in association with the controller for communication with that operation target; the controller recognizing a specific operation target among the plurality of operation targets that the controller should communicate with; the controller recognizing a communication specification stored in the storage device in association with the specific operation target as a specific communication specification; and the controller communicating with the specific operation target using the specific communication specification. In this eleventh embodiment, since the storage device stores each of the plurality of operation targets in association with the communication specification for that operation target, the controller can recognize the specific communication specification which is the communication specification for the specific operation target that the controller should communicate with, and can communicate with the specific operation target using the recognized specific communication specification. Therefore, even if the communication specifications for the plurality of operation targets are different from each other, the controller can communicate with each of the plurality of operation targets using a communication specification that is suitable for each of them.

Claims

1. A remote control system comprising a controller and a storage device, wherein the storage device is configured to store in association with each of a plurality of target devices and a communication specification for the controller to communicate with that target device, and the controller is configured to recognize a specific target device among the plurality of target devices that is to communicate with, recognize the communication specification stored in the storage device in association with the specific target device as a specific communication specification, and communicate with the specific target device using the specific communication specification.

2. The remote control system according to claim 1, wherein each of the plurality of targets for operation has a communication specification for the controller to establish communication with that target for operation, and the controller is configured to perform a communication establishment process for establishing communication with a specific target for operation using the communication establishment specification included in the specific communication specification.

3. The remote control system according to claim 1 or 2, wherein the storage device is configured to store in association with each of the plurality of targets for operation and a start specification for the controller to start the target for operation, the controller recognizes the start specification stored in the storage device in association with a specific target for operation as a specific start specification, and after communication between the controller and the specific target for operation is established, executes a start process, which is a process for starting the specific target for operation, using the specific start specification.

4. The remote control system according to any one of claims 1 to 3, wherein the communication specifications for each of the plurality of targets include format information relating to the format of the data signal for which the controller transmits a data signal to the target, and format information associated with the target and stored in the storage device, and the controller is configured to recognize a format associated with a specific target and stored in the storage device as a specific format, convert the format of the data signal to be transmitted to the specific target to the specific format, and transmit the converted data signal to the specific target.

5. The remote control system according to any one of claims 1 to 4, wherein the storage device is configured to store in association with each of the plurality of objects to be operated and a device communication specification which is the communication specification of an information acquisition device that acquires information about the conditions of the work site where the object to be operated is located, and the controller performs a communication establishment process for establishing communication with the information acquisition device using the device communication specification.

6. The remote control system according to claim 3, wherein the controller determines whether or not there is a malfunction in the startup process, and if there is a malfunction in the startup process, it interrupts the establishment of communication between the controller and the specific target of operation.

7. The remote control system according to any one of claims 1 to 6, further comprising a designated operation input device configured to receive designated operations by personnel involved in the work, wherein the controller recognizes the target of operation specified by the designated operation received by the designated operation input device as the specific target of operation from among the plurality of target of operation.

8. The remote control system according to any one of claims 1 to 7, further comprising a designated operation input device configured to receive designated operations by personnel involved in the work, wherein when the designated operation input device receives a second designated operation by personnel involved in the work while communication between the controller and the specific target of operation is established, the controller recognizes the target of operation designated by the second designated operation as the second specific target of operation from among the plurality of target of operation.

9. A remote control device configured to be used in the remote control system described in any one of claims 1 to 8, comprising either or both of the controller and the storage device.

10. A management device configured to be used in the remote control system described in any one of claims 1 to 8, comprising either or both of the controller and the storage device.

11. A remote operation support method comprising: a storage device storing a communication specification for each of a plurality of operation targets and a controller for communicating with that operation target in association with each of the operation targets; the controller recognizing a specific operation target among the plurality of operation targets that the controller should communicate with; the controller recognizing a communication specification stored in the storage device in association with the specific operation target as a specific communication specification; and the controller communicating with the specific operation target using the specific communication specification.

Images