Remote operating system

Abstract

Provided is a remote operation system capable of shortening the time from temporarily stopping the remote operation to resuming the remote operation. A remote control device (200) of a remote operation system (1) executes a restart process or an end process of a remote operation when an operator (2) executes a real machine stop operation during execution of the remote operation and the operator (2) executes a designation operation for restart or end of the remote operation with a hand terminal (230). The remote control device (200) and the real machine control device (400) maintain the communication connection state during a period from execution of the real machine stop operation to execution of the specified operation.

Description

[Technical field]

[0001] The invention relates to a remote operation system for an operator to remotely operate a working machine. [Background Technology]

[0002] Conventionally, as a remote operation system for a working machine, there is known a system described in Patent Document 1. The remote operation system includes a construction machine as the working machine and a transmission device, and the construction machine includes a receiving device. In the remote operation system, the start and stop of the working machine are remotely controlled by an operator operating the transmission device.

[0003] [Prior art literature]

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 10-219748 [Summary of the invention]

[0006] Problems to be solved by the invention

[0007] In recent years, in remote operation systems, a communication connection operation is usually performed when starting remote operation of a working machine. In such a remote operation system, when the remote operation system of Patent Document 1 is applied, a communication connection operation needs to be performed every time the working machine is restarted after temporarily stopping the working machine. As a result, there is a problem that it takes time from temporarily stopping the remote operation to restarting the remote operation.

[0008] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a remote operation system capable of shortening the time from temporarily stopping the remote operation to restarting the remote operation.

[0009] Means for solving problems

[0010] In order to achieve the above-mentioned purpose, the remote operation system of the present invention is used for an operator to remotely operate a working machine via a remote operation device having a remote input unit, and is characterized in that it comprises: a remote control element, which, when the operator performs an actual machine stop operation of the working machine via the remote operation device during the execution of the remote operation, performs a restart process or an end process of the remote operation when the operator performs a designated operation for restarting or ending the remote operation in the remote input unit; and a communication control element, which, when the remote operation starts, establishes a communication connection state between the remote operation device and the working machine to enable remote operation of the working machine, and maintains the communication connection state during the period from the execution of the actual machine stop operation to the execution of the designated operation. [Drawings]

[0011] Figure 1 This is a diagram showing the structure of a remote operation system according to one embodiment of the present invention.

[0012] Figure 2 is a block diagram showing the electrical structure of a remote operation system.

[0013] Figure 3 It is a diagram showing the structure of the remote control device on a plane.

[0014] Figure 4 It is a figure which shows the structure of a working machine.

[0015] Figure 5 : is a sequence diagram showing the control process when the remote operation starts.

[0016] Figure 6 : is a flowchart showing the startup preparation process.

[0017] Figure 7 : is a flowchart showing the authentication result confirmation process.

[0018] Figure 8 2 is a flowchart showing the real machine connection preparation process.

[0019] Fig. 9 It is a diagram showing an authentication confirmation screen.

[0020] Fig.10 This is a diagram showing an authentication success screen.

[0021] Fig.11 It is a diagram showing an example of a job selection screen.

[0022] Fig.12 This is a diagram showing an example of a machine selection screen.

[0023] Fig.13 It is a figure which shows an example of a confirmation screen.

[0024] Fig.14 It is a diagram showing an example of a confirmation instruction screen.

[0025] Fig.15 is a flowchart showing the job selection process.

[0026] Fig.16 is a flowchart showing the post-stop action selection process.

[0027] Fig.17 : is a timing chart showing the control process when the engine stop signal is output.

[0028] Fig.18 : is a timing chart showing the control process when the remote operation end signal is output.

[0029] Fig.19 It is a figure which shows the work completion screen.

[0030] Fig. 20 It is a diagram showing an engine operation selection screen.

[0031] Fig.21 It is a figure which shows the cutting screen.

[0032] Fig. 22 It is a figure which shows the post-stop action selection screen. [Specific implementation method]

[0033] A remote operating system according to an embodiment of the present invention will be described with reference to the accompanying drawings. Figure 1 As shown, the remote operation system 1 of the present embodiment is composed of a remote operation support server 10, a plurality of remote operation devices 20 (only one is shown), a plurality of working machines 40 (only two are shown), and the like.

[0034] In the remote operation system 1 , an operator 2 operates any one of a plurality of remote operation devices 20 to remotely operate any one of a plurality of working machines 40 .

[0035] In the remote operation system 1, the remote operation support server 10, the plurality of remote operation devices 20, and the plurality of working machines 40 are configured to be able to communicate with each other via a network. The communication network between the remote operation support server 10 and the remote operation device 20 and the communication network between the remote operation support server 10 and the working machine 40 may be the same or different.

[0036] like Figure 2 As shown, the remote operation support server 10 includes a server operation device 11, a storage device 12, and a server wireless communication device 13. The server operation device 11 is composed of a CPU, etc., and reads necessary data and software from the storage device 12 to execute various operation processes and various control processes.

[0037] The storage device 12 is composed of a memory, a memory, and the like, and stores and saves data such as an operation schedule of the working machine 40 and a work log of the operator 2 .

[0038] In the remote operation support server 10, the server wireless communication device 13 performs communication via the communication network between the remote wireless communication device 260 (described later) of the remote operation device 20 and the real machine wireless communication device 470 of the working machine 40. In addition, in the present embodiment, the remote wireless communication device 260 corresponds to the device-side communication device, and the real machine wireless communication device 470 corresponds to the real machine-side communication device.

[0039] like Figure 2and Figure 3 As shown, the remote control device 20 includes a remote control device 200, a remote operation mechanism 211, an image output device 221, an operator identification device 223, a hand terminal 230, a key switch 240, a speaker 250, and a remote wireless communication device 260. The remote control device 200 is a device composed of a plurality of controllers, each of which is composed of a CPU, a storage device (memory, RAM, ROM), and an input / output interface (none of which are shown in the figure). The remote control device 200 reads the required data and software programs from the storage device and performs various control processes as described later.

[0040] like Figure 2 As shown, the remote control device 200 has functions as a remote control element 201 , a communication control element 202 , an apparatus-side communication control element 203 , and an output control element 204 , and these functions are realized by the remote control device 200 executing a software program.

[0041] The remote control mechanism 211 includes a travel control device, a rotation control device, an arm control device, an arm control device, and a bucket control device. Each control device has an operating lever for receiving a rotation operation. The operating lever (travel lever) of the travel control device can be operated to move the lower mobile body 410 of the working machine 40.

[0042] The operating lever (rotating lever) of the swing operating device can be operated to actuate the swing mechanism 430 of the working machine 40. The operating lever (boom lever) of the boom operating device can be operated to actuate the boom cylinder 442 of the working machine 40. The operating lever (arm lever) of the arm operating device can be operated to actuate the arm cylinder 444 of the working machine 40. The operating lever (bucket lever) of the bucket operating device can be operated to actuate the bucket cylinder 446 of the working machine 40.

[0043] For example, Figure 3 As shown, the operating levers constituting the remote operating mechanism 211 are arranged around a seat 231 for the operator to sit on. In front of the seat 231, a pair of left and right travel levers 2110 corresponding to the left and right crawlers are arranged in a horizontal manner. One operating lever may also serve as multiple operating levers.

[0044] For example, it could also be, Figure 3 The left operating lever 2111 shown in the figure, which is arranged in front of the left side frame of the seat 231, functions as a boom lever when operated in the front-back direction, and functions as a swing lever when operated in the left-right direction. Figure 3The right operating lever 2112 provided in front of the right side frame of the seat 231 shown functions as a boom lever when operated in the front-rear direction, and functions as a bucket lever when operated in the left-right direction. The lever mode can also be arbitrarily changed according to the operator's operation instruction.

[0045] The blocking lever 2113 is arranged below the left operating lever 2111 in front of the left side frame of the seat 231. The blocking lever 2113 functions as an operating lever for the following actions: when lifted, it is locked so that the working machine 40 will not move even if the operating levers 2110, 2111, 2112 are operated, and on the other hand, the lock is released when lowered.

[0046] Specifically, the blocking lever 2113 is configured to be operable between an upper locking position and a release position below the blocking lever 2113, and when the blocking lever 2113 is pulled up from the release position to the locking position, the working machine 40 is locked, so that the working machine 40 will not move even if the operating levers 2110, 2111, and 2112 are operated. On the other hand, when the blocking lever 2113 is pulled down from the locking position to the release position, the lock is released.

[0047] The remote operating mechanism 211 is provided with a plurality of sensors (not shown) which output detection signals indicating the operation status thereof to the remote control device 200 when the operation levers of the remote operating mechanism 211 are operated. The remote control device 200 obtains the operation status of the remote operating mechanism 211 by the operator 2 based on the detection signals of the sensors.

[0048] like Figure 3 As shown, the image output device 221 is composed of a central image output device 2210, a left image output device 2211, and a right image output device 2212, each of which has a substantially rectangular screen and is disposed in front of, obliquely left in front of, and obliquely right in front of the seat 231. The shapes and sizes of the screens (image display areas) of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be the same or different.

[0049] like Figure 3 As shown, the screen of the central image output device 2210 and the screen of the left image output device 2211 are inclined at an angle θ1 (for example, 120°≤θ1≤150°), and the right edge of the left image output device 2211 is adjacent to the left edge of the central image output device 2210. Figure 3As shown, the left edge of the right image output device 2212 is adjacent to the right edge of the central image output device 2210 in such a manner that the screen of the central image output device 2210 and the screen of the right image output device 2212 are inclined at an angle θ2 (e.g., 120°≤θ2≤150°). The inclination angles θ1 and θ2 may be the same or different.

[0050] The screens of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be parallel to the vertical direction or inclined to the vertical direction. At least one of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be composed of a plurality of image output devices divided into a plurality of image output devices. For example, the central image output device 2210 may be composed of a pair of image output devices adjacent to each other in the vertical direction and having a substantially rectangular screen.

[0051] When the remote wireless communication device 260 receives the imaging data signal from the working machine 40 , the remote control device 200 causes the image output device 221 to display the imaging data included in the imaging data signal.

[0052] The handy terminal 230 (remote input unit) is a touch panel terminal, and as described later, displays various screens based on signals from the remote control device 200 , and outputs signals indicating the situation to the remote control device 200 when the operator 2 performs a touch operation.

[0053] The operator identification device 223 is a device for obtaining an image, voice, or personal identification number of the operator 2 to identify the individual operator. The operator identification device 223 is provided at the front portion of the remote operation mechanism 211 and is arranged backward toward the seat 231. After the remote control device 200 is started, the operator identification device 223 takes a picture of the operator 2 sitting on the seat 231. As described later, the image taken by the operator identification device 223 is obtained as authentication data and is sent to the remote operation assistance server 10 via the network. Thus, the remote operation assistance server 10 compares the authentication data as the information of the registered operator 2 stored in the storage device 12 of the remote operation assistance server 10, and identifies the individual operator through facial authentication.

[0054] In addition, the operator identification device 223 may be an operator information input device that is provided in the remote control device 200 instead of the camera and inputs the personal identification number of the operator 2, or may be both the camera and the operator information input device. The operator information input device may be a device that can input information via the handheld terminal 230, or may be a voice recognition device that has a microphone and recognizes the voice of the operator 2 itself or a password determined for each operator.

[0055] The key switch 240 is used to start the engine 460 of the working machine 40 or stop the operation of the engine 460, and is arranged near the remote operating mechanism 211. The key switch 240 is configured to be able to change its position to at least three of the off position, the on position and the start position, and when operated by the operator 2, a signal indicating the position of the key switch 240 (off position signal, on position signal and start position signal) is output to the remote control device 200.

[0056] The remote control device 200 transmits a signal instructing the start and stop of the engine 460 to the real machine wireless communication device 470 of the working machine 40 via the remote wireless communication device 260 based on the operation signal from the key switch 240 .

[0057] The speaker 250 is provided near the seat 231. When receiving an audio data signal from a working machine 40 described later via the remote wireless communication device 260, the remote control device 200 outputs the audio data included in the audio data signal from the speaker 250.

[0058] like Figure 2 As shown, the working machine 40 includes a machine control device 400 , a drive mechanism 450 , an engine 460 , a camera 461 , a microphone 462 , a machine wireless communication device 470 , and the like.

[0059] The real machine control device 400 is a device composed of a plurality of controllers, each of which is composed of a CPU, a storage device (memory, RAM, ROM), and an input / output interface, etc. (all not shown). The real machine control device 400 reads the required data and software programs from the storage device and performs various control processes as described below.

[0060] like Figure 2 As shown, the actual machine control device 400 has functions as a communication control element 401 and an actual machine-side communication control element 402 , and these functions are realized by the actual machine control device 400 executing a software program.

[0061] The working machine 40 is a construction machine of the crawler excavator type, such as Figure 4As shown, the crawler-type lower moving body 410 and the upper rotating body 420 rotatably mounted on the lower moving body 410 via a rotating mechanism 430 are provided.

[0062] A control room 424 (operator's cab) is provided at the front left side portion of the upper swing body 420. A working attachment 440 is provided at the front center portion of the upper swing body 420.

[0063] The working attachment 440 includes a boom 441 which is movably mounted on the upper rotating body 420, an arm 443 which is rotatably connected to the front end of the boom 441, and a bucket 445 which is rotatably connected to the front end of the arm 443. The working attachment 440 is mounted with a boom cylinder 442, an arm cylinder 444, and a bucket cylinder 446 which are composed of telescopic hydraulic cylinders.

[0064] The boom cylinder 442 is interposed between the boom 441 and the upper slewing body 420, and is extended and retracted by receiving the supply of hydraulic oil so that the boom 441 rotates in the up-and-down direction. The arm cylinder 444 is interposed between the arm 443 and the boom 441, and is extended and retracted by receiving the supply of hydraulic oil so that the arm 443 rotates around the horizontal axis relative to the boom 441. The bucket cylinder 446 is interposed between the bucket 445 and the arm 443, and is extended and retracted by receiving the supply of hydraulic oil so that the bucket 445 rotates around the horizontal axis relative to the arm 443.

[0065] The control room 424 is provided with a real machine operating mechanism (not shown) and a drive mechanism 450. The real machine operating mechanism includes a plurality of real machine operating levers arranged around a seat arranged inside the control room 424, similarly to the remote operating mechanism 211. The plurality of real machine operating levers are driven by the drive mechanism 450.

[0066] The real machine control device 400 controls the operating state of the driving mechanism 450 so that, when a signal indicating the operating state of the remote operating mechanism 211 is received from the real machine wireless communication device 470 , the real machine control device 400 drives a plurality of real machine operating levers of the real machine operating mechanism based on the signal.

[0067] The engine 460 is an internal combustion engine, and is provided with a fuel injection valve, etc. The actual machine control device 400 controls the operation state of the engine 460 by controlling the operation state of the fuel injection valve, as described later. In addition, the engine 460 is not limited to an internal combustion engine, and an electric motor or an engine combining an electric motor and an internal combustion engine may be used.

[0068] Generally speaking, in the case of a conventional crawler excavator type construction machine that is not remotely controlled, an operator sitting in a seat in a cab changes the position of an electric key switch mounted next to the seat to an off position, an on position, and a start position, thereby operating the electronic equipment and engine of the construction machine.

[0069] Specifically, when the key switch is in the off position, no power is supplied to the electronic device. By turning the key switch in the off position to the on position, power is supplied to the electronic device of the construction machine, and the electronic device is started. Power is also supplied to the circuit for starting the engine, and the engine is put into a standby state for starting the engine. Furthermore, by turning the key switch to the start position, the engine is started. After the engine is started, the key switch in the start position is maintained in the on position.

[0070] In contrast, the working machine 40 of this embodiment is a construction machine based on remote operation. Therefore, unlike a common crawler excavator type construction machine, the operator 2 changes the position of the key switch 240 of the remote control device 200 to operate the electronic equipment and the engine 460 of the working machine 40.

[0071] Specifically, when the key switch 240 is in the off position, no power is supplied to the electronic device. The electronic device here refers to the electronic device other than the actual machine control device 400, the drive mechanism 450, the camera 461, and the microphone 462 required for remote operation, and is also equipped in a typical crawler excavator type construction machine.

[0072] The key switch 240 is changed from the off position to the on position, thereby supplying power to the electronic equipment of the working machine 40, and the electronic equipment is started. Power is also supplied to the circuit for starting the engine 460, and the engine 460 is put into a standby state for starting the engine 460. Furthermore, by setting the key switch 240 to the start position, the engine 460 is started.

[0073] The photographing device 461 is a camera-type device installed inside the control room 424, and photographs the environment including the front of the control room 424 through the front window and a pair of left and right side windows, and outputs a photographing data signal containing the photographing data to the real machine control device 400. The real machine control device 400 sends the photographing data signal to the remote wireless communication device 260 via the real machine wireless communication device 470.

[0074] Microphone 462 outputs a sound data signal including the sound around working machine 40 as sound data to actual machine control device 400 . Actual machine control device 400 transmits the sound data signal to remote wireless communication device 260 via actual machine wireless communication device 470 .

[0075] Next, refer to Figure 5 The various control processes at the start of remote operation are described below. Figure 5 The various control processes shown are executed by at least one of the server operation device 11 in the remote operation support server 10, the remote control device 200 in the remote operation device 20, and the real machine control device 400 in the working machine 40. This also applies to the various control processes described later.

[0076] As shown in the figure, first, in the remote operation device 20, the startup preparation process ( Figure 5 / STEP1).

[0077] Specifically, the startup preparation process is as follows Figure 6 First, start the remote control device 20 ( Figure 6 / STEP 21). Specifically, the operator 2 turns on a power button (not shown) of the remote operating device 20, thereby starting up the remote operating device 20.

[0078] Next, the terminal 230 at hand displays Fig. 9 The authentication confirmation screen shown ( Figure 6 / STEP 22). On the authentication confirmation screen, a message "Do you want to be authenticated?" is displayed on the upper side, and an authentication button 500 is displayed on the lower side.

[0079] Next, it is determined whether the authentication start action has been performed ( Figure 6 / STEP 23) In this case, when the authentication button 500 is pressed by the operator 2, it is determined that the authentication start operation has been performed, and otherwise, it is determined that the authentication start operation has not been performed.

[0080] In the case where the judgment is negative ( Figure 6 / STEP23…NO), the process returns to STEP22. On the other hand, if the determination is positive ( Figure 6 / STEP23…YES) and the authentication start action is executed, the authentication data acquisition process is executed ( Figure 6 / STEP24) In the authentication data acquisition process, the authentication data for identifying the individual operator is acquired by the operator identification device 223 described above.

[0081] Next, an authentication signal ( Figure 6 / STEP25). After that, this process ends.

[0082] return Figure 5As described above, after the remote operation device 20 executes the startup preparation process, when the remote operation assistance server 10 receives the authentication signal, the remote operation assistance server 10 executes the user authentication process ( Figure 5 / STEP2).

[0083] In this user authentication process, after user authentication is performed based on the authentication data included in the authentication signal and the authentication data registered in the remote operation assistance server 10 , an authentication result signal is output to the remote operation device 20 .

[0084] In this case, the authentication result signal is configured to include authentication result data indicating the situation when user authentication is successful and work content data of the work plan of the operator 2 based on the successful user authentication, and is configured to include only authentication result data indicating the situation when user authentication fails.

[0085] As described above, after the remote operation support server 10 executes the user authentication process, when the remote operation device 20 receives the authentication result signal, the remote operation device 20 executes the authentication result confirmation process ( Figure 5 / STEP3).

[0086] Specifically, the authentication result confirmation process is as follows: Figure 7 As shown in the figure, first, based on the authentication result data in the authentication result signal, it is determined whether the user authentication of operator 2 is successful ( Figure 7 / STEP30).

[0087] In the case where the judgment is negative ( Figure 7 / STEP30…NO), and the user authentication of the operator 2 fails, a re-authentication process is executed. In this re-authentication process, a pattern indicating that the user authentication failed and a re-authentication button for re-authentication (neither of which is shown in the figure) are displayed on the hand terminal 230. Then, when the operator 2 presses the re-authentication button, the above-mentioned Figure 6 After the re-authentication process is executed as described above, this process ends.

[0088] On the other hand, in the case where the above determination is affirmative ( Figure 7 / STEP30…YES), and if the user authentication of the operator 2 is successful, the terminal 230 is displayed. Fig.10 The authentication success screen shown ( Figure 7 / STEP32) In the case of the authentication success screen, the text "Face Authentication" and a face text mark 502 indicating that the face authentication is successful are displayed on the upper side, and an OK button 503 is displayed on the lower side.

[0089] Next, determine whether the understanding action has been implemented ( Figure 7 / STEP 33) In this case, when the OK button 503 is pressed, it is determined that the understanding action is performed, and otherwise, it is determined that the understanding action is not performed.

[0090] In the case where the judgment is negative ( Figure 7 / STEP33…NO), the process returns to STEP32. On the other hand, if the determination is positive ( Figure 7 / STEP33 ... YES), and the OK button 503 is pressed, the work plan of the operator 2 stored in the storage device 12 of the remote operation support server 10 is displayed on the hand terminal 230. Fig.11 The job selection screen shown in the figure ( Figure 7 / STEP34).

[0091] In the case of the work selection screen, work selection buttons 504 to 506 showing three work content data that can be performed by the operator 2 are displayed, and a button 507 for returning to the previous screen is displayed at the bottom. In each frame of the work selection buttons 504 to 506, the location of the work site where the work is performed, the work content, and the work schedule are displayed as work content data.

[0092] In addition, Fig.11 In the example of the work selection screen shown, three work selection buttons 504 to 506 are displayed. However, depending on the work plan of the operator 2, two or less or four or more selection buttons for work content data may be displayed.

[0093] Next, it is determined whether the job selection has been performed ( Figure 7 / STEP35) In this case, when the operator 2 presses any one of the operation selection buttons 504 to 506, it is determined that the operation selection is performed, and otherwise, it is determined that the operation selection is not performed. Fig.11 The case where the operator 2 presses the job selection button 504 is shown.

[0094] In the case where the judgment is negative ( Figure 7 / STEP35…NO), the process returns to STEP34. On the other hand, if the determination is positive ( Figure 7 / STEP35…YES), and the job selection button is pressed, the terminal 230 at hand displays Fig.12 The machine selection screen shown in the figure ( Figure 7 / STEP36). Although not shown in the figure, if the button 507 is pressed, the process returns to STEP32.

[0095] In the case of this machine selection screen, machine selection buttons 508 to 510 indicating the contents of three work machines 40 that can be used for the work selected by the operator 2 and can be remotely operated are displayed, and a button 511 for returning to the work selection is displayed at the bottom.

[0096] For example, in the job selection ( Figure 7 / STEP35), when the excavation work to be performed in the work area A is selected, the work machine 40 existing in the work area A and having a bucket for excavation and capable of remote operation is displayed as a machine selection button, and on the other hand, the work machine 40 not existing in the work area A or the work machine 40 without a bucket is not displayed. The above processing is realized in the following manner: the work machine 40 has a unit for detecting the position such as GNSS, and the position information of the work machine 40 is received via the network, or when the specification information of the work machine 40 is changed, the change information is uploaded to the storage device 12 via the network.

[0097] In addition, Fig.12 In the example of the work selection screen shown, three machine selection buttons 508 to 510 are displayed, but two or less or four or more machine selection buttons may be displayed depending on the status of the remotely operable work machine.

[0098] Next, it is determined whether mechanical selection has been implemented ( Figure 7 / STEP37) In this case, when the operator 2 presses any one of the machine selection buttons 508 to 510, it is determined that the machine selection is performed, and otherwise, it is determined that the machine selection is not performed. Fig.12 The case where the machine selection button 509 is pressed by the operator 2 is shown.

[0099] In the case where the judgment is negative ( Figure 7 / STEP37…NO), the process returns to STEP36. On the other hand, if the determination is positive ( Figure 7 / STEP37…YES), and the machine selection button is pressed, the terminal 230 at hand displays Fig.13 The confirmation screen shown in the figure ( Figure 7 / STEP38).

[0100] In the case of this confirmation screen, data 512 indicating the work content selected by the operator 2 is displayed at the top, and data 513 indicating the work machine selected by the operator 2 is displayed below it. In addition, photo data 526 of the work machine 40 selected by the operator 2 and photo data 527 of the work site where the work machine 40 selected by the operator 2 is located are displayed below the data 513. These photo data 526 and 527 are photo data taken in advance and stored in the storage device 12.

[0101] Furthermore, weather data 528 is displayed below the photo data 526 and 527. In this case, current weather information (weather, temperature, wind direction) and weather forecast information of the work site where the work machine 40 selected by the operator 2 is located are displayed as the weather data 528. The weather data 528 is displayed by obtaining weather data corresponding to the position information of the work machine 40 selected by the operator 2.

[0102] In addition, a return list button 514 for returning to the previous screen is displayed on the left bottom of the confirmation screen, and a remote operation start button 515 is displayed on the right bottom. In this case, the operator 2 continues the remote operation start operation by pressing the remote operation start button 515 when the working machine 40 displayed on the confirmation screen is correct.

[0103] On the other hand, when the work machine 40 displayed on the confirmation screen is an error, the operator 2 can reselect the work machine 40 by pressing the return to list button 514. Although not shown, when the operator 2 presses the return to list button 514, the process returns to STEP 36.

[0104] Next, it is determined whether the remote operation start action is performed ( Figure 7 / STEP 39) In this case, when the remote operation start button 515 is pressed by the operator 2, it is determined that the remote operation start operation is performed, and otherwise, it is determined that the remote operation start operation is not performed.

[0105] In the case where the judgment is negative ( Figure 7 / STEP39…NO), the process returns to STEP38. On the other hand, if the determination is positive ( Figure 7 / STEP39...YES), and the remote operation start button 515 is pressed, an instruction signal ( Figure 7 / STEP40). After that, this process ends. This instruction signal requests the remote operation assistance server 10 for various communication setting information.

[0106] return Figure 5As described above, after the remote operation device 20 executes the authentication result confirmation process, when the remote operation assistance server 10 receives the instruction signal, the remote operation assistance server 10 executes the communication connection establishment process ( Figure 5 / STEP4).

[0107] In the communication connection establishment process, in order to establish a communication connection state between the remote operation device 20 and the working machine 40 selected by the operator 2 , a device-side communication setting signal is output to the remote operation device 20 , and a real machine-side communication setting signal is output to the working machine 40 .

[0108] The device-side communication setting signal is composed of various communication setting information such as the IP address of the actual machine wireless communication device 470 in the working machine 40 selected by the operator 2. The actual machine wireless communication device 470 can obtain information of the remote wireless communication device 260 that should establish communication as the recipient of various information.

[0109] In addition, the actual machine side communication setting signal is composed of various communication setting information such as the IP address of the remote wireless communication device 260 in the remote operation device 20 operated by the operator 2, and the remote wireless communication device 260 can obtain information of the actual machine wireless communication device 470 that should establish communication as the recipient of various information.

[0110] In this case, there may be multiple remote wireless communication devices 260 that can establish communication with the real machine wireless communication device 470 as the recipient of various information, and there may be multiple real machine wireless communication devices 470 that can establish communication with the remote wireless communication device 260 as the recipient of various information. Therefore, the device-side communication setting signal including various communication setting information such as the IP address to be used to establish communication with the remote wireless communication device 260 as the recipient of various information is output to the remote operation device 20, and the real machine-side communication setting signal including various communication setting information such as the IP address to be used to establish communication with the real machine wireless communication device 470 as the recipient of various information is output to the working machine 40, so that the remote wireless communication device 260 and the real machine wireless communication device 470 can obtain information related to the recipient of various information.

[0111] As described above, after the remote operation assistance server 10 executes the communication connection establishment process, when the remote operation device 20 receives the device-side communication setting signal, the remote operation device 20 executes the communication setting process ( Figure 5 / STEP 5) In this communication setting process, various communication setting information such as the IP address of the actual wireless communication device 470 is applied to the remote control device 200.

[0112] On the other hand, when the working machine 40 receives the real machine side communication setting signal, the working machine 40 executes the communication setting process ( Figure 5 / STEP6). In this communication setting process, various communication setting information such as the IP address of the remote wireless communication device 260 is applied to the actual machine control device 400. As described above, by executing the communication setting process in the remote operation device 20 and the working machine 40, a communication connection state with the working machine 40 is established, thereby enabling communication between the two.

[0113] Thus, in the working machine 40, information such as a shooting data signal including shooting data captured by the shooting device 461 can be sent to the remote wireless communication device 260 as the receiving party, and in the remote operation device 20, information such as operation information including operation information of the rod implemented in the remote operation mechanism 211 can be sent to the real machine wireless communication device 470 as the receiving party.

[0114] That is, when the operator 2 selects the working machine 40, an instruction signal for requesting various communication setting information to the remote operation support server 10 is sent from the remote operation device 20, and based on the instruction signal, various communication setting information such as the IP address is sent from the storage device 12 of the remote operation support server 10 to the real machine wireless communication device 470, and the communication setting process is performed in the remote operation device 20 and the working machine 40, thereby establishing a communication connection state with the working machine 40. Therefore, it takes time from the selection of the working machine 40 by the operator 2 to the establishment of the communication connection state.

[0115] In the remote operation device 20, the above-mentioned communication setting process is executed, and after the communication connection state is established, the real machine connection preparation process ( Figure 5 / STEP7).

[0116] Specifically, the real machine connection preparation process is as follows Figure 8 As shown in the figure, first, the display processing of the confirmation instruction screen is executed ( Figure 8 / STEP50). During the display process of the confirmation instruction screen, the terminal 230 displays Fig.14 The confirmation instruction screen is shown.

[0117] As shown in the figure, a message "Please confirm the main camera image" is displayed on the upper side of the confirmation instruction screen, and two messages "Switch the main camera image" and "Please confirm that it is the camera image of the construction machine you want to operate" are displayed on the lower side. Furthermore, below the above messages, a connection button 516 with a message "Connect to this construction machine" and a construction machine change button 517 with a message "Change construction machine" are displayed in order from top to bottom.

[0118] The connect button 516 is a button for confirming the start of the work in the working machine 40 after the communication connection establishment process is executed by the remote operation support server 10. In addition, the construction machine change button 517 is a button for canceling the work in the working machine 40 and reselecting another working machine 40 before the working machine 40 starts the work after the communication connection establishment process is executed by the remote operation support server 10.

[0119] Although not shown, while the confirmation screen is displayed on the hand terminal 230, in the working machine 40, a photographic data signal including photographic data photographed by the photographing device 461 is output from the real machine control device 400 to the remote operation device 20, and the photographic data photographed by the photographing device 461 is output from the image output device 221. In this case, the operator 2 can confirm whether the working machine 40 selected from the confirmation screen displayed on the hand terminal 230 and the photographic data displayed on the image output device 221 is the machine intended to be remotely operated.

[0120] Operator 2 receives the message and determines whether the shooting data displayed on the image output device 221 is correct. If the working machine 40 is correct, the operator starts remote operation by pressing the connection button 516. On the other hand, if the working machine 40 is wrong, the operator reselects the working machine 40 by pressing the construction machine change button 517.

[0121] Next, the operator 2 determines whether the actual machine connection operation has been performed ( Figure 8 / STEP 51) In this case, when the operator 2 presses the connection button 516 on the confirmation instruction screen, it is determined that the actual device connection operation has been performed, and otherwise, it is determined that the actual device connection operation has not been performed.

[0122] If the determination is negative (STEP 51 ... NO), the process returns to STEP 50. On the other hand, if the determination is positive (STEP 51 ... YES) and the connection button 516 is pressed, a real machine connection signal ( Figure 8 / STEP52). After that, the present process ends. In addition, although not shown in the figure, when the construction machinery change button 517 is pressed, the process returns to Figure 7 / Processing of STEP 36, display Fig.12 The machine selection screen is shown.

[0123] return Figure 5 After the above real machine connection preparation process, execute the connection operation process ( Figure 5 / STEP 8) In this on-operation process, as the operator 2 operates the key switch 240 from the off position to the on position, an on signal indicating this is output to the working machine 40.

[0124] As described above, after the power-on operation process is executed by the remote control device 20, when the power-on signal is received by the working machine 40, the power-on process ( Figure 5 / STEP9) In this power-on process, power is supplied to the electronic device including the circuit for starting the engine 460 mounted on the working machine 40 based on the power-on signal.

[0125] Furthermore, in the remote operation device 20, after the above-mentioned on-operation processing, a job log filling processing ( Figure 5 / STEP 10). In this job log filling process, the operator 2 operates the handy terminal 230 to fill in the job log data, thereby outputting a job log signal including the filled-in job log data to the remote operation support server 10.

[0126] When the remote operation assistance server 10 receives the job log signal, the remote operation assistance server 10 executes the job log update process ( Figure 5 / STEP 11) In this job log update process, the data of the job log of the operator 2 stored in the storage device 12 is updated.

[0127] The job log is configured to include the following information: Figure 7 / STEP30), on the machine selection screen ( Figure 7 / STEP36) The operator 2 who has successfully authenticated the work machine 40 selected in the work selection screen ( Figure 7 / STEP35). In addition, the configuration includes information on the start and end times of the job log update process, the operating status of the work machine 40 during the execution of the job log update process, and the operation information of the remote operation device 20.

[0128] Furthermore, in the remote operation device 20, after the above-mentioned job log filling process, a startup operation process ( Figure 5 / STEP 12) In this start operation process, as the operator 2 operates the key switch 240 from the on position to the start position, a start signal indicating this is output to the working machine 40.

[0129] As described above, after the remote control device 20 executes the start operation process, when the work machine 40 receives the start signal, the work machine 40 executes the engine start process ( Figure 5 / STEP12) In the engine start process, the actual machine control device 400 starts the engine 460 of the working machine 40. Then, when the engine 460 is started, an engine start signal indicating this is output to the remote control device 20.

[0130] As described above, after the engine start process is executed by the working machine 40, when the engine start signal is received by the remote operation device 20, the remote operation process ( Figure 5 / STEP13).

[0131] In this remote operation process, although not shown, first, the engine start signal is output from at least one of the image output device 221, the handy terminal 230, and the speaker 250. Thus, the operator 2 can confirm that the engine 460 has started.

[0132] Next, under the condition that the lever lock release operation is performed, the operation of the working machine 40 can be remotely controlled along with the operation of the remote control device 20 by the operator 2. In this case, when the blocking lever 2113 is lowered from the locked position to the released position by the operator 2, it is determined that the lever lock release operation is performed, and otherwise, it is determined that the lever lock release operation is not performed.

[0133] Next, refer to Fig.15 The work selection process is described below. The work selection process is a process executed by the remote control device 200 while the operator 2 is performing a remote operation, and is a process for the operator 2 to change a work based on the remote operation being performed.

[0134] As shown in the figure, first, it is determined whether remote operation is being performed ( Fig.15 / STEP60). In this case, while the remote operation process is being executed ( Figure 5 / STEP14), the judgment is affirmative, otherwise, the judgment is negative. Fig.15 / STEP60…NO), this process ends immediately.

[0135] On the other hand, in the case where the determination is positive ( Fig.15 / STEP60…YES), and when remote operation is being performed, determine whether the lever lock operation is performed ( Fig.15 / STEP61) In this case, when the blocking lever 2113 is pulled up to the locking position by the operator 2, it is determined that the lever locking operation is performed, and otherwise, it is determined that the lever locking operation is not performed.

[0136] In the case where the judgment is negative ( Fig.15 / STEP61…NO), the present process is terminated directly. On the other hand, if the determination is positive ( Fig.15 / STEP61…YES), and when the blocking lever 2113 is pulled up to the locked position, the terminal 230 is displayed. Fig.19 The job completion screen shown ( Fig.15 / STEP62).

[0137] In the case of this work completion screen, data related to various work machines 40 are displayed from the top to the center, and a work completion button 600 is displayed at the bottom.

[0138] Next, it is determined whether the lever lock release operation has been performed ( Fig.15 / STEP63) In this case, when the operator 2 lowers the blocking lever 2113 from the locked position to the released position, it is determined that the lever lock release operation has been performed, and otherwise, it is determined that the lever lock release operation has not been performed.

[0139] In the case where the judgment is positive ( Fig.15 / STEP63…YES), the process returns to STEP60. On the other hand, if the determination is negative ( Fig.15 / STEP63…NO), determine whether the job completion operation has been performed ( Fig.15 / STEP 64) In this case, when the operator 2 presses the work completion button 600, it is determined that the work completion operation has been performed, and otherwise, it is determined that the work completion operation has not been performed.

[0140] In the case where the judgment is negative ( Fig.15 / STEP64…NO), the process returns to STEP62. On the other hand, if the determination is positive ( Fig.15 / STEP64 ... YES), and when the work completion button 600 is pressed, the engine operation selection screen is displayed on the hand terminal 230. At the same time, although not shown in the figure, the work log update process in progress is terminated.

[0141] In the case of the engine operation selection screen, if Fig. 20 As shown, an engine ON continue button 601 and an engine OFF button 602 are displayed in a vertical arrangement at the lower part thereof. The words "engine ON continue" and "use this machine to perform other work" are displayed on the engine ON continue button 601.

[0142] The engine ON continue button 601 is a button that is pressed in the following cases: Figure 7 / STEP35) is completed, continue to use the machine selection screen ( Figure 7 / STEP36) and select the work machine 40 selected in the work selection screen ( Figure 7 / STEP35) is performed. In addition, the characters "Engine OFF", "·Switch to another machine" and "·End remote operation" are displayed on the engine OFF button 602.

[0143] Next, it is determined whether the engine ON continued operation is performed ( Fig.15 / STEP66) In this case, when the operator 2 presses the engine ON continue button 601, it is determined that the engine ON continue operation is performed, and otherwise, it is determined that the engine ON continue operation is not performed.

[0144] In the case where the judgment is positive ( Fig.15 / STEP66…YES), and the engine ON continue button 601 is pressed, other operation processing is performed ( Fig.15 / STEP67).

[0145] In the other job processing, the aforementioned Figure 7 After selecting other tasks, execute the above-mentioned STEP34 to STEP35. Figure 5 As described above, after executing other job processing, this processing ends.

[0146] On the other hand, when the above determination is negative ( Fig.15 / STEP66…NO), and the engine ON continue button 601 is not pressed, it is determined whether the engine OFF operation is performed ( Fig.15 / STEP68).

[0147] In this case, when the operator 2 presses the engine OFF button 602, it is determined that the engine OFF operation is performed, and otherwise it is determined that the engine OFF operation is not performed. The engine OFF button 602 is pressed by the operator 2 on the machine selection screen ( Figure 7 / STEP 36) is a button pressed when a different working machine 40 than the working machine 40 selected is performing work or when the remote operation itself in the remote operation device 20 is ended.

[0148] In the case where the judgment is negative ( Fig.15 / STEP68…NO), the process returns to the aforementioned STEP65. On the other hand, if the determination is positive ( Fig.15 / STEP68 ... YES), and the engine OFF button 602 is pressed, a portion of the image output device 221 displays Fig.21 The cut-off screen shown ( Fig.15 / STEP69).

[0149] In the case of the cut-off screen, if Fig.21 As shown, a "!" sign surrounded by a circle is displayed on the left side of the screen, a "Caution" sign is displayed below it, and a "Please disconnect" sign is displayed on the left side. Since the working machine 40 connected to the communication is not in use, these signs and signs are displayed to stop the engine of the working machine 40.

[0150] Next, it is determined whether the cutting operation has been performed ( Fig.15 / STEP 70) In this case, when the operator 2 operates the key switch 240 from the on position to the off position, it is determined that the off operation is performed, and otherwise, it is determined that the off operation is not performed.

[0151] In the case where the judgment is negative ( Fig.15 / STEP70…NO), the process returns to STEP68. On the other hand, if the determination is positive ( Fig.15 / STEP70...YES), and the key switch 240 is turned off, an engine stop signal ( Fig.15 / STEP71). As described above, in the operation selection process, when the operator 2 changes the key switch 240 in the remote operation device 20 from the on position to the off position, the off signal is sent to the operation machine 40, but otherwise, the off signal is not sent to the operation machine 40.

[0152] When the cut-off signal is received by the working machine 40, Fig.17 As shown, the engine stop process is executed in the working machine 40 ( Fig.17 / STEP73). As a result, engine 460 is stopped.

[0153] return Fig.15 After the cut-off signal is output to the working machine 40, the post-stop action selection process is executed ( Fig.15 / STEP72). Specifically, the post-stop action selection process is as follows: Fig.16 Execute as shown.

[0154] As shown in the figure, in this process, first, the terminal 230 is displayed. Fig. 22 The post-stop action selection screen shown ( Fig.16 / STEP80). In the case of the post-stop action selection screen, Fig. 22 As shown, the text "Please confirm that the key switch of the cab has been turned OFF" is displayed on the upper side. This text is displayed to prevent the operator 2 from operating the key switch 240 to the on position or the start position.

[0155] Furthermore, on the lower side of the above-mentioned text display, a restart button 603, a machine switching button 604, and a remote operation end button 605 are displayed in order from top to bottom.

[0156] The text "Restart with the same machine model" is displayed on the restart button 603, the text "Switch to another machine" is displayed on the machine change button 604, and the text "End remote operation" is displayed on the remote operation end button 605. In the case of this embodiment, displaying the post-stop action selection screen on the hand terminal 230 is equivalent to outputting confirmation information through the remote input unit.

[0157] Next, it is determined whether the remote operation end action has been performed ( Fig.16 / STEP81). In this case, when the remote operation end button 605 is pressed by the operator 2, it is determined that the remote operation end action is performed, otherwise, it is determined that the remote operation end action is not performed. In the case of this embodiment, the pressing of the remote operation end button 605 by the operator 2 is equivalent to the designated operation for performing the end of the remote operation.

[0158] In the case where the judgment is positive ( Fig.16 / STEP81...YES), and the remote operation end button 605 is pressed, a remote operation end signal ( Fig.16 / STEP82). The remote operation end signal is a signal for instructing the remote operation support server 10 to end the remote operation from the remote operation device 20. After that, this process ends.

[0159] When the remote operation support server 10 receives the engine stop signal, Fig.18 As shown, the communication setting cancellation process ( Fig.18 / STEP90). Along with this, a communication setting cancellation signal is output from the remote operation support server 10 to the remote operation device 20 and the working machine 40.

[0160] When the communication setting cancellation signal is received by the remote operation device 20, a communication setting cancellation process is executed ( Fig.18 / STEP91) In this communication setting cancellation process, the aforementioned various communication setting information set in the remote control device 200 is cancelled.

[0161] In addition, when the communication setting cancellation signal is received by the working machine 40, the communication setting cancellation process ( Fig.18 / STEP92) In this communication setting cancellation process, the aforementioned various communication setting information set in the actual machine control device 400 is cancelled. As a result, the wireless communication connection between the remote control device 200 and the working machine 40 is cut off.

[0162] return Fig.16 , in the above judgment is negative ( Fig.16 / STEP81…NO), and the remote operation end button 605 is not pressed, it is determined whether the transfer selection action is performed ( Fig.16 / STEP83). In this case, when the operator 2 presses the mechanical transfer button 604, it is determined that the transfer selection operation is performed, and otherwise, it is determined that the transfer selection operation is not performed. In the case of this embodiment, pressing the mechanical transfer button 604 is equivalent to the actual machine switching operation.

[0163] In the case where the judgment is positive ( Fig.16 / STEP83 ... YES), and the mechanical transfer button 604 is pressed, the remote operation end signal ( Fig.16 / STEP84). Thus, the aforementioned Fig.18 The communication setting cancellation process of the

[0164] Next, the work machine transfer process is executed ( Fig.16 / STEP85), after which the present process ends. In the present working machine transfer process, in the remote control device 20, the Figure 5 In the process of STEP1 Figure 6 The processing of STEP22 to 25 is performed, and the processing after STEP3 is executed.

[0165] At the same time, the remote operation support server 10 also executes the aforementioned Figure 5 The processing after STEP 2 of the above is also performed in the working machine 40. Figure 5 The work machine switching process is used to switch the work machine 40 remotely operated by the operator 2 to another work machine.

[0166] On the other hand, in the case where the above determination is negative ( Fig.16 / STEP83…NO), and the mechanical shift button 604 is not pressed, it is determined whether the restart action is performed ( Fig.16 / STEP86) In this case, when the restart button 603 is pressed by the operator 2, it is determined that the restart operation is performed, and otherwise, it is determined that the restart operation is not performed.

[0167] In the case where the judgment is negative ( Fig.16 / STEP86…NO), the process returns to the aforementioned STEP80. On the other hand, if the determination is positive ( Fig.16 / STEP86…YES), and the restart button 603 is pressed, the work machine restart process is executed ( Fig.16 / STEP87), and then this process ends.

[0168] In the work machine restart process, the remote control device 20 executes Figure 5 In the process of STEP1 Figure 6 At the same time, in the remote operation support server 10, the processing of STEP22 to STEP25 is executed, and the processing after STEP3 and STEP7 is executed. Figure 5 The processing of STEP 2 and STEP 11 is performed, and the processing of STEP 9 and subsequent steps is also performed in the working machine 40.

[0169] As described above, according to the remote operation system 1 of the present embodiment, after the lever locking operation is performed to stop the working machine 40, unless the remote operation end action or the transfer selection action is performed, the communication connection establishment process will not be performed again. That is, the communication connection state between the remote operation device 20 and the working machine 40 is maintained. As a result, compared with the case where the communication connection state between the remote operation device 20 and the working machine 40 is established again when the remote operation of the working machine 40 is restarted after the working machine 40 stops, the time from temporarily stopping the remote operation to restarting the remote operation can be shortened.

[0170] Furthermore, after the working machine 40 stops, when the remote operation termination action is performed or the transfer selection action is performed, the communication settings in the remote control device 200 and the real machine control device 400 are canceled, and the communication connection state between the remote control device 20 and the working machine 40 is terminated. Thus, the communication connection state can be reliably terminated in a situation where there is a high possibility that the remote operation can be terminated.

[0171] Furthermore, the remote control device 200 and the real machine control device 400 are configured with a communication setting that enables communication between the remote wireless communication device 260 and the real machine wireless communication device 470, thereby being able to establish a communication connection state between the remote control device 20 and the working machine 40 ( Figure 5 Thus, even when there are a plurality of remote operating devices 20 and a plurality of working machines 40 that can be remotely operated by the plurality of remote operating devices 20, a communication connection state can be appropriately established between the remote operating device 20 and the working machine 40 that should be the object of remote operation by the remote operating device 20.

[0172] In addition, after the working machine 40 stops and the engine 460 is stopped by the operator 2, a post-stop action selection screen ( Fig. 22 ). Thus, the operator 2 can accurately determine whether to restart or end the remote operation by operating the terminal 230 at hand.

[0173] Furthermore, by stopping the engine 460 before the post-stop operation selection screen is displayed on the hand terminal 230, it is possible to avoid wasting fuel by the engine 460.

[0174] In addition, in the post-stop action selection screen, when the machine transfer button 604 is pressed, the working machine transfer process is performed, so the transfer action of the working machine 40 can be accurately performed. Furthermore, in the post-stop action selection screen, when the restart button 603 is pressed, the working machine restart process is performed. In the working machine restart process and the working machine transfer process, the authentication process of the operator 2 is performed. Thereby, it is possible to confirm whether the operator 2 who restarts the remote operation is consistent with the registered operator stored in the storage device 12. This is because, when the post-stop action selection screen is displayed on the terminal 230, the previous remote operation-based operation is in a completed state, and the operator 2 who performed the remote operation may be replaced by another operator 2, so it is necessary to perform the authentication process of the operator 2 after the remote operation is restarted.

[0175] Furthermore, since the handy terminal 230 is provided near the seat 231 , the operator 2 can perform input operations on the handy terminal 230 while sitting on the seat 231 .

[0176] In addition, the embodiment uses the crawler excavator type working machine 40 as an example of the working machine, but the working machine of the present invention is not limited to this, as long as it can be remotely operated by the remote operating device. For example, a remotely operated crane or the like can also be used as the working machine.

[0177] In addition, the embodiment is an example of using the handheld terminal 230 as the remote input unit, but the remote input unit of the present invention is not limited to this, and the operator can use any remote input unit as long as it can be operated. For example, a portable terminal of the smartphone type can also be used as the remote input unit.

[0178] Furthermore, the embodiment is an example in which the operation of pressing a button by the operator 2 is used as the designation operation, but instead, the operation of inputting a voice uttered by the operator 2 to a device such as the handy terminal 230 may be used as the designation operation.

[0179] In addition, in the remote operation system 1, only one of the remote operation support server 10, the remote operation device 20, and the working machine 40 may have the functions of a remote control element and a communication control element. In other words, the remote operation system 1 may be constituted by only one of the remote operation support server 10, the remote operation device 20, and the working machine 40.

[0180] For example, when the remote operating device 20 constitutes the remote operating system 1, the remote control device 200 may also have the functions of the remote control element and the communication control element. Similarly, when the working machine 40 constitutes the remote operating system 1, the real machine control device 400 may also have the functions of the remote control element and the communication control element.

[0181] Furthermore, in the remote operation system 1, one of the remote operation support server 10 and the remote operation device 20 may function as a remote control element, and the other of the remote operation support server 10 and the remote operation device 20 may function as a communication control element.

[0182] Furthermore, in the remote operation system 1 , one of the remote operation support server 10 and the working machine 40 may function as a remote control element, and the other of the remote operation support server 10 and the working machine 40 may function as a communication control element.

[0183] Furthermore, in the remote operation system 1, one of the remote operation device 20 and the working machine 40 may function as a remote control element, and the other of the remote operation device 20 and the working machine 40 may function as a communication control element.

[0184] According to the remote operation system of the present invention, when the remote operation is resumed after the stop operation of the working machine is executed, the communication connection state is maintained until the specified operation is performed. Therefore, compared with the case where the communication connection state is established again, the time from temporarily stopping the remote operation to resuming the remote operation can be shortened.

[0185] Furthermore, in the present invention, preferably, after the actual machine stop operation is performed, when the operator performs a designated operation for ending the remote operation in the remote input unit, the communication control element ends the communication connection state.

[0186] According to the remote operation system, the operator can confirm that the remote operation performed by the remote operation device can be terminated for the working machine remotely operated by the remote operation device by performing a designated operation for terminating the remote operation. Thus, the communication connection state can be reliably terminated under the condition that the remote operation is likely to be terminated.

[0187] In addition, in the present invention, preferably, the communication control element comprises: an apparatus-side communication control element, which controls an apparatus-side communication device in a remote operating device; and an actual machine-side communication control element, which controls an actual machine-side communication device in an operating machine, and the apparatus-side communication control element and the actual machine-side communication control element are configured with a communication setting capable of performing communication between the apparatus-side communication device and the actual machine-side communication device, thereby establishing a communication connection state.

[0188] Furthermore, according to the remote operation system, the device-side communication control element and the real machine-side communication control element are configured with a communication setting that enables communication between the device-side communication device and the real machine-side communication device, thereby enabling a communication connection state between the remote operation device and the working machine. In this case, for example, in a situation where there are multiple remote operation devices and multiple working machines that can be remotely operated, there are multiple candidates for communication devices that can communicate with the device-side communication device and the real machine-side communication device, but by setting the communication setting corresponding to the communication device that establishes communication as described above, a communication connection state between the remote operation device and the working machine that should be the object of remote operation of the remote operation device can be reliably established.

[0189] Furthermore, in the present invention, preferably, after executing the actual machine stop operation, when the operator executes a designated operation for ending the remote operation in the remote input unit, the device side communication control element and the actual machine side communication control element end the maintenance of the communication connection state by canceling the communication setting.

[0190] According to this remote operation system, the operator can confirm that the work machine remotely operated by the remote operation device can terminate the remote operation by performing a designated operation for terminating the remote operation. Thus, the communication setting can be canceled in a situation where there is a high possibility that the remote operation can be terminated.

[0191] Furthermore, in the present invention, it is preferable to further include an output control means for outputting confirmation information for confirming whether to restart or end the remote operation from a remote output unit of the remote operation device when the real machine stop operation is executed.

[0192] According to the remote operation system, when the real machine stop operation is performed, confirmation information for confirming whether to restart or end the remote operation is output from the remote output unit to the operator. Thus, the operator can accurately decide whether to restart or end the remote operation, and can accurately perform the designated operation of designating the restart or end of the remote operation in the remote input unit.

[0193] In the present invention, preferably, after the actual machine stop operation is performed, when the operator performs an internal combustion engine stop operation for stopping the internal combustion engine of the working machine through the remote input unit, the remote control element stops the internal combustion engine.

[0194] According to this remote control system, after the operator stops the internal combustion engine by performing the actual machine stop operation, it is possible to avoid the internal combustion engine from wasting fuel.

[0195] Furthermore, in the present invention, preferably, after the actual machine stop operation is performed, when the operator performs a work end operation and an internal combustion engine stop operation for stopping the internal combustion engine of the working machine in the remote input unit, the remote control element stops the internal combustion engine.

[0196] According to this remote operation system, the internal combustion engine can be stopped in a situation where there is a high possibility that the operator has temporarily stopped the operation including the internal combustion engine stop operation, thereby preventing the internal combustion engine from wasting fuel thereafter.

[0197] Furthermore, in the present invention, preferably, after executing the actual machine stop operation, when the operator executes an actual machine switching operation for switching from the working machine to another working machine in the remote input unit, the remote control element executes a switching process to the other working machine.

[0198] According to this remote operation system, after the working machine stops, when the operator indicates an intention to switch the remotely operated working machine to another working machine, the switching operation can be accurately performed.

[0199] Furthermore, in the present invention, preferably, when the switching process is executed by the remote control means, the communication control means ends the communication connection state between the remote operating device and the working machine.

[0200] According to this remote operation system, by executing the switching process, it is possible to confirm that the remote operation performed by the remote operation device can be terminated in the working machine remotely operated by the remote operation device. Thus, the communication connection state can be reliably terminated in a situation where there is a high possibility that the remote operation can be terminated.

[0201] In addition, in the present invention, preferably, when prescribed conditions including successful authentication of the operator are satisfied, the remote control element establishes a communication connection state between the remote operating device and the working machine to enable remote operation of the working machine, and performs authentication of the operator before resuming remote operation.

[0202] According to the remote operating system, operator authentication is performed before resuming remote operation, so that it is possible to confirm whether the operator who resumes remote operation is the same person as the operator before resuming remote operation, and if the operator who resumes remote operation is a different person from the operator before resuming remote operation, it is possible to confirm whether the operator is qualified to perform remote operation of the working machinery.

[0203] Furthermore, in the present invention, preferably, the remote input unit is provided near a seat on which an operator sits.

[0204] According to this remote operation system, the operator can perform input operations on the remote input unit while sitting on the seat.

[0205] [Explanation of symbols]

[0206] 1: Remote operating system;

[0207] 2: Operator;

[0208] 20: Remote operation device;

[0209] 200: Remote control device;

[0210] 201: Remote control element;

[0211] 202: communication control element;

[0212] 203: device-side communication control element;

[0213] 204: output control element;

[0214] 230: terminal at hand (remote input unit);

[0215] 231: Seat;

[0216] 260: long-range wireless communication device (device-side communication device);

[0217] 40: Operating machinery;

[0218] 400: real machine control device (communication control element, real machine side communication control element);

[0219] 460: Engine (internal combustion engine);

[0220] 470: Real machine wireless communication device (real machine side communication device).

Claims

1. A remote operation system for an operator to remotely operate a working machine via a remote operation device having a remote input unit, characterized in that: have: a remote control element for executing a restart process or an end process of the remote operation when the operator performs a designated operation for restarting or ending the remote operation in the remote input unit, if the operator performs an actual machine stop operation of the working machine via the remote operation device during the execution of the remote operation; as well as A communication control element, which, when the remote operation begins, establishes a communication connection state between the remote operation device and the working machine to enable the remote operation of the working machine, and maintains the communication connection state during the period from executing the actual machine stop operation to executing the specified operation.

2. The remote operation system according to claim 1, characterized in that: After the real machine stop operation is performed, when the operator performs the designated operation for ending the remote operation in the remote input unit, the communication control element ends the communication connection state.

3. The remote operation system according to claim 1 or 2, characterized in that: The communication control element comprises: an apparatus-side communication control element for controlling an apparatus-side communication device in the remote operation device; and an actual machine-side communication control element for controlling an actual machine-side communication device in the working machine. The apparatus-side communication control element and the real machine-side communication control element configure communication settings that enable communication between the apparatus-side communication device and the real machine-side communication device, thereby establishing the communication connection state.

4. The remote operation system according to claim 3, characterized in that: After the real machine stop operation is performed, when the operator performs the designated operation for ending the remote operation in the remote input unit, the device-side communication control element and the real machine-side communication control element end the maintenance of the communication connection state by canceling the communication setting.

5. The remote operation system according to claim 1 or 2, characterized in that: An output control unit is further provided for outputting confirmation information for confirming whether to restart or end the remote operation from a remote output unit of the remote operation device when the real machine stop operation is performed.

6. The remote operation system according to claim 1 or 2, characterized in that: After the actual machine stop operation is performed, when the operator performs an internal combustion engine stop operation for stopping the internal combustion engine of the working machine through the remote input unit, the remote control element stops the internal combustion engine.

7. The remote operation system according to claim 1 or 2, characterized in that: After the actual machine stop operation is performed, when the operator performs a work ending operation and an internal combustion engine stop operation for stopping the internal combustion engine of the working machine in the remote input unit, the remote control element stops the internal combustion engine.

8. The remote operation system according to claim 1 or 2, characterized in that: After the real machine stop operation is performed, when the operator performs a real machine switching operation for switching from the working machine to another working machine in the remote input unit, the remote control element performs a switching process to the other working machine.

9. The remote operation system according to claim 8, characterized in that: When the switching process is executed by the remote control element, the communication control element ends the communication connection state between the remote operating device and the working machine.

10. The remote operation system according to claim 1 or 2, characterized in that: When prescribed conditions including successful authentication of the operator are satisfied, the remote control element establishes a communication connection state between the remote operating device and the working machine to enable the remote operation of the working machine, and performs the authentication of the operator before resuming the remote operation.

11. The remote operation system according to claim 1 or 2, characterized in that: The remote input unit is provided near a seat where the operator sits.

Images