Remote operation support system, remote operation support method, and remote operation support composite system

Abstract

To provide a system improved in remote operation efficiency of a work machine by improving efficiency of calibration processing, etc.SOLUTION: A result of determination of whether or not a remote operation factor corresponds to an actual machine operation factor is recognized by a first support processing element 121. The "remote operation factor" is a factor defining operation setting of a remote operation mechanism 211 forming a remote operation device 20. The "actual machine operation factor" is a factor defining operation setting of an actual operation mechanism 411 forming a work machine 40. With the determination result of being negative as a necessary condition, the processing for making the actual machine operation factor correspond to the remote operation factor is performed by a second support processing element 122.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a technique for remotely operating a work machine such as a hydraulic excavator by using a remote operation device. [Background technology]

[0002] When a work machine is remotely operated by a remote control device, the settings of the remote control device must be adjusted to match the settings of the work machine to be remotely operated.

[0003] In Patent Document 1, a correction value is registered for matching the operation characteristics of an actual operation mechanism such as an actual operation lever constituting a work machine with the operation characteristics of a remote operation mechanism such as a remote operation lever constituting a remote control device for remotely operating the work machine, and the operation amount input to the remote operation mechanism is corrected based on the correction value and transmitted to the actual operation mechanism, thereby executing correction control for operating the work machine according to the operation amount input to the remote operation mechanism. The correction value is associated with a combination of model information of the work machine and model information of the remote operation mechanism. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-167732 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the actual machine operation mechanism that is the target of remote operation by the remote control device may be changed. For example, the actual machine operation mechanisms that are mounted on the work machine that is the target of remote operation by the remote control device and can be remotely controlled may increase or decrease due to modification. In addition, when the remote control device switches the work machine that is the target of its remote operation from a first work machine to a second work machine, the actual machine operation mechanism provided on the first work machine and the actual machine operation mechanism provided on the second work machine may no longer correspond to each other. When the actual machine operation mechanism is changed, the registered correction values ​​may no longer be compatible, and new correction values ​​must be obtained and registered.

[0006] In addition, a remote control mechanism such as a remote control lever constituting a remote control device may be changed. For example, the operation pattern of the actual operation mechanism is changed. In this case, the actual operation mechanism corresponding to the actual operation mechanism is changed, so that the registered correction value may no longer be applicable, and it is necessary to obtain and register a new correction value.

[0007] Therefore, it is expected that at the start of remote operation, a process will be executed to reliably match the settings of the remote control device to the work machine to be remotely operated by checking and registering the correction values.

[0008] However, if this process is executed every time communication is established between the remote control device and the work machine, the process will be executed even when there are no changes to the actual operation mechanism or the remote operation mechanism, and remote operation of the work machine cannot be started until the process is completed.

[0009] Therefore, an object of the present invention is to provide a server etc. that can improve the efficiency of remote operation of a work machine by improving the efficiency of the process of matching the settings of a remote operation device to the work machine to be remotely operated. [Means for solving the problem]

[0010] The remote operation support system of the present invention comprises: A remote operation support system for supporting remote operation of a work machine by a remote operation device, comprising: A remote control mechanism constituting the remote control device The present invention represents the correspondence of the actual machine operation mechanism constituting the work machine intended for remote operation by the Remote control factors defining the control settings; Represents the correspondence of the actual machine operation mechanism that can be operated by receiving remote control. a first support processing element that recognizes a determination result of whether or not the operation setting corresponds to an actual machine operation factor that defines the operation setting; a second support processing element that executes a process for matching the actual machine operation factors and the remote operation factors based on communication with at least one of the remote operation device and the work machine, on the condition that the determination result recognized by the first support processing element is negative; It is equipped with:

[0011] According to the remote operation support system having this configuration, if the remote operation factors and the actual machine operation factors do not correspond to each other, a process is executed to make the operation method of the remote operation mechanism constituting the remote operation device correspond to the operation method of the actual machine operation mechanism constituting the work machine. As a result, if the two factors match, remote operation of the work machine can be started by the remote operation device without executing the process to make the two operation factors correspond to each other. Therefore, by improving the efficiency of this process, the efficiency of remote operation of the work machine is improved.

[0012] In the remote operation support system of the present invention, The first support processing element responds to the actual machine operation mechanism set as a remote operation target by the remote operation mechanism. The above A remote control mechanism that controls the remote operation of the working machine is provided. The above Whether the actual operation factors are supported The above Recognize the results It is preferred.

[0013] According to the remote operation support system having this configuration, the first support processing element can determine whether or not the actual machine operation mechanism that is the target of remote operation by the remote operation device has been changed. As a result, if the actual machine operation mechanism has not been changed, the remote operation of the work machine can be started by the remote operation device without executing the process of matching both operation factors. On the other hand, if the actual machine operation mechanism has been changed, the process of matching the factors is executed reliably.

[0014] In the remote operation support system of the present invention, On the condition that the determination result recognized by the first support processing element is negative, the second support processing element executes a process of acquiring a correction value for making the operation characteristics of the actual machine operation mechanism constituting the work machine correspond to the operation characteristics of the remote operation mechanism constituting the remote operation device for remotely operating the work machine. It is preferable.

[0015] According to the remote operation support system having the above configuration, when the process of matching the two operation factors is executed, a process of matching the operation characteristics of the real operation mechanism with the operation characteristics of the remote operation mechanism is further executed, so that even if the operation factor is changed, the same operability as before the change can be maintained.

[0016] In the remote operation support system of the present invention, the first support processing element recognizes an operation pattern when remotely operating the actual machine operation mechanism constituting the work machine that is the target of remote operation by the remote operation mechanism; The second support processing element executes a process of acquiring a correction value for making the operation characteristics of the actual operation mechanism constituting the work machine correspond to the operation characteristics of the remote operation mechanism constituting the remote operation device for remotely operating the work machine, on the condition that at least one of the determination result recognized by the first support processing element is negative and that there has been a change in the operation pattern recognized by the first support processing element. It is preferable.

[0017] According to the remote operation support system having the above configuration, when at least one of the operation pattern and the operation factor is changed, a process is performed to make the operation characteristics of the actual operation mechanism correspond to the operation characteristics of the remote operation mechanism, so that even when only the operation pattern is changed, the same operability as before the change can be maintained.

[0018] In the remote operation support system of the present invention, The second support processing element outputs a message prompting a change of the remote operation factor to a remote output interface constituting the remote operation device, and recognizes the new remote operation factor changed through a remote input interface constituting the remote operation device. It is preferred.

[0019] According to the remote operation support system having this configuration, when the remote operation factors and the actual machine operation factors are different, a process for matching the operation method of the remote operation mechanism constituting the remote operation device with the operation method of the actual machine operation mechanism constituting the work machine is executed in the remote operation device in response to the will of the user. As a result, when both factors match, the remote operation of the work machine can be started by the remote operation device without waiting for the completion of the execution of the process. Therefore, the efficiency of the process is improved, thereby improving the efficiency of the remote operation of the work machine.

[0020] In the remote operation support server of the present invention, It is preferable that after the second support processing element recognizes the new remote operation factor changed through the remote input interface, the first support processing element recognizes the determination result as to whether or not the remote operation factor corresponds to the actual machine operation factor based on communication with the work machine.

[0021] According to the remote operation support server having the above configuration, even if the remote operation factors and the actual machine operation factors do not correspond to each other, the remote operation factors and the actual machine operation factors are reliably matched to each other through the process executed in response to the user's intention in the remote operation device, thereby improving the efficiency of remote operation of the work machine. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is an explanatory diagram of a remote operation support system according to the present invention. [Diagram 2] FIG. 2 is an explanatory diagram relating to the configuration of a remote control device. [Diagram 3] FIG. 2 is an explanatory diagram relating to the configuration of a work machine. [Figure 4] FIG. 4 is an explanatory diagram relating to a first function of the operation support system. [Diagram 5] FIG. [Figure 6] FIG. 4 is an explanatory diagram relating to a first embodiment of a second function of the operation support system. [Figure 7] 5 is a diagram illustrating the relationship between a drive command and an operation amount of an actual machine control lever. FIG. [Figure 8] 5 is a diagram illustrating the relationship between a drive command and an operation amount of a remote control lever. [Figure 9] FIG. 13 is an explanatory diagram relating to a second embodiment of a second function of the operation support system. [Figure 10] FIG. 11 is an explanatory diagram relating to a modified example of the second function of the operation support system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] (Configuration of remote operation support system) The remote operation support complex system shown in Fig. 1 is composed of a remote operation support server 10, a remote operation device 20 and / or a work machine 40. The remote operation support server 10, the remote operation device 20 and the work machine 40 are each configured to be able to communicate with each other via a network. The mutual communication network of the remote operation support server 10 and the remote operation device 20 and the mutual communication network of the remote operation support server 10 and the work machine 40 may be the same as or different from each other.

[0024] The components of the present invention "recognize" information, which is a concept that encompasses any computational process for making the information available for subsequent computational processes, such as receiving the information, reading the information from a storage device, searching for the information from a database, measuring the information, determining, judging, estimating or predicting the information based on basic information received or otherwise, and storing the information in a storage device.

[0025] (Configuration of remote operation support server) The remote operation support server 10 constituting the remote operation support system includes a database 102, a first support processing element 121, and a second support processing element 122. The database 102 stores captured image data and the like. The database 102 may be configured as a database server separate from the remote operation support server 10. Each support processing element is configured with an arithmetic processing device (a single-core processor or a multi-core processor or a processor core constituting the same), reads necessary data and software from a storage device such as a memory, and executes arithmetic processing (described later) on the data in accordance with the software.

[0026] (Configuration of remote control device) The remote operation device 20 includes a remote control device 200, a remote input interface 210, and a remote output interface 220. The remote control device 200 is configured with an arithmetic processing device (a single-core processor or a multi-core processor or a processor core constituting the same), reads necessary data and software from a storage device such as a memory, and executes arithmetic processing on the data according to the software.

[0027] The remote input interface 210 includes a remote control mechanism 211. The remote output interface 220 includes a video output device 221, an audio output device 222, and a remote wireless communication device 223.

[0028] The remote operation mechanism 211 includes a traveling operation device, a slewing operation device, a boom operation device, an arm operation device, and a bucket operation device. Each operation device has an operation lever that receives a rotation operation. The operation lever (travel lever) of the traveling operation device is operated to move the lower traveling body 410 of the work machine 40. The travel lever may also serve as a travel pedal. For example, a travel pedal fixed to the base or lower end of the travel lever may be provided. The operation lever (slewing lever) of the slewing operation device is operated to move a hydraulic slewing motor that constitutes the slewing mechanism 430 of the work machine 40. The operation lever (boom lever) of the boom operation device is operated to move the boom cylinder 442 of the work machine 40. The operation lever (arm lever) of the arm operation device is operated to move the arm cylinder 444 of the work machine 40. The operation lever (bucket lever) of the bucket operation device is operated to move the bucket cylinder 446 of the work machine 40.

[0029] The operating levers constituting the remote control mechanism 211 are arranged around a seat St on which the operator sits, as shown in Fig. 2. The seat St is in the form of a high-back chair with armrests, but may be in the form of a low-back chair without a headrest, or in the form of a chair without a backrest, or in any other form on which the operator can sit.

[0030] A pair of left and right travel levers 2110 corresponding to the left and right crawlers are arranged side by side in front of the seat St. One operation lever may serve as multiple operation levers. For example, the left operation lever 2111 provided in front of the left frame of the seat St shown in FIG. 2 may function as an arm lever when operated in the front-rear direction, and may function as a rotation lever when operated in the left-right direction. Similarly, the right operation lever 2112 provided in front of the right frame of the seat St shown in FIG. 2 may function as a boom lever when operated in the front-rear direction, and may function as a bucket lever when operated in the left-right direction. The lever pattern may be arbitrarily changed by an operation instruction from the operator.

[0031] 2, the image output device 221 is composed of a central image output device 2210, a left image output device 2211, and a right image output device 2212, each having a substantially rectangular screen arranged in front of the seat St, diagonally forward to the left, and diagonally forward to the right. The shapes and sizes of the screens (image display areas) of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be the same or different.

[0032] As shown in Fig. 2, the right edge of the left image output device 2211 is adjacent to the left edge of the central image output device 2210 such that the screens of the central image output device 2210 and the left image output device 2211 form an inclination angle θ1 (e.g., 120°≦θ1≦150°). As shown in Fig. 2, the left edge of the right image output device 2212 is adjacent to the right edge of the central image output device 2210 such that the screens of the central image output device 2210 and the right image output device 2212 form an inclination angle θ2 (e.g., 120°≦θ2≦150°). The inclination angles θ1 and θ2 may be the same or different.

[0033] The screens of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be parallel to the vertical direction or may be inclined relative to the vertical direction. At least one of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be composed of a plurality of divided image output devices. For example, the central image output device 2210 may be composed of a pair of image output devices adjacent to each other vertically, each having a substantially rectangular screen.

[0034] The sound output device 222 is composed of one or more speakers, and is composed of a central sound output device 2220, a left sound output device 2221, and a right sound output device 2222, which are arranged behind the seat St, at the rear of the left armrest, and at the rear of the right armrest, respectively, as shown in Fig. 2. The specifications of the central sound output device 2220, the left sound output device 2221, and the right sound output device 2222 may be the same or different.

[0035] (Work machine configuration) The work machine 40 is equipped with an actual machine control device 400, an actual machine input interface 41, an actual machine output interface 42, and a work mechanism 440. The actual machine control device 400 is configured with an arithmetic processing device (a single-core processor or a multi-core processor or a processor core constituting the same), reads necessary data and software from a storage device such as a memory, and executes arithmetic processing on the data in accordance with the software.

[0036] The work machine 40 is, for example, a crawler excavator (construction machine), and as shown in Fig. 3, includes a crawler-type lower traveling body 410, and an upper rotating body 420 that is rotatably mounted on the lower traveling body 410 via a rotating mechanism 430. A cab 424 (operator's cabin) is provided on the front left side of the upper rotating body 420. A work mechanism 440 is provided in the front center of the upper rotating body 420.

[0037] The real machine input interface 41 includes a real machine operation mechanism 411, a real machine imaging device 412, and a positioning device 414. The real machine operation mechanism 411 includes a plurality of operation levers arranged around a seat arranged inside the cab 424 in the same manner as the remote control mechanism 211. A drive mechanism or robot that receives a signal according to the operation mode of the remote control lever and moves the real machine operation lever based on the received signal is provided in the cab 424. The real machine imaging device 412 is installed, for example, inside the cab 424, and images the environment including at least a part of the working mechanism 440 through the front window and a pair of left and right side windows. A part or all of the front window and the side windows may be omitted. The positioning device 414 is configured with a GPS and, if necessary, a gyro sensor or the like.

[0038] The real device output interface 42 includes a real device wireless communication device 422 .

[0039] 3, a working mechanism 440 as a working mechanism includes a boom 441 mounted on the upper rotating body 420 so as to be able to rise and fall, an arm 443 rotatably connected to the tip of the boom 441, and a bucket 445 rotatably connected to the tip of the arm 443. A boom cylinder 442, an arm cylinder 444, and a bucket cylinder 446 each composed of an extendable hydraulic cylinder are mounted on the working mechanism 440.

[0040] The boom cylinder 442 is interposed between the boom 441 and the upper rotating body 420 so as to extend and retract when supplied with hydraulic oil to rotate the boom 441 in the hoisting direction. The arm cylinder 444 is interposed between the arm 443 and the boom 441 so as to extend and retract when supplied with hydraulic oil to rotate the arm 443 about a horizontal axis relative to the boom 441. The bucket cylinder 446 is interposed between the bucket 445 and the arm 443 so as to extend and retract when supplied with hydraulic oil to rotate the bucket 445 about a horizontal axis relative to the arm 443.

[0041] (First function) The first function as a basic function of the remote operation support system having the above configuration will be described with reference to the flowchart shown in Fig. 4. In the flowchart, a block "C●" is used for the sake of simplicity, and means transmission and / or reception of data, and means a conditional branch in which processing in a branching direction is executed on the condition that the data is transmitted and / or received. The received data is stored in a storage device constituted by a database 102 and / or a non-volatile or volatile memory. These matters are the same in the flowcharts described below.

[0042] In the remote operation device 20, it is determined whether or not a designation operation has been performed by the operator through the remote input interface 210 (FIG. 4 / STEP 210). The "designation operation" is, for example, an operation such as tapping on the remote input interface 210 for the operator to designate the work machine 40 that he or she intends to remotely operate. If the determination result is negative (FIG. 4 / STEP 210...NO), the series of processes ends. On the other hand, if the determination result is positive (FIG. 4 / STEP 210...YES), an environment confirmation request is transmitted to the remote operation support server 10 through the remote wireless communication device 222 (FIG. 4 / STEP 212).

[0043] When the environment confirmation request is received in the remote operation support server 10, the first support processing element 121 transmits the environment confirmation request to the corresponding work machine 40 (FIG. 4 / C110).

[0044] When the work machine 40 receives an environment confirmation request through the actual machine wireless communication device 422 (FIG. 4 / C410), the actual machine control device 400 acquires a captured image through the actual machine imaging device 412 (FIG. 4 / STEP 410). Here, image processing may be performed by the actual machine control device 400 or an image processing device constituting the actual machine control device 400. The captured image data that has been subjected to the image processing is transmitted by the actual machine control device 400 through the actual machine wireless communication device 422 to the remote operation device 10 (FIG. 4 / STEP 412).

[0045] In the remote operation support server 10, when the first support processing element 121 receives captured image data (FIG. 4 / C112), the second support processing element 122 transmits environmental image data corresponding to the captured image to the remote operation device 20 (FIG. 4 / STEP 110). The environmental image data is not only the captured image data itself, but also image data representing a simulated environmental image generated based on the captured image. When the image processing device 30 is configured by the remote operation support server 10, the captured image data may be subjected to image processing by the image processing device 30 to generate the environmental image data.

[0046] When the remote operation device 20 receives environmental image data through the remote wireless communication device 222 (FIG. 4 / C210), the remote control device 200 outputs an environmental image corresponding to the environmental image data to the image output device 221 (FIG. 4 / STEP 214).

[0047] 5, an environmental image in which a boom 441, an arm 443, and a pile of rubble or earth and sand (the target of work by the bucket 445) that are part of the working mechanism 440 are reflected in front of the cab 424 through a window frame that is composed of a right window frame Q1, an upper window frame Q2, a left window frame Q3, and a lower window frame Q4 that define the cab 424 is output to the image output device 421. By image processing of the captured image or by adjusting the angle of view of the imaging device 412, the environmental image may be generated so that at least a part of the window frames Q1 to Q4 is not reflected. If the imaging device 421 is provided outside the cab 424 instead of inside it, it is possible to obtain a captured image, and therefore an environmental image, in which components of the cab 424, such as the window frames Q1 to Q4, are not reflected.

[0048] In the remote operation device 20, the operation mode of the remote operation mechanism 211 is recognized by the remote control device 200 (FIG. 4 / STEP 216), and a remote operation command corresponding to the operation mode is transmitted to the remote operation support server 10 via the remote wireless communication device 222 (FIG. 4 / STEP 218).

[0049] In the remote operation support server 10, when the second support processing element 122 receives the remote operation command, the first support processing element 121 transmits the remote operation command to the work machine 40 (FIG. 4 / C114).

[0050] In the work machine 40, when an operation command is received by the actual machine control device 400 via the actual machine wireless communication device 422 (FIG. 4 / C412), the operation of the work mechanism 440 and the like is controlled (FIG. 4 / STEP 414). For example, the work of scooping up soil in front of the work machine 40 with the bucket 445, rotating the upper rotating body 410, and dropping the soil from the bucket 445 is performed.

[0051] (Second Function (First Embodiment)) A first embodiment of the second function as a calibration function of the remote operation support system having the above configuration will be described with reference to the flowchart shown in FIG.

[0052] In the remote operation device 20, the remote control device 200 transmits the remote operation factor together with the actual machine identifier to the remote operation support server 10 via the remote wireless communication device 222 (FIG. 6 / STEP 220). In response to a trigger operation (input of the actual machine identifier or designation of a remote operation mode) in the remote input interface 210, the remote operation factor and the actual machine identifier may be transmitted from the remote operation device 20. The "remote operation factor" is a factor that defines the remote operation setting of the remote operation mechanism 211 that constitutes the remote operation device 20. The "remote operation factor" corresponds to the actual machine operation mechanism 411 that constitutes the work machine 40 intended to be remotely operated by the remote operation mechanism 211. The correspondence is illustrated in Table 1.

[0053] [Table 1]

[0054] As shown in Table 1, when the actual operation mechanism 411 constituting the work machine 40 intended to be remotely operated by the remote operation mechanism 211 is a boom operation lever, arm operation lever, bucket operation lever, and swing operation lever, the remote operation setting A1 is set. In the remote operation setting A1, remote operation that occurs by tilting the left and right operation levers is set, but remote operation that occurs by tilting the travel lever and optional pedal is not set. When the intention is for the work machine 40 to travel with the movement of the lower traveling body 410 through the operation of the remote operation mechanism 211, the remote operation setting A2 is set, and remote operation that occurs by tilting the travel lever is set. When the intention is for the remote operation mechanism 211 to operate an optional device such as a grapple, the remote operation setting A2 is set. teeth The remote operation setting A3 is set, and the remote operation that occurs by tilting the OPT pedal is set. It is possible to set remote operation settings other than the remote operation settings A1 to A3, for example, this corresponds to changing the setting of the operation pattern. The "actual machine identifier" is an identifier or data for identifying or specifying the work machine 40 that is the target of remote operation by the remote operation device 20.

[0055] In the remote operation support server 10, the first support processing element 121 receives the remote operation factor and the actual machine identifier (Figure 6 / C120), and a request to obtain the actual machine operation factor is sent to the work machine 40 identified by the actual machine identifier (Figure 6 / STEP120).

[0056] When the actual machine operation factor request is received by the actual machine control device 400 via the actual machine wireless communication device 422 in the corresponding work machine 40 (FIG. 6 / C420), the actual machine operation factor is recognized by being read or searched from a storage device or database (FIG. 6 / STEP420). Then, the actual machine control device 400 transmits the actual machine operation factor and the actual machine identifier to the remote operation support server 10 via the actual machine wireless communication device 422 (FIG. 6 / STEP422). The "actual machine operation factor" is a factor that defines the actual machine operation setting of the actual machine operation mechanism 411 that constitutes the work machine 40, and corresponds to the actual machine operation mechanism 411 that can be remotely operated. The correspondence is illustrated in Table 2.

[0057] [Table 2]

[0058] As shown in Table 2, when the actual machine operation mechanism 411 constituting the work machine 40 intended to be remotely operated by the remote operation mechanism 211 is a boom operation lever, an arm operation lever, a bucket operation lever, and a swing operation lever, the actual machine operation setting B1 is set. The actual machine operation mechanism marked with a circle in the actual machine operation setting B1 can be operated by receiving remote operation, and the actual machine operation mechanism without a circle cannot be operated even if it receives remote operation. When the actual machine operation mechanism 411 can receive remote operation of traveling, the actual machine operation setting B2 is set, and when the actual machine operation mechanism 411 can receive remote operation of an optional device such as a grapple, the actual machine operation setting B3 is set. The actual machine operation factors are set by the actual machine control device 400 of the work machine 40 recognizing the configuration of the actual machine operation mechanism 411 of the work machine 40, or by being input by a serviceman who performs maintenance on the work machine 40.

[0059] In the remote operation support server 10, the first support processing element 121 receives the actual machine operation factor and the actual machine identifier (FIG. 6 / C121), and the actual machine operation factor and the actual machine identifier are further received by the remote operation device 20 (FIG. 6 / C220).

[0060] When the remote control device 20 receives a real machine operation factor and a real machine identifier (Figure 6 / C220), it determines whether the real machine operation factor corresponds to the remote control factor associated with the real machine identifier (see Figure 6 / C120) (Figure 6 / STEP221).

[0061] For example, when the actual machine operation factor is operation setting B1, the machine operation mechanism 411 can remotely control the boom, arm, bucket, and swing, and when operation setting A1 is set as the remote operation factor to the remote operation mechanism 211, the two factors correspond to each other. Similarly, when operation setting A2 is set to operation setting B2, and operation setting A3 is set to operation setting B3, the two factors correspond to each other.

[0062] Even if the two factors correspond to each other, if the actual machine operation mechanisms that are mounted on the work machine 40 and can be remotely operated are increased or decreased due to modification, the two factors will no longer correspond to each other. Also, even if the two factors correspond to each other, if the remote operation device 20 switches the work machine that is the target of its remote operation, the two factors will no longer correspond to each other. Furthermore, if the setting of the operation pattern for the remote operation mechanism 211 is changed, the two factors will no longer correspond to each other.

[0063] If the determination result is positive (FIG. 6 / STEP 221...YES), the first flag f1 is set to "1" (FIG. 6 / STEP 222). On the other hand, if the determination result is negative (FIG. 6 / STEP 221...NO), the remote operation factor is changed to correspond to the real machine identifier (FIG. 6 / STEP 223), and the first flag f1 is set to "0" (FIG. 6 / STEP 223). That is, if the value of the first flag f1 is "1", it means that the remote operation factor and the real machine operation factor correspond to each other, and if the value of the first flag f1 is "0", it means that the remote operation factor and the real machine operation factor do not correspond to each other.

[0064] Next, in the remote operation device 20, it is determined whether or not there has been a change in the operation pattern by the remote operation mechanism 211 (FIG. 6 / STEP 225). For example, the operation pattern may change when the operator operating the remote operation device is changed. If there is a change in the operation pattern (FIG. 6 / STEP 225...YES), the operation input to the actual machine operation mechanism 211 will change, and therefore the amount by which the output of the actual machine operation mechanism should be corrected for the operation amount input to the remote operation mechanism 211 in order to operate the work machine 20 according to the operation amount input to the remote operation mechanism 211 will change.

[0065] If there is no change in the operation pattern (FIG. 6 / STEP 225...NO), it is determined whether or not the value of the first flag f1 is "0" (FIG. 6 / STEP 226). If the value of the first flag f1 is "0" (FIG. 6 / STEP 226...YES), there is a change in the remote operation factor and the operation input to the actual machine operation mechanism 211 changes, so the amount by which the output of the actual machine operation mechanism should be corrected for the operation amount input to the remote operation mechanism 211 in order to operate the work machine 20 according to the operation amount input to the remote operation mechanism 211 changes.

[0066] When the value of the first flag f1 is not "0" (FIG. 6 / STEP 225...NO), it is determined whether or not there is a change in the actual machine operation factor (FIG. 6 / STEP 227). For example, the actual machine operation mechanism 211 mounted on the work machine 40 that is the object of remote operation of the remote operation device 20 and that can be remotely operated increases or decreases due to modification, so that the actual machine operation factor at the time of the previous remote operation may not correspond to the actual machine operation factor at the time of the current remote operation. In addition, when the remote operation device 20 switches the work machine that is the object of its remote operation from the first work machine to the second work machine, the actual machine operation mechanism provided on the first work machine may differ from the actual machine operation mechanism provided on the second work machine, and in this case, the actual machine operation factor does not correspond. If the actual machine operation factors do not correspond, the amount by which the output of the actual machine operation mechanism 411 should be corrected for the operation amount input to the remote operation mechanism 211 in order to operate the work machine 20 according to the operation amount input to the remote operation mechanism 211 changes. If it is determined that there is no change in the actual machine operation factors (FIG. 6 / STEP 227...NO), the series of processes ends.

[0067] If there is a change in the operation pattern (FIG. 6 / STEP 225...YES), if the value of the first flag f1 is "0" (FIG. 6 / STEP 226...YES), or if there is a change in the actual machine operation factor (FIG. 6 / STEP 227...YES), a calibration request for adjusting the amount by which the output of the actual machine operation mechanism should be corrected for the operation amount input to the remote operation mechanism 211 in the remote operation device 20 is sent to the remote operation support server 10 together with the actual machine identifier (FIG. 6 / STEP 228).

[0068] When the remote operation support server 10 receives the calibration request and the actual machine identifier (FIG. 6 / C122), it transmits a calibration command to the remote operation device 20 and the work machine 40 that is the remote operation target of the remote operation device 20 (FIG. 6 / STEP 122).

[0069] When the work machine 40 receives a calibration command (FIG. 6 / C421), the calibration process of the actual machine operation mechanism 411 is executed (FIG. 6 / STEP 424), and when the remote operation device 20 receives a calibration command (FIG. 6 / C221), the calibration process of the remote operation mechanism 211 is executed (FIG. 6 / STEP 229).

[0070] For example, when a calibration process is executed to calibrate the operation amount of the real machine control lever constituting the real machine operation mechanism 411, the operation of the lever drive actuator is controlled to align the real machine control lever with the neutral position and the operation position with the maximum operation amount, the control command value or position of the lever drive actuator at each operation position is stored, and the lever drive control unit uses the stored control command value to create data that defines the correspondence relationship between the drive command of the operation lever and the control command value of the lever drive actuator. This determines the data so that the operation state or operation position of the real machine control lever constituting the real machine operation mechanism 411 in response to the drive command can be prevented from varying depending on the work machine 40 that is the remote control target.

[0071] After executing the process for creating the above-mentioned data, when the lever drive control unit operates the lever drive actuator in response to a drive command received from the lever operation command unit, the lever drive control unit executes operation control of the lever drive actuator using a control command value determined from the received drive command and the first stored and held data. This makes it possible to appropriately prevent the occurrence of variations in the operation state of the actual machine operation lever constituting the actual machine operation mechanism 411 of the work machine 40, which is realized in response to the operation of the remote operation mechanism 211 constituting the remote operation device 20. For example, it is possible to appropriately operate the first operation lever of the hydraulic work machine to the neutral position and / or to the maximum operation amount by operating the operation device, regardless of differences in the type, specifications, etc. of the work machine 40 to be remotely operated.

[0072] The operating lever is not limited to an operating portion that an operator operates manually, but may be an operating portion that an operator operates with his / her foot (for example, a pedal-type operating portion).

[0073] As shown by a solid line in Fig. 7, data related to the calibration process related to the operation of the actual machine control lever of the work machine 40 is generated as relationship data indicating the relationship between a drive command and a control command value so that the operation amount of the actual machine control lever corresponding to the control command value changes linearly with respect to the drive command in a drive command range of 0% to +100% and a drive command range of 0% to -100%. Here, 0% is a command not to drive the lever, +100% is a command to drive the lever maximally in the positive direction, and -100% is a command to drive the lever maximally in the negative direction. In Fig. 7, the relationship between the drive command represented by the relationship data before execution of the process and the operation amount of the actual machine control lever is illustrated by a two-dot chain line.

[0074] This makes it possible to remotely control the work machine 40 so that the hydraulic actuators operate in a similar manner in response to the operation of the control lever.

[0075] Data related to the calibration process related to the operation of the remote control lever constituting the remote control mechanism 211 of the remote control device 20 is created as relationship data that defines the relationship between the operation amount of the remote control lever and the drive command so that the drive command changes linearly with the change in the operation amount (swing angle) of the remote control lever in the drive command range from 0% to +100% and the drive command range from 0% to -100%, as shown by the solid line in Fig. 8. In Fig. 8, the relationship between the operation amount of the remote control lever and the drive command represented by the relationship data before the execution of the process is illustrated by a two-dot chain line.

[0076] This makes it possible to remotely control the work machine 20 so that the operation of the hydraulic actuator in response to the operation of the remote control lever constituting the remote operation mechanism 211 is similar to the operation of the hydraulic actuator in response to the operation of the actual machine operation lever constituting the actual machine operation mechanism 411. As a result of the above processing, the work machine 40 becomes capable of being remotely controlled as described above through the remote operation device 20 (see FIG. 4).

[0077] (Second Function (Second Embodiment)) A second embodiment of the second function as a calibration function of the remote operation support system having the above configuration will be described with reference to the flow chart shown in FIG.

[0078] In the remote operation device 20, the remote control device 200 transmits the remote operation factor together with the actual machine identifier to the remote operation support server 10 via the remote wireless communication device 222 (FIG. 7 / STEP 240). In response to a trigger operation (input of the actual machine identifier or designation of a remote operation mode) in the remote input interface 210, the remote operation factor and the actual machine identifier may be transmitted from the remote operation device 20.

[0079] In the work machine 40, the first support processing element 121 receives the remote operation factor and the actual machine identifier (FIG. 9 / C140), and transmits the remote operation factor to the work machine 40 identified by the actual machine identifier (FIG. 9 / STEP140).

[0080] In the corresponding work machine 40, when the real machine control device 400 receives a remote operation factor through the real machine wireless communication device 422 (FIG. 9 / C440), the real machine operation factor is recognized by reading or searching from a storage device or database (FIG. 9 / STEP 440). Then, the real machine control device 400 determines whether the real machine operation factor corresponds to the remote operation factor associated with the real machine identifier (see FIG. 9 / C420) (FIG. 9 / STEP 441).

[0081] If the determination result is positive (FIG. 9 / STEP 441...YES), the second flag f2 is set to "1" (FIG. 9 / STEP 443). On the other hand, if the determination result is negative (FIG. 9 / STEP 441...NO), the second flag f2 is set to "0" (FIG. 9 / STEP 444). That is, if the value of the second flag f2 is "1", it means that the remote operation factor and the real machine operation factor correspond to each other, and if the value of the second flag f2 is "0", it means that the remote operation factor and the real machine operation factor do not correspond to each other.

[0082] When the second flag f2 is set to "1" (Figure 9 / STEP 443) or the second flag f2 is set to "0" (Figure 9 / STEP 444), the real machine control device 400 transmits the second flag f2 to the remote operation support server 10 via the real machine wireless communication device 422 (Figure 9 / STEP 445).

[0083] In the remote operation support server 10, when the second flag f2 is received by the first support processing element 121, data representing the second flag f2 is transmitted to the remote operation device 20 (FIG. 9 / C141).

[0084] When the remote operation device 20 receives data indicating the value of the second flag f2 (FIG. 9 / C240), the second support processing element 122 determines whether the second flag f2 is "1" or not (FIG. 9 / STEP 241).

[0085] If the determination result is negative (FIG. 9 / STEP 241...NO), the remote control device 200 outputs a message for prompting the user to change the remote operation factor through the remote output interface 220 (FIG. 9 / STEP 242). If it is determined that the remote operation factor has not been changed (FIG. 9 / STEP 243...NO), the message is continuously output through the remote output interface 220 (FIG. 9 / STEP 242). If the duration of the message output exceeds a designated time, the series of processes may be terminated.

[0086] On the other hand, if it is determined that the remote control factor has been changed (FIG. 9 / STEP 243 . . . YES), the processing from transmission of the remote control factor (see FIG. 9 / STEP 240) onwards is repeated.

[0087] If the second support processing element 122 determines that the second flag f2 is “1” (FIG. 9 / STEP 241...YES), the remote control device 20 determines whether or not there has been a change in the operation pattern by the remote control mechanism 211 (FIG. 9 / STEP 244).

[0088] If there is no change in the operation pattern (FIG. 9 / STEP 244...NO), it is determined whether there is a change in the actual machine operation factor (FIG. 9 / STEP 245 ). If it is determined that there is no change in the actual machine operation factors (Fig. 9 / STEP 245 ...NO), the process ends.

[0089] If there is a change in the operation pattern (Fig. 9 / STEP 244 ‥ YES) or if there is a change in the actual operation factor (Fig. 9 / STEP 245 ...YES), a calibration request for adjusting the amount by which the output of the actual machine operating mechanism should be corrected for the operating amount input to the remote operation mechanism 211 by the remote operation device 20 is sent to the remote operation support server 10 together with the actual machine identifier (Figure 9 / STEP 246).

[0090] When the remote operation support server 10 receives a calibration request and an actual device identifier (FIG. 9 / C 142), and transmits a calibration command to the remote operation device 20 and the work machine 40 that is the remote operation target of the remote operation device 20 (FIG. 9 / STEP 140).

[0091] When the work machine 40 receives a calibration command (FIG. 9 / C441), the calibration process is executed (FIG. 9 / STEP 446), and when the remote control device 20 receives a calibration command (FIG. 9 / C241), the calibration process is executed (FIG. 9 / STEP 247).

[0092] In the second embodiment, a message is output to prompt the operator to change the remote control factor, so that the operator can be made aware of the content of the remote control factor.

[0093] (effect) According to the remote operation support system configured as above, when the remote operation factors do not correspond to the actual machine operation factors, a process is executed to make the remote operation factors of the remote operation mechanism 211 constituting the remote operation device 20 correspond to the actual machine operation factors of the actual machine operation mechanism 411 constituting the work machine 40. As a result, when both factors correspond to each other, the remote operation of the work machine 40 can be started by the remote operation device 20 without waiting for the completion of the execution of the calibration process. This improves the efficiency of the process of matching the settings of the remote operation device to the work machine to be remotely operated.

[0094] According to the remote operation support system having this configuration, the first support processing element 121 can determine whether or not the actual operation mechanism of the work machine 40 that is the target of remote operation by the remote operation device 20 has been changed. As a result, if the actual operation mechanism has not been changed, the remote operation of the work machine is started by the remote operation device without executing the process of matching the two operation factors, thereby improving efficiency. On the other hand, if the actual operation mechanism has been changed, the process of matching the factors is executed reliably.

[0095] According to the remote operation support system having the above configuration, when the process of matching both operation factors is executed, the process of matching the operation characteristics of the actual operation mechanism with the operation characteristics of the remote operation mechanism of the remote operation device 20 is executed. This makes it possible to maintain the same operability as before the change even if the operation factor is changed.

[0096] According to the remote operation support system having this configuration, when at least one of the operation pattern and the operation factor is changed, a process is performed to make the operation characteristics of the actual operation mechanism correspond to the operation characteristics of the remote operation mechanism of the remote operation device 20. As a result, even when only the operation pattern is changed, it is possible to maintain the same operability as before the change.

[0097] (Another embodiment of the present invention) At least some of the functional elements of the remote support processing server 10 may be configured by the remote operation device 20 and / or the work machine 40. For example, the first support processing element 121 may be configured by the remote control device 200 and / or the actual machine control device 400 as the first arithmetic processing device. The second support processing element 221 may be configured by the remote control device 200 and / or the actual machine control device 400 as the second arithmetic processing device. When the functional elements of the remote operation support server 10 are mounted on the remote operation device 20, information may be communicated by wired communication through a wired network mounted on the remote operation device 20 instead of the wireless communication in the above embodiment. Similarly, when the functional elements of the remote operation support server 10 are mounted on the work machine 40, information may be communicated by wired communication through a wired network mounted on the work machine 40 instead of the wireless communication in the above embodiment.

[0098] For example, with respect to the first embodiment of the second function, the functional elements represented by FIG. 6 / C121, STEP122, STEP123, STEP124, and STEP126 may be configured by the remote control device 200. The functional elements represented by FIG. 6 / C120, STEP120, C121, STEP122, STEP123, STEP124, and STEP126 may be configured by the real machine control device 400. The functional elements represented by FIG. 6 / C122, STEP127, and STEP128 may be configured by the remote control device 200 or the real machine control device 400.

[0099] Regarding the second embodiment of the second function, the functional elements represented by STEP 147 and STEP 148 in FIG.

[0100] Regarding the first embodiment of the second function, the recognition result of the actual machine operation factor by the work machine 40 may be sent to the remote operation support server 10 (FIG. 6 / STEP 222) before remote operation of the remote operation device 20. In this case, since the recognition result of the actual machine operation factor of the work machine 40 is stored in the database 102 of the remote operation support server 10, as shown in FIG. 10, a request to obtain an actual machine operation factor is transmitted from the remote operation device 20 (FIG. 10 / STEP 220), and when the request is received by the remote operation support server 10, the actual machine operation factor stored in the database 102 is recognized (FIG. 10 / STEP 520) without communicating with the work machine 40, thereby making it possible to transmit the actual machine operation factor to the remote operation device 20 (FIG. 10 / STEP 521).

[0101] Regarding the first embodiment of the second function, a message indicating the content of the remote control factor may be output through the remote output interface 220.

[0102] Regarding the second function, in the above-described embodiment, when the two factors do not correspond, when the operation pattern is changed, and when the actual machine operation factor is changed, the remote control device 20 requests calibration, but even when these conditions are met, the work efficiency may be improved by not requesting calibration as an exception. For example, when there is a history of past use of the work machine 40 remotely controlled by the remote control device 20, the settings of both operation factors, and the operation pattern, data related to the calibration process at that time can be used. [Explanation of symbols]

[0103] 10: Remote operation support server, 20: Remote operation device, 40: Work machine, 102: Database, 121: First support processing element, 122: Second support processing element, 200: Remote control device, 210: Remote input interface, 211: Remote operation mechanism, 220: Remote output interface, 221: Image output device, 222: Sound output device, 400: Actual machine control device, 410: Actual machine input interface, 420: Actual machine output interface, 424: Cab (operating room), 440: Work mechanism, 445: Bucket (working part).

Claims

1. A remote operation support system for supporting remote operation of a work machine by a remote operation device, comprising: a first support processing element that recognizes a determination result of whether or not remote operation factors that define operation settings that represent the correspondence of an actual machine operation mechanism that constitutes the work machine intended to be remotely operated by a remote operation mechanism that constitutes the remote operation device correspond to actual machine operation factors that define operation settings that represent the correspondence of the actual machine operation mechanism that can be operated by receiving remote operation; a second support processing element that executes a process for matching the actual machine operation factors and the remote operation factors based on communication with at least one of the remote operation device and the work machine, on the condition that the determination result recognized by the first support processing element is negative; A remote operation support system comprising:

2. 2. The remote operation support system according to claim 1, The first support processing element recognizes the determination result as to whether or not the remote operation factor corresponding to the actual machine operation mechanism set as a remote operation target by the remote operation mechanism corresponds to the actual machine operation factor corresponding to the actual machine operation mechanism constituting the work machine that is the target of remote operation of the remote operation mechanism. Remote operation support system.

3. 3. The remote operation support system according to claim 1, On the condition that the determination result recognized by the first support processing element is negative, the second support processing element executes a process of acquiring a correction value for making the operation characteristics of the actual machine operation mechanism constituting the work machine correspond to the operation characteristics of the remote operation mechanism constituting the remote operation device for remotely operating the work machine. Remote operation support system.

4. 3. The remote operation support system according to claim 2, the first support processing element recognizes an operation pattern when remotely operating the actual machine operation mechanism constituting the work machine that is the target of remote operation by the remote operation mechanism, The second support processing element executes a process of acquiring a correction value for making the operation characteristics of the actual operation mechanism constituting the work machine correspond to the operation characteristics of the remote operation mechanism constituting the remote operation device for remotely operating the work machine, on the condition that at least one of the determination result recognized by the first support processing element is negative and that there has been a change in the operation pattern recognized by the first support processing element. Remote operation support system.

5. In the remote operation support system according to any one of claims 1 to 4, The second support processing element outputs a message prompting a change of the remote operation factor to a remote output interface constituting the remote operation device, and recognizes the new remote operation factor changed through a remote input interface constituting the remote operation device. Remote operation support system.

6. 6. The remote operation support system according to claim 5, After the second support processing element recognizes the new remote operation factor changed through the remote input interface, the first support processing element recognizes the determination result as to whether or not the remote operation factor corresponds to the actual machine operation factor based on communication with the work machine. Remote operation support system.

7. A remote operation support system according to any one of claims 1 to 6, and at least one of the work machine and the remote operation device. Remote operation support complex system.

8. A remote operation assistance method for assisting remote operation of a work machine by a remote operation device, comprising: a first support processing step of recognizing a determination result of whether or not remote operation factors that define operation settings that represent the correspondence of an actual machine operation mechanism that constitutes the work machine intended to be remotely operated by a remote operation mechanism that constitutes the remote operation device correspond to actual machine operation factors that define operation settings that represent the correspondence of the actual machine operation mechanism that can be operated by receiving remote operation; a second support processing step of executing a process for matching the real-machine operation factors with the remote-operation factors, on the condition that the determination result recognized in the first support processing step is negative; A remote operation assistance method comprising:

9. A remote operation support server for supporting remote operation of a work machine by a remote operation device, a first support processing element that recognizes a determination result of whether or not remote operation factors that define operation settings that represent the correspondence of an actual machine operation mechanism that constitutes the work machine intended to be remotely operated by a remote operation mechanism that constitutes the remote operation device correspond to actual machine operation factors that define operation settings that represent the correspondence of the actual machine operation mechanism that can be operated by receiving remote operation; a second support processing element that executes a process for matching the actual machine operation factors and the remote operation factors based on communication with at least one of the remote operation device and the work machine, on the condition that the determination result recognized by the first support processing element is negative; A remote operation support server comprising:

10. A remote control device for assisting in remote control of a work machine, comprising: a first support processing element that recognizes a determination result of whether or not remote operation factors that define operation settings that represent the correspondence of an actual machine operation mechanism that constitutes the work machine intended to be remotely operated by a remote operation mechanism that constitutes the remote operation device correspond to actual machine operation factors that define operation settings that represent the correspondence of the actual machine operation mechanism that can be operated by receiving remote operation; a second support processing element that executes a process for matching the actual machine operation factors with the remote operation factors, on the condition that the determination result recognized by the first support processing element is negative; A remote control device comprising:

Images