Teleoperation assistance system, teleoperation assistance method, and teleoperation assistance complex system

Abstract

The present application provides a remote operation assistance system, a remote operation assistance method, and a remote operation assistance complex system. With the remote operation assistance system or the like, it is possible to achieve an increase in remote operation efficiency of a work machine through an increase in efficiency of calibration processing. A first assistance processing element (121) identifies a determination result of whether remote operation factors and actual machine operation factors correspond. The "remote operation factors" are factors that define operation settings of remote operation mechanisms (211) that constitute a remote operation device (20). The "actual machine operation factors" are factors that define operation settings of actual machine operation mechanisms (411) that constitute a work machine (40). With the determination result being negative as a necessary condition, a second assistance processing element (122) executes processing for causing the actual machine operation factors and the remote operation factors to correspond.

Description

Technical Field

[0001] This invention relates to a technology for remotely operating hydraulic excavators and other construction machinery via a remote control device. Background Technology

[0002] When operating machinery remotely via a remote control device, the settings of the remote control device must be matched with the machinery being operated.

[0003] Patent Document 1 discloses a correction value used to ensure that the operating characteristics of the actual machine operating mechanism, such as the operating lever, constituting the working machinery are consistent with the operating characteristics of the remote operating mechanism, such as the remote operating lever, constituting the remote operating device for remotely operating the working machinery. By correcting the operating quantity input to the remote operating mechanism based on this correction value and sending it to the actual machine operating mechanism, correction control is executed, causing the working machinery to operate according to the operating quantity input to the remote operating mechanism. A correspondence is established between the correction value and a combination of the form information of the working machinery and the form information of the remote operating mechanism.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-167732 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, the actual operating mechanism of the machine being remotely operated by the remote operating device may sometimes change. For example, the actual operating mechanism capable of remote operation mounted on the work machine being remotely operated by the remote operating device may be added or removed due to modifications. Furthermore, when the remote operating device switches the work machine being remotely operated from a first work machine to a second work machine, the actual operating mechanism installed on the first work machine may sometimes become incompatible with the actual operating mechanism installed on the second work machine. If the actual operating mechanism changes, the registered calibration values ​​may become incompatible, necessitating the acquisition and re-registration of calibration values.

[0009] Furthermore, the remote operating mechanisms, such as the remote control joystick, that constitute the remote operating device may sometimes change. For example, the operating mode of the remote operating mechanism may change. As a result, because the remote operating mechanism corresponding to the actual machine operating mechanism has changed, the registered calibration values ​​may no longer be corresponding, thus requiring the calibration values ​​to be obtained and registered again.

[0010] Therefore, the following scenario is envisioned: at the start of remote operation, by confirming and registering the calibration value, a process is performed that enables the settings of the remote operation device to reliably match the working machinery of the remotely operated object.

[0011] However, if this process is performed every time communication between the remote operating device and the working machine is established, the process will be performed even if there are no changes to the actual machine operating mechanism or the remote operating mechanism, and the remote operation of the working machine cannot begin until the process is completed.

[0012] Therefore, the object of the present invention is to provide a server or the like that can improve the remote operation efficiency of machinery by optimizing the process of matching the settings of the remote operation device with the machinery to be remotely operated.

[0013] Methods for solving problems

[0014] The remote operation assistance system of the present invention is used to assist a remote operation device in remotely operating a work machine. It is characterized by comprising: a first auxiliary processing element that identifies a determination result indicating whether a remote operation factor corresponds to a physical machine operation factor, wherein the remote operation factor defines the operation settings of the remote operation mechanism constituting the remote operation device, and the physical machine operation factor defines the operation settings of the physical machine operation mechanism constituting the work machine; and a second auxiliary processing element that, on the condition that the determination result identified by the first auxiliary processing element is negative, performs processing to make the physical machine operation factor correspond to the remote operation factor based on communication with at least one of the remote operation device and the work machine.

[0015] According to the remote operation assistance system with this configuration, when the remote operation factors and the actual machine operation factors do not correspond, a process is executed to correspond the operation mode of the remote operation mechanism constituting the remote operation device with the operation mode of the actual machine operation mechanism constituting the work machinery. Therefore, when the two factors are consistent, remote operation of the work machinery by the remote operation device can begin without executing the process for corresponding the two operation factors. Thus, by improving the efficiency of this process, the remote operation efficiency of the work machinery can be increased.

[0016] Brief description of the attached diagram

[0017] Figure 1 This is an explanatory diagram of the remote operation assistance system of the present invention.

[0018] Figure 2 This is an explanatory diagram of the configuration of a remote operation device.

[0019] Figure 3 It is an explanatory diagram related to the structure of the operating machinery.

[0020] Figure 4 This is an explanatory diagram related to the primary function of the operation assistance system.

[0021] Figure 5 It is an explanatory diagram of the working environment.

[0022] Figure 6 This is an explanatory diagram related to the first embodiment of the second function of the operation assistance system.

[0023] Figure 7 This is an explanatory diagram about the relationship between drive commands and the actual operation of the joystick.

[0024] Figure 8 This is an explanatory diagram about the relationship between drive commands and the operation of the remote control lever.

[0025] Figure 9 This is an explanatory diagram related to a second implementation of the second function of the operation assistance system.

[0026] Figure 10 This is an explanatory diagram related to a variation of the second function of the operation assistance system. Detailed Implementation

[0027] (Composition of a remote operation assistance system)

[0028] Figure 1 The remote operation assistance system shown comprises a remote operation assistance server 10, a remote operation device 20, and / or a working machine 40. The remote operation assistance server 10, the remote operation device 20, and the working machine 40 are configured to communicate with each other via a network. The communication networks between the remote operation assistance server 10 and the remote operation device 20, and between the remote operation assistance server 10 and the working machine 40, can be the same or different communication networks.

[0029] The concept of "identification information" as a constituent element of the present invention includes the following forms: receiving the information, reading the information from a storage device, retrieving the information from a database, measuring the information, determining, judging, presuming or predicting the information based on the received basic information, storing the information in a storage device, and all such computational processes that make the information available for use in subsequent computational processing.

[0030] (Composition of a remote operation auxiliary server)

[0031] The remote operation assistance server 10, constituting the remote operation assistance system, includes a database 102, a first auxiliary processing element 121, and a second auxiliary processing element 122. The database 102 stores and saves captured image data, etc. The database 102 may also be composed of a database server different from the remote operation assistance server 10. Each auxiliary processing element is composed of a computing processing device (a single-core processor or a multi-core processor, or a processor core constituting a single-core or multi-core processor), which reads the required data and software from storage devices such as memory, and performs computational processing according to the software, as described later, on the object of that data.

[0032] (Composition of the remote operation device)

[0033] The remote operation device 20 includes a remote control device 200, a remote input interface 210, and a remote output interface 220. The remote control device 200 is composed of a computing device (a single-core processor or a multi-core processor, or a processor core constituting a single-core processor or a multi-core processor), which reads the required data and software from storage devices such as memory, and performs computing processing according to the software with the data as the object.

[0034] The remote input interface 210 includes a remote operation mechanism 211. The remote output interface 220 includes an image output device 221, an audio output device 222, and a remote wireless communication device 224.

[0035] The remote operating mechanism 211 includes a travel operating device, a slewing operating device, a boom operating device, a stick operating device, and a bucket operating device. Each operating device has an operating lever that accepts rotational operation. The operating lever (travel lever) of the travel operating device is operated to move the lower traveling body 410 of the working machine 40. The travel lever can also serve as a travel pedal. For example, a travel pedal fixed to the base or lower end of the travel lever can also be provided. The operating lever (slewing lever) of the slewing operating device is operated to move the hydraulic slewing motor that constitutes the slewing mechanism 430 of the working machine 40. The operating lever (boom lever) of the boom operating device is operated to actuate the boom cylinder 442 of the working machine 40. The operating lever (stick lever) of the stick operating device is operated to actuate the stick cylinder 444 of the working machine 40. The operating lever (bucket stick) of the bucket operating device is operated to actuate the bucket cylinder 446 of the working machine 40.

[0036] like Figure 2 As shown, the levers constituting the remote operation mechanism 211 are arranged, for example, around a seat St for the operator to sit on. The seat St is a high-backed chair with armrests, but it can also be a low-backed chair without a headrest, or a chair without a backrest, or any other seating arrangement that allows the operator to sit on it.

[0037] A pair of left and right travel levers 2110 are arranged in front of the seat St, corresponding to the left and right tracks. One lever can also function as multiple levers. For example, it can be used in… Figure 2 The left-side operating lever 2111, located in front of the left-side frame of the seat St, functions as an arm when operated in the forward / backward direction and as a swivel lever when operated in the left / right direction. Similarly, it can be used to... Figure 2 The right-side operating lever 2112, located in front of the right-side frame of the seat St, functions as a boom lever when operated in the forward / backward direction and as a bucket lever when operated in the left / right direction. The lever pattern can be changed arbitrarily according to the operator's instructions.

[0038] For example, such as Figure 2 As shown, the image output device 221 comprises a central image output device 2210, a left image output device 2211, and a right image output device 2212, each having a roughly rectangular screen positioned in front of, to the left diagonally in front of, and to the right diagonally in front of the seat St. The shape and size of the screens (image display areas) of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be the same or different.

[0039] like Figure 2 As shown, with the screen of the central image output device 2210 and the screen of the left image output device 2211 tilted at an angle θ1 (e.g., 120°≤θ1≤150°), the right edge of the left image output device 2211 is adjacent to the left edge of the central image output device 2210. Figure 2 As shown, the screen of the central image output device 2210 and the screen of the right image output device 2212 are tilted at an angle θ2 (e.g., 120°≤θ2≤150°), and the left edge of the right image output device 2212 is adjacent to the right edge of the central image output device 2210. The tilt angle θ1 and the tilt angle θ2 can be the same or different.

[0040] The images of the central image output device 2210, the left image output device 2211, and the right image output device 2212 can each be parallel to or tilted relative to the vertical direction. At least one of the central image output device 2210, the left image output device 2211, and the right image output device 2212 can also be composed of multiple image output devices. For example, the central image output device 2210 can also be composed of a pair of vertically adjacent image output devices with approximately rectangular images.

[0041] The audio output device 222 consists of one or more speakers, for example, such as Figure 2 As shown, the audio output device 222 consists of a central audio output device 2220, a left audio output device 2221, and a right audio output device 2222, respectively located behind the seat St, behind the left armrest, and behind the right armrest. The specifications of the central audio output device 2220, the left audio output device 2221, and the right audio output device 2222 can be the same or different.

[0042] (Structure of the operating machinery)

[0043] The operating machinery 40 includes a machine control device 400, a machine input interface 41, a machine output interface 42, and a working mechanism 440. The machine control device 400 is composed of a computing processing unit (a single-core processor or a multi-core processor, or a processor core constituting a single-core processor or a multi-core processor), which reads the required data and software from storage devices such as memory, and performs computing processing according to the software based on the data.

[0044] Construction machinery 40, for example, is a tracked excavator (construction machinery), such as Figure 3 As shown, the device includes a tracked lower traveling body 410 and an upper rotating body 420 rotatably mounted on the lower traveling body 410 via a rotating mechanism 430. A driver's cab 424 is provided on the front left side of the upper rotating body 420. A working mechanism 440 is provided on the front center of the upper rotating body 420.

[0045] The machine input interface 41 includes a machine operation mechanism 411, a machine imaging device 412, and a positioning device 414. The machine operation mechanism 411 has multiple joysticks arranged similarly to those of the remote operation mechanism 211 around the seat located inside the cab 424. A drive mechanism or robot is installed inside the cab 424, which receives signals corresponding to the operation mode of the remote joysticks and actuates the machine operation joysticks based on the received signals. The machine imaging device 412 is installed, for example, inside the cab 424, and captures images of at least a portion of the environment including the operating mechanism 440 through the front window and a pair of side windows. Alternatively, some or all of the front and side windows may be omitted. The positioning device 414 is composed of a GPS device or, if necessary, a gyroscope sensor, etc.

[0046] The actual output interface 42 is equipped with actual wireless communication device 422.

[0047] like Figure 3As shown, the working mechanism 440, which serves as the working mechanism, includes: a boom 441, which is mounted on the upper slewing body 420 for lifting and lowering; a stick 443, which is rotatably connected to the top of the boom 441; and a bucket 445, which is rotatably connected to the top of the stick 443. The working mechanism 440 is equipped with a boom cylinder 442, a stick cylinder 444, and a bucket cylinder 446, all of which are telescopic hydraulic cylinders.

[0048] The boom cylinder 442 extends and retracts by receiving a supply of hydraulic fluid, allowing the boom 441 to rotate in the lifting direction in a manner between the boom 441 and the upper slewing body 420. The stick cylinder 444 is located between the stick 443 and the boom 441, extending and retracting by receiving a supply of hydraulic fluid, allowing the stick 443 to rotate about a horizontal axis relative to the boom 441. The bucket cylinder 446 is located between the bucket 445 and the stick 443, extending and retracting by receiving a supply of hydraulic fluid, allowing the bucket 445 to rotate about a horizontal axis relative to the stick 443.

[0049] (Primary Function)

[0050] Figure 4 This is a flowchart explaining the first function, which is the basic function of the remote operation assistance system having the above-described configuration. In this flowchart, the "C●" box is a symbol used for simplification and refers to the transmission and / or reception of data, and conditional branches in which branch-direction processing is performed based on the transmission and / or reception of such data. The received data is stored in a storage device consisting of database 102 and / or non-volatile or volatile memory. These explanatory points are also the same in the flowcharts described later.

[0051] In the remote operation device 20, the operator determines whether a specified operation has been performed via the remote input interface 210. Figure 4 / Step (STEP) 210). "Specified operation" is, for example, an operation such as a click in the remote input interface 210 by the operator to specify the work machinery 40 to be remotely operated. In the case where the determination result is negative ( Figure 4 / Step 210: No (NO)), end the series of processes. On the other hand, if the determination result is positive ( Figure 4 / Step 210: Yes, send an environment confirmation request to the remote operation assistance server 10 via the remote wireless communication device 224. Figure 4 / Step 212).

[0052] In the remote operation assistance server 10, upon receiving an environmental confirmation request, the first auxiliary processing element 121 sends the environmental confirmation request to the operating machinery 40 corresponding to the environmental confirmation request. Figure 4 / C110).

[0053] In the operating machinery 40, when an environmental confirmation request is received via the actual machine wireless communication device 422 ( Figure 4 / C410), the actual control device 400 uses the actual shooting device 412 to acquire the captured image ( Figure 4 / Step 410). Here, image processing can also be performed by the physical control device 400 or the image processing device constituting the physical control device 400. The physical control device 400 uses the physical wireless communication device 422 to send the captured image data that has undergone the image processing to the remote operation assistance server 10. Figure 4 / Step 412).

[0054] In the remote operation auxiliary server 10, when the captured image data is received by the first auxiliary processing element 121 ( Figure 4 / C112), the second auxiliary processing element 122 sends environmental image data corresponding to the captured image to the remote operation device 20. Figure 4 (Step 110). Environmental image data can be, in addition to the captured image data itself, image data representing a simulated environmental image generated based on the captured image. When the image processing device is configured as a remote operation auxiliary server, environmental image data can also be generated by performing image processing on the captured image data by the image processing device.

[0055] In the remote operation device 20, when environmental image data is received via the remote wireless communication device 224 ( Figure 4 / C210), via remote control device 200, outputs the environmental image corresponding to the environmental image data to image output device 221 (C210). Figure 4 / Step 214).

[0056] Thus, for example, such as Figure 5 As shown, an environmental image of the boom 441, stick 443 (which is part of the working mechanism 440), and the hill of rubble or sand (which is the work object of the bucket 445) that is reflected in front of the cab 424 through the window frames formed by the right window frame Q1, upper window frame Q2, left window frame Q3, and lower window frame Q4 of the cab 424 is output to the image output device 421. An environmental image can also be generated by image processing of the captured image or by adjusting the field of view of the actual shooting device 412 so that at least a portion of the window frames Q1 to Q4 are not reflected in the environmental image. When the actual shooting device 412 is located outside the cab 424 instead of inside the cab 424, it is possible to obtain captured images in which the components of the cab 424, such as the window frames Q1 to Q4, are not reflected, thereby obtaining an environmental image.

[0057] In the remote operation device 20, the operation mode of the remote operation mechanism 211 is identified by the remote control device 200. Figure 4 / Step 216), and, through the remote wireless communication device 224, send a remote operation command corresponding to the operation mode to the remote operation assistance server 10 ( Figure 4 / Step 218).

[0058] In the remote operation auxiliary server 10, when the second auxiliary processing element 122 receives the remote operation command, the first auxiliary processing element 121 sends the remote operation command to the working machine 40. Figure 4 / C114).

[0059] In the operating machinery 40, when the actual machine control device 400 receives the operation command through the actual machine wireless communication device 422 ( Figure 4 / C412), controls the actions of the working mechanism 440, etc. Figure 4 / Step 414). For example, perform the following operation: use the bucket 445 to scoop up the soil in front of the working machine 40, and after the upper rotating body 420 rotates, the soil falls off the bucket 445.

[0060] (Second Function (First Implementation))

[0061] Figure 6 This is a flowchart illustrating a first embodiment of the second function, wherein the second function is a calibration function of a remote operation assistance system having the above-described configuration.

[0062] In the remote operation device 20, the remote control device 200 sends the remote operation factors along with the physical machine identifier to the remote operation auxiliary server 10 via the remote wireless communication device 224. Figure 6 (Step 220). Alternatively, if a trigger operation (input of a machine identifier or specification of a remote operation mode) is performed at the remote input interface 210, remote operation factors and a machine identifier can be sent from the remote operation device 20. "Remote operation factors" are factors that define the remote operation settings of the remote operation mechanism 211 constituting the remote operation device 20. "Remote operation factors" correspond to the machine operation mechanism 411 of the working machine 40, where the working machine 40 is the machine that the remote operation mechanism 211 intends to remotely operate. The above correspondence is illustrated in Table 1.

[0063] [Table 1]

[0064] Remote operating mechanism Operating direction Remote operation settings A1 Remote operation settings A2 Remote operation settings A3 … Left control lever Forward tilt Push rod Push rod Push rod … Left control lever Rear tilt Pulling rod Pulling rod Pulling rod … Left control lever Rear tilt Turn right Turn right Turn right … Left control lever tilt to the left Left turn Left turn Left turn … Right control lever Forward tilt Landing boom Landing boom Landing boom … Right control lever Rear tilt Raise the boom Raise the boom Raise the boom … Right control lever tilt to the right Bucket digging Bucket digging Bucket digging … Right control lever tilt to the left Bucket dumping soil Bucket dumping soil Bucket dumping soil … Left driving lever Forward tilt - Drive forward on the left Drive forward on the left … Left driving lever Rear tilt - Drive to the left rear Drive to the left rear … Right-hand drive lever Forward tilt - Drive to the right front Drive to the right front … Right-hand drive lever Rear tilt - Drive to the right rear Drive to the right rear … OPT pedals Forward tilt - - Open … OPT pedals Rear tilt - - closure … … … … … … …

[0065] As shown in Table 1, when the actual machine operating mechanism 411 constituting the work machine 40 intended to be remotely operated by the remote operating mechanism 211 is the boom control lever, stick control lever, bucket control lever, and swing control lever, remote operation setting A1 is set. Remote operation setting A1 sets remote operation generated by tilting the left and right control levers, but does not set remote operation generated by tilting the travel lever or optional pedal. When the operation of the remote operating mechanism 211 is intended to drive the work machine 40, which moves in tandem with the lower traveling body 410, remote operation setting A2 is set, setting remote operation generated by tilting the travel lever. When the operation of the remote operating mechanism 211 is intended to operate optional devices such as grab buckets, remote operation setting A3 is set, setting remote operation generated by tilting the OPT pedal. The remote operation setting can also be set to settings other than remote operation settings A1 to A3, such as changing the operating mode setting. "Actual machine identifier" is an identifier or data used to identify or determine the work machine 40 as the object of remote operation by the remote operating device 20.

[0066] In the remote operation assistance server 10, the first assistance processing element 121 receives remote operation factors and the physical machine identifier ( Figure 6 / C120), sends a request to obtain machine operation factors to the operating machine 40 identified by the machine identifier. Figure 6 / Step 220).

[0067] In the operating machinery 40, when the actual machine control device 400 receives a request for obtaining actual machine operation factors through the actual machine wireless communication device 422 ( Figure 6 / C420) identifies it by reading or retrieving real-machine operating factors from a storage device or database. Figure 6 / Step 420). Then, the machine control device 400 sends the machine operation factors and machine identifier to the remote operation assistance server 10 via the machine wireless communication device 422. Figure 6 / Step 422). "Actual operation factors" are factors that define the actual operation settings of the actual operation mechanism 411 constituting the working machine 40, and correspond to the actual operation mechanism 411 that can accept remote operation. This correspondence is illustrated in Table 2.

[0068] [Table 2]

[0069] Actual machine operating mechanism Actual operation settings B1 Actual operation settings B2 Actual operation settings B3 … Push rod 〇 〇 〇 … Pulling rod 〇 〇 〇 … Turn right 〇 〇 〇 … Left turn 〇 〇 〇 … Landing boom 〇 〇 〇 … Raise the boom 〇 〇 〇 … Bucket digging 〇 〇 〇 … Bucket dumping soil 〇 〇 〇 … Drive forward on the left - 〇 〇 … Drive to the left rear - 〇 〇 … Drive to the right front - 〇 〇 … Drive to the right rear - 〇 〇 … OPT Open - - 〇 … OPT Closure - - 〇 … … … … … …

[0070] As shown in Table 2, when the actual operating mechanism 411 constituting the work machinery 40 intended to be remotely operated by the remote operating mechanism 211 is a boom control lever, stick control lever, bucket control lever, and swing control lever, actual operation setting B1 is set. Actual operating mechanisms marked with ○ in actual operation setting B1 can operate by receiving remote operation; actual operating mechanisms not marked with ○ in actual operation setting B1 cannot operate even if remote operation is received. When the actual operating mechanism 411 can receive remote operation for travel, actual operation setting B2 is set; when the actual operating mechanism 411 can receive remote operation for optional devices such as grab buckets, actual operation setting B3 is set. Actual operation factors are set by: the work machinery control device 400 recognizing the structure of the actual operating mechanism 411 of the work machinery 40; and input from maintenance personnel performing maintenance on the work machinery 40.

[0071] In the remote operation assistance server 10, the first assistance processing element 121 receives the actual operation factors and the actual identifier ( Figure 6 / C121), further, the remote operation device 20 receives the actual machine operation factors and the actual machine identifier ( Figure 6 / C220).

[0072] In the remote operation device 20, when the actual machine operation factors and the actual machine identifier are received ( Figure 6 / C220), determine the actual machine operation factor and the remote operation factor corresponding to the actual machine identifier (refer to...). Figure 6 Does / C120 correspond to ( Figure 6 / Step 221).

[0073] For example, when the actual machine operation factor is actual machine operation setting B1, the actual machine operation mechanism 411 can accept remote operation of the boom, stick, bucket, and swing. When the remote operation mechanism 211 is set to remote operation setting A1 as the remote operation factor, the two factors are in corresponding states. Similarly, when the remote operation setting A2 is set to correspond to actual machine operation setting B2 and the remote operation setting A3 is set to correspond to actual machine operation setting B3, the two factors are also in corresponding states.

[0074] Even if the two factors correspond, they will become incompatible if the actual machine operating mechanism mounted on the work machinery 40 and capable of remote operation is added or removed due to modifications. Furthermore, even if the two factors correspond, they will become incompatible if the remote operating device 20 switches the work machinery to be remotely operated. Moreover, they will become incompatible if the operating mode setting of the remote operating mechanism 211 is changed.

[0075] If the judgment result is affirmative ( Figure 6 Step 221: Yes, set the first flag f1 to "1" ( Figure 6 / Step 222). On the other hand, when the determination result is negative ( Figure 6 / Step 221: No), the remote operation factor is changed to correspond to the physical machine identifier ( Figure 6 / Step 223), and set the first label f1 to "0" ( Figure 6 / Step 224). That is, when the value of the first label f1 is "1", it means that the remote operation factor corresponds to the actual operation factor, while when the value of the first label f1 is "0", it means that the remote operation factor does not correspond to the actual operation factor.

[0076] Next, in the remote operation device 20, it is determined whether the operation mode has been changed by the remote operation mechanism 211. Figure 6 / Step 225). For example, the operating mode may be changed when the operator of the remote operating device changes. In cases where the operating mode changes ( Figure 6 / Step 225: Yes), the operation input to the remote operating mechanism 211 is changed, therefore, the amount by which the output of the actual operating mechanism 411 should be corrected changes, wherein the output of the actual operating mechanism 411 corresponds to the operation amount input to the remote operating mechanism 211, which is the operation amount that makes the working machine 40 work according to the operation amount input to the remote operating mechanism 211.

[0077] Without changing the operating mode ( Figure 6 / Step 225: No), determine whether the value of the first label f1 is "0" ( Figure 6 / Step 226). In the case where the value of the first label f1 is "0" ( Figure 6 / Step 226: Yes), because the remote operation factors have changed, the operation input to the remote operation mechanism 211 has changed. Therefore, the amount by which the output of the actual machine operation mechanism should be corrected relative to the operation amount input to the remote operation mechanism 211 has changed. The operation amount input to the remote operation mechanism 211 is the operation amount that enables the working machine 40 to work according to the operation amount input to the remote operation mechanism 211.

[0078] If the value of the first label f1 is not "0" ( Figure 6 Step 226: No), determine whether the actual machine operation factors have changed ( Figure 6 / Step 227). For example, if the machine operation mechanism 411, which is the object of remote operation of the remote operation device 20, is added or removed due to modification, there may be a situation where the machine operation factors at the time of the last remote operation do not correspond to the machine operation factors at the time of the current remote operation. In addition, when the remote operation device 20 switches the machine operation object from the first machine operation to the second machine operation, the machine operation mechanism installed in the first machine operation may be different from the machine operation mechanism installed in the second machine operation. In this case, the machine operation factors do not correspond. If the machine operation factors do not correspond, the amount by which the output of the machine operation mechanism 411 should be corrected changes. The output of the machine operation mechanism 411 corresponds to the operation amount input to the remote operation mechanism 211, which is the operation amount that makes the machine operation 40 work according to the operation amount input to the remote operation mechanism 211. If it is determined that the machine operation factors have not changed ( Figure 6 / Step 227: No), end the series of processes.

[0079] In the event of a change in operating mode ( Figure 6 / Step 225: Yes), in the case that the value of the first label f1 is "0" ( Figure 6 / Step 226: Yes), and in the event of a change in actual machine operation factors ( Figure 6 / Step 227: Yes), send the calibration request along with the physical identifier to the remote operation assistance server 10 ( Figure 6 / Step 228), wherein the aforementioned correction request is a request in the remote operating device 20 for adjusting the amount by which the output of the physical operating mechanism should be corrected, the output of the physical operating mechanism being the output relative to the operating amount input to the remote operating mechanism 211.

[0080] When the remote operation auxiliary server 10 receives the correction request and the physical identifier ( Figure 6 / C122), sends a correction command to the remote operation object of the remote operation device 20, namely the working machinery 40 and the remote operation device 20. Figure 6 / Step 122).

[0081] When the working machine 40 receives the correction command ( Figure 6 / C421), performs the calibration process of the actual machine operating mechanism 411 ( Figure 6 / Step 424), when the remote operating device 20 receives the calibration command ( Figure 6 / C221), performs the calibration process of the remote operating mechanism 211 ( Figure 6 / Step 229).

[0082] For example, when performing a correction process to adjust the operating amount of the actual operating lever constituting the actual operating mechanism 411, the lever drive actuator is activated to align the actual operating lever with the neutral position and the operating position with the maximum operating amount, respectively, and the control command value or position of the lever drive actuator at each operating position is stored. The lever drive control unit uses the stored control command value to create data defining the correspondence between the drive command of the operating lever and the control command value of the lever drive actuator. As a result, this data can be determined in a way that prevents deviations from the operating state or operating position of the actual operating lever constituting the actual operating mechanism 411 corresponding to the drive command corresponding to the work machine 40, which is the object of remote operation.

[0083] After processing the aforementioned data, the lever drive control unit, upon actuating the lever drive actuator according to the drive command received from the lever operation command unit, executes the operation control of the lever drive actuator based on the control command value determined using the received drive command and the stored data. This appropriately prevents deviations in the operating state of the actual operating lever of the actual operating mechanism 411 constituting the working machine 40, which is achieved based on the operation of the remote operating mechanism 211 constituting the remote operating device 20. For example, regardless of the type or specifications of the working machine 40 being remotely operated, the operation of the operating device can appropriately achieve the operation of the first operating lever of the hydraulic working machine to the neutral position and / or the operation of the first operating lever to the maximum operating amount.

[0084] The control lever is not limited to a control part that is operated manually by the operator; it can also be a control part that is operated by the operator using the foot (e.g., a pedal-type control part).

[0085] The data associated with the calibration processing related to the operation of the control lever of the machine tool 40 is generated as the following relational data representing the relationship between drive commands and control commands: such as Figure 7 As shown by the solid line, within the range of drive commands from 0% to +100% and from 0% to -100%, the operation of the actual control stick corresponding to the control command value varies linearly with respect to the drive command. Here, 0% is a command to not drive the stick, +100% is a command to drive the stick to its maximum extent in the positive direction, and -100% is a command to drive the stick to its maximum extent in the negative direction. Figure 7 In the diagram, the relationship between the drive commands represented by the relational data before the correction process is executed and the actual operation of the joystick is illustrated by the double-dotted line.

[0086] Therefore, the machine 40 can be remotely operated in a manner that makes the movement of the machine 40 the same as the movement of the hydraulic actuator corresponding to the operation of the control lever.

[0087] The data associated with the correction processing related to the operation of the remote control lever of the remote control mechanism 211 constituting the remote control device 20 is generated as relational data defining the relationship between the operation amount of the remote control lever and the drive command as follows: Figure 8 As shown by the solid line, the change in drive command relative to the amount of joystick manipulation (swing angle) varies linearly within the ranges of drive command from 0% to +100% and from 0% to -100%. Figure 8 In the diagram, the relationship between the operation amount of the remote control lever and the drive command, represented by the relational data before the correction process is executed, is illustrated by the double-dotted line.

[0088] Therefore, the machine tool 40 can be remotely operated so that the operation of the hydraulic actuator corresponding to the operation of the remote control lever constituting the remote operation mechanism 211 is the same as the operation of the hydraulic actuator corresponding to the operation of the actual machine control lever constituting the actual machine operation mechanism 411. As a result of the above processing, a state is formed in which the machine tool 40 can be remotely operated by the remote operation device 20 in the above manner (see reference). Figure 4 ).

[0089] (Second Function (Second Implementation))

[0090] Figure 9 This is a flowchart describing a second implementation of the calibration function as a second function of a remote operation assistance system having the above-described configuration.

[0091] In the remote operation device 20, the remote control device 200 sends remote operation factors and a physical identifier to the remote operation auxiliary server 10 via the remote wireless communication device 224 (Figure / Step 240). Alternatively, the remote operation factors and the physical identifier can be sent from the remote operation device 20 depending on whether a trigger operation (input of the physical identifier or specification of the remote operation mode) is performed at the remote input interface 210.

[0092] In the operating machinery 40, the first auxiliary processing element 121 receives remote operation factors and the actual machine identifier ( Figure 9 / C140), and sends remote operation factors ( / C140) to the operating machinery 40 identified according to the actual machine identifier. Figure 9 / Step 240).

[0093] In the operating machinery 40, when the actual machine control device 400 receives remote operation factors (via the actual machine wireless communication device 422) Figure 9When / C440), the actual operational factors are identified by reading or retrieving from the storage device or database. Figure 9 / Step 440). Then, the machine control device 400 determines the machine operation factor and the remote operation factor corresponding to the machine identifier (see...). Figure 9 Does / C420 correspond?

[0094] If the judgment result is affirmative ( Figure 9 / Step 441: Yes, the second label f2 is set to "1" ( Figure 9 / Step 443). On the other hand, if the determination result is negative ( Figure 9 / Step 441…No (NO)), the second label f2 is set to "0" ( Figure 9 / Step 444). That is, when the value of the second label f2 is "1", it means that the remote operation factor corresponds to the actual operation factor; when the value of the second label f2 is "0", it means that the remote operation factor does not correspond to the actual operation factor.

[0095] When the second label f2 is set to "1" ( Figure 9 / step 443), or the second label f2 is set to "0" ( Figure 9 During step 444), the machine control device 400 sends the second tag f2 to the remote operation assistance server 10 via the machine wireless communication device 422. Figure 9 / Step 445).

[0096] In the remote operation assistance server 10, when the first assistance processing element 121 receives the second tag f2, it sends the data representing the second tag f2 to the remote operation device 20. Figure 9 / C141).

[0097] In the remote operation device 20, when data representing the value of the second tag f2 is received ( Figure 9 When / C240), the second auxiliary processing element 122 determines whether the second tag f2 is "1" ( Figure 9 / STEP241).

[0098] When the judgment result is negative ( Figure 9 / Step 241: No), the remote control device 200 outputs a message through the remote output interface 220 to prompt changes in remote operation factors. Figure 9 / Step 242). If it is determined that the remote operation factors have not changed ( Figure 9 / Step 243…No), continuously output the above message through remote output interface 220 ( Figure 9(Step 242). If the output duration of this message exceeds the specified time, the series of processes can also be terminated.

[0099] On the other hand, in cases where it is determined that the factors for remote operation have changed ( Figure 9 / Step 243… is), repeatedly send remote operation factors (refer to Figure 9 / Step 240) and subsequent processing.

[0100] When the second auxiliary processing element 122 determines that the second tag f2 is "1" ( Figure 9 / STEP241: Yes), in the remote operating device 20, it is determined whether the remote operating mechanism 211 has changed the operating mode ( Figure 9 / STEP244).

[0101] Without a change in operating mode ( Figure 9 / Step 244: No), determine whether the actual machine operation factors have changed ( Figure 9 / Step 245). If it is determined that the actual machine operation factors have not changed ( Figure 9 / Step 245: No), end the series of processes.

[0102] In the event of a change in operating mode ( Figure 9 / Step 244: Yes), and in the event of a change in actual machine operation factors ( Figure 9 / Step 245: Yes), the calibration request is sent to the remote operation assistance server 10 along with the physical identifier. Figure 9 / Step 246), wherein the correction request is a request to adjust the amount by which the output of the physical operating mechanism should be corrected, the output of which corresponds to the operating amount input to the remote operating mechanism 211 in the remote operating device 20.

[0103] When the remote operation auxiliary server 10 receives the calibration request and the physical identifier ( Figure 9 / C142), sends a correction command to the remote operation object of the remote operation device 20, namely the working machinery 40 and the remote operation device 20. Figure 9 / Step 140).

[0104] When the operating machine 40 receives a correction command ( Figure 9 / C441), perform correction processing ( Figure 9 / Step 446), the remote operating device 20 receives the calibration command ( Figure 9 When / C241), perform the correction process ( Figure 9 / Step 247).

[0105] In the second embodiment, a message urging the operator to change remote operating factors is output. Therefore, the operator is made aware of the content of the remote operating factors.

[0106] (Effect)

[0107] According to the remote operation assistance system with this configuration, when the remote operation factors and the actual machine operation factors do not correspond, a process is performed to match the remote operation factors of the remote operation mechanism 211 constituting the remote operation device 20 with the actual machine operation factors of the actual machine operation mechanism 411 constituting the work machine 40. Therefore, when the two factors correspond, remote operation of the work machine 40 by the remote operation device 20 can begin without waiting for the calibration process to be completed. Thus, the process of matching the settings of the remote operation device with the work machine to be remotely operated can be made more efficient.

[0108] According to the remote operation assistance system with this configuration, the first auxiliary processing element 121 can determine whether the actual operating mechanism of the working machine 40 has changed, wherein the working machine 40 is the object of remote operation by the remote operation device 20. Therefore, when the actual operating mechanism has not changed, remote operation of the working machine by the remote operation device can begin without performing processing to match two operating factors, achieving high efficiency. On the other hand, when the actual operating mechanism has changed, the processing to match the factors is reliably performed.

[0109] According to the remote operation assistance system with this configuration, when performing the process of matching two operating factors, the operating characteristics of the actual machine operating mechanism are made consistent with the operating characteristics of the remote operating mechanism of the remote operating device 20. Therefore, even if the operating factors change, the same operability as before the change can be maintained.

[0110] According to the remote operation assistance system with this configuration, when at least one of the operating mode and operating factors changes, processing is performed to make the operating characteristics of the physical machine's operating mechanism correspond to the operating characteristics of the remote operating mechanism of the remote operating device 20. Therefore, even if only the operating mode changes, the same operability as before the change can be maintained.

[0111] (Other embodiments of the present invention)

[0112] At least some of the functional elements of the remote operation assistance server 10 may also be constituted by the remote operation device 20 and / or the working machine 40. For example, the first auxiliary processing element 121 may also be constituted by the remote control device 200 and / or the actual machine control device 400, which serve as the first arithmetic processing device. The second auxiliary processing element 122 may also be constituted by the remote control device 200 and / or the actual machine control device 400, which serve as the second arithmetic processing device. When the functional elements of the remote operation assistance server 10 are mounted on the remote operation device 20, information may be communicated via wired communication through a wired network mounted on the remote operation device 20, instead of the wireless communication described in the above embodiments. Similarly, when the functional elements of the remote operation assistance server 10 are mounted on the working machine 40, information may be communicated via wired communication through a wired network mounted on the working machine 40, instead of the wireless communication described in the above embodiments.

[0113] For example, regarding the first implementation of the second function, Figure 6 / C121, The functional element represented by step (STEP) 122 can also be constituted by the remote control device 200. Figure 6 The functional elements represented by / C120, C121 and STEP122 can also be constituted by the actual machine control device 400. Figure 6 The functional element represented by / C122 can also be composed of a remote control device 200 or a physical control device 400.

[0114] Furthermore, regarding the second implementation of the second function, by Figure 9 The functional element represented by / C142 can also be composed of a remote control device 200 or a physical control device 400.

[0115] Regarding the first implementation of the second function, the identification results of the working machinery 40 on the actual machine operation factors can also be sent to the remote operation auxiliary server 10 before the remote operation of the remote operation device 20. Figure 6 / Step 422). In this case, the identification results of the actual machine operation factors of the operating machinery 40 are stored in the database 102 of the remote operation assistance server 10, therefore, as Figure 10 As shown, a request to obtain actual machine operation factors is sent from the remote operation device 20. Figure 10 / Step 220), upon receiving the request, the remote operation assistance server 10 can identify the actual machine operation factors stored in the database 102 without communicating with the operating machinery 40. Figure 10 / Step 520), thereby enabling the transmission of actual machine operation factors to the remote operation device 20 ( Figure 10 / Step 521).

[0116] Regarding the first implementation of the second function, a message representing remote operation factors can also be output through the remote output interface 220.

[0117] Regarding the second function, in the above-described implementation, the remote operating device 20 requests a correction when the two factors do not correspond, when the operating mode changes, or when the actual machine operating factors change. However, even when these conditions are met, a correction request may be waived to improve work efficiency. For example, if the remote operating device 20 has a history of using remotely operated machinery 40, the settings of the two operating factors, and the operating mode, data associated with the correction process at that time can be used.

[0118] In the remote operation assistance system of the present invention, preferably, the first auxiliary processing element identifies the determination result of whether the remote operation factor and the actual machine operation factor correspond, wherein the remote operation factor is the operation factor corresponding to the actual machine operation mechanism that is set as the object of remote operation by the remote operation mechanism; the actual machine operation factor is the operation factor corresponding to the actual machine operation mechanism, the actual machine operation mechanism constitutes the working mechanism, and the working mechanism is the object of remote operation by the remote operation mechanism.

[0119] According to the remote operation assistance system with this configuration, the first auxiliary processing element can determine whether the actual machine operating mechanism, which is the object of remote operation by the remote operation device, has changed. Therefore, if the actual machine operating mechanism has not changed, remote operation of the machine by the remote operation device can begin without performing processing to match the two operating factors. On the other hand, if the actual machine operating mechanism has changed, processing to match the factors is reliably performed.

[0120] Furthermore, in the remote operation assistance system of the present invention, it is preferable that the second auxiliary processing element performs a process of obtaining a correction value, provided that the determination result identified by the first auxiliary processing element is negative. The correction value is used to make the operating characteristics of the actual machine operating mechanism constituting the work machine correspond to the operating characteristics of the remote operating mechanism, which constitutes the remote operation device for remotely operating the work machine.

[0121] According to the remote operation assistance system with this configuration, after performing the process of matching two operating factors, further processing is performed to make the operating characteristics of the physical operating mechanism consistent with the operating characteristics of the remote operating mechanism. Therefore, even if the operating factors change, the same operability as before the change can be maintained.

[0122] Furthermore, in the remote operation assistance system of the present invention, preferably, the first auxiliary processing element identifies the operation mode when the remote operation mechanism remotely operates the physical machine operation mechanism, wherein the physical machine operation mechanism constitutes the work machine to be remotely operated, and the second auxiliary processing element performs a process to obtain a correction value when at least one of the determination result identified by the first auxiliary processing element is negative or when the first auxiliary processing element identifies that the operation mode has changed is a necessary condition. The correction value is used to make the operation characteristics of the physical machine operation mechanism constituting the work machine correspond to the operation characteristics of the remote operation mechanism, and the remote operation mechanism constitutes the remote operation device for remotely operating the work machine.

[0123] According to the remote operation assistance system with this configuration, when at least one of the operating mode and operating factors changes, processing is performed to make the operating characteristics of the physical operating mechanism correspond to the operating characteristics of the remote operating mechanism. Therefore, even if only the operating mode changes, the same operability as before the change can be maintained.

[0124] Furthermore, in the remote operation assistance system of the present invention, it is preferable that the second auxiliary processing element causes the remote output interface constituting the remote operation device to output a message urging the change of the remote operation factor, and identifies the new remote operation factor that has been changed through the remote input interface constituting the remote operation device.

[0125] According to the remote operation assistance system with this configuration, when the remote operation factors and the actual machine operation factors differ, the processing to correspond the operation mode of the remote operation mechanism constituting the remote operation device with the operation mode of the actual machine operation mechanism constituting the work machinery is executed according to the user's intention at the remote operation device. Therefore, when the two factors correspond, remote operation of the work machinery can begin without waiting for the processing to complete. Thus, by improving the efficiency of this processing, the remote operation efficiency of the work machinery can be increased.

[0126] Furthermore, in the remote operation assistance system of the present invention, preferably, after the second auxiliary processing element identifies the new remote operation factor after the change through the remote input interface, the first auxiliary processing element identifies, based on the communication with the operating machine, the determination result of whether the remote operation factor corresponds to the actual machine operation factor.

[0127] Based on the remote operation assistance server with this configuration, processing executed according to the user's intentions at the remote operation device enables reliable coordination between the two factors, even when remote operation factors and actual machine operation factors do not correspond. Therefore, the efficiency of remote operation of machinery can be improved.

[0128] Explanation of reference numerals in the attached figures

[0129] 10 Remote Operation Assist Server

[0130] 20 Remote operating devices

[0131] 40 Operating machinery

[0132] 41. Actual Input Interface

[0133] 42. Actual output interfaces

[0134] 102 Database

[0135] 121 First Auxiliary Processing Element

[0136] 122 Second Auxiliary Processing Element

[0137] 200 Remote control devices

[0138] 210 Remote Input Interface

[0139] 211 Remote operating mechanism

[0140] 220 Remote Output Interface

[0141] 221 Image output device

[0142] 222 Audio output device

[0143] 224 Remote wireless communication devices

[0144] 400 Real Machine Control Device

[0145] 424 Driver's Cab

[0146] 440 Operating Mechanism

[0147] 445 Bucket (Working Section).

Claims

1. A remote operation assistance system for assisting a remote operation device in remotely operating machinery, characterized in that, include: The first auxiliary processing element identifies the determination result of whether remote operation factors and actual machine operation factors correspond. The remote operation factors define operation settings, which represent the correspondence of the actual machine operation mechanism constituting the working machine. The working machine is a working machine that is intended to be remotely operated by the remote operation mechanism constituting the remote operation device. The actual machine operation factors define operation settings, which represent the correspondence of the actual machine operation mechanism that can work by receiving remote operation. as well as The second auxiliary processing element, with the condition that the determination result identified by the first auxiliary processing element is negative, performs processing to match the actual machine operation factor with the remote operation factor based on communication with at least one of the remote operating device and the operating machinery.

2. The remote operation assistance system according to claim 1, characterized in that, The first auxiliary processing element identifies the determination result of whether the remote operation factor corresponds to the actual operation factor, wherein the remote operation factor is the factor corresponding to the actual operation mechanism of the object to which the remote operation mechanism is set to remote operation; The actual machine operation factor is the factor corresponding to the actual machine operation mechanism, which constitutes the working machinery that is the object of remote operation by the remote operation mechanism.

3. The remote operation assistance system according to claim 1, characterized in that, If the determination result identified by the first auxiliary processing element is negative, the second auxiliary processing element performs a process to obtain a correction value. This correction value is used to make the operating characteristics of the physical operating mechanism constituting the working machine correspond to the operating characteristics of the remote operating mechanism, which constitutes the remote operating device for remotely operating the working machine.

4. The remote operation assistance system according to claim 2, characterized in that, The first auxiliary processing element identifies the operating mode when the remote operating mechanism remotely operates the physical operating mechanism, wherein the physical operating mechanism constitutes the working machinery that is the object of remote operation. The second auxiliary processing element performs a process to obtain a correction value, provided that at least one of the following conditions is met: the determination identified by the first auxiliary processing element is negative, or the operating mode identified by the first auxiliary processing element has changed. This correction value is used to make the operating characteristics of the physical operating mechanism constituting the working machine correspond to the operating characteristics of the remote operating mechanism, which constitutes the remote operating device for remotely operating the working machine.

5. The remote operation assistance system according to claim 1, characterized in that, The second auxiliary processing element causes the remote output interface constituting the remote operation device to output a message urging the change of the remote operation factor, and identifies the new remote operation factor changed through the remote input interface constituting the remote operation device.

6. The remote operation assistance system according to claim 5, characterized in that, After the second auxiliary processing element identifies the new remote operation factor after the change through the remote input interface, the first auxiliary processing element identifies the determination result of whether the remote operation factor corresponds to the actual machine operation factor based on the communication with the operating machinery.

7. A remote operation assistance composite system, characterized in that, It is composed of the remote operation assistance system as described in claim 1, and at least one of the operating machinery and the remote operation device.

8. A remote operation assistance method, the method being used to assist a remote operation device in remotely operating machinery, characterized in that, have: The first auxiliary processing step involves identifying the determination result of whether remote operation factors correspond to actual machine operation factors. The remote operation factors define operation settings, which represent the correspondence between the actual machine operation mechanism constituting the working machinery. This working machinery is intended to be remotely operated by the remote operation mechanism constituting the remote operation device. The actual machine operation factors define operation settings, which represent the correspondence between the actual machine operation mechanism that can perform work by receiving remote operation. The second auxiliary processing step performs processing to match the physical operation factor with the remote operation factor, provided that the determination result identified in the first auxiliary processing step is negative.

9. A remote operation auxiliary server, used to assist a remote operation device in remotely operating machinery, characterized in that, include: The first auxiliary processing element identifies the determination result of whether remote operation factors and actual machine operation factors correspond. The remote operation factors define operation settings, which represent the correspondence of the actual machine operation mechanism constituting the working machine. The working machine is a working machine that is intended to be remotely operated by the remote operation mechanism constituting the remote operation device. The actual machine operation factors define operation settings, which represent the correspondence of the actual machine operation mechanism that can work by receiving remote operation. as well as The second auxiliary processing element, with the condition that the determination result identified by the first auxiliary processing element is negative, performs processing to match the actual machine operation factor with the remote operation factor based on communication with at least one of the remote operating device and the operating machinery.

10. A remote operation device for assisting in the remote operation of machinery, characterized in that, include: The first auxiliary processing element identifies the determination result of whether remote operation factors and actual machine operation factors correspond. The remote operation factors define operation settings, which represent the correspondence of the actual machine operation mechanism constituting the working machine. The working machine is a working machine that is intended to be remotely operated by the remote operation mechanism constituting the remote operation device. The actual machine operation factors define operation settings, which represent the correspondence of the actual machine operation mechanism that can work by receiving remote operation. as well as The second auxiliary processing element performs processing to match the physical operation factor with the remote operation factor, provided that the determination result identified by the first auxiliary processing element is negative.

Images