Teleoperated assistive device and teleoperated assistive method

Abstract

The present invention aims to provide a remote operation assistance device which, when an operator remotely operates a work machine to perform work, moves the work machine to a position suitable for the work scheduled to be performed by the work machine or changes the posture thereof in advance. To achieve the above object, the present invention has a first assistance processing element which acquires schedule information from a database, determines whether or not the work scheduled to be performed by the work machine is ready to start the preparation of the work machine based on the schedule information, and if the determination is YES, executes a first assistance processing for transmitting a preparation signal to the first work machine, the preparation signal being a control signal for starting the preparation of the work scheduled to be performed by the work machine.

Description

Technical Field

[0001] This invention relates to a remote operation assistance device and a remote operation assistance system, which assists operators in remotely operating machinery by operating the remote operation device. Background Technology

[0002] Previously, a remote operating system for a work machine was known, characterized in that it controls the work machine to a standby state when an abnormality occurs (for example, see Patent Document 1).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2015-192163 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] In Patent Document 1, when the remote operating system malfunctions, the control of the working machinery is put into standby mode, thereby preventing the operator from operating the working machinery.

[0008] However, in the remote operating system of machinery, there are situations where a first task is performed with the first machinery as the remotely operated object, and a second task is performed by switching the remotely operated object from the first machinery to the second machinery. To smoothly begin the second task, it is necessary to prepare by changing the position and posture of the second machinery based on the conditions prior to its commencement. However, if the operator performing the first task remotely with the first machinery attempts to prepare for the second task concurrently with the second machinery, not only will the machinery efficiency of the first task decrease, but the preparation for the second task will also be insufficient. Furthermore, starting the second task without preparation will reduce the efficiency of the second task at its commencement.

[0009] Therefore, the object of the present invention is, in view of the above background, to provide a remote operation assistance device that, when selectively switching operations on multiple machines, moves the remotely operated object to a position suitable for the specified operation in advance, or changes the machine to a posture suitable for the specified operation in advance.

[0010] Methods for solving problems

[0011] To achieve the above objectives, the present invention includes a first auxiliary processing element. The first auxiliary processing element performs a first auxiliary process, which involves retrieving information, i.e., plan information, containing the content of the operation to be performed by the first working machine (i.e., the first operation), from a database storing plan information; determining, based on the plan information, whether conditions for preparing the first working machine to begin the first operation (i.e., preparation conditions) are met; and if the determination is yes, sending a control signal, i.e., a preparation signal, to the first working machine to prepare for starting the first operation.

[0012] (Effects)

[0013] According to the present invention, the first auxiliary processing element determines whether the preparation conditions are met based on schedule information. If the determination is yes, a preparation signal is sent to the first working machine. This preparation signal is a control signal for initiating the preparation for the work to be performed by the working machine. Thus, control is initiated to move the working machine, which is the object of remote operation, to a position suitable for the work in advance, or to assume a posture suitable for the work in advance. Therefore, compared to operating a working machine that is not pre-prepared, the operator can begin work immediately, thereby improving work efficiency.

[0014] Brief description of the attached diagram

[0015] Figure 1 This is an explanatory diagram of the configuration of a remote operation assistance system as an embodiment of the remote operation assistance device constituted by the present invention.

[0016] Figure 2 This is an explanatory diagram about the structure of a database.

[0017] Figure 3 This is an explanatory diagram of the configuration of a remote operation device.

[0018] Figure 4 It is an explanatory diagram of the structure of the operating machinery.

[0019] Figure 5 This is an explanatory diagram of the function of the first auxiliary processing element, which is an embodiment of the remote operation assistance device constituted by the present invention.

[0020] Figure 6 This is an explanatory diagram of the function of the second auxiliary processing element, which is an embodiment of the remote operation assistance device constituted by the present invention.

[0021] Figure 7 This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0022] Figure 8A This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0023] Figure 8B This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0024] Figure 9A This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0025] Figure 9B This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0026] Figure 10 This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0027] Figure 11 This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0028] Figure 12 This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention.

[0029] Figure 13 This is an explanatory diagram illustrating the preparation control as another embodiment of the present invention. Detailed Implementation

[0030] (Composition of a remote operation assistance system)

[0031] Figure 1 The remote operation assistance system shown as one embodiment of the remote operation assistance device 100 of the present invention comprises a remote operation assistance server 10 and a remote operation device 20 for remotely operating a work machine 40. The remote operation assistance server 10, the remote operation device 20, and the work machine 40 are configured to communicate with each other via a network. The communication network between the remote operation assistance server 10 and the remote operation device 20 may be the same as or different from the communication network between the remote operation assistance server 10 and the work machine 40.

[0032] Furthermore, the number of operating machines 40 can be one or multiple. In the case of multiple operating machines 40, there can be a first operating machine 40A that serves as the primary object of remote operation by the operator, and a second operating machine 40B that the operator can switch from remote operation of the first operating machine 40A to become the object of remote operation. The first operating machine 40A and the second operating machine 40B can be at the same work site or at different work sites. Moreover, the types of operating machines 40 can be the same or different models.

[0033] For example, an operator can remotely operate the first working machine 40A to primarily perform slope formation operations. Then, when a dump truck, acting as a cooperating working machine 50, periodically approaches the vicinity of the second working machine 40B, the operator can switch the remotely operated working machine from the first working machine 40A to the second working machine 40B each time the dump truck arrives, thereby performing the loading operation of sand onto the dump truck. The first working machine 40A and the second working machine 40B can be at the same work site (work site A) or at different work sites (work site A is the first working machine 40A, and work site B is different from work site A).

[0034] In addition, remote operation refers to the concept of operating the work machinery 40 from a location far away from the work machinery 40 without the operator being on board.

[0035] In addition, the term "operator" refers to the person who operates the remote control device 20 to operate the work machinery 40.

[0036] (Composition of a remote operation auxiliary server)

[0037] The remote operation assistance server 10 includes a remote operation assistance device 100, a database 110, and a server wireless communication device 122. The remote operation assistance device 100 includes a first auxiliary processing element 101 and a second auxiliary processing element 102. Each auxiliary processing element is composed of a computational processing device (a single-core processor, a multi-core processor, or a processor core constituting it), which reads necessary data and software from a storage device such as a memory, and performs computational processing (described later) based on the data and the software.

[0038] Database 110 stores and maintains the attributes of the operating machinery 40, the operation plan of the operating machinery 40, the location information of the operation site where the operating machinery 40 operates, the location information of the object to be operated by the operating machinery 40 in the operation site, i.e., the area where the object is located, i.e., the operation object area 80, the captured image data of the environment of the operation site where the operating machinery 40 operates, and the image data of the cooperating operating machinery 50, etc.

[0039] In addition, database 110 can also store information such as whether the working machine 40 is a hydraulic excavator, manufacturer-related information, product number information, etc., as attributes of the working machine 40.

[0040] In addition, the database 110 can also store information such as the content of keeping the machine 40 in operation, the start time of the machine 40 in operation, the end time of the machine 40 in operation, and the time when the machine 40 is not in use, as an operation plan table for the machine 40.

[0041] In addition, the database 110 can also be interconnected and store the attributes of the operating machinery 40, the operation plan of the operating machinery 40, the location information of the operating site of the operating machinery 40, and the captured image data of the environment of the operating site of the operating machinery 40.

[0042] In addition, when there are multiple operating machines 40 (for example, there is a first operating machine 40A and a second operating machine 40B), the database 110 stores and maintains the attributes of each operating machine 40, the operation plan of each operating machine 40, the location information of the operation site of each operating machine 40, and the captured image data of the environment of the operation site of the operating machine 40.

[0043] For example, such as Figure 2 As shown, database 110 will associate the following information with the operator's name using the first working machine 40A, the location information of the working site using the first working machine 40A, the name of the first operation performed by the first working machine 40A (slope formation), the start time (10:00, 13:00) of the name of the first operation performed by the first working machine 40A, the end time (12:00, 18:00) of the name of the first operation performed by the first working machine 40A, and the time when the first working machine 40A is not used (12:00 to 13:00), and store and maintain it as relevant information of the first working machine 40A.

[0044] In addition, database 110 will associate the following information with the operator's name, the location information of the work site where the second working machine 40B is used, the name of the second operation performed by the second working machine 40B (trenching operation), the start time of the second operation performed by the second working machine 40B (13:00), the end time of the name of the second operation performed by the second working machine 40B (18:00), and the time when the second working machine 40B is not used (10:00 to 13:00), and store and maintain it as relevant information of the second working machine 40B.

[0045] Therefore, the database 110 stores at least the start time of the first task and the start time of the second task. Thus, the first auxiliary processing element 101 can read the start time of the first task from the database 110 and send a command signal to the first working machine 40A to move to a position suitable for the first task or to adopt a posture suitable for the first task before the start time of the first task.

[0046] Similarly, the first auxiliary processing element 101 can read the relevant information of the start time of the second operation from the database 110 and send a command signal to the second working machine 40B, so that the second working machine 40B moves to a position suitable for the second operation or changes to a posture suitable for the second operation before the start time of the second operation.

[0047] (Composition of the remote operation device)

[0048] The remote operation device 20 includes a remote control device 200, a remote input interface 210, and a remote output interface 220. The remote control device 200 is composed of a processing unit (a single-core processor, a multi-core processor, or a processor core thereof), which reads necessary data and software from a storage device such as a memory, and performs calculations based on the data and the software. The remote input interface 210 has a remote operation mechanism 211. The remote output interface 220 includes an image output device 221 and a remote wireless communication device 222.

[0049] The remote control mechanism 211 includes a travel control device, a slewing control device, a boom control device, a stick control device, and a bucket control device. Each control device has a lever for rotational operation. Operating the lever (travel lever) of the travel control device moves the lower travel body 450. The travel lever can also serve as a travel pedal. For example, a travel pedal fixed to the base or lower end of the travel lever can be provided. Operating the lever (slewing lever) of the slewing control device actuates the hydraulic slewing motor constituting the slewing mechanism 430. Operating the lever (boom lever) of the boom control device actuates the boom cylinder 442 of the working machine 40. Operating the lever (stick lever) of the stick control device actuates the stick cylinder 444 of the working machine 40. Operating the lever (bucket stick) of the bucket control device actuates the bucket cylinder 446 of the working machine 40.

[0050] For example, Figure 3 As shown, the levers constituting the remote operation mechanism 211 are arranged around the seat St for the operator. The seat St is similar in shape to a high-backed chair with armrests, but the operator can also sit in a low-backed chair without a headrest or a chair without a backrest. The seat St can be any shape of seating.

[0051] A pair of left and right travel levers 2110 are arranged side-by-side in front of the seat St, corresponding to the left and right tracks. One lever can also function as multiple levers. For example, Figure 3 The left-side operating lever 2111, located in front of the left side frame of the seat St, can function as a boom lever when operated in the forward and backward direction, and as a swivel lever when operated in the left and right direction. Similarly, Figure 3 The right-side operating lever 2112, located in front of the right-side frame of the seat St, can function as a boom lever when operating in the forward and backward direction, and as a bucket lever when operating in the left and right direction. The lever mode can be changed arbitrarily according to the operator's operating instructions.

[0052] For example, Figure 3 As shown, the image output device 221 consists of a central image output device 2210, a left image output device 2211, and a right image output device 2212, each with a roughly rectangular screen, respectively positioned in front of, to the left diagonally in front of, and to the right diagonally in front of the seat St. The shape and size of the screens (image display areas) of the central image output device 2210, the left image output device 2211, and the right image output device 2212 may be the same or different.

[0053] like Figure 3 As shown, the right edge of the left image output device 2211 is adjacent to the left edge of the central image output device 2210, so that the image of the central image output device 2210 and the image of the left image output device 2211 form a tilt angle θ1 (e.g., 120°≤θ1≤150°). Figure 3 As shown, the left edge of the right image output device 2212 is adjacent to the right edge of the central image output device 2210, so that the screen of the central image output device 2210 and the screen of the right image output device 2212 form a tilt angle θ2 (e.g., 120°≤θ2≤150°). The tilt angles θ1 and θ2 can be the same or different.

[0054] The images of the central image output device 2210, the left image output device 2211, and the right image output device 2212 can be parallel or tilted relative to the vertical direction. At least one of the image output devices 2210, 2211, and 2212 can be composed of multiple image output devices. For example, the central image output device 2210 can also be composed of a pair of vertically adjacent image output devices that form a roughly rectangular image. The image output devices 221 (central image output device 2210, left image output device 2211, and right image output device 2212) can also include a speaker (voice output device).

[0055] (Composition of operating machinery)

[0056] Operational machinery 40 refers to an operational vehicle that can operate at a work site. Operational machinery 40 includes at least one type of engineering machinery capable of bulldozing or excavating sand. Examples of engineering machinery include hydraulic excavators with buckets and bulldozers with scrapers.

[0057] Construction machinery 40, for example, is a tracked excavator (construction machinery), such as... Figure 4 As shown, the device includes a tracked lower traveling body 450 and an upper slewing body 460. The upper slewing body 460 is rotatably mounted on the lower traveling body 450 via a slewing mechanism 430. A driver's cab 424 is located on the front left side of the upper slewing body 460. A working mechanism 440 is located on the front center of the upper slewing body 460.

[0058] The machine input interface 410 includes a machine operation mechanism 411, a machine imaging device 412, a machine positioning device 414, and a collaborative positioning device 416. Similar to the remote operation mechanism 211, the machine operation mechanism 411 has multiple control levers around the seat located within the cab 424. The cab 424 is equipped with a drive mechanism or robot that receives signals corresponding to the operating states of the remote control levers and actuates the machine control levers based on these received signals. The machine imaging device 412 is located, for example, inside the cab 424, and captures images of the environment, including at least a portion of the working mechanism 440, through a front window and a pair of side windows separated by a pair of left and right pillars 4240 located at the front of the cab 424 (the left and right sides are distinguished by "L" and "R" in the symbol). The front and side windows may be omitted, either partially or entirely. The machine positioning device 414 is a device for detecting the position of the working machinery 40, and may be, for example, composed of a GNSS receiver (GNNS: Global Navigation Satellite System).

[0059] The collaborative positioning device 416 has the function of specifying the position of the collaborative work machinery 50 (described later). The collaborative positioning device 416 can be composed of, for example, a camera or sensor (range sensor) other than a camera or sensor capable of acquiring information related to the working environment. The collaborative positioning device 416 specifies the position of the collaborative work machinery 50 relative to the work machinery 40 by detecting the shape and size of the collaborative work machinery 50 using these cameras or sensors.

[0060] The machine output interface 420 is equipped with a machine wireless communication device 422. The position-related information of the operating machinery 40 detected by the machine positioning device 414 is transmitted to the remote operation auxiliary server 10 through the machine wireless communication device 422.

[0061] The working attachments of the working mechanism 440 include a boom 441 that can be raised and lowered and mounted on the upper slewing body 460, a stick 443 rotatably connected to the front end of the boom 441, and a bucket 445 rotatably connected to the front end of the stick 443. The working mechanism 440 is equipped with a boom cylinder 442, a stick cylinder 444, and a bucket cylinder 446, all of which are telescopic hydraulic cylinders.

[0062] Boom cylinder 442 is located between boom 441 and upper slewing body 460, and extends and retracts by receiving a supply of working oil, causing boom 441 to rotate in the lifting direction. Stick cylinder 444 is located between stick 443 and boom 441, and extends and retracts by receiving a supply of working oil, causing stick 443 to rotate about a horizontal axis relative to boom 441. Bucket cylinder 446 is located between bucket 445 and stick 443, and extends and retracts by receiving a supply of working oil, causing bucket 445 to rotate about a horizontal axis relative to stick 443.

[0063] (Composition of collaborative work machinery)

[0064] The cooperative work machine 50 is a work machine that works in conjunction with the work machine 40. For example, the cooperative work machine 50 is a dump truck equipped with a loading platform. The cooperative work machine 50 has a cooperative control device 500, a cooperative input interface 510, a cooperative output interface 520, and a cooperative working mechanism 540 (e.g., a loading platform).

[0065] In addition, the cooperative work machine 50 is operated by personnel other than the operator of the work machine 40 who remotely operates the work machine via the remote operation device 20.

[0066] In addition, the cooperative operation machinery 50 also includes a loading platform, a vehicle body supporting the loading platform, and a driving device that supports the vehicle body and drives the vehicle.

[0067] The loading platform can be used to load sand. The cooperative machinery 50, for example, is a rear-discharge type, with a gate at the rear of the loading platform that opens when the front of the loading platform is raised to tilt it backward.

[0068] The driving device has wheels, and the cooperative working machine 50 can move by rotating the wheels.

[0069] (Function)

[0070] use Figure 5 and Figure 6 The flowchart shown illustrates the function of the constructed remote operation assistance system. In this flowchart, for simplicity, boxes such as "C●" are used to represent data transmission and / or reception, and to represent conditional branches in which processing in the branch direction is performed based on the transmission and / or reception of the data.

[0071] Reference Figure 5 The flowchart shown illustrates the first auxiliary process of this embodiment. The first auxiliary process is a process performed to pre-move the working machine 40 to a suitable working position or to pre-position the working machine 40 into a suitable working posture.

[0072] In the remote operation device 20, it is determined whether a first specified operation performed by an operator through the remote input interface 210 exists. Figure 5 / STEP211). The “first designated operation” is, for example, a click, press, pinch, slide, or other operation on the remote input interface 210, which is used on the image output device 221 to designate the work machine 40 that the operator intends to remotely operate.

[0073] Other examples of the “first designated operation” include operations such as clicking, pressing, pinching, and sliding on the remote input interface 210, which are used to designate the task that the operator intends to perform on the image output device 221. If the determination result is negative ( Figure 5 If / STEP211..NO), then the process ends. On the other hand, if the result of this determination is ( Figure 5 / STEP211..YES), then a preparation request is sent to the remote operation assistance server 10 via the remote wireless communication device 222. Figure 5 / STEP212).

[0074] When the remote operation assistance server 10 receives the preparation request ( Figure 5 / STEP111..YES), the first auxiliary processing element 101 reads the schedule information from the database 110. Figure 5 / STEP112). On the other hand, if the result of this determination is negative ( Figure 5 If / STEP111..NO), then the decision process of STEP111 will be repeated.

[0075] The first auxiliary processing element 101 determines, based on the obtained schedule information, whether the conditions for preparing the machine 40 to perform the scheduled operation are met, i.e., the preparation conditions (STEP113).

[0076] In STEP 113, for example, the first auxiliary processing element 101 acquires information related to the current time of the first working machine 40A and determines whether the current time is a time point that is only a predetermined time (e.g., 10 minutes) earlier than the start time of the first operation (e.g., 10 o'clock).

[0077] like Figure 2As shown, relative to the start time of the first operation (e.g., a slope formation operation performed by the first operating machine 40A in the morning, e.g., 10:00 AM), when the current time is a time t1 that is only a predetermined time earlier than the start time of the first operation (e.g., 9:50 AM), the first auxiliary processing element 101 determines that the preparation conditions are met. On the other hand, when the current time is a time point earlier than the time t1 that is only a predetermined time earlier than the start time of the first operation (e.g., a slope formation operation performed in the morning, e.g., 10:00 AM, e.g., 9:40 AM), the determination result is negative. Figure 5 / STEP113..NO), the first auxiliary processing element 101 repeats the determination process of STEP113.

[0078] As another example of STEP 113, consider the example where the first auxiliary processing element 101 acquires information about the current time of the second working machine 40B and determines whether the current time is a time point that is only a predetermined time (e.g., 10 minutes) earlier than the start time of the second operation (e.g., 13:00). When the current time is... Figure 2 The first auxiliary processing element 101 determines that the preparation conditions are met when the time point t3 (e.g., 12:50) is only a predetermined time (e.g., 10 minutes) earlier than the start time of the second operation (e.g., a loading operation that the operator interrupts the first operation to start at any time). On the other hand, when the current time is an earlier time point (e.g., 12:40) than the start time of the second operation (e.g., a loading operation that the operator interrupts the first operation to start at any time) (e.g., 13:00), the determination result is negative. Figure 5 / STEP113..NO), the first auxiliary processing element 101 repeats the determination process of STEP113.

[0079] When STEP113 determines that ( Figure 5 / STEP113..YES), the first auxiliary processing element 101 sends a preparation command to the operating machine 40 (STEP114). On the other hand, when the determination result is negative ( Figure 5 / STEP113..NO), the first auxiliary processing element 101 repeats the determination process.

[0080] The processes described in STEP112 to STEP114 are the concept of the first auxiliary processing element in this embodiment.

[0081] When the operating machinery 40 receives the preparation signal ( Figure 5 / STEP411..YES), the working machine 40 executes preparation control, which is a control used to begin the work preparation that the working machine 40 is scheduled to perform. Figure 5 / STEP412). On the other hand, if the result of this determination is negative ( Figure 5 / STEP411..NO), repeat the decision process of STEP411.

[0082] In STEP 412, control is initiated to pre-position the machine 40 in a suitable work location or pre-adjust it to a suitable work posture. Therefore, compared to operating a machine that is not pre-prepared, the operator can begin work immediately, thus improving work efficiency.

[0083] For example, in STEP 412, when the first working machine 40A, which is waiting near the work target area 80 (e.g., slope 80A), is scheduled to perform slope formation work in the work target area 80 (e.g., slope 80A), the first working machine 40A begins to move towards the area where the work target area 80 (e.g., slope 80A) ((latitude w, longitude z)) is located in the work site ((latitude XX, longitude YY)).

[0084] As an example of preparing for control, an example is shown where the first working machine 40A moves toward the work target area 80 (e.g., ramp 80A), but the examples of preparing for control are not limited to this example. As another example of preparing for control, an example is given where the first working machine 40A, after moving to the work target area 80 (e.g., ramp 80A), rotates its upper rotating body 460A to pre-orient the working mechanism 440A toward the direction of the work target area 80 (e.g., ramp 80A).

[0085] As an example of preparatory control, an example of preparatory control performed by the first working machine 40A is shown, but the examples of preparatory control are not limited to this example. For example, as another example of preparatory control, an example is given where a second working machine 40B, which is waiting near trench 80B and is scheduled to perform excavation work at trench 80B, begins to move towards the area of ​​the hole to be excavated in the work site ((latitude AA, longitude BB)), i.e., trench 80B ((latitude a, longitude b)).

[0086] As an example of preparatory control, an example of the second working machine 40B moving toward the trench 80B is shown, but the examples of preparatory control are not limited to this example. As another example of preparatory control, an example is given where the second working machine 40B rotates the upper rotating body 460B relative to the lower traveling body 450 in advance so that the actuating mechanism 440B is oriented toward the direction of the trench 80B.

[0087] As an example of preparatory control, it is shown that the second working machine 40B rotates the upper rotating body 460B relative to the lower traveling body 450 to pre-orient the actuating mechanism 440B toward the direction of the trench 80B, but the examples of preparatory control are not limited to this example. As another example of preparatory control, a control example is given in which the second working machine 40B digs the trench 80B in the direction of the second working machine 40B to extend the length of the trench 80B.

[0088] In this example, such as Figure 8A As shown, with the trench 80B already excavated, the lower traveling body 450 of the second working machine 40B is operated to move the second working machine 40B to a position near the trench 80B in preparation for excavating the trench 80B in the direction of the second working machine 40B. This allows the working mechanism 440B to be positioned beforehand at the base end of the trench 80B. (It should be noted that the end of the trench 80B closest to the second working machine 40B is called the base end side.)

[0089] Therefore, after the remote operation object is switched from the first operating machine 40A to the second operating machine 40B, the operator can immediately use the working mechanism 440B (bucket 445B) to dig the base side of the trench 80B.

[0090] Alternatively, in this case, control can be performed to rotate the upper rotating body 460B relative to the lower traveling body 450B, with the operating mechanism 440B precisely oriented in the direction of the slot 80B.

[0091] When the preparation for the operation is completed, the operating machinery 40 sends a preparation completion signal to the remote operation assistance server 10 via the machine wireless communication device 422. This preparation completion signal includes a signal to notify the operator that the preparation for the operation is complete. Figure 5 / STEP413).

[0092] In the remote operation auxiliary server 10, when a ready-to-end signal is received ( Figure 5 / STEP115..YES), the first auxiliary processing element 101 sends a readiness-to-end signal to the remote operating device 20 ( Figure 5 / STEP116). On the other hand, if the result of this determination is negative ( Figure 5 If / STEP115..NO), then the first auxiliary processing element 101 repeats the determination process of STEP115.

[0093] In the remote operation device 20, when a ready-to-end signal is received ( Figure 5 / STEP213..YES), the remote control device 200 performs notification control ( Figure 5 / STEP214), this notification control is used to notify the operator that preparation for the operation to be performed by the machine 40 has been completed. For example, as a notification control, the remote operating device 200 performs control processing to display the text message "Preparation for the machine 40 has been completed" on the image output device 221. Of course, examples of text messages displayed on the image output device 221 are not limited to this example.

[0094] As an example of notification control, an example of displaying a text message on the image output device 221 is shown, but examples of notification control are not limited to this example. For example, as another example of notification control, an example is given of the speaker provided by the image output device 221 playing the voice message "The preparation of the working machine 40 has been completed". Of course, examples of playing voice messages by the speaker provided by the image output device 221 are not limited to this example.

[0095] On the other hand, if the result of the judgment is no ( Figure 5 If / STEP213..NO), then the decision process of STEP213 will be repeated.

[0096] use Figure 6 The flowchart shown illustrates the second auxiliary process of this embodiment. The second auxiliary process is used to assist the operator in actually remotely operating the work machinery 40 using the remote operating mechanism 211.

[0097] In the remote operation device 20, the remote control device 200 determines whether a second specified operation has been performed by an operator through the remote input interface 210. Figure 6 / STEP221). The "second designated operation" is, for example, a click, press, pinch, or slide operation performed on the remote input interface 210, used to designate the work machinery 40 that the operator intends to remotely operate on the image output device 221. If the determination result is no ( Figure 6 If / STEP221..NO), the remote operating device 200 repeats the processing after determining whether the specified operation exists. On the other hand, if the determination result is yes ( Figure 6 / STEP221..YES), then the remote control device 200 sends an environment confirmation request to the remote operation assistance server 10 through the remote wireless communication device 222. Figure 6 / STEP222).

[0098] In the remote operation assistance server 10, when an environmental confirmation request is received, the second auxiliary processing element 102 sends the environmental confirmation request to the corresponding operating machinery 40. Figure 6 / C10).

[0099] In the operating machinery 40, when an environmental confirmation request is received via the actual machine wireless communication device 422 ( Figure 6 / C40), the actual control device 400 acquires the captured image through the actual shooting device 412 ( Figure 6 / STEP421). The actual control device 400 transmits the captured image data representing the captured image to the remote operation auxiliary server 10 via the actual wireless communication device 422. Figure 6 / STEP422).

[0100] In the remote operation assistance server 10, when the second auxiliary processing element 102 receives the captured image data ( Figure 6 / C11), the second auxiliary processing element 102 sends the captured image data to the remote control device 20 ( Figure 6 / STEP121). The second auxiliary processing element 102 may also send environmental image data, representing a simulated environmental image generated based on the captured image, to the remote operation device 20 instead of the captured image data. At this time, the second auxiliary processing element 102 may also send a command to the remote operation device 20 to display the captured image data in a split manner on the central image output device 2210, the left image output device 2211, and the right image output device 2212.

[0101] In the remote operation device 20, when the remote control device 200 receives the captured image data through the remote wireless communication device 222 ( Figure 6 / C21), the remote control device 200 controls the captured image data to be displayed in a split manner on the central image output device 2210, the left image output device 2211, and the right image output device 2212. Figure 6 / STEP223).

[0102] In the remote operation device 20, the remote control device 200 identifies the operating status of the remote operation mechanism 211. Figure 6 / STEP224), and the remote control device 200 sends the remote operation command corresponding to the operation status to the remote operation auxiliary server 10 via the remote wireless communication device 222. Figure 6 / STEP225).

[0103] In the remote operation auxiliary server 10, when the second auxiliary processing element 102 receives the remote operation command, the second auxiliary processing element 102 sends the remote operation command to the operating machine 40. Figure 6 / C12).

[0104] In the operating machinery 40, when the actual machine control device 400 receives an operation command through the actual machine wireless communication device 422 ( Figure 6 / C42), controls the actions of the working mechanism 440, etc. Figure 6 / STEP423). For example, performing an operation in which the bucket 445 digs up the soil in front of the working machine 40, and after rotating the upper rotating body 460, dumps the soil from the bucket 445.

[0105] (Other embodiments of the present invention)

[0106] In the above embodiments, an example was described in which the first auxiliary processing element 101 sends a preparation signal to the working machine 40 based on schedule information (e.g., information related to the start time of the operation), but the method is not limited to this example. For example, an example is given in which the first auxiliary processing element 101 obtains distance-related information between the second working machine 40B and the cooperating working machine 50, and sends a preparation signal to the second working machine 40B.

[0107] use Figure 2 and Figure 7 This section describes other implementation methods. It should be noted that for processing flows identical to those described in the implementation method, the same STEP numbers are assigned, and descriptions are appropriately omitted.

[0108] like Figure 2 As shown, database 110 stores and maintains the first operation (slope formation operation) performed by the first operating machine 40A after 13 o'clock and the second operation (e.g., trenching operation performed by the operator after interrupting the first operation) performed by the second operating machine 40B, as an operation plan table for the operating machine 40.

[0109] In the remote operation device 20, it is determined whether the first task is being performed. Figure 7 / STEP231). As an example of performing the first task, an example is given where the current time is the designated time (13:00-18:00) and the remote control device 20 is establishing communication to remotely control the first task machine 40A. If the determination result is negative ( Figure 7 If / STEP231..NO), then the process ends. On the other hand, if the result of this determination is ( Figure 7 / STEP231..YES), then a preparation request is sent to the remote operation assistance server 10 via the remote wireless communication device 222. Figure 7 / STEP232).

[0110] In the remote operation auxiliary server 10, when a preparation request is received ( Figure 7 / STEP121..YES), the first auxiliary processing element 101 requests information about the distance between the second working machine 40B and the cooperating working machine 50 from the second working machine 40B. Figure 7 / STEP117). On the other hand, if the result of this determination is negative ( Figure 7 If / STEP121..NO), then the decision process of STEP121 will be repeated.

[0111] In the second working machine 40B, when a request for distance information is received ( Figure 7 / STEP414..YES), the machine control device 400 acquires relevant information (hereinafter sometimes referred to as distance information, etc.) regarding the distance between the second working machine 40B and the cooperating working machine 50. On the other hand, if the determination result is negative ( Figure 7 If / STEP414..NO), then repeat the decision process.

[0112] Here, regarding distance information, the collaborative positioning device 416 (e.g., a camera or sensor such as the actual machine camera device 412) measures the distance between the second working machine 40B and the collaborative working machine 50, thereby the actual machine control device 400 acquires the distance information. Figure 7 / STEP415).

[0113] Additionally, an example of using a collaborative positioning device 416 (a camera or sensor such as a live camera device 412) to acquire distance information is shown, but the examples of acquiring distance information are not limited to this one. For example, when a camera or sensor is installed at the work site where the second working machine 40B is located, the first auxiliary processing element 101 can use the camera and the sensor, or one of them, to specify the positions of the working machine 40 and the collaborative working machine 50, calculate the distance between the coordinates of the second working machine 40B and the coordinates of the collaborative working machine 50, and thus acquire distance information.

[0114] In addition, when the collaborative work machine 50 is equipped with a positioning device (e.g., a GNSS receiver) to detect the position of the collaborative work machine 50, the first auxiliary processing element 101 calculates the distance between the coordinates of the second work machine 40B and the coordinates of the collaborative work machine 50 based on the position information of the second work machine 40B located by the actual positioning device 414 and the position information of the collaborative work machine 50 located by the positioning device of the collaborative work machine 50, thereby obtaining distance information.

[0115] Regarding the location information of the second working machine 40B, the machine positioning device 414 (e.g., a GNSS receiver) acquires relevant information about the coordinate position (e.g., global coordinate position in GNSS) of the second working machine 40B. Then, the machine control device 400 transmits this coordinate position information to the remote operation assistance server 10 via the machine wireless communication device 422. Thus, the remote operation assistance server 10 acquires the location information of the second working machine 40B.

[0116] Additionally, an example of using global coordinates in GNSS to obtain coordinate position information of the second working machine 40B is shown, but the examples of the remote operation assistance server 10 obtaining coordinate position information of the second working machine 40B are not limited to this example. For example, as another example of the remote operation assistance server 10 obtaining the position information of the second working machine 40B, an example is given where the database 110 pre-stores local coordinates related to the work site of the second working machine 40B, and the machine positioning device 414 locates the second working machine 40B within those local coordinates.

[0117] On the other hand, regarding the location information of the collaborative work machinery 50, the collaborative positioning device 416 acquires relevant information about the coordinate position (e.g., global coordinate position in GNSS) of the collaborative work machinery 50. Then, the collaborative control device sends this coordinate position-related information to the remote operation assistance server 10 via the collaborative wireless communication device. Thus, the remote operation assistance server 10 acquires the location information of the collaborative work machinery 50.

[0118] Additionally, an example of using global coordinates in GNSS to obtain coordinate location information of the collaborative work machinery 50 is shown, but examples of the remote operation assistance server 10 obtaining coordinate location information of the collaborative work machinery 50 are not limited to this example. For example, as another example of the remote operation assistance server 10 obtaining location information of the collaborative work machinery 50, an example is given where the database 110 pre-stores local coordinates related to the work site of the collaborative work machinery 50, and the positioning device of the collaborative work machinery 50 is located in which coordinates of the collaborative work machinery 50 within those local coordinates.

[0119] As described above, the first auxiliary processing element 101 can also compare the coordinates of the second working machine 40B with the coordinates of the cooperative working machine 50 obtained above, and calculate the distance between the two coordinates to obtain distance information.

[0120] In the operating machinery 40, when distance information is obtained, the machine control device 400 sends the distance information to the remote operation auxiliary server 10 via the machine wireless communication device 422. Figure 7 / STEP416).

[0121] In the remote operation auxiliary server 10, when the first auxiliary processing element 101 acquires distance information ( Figure 7 / STEP118..YES), the first auxiliary processing element 101 determines whether the cooperating machine 50 has entered the reference area ( Figure 7 / STEP119). On the other hand, if the result of STEP118 is negative ( Figure 7If / STEP118..NO), then repeat the process before that decision.

[0122] In STEP 119, the first auxiliary processing element 101 reads the schedule information from the database 110 and determines, during the period from the start time to the end time of the first operation of the first working machine 40A, whether the working machine that works in coordination with the second working machine 40B, namely the cooperating working machine 50, has moved from outside the reference area AR determined based on the location of the second working machine 40B to inside it.

[0123] For example, Figure 8A As shown, when the second working machine 40B is performing trench excavation, if the cooperating working machine 50 is located outside the reference area AR ( Figure 7 If / STEP119..NO), then the first auxiliary processing element 101 repeats the determination process.

[0124] In addition, in this case, such as Figure 8B As shown, the first auxiliary processing element 101 can also send instructions to the remote operation device 20 to display the environmental images of the work site captured by the real-time imaging device 412 of the second working machine 40B in a segmented manner on the central image output device 2210, the left image output device 2211, and the right image output device 2212. In this case, the second working machine 40B does not perform preparatory control (rotation), so a portion of the work object area 80 (e.g., sand dune 80C) in the second working machine 40B will not be captured.

[0125] On the other hand, such as Figure 9A As shown, when the second working machine 40B is performing loading operations, if the cooperating working machine 50 moves from outside the reference area AR into it ( Figure 7 / STEP119..YES), then the first auxiliary processing element 101 sends a preparation signal to the second working machine 40B (e.g., a command signal for driving the lower traveling body 450 of the excavation work object area 80 (e.g., trench 80B) to make the second working machine 40B reverse and rotating the upper rotating body 460 to make the working mechanism 440 turn towards the trench 80B in the predetermined excavation direction). Figure 7 / STEP124).

[0126] In the second working machine 40B, when the actual machine control device 400 receives the preparation signal ( Figure 7 / STEP431..YES), as shown Figure 9B As shown, preparation for control begins in the second working machine 40B. Figure 7 / STEP432), the second working machine 40B drives the lower traveling body 450 to make the second working machine 40B move backward, or rotates the upper rotating body 460 to make the working mechanism 440 turn towards the predetermined excavation direction of the trench 80B.

[0127] In addition, in this case, such as Figure 9B As shown, the first auxiliary processing element 101 can also send instructions to the remote operation device 20 to display the environmental image of the work site captured by the real-time imaging device 412 of the second working machine 40B on the central image output device 2210, the left image output device 2211, and the right image output device 2212. In this case, the second working machine 40B performs preparation control (rotation), and thus the trench 80B is captured and displayed on the central image output device 2210. In addition, the cooperating working machine 50 is displayed on the left image output device 2211.

[0128] When the preparation for the operation is completed, the operating machinery 40 sends a preparation completion signal to the remote operation assistance server 10 via the machine wireless communication device 422. This preparation completion signal includes a signal to notify the operator that the preparation for the operation is complete. Figure 7 / STEP433).

[0129] In the remote operation auxiliary server 10, when a ready-to-end signal is received ( Figure 7 / STEP125..YES), the first auxiliary processing element 101 sends a preparation end signal to the remote control device 20. On the other hand, if the determination result is negative ( Figure 7 If / STEP125..NO), then the STEP125 decision process will be repeated.

[0130] In the remote operation device 20, when a ready-to-end signal is received ( Figure 7 / STEP233..YES), the remote control device 200 prepares for the end of the display control ( Figure 7 / STEP234), this control is used to notify the operator that preparation for work on the operating machine 40 has been completed. On the other hand, if the determination result is negative ( Figure 7 If / STEP233..NO), then the STEP233 decision process will be repeated.

[0131] The remote operation assistance device 100 and remote operation assistance system of the present invention can also be applied to this situation. In the remote operation device 20, notification control is performed ( Figure 5 / STEP214), the operator can interrupt the first operation performed using the first working machine 40A and begin the second operation performed using the second working machine 40B. For example... Figure 9B As shown, when executing notification control ( Figure 5 When / STEP214), the cooperating machine 50 is displayed on the left image output device 2211, so the operator can accurately confirm when to start using the second machine 40B for the second operation.

[0132] Furthermore, after the second task performed using the second working machine 40B (digging trench 80B using the working mechanism 440 and loading the excavated sand onto the cooperating working machine 50) is completed, when the operator returns to the first task performed using the first working machine 40A, the remote operating device 20 again determines whether the first task is being performed. Figure 7 / STEP231). In this way, database 110 can store and maintain information on the first operation (ramp forming operation) performed using the first working machine 40A and the second operation (e.g., loading operation performed by the operator after interrupting the first operation) as an operation schedule of the working machine 40 after 13:00.

[0133] Furthermore, before the collaborative work machine 50 moves from outside the reference area AR to within it, the first auxiliary processing element 101 does not send a preparation signal to the second work machine 40B. Therefore, if the first operation performed by the first work machine 40A is interrupted at any time, and an operation other than the second operation performed by the second work machine 40B is performed, the operator can operate the second work machine 40B without any sense of incongruity because the second work machine 40B maintains the position and posture when the second operation was interrupted. After the collaborative work machine 50 moves from outside the reference area AR to within it, the first auxiliary processing element 101 sends a preparation signal to the second work machine 40B, causing the second work machine 40B to assume a ready posture, so that the operator can smoothly begin the second operation performed by the second work machine 40B.

[0134] In the above embodiments, an example of preparation control performed by the first auxiliary processing element is shown, which involves determining whether the cooperating machine 50 has moved from outside to inside the reference area AR. If the determination is yes, a preparation signal is sent to the second machine 40B. However, the examples of preparation control performed by the first auxiliary processing element are not limited to this example. Another example of preparation control performed by the first auxiliary processing element includes a reference area AR comprising a first reference area AR1 and a second reference area AR2 located outside the first reference area AR1. The first auxiliary processing element determines whether the cooperating machine 50 has entered the first reference area AR1 from the second reference area AR2. If the determination is yes, control is performed to send a signal different from the preparation signal, i.e., a target preparation signal.

[0135] For example, such as Figure 10 As shown, when the second working machine 40B is digging trench 80B and loading the excavated sand, when the cooperating working machine 50 is located within the area of ​​the second reference area AR2, the second working machine 40B drives the lower traveling body 450 to move the second working machine 40B backward, or rotates the upper rotating body 460 to turn the working mechanism 440 toward the direction of the work object area 80 (e.g., trench 80B, sand dune 80C).

[0136] Then, as Figure 11 As shown, when the cooperative working machine 50 enters the first reference area AR1 from the second reference area AR2, the first auxiliary processing element sends a target preparation signal to the second working machine 40B.

[0137] For example, the target preparation signal is a control signal used to extend the boom 441, stick 443 and bucket 445, and to bring the bucket 445 into contact with the ground at the predetermined excavation position in the trench 80B.

[0138] Therefore, as Figure 11 As shown, by using the second working machine 40B to turn the working mechanism 440 to the predetermined excavation position of the trench 80B and put it into standby mode, the operator can quickly begin loading operations. Therefore, compared to operating the second working machine 40B without prior preparation, the operator can start the work immediately, thereby improving work efficiency.

[0139] The remote operation assistance device 100 and remote operation assistance system of the present invention can also be applied to this situation.

[0140] In the described embodiment, an example is shown in which the reference area AR is defined as a circle with the location of the second working machine 40B as a reference, but the method of determining the reference area is not limited to this example.

[0141] For example, Figure 12 As shown, as another example of the method for determining the reference area AR, an example is given where the reference area AR is set in a shape that extends eccentrically from the location of the second working machine 40B toward the entrance and exit of the work site, the entrance and exit of which is used for the entry and exit of the working machine 50 to work in conjunction with the second working machine 40B.

[0142] Based on the above configuration, the reference area AR has a shape that extends eccentrically toward the entrance and exit of the work site. Therefore, in the work site, the entrance and exit, as the part most likely to be entered by the cooperating machine 50, will definitely be included in the reference area, and the reference area AR is set to narrow in other parts, thereby allowing for the setting of a reasonable reference area AR. This reduces the probability that a work machine 40 unrelated to preparation control is mistakenly identified as the cooperating machine 50, thus reducing the probability of initiating preparation control of the second work machine 40B due to misidentification, and improving work efficiency.

[0143] In the described embodiment, an example is shown where the first auxiliary processing element 101 sends a preparation signal to the working machine 40 when the collaborative working machine 50 moves from outside the reference area AR into it, but the example of the first auxiliary processing element 101 sending a preparation signal to the working machine 40 is not limited to this example.

[0144] For example, other examples of the first auxiliary processing element 101 sending a preparation signal to the working machine 40, such as Figure 13 As shown, an example is given whereby the first auxiliary processing element 101 sends a preparation signal to the second working machine 40B when the cooperating machine 50 moves from inside the reference area AR to outside of it.

[0145] According to the above configuration, when the cooperating machine 50 moves from within the reference area AR, the first auxiliary processing element 101 sends a preparation signal to the second machine 40B. Thus, after the cooperating machine 50 moves to transport the sand loaded on the loading platform, the second machine 40B can rotate its upper rotating body 460, causing its working mechanism 440 to face the trench 80B, ready to resume digging operations. Therefore, compared to operating a second machine 40B that is not pre-prepared, the operator can begin work immediately, thereby improving work efficiency.

[0146] The remote operation assistance device 100 and remote operation assistance system of the present invention can also be applied to this situation.

[0147] In the described embodiment, an example is shown in which the first auxiliary processing element 101 sends a preparation signal to the second working machine 40B based on the distance between the second working machine 40B and the cooperating working machine 50, but the example of the first auxiliary processing element 101 sending a preparation signal is not limited to this example.

[0148] For example, consider the example where the first auxiliary processing element 101 sends a preparation signal to the first working machine 40A based on the distance between the first working machine 40A and the cooperating working machine 50.

[0149] The remote operation assistance device 100 and remote operation assistance system of the present invention can also be applied to this situation.

[0150] In the described embodiment, as a second operation performed using the second working machine 40B, an operation of digging a trench 80B and loading the excavated sand into the cooperating working machine 50 is shown, but examples of the second operation are not limited to this example.

[0151] For example, as a second operation, an example is given of loading and unloading a transported object loaded on the loading platform of the cooperative operation machinery 50. The transported object is, for example, sand, which is loaded and unloaded by scooping it up using a bucket.

[0152] Alternatively, the transported object can also be a structure (such as reinforcing bars). For example, reinforcing bars can be loaded and unloaded by using a grab bucket, which is an example of a working mechanism 440.

[0153] The remote operation assistance device 100 and remote operation assistance system of the present invention can also be applied to this situation.

[0154] In the described embodiment, an example of a structure is shown as an example of a transported object loaded on the loading platform of the cooperative operation machinery 50, but the transported object loaded on the loading platform of the cooperative operation machinery 50 is not limited to this example.

[0155] For example, consider timber loaded on the loading platform of the cooperative operation machinery 50. In this case, the timber loaded on the loading platform of the cooperative operation machinery 50 is loaded and unloaded by using a grab bucket, which is an example of a working mechanism 440, to grasp and transport it simultaneously.

[0156] The remote operation assistance device 100 and remote operation assistance system of the present invention can also be applied to this situation.

[0157] Furthermore, in this invention, it is preferable that the first auxiliary processing element determines that the preparation conditions are met when the current time is before the start time of the operation to be performed by the machine.

[0158] (Assignment Results)

[0159] Based on the above configuration, the first auxiliary processing element determines whether the preparation conditions are met based on the schedule information including the time. Therefore, when the operator begins work, the machine has already moved to a suitable position or assumed a suitable posture for work. Thus, compared to operating machine that is not pre-prepared, the operator can begin work immediately, thereby improving work efficiency.

[0160] Furthermore, in this invention, it is preferred that the first auxiliary processing element determines that the preparation condition is met when the time from the start time to the end time of the first job stored in the database includes the start time of the second job stored in the database.

[0161] (Effects)

[0162] Based on the above configuration, the database stores a schedule containing at least the content of the first task performed by a remotely operated first machine, the start time of the first task, the content of the second task performed by a remotely operated second machine, and the start time of the second task. Based on this schedule information, when the scheduled time slot for the second task stored in the database is included within the scheduled time slot for the first task stored in the database, the first auxiliary processing element determines that the preparation conditions are met. Therefore, even if a situation arises where the second task needs to be performed while the operator is using the first machine to perform the first task, the second machine will be controlled to move to a position suitable for the second task before the start time of the second task, or to assume a posture suitable for the second task. Therefore, compared to operating a second machine that is not pre-prepared, the operator can immediately begin the second task, thereby improving the efficiency of the second task.

[0163] In addition, in this invention, it is preferred that the first auxiliary processing element sends a preparation signal to the first working machine, the preparation signal being a signal that includes a signal for driving the lower traveling body of the first working machine.

[0164] (Effects)

[0165] Based on the above configuration, when the preparation conditions are met, the first auxiliary processing element sends a preparation signal for driving the lower traveling body to the first working machine. Therefore, the working machine can pre-move to a suitable working position. Thus, the working machine, as the object of remote operation, is pre-moved to a suitable working position. Therefore, compared to operating a working machine that has not been pre-moved, the operator can begin work immediately, thereby improving work efficiency.

[0166] In addition, in this invention, it is preferred to send a preparation signal to the working machine, the preparation signal being a signal that includes a signal for rotating the upper rotating body of the working machine.

[0167] (Effects)

[0168] Based on the above configuration, when the preparation conditions are met, the first auxiliary processing element sends a preparation signal to the working machine to rotate the upper rotating body. Therefore, the working machine can pre-position itself in a suitable working direction and wait. Thus, the working machine, as the object of remote operation, is pre-positioned in a suitable working direction and ready to operate. Therefore, compared to operating a working machine that has not been pre-positioned, the operator can begin work immediately, thereby improving work efficiency.

[0169] Furthermore, in this invention, it is preferred that the first auxiliary processing element sends a preparation signal to the first working machine, the preparation signal being a signal that includes a signal for activating the working mechanism of the first working machine.

[0170] (Effects)

[0171] According to the above configuration, when the preparation conditions are met, the first auxiliary processing element sends a preparation signal to the working machine to enable the working mechanism to operate. Therefore, the working machine can activate the working mechanism to prepare for operation in advance. Thus, the working machine, as the object of remote operation, is prepared for operation in advance. For example, if a safer position is desired, a preparation signal can be sent to prepare the bucket in a position in contact with the ground. Alternatively, if a faster start to work is desired, a preparation signal can be sent to prepare the machine in a position with the bucket raised. Therefore, compared to operating a working machine that has not been prepared for operation in advance, the operator can begin work immediately, thereby improving work efficiency.

[0172] Furthermore, in this invention, it is preferred that when the operator interrupts the remote operation of the first working machine and the machine that works in coordination with the second working machine, which is the object of the remote operation, moves from outside the reference area determined based on the location of the second working machine to the inside of that area, the first auxiliary processing element determines that the preparation condition is met.

[0173] (Effects)

[0174] Based on the above configuration, when the cooperating machine moves from outside the reference area into it, the first auxiliary processing element determines that the preparation conditions are met. Therefore, even if the cooperating machine approaches the second machine outside the predetermined work time slot, the second machine will begin control to move to a suitable position for cooperating or assume a suitable posture for cooperating. Thus, compared to operating a second machine that is not pre-prepared, the operator can immediately begin the second operation, thereby improving work efficiency.

[0175] Furthermore, in this invention, it is preferred that, during the time from the start time to the end time of the first operation, when the machine that works in cooperation with the second operating machine moves from outside to inside the reference area determined based on the location of the second operating machine, the first auxiliary processing element determines that the preparation condition is met.

[0176] (Effects)

[0177] Based on the above configuration, during the time from the start to the end of the first operation, when the cooperating machine moves from outside to within the reference area defined by the location of the second operating machine, the first auxiliary processing element determines that the preparation conditions are met. Therefore, even if the cooperating machine approaches the second operating machine while the operator is performing the first operation, the second operating machine will have already moved to a suitable position or assumed a suitable posture for the second operation while the operator continues the first operation. Thus, the operator can continue the first operation until just before the start of the second operation, and then switch to the ready second operating machine to begin the second operation, thereby improving work efficiency.

[0178] Therefore, even if the second machine cannot move to a suitable position before the start time of the second operation, it can still move to a suitable position as much as possible. Furthermore, even if the second machine cannot reach a suitable ready position before the start time of the second operation, it can still reach a suitable position as much as possible. Therefore, compared to operating a second machine without any preparation, the operator can start the second operation as quickly as possible, thus improving work efficiency.

[0179] Furthermore, in this invention, it is preferred that the reference area is set in a shape that extends eccentrically from the location of the second working machine toward the entrance and exit of the work site, the entrance and exit of which is used for the entry and exit of the cooperating working machine to work in coordination with the second working machine.

[0180] (Effects)

[0181] Based on the above configuration, the reference area has a shape that extends eccentrically towards the entrance and exit of the work site. Therefore, in the work site, the entrance and exit, as the part most likely to be entered by the cooperating machinery, will definitely be included in the reference area, and the reference area AR is narrowed in the remaining parts, thereby allowing for the setting of a reasonable reference area. This reduces the probability of irrelevant working machinery being misidentified as cooperating machinery, thus reducing the probability of starting preparation control for a second working machine due to misidentification, and improving work efficiency.

[0182] In addition, in this invention, it is preferred that the remote operation assistance device has a second auxiliary processing element, which performs processing for identifying the operation status of the remote operation mechanism being operated, and sending remote operation instructions according to the operation status to remotely operate the work machinery.

[0183] (Effects)

[0184] Based on the above configuration, the working machinery is remotely operated according to the operating status of the remote control mechanism. Therefore, the operator can actually remotely operate the ready-to-work machinery through the second auxiliary processing element. Thus, compared to operating machinery that is not pre-prepared, the operator can begin work immediately, thereby improving work efficiency.

[0185] Symbol Explanation

[0186] 10: Remote operation auxiliary server; 20: Remote operation device; 40: Working machinery; 40A: First working machinery; 40B: Second working machinery; 50: Cooperative working machinery; 100: Remote operation auxiliary device; 101: First auxiliary processing element; 102: Second auxiliary processing element; 110: Database; 122: Server wireless communication equipment; 450: Lower traveling body; 460: Upper rotating body; 440: Working mechanism; AR: Reference area; 211: Remote operation mechanism.

Claims

1. A remote operation assistance device for performing control to initiate preparations for a first working machine, which is remotely operated by an operator, to begin a task to be performed by the working machine, characterized in that: The remote operation assistance device has a first auxiliary processing element. The first auxiliary processing element performs first auxiliary processing, which involves retrieving information, i.e., planning information, containing the content of the first operation (i.e., the first task) to be performed by the first operating machine from a database storing planning information. Based on this planning information, it determines whether the conditions for preparing the first operating machine to start the first task (i.e., preparation conditions) are met. If the determination is yes, it sends a control signal (i.e., a preparation signal) to the first operating machine to start the preparation for the first task. The preparation signal is a control signal that, based on the content of the first task, controls the first working machine to move or change its posture towards the target area of ​​the first task. The first auxiliary processing element is configured to determine that the preparation conditions are met when the current time is a time before the start time of the first job.

2. The remote operation assistance device according to claim 1, characterized in that: When the time between the start and end times of the first job stored in the database includes the start time of the second job stored in the database, the first auxiliary processing element determines that the preparation condition is met.

3. The remote operation assistance device according to claim 1 or 2, characterized in that: The first auxiliary processing element sends a preparation signal to the first working machine, the preparation signal being a signal that includes a signal for driving the lower traveling body of the first working machine.

4. The remote operation assistance device according to claim 1 or 2, characterized in that: The first auxiliary processing element sends a preparation signal to the first working machine, the preparation signal being a signal that includes a signal for rotating the upper rotating body of the first working machine.

5. The remote operation assistance device according to claim 1 or 2, characterized in that: The first auxiliary processing element sends a preparation signal to the first working machine, the preparation signal being a signal that includes signals for activating the working mechanism of the first working machine.

6. The remote operation assistance device according to claim 1, characterized in that: When the operator interrupts the remote operation of the first working machine and the machine that works in coordination with the second working machine (which is the object of the remote operation) moves from outside the reference area determined by the location of the second working machine to inside the reference area, the first auxiliary processing element determines that the preparation conditions are met.

7. The remote operation assistance device according to claim 6, characterized in that: During the time from the start to the end of the first operation, when the machine that works in conjunction with the second operating machine moves from outside the reference area determined by the location of the second operating machine to inside it, the first auxiliary processing element determines that the preparation conditions are met.

8. The remote operation assistance device according to claim 6 or 7, characterized in that: The reference area is defined as a shape that extends eccentrically from the location of the second working machine toward the entrance / exit of the work site, the entrance / exit of which is used for the entry and exit of the cooperating working machine to work in conjunction with the second working machine.

9. The remote operation assistance device according to any one of claims 1, 2, 6 and 7, characterized in that: It has a second auxiliary processing element. The second auxiliary processing element performs processing to identify the operating status of the remote operating mechanism being operated, and sends remote operation commands based on the operating status to remotely operate the machine.

10. A remote operation assistance method for performing control to initiate preparations for a first working machine, which is an object to be remotely operated by an operator, to begin a task to be performed by the working machine, characterized in that: The process includes a first auxiliary processing step, which involves retrieving information, i.e., planning table information, containing the content of the first operation performed by the first working machine from a database storing planning table information. Based on this planning table information, it determines whether the conditions for preparing the first working machine to start the first operation, i.e., preparation conditions, are met. If the determination is yes, a control signal for starting the first operation, i.e., a preparation signal, is sent to the first working machine. The preparation signal is a control signal that, based on the content of the first task, controls the first working machine to move or change its posture towards the target area of ​​the first task. In the first auxiliary processing step, the preparation condition is determined to be met when the current time is a time before the start time of the first operation.

Images