Work support system

Abstract

This system provides a work support system that allows for training in the operation of a work machine in a simulated environment, even if there is no operational data recorded when a specific task or task using a specific function is performed. [Solution] The server 13 extracts specific operational data from the operational data stored in the database 11, generates an initial simulation video to start training for the virtual excavator 15 and the virtual construction site 14 based on the specific operational data, and outputs an operation simulation video to the first output interface 10b that reflects the operation of the virtual excavator 15 and the changes in the virtual construction site 14 corresponding to the operation of the virtual excavator 15 in the initial simulation video.

Description

Technical Field

[0001] The present invention relates to a work support system that supports the improvement of the operation technology of a work machine by an operator.

Background Art

[0002] In recent years, in work machines such as hydraulic excavators and bulldozers, work support functions have become development items, such as a control system intervening in the control of the work machine and moving the work machine semi-automatically in response to a specific operation by an operator. By semi-automating some of the work of the work machine, the operator operations required for that work are reduced, so the fatigue of the operator can be reduced. Also, for operators with little experience, it is desirable to accumulate training for work using the above new functions by simulation in order to improve work efficiency at the actual work site.

[0003] As a prior art document regarding the improvement of the operation technology of a work machine for an operator who is not used to operating, there is Patent Document 1. Patent Document 1 describes a work support server that reproduces a work environment video from the time specified by the operator, or generates and displays a virtual work site video that is the same as the work environment at the specified time and is operable, superimposed on it. Thereby, the operator can accumulate operation experience by simulation while referring to the state of operation of the work machine by himself or other operators.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with the technology described in Patent Document 1, operators cannot gain operational experience through simulation for tasks for which operational data such as video data has not been accumulated (for example, tasks using functions that have never been used in past actual work, such as new functions that are under development or have just been commercialized).

[0006] The present invention has been made in view of the above-mentioned problems, and its objective is to provide a work support system that enables training in the operation of a work machine for performing a specific task or a task using a specific function to be conducted in a simulation environment, even if there is no operational data of the work machine recorded when the task is performed. [Means for solving the problem]

[0007] To achieve the above objective, the present invention provides a work support system comprising: a database for storing operational data including the operation of a work machine and external environment recognition information; a server that generates simulation videos of a virtual work machine that simulates the work machine and a virtual work site that simulates the work site of the work machine based on the operational data stored in the database; and an output interface for outputting the simulation videos generated by the server, wherein the system includes an input interface for inputting operations to operate the virtual work machine and specifying the work to be trained or the work using the work machine's training target function to the server, the server extracts specific operational data from the operational data stored in the database based on the training target work or the work using the training target function specified by the input interface, generates an initial simulation video which is the simulation video for starting training of the virtual work machine and the virtual work site based on the specific operational data and outputs it to the output interface, and generates an operation simulation video which is the simulation video that reflects the operation of the virtual work machine and the changes in the virtual work site corresponding to the operation of the virtual work machine in the initial simulation video and outputs it to the output interface in response to the operation input from the input interface. [Effects of the Invention]

[0008] According to the present invention, even if there is no operational data recorded when a work machine performs a specific task or a task using a specific function, it becomes possible to conduct training on the operation of a work machine for performing such a task in a simulation environment. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of a shovel in an embodiment of the present invention. [Figure 2] This is an external view of a work support system according to an embodiment of the present invention. [Figure 3] This is a functional block diagram of a work support system in an embodiment of the present invention. [Figure 4] This figure shows an example of a simulation image displayed on the first output interface in an embodiment of the present invention. [Figure 5] This figure shows an example of the display screen of the second input interface in an embodiment of the present invention. [Figure 6] This is a flowchart showing the processing of the training condition generation unit in an embodiment of the present invention. [Figure 7] This figure shows an example of the comparison result between the first evaluation index value and the second evaluation index value displayed on the second output interface in an embodiment of the present invention. [Figure 8] This figure shows an example of a simulation image displayed on the first output interface in an embodiment of the present invention. [Figure 9] This is a conceptual diagram illustrating the operation of a shovel using the excavation support function in an embodiment of the present invention. [Figure 10] This figure shows an example of specific operational data extracted from a database in an embodiment of the present invention. [Figure 11] This figure shows another example of specific operational data extracted from a database in an embodiment of the present invention. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, a work support system for simulating the operation of a large hydraulic excavator (hereinafter referred to as "excavator") equipped with a function to assist the operator's operation when performing excavation work will be described as an example. However, the present invention is also applicable to other work machines such as small hydraulic excavators and large cranes.

[0011] Figure 1 is a schematic diagram of the shovel 1. The shovel 1 comprises a lower traveling body 5, an upper rotating body 3 that is rotatably mounted on the lower traveling body 5, and a working device 4 attached to the front of the upper rotating body 3. The upper rotating body 3 is driven to rotate relative to the lower traveling body 5 by a rotating drive device (not shown).

[0012] The working device 4 includes a boom 4a mounted on the front side of the upper slewing body 3 so as to be rotatable in the vertical direction, an arm 4b mounted on the tip of the boom 4a so as to be rotatable in the vertical or longitudinal direction, a bucket 4c mounted on the tip of the arm 4b so as to be rotatable in the vertical or longitudinal direction, a boom cylinder 4d for driving the boom 4a, an arm cylinder 4e for driving the arm 4b, and a bucket cylinder 4f for driving the bucket 4c.

[0013] The upper rotating body 3 is equipped with a driver's cab 2 where the operator sits, a drive unit 6 that drives the work equipment 4 and the slewing drive unit, a controller 7 that controls the drive unit 6, and a communication device 8 that can communicate with external equipment. The driver's cab 2 is equipped with an operating lever (not shown) for instructing the operation of the work equipment 4 or the slewing drive unit. The drive unit 6 drives the work equipment 4 or the slewing drive unit in response to the operation of the operating lever. The drive unit 6 is often composed of hydraulic equipment, but may also be composed of an electric motor or the like.

[0014] The controller 7 is connected to the drive device 6 via a signal line, intervenes in the operator's operation as necessary, and outputs a command signal for assisting the operator's operation to the drive device 6. Further, the controller 7 is connected to the communication device 8 via a signal line. The controller 7 can also output a control signal corresponding to a command signal from a cab (not shown) outside the excavator 1 to the drive device 6. Thereby, remote operation of the excavator 1 becomes possible.

[0015] FIG. 2 is an external view of the work support system 9. The work support system 9 includes a server 13, a first input interface 10a, a first output interface 10b, a second input interface 10c, a second output interface 10d, and a database 11 (shown in FIG. 3).

[0016] The first input interface 10a is composed of a training operation lever that simulates the operation lever provided in the cab 2, and is connected to the server 13 via a signal line. The first input interface 10a is disposed near the training seat 12 that simulates a seat (not shown) provided in the cab 2. The first input interface 10a instructs the server 13 about the operation of the virtual excavator 15 described later. That is, the first input interface 10a is an input interface for performing an operation input for operating the virtual excavator 15 to the server 13.

[0017] The first output interface 10b is composed of at least one monitor arranged near the training sheet 12 and is connected to the server 13 via a signal line. The first output interface 10b displays a simulation video 18 (initial simulation video) for starting training as shown in FIG. 4. The simulation video 18 (initial simulation video) includes a virtual site 14 simulating the work site, a virtual excavator 15 simulating the excavator 1 at the work site, a virtual dump truck 16 simulating the dump truck at the work site, a virtual bulldozer (not shown) simulating the bulldozer at the work site, etc. The virtual excavator 15 includes a virtual lower traveling body 15a simulating the lower traveling body 5, a virtual upper slewing body 15b simulating the upper slewing body 3, and a virtual working device 15c simulating the working device 4. Note that the first output interface 10b may be composed of VR goggles or the like to be worn by the operator. When the first input interface 10a is operated, an operation simulation video is displayed on the first output interface 10b. The operation simulation video is a simulation video 18 in which the operation of the virtual excavator 15 and the change of the virtual site 14 corresponding to the operation are reflected in the initial simulation video according to the operation input from the first input interface 10a. The operator sitting on the training sheet 12 can accumulate the operation experience of the excavator 1 by operating the first input interface 10a while observing the operation of the virtual excavator 15 and the change of the virtual site 14 in the simulation video 18 (operation simulation video) displayed on the first output interface 10b.

[0018] The second input interface 10c is composed of, for example, a touch panel monitor and is connected to the server 13 via a signal line. The second input interface 10c is an input interface for performing a designated input to the server 13 for the training target work of the excavator 1 or the work using the training target function described later. The second input interface 10c instructs the server 13 of the training target work or the training target function selected by the operator.

[0019] The second output interface 10d consists of a monitor and is connected to the server 13 via a signal line. The second output interface 10d outputs evaluation index values ​​(described later) for tasks or tasks using the tasks to be trained or the functions to be trained.

[0020] Figure 3 is a functional block diagram of the work support system 9. The work support system 9 comprises a server 13, a first input interface 10a, a first output interface 10b, a second input interface 10c, a second output interface 10d connected to the server 13, and a database 11. The database 11 is connected to the server 13 via signal lines. The database 11 may be configured on a computer independent of the server 13, or it may be built into the server 13.

[0021] In this embodiment, the case in which the work support system 9 has a first input interface 10a and a second input interface 10c is described, but the configuration of the input interfaces of the work support system 9 is not limited to this. For example, the work support system 9 may be configured to accept inputs via these two input interfaces 10a and 10c through a single interface.

[0022] Furthermore, in this embodiment, the case in which the work support system 9 has a first output interface 10b and a second output interface 10d is described, but the configuration of the output interfaces of the work support system 9 is not limited to this. For example, the work support system 9 may be configured to output information from these two output interfaces 10b and 10d through a single interface.

[0023] Database 11 has the function of storing past operational data of Excavator 1. The operational data includes video data from the imaging device installed on Excavator 1, the attitude angle of Excavator 1, position and orientation information from GNSS (Global Navigation Satellite System), and various pressure values ​​of the hydraulic circuit that drives Excavator 1. In addition, the operational data may also include position information of other work machines that work in conjunction with Excavator 1, such as dump trucks and bulldozers, which are used at the work site.

[0024] Server 13 comprises a shovel control calculation unit 13a, a shovel motion calculation unit 13b, an environmental change calculation unit 13c, a training condition generation unit 13d, a training evaluation index generation unit 13e, an operation command transmission unit 13f, and an external environment recognition information receiving unit 13g. Server 13 is equipped with a processing unit (processor) such as a CPU, a storage device (memory) such as ROM and RAM, and an input / output interface for signal input and output with external devices, and realizes the functions of each unit by executing programs stored in ROM, etc.

[0025] The shovel control calculation unit 13a calculates the control of the shovel 1 (controls executed by the controller 7 and the behavior of hydraulic components, etc.) that will be executed when the operating lever of the operator's cab 2 is operated in the same way as the first input interface 10a, and outputs the result to the shovel motion calculation unit 13b.

[0026] The shovel motion calculation unit 13b calculates the motion of shovel 1 based on the calculation results of the shovel control calculation unit 13a. The motion of shovel 1 referred to here is the motion that can be observed externally and is calculated, for example, using the equation of motion of shovel 1.

[0027] The environmental change calculation unit 13c generates a virtual work site 14, which is a CG-based reproduction of the work site where the operator is trained, as shown in Figure 4, and a virtual shovel 15, which is a CG-based representation of the operation of shovel 1 calculated by the shovel operation calculation unit 13b. The time-series video of the virtual work site 14 and virtual shovel 15 generated by the environmental change calculation unit 13c is displayed on the first output interface 10b. The environmental change calculation unit 13c also performs calculations such that the shape of the excavation surface 17 in the virtual work site 14 changes when the virtual shovel 15 performs the operation of digging the excavation surface 17 with its bucket. Other work machines, such as a virtual dump truck 16 which is the target of loading by the virtual shovel 15, may also be represented in CG-based in the virtual work site 14. The video displayed on the first output interface 10b may be a bird's-eye view as shown in Figure 4, or a view from the driver's cab 2 of shovel 1.

[0028] As shown in Figure 5, the second input interface 10c has the function of specifying the training target function and the training target operation. The training target operation is excavation, leveling, movement, etc. The training target function is an operation support function that supports the operation, such as an excavation support function that performs excavation work semi-automatically, or an automatic loading control that assists in loading soil into a dump truck. The second input interface 10c may also specify the type of shovel 1 to be trained, and the operation method such as onboard operation or remote operation. If it is to train a remote operation site, the wireless communication quality to be used for remote operation may be selected. In this case, the second input interface 10c is configured to accept input for selecting the communication quality when remotely operating the shovel 1. According to the selected communication quality, the environment change calculation unit 13c sets a delay time in the display of the behavior of the shovel 1 in response to the operation input (operation signal) from the first input interface 10a, and reproduces the operation delay in remote operation. In other words, the environmental change calculation unit 13c outputs a simulation video 18 (operation simulation video) to the first output interface 10b to change the operation of the virtual excavator 15 and the virtual work site by introducing a delay in response to the operation input from the first input interface 10a according to the communication quality. Furthermore, according to the selected communication quality, the environmental change calculation unit 13c may process the video displayed on the first output interface 10b to simulate video distortion.

[0029] The training condition generation unit 13d calculates environmental conditions that enable effective operation training related to a specific task (training target task) selected by the second input interface 10c, or a specific function (training target function) selected by the second input interface 10c, and extracts specific operational data that satisfies these environmental conditions from the database 11. Furthermore, the training condition generation unit 13d instructs the environmental change calculation unit 13c to generate a virtual work site 14 that simulates the environmental conditions of the extracted specific operational data. The environmental change calculation unit 13c may also generate multiple patterns of virtual work sites 14 and virtual excavators 15 by adding fluctuation values ​​to the specific operational data.

[0030] The operation command transmission unit 13f has the function of outputting instructions from the first input interface 10a to the communication device 8 of the shovel 1. The external environment recognition information receiving unit 13g has the function of receiving external environment recognition information of the shovel 1 via the communication device 8 and outputting it to the first output interface 10b. As a result, the operator seated in the training seat 12 can remotely operate the shovel 1 by operating the first input interface 10a while checking the external environment recognition information of the shovel 1 displayed on the first output interface 10b.

[0031] Figure 6 is a flowchart 19 showing the processing of the training condition generation unit 13d. The processing of the training condition generation unit 13d starts simultaneously with the activation of the work support system 9.

[0032] The training condition generation unit 13d first displays a screen to select whether to train (operation training) the shovel 1 or to remotely operate it (step S101).

[0033] Following step S101, it is determined whether training was selected in step S101 (step S102).

[0034] If the result of step S102 is Yes (training is selected), select the task (task to be trained) or function (function to be trained) for which you want to perform operational training (step S103).

[0035] Following step S103, the operating conditions and decision conditions for the training target operation or training target function are acquired and defined as decision variables. The operating conditions are, for example, the operating patterns of the first input interface 10a. The decision conditions are, for example, sensor signals used to determine whether or not to intervene with operation support control when the selected training target function is an operation support function. The sensor signals are, for example, the pressure of the hydraulic actuator equipped on the shovel 1 and the attitude angle of the shovel 1.

[0036] Following step S104, operational data is obtained from database 11, and data intervals in which the fluctuation value and fluctuation time of the decision variable are greater than or equal to a predetermined threshold are searched (step S105).

[0037] Following step S105, it is determined in the operational data within the database 11 whether there is a specific data interval where the fluctuation value and fluctuation time of the determination variable are greater than or equal to a predetermined threshold (step S106).

[0038] If the result of step S106 is Yes, the operational data (specific operational data) for the specific data interval [Ts, Te] is extracted and set as training data. Here, time Ts is the start time of the specific data interval, and time Te is the end time of the specific data interval.

[0039] Following step S107, the training start time is set to time Ts-ΔT1, which is obtained by subtracting a predetermined margin time ΔT1 from time Ts (step S108). The margin time ΔT1 is a few seconds and is a constant used to start the simulation a few seconds before the operator performs the task or function to be trained during operational training.

[0040] Following step S108, the training condition generation unit 13d extracts operational data for a specific data interval [Ts-ΔT1,Te] from the database 11 and instructs the environment change calculation unit 13c to generate a virtual site 14 and a virtual excavator 15 similar to the work site and excavator 1 in said operational data (step S109), and then terminates the flow.

[0041] If the result of step S106 is No, then because there is no operational data in database 11 that satisfies the conditions for effectively conducting training, the environment change calculation unit 13c is instructed to generate a virtual work site 14 and a virtual excavator 15 for the training target work or training target function that have been prepared in advance (step S110), and the flow is terminated.

[0042] If the result of step S102 is No (remote control is selected), select shovel 1 to be remotely controlled (step S111).

[0043] Following step S111, the operation command transmission unit is instructed to send an operation command corresponding to the operation of the first input interface 10a to the shovel 1 selected in step S111, and the external environment recognition information receiving unit 13g is instructed to acquire external environment recognition information of the shovel 1 (step S112), and the flow ends.

[0044] Returning to Figure 3, the training evaluation index generation unit 13e calculates an evaluation index value for the training target task or the task using the training target function based on the operation of the shovel 1 in specific operational data extracted from the database 11, and stores it as the first evaluation index value. Furthermore, it calculates an evaluation index value for the training target task or the task using the training target function based on the operation of the virtual shovel 15, and stores it as the second evaluation index value. Then, it compares the first evaluation index value and the second evaluation index value and outputs the result to the second output interface 10d.

[0045] Figure 7 shows an example of the comparison result between the first evaluation index value and the second evaluation index value displayed on the second output interface 10d. When work speed is shown as an example of an evaluation index value, the first evaluation index value and the second evaluation index value are compared, and the increase or decrease ratio of the second evaluation index value relative to the first evaluation index value may be shown as a percentage, such as the index value ratio 20a. Alternatively, as shown in the graph on the right of Figure 7, the time series change of the first evaluation index value may be displayed by comparing it with the time series of the second evaluation index value, such as waveform 20b, and the time series of the second evaluation index value, such as waveform 20c. Furthermore, the highest or lowest value of the second evaluation index value calculated during training with the work support system 9 in the past may be used as the first evaluation index value.

[0046] Furthermore, the training evaluation index generation unit 13e may, following the flowchart 19 in Figure 6, generate a CG simulation video 18 (operation simulation video) that simulates the operation of virtual work devices 21 and 22 based on training data extracted from the database 11, as shown in Figure 8, and superimpose it on the first output interface 10b. For example, the superimposed virtual work device 21 is displayed as a guideline for operation training based on training data extracted from the database 11, and the virtual work device 21 is displayed at a time and position preceding the virtual shovel 15 operated by the first input interface 10a. If the virtual shovel 15 is rotated by the first input interface 10a, it may be displayed at a time and position preceding the input rotation direction, similar to the virtual work device 22. Also, when displaying virtual work devices 21 and 22 as guidelines for operation training, adjustments such as reducing the superimposed display speed of virtual work devices 21 and 22 may be made to match the movement of the virtual shovel 15 operated by the first input interface 10a.

[0047] (effect) The effects of the invention in this embodiment will be explained using the situation in Figure 9 as an example. The explanation will be based on the virtual site 14 and virtual shovel 15 generated when the training target function selected in the second input interface 10c is the excavation support function. The excavation support function is a control that, when the shovel 1 excavates the excavation surface 17 and the tip of the bucket 4c gets stuck on rock 23, causing the pressure of the arm cylinder 4e and bucket cylinder 4f to exceed a threshold, automatically raises the boom 4a through operational intervention to relieve the load on the tip of the bucket and prevent the excavation work from stopping. This excavation support function can suppress a decrease in work speed. In order to effectively train the excavation support function, in step S104 of the flowchart 19 in Figure 6, the arm operation amount, bucket operation amount, arm cylinder pressure, bucket cylinder pressure, and control intervention judgment threshold are selected from the database 11 as judgment variables for the excavation support function. An example of specific operational data extracted from the database 11 in steps S105 to S107 is shown in Figure 10. Figure 10 shows the time history of the decision variables: arm operation amount 24, bucket operation amount 25, arm cylinder pressure 26, and bucket cylinder pressure 27. In step S105, the operational data for the data interval [Ts, Te] before and after the arm cylinder pressure 26 and bucket cylinder pressure 27 exceed the control intervention decision threshold 28 provided for the excavation support function is extracted as specific operational data.

[0048] In this case, it is not necessary for the database 11 to contain operational data in which the excavation support function is actually being used. If there is operational data from when the bucket tines of the shovel 1 got stuck on rocks while excavating without using the excavation support function, it is possible to extract data intervals for training. Therefore, even if there is no prior experience using the excavation support function, the work support system 9 can generate a virtual work site 14 in which the excavation support function can be effectively used, and the operator can receive training on operating the excavation support function using the work support system 9.

[0049] Figure 11 shows another example of specific operational data extracted from the database 11 in steps S105-S107 of flowchart 19 in Figure 6. Figure 11 shows the time history of the decision variables: the amount of operation of the slewing lever 29a, 29b, the amount of operation of the travel lever 29c, and the absolute attitude angle 30 of the shovel 1 (0 degrees when the ground is horizontal). As shown by the amount of operation of the travel lever 29c, if the travel lever is pushed down and the shovel 1 runs onto a rock on the ground, for example, the shovel 1 will tilt and the absolute attitude angle 30 will increase. At this time, if the slewing lever is continuously operated as shown by the amount of operation of the slewing lever 29b, the center of gravity of the shovel 1 will change and there is a risk that the shovel 1 will tip over. In cases where there is a risk of tipping over due to slewing, it is effective to disable the operation of the slewing lever by using the tipping prevention support function. Specifically, when the slewing lever is operated as indicated by the slewing lever operation amount 29b, the slewing lever operation is disabled from time 32 onward, when the absolute attitude angle 30 of the shovel 1 exceeds the threshold 31, as indicated by the slewing lever operation amount 29a. By training on this anti-tipping support function in advance using the work support system 9, the operator can perform operations to restore the shovel 1 from an unstable state without inadvertently performing other operations.

[0050] Furthermore, by gaining operational training while checking the evaluation index values, which are training results displayed on the second output interface 10d shown in Figure 7, it is possible to conduct effective operational training that further improves the evaluation index values.

[0051] Furthermore, as shown in Figure 8, the environmental change calculation unit 13c adjusts the display speed of the virtual work device 21, which is superimposed and displayed in accordance with the movement of the virtual excavator 15, thereby enabling focused training on only the work trajectory of the virtual excavator 15. Therefore, it can provide appropriate training tailored to the operator's experience level, such as starting with a slower display speed for less experienced operators.

[0052] (summary) The work support system 9 according to this embodiment includes a database 11 that stores operational data including the operation of the shovel 1 (working machine) and external environment recognition information, and a server 13 that generates simulation video 18 of a virtual shovel 15 (virtual working machine) that simulates the shovel 1 and a virtual work site 14 that simulates the work site of the shovel 1, based on the operational data stored in the database 11. The work support system 9 also includes a first output interface 10b, which is an output interface for outputting the simulation video 18 generated by the server 13, a first input interface 10a, which is an input interface for making operation inputs to the server 13 for operating the virtual shovel 15, and a second input interface 10c, which is an input interface for making specification inputs to the server 13 for the training target work or work using the training target function of the shovel 1. Based on the training target work or work using the training target function specified in the second input interface 10c, the server 13 extracts specific operational data from the operational data stored in the database 11. Server 13 generates an initial simulation video, which is a simulation video 18 for starting training of the virtual excavator 15 and the virtual construction site 14, based on specific operational data, and outputs it to the first output interface 10b. In response to the operation input from the first input interface 10a, Server 13 generates an operation simulation video, which is a simulation video 18 that reflects the operation of the virtual excavator 15 and the changes in the virtual construction site 14 corresponding to the operation of the virtual excavator 15, in the initial simulation video, and outputs it to the first output interface 10b.

[0053] According to this embodiment configured as described above, even if there is no operational data recorded when the shovel 1 performs a specific task or a task using a specific function, it is possible to simulate the operation of the shovel 1 to perform the task. Therefore, by conducting operational training in advance for tasks that have never been performed before, or for tasks using functions that have never been used before (for example, functions that are under development or have just been commercialized), high productivity can be achieved at the actual work site.

[0054] Furthermore, in this embodiment, the second input interface 10c is configured to accept input for selecting the communication quality when remotely operating the shovel 1 (working machine). The server 13 processes the simulation video 18 output to the first output interface 10b according to the communication quality. The server 13 outputs the operation simulation video to the first output interface 10b in such a way that it changes the operation of the virtual shovel 15 (virtual working machine) and the virtual work site 14 by adding a delay according to the communication quality to the operation input from the first input interface 10a. This makes it possible to conduct operation training for remotely operating the shovel 1.

[0055] Furthermore, in this embodiment, the work support system 9 is equipped with a second output interface 10d as an output interface. The server 13 calculates an evaluation index value for the work to be trained or the work using the trained function as the first evaluation index value based on the operation of the shovel 1 (work machine) in specific operational data. The server 13 calculates an evaluation index value for the work to be trained or the work using the trained function as the second evaluation index value based on the operation of the virtual shovel 15 (virtual work machine). The server 13 compares the first evaluation index value and the second evaluation index value and outputs the comparison result to the second output interface 10d. This makes it possible to evaluate the degree of achievement of operation training using the work support system 9.

[0056] Furthermore, in this embodiment, the virtual excavator 15 (working machine) comprises a virtual lower traveling body 15a that simulates the lower traveling body 5 of the excavator 1 (working machine), a virtual upper rotating body 15b that simulates the upper rotating body 3 that is rotatably attached to the lower traveling body 5, and a virtual working device 15c that simulates the working device 4 that is rotatably attached to the upper rotating body 3. The server 13 has a function that, when the virtual upper rotating body 15b rotates, superimposes the virtual working device 22 at a preceding time point onto the simulation image 18, using the position and orientation of the virtual lower traveling body 15a as a reference. This makes it possible to effectively conduct operational training when aligning the working device 4 by rotating.

[0057] Furthermore, the server 13 in this embodiment has a function to intervene in the operation of the virtual shovel 15 (virtual work machine) input from the first input interface 10a. This makes it possible to perform operation training using the function to intervene in the operation of the shovel 1.

[0058] Furthermore, in this embodiment, the virtual excavator 15 (working machine) comprises a virtual lower traveling body 15a that simulates the lower traveling body 5 of the excavator 1 (working machine), a virtual upper rotating body 15b that simulates the upper rotating body 3 that is rotatably attached to the lower traveling body 5, and a virtual working device 15c that simulates the working device 4 that is rotatably attached to the upper rotating body 3. The training target function includes an excavation support function that assists in excavation work performed by the working device 4. The server 13 has a function to intervene in the operation of the virtual working device 15c input from the first input interface 10a when the load of the virtual working device 15c (arm cylinder pressure 26, bucket cylinder pressure 27) exceeds a predetermined threshold. This makes it possible to perform operation training for work using the excavation support function.

[0059] Furthermore, in this embodiment, the virtual shovel 15 (working machine) comprises a virtual lower traveling body 15a that simulates the lower traveling body 5 of the shovel 1 (working machine), a virtual upper rotating body 15b that simulates the upper rotating body 3 that is rotatably attached to the lower traveling body 5, and a virtual working device 15c that simulates the working device 4 that is rotatably attached to the upper rotating body 3. The training target function includes a fall prevention support function that assists in preventing the shovel 1 from tipping over. The server 13 has a function that intervenes in the operation of the virtual upper rotating body 15b input from the first input interface 10a when the attitude angle of the virtual shovel 15 exceeds a predetermined threshold. This makes it possible to perform operation training using the fall prevention support function.

[0060] Furthermore, in this embodiment, the server 13 has the function of generating a virtual dump truck 16 (virtual transport machine) or a virtual bulldozer (virtual earthmoving machine) in addition to the virtual excavator 15 (virtual work machine). This makes it possible for the excavator 1 and the dump truck or bulldozer to work together and conduct operational training.

[0061] Furthermore, the server 13 in this embodiment has the function of generating multiple patterns of virtual work sites 14 and virtual excavators 15 (virtual work machines) by adding fluctuation values ​​to specific operational data. This makes it possible to perform flexible operation training.

[0062] Furthermore, the server 13 in this embodiment has the function of transmitting instructions from the first input interface 10a to the shovel 1 (working machine) and the function of receiving external environment recognition information from the shovel 1 and outputting it to the first output interface 10b. This makes it possible for an operator trained with the work support system 9 to remotely control the shovel 1 via the first input interface 10a.

[0063] Although embodiments of the present invention have been described in detail above, the present invention is not limited to the embodiments described above and includes various modifications. For example, the embodiments described above have been described in detail for the purpose of explaining the present invention in an easy-to-understand manner and are not necessarily limited to those having all the configurations described. [Explanation of symbols]

[0064] 1...Shovel, 2...Operator's cab, 3...Upper slewing body, 4...Working equipment, 4a...Boom, 4b...Arm, 4c...Bucket, 4d...Boom cylinder, 4e...Arm cylinder, 4f...Bucket cylinder, 5...Lower traveling body, 6...Drive unit, 7...Controller, 8...Communication device, 9...Work support system, 10a...First input interface (input interface), 10b...First output interface (output interface), 10c...Second input interface (input interface), 10d...Second output interface (output interface) 11...Database, 12...Training sheet, 13...Server, 13a...Shovel control calculation unit, 13b...Shovel motion calculation unit, 13c...Environmental change calculation unit, 13d...Training condition generation unit, 13e...Training evaluation index generation unit, 13f...Operation command transmission unit, 13g...External environment recognition information reception unit, 14...Virtual work site, 15...Virtual shovel, 15a...Virtual lower traveling body, 15b...Virtual upper rotating body, 15c...Virtual working device, 16...Virtual dump truck, 17...Excavation surface, 18...Simulation video, 19...Flowchart, 20a...Indicator value ratio, 20b...Waveform, 20c...Waveform, 2

Claims

1. A database that stores operational data including the operation of the work machine and external environment recognition information, A server that generates simulation images of a virtual work machine that simulates the work machine and a virtual work site that simulates the work site of the work machine, based on the operational data stored in the database, In a work support system comprising an output interface that outputs a simulation image generated by the server, The system includes an input interface for inputting operations to operate the virtual work machine and specifying the work to be trained or the work using the function to be trained for the work machine to be trained, to be sent to the server. The aforementioned server, Based on the training target task or task using the training target function specified in the input interface, specific operational data is extracted from the operational data stored in the database. Based on the aforementioned specific operational data, an initial simulation video, which is the simulation video for starting training on the virtual work machine and the virtual work site, is generated and output to the output interface. In response to the operation input from the input interface, the system generates an operation simulation video, which is the simulation video that reflects the operation of the virtual work machine and the changes in the virtual work site corresponding to the operation of the virtual work machine, and outputs it to the output interface. A work support system characterized by the following features.

2. In the work support system described in claim 1, The input interface is configured to accept input for selecting the communication quality when remotely operating the work machine. The aforementioned server, The simulation video output to the output interface is processed according to the communication quality. The operation simulation video is output to the output interface in such a way that the operation of the virtual work machine and the virtual work site are changed by adding a delay to the operation input from the input interface according to the communication quality. A work support system characterized by the following features.

3. In the work support system described in claim 1, The aforementioned server, Based on the operation of the work machine in the aforementioned specific operational data, the evaluation index value of the training target work or the work using the training target function is calculated as the first evaluation index value. Based on the operation of the virtual work machine, the evaluation index value of the training target work or the work using the training target function is calculated as the second evaluation index value. The first evaluation index value and the second evaluation index value are compared, The results of the comparison are output to the output interface. A work support system characterized by the following features.

4. In the work support system described in claim 1, The aforementioned virtual work machine is A virtual lower traveling body that simulates the lower traveling body of the aforementioned work machine, A virtual upper rotating body that simulates an upper rotating body that is rotatably attached to the lower traveling body, The system includes a virtual work device that simulates a work device rotatably attached to the upper rotating body, The server has a function that, when the virtual upper rotating body rotates, superimposes the virtual work device at a preceding time point onto the simulation image, using the position and orientation of the virtual lower traveling body as a reference. A work support system characterized by the following features.

5. In the work support system described in claim 1, The server has the function of intervening in the operation of the virtual work machine input from the input interface. A work support system characterized by the following features.

6. In the work support system described in claim 5, The aforementioned virtual work machine is A virtual lower traveling body that simulates the lower traveling body of the aforementioned work machine, A virtual upper rotating body that simulates an upper rotating body that is rotatably attached to the lower traveling body, The system includes a virtual work device that simulates a work device rotatably attached to the upper rotating body, The training target function includes an excavation support function that assists in excavation work performed by the work device, The server has a function to intervene in the operation of the virtual work device input from the input interface when the load of the virtual work device exceeds a predetermined threshold. A work support system characterized by the following features.

7. In the work support system described in claim 5, The aforementioned virtual work machine is A virtual lower traveling body that simulates the lower traveling body of the aforementioned work machine, A virtual upper rotating body that simulates an upper rotating body that is rotatably attached to the lower traveling body, The system includes a virtual work device that simulates a work device rotatably attached to the upper rotating body, The training target function includes a fall prevention support function that assists in preventing the work machine from tipping over, The server has a function to intervene in the operation of the virtual upper rotating body input from the input interface when the attitude angle of the virtual work machine exceeds a predetermined threshold. A work support system characterized by the following features.

8. In the work support system described in claim 1, The server has the function of generating virtual transport machines or virtual land leveling machines in addition to the virtual work machines. A work support system characterized by the following features.

9. In the work support system described in claim 1, The server has the function of generating multiple patterns of the virtual work site and the virtual work machine by adding fluctuation values ​​to the specific operational data. A work support system characterized by the following features.

10. In the work support system described in claim 1, The server has the function of transmitting instructions from the input interface to the work machine and the function of receiving external environment recognition information from the work machine and outputting it to the output interface. A work support system characterized by the following features.

Images