Work machine operation system

Abstract

To enable a user to learn operations for a work machine efficiently.SOLUTION: A work machine operation system 1 comprises an output unit 30, an input unit 11, and a computer 20. The output unit 30 can display an image. The input unit 11 allows for operating a work machine 60. The calculator 20 comprises a recording unit 21 and a reproduction unit 23. The recording unit 21 records a first action (M1), which is an action of the work machine 60 corresponding to the operation on the input unit 11. The reproduction unit 23 reproduces the first action (M1) recorded in the recording unit 21. The computer 20 causes the output unit 30 to display: the first action (M1) reproduced by the reproduction unit 23; and a second action (M2), which is the action of the work machine 60 corresponding to the operation performed on the input unit 11.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a work machine operation system for operating a work machine. [Background technology]

[0002] For example, Patent Document 1 describes a system configured to allow a second operator to understand the operation of a work machine being performed by a first operator (see the abstract of Patent Document 1,

[0004] ,

[0044] , etc.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-161934 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in the document, the first operator needs to operate the work machine at the same time as having the second operator understand the operation. As a result, it is difficult to have the user (second operator) learn how to operate the work machine efficiently. Therefore, it is desired to enable the user to learn how to operate the work machine more efficiently than ever before.

[0005] Therefore, an object of the present invention is to provide a work machine operation system that allows a user to efficiently learn how to operate a work machine. [Means for solving the problem]

[0006] The work machine operation system comprises an output unit, an input unit, and a computer. The output unit is capable of displaying images. The input unit is capable of operating a work machine. The computer comprises a recording unit and a playback unit. The recording unit records a first action, which is an action of the work machine in response to operation of the input unit. The playback unit plays back the first action recorded in the recording unit. The computer causes the output unit to display the first action played back by the playback unit and a second action, which is an action of the work machine in response to operation performed on the input unit. [Effects of the Invention]

[0007] The above-described work machine operation system allows the user to learn how to operate a work machine more efficiently. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a work machine operation system 1. FIG. [Figure 2] 2 is a diagram showing an object 50 and the like displayed on the display unit 31 shown in FIG. [Figure 3] 2 is a flowchart of the operation of the work machine operation system 1 shown in FIG. [Figure 4] 1. FIG. 4 is a diagram showing a recording instruction section G21 displayed by the display section 31 shown in FIG. [Figure 5] 1. FIG. 4 is a diagram showing a playback instruction section G23 displayed by the display section 31 shown in FIG. [Figure 6] 2 is a diagram showing a work machine 60 and the like as viewed from the side, displayed on the display unit 31 shown in FIG. 1. FIG. [Figure 7] 2 is a diagram showing a work machine 60 and the like as viewed from inside a cab 62c, displayed on the display unit 31 shown in FIG. [Figure 8] 1. FIG. 4 is a diagram showing a future reproduction operation M1f at a predetermined time interval, displayed on the display unit 31 shown in FIG. [Figure 9] 1. FIG. 4 is a diagram showing a future reproduction action M1f for each step of a task, displayed by the display unit 31 shown in FIG. [Figure 10]1. FIG. 4 is a diagram showing a graph G31a that indicates the operation of the work machine 60 and is displayed on the display unit 31 shown in FIG. [Figure 11] 1. FIG. 4 is a diagram showing an operation indicator G31c based on a graph G31a displayed on the display unit 31 shown in FIG. [Figure 12] 1. FIG. 4 is a diagram showing a graphical operation indicator G31c displayed by the display unit 31 shown in FIG. [Figure 13] 1. FIG. 4 is an explanatory diagram of chapter C when the playback unit 23 shown in FIG. 1 performs divided playback. DETAILED DESCRIPTION OF THE INVENTION

[0009] The work machine operation system 1 will be described with reference to FIGS.

[0010] As shown in FIG. 1, the work machine operation system 1 is a system that records and plays back the situation within a work space. The "work space" mentioned above is a space where work is performed by a work machine 60 (described later). The work space is a space where the situation is recorded by a recording unit 21 (described later), and is a space where the situation is played back by a playback unit 23. The work space includes the space in which the work machine 60 exists. The work space includes the space inside the work machine 60 (for example, the space inside the driver's cab 62c). The work space includes the space around the work machine 60 (surrounding situation). The work space may include a virtual space, or may include an actual space (real space). The work machine operation system 1 may be a system (work machine operation simulation system) that simulates the operation (piloting) of a work machine 60 (described later) in a virtual space. The work machine operation system 1 may also be a system that operates (specifically, remotely controls) a work machine 60 in real space.

[0011] The work machine operation system 1 records and plays back at least the situation (phenomena, states, operations, etc.) of the work machine 60. The work machine operation system 1 moves the work machine 60 in response to operation of an input unit 11 (described later) by a user (operator). The work machine operation system 1 is a device (steering device) that allows the user to operate the work machine 60. The work machine operation system 1 may also record and play back the situation (phenomena, states, operations, etc.) of surrounding objects 80 (see FIG. 2, described later).

[0012] This work machine operation system 1 may be used for a variety of purposes. The work machine operation system 1 may be used as a training device (training device, teaching device) for a user to learn how to operate (one or both of on-board operation and remote operation) a real work machine (a machine or actual device corresponding to the work machine 60 displayed on the display unit 31). The following mainly describes a case where the work space is a virtual space. The work machine operation system 1 may be used as an operation simulator for simulating the operation of a real work machine. The work machine operation system 1 may also be used as a training device for a user to learn how to operate a real work machine by operating the work machine 60 in the virtual space. In this case, the user can efficiently learn (practice) how to operate the real work machine by operating the work machine 60 in the virtual space, and can efficiently become proficient in operation (quickly become proficient).

[0013] For example, the work machine operation system 1 may be used to create a plan (construction plan) for work using a real work machine at a real work site. The work machine operation system 1 may be used as an operation verification device for verifying the operation (behavior) of the work machine 60. The work machine operation system 1 may be used as a situation verification device for verifying various situations at a work site other than the operation of the work machine 60. The work machine operation system 1 may be used to reproduce the operation of a real work machine in a virtual space (performing a digital twin) based on the operating data of the real work machine.

[0014] This work machine operation system 1 may include one or more devices selected from the group consisting of a personal computer, a tablet, and a smartphone. The work machine operation system 1 may also include a remote control device that remotely controls an actual work machine (described below).

[0015] This work machine operation system 1 may include a client device 10c and a server device 10s. The work machine operation system 1 includes an input unit 11, a computer 20, and an output unit 30.

[0016] The client device 10c and the server device 10s are each a computer. The server device 10s operates in response to commands from the client device 10c. The functions of the work machine operation system 1 may be realized by the client device 10c and the server device 10s. All or part of the input unit 11, the calculator 20, and the output unit 30 may be provided in either the client device 10c or the server device 10s, or in both. For example, the memory unit 20a and the calculation unit 20b of the calculator 20, which will be described later, may be provided in either the client device 10c or the server device 10s, or in both. Only one client device 10c and one server device 10s may be provided, or multiple client devices 10c and multiple server devices 10s may be provided. The client device 10c and the server device 10s may be connected via wireless communication or wired communication. Specifically, information is exchanged between the client device 10c and the server device 10s via communication means such as a mobile phone line, an optical fiber line, a wireless LAN (Local Area Network), or a wired LAN. It should be noted that the work machine operation system 1 does not necessarily have to include the server device 10s.

[0017] The input unit 11 is a device for inputting information to the calculator 20. The input unit 11 is a device (operation device) operated by a user. The input unit 11 may be used to operate a GUI (Graphical User Interface) that allows a user to perform operations in the work machine operation system 1. The input unit 11 may include one or more devices selected from a mouse, a keyboard, a touch panel, and a game pad. The input unit 11 may include a device that inputs information without being operated by a user. The input unit 11 includes an operation input unit 11a.

[0018] The operation input unit 11a is used to operate the work machine 60 (described later). For example, the operation input unit 11a may be used to travel the work machine 60, to rotate the upper rotating body 62 (see FIG. 2), and to operate the work implement 64 (see FIG. 2). The operation input unit 11a includes a device that simulates a device (cockpit) for operating the work machine that is located in the driver's cab of the actual work machine. The operation input unit 11a includes real (physical) levers, pedals, buttons, etc. that correspond to the operation device 63 (see FIG. 7, described later) displayed on the display unit 31. The operation input unit 11a may also be part of a remote operation device for remotely operating the actual work machine (described later).

[0019] The computer 20 performs signal input / output, calculations, information storage, etc. If the work space is a virtual space, the computer 20 simulates the situation in the virtual space. The computer 20 simulates the situation (movement, etc.) of the work machine 60 in the virtual space. For example, the computer 20 simulates the operation of the work machine 60 in the virtual space in response to input from the operation input unit 11a. The computer 20 may also simulate situations (movement, etc.) other than that of the work machine 60 in the virtual space. Each function of the computer 20 is realized by the calculation unit 20b executing a program stored in the storage unit 20a. The computer 20 and other devices (e.g., the input unit 11, the output unit 30, etc.) may be connected by wireless communication or by wired communication (a specific example of communication is the same as the communication between the client device 10c and the server device 10s). Information output by the input unit 11 is input to the computer 20. The computer 20 outputs commands to the output unit 30. The computer 20 includes a storage unit 20a and a calculation unit 20b. Focusing on the functions of the computer 20, the computer 20 includes a recording unit 21 and a playback unit 23.

[0020] The memory unit 20a stores various types of information. The memory unit 20a stores programs to be executed by the calculator 20. For example, the memory unit 20a stores a work machine operation program for operating the work machine operation system 1. The memory unit 20a stores information recorded by the recording unit 21. The calculation unit 20b performs calculations (processing such as calculations and judgments).

[0021] The recording unit 21 records the situation within the work space. For example, the recording unit 21 records the operation (first operation) of the work machine 60 in response to the operation of the operation input unit 11a (details will be described later).

[0022] The playback unit 23 plays back the situation in the workspace recorded in the recording unit 21. For example, the playback unit 23 plays back the first action (playback action M1) recorded in the recording unit 21 (details will be described later).

[0023] The output unit 30 is capable of outputting information. The output unit 30 performs output in response to a command input from the computer 20. The output unit 30 outputs the situation in the workspace. The output unit 30 includes a display unit 31 and an audio output unit 33.

[0024] The display unit 31 performs display in response to commands input from the computer 20. The display unit 31 displays the calculation results of the computer 20. The display unit 31 displays an image showing the situation in the work space (details will be described later). Specifically, the display unit 31 is a monitor (screen). The display unit 31 may also be a monitor of a remote control device (described later).

[0025] The audio output unit 33 outputs a sound (output sound) in response to a command input from the computer 20. Specifically, the audio output unit 33 is a speaker. The audio output unit 33 may be a speaker of a remote control device.

[0026] (remote control device) The work machine operation system 1 may include a remote control device (a cockpit for remote operation) for remotely operating a real work machine. The work machine operation system 1 may be implemented in the remote control device. The operation input unit 11a may include devices (levers, pedals, buttons, etc.) for inputting to the remote control device. If the operation input unit 11a is used to operate the work machine 60 in a virtual space and is configured to be able to remotely operate a real machine, the user can experience an operation sensation similar to that of operating a real machine by operating the work machine 60 in the virtual space. As a result, for example, the user's mastery of operating the remote control device is promoted. The computer 20 may be part of the remote control device. In this case, during remote operation, the computer 20 exchanges signals with the real machine that is the target of remote operation via communication means. During remote operation, the computer 20 transmits a signal (operation command signal) to operate the real machine based on an operation signal input from the operation input unit 11a. During remote control, the computer 20 receives signals output by sensors mounted on the actual device (e.g., image data from a camera mounted on the actual device, audio data from a microphone mounted on the actual device, etc.) via communication means. The output unit 30 may include a device for outputting the remote control device. For example, if the display unit 31 is a monitor of the remote control device, the display unit 31 displays image data from a camera mounted on the actual device during remote control. For example, if the audio output unit 33 is a speaker of the remote control device, the audio output unit 33 outputs audio data from a microphone mounted on the actual device during remote control.

[0027] When the work machine operation system 1 is configured to be able to operate a work machine 60 in a virtual space (having a normal operation mode described later) and the output unit 30 is equipped with a device for outputting from a remote control device, the following effects can be obtained. In this case, the user can operate the work machine 60 in the virtual space while perceiving the same output (display, sound) from the output unit 30 as that perceived (seen, heard) when remotely operating a real machine. Also, in this case, the output unit 30 of the work machine operation system 1 can also be used as a device for outputting from the remote control device. Therefore, there is no need to use a device (such as a monitor) separate from the device for outputting from the remote control device as the output unit 30 of the work machine operation system 1. When the work machine operation system 1 is configured to be able to operate a work machine 60 in a virtual space (having a normal operation mode described later) and both the operation input unit 11a and the output unit 30 are also used when remotely operating a real machine, the following effects can be obtained. In this case, the user can experience a sensation closer to that of operating the actual machine, compared to when only one of the operation input unit 11a and the output unit 30 is used during remote operation of the actual machine.

[0028] (Activated) The work machine operation system 1 (mainly the computer 20) is configured to perform the following operations. Below, the input unit 11, computer 20, output unit 30, and their components (e.g., recording unit 21, playback unit 23, etc.) will be described with reference to Fig. 1. The computer 20 displays an object 50 on the display unit 31, as shown in Fig. 2.

[0029] The object 50 is an object within the work space and is an object displayed on the display unit 31. The object 50 is an object for which the situation is recorded by the recording unit 21 and an object for which the situation is played back by the playback unit 23. When the work space is a real space, the object 50 is an object in the real space. When the work space is a virtual space, the object 50 (object, model) is a simulation (model) of an object in a real work site. In this case, the object 50 is a two-dimensional model or a three-dimensional model. Only one object 50 may be set, or multiple objects may be set. There may be one type of object 50, or multiple types. Only one object 50 of a certain type may be set, or multiple objects may be set. For example, the object 50 includes a work machine 60 and a peripheral object 80.

[0030] The work machine 60 is a machine that performs work (e.g., heavy machinery). The work machine 60 may be a construction machine that performs construction work, or a loading and unloading machine that performs loading and unloading work. The work machine 60 may be a shovel, a crane, a bulldozer, or a wheel loader. The following mainly describes a case where the work machine 60 is a shovel. The work machine 60 operates within the work space (specific examples of operation will be described later). The work machine 60 operates in response to operation (manual operation) of the operation input unit 11a by the user (see the normal operation mode described later). The work machine 60 may sometimes operate based on information that is set in advance in the calculator 20 (before the work machine 60 operates). A plurality of work machines 60 may be provided within the work space. A work machine 60 to be operated (for example, manually operated by the user) may be selected from the plurality of work machines 60. The work machine 60 comprises a machine main body 60a, an operation device 63 (see FIG. 7), and a work device 64.

[0031] The machine body 60a is the main body portion of the work machine 60. The machine body 60a includes a lower body 61 and an upper rotating body 62.

[0032] The lower body 61 supports the upper revolving body 62 from below. The lower body 61 is a lower running body that can run on a running surface (for example, the ground). Note that the lower body 61 does not have to be able to run. Below, an example will be described in which the lower body 61 is able to run (when it is a lower running body).

[0033] The upper rotating body 62 is rotatable relative to the lower body 61. The upper rotating body 62 includes a driver's cab 62c.

[0034] The operator's cab 62c (cab) is a portion where an operator operates a real work machine when the operator boards the work machine and operates it. The operator's cab 62c may be fixed to the bottom (swivel frame) of the upper rotating body 62. The operator's cab 62c may be movable relative to the bottom of the upper rotating body 62. In this case, the operator's cab 62c may be vertically movable in parallel with the bottom of the upper rotating body 62, may be vertically rotatable (rotating), or may be vertically movable in parallel with and rotatable. For example, the operator's cab 62c may be an elevator cab, a link cab, or a tilt cab. The operator's cab 62c, which is an elevator cab, is vertically movable relative to the bottom of the upper rotating body 62. The operator's cab 62c, which is a link cab, is installed at the bottom of the upper rotating body 62 via a link mechanism. By operating the link mechanism, the operator's cab 62c, which is a link cab, can move up and down and back and forth relative to the bottom of the upper rotating body 62. The operator's cab 62c, which is a tilt cab, can rotate (tilt) up and down relative to the bottom of the upper rotating body 62 about a rotation axis extending laterally.

[0035] As shown in Figure 7, the operating device 63 is a device for operating the work machine 60. The operating device 63 is arranged inside the driver's cab 62c. The operating device 63 includes, for example, levers, pedals, buttons (including switches), etc. Specifically, the operating device 63 includes an operating lever 63a and a travel operating unit 63b. The operating device 63 may include devices other than the operating lever 63a and the travel operating unit 63b (see "Record Items" below for specific examples).

[0036] The operating lever 63a is a device (lever) for operating the attachment 65 shown in FIG. 2 and for operating the rotation of the upper rotating body 62 relative to the lower main body 61. As shown in FIG. 4, the operating lever 63a is arranged on the left and right sides inside the cab 62c. The terms "front," "rear," "left," and "right" in relation to the operating device 63 refer to directions as seen from the perspective of an operator (a virtual operator) sitting in the cab 62c. The same applies hereinafter. The operating lever 63a includes a left operating lever 63al and a right operating lever 63ar. The left operating lever 63al is arranged on the left side inside the cab 62c. The left operating lever 63al is operated forward, backward, left, and right (moves forward, backward, left, and right) (similar to the right operating lever 63ar). The right operating lever 63ar is arranged on the right side inside the cab 62c.

[0037] A lever pattern is set which is the correspondence between the operation of this control lever 63a and the operation of the work machine 60. The lever pattern of the control lever 63a can be set as desired. The operation of the work machine 60 operated by the control lever 63a includes the rotation (right turn, left turn) of the upper rotating body 62 relative to the lower main body 61 shown in FIG. 2. In addition, the operation of the work machine 60 operated by the control lever 63a (see FIG. 7) includes the rotation of the boom 65a relative to the upper rotating body 62 (boom up, boom down) and the rotation of the arm 65b relative to the boom 65a (arm push, arm pull). In addition, the operation of the work machine 60 operated by the control lever 63a includes the rotation of the tip attachment 65c relative to the arm 65b (for example, bucket digging, bucket release). An example of a lever pattern (ISO pattern) is shown in FIG. 11 as an operation indicator G31c (described below). In FIG. 11, the left operating lever 63al is referred to as the "left lever," and the right operating lever 63ar is referred to as the "right lever."

[0038] The travel operation unit 63b (see FIG. 4) is a device for operating the travel of the work machine 60 (of the lower body 61) shown in FIG. 2. As shown in FIG. 4, the travel operation unit 63b is located in the front portion of the driver's cab 62c. The travel operation unit 63b may be a lever, a pedal, or a lever and a pedal. The travel operation unit 63b includes a left travel operation unit 63bl and a right travel operation unit 63br. The left travel operation unit 63bl is a device for operating one portion of the lower body 61 in the width direction (the left traveling body, for example, the left crawler). The left travel operation unit 63bl controls the forward and reverse movement of the left traveling body. The right travel operation unit 63br is a device for operating the other portion of the lower body 61 in the width direction (the opposite side to the above-mentioned "one portion") (the right traveling body, for example, the right crawler). The right travel operation unit 63br controls the forward and reverse movement of the right traveling body.

[0039] The working device 64 is a device that performs work, as shown in Fig. 2. The working device 64 performs work (for example, movement) on a work target 83. As shown in Fig. 6, the working device 64 includes a dozer 64d and an attachment 65.

[0040] The dozer 64d is a work device 64 attached to the lower body 61. The dozer 64d includes, for example, a plate-shaped member (e.g., a blade) extending in the width direction and up and down direction of the lower body 61. The dozer 64d is movable up and down relative to the lower body 61.

[0041] The attachment 65 is a work device 64 attached to the upper rotating body 62. For example, the attachment 65 includes a boom 65a, an arm 65b, and a tip attachment 65c. The boom 65a is capable of raising and lowering (rotating up and down) relative to the upper rotating body 62. The arm 65b is rotatable relative to the boom 65a. The tip attachment 65c is rotatable relative to the arm 65b. The tip attachment 65c is provided at the tip of the attachment 65. As shown in FIG. 2, the tip attachment 65c is capable of capturing and releasing a work object 83. The tip attachment 65c may include a bucket capable of performing operations such as scooping and excavating the work object 83. The tip attachment 65c may include a magnet that attracts a magnetic work object 83, such as a metal work object 83. The tip attachment 65c may be equipped with a device for clamping the work piece 83 (such as a grapple, nibbler, or rotary fork), or may be equipped with a device for crushing the work piece 83 (such as a breaker). The device for clamping the work piece 83 (such as a rotary fork) may be configured to rotate relative to the base end (the part on the arm 65b side) of the tip attachment 65c, so that the opening and closing direction of the clamping device (the direction in which the work piece 83 is clamped) can be changed in various directions. The attachment 65 may be equipped with a hook for lifting a load. This hook is provided, for example, at the tip of the arm 65b.

[0042] The surrounding objects 80 are objects around the work machine 60. The surrounding objects 80 include, for example, the ground 81, a work target object 83, a vehicle 85, and an obstacle 87.

[0043] The ground 81 is the land of the work site. The ground 81 may include, for example, flat ground, a slope, unevenness, holes, etc.

[0044] The work object 83 is an object that is the target of work by the work machine 60. The work object 83 may be soil (soil, sand, gravel, etc.), rock, an object attracted by a magnet (a magnetic material such as metal, for example, scrap material), resin, waste material, wood (logs, etc.), or a structure (blocks, etc.). The work object 83 may be a load lifted by the work machine 60. The work object 83 may be placed on the ground (the surface of the ground 81). The work object 83 may be housed in a container, for example, a container placed on the ground, a loading platform 85a of the vehicle 85, or a container (such as a soil pit) provided in a hole formed in the ground 81. If the work object 83 is soil, the work object 83 may be part or all of the ground 81, or a pile of soil and sand piled up on the ground.

[0045] The shape of this work object 83 may be set in various ways. The work object 83 may be a rectangular parallelepiped (including an approximately rectangular parallelepiped). The rectangular work object 83 may be, for example, a structure (such as a block). The rectangular work object 83 may be a simulation of an object attracted by a magnet in virtual space. The work object 83 may be a cylinder (including an approximately cylindrical shape). The cylindrical work object 83 may be, for example, a log or a structure (such as a pillar).

[0046] The vehicle 85 is a machine (transport vehicle) capable of transporting an object. The vehicle 85 is, for example, a dump truck. The vehicle 85 is equipped with a loading platform 85a. The loading platform 85a is a container capable of storing an object (a container, for example, a work object 83) to be transported.

[0047] Obstacle 87 is an object that impedes the work of work machine 60. Obstacle 87 is an object that work machine 60 must avoid. For example, obstacle 87 may be a building or a worker. For example, obstacle 87 may be equipment installed at the work site, and more specifically, may be equipment such as markers, such as pylons (cones) or bars. Depending on the purpose of the work of work machine 60, the ground 81, work object 83, vehicle 85, etc. may also become obstacle 87.

[0048] (Operation mode) The calculator 20 switches between a plurality of operation modes (see steps S10, S21, and S22 in FIG. 3). Each step shown in FIG. 3 will be described below with reference to FIG. 3. The operation mode is the processing state of the calculator 20 and the operating state of the work machine operation system 1. For example, the operation modes include a normal operation mode (steps S10, S60), a recording mode (step S30), and a playback mode (step S40). The operation modes may also include a training mode.

[0049] The computer 20 may switch the operation mode in response to an operation of a GUI. Specifically, the computer 20 may display a GUI (such as a window) for switching the operation mode on the display unit 31 shown in FIG. 1 and switch the operation mode in response to an operation of the GUI by a user. Hereinafter, the computer 20 displaying an image on the display unit 31 may also be referred to as the computer 20 displaying an image, etc. The computer 20 may switch the operation mode in response to an operation of the input unit 11 (for example, the operation input unit 11a). Specifically, the relationship between the operation of the input unit 11 and the operation mode is set (linked, associated, assigned) in advance in the computer 20 (before switching the operation mode). Then, the computer 20 may switch the operation mode when a specific operation for switching the operation mode is performed on the input unit 11. The computer 20 may switch the operation mode in response to a situation in the work space. Specifically, the relationship between the attitude, etc. of the work machine 60 shown in FIG. 2 (an example of a situation in the work space) and the operation mode is set in advance in the computer 20. Then, the calculator 20 may switch the operation mode when the attitude of the work machine 60 becomes a specific attitude that switches the operation mode.

[0050] (Normal operation mode (step S10, step S60)) The normal operation mode is an operation mode in which the user can operate the work machine 60. For example, the normal operation mode may be an operation mode that simulates the operation of a real work machine within a virtual work space. For example, the normal operation mode may be an operation mode in which a real work machine is remotely operated within a work space in real space. For example, in the normal operation mode, when the user operates (manually operates) the operation input unit 11a, the work machine 60 operates in accordance with the user's manual operation. Specifically, one or more of the lower body 61, upper rotating body 62, dozer 64d (see FIG. 6), boom 65a, arm 65b, and tip attachment 65c move in accordance with the user's manual operation.

[0051] In this normal operation mode, the situation of the surrounding objects 80 (the situation at the work site) may change in accordance with the operation (e.g., work) of the work machine 60. For example, the work target object 83 may move in accordance with the operation of the work machine 60.

[0052] (Training mode) The training mode is an operating mode for having the user learn (teach) how to operate the work machine 60 in an order (sequentially) determined by the program. In detail, the training content for the operation method in the training mode is determined in advance (before the training) by the program and is stored in advance in the storage unit 20a.

[0053] (Recording mode (step S30) and playback mode (step S40)) The recording mode (step S30) is an operation mode for recording the situation in the workspace, and the playback mode (step S40) is an operation mode for playing back the recorded situation in the workspace.

[0054] In the recording mode and playback mode, at least the operation of the work machine 60 is recorded and played back. Examples of reasons for recording and playing back the operation of the work machine 60 are as follows: The operation methods (ways of moving) of real work machines (actual machines) vary enormously depending on various circumstances, such as the work site where the actual machine is used, the type of actual machine, the operator who operates the actual machine, the company that uses the actual machine, and the work content of the actual machine. For this reason, it is difficult to determine the training content for all operation methods in advance (before training) in a program.

[0055] Therefore, for example, an experienced operator performs a model (exemplary) operation using the operation input unit 11a, causing the work machine 60 to operate (first operation). The recording unit 21 records this model first operation of the work machine 60. The playback unit 23 then plays back the model first operation of the work machine 60. A user (for example, a new operator or trainee) learning the operation method operates the work machine 60 using the operation input unit 11a while checking the played back first operation. As described above, the work machine operation system 1 is configured to be able to record and play back the first operation of the work machine 60, so it is possible for the user to learn operation methods in a variety of situations. Below, details of the recording and playback of the first operation of the work machine 60, and the display by the display unit 31, will be described.

[0056] (Recording (Step S30)) The recording unit 21 records the situation within the work space. The recording unit 21 may record a portion of the situation within the work space, or may record the entire situation. The recording unit 21 records at least a first action, which is an action of the work machine 60 in response to operation of the operation input unit 11a (manual operation by the user). The recording unit 21 may also record situations within the work space other than the first action.

[0057] The recording unit 21 may record a situation in a workspace that is a virtual space. The recording unit 21 may record a situation in a workspace that is a real space. A specific example of the recording unit 21 recording a situation in a workspace that is a real space is as follows. In this case, various sensors detect the situation (position, speed, attitude, etc.) of an object 50 in the real space. Then, the recording unit 21 records the information detected by the sensor. The sensor may directly detect the position, speed, attitude, etc. of the object 50. The sensor may include an imaging device that captures an image of the object 50. The computer 20 may calculate the situation of the object 50 based on an image captured by the imaging device (for example, by image recognition, clustering, etc.). The captured image may be a two-dimensional image or a three-dimensional image (for example, including point cloud information, etc.). Furthermore, the recording unit 21 may simply record (video) the image captured by the imaging device.

[0058] The computer 20 displays, for example, the recording instruction section G21 shown in Fig. 4. It is to be noted that the computer 20 does not have to display the recording instruction section G21.

[0059] The recording instruction section G21 is a section where the user instructs the state (recording state) such as start and stop (end) of recording. The recording instruction section G21 is a section where the user instructs the timing of start and stop of recording. The recording section 21 may display information related to recording. For example, the recording instruction section G21 is a window (panel, operation recording window) displayed on the display section 31, and is a GUI. The recording instruction section G21 is operated by the input section 11 (for example, a mouse, a touch panel, etc.). The recording instruction section G21 includes a recording file designation section G21a, a recording state designation section G21b, and a recording information display section G21c.

[0060] The recording file designation section G21a is a section for designating a file name (described in detail later). The recording state instruction section G21b is a section for designating a recording state (described in detail later). The recording state instruction section G21b includes a recording start instruction section G21b1, a recording stop instruction section G21b2, and a chapter setting instruction section G21b3 (described in detail later).

[0061] The recording information display section G21c is a section that displays information related to recording. The recording information display section G21c may display time information. For example, the recording information display section G21c may display the current recording position during recording (the elapsed time from the start of recording to the present). The recording information display section G21c may display the recording time (the time from the start of recording to the stop) after recording is stopped. The recording information display section G21c may display the upper limit of the recording time or the target recording time. The recording information display section G21c may display information other than time information, such as characters or figures indicating that recording is in progress.

[0062] (Record file) The recording unit 21 stores the information to be recorded as a recording file in the storage unit 20a.

[0063] The file name of the record file can be set in various ways (step S31). The file name of the record file may be arbitrarily set (specified) by the user, and may be, for example, a file name input in the record file designation section G21a. The file name of the record file may also be automatically set by the computer 20. For example, the file name of the record file may be automatically set by the computer 20 when the record file designation section G21a is blank. The computer 20 may automatically reflect information to be saved as the record file in the file name. For example, the computer 20 may automatically reflect information set in the computer 20 at the time of recording by the storage unit 20a (see specific examples of "record items" described later) in the file name. For example, if the working space is a virtual space, the computer 20 may automatically reflect information set in the computer 20 (information set in the simulator) at the time of recording by the storage unit 20a in the file name.

[0064] The file format of the record file can be set in various ways. The file format of the record file may be a format that allows the contents of the record file to be viewed (opened) on a device external to the work machine operation system 1 shown in FIG. 1. Specifically, the record file may be in text data format or a CSV (Comma Separated Values) file format. The file format of the record file may be a format that allows the contents to be viewed only within the software of the work machine operation system 1.

[0065] The memory unit 20a that stores (saves) the record file may be the memory unit 20a of the client device 10c, or the memory unit 20a of the server device 10s. Multiple record files may be collectively stored in the server device 10s (for example, a cloud server). In particular, multiple record files recorded in one or more work machine operation systems 1 may be collectively stored in the server device 10s. The record files collectively stored in the server device 10s may be used (specifically, played back) in one or more work machine operation systems 1.

[0066] As will be described later, the recording unit 21 may record the situation in the workspace (for example, the first action) by dividing (segmenting) it into a plurality of chapters C (playback areas, see FIG. 13) (see the description of "divided playback" below). When the situation recorded by the recording unit 21 is divided into a plurality of chapters C, a recording file may be stored (generated) for each chapter C, or one recording file may be stored (generated) in which information (chapter information) about the division positions of the chapters C is set.

[0067] As will be described later, the recording unit 21 may record a plurality of first actions. In this case, a plurality of recording files may be stored (generated) for each first action, or a plurality of first actions may be collectively stored (generated) as one recording file.

[0068] As will be described later, the recording unit 21 may record the first actions of multiple work machines 60 (see Figure 2). In this case, a record file may be stored (generated) for each work machine 60 shown in Figure 2, or the first actions of multiple work machines 60 may be collectively stored (generated) as a single record file.

[0069] (Recording items (Step S32)) The items (recording items) (content to be recorded, recording targets) recorded by the recording unit 21 can be set in various ways. The recording items may be selected by the user. For example, the computer 20 displays a GUI (such as a list) for selecting recording items. The recording unit 21 may then record the items selected by the user on the GUI as recording items. The recording items may be set in advance (before recording) in the computer 20, or may be automatically set in the computer 20 in accordance with predetermined conditions. The recording items include, for example, some or all of the information on the recording itself, time information, machine information on the work machine 60, operation information on the work machine 60, work information on the work machine 60, and surrounding situation information.

[0070] (Example of record items: information about the record itself) The recording unit 21 may record information about the recording itself. This "information about the recording itself" may include, for example, the date and time when the recording was performed (for example, the date and time when the recording started, or the date and time when the recording stopped), or may include information about the user who performed the first action to be recorded (such as information indicating who the user is).

[0071] (Example of record items: time information) The recording unit 21 may record time information. The time information may include, for example, information on the time from the start of recording (recording position), or information on the time from the start of recording to the stop (end) of recording (recording time). The time information may include information on the division position of chapter C (see FIG. 13) (chapter information).

[0072] (Example of record items: machine information of work machine 60) The recording unit 21 may record machine information of the work machine 60. This "machine information" is information about the machine of the work machine 60 that is the target of operation by the user (machine to be operated). The machine information may include, for example, information about the type of work machine 60, or may include information about the specifications of the work machine 60. Specifically, the machine information may include all or some of the information about the model, class, type of attachment 65, type of cab 62c, type of operating device 63 (see FIG. 7), and lever pattern of the operating device 63 of the work machine 60.

[0073] (Example of record items: operation information of the work machine 60) The recording unit 21 records operation information of the work machine 60. This "operation information" is information on the operation of the work machine 60. The operation information is chronological information on the situation (state) of the work machine 60. For example, the operation information includes information indicating the relationship between information on the situation of the work machine 60 and the time from the start of recording (recording position). The following specific examples of "operation" of the work machine 60 are specific examples of a first operation and a second operation. Note that a situation in which the work machine 60 is stopped in whole or in part is included in the "operation" of the work machine 60.

[0074] (operation information) The operation information of the work machine 60 may include information on the operation of the operation device 63 shown in Figure 7 (also referred to as operation information). The "operation" of the work machine 60 may include the operation of the operation device 63. The operation information may include information on the operation of the operation device 63 displayed on the display unit 31 in response to the operation of the actual operation input unit 11a. The operation information includes chronological information on the status of the operation device 63. For example, the operation information includes information indicating the relationship between the information on the status of the operation device 63 and the time from the start of recording (recording position).

[0075] The operation information may include information on whether or not the operation device 63 has been operated, for example, information on whether or not a button or the like has been operated, or information on whether a switch has been turned on or off. The operation information may include information on the operation position (amount of operation) of the operation device 63, or information on the operation position of a lever, the operation position of a pedal, etc. Specifically, the operation information may include information on the operation position of the operation lever 63a in the front, rear, left, or right directions, or information on the operation position of the travel operation unit 63b.

[0076] The operation information may include information on the operation of the operating device 63 for operating the optional attachment. The above-mentioned "optional attachment" is, for example, a tip attachment 65c other than a bucket, and specifically, a device for clamping the work object 83 (see FIG. 2), a magnet for attracting the work object 83 with magnetic force, etc. The operation information may include information on the operation of opening, closing, and rotating the device for clamping the work object 83. The operation information may also include information on the operation (for example, on / off) of the magnet. The operating device 63 for operating the optional attachment may be a pedal (optional pedal), a lever (optional lever), a button, etc.

[0077] The operation information may include information on the operation of the operating device 63 (e.g., a dozer lever) for operating the dozer 64d (see FIG. 6). In the case where the cab 62c is movable with respect to the bottom of the upper rotating body 62 shown in FIG. 2, the operation information may include information on the operation of the operating device 63 for moving the cab 62c with respect to the bottom of the upper rotating body 62. For example, the operation information may include information on the operation of the operating device 63 for changing one or both of the elevation and angle of the cab 62c. The operation information may also include information on the operation (on / off, etc.) of the operating device 63 for changing the status (on, off, etc.) of a light provided to the work machine 60.

[0078] (Operation information other than operation information) The operation information of the work machine 60 may include information about the operation of the work machine 60 other than the operation of the operating device 63 .

[0079] The operation information may include information on the traveling operation of the work machine 60. The "operation" of the work machine 60 may include the traveling operation of the work machine 60. In this case, the operation information includes information on the position (coordinates) of the work machine 60. The operation information may include information on the position (traveling position, machine position) of the travelling work machine 60 at the work site. The information on the position of the work machine 60 is, for example, information on the position of the "reference part" of the work machine 60. The "reference part" of the work machine 60 may be, for example, the position of the center of rotation of the upper rotating body 62 relative to the lower body 61, or the position of the center of gravity of the work machine 60. The operation information may include information on the traveling direction of the work machine 60. For example, the operation information may include information on the orientation (heading) of the lower body 61 relative to the work site.

[0080] The operation information may include information on the swing operation of the upper rotating body 62. The "operation" of the work machine 60 may include the swing operation of the upper rotating body 62. In this case, the operation information may include information on the swing angle of the upper rotating body 62 relative to the lower body 61.

[0081] The operation information may include information about the operation of the work implement 64. The "operation" of the work machine 60 may include the operation of the work implement 64. For example, the operation information may include information about the operation of the attachment 65. In this case, the operation information may include information about the attitude of the attachment 65. Specifically, the operation information may include information about the angle (boom angle) of the boom 65a relative to the upper rotating body 62 or the horizontal direction, or information about the angle (arm angle) of the arm 65b relative to the boom 65a or the horizontal direction. The operation information may include information about the angle (tip attachment angle, for example, bucket angle) of the tip attachment 65c relative to the arm 65b or the horizontal direction. The operation information may include information about the operation of an optional attachment (an example of a tip attachment 65c). For example, if the optional attachment rotates and opens and closes, the operation information may include information about the rotation angle and opening / closing status (opening / closing angle, etc.) of the optional attachment. The operation information may include information about the position (coordinates) of a specific part of the work implement 64. For example, the operation information may include information on the position of the tip of the boom 65a (boom top), or may include information on the position of the tip of the arm 65b (arm top).The operation information may also include information on the position of the tip of the tip attachment 65c (for example, the tip of the bucket).

[0082] (Example of record item: work information of work machine 60) The recording unit 21 may record work information of the work machine 60. This "work information" is information relating to the work that the work machine 60 performs.

[0083] The work information may include information on the type of work performed by the work machine 60. Specifically, the work information may include information on the type of work, such as excavation, loading, leveling, magnetic attraction, and cargo handling.

[0084] The work information may include information on the amount of work (work volume) that the work machine 60 performs on the work object 83. For example, when the work machine 60 excavates earth and sand (an example of the work object 83) with a bucket (an example of the tip attachment 65c), the work information may include information on the amount of soil in the bucket (e.g., mass, volume, etc.), and may also include information on the excavation depth of the ground 81 by the bucket. For example, when the work machine 60 attracts metal (an example of the work object 83) with a magnet (an example of the tip attachment 65c), the work information may include information on the amount of metal attracted by the magnet (magnet attraction amount). For example, when the work machine 60 performs work to grip lumber (an example of the work object 83), the work information may include information on the amount of lumber to be gripped. For example, when the work machine 60 performs work to move a suspended load (an example of the work object 83) (load handling work), the work information may include information on the weight of the suspended load.

[0085] (Example of recording items: Surrounding situation information) The recording unit 21 may record surrounding situation information. This "surrounding situation information" is information about the situation around the work machine 60 (surrounding situation, surrounding environment), for example, information about the situation of surrounding objects 80. The surrounding situation information is information about the situation around the work machine 60 when the work machine 60 is operating.

[0086] The surrounding situation information may include information on the position of the peripheral object 80. The information on the position of the peripheral object 80 may include information on the position (coordinates) at the work site, or may include information on the relative position of the peripheral object 80 with respect to the work machine 60. The surrounding situation information may include information on the distance from the work machine 60 to the peripheral object 80, etc. The surrounding situation information may include information on the shape of the peripheral object 80. The surrounding situation information may include information on the operation (time series) of the peripheral object 80.

[0087] For example, the surrounding situation information may include information on the condition (e.g., shape) of the ground 81. The surrounding situation information may include information on the condition of the work object 83, and may include, for example, information on the position and shape (such as the soil surface shape) of the work object 83. The surrounding situation information may include information on the condition of a machine other than the work machine 60 performing the first operation recorded in the recording unit 21. The above-mentioned "other machine" may include a machine similar to the work machine 60 (such as an excavator), or may include a vehicle 85. The surrounding situation information may include the condition of an obstacle 87, and may include, for example, information such as the position of a person.

[0088] The surrounding situation information may include the surrounding situation at the start of the first operation (the recording unit 21 may record the surrounding situation at the start of the first operation). This "surrounding situation at the start of the first operation" is the situation around the work machine 60 at the start of recording the first operation of the work machine 60.

[0089] The surrounding condition information may include information about the surrounding conditions of the work machine 60 after the start of recording of the first operation of the work machine 60. For example, the surrounding condition information may include chronological information about the surrounding conditions of the work machine 60 from the start to the end of recording of the first operation.

[0090] (Examples of record items: operation information of the work machine 60 and information on the surrounding conditions) The recording unit 21 may record scene information (for example, simulator scene information). This "scene information" is information that includes operation information of the work machine 60 and information about the surrounding situation. The scene information is information about the scene (scene, situation) at the work site. Specific examples of the "scene" include a scene in which the work machine 60 excavates earth and sand with a bucket, a scene in which the work machine 60 handles scrap material with a magnet, a scene in which the work machine 60 moves a load suspended by a hook (cargo handling work), and the like.

[0091] (Instruction of recording status (steps S33, S34, S35)) The recording unit 21 changes the recording state in response to the recording state instruction. The recording state instruction is an instruction for the recording state of the recording unit 21, and specifically includes starting recording (step S33), setting a chapter (step S34), and stopping recording (step S35). The recording unit 21 starts recording in response to the instruction to start recording, stops recording in response to the instruction to stop recording, and sets chapter C (see FIG. 13) in response to the instruction to set a chapter. The recording state instruction may be given by the user or at a timing of the user's choosing. The recording state instruction may also be given automatically by the recording unit 21 (computer 20). Specific examples of recording state instructions are as follows:

[0092] The recording state may be instructed by the user operating the recording instruction unit G21 shown in FIG. 4. For example, when the recording start instruction unit G21b1 (start button) is selected, the recording unit 21 starts recording (step S33). This "selection" of the start button refers to selecting the start button by, for example, a mouse operation, a touch operation, a keyboard key operation, or the like, specifically, pressing the button (the same applies to other recording state instructions). When the recording stop instruction unit G21b2 (stop button) is selected, the recording unit 21 stops (ends) recording (step S35) and saves the recorded information as a recording file. When the chapter setting instruction unit G21b3 is selected, the recording unit 21 divides the recording situation into multiple chapters C (see FIG. 13) during recording (described later) (step S34). Note that the chapter setting instruction unit G21b3 may be included in the playback instruction unit G23 (see FIG. 5).

[0093] The user may instruct the recording state (directly) by operating the input unit 11 (for example, the operation input unit 11a) without operating the recording instruction unit G21. That is, the input unit 11 (for example, the operation input unit 11a) may be configured to be able to directly instruct the recording state. Specifically, the relationship between the operation of the input unit 11 (for example, the operation input unit 11a) and the instruction of the recording state is set (linked, associated, assigned) in advance (before recording) in the recording unit 21. Then, the recording unit 21 may change the recording state when a specific operation for instructing the recording state is performed on the input unit 11.

[0094] The user who instructs the recording state may be the user who operates the work machine 60 (see Figure 2) that performs the first operation to be recorded using the operation input unit 11a, or may be someone other than this user (for example, someone around the user who operates the operation input unit 11a).

[0095] The instruction of the recording state may be automatically performed by the recording unit 21 (computer 20). The recording unit 21 may automatically change the recording state when a predetermined condition (recording state instruction condition) is satisfied. For example, the recording unit 21 may automatically change the recording state when the situation (posture, etc.) of the work machine 60 shown in FIG. 2 becomes a specific situation. Specifically, the relationship between the situation of the work machine 60 and the instruction of the recording state is set (linked, associated) in advance (before recording) in the recording unit 21. Then, the recording unit 21 may automatically change the recording state when the situation of the work machine 60 becomes a specific situation for instructing the recording state. Furthermore, the recording unit 21 may automatically change the recording state when the situation around the work machine 60 becomes a specific situation.

[0096] (Playback (Step S40)) The playback unit 23 plays back the situation in the workspace recorded in the recording unit 21. Specifically, the playback unit 23 makes the situation in the workspace the same as the situation recorded by the recording unit 21. The playback unit 23 may play back only a part of the situation recorded by the recording unit 21, or may play back the entire situation.

[0097] The playback unit 23 plays back at least the first action of the work machine 60 recorded in the recording unit 21. The first action of the work machine 60 played back by the playback unit 23 is also referred to as played back action M1. The playback unit 23 sets the first action played back by the playback unit 23 to be the same as part or all of the first action of the work machine 60 recorded in the recording unit 21. The playback unit 23 may also play back situations in the work space other than played back action M1.

[0098] The playback unit 23 may play back the status of the virtual workspace recorded in the recording unit 21 as a virtual workspace. The playback unit 23 may play back the status of the real workspace recorded in the recording unit 21 as a virtual workspace (for example, by model display, which will be described later). In particular, the computer 20 may convert the status of the real workspace recorded in the recording unit 21 into the status of the virtual workspace, and the playback unit 23 may play back the status of the virtual workspace after the conversion. The playback unit 23 may play back values ​​indicating the status of the real workspace recorded in the recording unit 21 as a graph G31a, which will be described later. The playback unit 23 may simply play back (for example, display as a video) all or part of the captured image of the real workspace recorded in the recording unit 21.

[0099] Specifically, the playback unit 23 reads the information recorded in the recording unit 21. The playback unit 23 sets the situation in the workspace to be played back in accordance with the read information. Then, the playback unit 23 (computer 20) causes the display unit 31 to display the situation in the workspace. For example, when the playback unit 23 plays back a virtual workspace, the playback unit 23 sets the situation in the virtual space in accordance with the read information and moves the object 50 in the virtual space.

[0100] The computer 20 displays, for example, the reproduction instruction section G23 shown in Fig. 5. It should be noted that the computer 20 does not have to display the reproduction instruction section G23.

[0101] The playback instruction unit G23 is a unit where the user instructs the state (playback state) of playback, such as start and stop. The playback instruction unit G23 is a unit where the user instructs the timing of playback, such as start and stop. The playback unit 23 may display information related to playback. For example, the playback instruction unit G23 is a window (panel, action playback window) displayed on the display unit 31, and is a GUI. The playback instruction unit G23 is operated by the input unit 11 (for example, a mouse, a touch panel, etc.). The playback instruction unit G23 includes a playback file designation unit G23a, a playback state instruction unit G23b, a playback method instruction unit G23c, and a playback information display unit G23d.

[0102] The playback file designation section G23a is a section for designating a file to be played (playback file) (details will be described later). The playback state designation section G23b is a section for designating a playback state (details will be described later). The playback state designation section G23b includes a playback speed designation section G23b1, a playback position designation section G23b2, a playback start designation section G23b3, a playback pause designation section G23b4, and a playback stop designation section G23b5 (each of which will be described later). The playback method designation section G23c is a section for designating a playback method (repeat playback, etc.) (details will be described later).

[0103] The playback information display section G23d is a section that displays information related to playback. The playback information display section G23d may display time information. For example, the playback information display section G23d may display the current playback position (the time from the playback start position (0 seconds) to the present). The playback information display section G23d may display the playback end time (the time from the playback start position (0 seconds) to the end position). The playback information display section G23d may display information other than time information, such as characters or figures indicating that playback is in progress.

[0104] The playback instruction unit G23 may be able to select a chapter C (see FIG. 13) to be played back (chapter C will be described later) and may be able to select a playback method for chapter C (such as repeat playback). The playback instruction unit G23 may include a section (chapter setting instruction unit G21b3 (see FIG. 4)) that instructs the playback unit G23 to divide the situation in the workspace into chapters C. The playback instruction unit G23 may be able to select various display methods, which will be described later. The various display methods, which will be described later, may be selected by a GUI, or the like, that is separate from the playback instruction unit G23 and that is used to select a display method.

[0105] (Regarding playback files, etc.) The playback unit 23 may play back the situation in the workspace that was recorded in the recording unit 21 and saved as a recording file in the storage unit 20a (step S41). In this case, the recording file that the playback unit 23 reads from the storage unit 20a is referred to as a "playback file." The playback unit 23 may play back the situation in the workspace that was recorded in the recording unit 21 but not saved as a recording file. For example, the playback unit 23 may play back the situation in the workspace that was recorded immediately before playback and that was temporarily stored in the storage unit 20a. The following mainly describes the case where the playback unit 23 plays back the situation in the workspace that was saved as a recording file in the storage unit 20a.

[0106] The playback file may be arbitrarily selected (specified) by the user. The playback file may be selected by the playback file designation unit G23a. For example, the playback file designation unit G23a may display a list of playback file candidates in a specific folder. The playback file designation unit G23a may be an input box into which the user can input the file name of the playback file by operating the input unit 11 (for example, by key input). The playback file may also be automatically selected by the computer 20 according to predetermined conditions.

[0107] (Playback item (step S42)) The items (playback items) (contents to be played, playback targets) played back by the playback unit 23 can be set in various ways. Like the recorded items, the playback items may be selected by the user, may be set in advance (before playback) in the computer 20, or may be automatically set (selected) by the computer 20 according to predetermined conditions. The playback items may be all of the recorded items or only some of them (the playback unit 23 may play back all or only some of the situations recorded in the recording unit 21).

[0108] Based on the machine information of the work machine 60 shown in Figure 2 (details described above) recorded in the recording unit 21, the playback unit 23 may set the model, specifications, etc. of the work machine 60 to be played back to be the same as the model, specifications, etc. of the recorded work machine 60.

[0109] Based on the operation information of the work machine 60 recorded in the recording unit 21 (details described above), the playback unit 23 makes the first operation of the work machine 60 to be played back (played operation M1) the same as the recorded first operation of the work machine 60. For example, based on the operation information of the operation device 63 recorded in the recording unit 21, the playback unit 23 may make the operation of the operation device 63 to be played back the same as the recorded operation of the operation device 63. For example, the playback unit 23 may make the traveling operation, turning operation, operation of the work device 64, etc. of the work machine 60 recorded in the recording unit 21 the same as the recorded traveling operation, turning operation, operation of the work device 64, etc. of the work machine 60.

[0110] Based on the work information of the work machine 60 recorded in the recording unit 21 (details described above), the playback unit 23 may set the content of work, amount of work, etc. of the work machine 60 to be played back to be the same as the content of work, amount of work, etc. of the work machine 60 recorded.

[0111] The playback unit 23 may set the surrounding conditions of the work machine 60 to be played back to the same conditions as the recorded surrounding conditions of the work machine 60, based on the surrounding condition information recorded in the recording unit 21 (details described above). In this case, the user can learn to operate the work machine 60 in accordance with the surrounding conditions of the work machine 60. For example, the playback unit 23 may set the surrounding conditions of the work machine 60 to the recorded surrounding conditions at the start of the first operation (details described above) at the start of playback of the playback operation M1 (step S51). Furthermore, for example, the playback unit 23 may set the surrounding conditions of the work machine 60 after the start of playback of the playback operation M1 to the same conditions as the surrounding conditions of the work machine 60 after the start of recording.

[0112] (Selection of playback method (step S43)) The playback unit 23 may play back the situation in the workspace using various playback methods. The playback method may be set (specified) by the user, or may be set, for example, in response to the user's operation of the playback method instruction unit G23c (see FIG. 5). In the example shown in FIG. 5, the playback method (repeat playback in FIG. 5) is specified in response to whether or not a check box is checked. The playback method may be set in advance (before playback) in the computer 20, or may be automatically set in the computer 20 in response to predetermined conditions.

[0113] Specific examples of playback methods are as follows: The playback method may include repeat playback, which is a playback method that repeatedly plays back the situation in the workspace within a specific time range (repeat playback range). The repeat playback range may be the range from the start position (0 seconds) to the end position of playback, the time range of chapter C (see FIG. 13), or a time range arbitrarily specified by the user. As will be described later, the content played back by the playback unit 23 can be displayed in various display methods, and each of the various display methods of the content played back by the playback unit 23 is also a playback method.

[0114] (Change of playback speed (step S44)) The playback speed of the situation in the workspace (e.g., playback action M1) played back by the playback unit 23 may be variable. In this case, the playback speed may be arbitrarily set (instructed) by the user, for example, by the playback speed instruction unit G23b1 (see FIG. 5). For example, when the speed of the situation recorded in the recording unit 21 is set to "1x," the playback speed may be 1x or may not be 1x. The range of the playback speed can be set in various ways, and in the example shown in FIG. 5, is equal to or greater than 0x and equal to or less than 3x. The lower limit of the playback speed may be less than 0x (reverse playback). The upper limit of the playback speed may be greater than 3x.

[0115] The playback speed may be set to 0x. In other words, the playback unit 23 may stop the situation in the work space being played back. When the playback unit 23 stops the playback operation M1 of the work machine 60 shown in Figure 2, it is possible to allow the user to check the situation of the work machine 60 (posture, operation status, etc.) while the work machine 60 is stopped.

[0116] The playback speed may be set to a speed greater than 0x and less than 1x. In other words, the playback unit 23 may play back the situation within the workspace at a slower speed than the situation within the workspace recorded in the recording unit 21. If the playback unit 23 plays back the played-back action M1 of the work machine 60 at a speed greater than 0x and less than 1x, the user can easily confirm the played-back action M1. For example, by playing back fast actions of the work machine 60 operated by an experienced operator at a slow speed, a novice operator (trainee) can easily imitate the actions of the work machine 60 operated by the experienced operator.

[0117] The playback speed may be set to a speed greater than 1x. That is, the playback unit 23 may play back the situation within the work space at a speed faster than the situation within the work space recorded in the recording unit 21. In this case, the user can check the content played back by the playback unit 23 in a short period of time. Furthermore, when the user operates the operation input unit 11a to operate the work machine 60 so as to imitate the played back action M1 (when practicing an operation), if the playback speed is set to a speed greater than 1x, the user can learn a faster operation. For example, it may be possible to increase the difficulty of the training.

[0118] (Playback state instruction (step S53, step S55, etc.)) The playback unit 23 changes the playback state in response to the playback state instruction. The playback state instruction is an instruction regarding the playback state of the playback unit 23, and specifically, is to start playback (step S53), stop playback (step S55), etc. The playback unit 23 starts playback in response to the playback start instruction, and stops playback in response to the playback stop instruction.

[0119] As with the specific example of the instruction for the recording state, the instruction for the playback state may be given by the user or at a timing of the user's choosing. Also, as with the specific example of the instruction for the recording state, the instruction for the playback state may be given automatically by the playback unit 23 (computer 20). Below, the specific example of the instruction for the playback state will be described, focusing mainly on the differences from the specific example of the instruction for the recording state.

[0120] The playback state may be instructed by the user operating the playback instruction unit G23 shown in FIG. 5. When the playback start instruction unit G23b3 (start button) is selected, the playback unit 23 starts playback (step S53). This "selection" of the start button refers to, for example, operating the start button by a mouse operation, a touch operation, a key operation, or the like, and specifically, pressing the button (the same applies to other playback state instructions). When the playback pause instruction unit G23b4 (pause button) is selected during playback, the playback unit 23 pauses playback and holds the playback position. Furthermore, when the playback pause instruction unit G23b4 (playback resume instruction unit, playback resume button) is selected while playback is paused, the playback unit 23 resumes playback. When the playback stop instruction unit G23b5 (stop button) is selected, the playback unit 23 stops playback (step S55). In this case, the playback unit 23 may stop playback with the playback position returned to the playback start position (see FIG. 13). When the playback position reaches the playback end position (see FIG. 13), the playback unit 23 may automatically stop the playback.

[0121] The playback position may be specified by the playback position specification section G23b2. The playback position specification section G23b2 is a section where the user specifies (specifies, seeks) the playback position. The playback position specification section G23b2 may include a graphic that displays the current playback position in the time from the playback start position (see FIG. 13) to the playback end position (see FIG. 13). Specifically, the playback position specification section G23b2 may be in the shape of a bar (seek bar). The playback unit 23 may change the playback position in response to an operation of the seek bar. Note that the playback unit 23 may change the playback position in response to an instruction different from that of the playback position specification section G23b2. For example, the playback position may be changed in response to an operation of the input unit 11 by the user (e.g., a numerical input, etc.).

[0122] The playback state may be instructed (directly) by the user operating the input unit 11 (for example, the operation input unit 11a) without operating the playback instruction unit G23. That is, the input unit 11 (for example, the operation input unit 11a) may be configured to be able to directly instruct the playback state. Specifically, the relationship between the operation of the input unit 11 (for example, the operation input unit 11a) and the instruction of the playback state is set (linked, associated, assigned) in advance (before playback) in the playback unit 23. Then, the playback unit 23 may change the playback state when a specific operation for instructing the playback state is performed on the input unit 11. The user instructing the playback state may be the user operating the work machine 60 while checking the situation to be played back (the user operating the operation input unit 11a), or may be someone different from this user.

[0123] The instruction of the regeneration state may be automatically given by the regeneration unit 23 (computer 20). For example, the computer 20 may automatically change the regeneration state when one or both of the state of the work machine 60 and the situation around the work machine 60 become specific situations.

[0124] (The timing of the instruction to start playback and the timing of the actual start of playback) The playback unit 23 may start playback when an instruction to start playback is given and, further, when a predetermined condition (playback start condition) is satisfied. An example of the reason for this is as follows. If an instruction to start playback (for example, an instruction in the playback start instruction unit G23b3) is given and playback starts immediately after the instruction, there is a possibility that the user operating the operation input unit 11a is not ready. Therefore, even if an instruction to start playback is given, the playback unit 23 does not start playback until the playback start condition is satisfied, and starts playback when the playback start condition is satisfied. The playback start condition can be set in various ways. The playback start condition may include a specific operation being performed in the operation input unit 11a. The playback start condition may include the work machine 60 shown in FIG. 2 assuming a specific posture (for example, a posture ready to start work). The playback start condition may include the elapse of a predetermined time from when the instruction to start playback was given.

[0125] (Various examples of output) As described above, the calculator 20 causes the display unit 31 to display an image (the calculator 20 displays an image). The calculator 20 may also cause the audio output unit 33 (see FIG. 1) to output a sound (described later).

[0126] The computer 20 displays an image representing the situation within the work space. The computer 20 displays an image of the object 50. The computer 20 displays an image (video) showing the movement of the object 50 when the object 50 moves within the work space. The computer 20 displays the work machine 60. The computer 20 may also display surrounding objects 80 (surrounding situation). The computer 20 may also display an image different from the situation within the work space. For example, the computer 20 may display a GUI.

[0127] The items (display items) (contents to be displayed, display targets) that the computer 20 causes to be displayed on the display unit 31 can be set in various ways, similar to the record items and playback items. The display items may be selected by the user, may be set in advance (before display) in the computer 20, or may be automatically set in the computer 20 according to predetermined conditions. Specific examples of display items are the same as the record items and playback items (however, information that cannot be displayed, such as information on the recording itself, is excluded).

[0128] The calculator 20 may perform display using various display methods, such as a model display, display of a graph G31a (see FIG. 10, etc.), or display of an operation indicator G31c (see FIG. 11, etc.), which will be described later. The display method may be selected by the user, may be set in the calculator 20 in advance (before display), or may be automatically set in the calculator 20 according to predetermined conditions. The calculator 20 may perform display using only one display method, or may combine (simultaneously) display using a plurality of display methods. The display method may be set (individually) for each display item. The display items may be set (individually) for each display method.

[0129] (Operation display of work machine 60) The calculator 20 displays the operation of the work machine 60. The calculator 20 displays a first operation (reproduced operation M1) that is a first operation of the work machine 60 recorded in the recording unit 21 and that is reproduced by the reproduction unit 23. The calculator 20 displays a second operation (also referred to as a current operation M2), which is the operation of the work machine 60 in accordance with the operation being performed on the operation input unit 11a (the operation currently being performed by the user). The operation of the work machine 60 (either or both of the reproduced operation M1 and the current operation M2) displayed by the calculator 20 includes the operation of the work machine 60 itself (traveling operation, turning operation, change in posture, etc.) and the operation (operation) of the operation device 63 (see Figure 7).

[0130] In some cases, the computer 20 may display the playback action M1 but not the current action M2 (see preview display, described below). In other cases, the computer 20 may display the current action M2 but not the playback action M1 (see normal operation mode, above).

[0131] (Simultaneous display of playback action M1 and current action M2) The calculator 20 may simultaneously display the reproduced action M1 and the current action M2. For example, the calculator 20 simultaneously displays the reproduced action M1, which is the action of the work machine 60 operated by an experienced operator, and the current action M2, which is the action of the work machine 60 operated by a novice operator (trainee) currently using the operation input unit 11a.

[0132] (Various combinations of the operation of the work machine 60) Actions in the virtual workspace and actions in the real workspace may be combined in various ways. The replayed action M1 may include a replayed action M1 in the virtual workspace, a replayed action M1 in the real workspace, or both. All or part of the current action M2 may include a current action M2 in the virtual workspace, a current action M2 in the real workspace, or both of these current actions M2. For example, the replayed action M1 may include an operation of the operating device 63 in the virtual space, an operation of the operating device 63 in the real space (operation of a real device by remote control), or both of these operations (the same applies to the current action M2).

[0133] (Model display) The computer 20 may display an object 50 in the work space as a model (one with a shape similar to that of a real object). The computer 20 may display an object 50 in a virtual work space as a model. The computer 20 may convert an object 50 in the real work space into an object 50 in the virtual work space and display the converted object 50 in the virtual space as a model. The computer 20 may display the object 50 as a two-dimensional model or a three-dimensional model. The content displayed by the computer 20 as a model (display items of the model) can be set in various ways, similar to the recording items and playback items. For example, the computer 20 may display all or part of the work machine 60 as a model. The computer 20 may display a cab 62c including an operating device 63 as a model, as shown in FIG. 7. The computer 20 may display all or part of the surrounding objects 80 shown in FIG. 2 as a model.

[0134] The viewpoint (how the model appears) (for example, viewpoint position, angle of view, direction) from which the object 50 in the virtual space is displayed as a model can be set in various ways. For example, the computer 20 may display an image of the interior and exterior of the cab 62c as seen from a viewpoint inside the cab 62c, as shown in FIG. 7. The computer 20 may display an image of the work machine 60 as seen from a viewpoint outside the work machine 60 (an overhead image of the work site). The computer 20 may display an image from one viewpoint, or may simultaneously display images from multiple viewpoints.

[0135] (Operation display of work machine 60 model) The computer 20 may display the motion of the work machine 60 (either or both of the replayed motion M1 and the current motion M2) as a three-dimensional model of all or part of the work machine 60. In this case, the user can easily confirm (imagine) the motion of the actual work machine. The computer 20 may simultaneously display the replayed motion M1 and the current motion M2 as a three-dimensional model of the work machine 60. For example, as shown in FIG. 7, the computer 20 may simultaneously display the replayed motion M1 and the current motion M2 as a three-dimensional model of the operation device 63.

[0136] (Examples of model displays and viewpoints) 2, the calculator 20 may display the work machine 60 as seen from a viewpoint outside the work machine 60. In this case, the user can check the state (posture, etc.) of the work machine 60 that cannot be seen from inside the cab 62c.

[0137] The calculator 20 may display the work machine 60 as seen from a viewpoint higher than the work machine 60. In this case, it may be possible for the user to check the rotation state (rotation angle, etc.) of the upper rotating body 62, and it may be possible for the user to check the traveling state (traveling position, etc.) of the work machine 60.

[0138] The computer 20 may display an image of the upper rotating body 62 and the attachment 65 viewed from the side (for example, directly or substantially directly from the side), as shown in FIG. 6. This "side" refers to the direction in which the axis of rotation of the boom 65a relative to the upper rotating body 62 extends. When the computer 20 displays an image of the upper rotating body 62 and the attachment 65 viewed from the side, the user can easily confirm the state (posture, position, etc.) of the attachment 65. For example, even if the attachment 65 shown in FIG. 6 is located in a position that cannot be seen from inside the operator's cab 62c shown in FIG. 7 (for example, when the tip attachment 65c is in the soil), the user can easily confirm the position of the attachment 65.

[0139] As shown in FIG. 7 , the computer 20 may display an image of the inside and outside (e.g., forward) of the cab 62c viewed from a viewpoint inside the cab 62c (e.g., near the seat). In this case, the user can experience the view of an operator sitting in the cab of a real work machine. The computer 20 may display an image of the operation device 63 viewed from a viewpoint inside the cab 62c (e.g., near the seat). In this case, the user can experience the view of an operator operating a real work machine. The computer 20 may display an image of the work device 64 and work object 83 outside the cab 62c viewed from a viewpoint inside the cab 62c. In this case, the user can easily confirm the content of the work performed by the work device 64 on the work object 83. Specifically, for example, the computer 20 may display an image of the tip attachment 65c and work object 83 outside the cab 62c viewed from inside the cab 62c. In this case, the user can easily check how the soil is being put into the bucket, how the work object 83 is being gripped, and the like.

[0140] 2, the calculator 20 may display an image of another machine (for example, a vehicle 85, an excavator, etc.) viewed from a viewpoint inside or outside the work machine 60. In this case, the user can check the status (position, movement, operation, etc.) of the work machine 60 during cooperative work carried out between the work machine 60 and another machine. This cooperative work is, for example, the loading of a work object 83 onto a vehicle 85 (dump truck).

[0141] (Display of the operation device 63 and the part operated by the operation device 63) The calculator 20 may simultaneously display the operation (operation) of the operating device 63 shown in Figure 7 and the operation of the part of the work machine 60 that is operated by the operating device 63 (the part to be operated, such as the attachment 65). In this case, the calculator 20 may simultaneously display the operation of the operating device 63 as seen from inside the cab 62c and the operation of the part to be operated (such as the attachment 65) as seen from outside the work machine 60, as shown in Figure 6. In this case, the user can confirm (understand) the relationship between the operation of the operating device 63 (see Figure 7) and the operation of the work machine 60. As a result, the user can efficiently learn how to operate the work machine 60 (accelerating mastery of the operation).

[0142] (Overlapping display and parallel display in model display of work machine 60) As shown in Fig. 2, the calculator 20 may simultaneously display multiple work machines 60 displayed as models. For example, the calculator 20 may simultaneously display a reproduced action M1 and a current action M2, or may simultaneously display multiple reproduced actions M1 (described later). Here, a case will be described in which the calculator 20 simultaneously displays a first work machine 60-1 performing a certain action (for example, reproduced action M1) and a second work machine 60-2 performing another action (for example, current action M2).

[0143] As shown in FIG. 6, the calculator 20 may display the first work machine 60-1 and the second work machine 60-2 in an overlapping manner (overlapping display) (see FIGS. 6 to 9). In this case, the calculator 20 may set the reference positions (for example, the position of the center of rotation, the position of the center of gravity, etc.) of the first work machine 60-1 and the second work machine 60-2 to be the same. In this case, only one part (the machine main body 60a in the example of FIG. 6) that is located in the same position between the first work machine 60-1 and the second work machine 60-2 is displayed. A part (the attachment 65 in the example of FIG. 6) that is located in a different position between the first work machine 60-1 and the second work machine 60-2 is displayed for both the first work machine 60-1 and the second work machine 60-2. This allows the user to easily confirm differences in the operation of the multiple work machines 60.

[0144] The calculator 20 may display the first work machine 60-1 and the second work machine 60-2 side by side (parallel display), as shown in Fig. 2. In this case, the calculator 20 displays the first work machine 60-1 and the second work machine 60-2 offset from each other so that they do not overlap. In this case, for example, even if it is difficult for the user to identify the first work machine 60-1 and the second work machine 60-2 in the above-mentioned overlapping display (for example, they are difficult to see due to overlapping), the user can easily identify the first work machine 60-1 and the second work machine 60-2.

[0145] When displaying a plurality of work machines 60 to be displayed as models, the calculator 20 may display the plurality of work machines 60 (specifically, the first work machine 60-1 and the second work machine 60-2) in a manner that allows the user to easily distinguish between them. For example, the calculator 20 may use different display formats for the first work machine 60-1 and the second work machine 60-2. Specifically, the calculator 20 may use different display colors or different degrees of transparency for the first work machine 60-1 and the second work machine 60-2. Specifically, the calculator 20 may display the first work machine 60-1 in a transparent manner and the second work machine 60-2 in a non-transparent manner.

[0146] (Display of captured image) The computer 20 may display a captured image showing the situation within the work space in real space. The object 50 in the above description of the model display may refer to the object 50 shown in the captured image. The computer 20 may display only a part of the captured image captured by the imaging device, or may display the entire image. For example, the computer 20 may display only the work machine 60 from the captured image of the work site, or may display only a part of the work machine 60 (for example, only the attachment 65). For example, the computer 20 may display only the range in which the work machine 60 performs work (for example, the range that the attachment 65 can reach) from the captured image of the surroundings of the work machine 60. For example, the computer 20 may display only the operation input unit 11a from the captured image of the inside of the cab 62c.

[0147] (Graph G31a) The calculator 20 may display (graph display) the operation of the work machine 60 (see FIG. 2) as shown in FIG. 10 as a graph G31a (see also FIG. 11). The "operation of the work machine 60" displayed as the graph G31a is one or both of the reproduced operation M1 and the current operation M2. One example of the reason why the calculator 20 displays the operation of the work machine 60 as the graph G31a is as follows. When the work machine 60 is displayed as a model only, as shown in FIG. 2, it may be difficult for the user to confirm the operation of the work machine 60 depending on the viewpoint, etc. For example, it may be difficult for the user to confirm the difference between the reproduced operation M1 and the current operation M2, or it may be difficult for the user to confirm the difference between multiple reproduced operations M1. Therefore, the calculator 20 displays the operation of the work machine 60 as a graph G31a (see FIG. 10). This may enable the user to easily confirm the operation of the work machine 60, and may also enable the user to easily confirm the difference between multiple operations of the work machine 60.

[0148] For example, calculator 20 may display information about the operation of work machine 60 (position, angle, etc.) as graph G31a (see FIG. 10). Calculator 20 may display graph G31a (time series graph) showing the relationship between time and operation. Calculator 20 may display coordinate axes of graph G31a, or may display images (text, graphics, etc.) that represent the meaning of the coordinate axes.

[0149] The calculator 20 may display various information about the operation of the work machine 60 as a graph G31a (see FIG. 10). For example, the calculator 20 may display information about the traveling operation (position, speed, etc.) as a graph G31a, or may display information about the turning operation (turning angle, etc.) as a graph G31a. The calculator 20 may display information about the work device 64 (such as the angle of the attachment 65) as a graph G31a. The calculator 20 may display information about the operation device 63 shown in FIG. 7 as a graph G31a. For example, the calculator 20 may display the position (operation amount) of the operation lever 63a as a graph G31a, or may display the position (operation amount) of the traveling operation unit 63b as a graph G31a. The calculator 20 may display the state of the operation device 63, such as a button (such as the on / off state of a switch), as a graph G31a. The calculator 20 may display information about the work performed by the work machine 60 as a graph G31a. For example, the computer 20 may display the amount of work (such as the amount of earthwork) performed by the work machine 60 on the work object 83 as a graph G31a.

[0150] A specific example of graph G31a shown in FIG. 10 will be described. In this example, calculator 20 displays graph G31a showing the relationship between the position of control lever 63a (see FIG. 7) and time. Calculator 20 displays the position of control lever 63a for current action M2 as graph G31a. Specifically, calculator 20 displays a graphic (plot, dot, black circle in FIG. 10) showing the current position of control lever 63a. Calculator 20 also displays a graphic (line, solid line in FIG. 10) showing the operation history (past positions) of current action M2. Calculator 20 displays the position (target position) of control lever 63a for replay action M1 as graph G31a. Specifically, calculator 20 displays a graphic (circle (white circle) and line (two-dot chain line) in FIG. 10) showing the position of control lever 63a for replay action M1. Calculator 20 may also display a graphic (two-dot chain circle in FIG. 10) showing the position of control lever 63a for replay action M1 at the current playback position. Of the positions of the operating lever 63a in the reproducing operation M1, the position of the operating lever 63a at a time later than the current reproducing position is the position of the operating lever 63a in a future reproducing operation M1f, which will be described later.

[0151] A specific example of graph G31a shown in Figure 11 will be described. In this example, calculator 20 displays an operation indicator G31c (described below) for operation lever 63a. This operation indicator G31c is also graph G31a. Specifically, calculator 20 displays a graphic indicating the position of operation lever 63a for playback action M1. Calculator 20 displays a graphic (a dot, or a white circle in Figure 11) indicating the position of operation lever 63a for playback action M1 at the current playback position. Calculator 20 displays a graphic (a two-dot chain line in Figure 11) indicating the position of operation lever 63a for future playback action M1f. Calculator 20 displays a graphic (a black circle in Figure 11) indicating the position of operation lever 63a for current action M2. Calculator 20 may also display a graphic indicating the operation history (past positions) of current action M2 (see "Operation History" in Figure 10).

[0152] 9 (described later) may be included in the graph G31a. The calculator 20 may display coordinate axes (not shown) in the posture-related information G31b.

[0153] (Posture-related information G31b) The calculator 20 may display the posture-related information G31b (may perform posture-related information display). The posture-related information G31b is information that indicates the posture of the work machine 60. The calculator 20 may superimpose the posture-related information G31b and a model of the work machine 60. The calculator 20 may display the posture-related information G31b without displaying the model of the work machine 60. Note that displaying the posture of the work machine 60 by simply displaying a model of the work machine 60 is not included in the posture-related information G31b.

[0154] The computer 20 displays information about the position of a specific part of the work machine 60 as posture-related information G31b. For example, the computer 20 may display the position (trajectory, etc.) of a specific part of the work machine 60 when the work machine 60 operates over a certain time range as a graphic (e.g., a line, etc.). The computer 20 may display the position of a specific part of the work machine 60 at a certain point in time as a graphic (e.g., a point, a circle, etc.). The above-mentioned "position of a specific part of the work machine 60" may be, for example, the position of the tip of the tip attachment 65c (e.g., the toe position), the position of the tip of the arm 65b (the arm top position), or the position of the tip of the boom 65a (the boom top position). The above-mentioned "position of a specific part of the work machine 60" may be the position (the amount of operation) of the operation lever 63a (see FIG. 7), or the position (the amount of operation) of the travel operation unit 63b (see FIG. 7). The calculator 20 may display the posture-related information G31b as a graph G31a (see FIG. 10).

[0155] (Operation indicator G31c) Calculator 20 may display an operation indicator G31c (may perform an operation indicator display) as shown in FIG. 11 . Operation indicator G31c is a display (indicator, schematic diagram) that indicates the state (operation state) of operation device 63. Examples of the reason why calculator 20 displays operation indicator G31c are as follows. When operation device 63 is displayed only as a model, it may be difficult for the user to confirm the state of operation device 63. For example, depending on the angle at which operation device 63 is viewed (the angle of operation device 63 reflected on display unit 31), it may be difficult for the user to confirm the state of operation device 63. Therefore, calculator 20 displays operation indicator G31c that indicates the state of operation device 63. This allows the user to easily confirm the state of operation device 63. This allows the user to easily confirm the relationship between the state of (actual) operation input unit 11a at the user's hand and the state of operation device 63 displayed on display unit 31.

[0156] Calculator 20 may display operation indicators G31c that indicate the state of left and right operation levers 63a (e.g., operation direction and operation amount). Calculator 20 may display operation indicators G31c that indicate the operation state of travel operation unit 63b (e.g., operation amount). Calculator 20 may display operation indicators G31c that indicate the state of operation devices 63 such as buttons (e.g., on / off state of switches).

[0157] A specific example of the operation indicator G31c shown in FIG. 11 will be described. In this example, the calculator 20 displays the state (operation direction and operation amount) of the operation lever 63a as a graph G31a. The calculator 20 displays, as the graph G31a, figures (circles and lines) indicating the operation position of the operation lever 63a, operation axes (coordinate axes), and explanations of the operation axes. The explanations of the operation axes indicate (explain) the relationship between the operation direction of the operation device 63 (the operation lever 63a in this example) and the details of the operation (specifically, boom up, boom down, etc.). The explanations of the operation axes may include characters or figures indicating the name of the details of the operation. The explanations of the operation axes are switched depending on the lever pattern of the operation lever 63a.

[0158] In this example, the operation indicator G31c simultaneously displays the forward / backward operation amount and the left / right operation amount of the right operating lever 63ar. Specifically, the forward / backward operation amount of the right operating lever 63ar is represented by the up / down coordinate axis of the operation indicator G31c, and the left / right operation amount of the right operating lever 63ar is represented by the left / right coordinate axis of the operation indicator G31c. In this example, the forward / backward operation of the right operating lever 63ar is assigned to the operation of the boom 65a (see FIG. 2), and the left / right operation of the right operating lever 63ar is assigned to the operation of the bucket (an example of the tip attachment 65c (see FIG. 2)). This operation indicator G31c allows the user to simultaneously check the operation amounts of both the operation of the boom 65a and the operation of the bucket (tip attachment 65c). Like the operation indicator G31c of the right operating lever 63ar, the operation indicator G31c of the left operating lever 63al allows the user to simultaneously check the operation amounts of both the operation of the arm 65b (see FIG. 2) and the operation of the swing of the upper rotating body 62 (see FIG. 2).

[0159] A specific example of the operation indicator G31c shown in FIG. 12 will be described. In this example, calculator 20 displays graphics (for example, an arrow and a circle) indicating the state (operation direction and operation amount) of operation lever 63a. Specifically, calculator 20 displays graphics (arrows in FIG. 12) indicating the range of the operation amount of operation lever 63a. Calculator 20 also displays graphics (circles in FIG. 12) indicating the current operation amount of operation lever 63a. Calculator 20 displays graphics (white circles in FIG. 12) indicating the operation amount of operation lever 63a for replay action M1 at the current playback position. Calculator 20 displays graphics (black circles in FIG. 12) indicating the operation amount of operation lever 63a for current action M2. In the example of FIG. 12, calculator 20 displays graphics (circles in FIG. 12) indicating the operation amount of operation lever 63a in the forward / backward direction. Calculator 20 may also display graphics indicating the operation amount of operation lever 63a in the left / right direction.

[0160] (Future playback action M1f) The calculator 20 may display a future replay action M1f, as shown in FIG. 8 (see FIGS. 8 to 11). The future replay action M1f is a replay action M1 at a point in time in the future than the replay action M1 at the time (at the current replay position) being replayed by the replay unit 23. By the calculator 20 displaying the future replay action M1f, the user can confirm the operation of the work machine 60 at a replay position beyond the current replay position (for example, at the next moment). The calculator 20 may simultaneously display the replay action M1 at the current replay position and the future replay action M1f.

[0161] The calculator 20 may display the future replay action M1f as a model of the work machine 60, as a graph G31a (see FIG. 10), or as posture-related information G31b (see FIG. 9). The calculator 20 may display only one future replay action M1f, or may display multiple future replay actions M1f. The calculator 20 may display the future replay action M1f in stages, or continuously (as a moving image). Specific examples of the future replay action M1f will be further described below.

[0162] The computer 20 displays the future replay action M1f as a model of the work machine 60, for example, as follows: The computer 20 may display a model of the work machine 60 performing the replay action M1 a predetermined time after the current replay position (for example, a predetermined number of frames after). Specifically, the computer 20 may display the future replay action M1f at regular time intervals (frame intervals), such as 1 second, 2 seconds, or 3 seconds after the present.

[0163] Calculator 20 may display future playback actions M1f for each step of work by the work machine 60, as shown in Fig. 9. In particular, calculator 20 may display a model of the work machine 60 at the time when the work content (work situation) changes during a series of work actions (playback actions M1f from the present to the future) of the work machine 60. Specifically, a case will be described in which the series of works by the work machine 60 (playback actions M1) is work to excavate earth and sand with a bucket (an example of a tip attachment 65c). In this case, calculator 20 may display future playback actions M1f at one or more of the following times, for example, before excavation, when the bucket contacts the ground, when the bucket is excavating the ground, when the bucket is raised from the ground, and after the bucket has raised from the ground.

[0164] The calculator 20 may display future playback actions M1f so as to allow the user to confirm (understand) the entire flow of a series of work tasks performed by the work machine 60. For example, the calculator 20 may display future playback actions M1f from the current playback position until the end of a series of work tasks (for example, until the bucket is raised from the ground). In this case, the user can confirm the state of the work machine 60 at the current playback position within the entire flow of a series of work tasks performed by the work machine 60. Note that the calculator 20 may display only a portion of the series of work tasks performed by the work machine 60 as future playback actions M1f. In Figures 8 and 9, an example is shown in which the calculator 20 displays a model of the entire work machine 60 as future playback actions M1f. On the other hand, the calculator 20 may display a model of only a portion of the work machine 60 (for example, the operating device 63 (see Figure 7)) as future playback actions M1f.

[0165] Calculator 20 displays future reproduced operation M1f as a graph G31a, for example, as follows, as shown in Figure 10. Calculator 20 may display a graph G31a showing target values ​​(angle, position, operation amount, etc.) for the operation of work machine 60 (see Figure 2). Calculator 20 may display a graph G31a showing time-series information on the target values ​​(two-dot chain line in Figure 10). Calculator 20 may display a shape (circle in Figure 10) showing the target value at a certain point in the future.

[0166] The calculator 20 may display, on the operation indicator G31c, a graph G31a indicating a future reproduced operation M1f, as shown in Fig. 11. For example, the calculator 20 may display a graph G31a (a ghost operation trajectory) (indicated by a two-dot chain line in Fig. 11) indicating a target operation position of the operation device 63 (e.g., the operation lever 63a) from the current reproduced position until a predetermined time later (e.g., three seconds later). In this case, the user can easily confirm how to move the operation device 63. Figs. 10 and 11 show an example in which the calculator 20 displays a graph G31a indicating the position (lever operation amount) of the operation device 63 as the graph G31a indicating the future reproduced operation M1f. On the other hand, the calculator 20 may display, as the graph G31a indicating the future reproduced operation M1f, a graph G31a (see Fig. 10) indicating the state of the work machine 60 itself (e.g., the posture of the attachment 65) shown in Fig. 2.

[0167] The computer 20 may display posture-related information G31b indicating a future reproduction motion M1f, as shown in Fig. 9. Specifically, the computer 20 may display information indicating a future position of the tip of the arm 65b (arm top position), or may display information indicating a future position of the tip of the tip attachment 65c (for example, a toe position).

[0168] (Difference output (Difference display G31d)) The calculator 20 may calculate the differences between a plurality of actions and cause the output unit 30 to output information about the calculated differences. Hereinafter, causing the calculator 20 to cause the output unit 30 to output information will also be referred to as the calculator 20 outputting information, etc. The display of the difference information is referred to as a difference display G31d, as shown in FIG. 6. For example, the calculator 20 calculates the difference between a replay action M1 and a current action M2, and outputs information about the calculated difference. When a plurality of replay actions M1 are replayed, the calculator 20 may calculate the differences between the plurality of replay actions M1 and output information about this difference.

[0169] The "difference" calculated and output by the calculator 20 may be a difference in the overall position of the work machine 60 (for example, the traveling position). This difference may be a difference in the attitude of the work machine 60, for example, a difference in the angle of the work implement 64 (for example, a difference in the angle of the attachment 65). This difference may be a difference in the operation of the operating device 63 (see FIG. 7), for example, a difference in the amount of operation (operation position) of the operating device 63, or a difference in the state of a button (such as the on / off state of a switch).

[0170] The difference information output by the calculator 20 may be a difference (amount) or a match rate. The difference information output by the calculator 20 may also be information prompting correction, more specifically, information on corrective action (direction, amount, etc.) to be taken to reduce the difference (specific examples will be described later).

[0171] Calculator 20 may display the difference information as a graphic. Specifically, calculator 20 may display an arrow indicating the difference, or an arrow indicating the direction and amount of correction to be made. Calculator 20 may display the difference information as a numerical value (e.g., a numerical value indicating the difference, a numerical value indicating the matching rate, etc.). Calculator 20 may output the difference information as characters (words, phrases) other than numerical values. Specifically, calculator 20 may output information on the corrective action to be taken to reduce the difference (information encouraging correction) as text, or may cause audio output unit 33 (see FIG. 1) to output the information as audio.

[0172] A specific example of the difference display G31d shown in FIG. 6 will be described. In this example, the calculator 20 displays the difference display G31d in a model display of the work machine 60. In this example, there is a difference in the angle of the boom 65a between the replayed operation M1 and the current operation M2. Therefore, the calculator 20 displays information about the operation (corrective operation) of the boom 65a that should be performed to reduce this difference. Specifically, the calculator 20 graphically displays the direction and amount of movement of the boom 65a to reduce this difference. More specifically, in FIG. 6, the calculator 20 displays an arrow instructing the user to raise the boom 65a to reduce this difference. The calculator 20 indicates the direction and amount of movement of the boom 65a to reduce this difference by the direction and length of the arrow. The calculator 20 may output, in text or audio, a message urging the user to make the correction, specifically, a message such as "Please raise the boom 65a."

[0173] A specific example of the difference display G31d shown in FIG. 12 will be described. In this example, calculator 20 displays difference display G31d in operation indicator G31c. In this example, there is a difference in the operation position of operating lever 63a between reproduced action M1 and current action M2. Therefore, calculator 20 displays information about the operation of operating lever 63a to reduce this difference (correction operation). Specifically, calculator 20 displays, in a graphic form, the direction and amount of operation of operating lever 63a to reduce this difference. More specifically, in the example shown in FIG. 12, calculator 20 displays an arrow instructing the user to operate right operating lever 63ar toward the user in order to reduce this difference. Calculator 20 indicates, by the direction and length of the arrow, the direction and amount of movement of right operating lever 63ar to reduce this difference (correction operation). Calculator 20 may output, in text or audio, a message urging the user to make the correction, specifically, a message such as "Pull the right lever."

[0174] (1st operation only) The calculator 20 may display only the first action. Displaying only the first action means that the calculator 20 displays the reproduced action M1 (first action) shown in Figure 2, but does not display the current action M2 (second action). When displaying only the first action, the calculator 20 does not reflect the operation of the operation input unit 11a by the user in the operation of the work machine 60.

[0175] The first action independent display includes, for example, preview playback and retrospective playback (described later). In addition, playback of playback action M1, which is performed only to allow the user to check the content recorded in the recording unit 21 (the content of the recording file), may be included in the first action independent display.

[0176] (Preview playback (step S52)) The computer 20 may perform preview playback. More specifically, the computer 20 may perform preview playback of a first action alone, and then simultaneously display the playback action M1 and the current action M2. The preview playback may be used, for example, as follows: Before the user operates the operation input unit 11a (in advance, before practicing the operation), a model playback action M1 (pre-check action) operated by an experienced operator is shown to the user (preview playback). Thereafter, the user operates the operation input unit 11a while checking (for example, imitating) the model playback action M1. At this time, the current action M2 is displayed in response to the user's operation. In this example, the preview playback allows the user to confirm (understand) the content of the operation (practice content) of the operation input unit 11a in advance, and then the user can operate the operation input unit 11a (practice the operation). This allows the user to efficiently learn the operation.

[0177] (Record and play back multiple situations) The recording unit 21 may record a plurality of situations in the work space. Then, the playback unit 23 may play back one or more situations from among the plurality of situations recorded in the recording unit 21. The recording unit 21 may record a plurality of first actions of the work machine 60. Then, the playback unit 23 may play back one or more first actions (playback actions M1) from among the plurality of first actions recorded in the recording unit 21.

[0178] (Retrospective playback (step S57)) The recording unit 21 is capable of recording multiple first actions, which enables retrospective playback (playback of the recorded current action M12). More specifically, as described above, the calculator 20 may simultaneously display the played-back action M1 and the current action M2. At this time, the recording unit 21 records the current action M2 (records the second action as the first action). The current action M2 recorded by the recording unit 21 is designated as the "recorded current action M12." Thereafter, the playback unit 23 plays back (i.e., retrospective playback) the recorded current action M12. This retrospective playback allows the user to confirm the actions of the work machine 60 in response to the operations of the operation input unit 11a performed by the user (allowing the user to look back on their own operations). This allows the user to efficiently learn operations.

[0179] (Simultaneous playback of multiple actions) As described above, the recording unit 21 may record multiple first actions of the work machine 60. The recording unit 21 may record multiple actions of a single work machine 60, or may record one or more actions of multiple work machines 60. The playback unit 23 may then play back multiple actions of the work machine 60 simultaneously. In this case, the playback unit 23 may play back multiple actions of a single work machine 60 simultaneously, or may play back one or more actions of multiple work machines 60 simultaneously.

[0180] For example, the playback unit 23 may simultaneously play back the playback action M1 of the model operation of an experienced operator (normal playback) and the playback action M1 of the model operation of a novice operator (trainee) (review playback). In this case, the user can compare the playback action M1 of the model operation with the playback action M1 performed by the user's own operation. This allows the user to efficiently learn the operation.

[0181] Furthermore, for example, a plurality of playbacks (retrospective playbacks) of playback actions M1 performed by a new operator may be performed simultaneously. Specifically, the playback unit 23 may simultaneously play back a playback action M1 performed by a user immediately before playback (for example, one minute ago) and a playback action M1 performed by a user even earlier than the "immediate" playback (for example, one hour ago). In this case, the user can confirm that their operation has progressed. This allows the user to efficiently learn the operation.

[0182] (split playback) The playback unit 23 may play back the situation in the workspace recorded in the recording unit 21 for each chapter C (playback area) shown in FIG. 13. Specifically, the recording unit 21 records a series of situations in the workspace. The computer 20 divides the series of situations into a plurality of chapters C. Then, the playback unit 23 may play back the situation in the workspace for each chapter C.

[0183] The playback unit 23 may play back the playback action M1 for each chapter C. In particular, the recording unit 21 records a series of first actions of the work machine 60. The calculator 20 divides the series of first actions into a plurality of chapters C. The playback unit 23 may then play back the first action (playback action M1) for each chapter C. By the playback unit 23 playing back the playback action M1 for each chapter C, the user can confirm the series of actions of the work machine 60 for each chapter C. Below, a case where the playback action M1 is divided into chapters C will be described as an example in which the situation in the workspace is divided into chapters C.

[0184] The instruction to divide the playback operation M1 into chapters C and the designation of the division position may be given by the user. For example, the computer 20 (recording unit 21) may set the timing when the chapter setting instruction unit G21b3 (see FIG. 4) is selected (operated) as the division position of chapter C.

[0185] The instruction to divide the playback action M1 into chapters C and the specification of the division positions may be automatically performed by the computer 20. For example, the computer 20 may divide a series of actions of the work machine 60 (see FIG. 2) into chapters C at predetermined time intervals. Furthermore, for example, the computer 20 may divide a series of actions of the work machine 60 into chapters C at the point in time when the work content (work situation) of the work machine 60 switches (see the explanation of "future playback action M1f" above for a specific example). In this case, the computer 20 may determine the work content of the work machine 60 according to the situation (position, posture, etc.) of the work machine 60, and determine the point in time when the work content switches.

[0186] The division of the replay operation M1 into chapters C may be performed while recording the first operation of the work machine 60 (see FIG. 2) (step S34), or may be performed after recording of the first operation is stopped (step S35). The division of the replay operation M1 into chapters C may be performed after recording of the first operation is stopped and before playback, or may be performed during playback of the first operation (replay operation M1) (steps S52, S53 to S55) (the operation may be divided into chapters C in playback mode).

[0187] (Example of division playback) The playback unit 23 may perform playback for each chapter C, for example, as follows. The playback unit 23 may start playback of the playback operation M1 from the start position of the chapter C. In this case, the playback unit 23 may start playback of the playback operation M1 from the start position of a chapter C in the middle of the series of playback operations M1, rather than from the first playback position of the series of playback operations M1 (except when playing the first chapter C of multiple chapters C). The playback unit 23 may stop playback of the playback operation M1 at the end position of the chapter C. In this case, the playback unit 23 may stop playback of the playback operation M1 not at the end position of the series of playback operations M1, but at the end position of a chapter C in the middle of the series of playback operations M1. The playback unit 23 may repeat playback of one or more chapters C.

[0188] (Divided playback determination process) The playback unit 23 may determine whether to start playback of the (n+1)th chapter C (next chapter C) when the playback position of the playback operation M1 reaches the end of the nth chapter C. "n" is the order of the chapters C counted from the start position (0 seconds) of the series of playback operations M1. "n" is an integer greater than or equal to 1 and less than or equal to the number of chapters C (number of divisions).

[0189] In detail, the playback unit 23 starts playback of the (n+1)th chapter C when the playback position of the playback operation M1 reaches the end of the nth chapter C and a predetermined condition (next chapter start condition) is satisfied. The playback unit 23 may start playback of the (n+1)th chapter C if the next chapter start condition is satisfied after the playback position of the playback operation M1 reaches the end of the nth chapter C. Even if the playback position of the playback operation M1 reaches the end of the nth chapter C, the playback unit 23 does not start playback of the (n+1)th chapter C unless the next chapter start condition is satisfied. Note that the playback unit 23 may stop playback when the playback position of the playback operation M1 reaches the end of the last chapter C of the multiple chapters C.

[0190] The next chapter start condition can be set in various ways. The next chapter start condition may be a condition for the difference in the state of the work machine 60 shown in FIG. 6. The next chapter start condition may be that the difference between the state of the first work machine 60-1 performing the reproduced operation M1 and the state of the second work machine 60-2 performing the current operation M2 at the end of the nth chapter C (see FIG. 13) is within a predetermined range. In other words, the playback unit 23 may start playback of the (n+1)th chapter C when the difference between the state of the first work machine 60-1 performing the reproduced operation M1 and the state of the second work machine 60-2 performing the current operation M2 at the end of the nth chapter C is within a predetermined range. The above "difference is within a predetermined range" may also mean that there is no difference (it is zero).

[0191] Specifically, for example, the condition for starting the next chapter may be that the difference between the attitude of the first work machine 60-1 and the attitude of the second work machine 60-2 (for example, the difference in the angle of the attachment 65) at the end of the nth chapter C (see Figure 13) falls within a predetermined range. The condition for starting the next chapter may be that the difference between the state of the operating device 63 of the first work machine 60-1 (see Figure 7) and the state of the operating device 63 of the second work machine 60-2 (for example, the difference in the amount of operation) at the end of the nth chapter C falls within a predetermined range.

[0192] The next chapter start condition may include a condition different from the condition regarding the difference in the state of the work machine 60. For example, the next chapter start condition may include a condition regarding the difference in the amount of work performed by the work machine 60. In more detail, the next chapter start condition may include the difference between the amount of work performed by the first work machine 60-1 and the amount of work performed by the second work machine 60-2 falling within a predetermined range. Note that the next chapter start condition does not have to include a condition regarding the difference in the state of the work machine 60.

[0193] (Multiple machine recording and playback) The computer 20 (recording unit 21 and reproducing unit 23) may record and reproduce the operations performed by multiple machines simultaneously.

[0194] For example, the calculator 20 may record and play back the actions of multiple work machines 60. In particular, the recording unit 21 may record the actions of multiple work machines 60, and the playback unit 23 may simultaneously play back the first actions (playback actions M1) of the multiple work machines 60 recorded by the recording unit 21. In this case, it is possible to play back coordinated work between multiple work machines 60. Therefore, by having the user operate the operation input unit 11a while playing back the coordinated work between multiple work machines 60, the user can practice operating the coordinated work.

[0195] For example, the calculator 20 may record and play back the operations of the work machine 60 and vehicle 85 shown in Figure 2. In this case, it is possible to play back coordinated work between the work machine 60 and the vehicle 85. Specifically, it is possible to play back coordinated work in which, for example, the vehicle 85 moves (travels) close to the work machine 60, the work machine 60 loads the work object 83 onto the loading platform 85a of the vehicle 85, and then the vehicle 85 moves (travels) after loading is complete.

[0196] (combination) The various recording, playback, and output (mainly display) processes described above can be combined in various ways and executed in various orders.

[0197] (An example of processing order) An example of the processing sequence of the computer 20 will be described below, mainly with reference to the flowchart shown in FIG. 3. Unless otherwise specified, the processing sequence will be described below. Note that the processing sequence can be changed in various ways. Note that, as described above, the input unit 11, computer 20, output unit 30, and their components (e.g., recording unit 21, playback unit 23, etc.) will be described with reference to FIG. 1.

[0198] In step S10, the computer 20 sets the operation mode to the normal operation mode.

[0199] In step S21, the computer 20 determines whether or not to set the operation mode to the playback mode (step S40). For example, the computer 20 determines whether or not a specific operation for setting the operation mode to the playback mode has been performed on the input unit 11. If the computer 20 determines that the operation mode should be set to the playback mode (YES in step S21), it switches the operation mode to the playback mode, starts the playback mode, and performs the processing of step S41. If the computer 20 determines that the operation mode should not be set to the playback mode (NO in step S21), it performs the processing of step S22.

[0200] In step S22, the computer 20 determines whether or not to set the operation mode to the recording mode (step S30). For example, the computer 20 determines whether or not a specific operation for setting the operation mode to the recording mode has been performed on the input unit 11. If the computer 20 determines that the operation mode should be set to the recording mode (YES in step S22), it switches the operation mode to the recording mode, starts the recording mode, and performs the processing of step S31. If the computer 20 determines that the operation mode should not be set to the recording mode (NO in step S22), it does not switch the operation mode to the recording mode, but sets the operation mode to the normal operation mode (step S60).

[0201] (Step S30: Recording mode) In step S31, the file name of the recording file is set in the recording unit 21. As described above, the file name of the recording file may be set by the user or may be set automatically by the computer 20.

[0202] In step S32, the items to be recorded by the recording unit 21 (recording items) are set in the recording unit 21. As described above, the recording items may be selected (set) by the user or may be automatically set by the computer 20.

[0203] In step S33, the recording unit 21 starts recording the situation in the workspace. As described above, the recording unit 21 starts recording, for example, when the user selects (presses) the recording start instruction unit G21b1 (see FIG. 4) at any timing of the user's choosing.

[0204] In step S34, the computer 20 (for example, the recording unit 21) divides the situation in the workspace to be recorded into chapters C (see FIG. 13). As described above, the division into chapters C may be performed, for example, when the chapter setting instruction unit G21b3 (see FIG. 4) is selected (pressed) by the user at any timing of the user's choosing, or may be performed automatically by the computer 20.

[0205] In step S35, the recording unit 21 stops (ends) recording. As described above, the recording unit 21 stops recording, for example, when the user selects (presses) the recording stop instruction unit G21b2 (see FIG. 4) at a timing of the user's choosing. After the recording unit 21 stops recording, the computer 20 returns the operation mode to the normal operation mode (step S60).

[0206] (Step S40: Playback mode) In step S41, a playback file to be played by the playback unit 23 (for example, the file name of the playback file) is set in the playback unit 23. As described above, the playback file may be selected (set) by the user or may be automatically set in the computer 20.

[0207] In step S42, the items to be played back by the playback unit 23 (playback items) are set in the playback unit 23. As described above, the playback items may be selected (set) by the user or may be automatically set by the computer 20.

[0208] In step S43, the playback method to be performed by the playback unit 23 is set in the playback unit 23. For example, the playback method to be set may include whether or not to perform repeat playback. The playback method to be set may include a display method for the playback situation by the playback unit 23 (for example, display of a model, display of a graph G31a (see FIG. 10), display of an operation indicator G31c (see FIG. 11), etc.).

[0209] In step S44, the playback speed is set in the playback unit 23. As described above, the playback speed may be set by the user using the playback speed instruction unit G23b1 (see FIG. 5), or may be automatically set by the computer 20.

[0210] In step S51, the playback unit 23 sets the situation within the work space to the situation at the start of recording. In particular, the playback unit 23 sets the situation within the work space (situation at the work site) at the start of playback to the situation within the work space when recording of the series of situations that the playback unit 23 is to play back started (at the start of recording). The playback unit 23 sets the surrounding situation of the work machine 60 shown in FIG. 2 (specifically, the situation of the peripheral object 80) to the surrounding situation at the start of recording. The playback unit 23 may also set the situation of the work machine 60 (for example, position, posture, etc.) to the situation of the work machine 60 at the start of recording.

[0211] In step S52, the playback unit 23 performs the above-mentioned preview playback. The playback unit 23 may perform preview playback if the user or the computer 20 has selected (set) to perform preview playback. The playback unit 23 may not perform preview playback if the user or the computer 20 has selected (set) not to perform preview playback.

[0212] In step S53, the playback unit 23 starts playing back the situation in the work space (for example, playback action M1, etc.). As described above, playback is started, for example, when the playback start instruction unit G23b3 (see FIG. 5) is selected (pressed) by the user. Furthermore, the computer 20 displays the situation to be played back (for example, playback action M1) on the playback unit 23, while also displaying the action (current action M2) of the work machine 60 in response to the user's operation of the operation input unit 11a. The computer 20, for example, simultaneously displays the playback action M1 and the current action M2. The recording unit 21 may record this current action M2 (step S33) (the computer 20 may execute the playback mode and the recording mode simultaneously).

[0213] In step S54, the calculator 20 performs a difference display G31d (see FIG. 6). Specifically, when the reproduced motion M1 and the current motion M2 are simultaneously displayed, the calculator 20 calculates and displays information on the difference between the reproduced motion M1 (target) and the current motion M2.

[0214] In step S55, the playback unit 23 stops (ends) playback. As described above, playback may be stopped, for example, when the playback stop instruction unit G23b5 (see FIG. 5) is selected (pressed) by the user. When the playback position reaches the playback end position (see FIG. 13), the playback unit 23 stops playback. If recording of the current action M2 has started in step S53, the recording unit 21 may automatically stop recording in conjunction with the stopping of playback in step S55 (for example, simultaneously with the stopping of playback). If recording of the current action M2 has started in step S53, the recording unit 21 may stop recording when the recording stop instruction unit G21b2 (see FIG. 4) is selected (pressed) by the user.

[0215] In step S56, the computer 20 determines whether or not to re-practice. This "re-practice" means to simultaneously display the reproduced action M1 and the current action M2 again. If the computer 20 determines that the re-practice will be performed, it performs the process of step S51. For example, the computer 20 may display a GUI that allows the user to select whether or not to re-practice, and determine whether or not to re-practice based on the selection (instruction) on this GUI. For example, the computer 20 may determine whether or not to re-practice based on a predetermined condition (for example, the content of the operation performed by the user). If the computer 20 determines that the re-practice will not be performed, it performs the process of step S57.

[0216] In step S57, the playback unit 23 performs retrospective playback. More specifically, when the current action M2 is recorded in steps S53 to S55, the playback unit 23 can play back the recorded current action M12. The computer 20 (playback unit 23) may display a GUI that allows the user to select whether or not to perform retrospective playback, and may determine whether or not to perform retrospective playback based on the selection (instruction) on this GUI. Furthermore, the computer 20 may automatically start retrospective playback in accordance with a predetermined condition (retrospective playback start condition).

[0217] Thereafter, when a condition for terminating the playback mode is satisfied, the computer 20 returns the operation mode to the normal operation mode (step S60). For example, the computer 20 may display a GUI that allows the user to select whether to terminate the playback mode, and terminate the playback mode when the user selects to terminate the playback mode. Alternatively, the computer 20 may automatically terminate the playback mode in accordance with a predetermined condition (a playback mode termination condition). For example, the computer 20 may automatically terminate the playback mode when the playback position of the retrospective playback reaches the playback end position (see FIG. 13).

[0218] (Effects of the first invention) The effects of the work machine operation system 1 shown in FIG. 1 are as follows.

[0219] The work machine operation system 1 comprises an output unit 30, an input unit 11, and a computer 20. The output unit 30 is capable of displaying images. The input unit 11 (operation input unit 11a) is capable of operating the work machine 60. The computer 20 comprises a recording unit 21 and a playback unit 23.

[0220] [Configuration 1-1] The recording unit 21 records a first action, which is an action of the work machine 60 in response to the operation of the input unit 11. The playback unit 23 plays back the first action (playback action M1) recorded in the recording unit 21.

[0221] [Configuration 1-2] The calculator 20 causes the output unit 30 to display the first action (playback action M1 (see Figure 2)) played back by the playback unit 23 and the second action (current action M2 (see Figure 2)), which is the action of the work machine 60 in response to the operation being performed on the input unit 11.

[0222] In the above [Configuration 1-2], as shown in Fig. 2, the first action (reproduced action M1) and the second action (current action M2) are displayed on the output unit 30. Therefore, while or after the user is checking the first action (reproduced action M1, for example, a model action) of the work machine 60, the user can operate the input unit 11 (see Fig. 1) and check the second action (current action M2) of the work machine 60 that corresponds to this operation. Therefore, the user can efficiently learn how to operate the work machine 60.

[0223] In the above [Configuration 1-1], the recording unit 21 (see FIG. 1) records a first action of the work machine 60 in response to operation of the input unit 11 (see FIG. 1) by the user. Thus, the first action (e.g., a model action) can be recorded by this work machine operation system 1. Therefore, even if the action (e.g., a model action) to be confirmed by the user is not set in advance in the work machine operation system 1, the first action to be confirmed by the user can be recorded and played back by the work machine operation system 1. Therefore, the above [Configuration 1-1] and [Configuration 1-2] enable the user to efficiently learn operations related to actions of the work machine 60 that are not set in advance in the work machine operation system 1.

[0224] (Effects of the second invention) [Configuration 2] The first action (reproduced action M1) and the second action (current action M2) each include an action of the operating device 63 inside the work machine 60 (see FIG. 7).

[0225] 7, the above [Configuration 2] and [Configuration 1-1] make it possible to display the operation of the operating device 63, which is the first operation (reproduced operation M1), and the operation of the operating device 63, which is the second operation (current operation M2), on the output unit 30. Therefore, it is possible to allow the user to efficiently learn the operation (operation) of the operating device 63.

[0226] (Effect of the third invention) [Configuration 3] The recording unit 21 (see FIG. 1) records the position of a travellable work machine 60 (see FIG. 2).

[0227] By using the above [Configuration 3] and [Configuration 1-1], the work machine operation system 1 can record and play back the traveling operation (traveling operation) of the work machine 60 shown in Fig. 2. As a result, the user can efficiently learn the operation of traveling the work machine 60 (traveling operation).

[0228] (Effect of the fourth invention) [Configuration 4] The recording unit 21 (see Figure 1) records the ambient conditions at the start of the first action, which are the conditions around the work machine 60 when recording of the first action begins. The playback unit 23 (see Figure 1) sets the conditions around the work machine 60 to the ambient conditions at the start of the first action when playback of the first action (playback action M1) begins.

[0229] The above [Configuration 4] allows the user to learn how to operate the work machine 60 in accordance with the situation around the work machine 60 at the start of recording the first action.

[0230] (Effect of the fifth invention) [Configuration 5] The playback speed of the first action (playback action M1) played back by the playback unit 23 (see FIG. 1) is variable (see playback speed instruction unit G23b1 in FIG. 5).

[0231] The above [Configuration 5] provides the following effects. When the first action (played back action M1) is played back at a speed slower than the actual speed of the first action (the speed at which it was recorded), the user can easily confirm the first action (played back action M1). As a result, the user can efficiently learn how to operate the work machine 60. Furthermore, when the first action (played back action M1) is played back at a speed faster than the actual speed of the first action, the user can confirm the content of the played back action M1 in a short time. Furthermore, when the first action (played back action M1) is played back at a speed faster than the actual speed of the first action, the user may be able to learn a faster operation (for example, increasing the difficulty of the training).

[0232] (Effect of the sixth aspect of the invention) [Configuration 6] As shown in FIG. 6, the calculator 20 (see FIG. 1) calculates the difference between the first action (playback action M1) and the second action (current action M2), and outputs information about the calculated difference to the output unit 30 (see difference display G31d).

[0233] The above [Configuration 6] provides the following effects. In the above [Configuration 1-2], the reproduced motion M1 and the current motion M2 are displayed on the output unit 30. Simply displaying the reproduced motion M1 and the current motion M2 may make it difficult for the user to understand the difference between the reproduced motion M1 and the current motion M2. Therefore, in the above [Configuration 6], information on the difference between the reproduced motion M1 and the current motion M2 is calculated and output. This allows the user to easily confirm this difference. As a result of allowing the user to easily confirm the difference between the reproduced motion M1 and the current motion M2, the user can efficiently learn how to operate the work machine 60. For example, the user can easily confirm (understand) how to modify the operation currently being performed by the user on the input unit 11 so that the current motion M2 approaches the reproduced motion M1.

[0234] (Effect of the seventh invention) [Configuration 7] The computer 20 (see FIG. 1) causes the output unit 30 to display one or both of the first action (playback action M1) and the second action (current action M2) as a model of the work machine 60.

[0235] The above [Configuration 7] allows the user to easily visualize an actual work machine. For example, as shown in Fig. 7, when an image of a model of the cab 62c seen from a viewpoint inside the cab 62c is displayed, the user can experience how an operator sitting in the cab of an actual work machine would look. Also, for example, as shown in Fig. 2, when an image of a model of the work machine 60 seen from a viewpoint outside the work machine 60 is displayed, the user can easily visualize the movement of the actual work machine when the user operates the input unit 11 (see Fig. 1).

[0236] (Effect of the eighth aspect of the invention) [Configuration 8] The computer 20 (see FIG. 1) causes the output unit 30 to display one or both of the first action (reproduced action M1) and the second action (current action M2) as a graph G31a, as shown in FIG.

[0237] The above [Configuration 8] provides the following effects. As shown in FIG. 2, when the work machine 60 is displayed as a model, it may be difficult for the user to confirm the operation of the work machine 60 depending on the viewpoint setting, etc. Therefore, in the above [Configuration 8], as shown in FIG. 10, one or both of the replay action M1 and the current action M2 are displayed as a graph G31a. This makes it possible for the user to easily confirm the action displayed as the graph G31a. As a result, the user can efficiently learn how to operate the work machine 60.

[0238] (Effect of the ninth invention) [Configuration 9] As shown in Figure 8, the computer 20 (see Figure 1) causes the output unit 30 to display a first action (future playback action M1f) at a time in the future than the first action (playback action M1) currently being played back by the playback unit 23 (see Figure 1).

[0239] The above [Configuration 9] allows the user to confirm (understand) how the playback action M1 will change after the playback action M1 at the current playback position. For example, the user can perform operations on the input unit 11 (see FIG. 1) while checking the future playback action M1f. As a result, the user can efficiently learn how to operate the work machine 60.

[0240] (Effect of the 10th invention) [Configuration 10] The computer 20 (see FIG. 1) displays the first action (reproduced action M1) shown in FIG. 2, but causes the output unit 30 to display a first action only display in which the second action (current action M2) is not displayed.

[0241] The above [Configuration 10] allows the user to reliably confirm the content of the regenerated operation M1, and as a result, the user can learn how to operate the work machine 60 efficiently.

[0242] (Effects of the eleventh invention) [Configuration 11] The computer 20 (see FIG. 1) displays the first action alone (above [Configuration 10]) on the output unit 30 (see FIG. 1) (step S52 in FIG. 3). Thereafter, the computer 20 displays the first action (playback action M1) and the second action (current action M2) simultaneously on the output unit 30 (step S53 in FIG. 3).

[0243] With the above [Configuration 11], after the user has confirmed the content of the reproduced motion M1, the user can operate the input unit 11 (see FIG. 1) (learn how to operate) while simultaneously displaying the reproduced motion M1 and the current motion M2. For example, before the user operates (practices) the input unit 11, the user can be made to understand the practice content, and then the user can operate (practice) the input unit 11. Therefore, the user can efficiently learn how to operate the work machine 60.

[0244] (Effects of the twelfth invention) [Configuration 12] The recording unit 21 (see FIG. 1) records a plurality of first actions of the work machine 60. The playback unit 23 (see FIG. 1) plays back one or more first actions (playback actions M1) from among the plurality of first actions recorded in the recording unit 21.

[0245] The above [Configuration 12] makes it possible to accommodate various recordings and various playbacks, improving the convenience of the work machine operation system 1 (see FIG. 1).

[0246] For example, the above [Configuration 12] may provide the following effect. For example, it becomes possible to select a first action to be played back (playback action M1) from a plurality of first actions (e.g., a plurality of model actions). For example, it becomes possible for the recording unit 21 (see FIG. 1) to record a current action M2 corresponding to a user's operation of the input unit 11 (see FIG. 1) while the playback unit 23 (see FIG. 1) plays back a recorded first action (playback action M1). For example, when the playback unit 23 plays back a plurality of first actions (playback actions M1), it becomes possible to simultaneously display the model played back action M1 and the recorded current action M12 (action in which the current action M2 of the work machine 60 resulting from the user's operation of the input unit 11 is recorded). For example, when the playback unit 23 plays back a plurality of first actions (playback actions M1), it becomes possible to display the current action M2 while displaying the model played back action M1 and the recorded current action M12.

[0247] (Effect of the thirteenth invention) [Configuration 13] The recording unit 21 (see FIG. 1) records the second action (current action M2) as the first action.

[0248] With the above [Configuration 13], the playback unit 23 (see FIG. 1) can record a current action M2 in response to the user's operation of the input unit 11 (see FIG. 1) while playing back the first action (playback action M1) recorded in the recording unit 21 (see FIG. 1). This allows the user to confirm (look back on) the action of the work machine 60 (recorded current action M12) caused by the user's operation of the input unit 11. As a result, the user can efficiently learn how to operate the work machine 60.

[0249] (Effect of the 14th invention) [Configuration 14] The recording unit 21 (see FIG. 1) records a series of first actions of the work machine 60. The calculator 20 divides the series of first actions into a plurality of chapters C, as shown in FIG. 13. The playback unit 23 (see FIG. 1) plays back the first action (playback action M1) for each chapter C.

[0250] The above [Configuration 14] allows the user to confirm (step by step, little by little) the series of operations of the work machine 60 shown in Figure 2 for each chapter C (see Figure 13). As a result, the user can learn how to operate the work machine 60 efficiently.

[0251] (Effect of the 15th invention) [Configuration 15] The playback unit 23 (see Figure 1) starts playback of the (n+1)th chapter C (see Figure 13) when the following [Condition A] is satisfied: [Condition A] The playback position of the first action reaches the end of the nth chapter C, and the difference between the state of the first work machine 60-1 (see Figure 6) performing the first action and the state of the second work machine 60-2 (see Figure 6) performing the second action (current action M2) falls within a predetermined range.

[0252] The above [Configuration 15] provides the following effect. The user can operate the input unit 11 (see FIG. 1) so that the difference between the state of the first work machine 60-1 and the state of the second work machine 60-2 shown in FIG. 6 falls within a predetermined range, and then the (n+1)th (next) chapter C (see FIG. 13) can be played back. This allows the user to learn how to operate the work machine 60 (in stages) for each chapter C. As a result, the user can learn how to operate the work machine 60 efficiently.

[0253] (Effect of the 16th invention) [Configuration 16] The computer 20 (see FIG. 1) causes the output unit 30 to simultaneously display the operation of the operating device 63 (see FIG. 7) and the operation of the part of the work machine 60 that is operated by the operating device 63.

[0254] The above [Configuration 16] allows the user to confirm (understand) the relationship between the operation (operation) of the operating device 63 (see FIG. 7) and the operation of the work machine 60. As a result, the user can efficiently learn how to operate the work machine 60.

[0255] (Effects of the 17th invention) [Configuration 17] The recording unit 21 (see FIG. 1) records the first actions of multiple work machines 60. The playback unit 23 simultaneously plays back the first actions (playback actions M1) of the multiple work machines 60 recorded by the recording unit 21.

[0256] The above [Configuration 17] allows the user to learn how to operate the work machine 60 when multiple work machines 60 are working together (collaborative work).

[0257] (Effects of the 18th invention) [Configuration 18] The input unit 11 shown in Fig. 1 is configured to be able to directly issue instructions regarding the recording state (recording state instructions) of the recording unit 21. The input unit 11 is configured to be able to directly issue instructions regarding the playback state (playback state instructions) of the playback unit 23.

[0258] In the above [Configuration 18], the user can directly instruct the recording state and the playback state using the input unit 11. For example, the user can instruct the recording state and the playback state by simply operating the input unit 11, without having to perform complicated operations on a GUI for instructing these (for example, a recording instruction unit G21 (see FIG. 4), a playback instruction unit G23 (see FIG. 3), etc.). Therefore, the user can efficiently (quickly and simply) instruct the recording state and the playback state.

[0259] (Variation) The above-described embodiments (including modified examples within the embodiments (the same applies hereinafter)) may be modified in various ways. For example, the number of components in the above-described embodiments may be changed, or some of the components may not be provided. For example, the connections between the components shown in FIG. 1 may be changed. For example, the inclusion relationships between the components may be changed in various ways. For example, a component described as a lower-level component included in a higher-level component may not be included in this higher-level component, but may be included in another component. For example, what is described as multiple different elements may be combined into a single element. For example, what is described as a single element may be provided as multiple different elements. For example, the order of the steps in the flowchart shown in FIG. 3 may be changed, or some of the steps may not be performed. For example, various information (values, ranges, etc.) may be set in advance in the computer 20 shown in FIG. 1, or may be set by being read into the computer 20 from an external storage device of the computer 20. The various information may be set in the computer 20 based on information manually set by an operator. The various information may also be automatically set by the computer 20 in accordance with certain conditions. For example, various pieces of information may not be changed, may be changed by manual operation, or may be changed automatically by the computer 20 in response to certain conditions. For example, the computer 20 may perform substantially the same processing (calculation, determination, etc.) as the processing of the above-described embodiment. For example, the processing procedures used in the processing, the information used in the processing, etc. may be changed in various ways. The processing performed by the computer 20 may be combined in various ways. For example, each component may have only a part of each feature (function, arrangement, shape, operation, etc.).

[0260] The work machine operation system 1 is configured to perform each of the above operations. A work machine operation program may be set that causes the computer 20 to execute processing to perform each of the above operations. A work machine operation method may be implemented that performs each of the above operations. Each of the above operations may be considered a "step" in the work machine operation program and work machine operation method. For example, the playback of the first operation of the work machine 60 by the playback unit 23 may be considered a "first operation playback step." [Explanation of symbols]

[0261] 1. Work machine operation system 11 Input section 20 Calculator 21 Recording Section 23 Playback Department 30 Output section 60 Work Machinery 60-1 First work machine (work machine performing first operation) 60-2 Second work machine (work machine performing second operation) 63 Operating device Chapter C M1 Playback operation (first operation played by the playback unit) M2 Current operation (second operation)

Claims

1. an output unit capable of displaying an image; an input unit capable of operating a work machine; A calculator and Equipped with The computer a recording unit that records a first operation, which is an operation of the work machine in response to an operation of the input unit; a playback unit that plays back the first action recorded in the recording unit; Equipped with the computer causes the output unit to display the first action reproduced by the reproduction unit and a second action that is an action of the work machine in accordance with an operation being performed on the input unit; Work machine operating system.

2. 2. The work machine operation system according to claim 1, each of the first operation and the second operation includes an operation of an operating device inside the work machine; Work machine operating system.

3. 2. The work machine operation system according to claim 1, The recording unit records the position of the travellable work machine. Work machine operating system.

4. 2. The work machine operation system according to claim 1, the recording unit records a surrounding situation at the start of a first action, which is a situation around the work machine at the start of recording of the first action; the reproduction unit sets the surrounding conditions of the work machine to the surrounding conditions at the start of the first operation at the start of reproduction of the first operation. Work machine operating system.

5. 2. The work machine operation system according to claim 1, a playback speed of the first action played back by the playback unit is variable; Work machine operating system.

6. 2. The work machine operation system according to claim 1, the computer calculates a difference between the first operation and the second operation, and causes the output unit to output information on the calculated difference; Work machine operating system.

7. 2. The work machine operation system according to claim 1, the computer causes the output unit to display one or both of the first operation and the second operation as a model of the work machine; Work machine operating system.

8. 2. The work machine operation system according to claim 1, the computer causes the output unit to display one or both of the first operation and the second operation as a graph; Work machine operating system.

9. 2. The work machine operation system according to claim 1, the computer causes the output unit to display the first action at a time point in the future relative to the first action at the time point being reproduced by the reproduction unit; Work machine operating system.

10. 2. The work machine operation system according to claim 1, the computer causes the output unit to display a first action only display that displays the first action but does not display the second action; Work machine operating system.

11. 11. A work machine operation system according to claim 10, the computer causes the output unit to display the first action alone, and then causes the output unit to simultaneously display the first action and the second action; Work machine operating system.

12. 2. The work machine operation system according to claim 1, the recording unit records a plurality of the first operations of the work machine; the reproduction unit reproduces one or more of the first actions among the plurality of first actions recorded in the recording unit; Work machine operating system.

13. 13. A work machine operation system according to claim 12, The recording unit records the second action as the first action. Work machine operating system.

14. 2. The work machine operation system according to claim 1, the recording unit records a series of the first operations of the work machine; The computer divides the series of first actions into a plurality of chapters; the playback unit plays back the first action for each chapter. Work machine operating system.

15. 15. A work machine operation system according to claim 14, comprising: the playback unit starts playback of the (n+1)th chapter when the playback position of the first action reaches the end of the nth chapter and the difference between the state of the work machine performing the first action and the state of the work machine performing the second action falls within a predetermined range. Work machine operating system.

16. 3. A work machine operation system according to claim 2, the computer causes the output unit to simultaneously display the operation of the operation device and the operation of the part of the work machine operated by the operation device. Work machine operating system.

17. 2. The work machine operation system according to claim 1, the recording unit records the first operations of the plurality of work machines; the playback unit simultaneously plays back the first actions of the plurality of work machines recorded by the recording unit. Work machine operating system.

18. 2. The work machine operation system according to claim 1, the input unit is configured to be able to directly give instructions regarding the recording state of the recording unit, and is configured to be able to directly give instructions regarding the reproduction state of the reproduction unit. Work machine operating system.

Images