Automated driving information processing device, automated driving information processing method, and automated driving information processing program

Abstract

The present invention facilitates the confirmation and input of information pertaining to an automatic operation of a work machine. This automatic operation information processing device 20 is connected in a communicable manner to a work machine 10 capable of an automatic operation. The automatic operation information processing device 20 comprises a display unit 23o and an input unit 23i. The display unit 23o displays information pertaining to an automatic operation. The input unit 23i receives the information pertaining to the automatic operation according to an operation of an operator.

Description

Technical Field

[0001] The present invention relates to an automatic driving information processing apparatus, an automatic driving information processing method, and an automatic driving information processing program that process information related to automatic driving of a working machine.

Background Art

[0002] For example, Patent Document 1 describes a working machine that performs automatic driving.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is desired that an operator can easily check and input information related to automatic driving of a working machine.

[0005] Therefore, an object of the present invention is to provide an automatic driving information processing apparatus, an automatic driving information processing method, and an automatic driving information processing program that can easily check and input information related to automatic driving of a working machine.

Means for Solving the Problems

[0006] The automatic driving information processing apparatus according to the first means is communicably connected to a working machine capable of performing automatic driving. The automatic driving information processing apparatus includes a display unit and an input unit. The display unit displays a display image including information related to the automatic driving. The input unit inputs information related to the automatic driving in response to an operator's operation.

[0007] The second means of the automated driving information processing method comprises a connection step, a display step, and an input step. The connection step connects to a work machine capable of automated driving in a communicative manner. The display step displays a display image containing information about the automated driving on a display unit. The input step acquires information about the automated driving that has been input to the input unit in response to an operator's operation.

[0008] The third means of the automated driving information processing program causes a computer to perform a connection step, a display step, and an input step. The connection step connects to a work machine capable of automated driving in a communicative manner. The display step displays a display image containing information about the automated driving on a display unit. The input step acquires information about the automated driving that has been input on an input unit in response to operator operations. [Effects of the Invention]

[0009] Each of the above methods 1 to 3 makes it easy to check and input information regarding the automatic operation of the work machine. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an automated driving system 1, including an automated driving information processing device 20. [Figure 2] Figure 1 shows the display image G displayed on the display unit 23o, and is a diagram of the basic screen. [Figure 3A] This figure shows the home screen of the display image G shown in Figure 2. [Figure 3B] Figure 2 shows the home screen when multiple work machines 10 are connected, as shown in the display image G. [Figure 4A] Figure 2 shows the top screen of the instruction display image G. [Figure 4B] This figure shows the screen before teaching the display image G shown in Figure 2. [Figure 4C] This figure shows the screen during the teaching process for the display image G shown in Figure 2. [Figure 5A] It is a diagram showing the trajectory confirmation screen of the display image G shown in FIG. 2. [Figure 5B] It is a diagram showing the trajectory change screen of the display image G shown in FIG. 2. [Figure 6] It is a diagram showing the target range setting screen of the display image G shown in FIG. 2. [Figure 7A] It is a diagram showing the initial posture preparation screen of the display image G shown in FIG. 2. [Figure 7B] It is a diagram showing the working machine state preparation screen of the display image G shown in FIG. 2. [Figure 7C] It is a diagram showing the automatic driving mode start notification screen of the display image G shown in FIG. 2. [Figure 8] It is a diagram showing the automatic driving initial setting screen of the display image G shown in FIG. 2. [Figure 9A] It is a diagram showing the screen before the start of automatic driving of the display image G shown in FIG. 2. [Figure 9B] It is a diagram showing the operating speed change screen of the display image G shown in FIG. 2. [Figure 9C] It is a diagram showing the number of working times change screen of the display image G shown in FIG. 2. [Figure 9D] It is a diagram showing the confirmation screen before the start of automatic driving of the display image G shown in FIG. 2. [Figure 10A] It is a diagram showing the screen during automatic driving of the display image G shown in FIG. 2. [Figure 10B] It is a diagram showing the display setting screen of the display image G shown in FIG. 2. [Figure 10C] It is a diagram showing the sensor status confirmation screen of the display image G shown in FIG. 2. [Figure 10D] It is a diagram showing the screen during automatic driving of the display image G shown in FIG. 2, including the operation phase display part PD, etc. [Figure 11] It is a diagram showing the simple screen during automatic driving of the display image G shown in FIG. 2. [Figure 12A] It is a diagram showing the automatic driving work end screen of the display image G shown in FIG. 2. [Figure 12B] It is a diagram showing the intermediate end screen of the display image G shown in FIG. 2. [Figure 13A]Figure 2 shows the emergency stop screen of the display image G. [Figure 13B] Figure 2 shows the emergency stop screen corresponding to the work being performed, as indicated by the display image G. [Figure 14A] Figure 2 shows the connection settings screen for the display image G. [Figure 14B] Figure 2 shows the new connection operation selection screen, as indicated by the display image G. [Figure 14C] Figure 2 shows the sensor status display screen for display image G. [Figure 14D] Figure 2 shows the accessory operation screen for the displayed image G. [Figure 15A] Figure 1 is a flowchart showing the operation of the autonomous driving system 1. [Figure 15B] This flowchart shows the operation of the automatic driving system 1 shown in Figure 1 when the all-machine stop selection unit F13A2 shown in Figure 13A is selected. [Modes for carrying out the invention]

[0011] The automated driving system 1 will be described with reference to Figures 1 to 15B.

[0012] As shown in Figure 1, the automated driving system 1 is a system for the automated driving of the work machine 10. The automated driving system 1 comprises the work machine 10 and the automated driving information processing device 20.

[0013] The work machine 10 is a machine that performs work, for example, a construction machine that performs construction work, and may be an excavator or a crane. The following description will focus on the case where the work machine 10 is an excavator. The work machine 10 is configured to be automatically operated. The work machine 10 may be operated by a worker (operator) in the driver's cab 13a (described later), or it may be remotely operated. The number of work machines 10 included in the automatic driving system 1 may be one or multiple. The work machine 10 comprises a machine body 10a and an attachment 15.

[0014] The machine body 10a is the main body of the work machine 10. The machine body 10a comprises a lower traveling body 11 and an upper rotating body 13. The lower traveling body 11 is capable of traveling on a traveling surface (such as the ground). The lower traveling body 11 may be equipped with crawlers or wheels. The upper rotating body 13 is rotatably mounted on the lower traveling body 11. The upper rotating body 13 comprises a cab 13a and a status display device 13b. The cab 13a is the part from which an operator can operate the work machine 10. The status display device 13b is a device that displays the status of the work machine 10 (described later). The status display device 13b is, for example, a rotating light, a screen, or a light. The status display device 13b is, for example, installed on the upper surface of the upper rotating body 13 (for example, the upper surface of the cab 13a).

[0015] The attachment 15 is the work unit and comprises, for example, a boom 15a, an arm 15b, and a tip attachment 15c. The boom 15a is mounted on the upper slewing body 13 in a luffable manner. The arm 15b is mounted on the boom 15a in a rotatable manner.

[0016] The tip attachment 15c is provided at the tip of the attachment 15 and is rotatably attached to the arm 15b. The tip attachment 15c performs work on the workpiece. The workpiece may be soil, sand, stone, wood, metal, resin, waste, or structure (such as a block). The tip attachment 15c is set to a type corresponding to the workpiece. For example, the tip attachment 15c may be a bucket used for scooping or excavating workpieces such as soil, granular, chipped, or powdered materials. The tip attachment 15c may also be a lifting magnet that uses magnetic force to attract and lift metal (workpiece). The tip attachment 15c may also be a device for gripping workpieces (such as a grapple or nibbler) or a device for crushing workpieces (such as a breaker).

[0017] (Types of work performed by machine 10) For example, the work machine 10 may capture an object to be worked on and move the captured object to a predetermined position. Specific examples of the work performed by the work machine 10 are as follows: The work machine 10 may also perform loading operations. Loading operations involve the work machine 10, equipped with a bucket (an example of a tip attachment 15c), excavating (capturing) soil (object to be worked on), unloading the excavated soil onto a transport vehicle (e.g., a dump truck), and loading it onto the vehicle. The work machine 10 may also perform lifting magnet operations. Lifting magnet operations involve the work machine 10, equipped with a lifting magnet (an example of a tip attachment 15c), attracting (capturing) metal (object to be worked on) and moving the captured metal to a predetermined position. The work machine 10 may also perform soil inversion operations. Soil inversion operations involve the work machine 10, equipped with a bucket or the like, inverting soil (object to be worked on). The work machine 10 may also perform structural transport operations. The work of transporting structures involves a work machine 10 equipped with a tip attachment 15c capable of gripping a structure (object to be worked on), gripping the structure, and placing the gripped structure in a predetermined position. A single work machine 10 may be capable of performing multiple types of work. Whether or not a work machine 10 can perform a particular type of work depends, for example, on the type of tip attachment 15c and the capabilities of the work machine 10.

[0018] The automated driving information processing device 20 is a device that processes information related to the automated driving of the work machine 10. The automated driving information processing device 20 may also have a function to operate the work machine 10 that is capable of automated driving. The automated driving information processing device 20 may also have a function to support the management of the work machine 10 that is capable of automated driving. The automated driving information processing device 20 is connected to the work machine 10 in a communicative manner. Hereinafter, a communicative connection will also be simply referred to as "connection". The automated driving information processing device 20 may be equipped with various information processing devices (equipment). Specifically, for example, the automated driving information processing device 20 may be equipped with a server 21 (computer) and an information terminal 23.

[0019] Server 21 is a computer (controller) that performs information input / output, calculations (processing), and information storage. For example, the functions of Server 21 are realized by the execution of programs stored in the Server 21's memory unit by its calculation unit. Server 21 is located outside the work machine 10 and the information terminal 23. Server 21 is connected to the work machine 10.

[0020] The information terminal 23 is connected to the work machine 10. The information terminal 23 may be connected to the work machine 10 via the server 21, or it may be connected to the work machine 10 without going through the server 21. Below, we will mainly describe the case in which the information terminal 23 is connected to the work machine 10 as an example of how the automated driving information processing device 20 is connected to the work machine 10. The information terminal 23 is a device capable of inputting and outputting information. The information terminal 23 may be portable or not. The information terminal 23 may be a tablet, a smartphone, or a personal computer. The information terminal 23 may also be used as the server 21. The information terminal 23 may be located outside the work machine 10, or it may be located inside the work machine 10 (for example, it may be brought in). The information terminal 23 comprises a control unit 23c (computer), an input unit 23i, and a display unit 23o.

[0021] The control unit 23c is a computer (controller) that performs signal input / output, calculations (processing), and information storage. For example, the functions of the control unit 23c are realized by the execution of a program stored in the memory unit of the control unit 23c by the calculation unit. The control unit 23c controls information related to the display unit 23o, for example, causing the display unit 23o to display information. The control unit 23c controls information related to the input unit 23i, for example, processing the information input to the input unit 23i. The control unit 23c may also perform other controls, such as information communication control. The control unit 23c may be distributed and arranged in multiple parts (it may constitute a distributed system). The control unit 23c may be provided integrally with one or both of the input unit 23i and the display unit 23o, or they may not be provided integrally. For example, the input unit 23i and the display unit 23o may be provided in the information terminal 23, and part or all of the control unit 23c may be provided in the server 21 or the like.

[0022] The input unit 23i receives information related to autonomous driving in response to the operator's actions. Specifically, for example, the input unit 23i may be a touch panel, a mouse, a keyboard, a device that inputs information based on the operator's line of sight, or a device that inputs information based on the position of the operator's fingers in space.

[0023] The display unit 23o displays information related to autonomous driving (display output unit). Specifically, for example, the display unit 23o may be a monitor (display), a device that projects images onto an object, or it may use AR (Augmented Reality) technology. The image displayed by the display unit 23o (the image that the control unit 23c causes the display unit 23o to display) is called the display image G. The display image G includes information related to autonomous driving. The display image G includes shapes, patterns, colors, characters, symbols, etc. The display image G includes a GUI (Graphical User Interface). The information terminal 23 may also be equipped with parts that perform output other than display (output units), for example, an audio output unit and a vibration output unit.

[0024] (Automatic operation of work machine 10) The work machine 10 is configured to be able to operate automatically based on a work plan. The work plan is information about the objectives of the work machine 10's operation and is pre-set in the automatic operation information processing device 20 (before automatic operation begins). The work plan may also include information about the target route for the work machine 10's travel. The work plan may also include information about the target range TA (for example, the target acquisition range TA1 and target release range TA3 shown in Figure 10D) in which the tip attachment 15c will perform its work.

[0025] (Target path PA, target trajectory) The work plan may include information on the target path PA (see Figure 10D) (also called the target trajectory) of a specific part of the attachment 15. The specific part of the attachment 15 may be, for example, the tip of the tip attachment 15c, or the base end of the tip attachment 15c (the tip of the arm 15b). The specific part of the attachment 15 may be set in only one location, or in multiple locations. The target path PA is information that includes, for example, information on the positions (coordinates) of multiple target points and information on the order of each target point. The work plan may also include information on the target trajectory of a specific part of the attachment 15. The above "target trajectory" is information that adds time information to the information on the target path PA.

[0026] (Cycle, work phase P) The work plan may include information for having the work machine 10 perform a certain task (e.g., loading work, lifting magnet work) in multiple cycles. One cycle may include multiple work phases P (see Figures 8 and 10D) (operation phases). A specific example of a work phase P is as follows. Here, a specific example of a work phase P is described when the work machine 10 automatically performs the task of capturing a work object and moving the captured work object to a predetermined position. In this case, as shown in Figure 10D, the work phase P includes a capture phase P1 (e.g., excavation phase), a lifting and rotating phase P2 (soil removal and rotating phase), a release phase P3 (soil removal phase), and a return and rotating phase P4. The capture phase P1 is the phase in which the tip attachment 15c captures a work object in the target capture range TA1 (e.g., excavating soil). For example, the target capture range TA1 is set to a place where work objects are collected (e.g., a pile of soil). The lifting and turning phase P2 (see Figure 8) is the phase in which the tip attachment 15c moves from the target acquisition range TA1 to the target release range TA3 while the tip attachment 15c has acquired the work object. The release phase P3 is the phase in which the tip attachment 15c releases the work object in the target release range TA3 (for example, by removing soil). The target release range TA3 may be set to a predetermined range on the bed of a transport vehicle (such as a dump truck), a predetermined range on a container other than a bed, or a predetermined range on the ground. The return and turning phase P4 is the phase in which the tip attachment 15c moves from the target release range TA3 to the target acquisition range TA1. For example, in automated driving, the series of work phases P (one cycle of automated driving work) consisting of the acquisition phase P1, the lifting and turning phase P2, the release phase P3, and the return and turning phase P4 is repeated.

[0027] (teaching) At least a portion of the work plan may be set by teaching, or by other methods (e.g., numerical input). Information set by teaching is called teaching data. Teaching data is set by an operator operating the work machine 10 shown in Figure 1. Specifically, for example, the operator may board the work machine 10 and operate it, or the operator may remotely operate the work machine 10. For example, by operating the work machine 10, the operator places a specific part of the attachment 15 at a location (path, range, etc.) that they want to set as teaching data. Then, based on the position where the specific part of the attachment 15 is placed, the teaching data is set in the automatic driving information processing device 20. For example, by operating the work machine 10, the operator places a specific part of the attachment 15 at a specific position within the range that they want to set as the target range TA (target acquisition range TA1, target release range TA3) shown in Figure 10D (see the explanation of Figure 6 below). The position of a specific part of the attachment 15 shown in Figure 1 is calculated based on the detection result of an attitude sensor (not shown) that detects the attitude of the work machine 10. Then, based on the position of the specific part of the attachment 15, the target range TA shown in Figure 10D is set. For example, the operator moves the specific part of the attachment 15 along the path that they want to set as the target path PA by operating the work machine 10 (see the explanations of Figures 4A, 4B, and 4C below). For example, the operator moves the specific part of the attachment 15 along the path that they want to set as the target trajectory at the speed they want to set as the target trajectory by operating the work machine 10. Then, the automatic driving information processing device 20 sets the path (or trajectory) along which the specific part of the attachment 15 has moved as the target path PA (or trajectory).

[0028] (Operation) The operation of the automatic driving information processing device 20 shown in Figure 1, the automatic driving information processing method, and the automatic driving information processing program will be described below. The following will mainly describe the display image G (see Figures 2 to 14D) that the control unit 23c displays on the display unit 23o. Each component shown in Figure 1 (work machine 10, automatic driving information processing device 20, etc.) will be explained with reference to Figure 1. The display image G shown in Figure 2 changes in various ways depending on the situation. The display image G comprises various parts (elements such as various selection units, various display units, and various setting units). The presence or absence of display of each part of the display image G, the manner of display, and whether or not selection or operation is possible by the input unit 23i (active or inactive) will vary in various ways depending on the situation. The following will first describe the parts of the display image G that are commonly displayed in many situations, and then describe the other parts and screen transitions.

[0029] Figure 2 shows the basic screen. The basic screen includes a task selection unit A, a task addition unit Aadd, a function selection unit B, a menu selection unit F2A1, an information terminal operation content display unit F2A2, a machine status display unit F2A3, and a communication status display unit F2A4. The basic screen also includes a home selection unit H, a main display area M, a machine selection unit T, and a machine addition unit Tadd.

[0030] The work selection unit A is the part that allows the operator to select the type of work that the work machine 10 will perform in automatic operation. The work selection unit A displays the types of work that can be selected by the input unit 23i.

[0031] This "selection" is a selection made by the operator operating the input unit 23i. Specifically, for example, if the input unit 23i is a touch panel, the selection is made by touching the work selection unit A. If the input unit 23i is a mouse or keyboard, the selection may also be made by placing a cursor on the work selection unit A and then performing an operation to confirm the selection (such as clicking or pressing a designated key). Alternatively, the selection may also be made by performing a specific mouse operation or keyboard operation (such as a shortcut key operation) to select the work selection unit A. Hereafter, the selection made by the operator operating the input unit 23i will also be simply referred to as "selection" (the same applies to "selection" and "setting" of parts other than the work selection unit A).

[0032] The selection result of the work type by the work selection unit A can be used for various processes. For example, if some or all of the application software used for autonomous driving differs depending on the type of work, the work selection result by the work selection unit A may be used to select which application software to use. In this case, the work selection unit A is the application selection unit (app selection unit). Also, for example, there may be cases where a work type and a work machine 10 are associated. In this case, when narrowing down the work machines 10 associated with a specific type of work from among multiple work machines 10, the work selection result by the work selection unit A may be used to select the type of work to be narrowed down. Also, for example, when newly connecting a work machine 10 to the information terminal 23, the work selection result by the work selection unit A may be used to select which type of work to associate the connected work machine 10 with (see explanation of Figure 14B).

[0033] The position of the work selection unit A in the display image G can be set in various ways. The position of the work selection unit A in the display image G may be the position shown or a position close to the position shown (the same applies to other parts). In the example shown in Figure 2, the work selection unit A is placed in the leftmost area of ​​the display image G (application selection area). The work selection unit A may also be placed in the main display area M (see loading work selection unit F3A1 and lifting magnet work selection unit F3A2 in Figure 3A). Note that the position of the work selection unit A may be different from the position shown (the same applies to other parts).

[0034] Task selection section A may include at least one of the following: shapes (shapes, patterns, colors, etc.), characters, symbols, or image changes (movements, etc.). Specifically, for example, task selection section A may include a shape like a button (button shape), or it may include an icon. Task selection section A may also include characters that describe the function of task selection section A ("Loading," "Lifting Magnet," etc.). Unless otherwise specified, the presence of shapes, characters, symbols, etc., is also the case for other parts.

[0035] When a work selection area A (a portion of the displayed image G) is selected, the selected work selection area A and the unselected portion are displayed in a way that allows the operator to distinguish between them (the same applies to other selectable portions). For example, at least one of the changes in shapes, characters, symbols, and images may differ between the selected portion and the unselected portion. Specifically, for example, the brightness of the selected portion may be higher (brighter) than the brightness of the unselected portion (it may be highlighted).

[0036] The work selection unit A displays the types of work that the work machine 10 will perform in automatic operation. Specifically, for example, if there are loading work and lifting magnet work as types of work, the work selection unit A includes a loading work selection unit A1 and a lifting magnet work selection unit A2. The loading work selection unit A1 is the part that allows the operator to select loading work as the "type of work". The lifting magnet work selection unit A2 is the part that allows the operator to select lifting magnet work as the "type of work".

[0037] The task addition section Aadd is selected when an additional type of task is available for selection in the task selection section A. When the task addition section Aadd is selected, the control unit 23c performs the function of adding a type of task. For example, the task addition section Aadd is positioned consecutively with the task selection section A, specifically, for example, below the task selection section A and adjacent to the task selection section A.

[0038] Function selection unit B is the part that allows the operator to select functions related to autonomous driving. For example, function selection unit B is located adjacent to work selection unit A (e.g., to the right). Function selection unit B is displayed for each of several functions. Function selection unit B allows the selection of functions corresponding to the type of work selected in work selection unit A. Specifically, for example, if the type of work selected in work selection unit A is loading work, function selection unit B includes teaching function selection unit B1, loading function selection unit B2, trajectory confirmation function selection unit B3, release range function selection unit B4, and setting unit B5. The operation of the autonomous driving information processing device 20 when each selection unit is selected will be described later. Note that a part for adding a new selection unit to function selection unit B (similar to the part for adding work unit Aadd) may be provided in the display image G. For example, function selection unit B may include multiple button-shaped figures.

[0039] The menu selection section F2A1 is the part that is selected when the menu screen (not shown) is displayed. The "menu screen" mentioned above is a screen that allows the operator to select the menu items to be displayed as display image G (see, for example, the explanations in Figures 10B and 10C).

[0040] The information terminal operation content display unit F2A2 displays information indicating what is currently being done (operation, settings, etc.) on the information terminal 23. Specifically, for example, the information terminal operation content display unit F2A2 displays information such as which function is selected in the function selection unit B, and what is being done in the function selected in the function selection unit B (operation settings in Figure 2).

[0041] The machine status display unit F2A3 displays the status of the work machine 10. The machine status display unit F2A3 displays the status of the work machine 10 in a way that is distinguishable to the operator. For example, the status of the work machine 10 displayed on the machine status display unit F2A3 may include statuses related to automatic operation, or statuses other than automatic operation. For example, the status of the work machine 10 displayed on the machine status display unit F2A3 may include whether the work machine 10 is normal or not. Specifically, for example, the status of the work machine 10 displayed on the machine status display unit F2A3 may include at least one of the following: emergency stop, lever lock engaged (a state in which the work machine 10 cannot be operated), automatic operation, and remote operation. In this case, if the status of the work machine 10 displayed on the machine status display unit F2A3 is "automatic operation," "remote manual," or "hydraulic lock," it may indicate that the work machine 10 is normal. Also, if the status of the work machine 10 displayed on the machine status display unit F2A3 is "emergency stop," it may indicate that the work machine 10 is not normal. Specifically, for example, the machine status display unit F2A3 may include an image in which a light corresponding to the status of the work machine 10 illuminates.

[0042] For example, a status display device 13b (see Figure 1) that displays the status of the work machine 10 may be mounted on the actual work machine 10. In this case, the image of the machine status display unit F2A3 shown in Figure 2 and the display of the status display device 13b of the actual work machine 10 are linked. Specifically, for example, the on or off state of the rotating light (an example of the status display device 13b) on the actual work machine 10 is linked to the on or off state in the image of the machine status display unit F2A3. For example, the color of the rotating light that is lit on the actual work machine 10 is linked to the color of the rotating light graphic in the image of the machine status display unit F2A3.

[0043] Here, depending on the relative position (e.g., angle) between the status display device 13b of the actual work machine 10 and the worker, it may be difficult for the worker to visually view the status display device 13b. Also, if the actual work machine 10 is located far from the worker, it may be impossible or difficult for the worker to visually view the status display device 13b. In such cases, it is impossible or difficult for the worker to distinguish (judge, understand) the state of the work machine 10 by visually viewing the status display device 13b. On the other hand, in this embodiment, the image of the machine status display unit F2A3 is linked to the display of the status display device 13b of the actual work machine 10. In this case, the worker can distinguish (judge, understand) the state of the work machine 10 by looking at the image of the machine status display unit F2A3 (for example, in their hand) on the display image G, even without looking at the status display device 13b of the actual work machine 10. As a result, the worker who has looked at the image of the machine status display unit F2A3 may take action to draw attention to the surroundings of the work machine 10 according to the state of the work machine 10, or may take action such as operating the information terminal 23.

[0044] The status of the work machine 10 may also be displayed in the machine selection unit T, which will be described later. In this case, the machine selection unit T may also be called the "machine status display unit".

[0045] The communication status display unit F2A4 displays the communication status of the information terminal 23. The communication status display unit F2A4 may display the communication status between the information terminal 23 and the work machine 10, or it may display the communication status between the information terminal 23 and the server 21. The communication status includes, for example, "good," "unstable," and "not connected." The communication status display unit F2A4 displays the communication status in a way that allows the operator to distinguish between them. For example, the communication status display unit F2A4 changes at least one of the icon's color and shape according to the communication status.

[0046] The home selection section H is the part that is selected when displaying the home screen (for example, Figure 3A or Figure 3B).

[0047] The main display area M occupies the majority (e.g., more than half) of the display image G. For example, when the work machine 10 is selected in the machine selection unit T, the main display area M displays information about the work machine 10 selected in the machine selection unit T. Also, for example, when a function is selected in the function selection unit B, the main display area M displays information about the function selected in the function selection unit B. Specific examples of the main display area M will be described later.

[0048] The machine selection section T is the part that allows the operator to select one work machine 10 from a group of work machines 10. For example, the machine selection section T is the part that allows the operator to select a work machine 10 whose information is displayed in the main display area M. For example, the machine selection section T is displayed at the top of the display image G. The machine selection section T may include a tab-shaped graphic. The machine selection section T may include icons, etc. (not shown). The machine selection section T may include text. The machine selection section T displays information to identify the work machine 10. For example, the machine selection section T may display the name of the work machine 10, or the number of the work machine 10, etc. The machine selection section T is displayed for each work machine 10. More specifically, the machine selection section T has one tab-shaped graphic for each work machine 10.

[0049] The machine selection unit T, like the work selection unit A described above, is displayed in a way that allows the operator to distinguish between selected and unselected machine selection units T. Specifically, for example, the brightness of the selected machine selection unit T is higher (brighter) than that of the unselected machine selection unit T (highlighted). In the example shown in Figure 2, two machine selection units T are displayed, comprising a first machine selection unit T1 and a second machine selection unit T2. The number of machine selection units T may be three or more. There may be cases where only one machine selection unit T is displayed. There may be cases where the machine selection unit T is not displayed in the display image G.

[0050] (Changes in the machine selection unit T according to the state of the work machine 10) The image on the machine selection unit T changes according to the state of the work machine 10. For example, the image on the machine selection unit T changes according to at least one of the following states: automatic operation is in progress, automatic operation has completed successfully, and automatic operation has come to an emergency stop. The image on the machine selection unit T may also change according to other states of the work machine 10. The image on the machine selection unit T is displayed in a way that allows the operator to distinguish the states of the work machine 10. Specifically, for example, the color of the machine selection unit T may differ depending on the state of the work machine 10.

[0051] (Changes in the machine selection unit T depending on whether a selection is made or not) The image of the machine selection unit T changes depending on whether the machine selection unit T corresponding to the work machine 10 whose state has changed is selected (already selected) or not selected (not selected). The machine selection unit T displays whether the machine selection unit T corresponding to the work machine 10 whose state has changed has been selected or not, so that the operator can distinguish between them. Specifically, for example, an unselected machine selection unit T may be displayed in a blinking manner (alternating between images of high and low brightness). A selected machine selection unit T may be displayed in a non-blinking manner. More specifically, for example, suppose the state of the work machine 10 corresponding to the unselected second machine selection unit T2 has changed (for example, automatic operation has completed successfully). At this time, the second machine selection unit T2 displays an indication that the state has changed (automatic operation has completed successfully) (for example, it turns blue) and also displays an indication that it is unselected (for example, it blinks). Then, when the second machine selection unit T2 is selected, the indicator (e.g., blue) showing that the state has changed (automatic operation has completed successfully) is maintained, the indicator (e.g., blinking) showing that it is not selected ends, and the indicator (e.g., blinking stops) showing that it is selected becomes the indicator (e.g., blinking stops).

[0052] (Display of machine selection unit T according to the type of work) The image on the machine selection unit T changes according to the type of work performed by the automated operation. The image on the machine selection unit T is displayed in a way that allows the operator to distinguish between different types of work. Specifically, for example, the image of the frame on the machine selection unit T (e.g., the color of the frame) may be changed according to the type of work associated with the work machine 10 corresponding to the machine selection unit T.

[0053] (Switching of machine selection unit T) A specific example of the transition of the display image G when the selected machine selection unit T (selected work machine 10) is switched is as follows. For example, suppose the first machine selection unit T1 is selected and information about the work machine 10 corresponding to the first machine selection unit T1 is displayed in the main display area M. At this time, the second machine selection unit T2 is selected (let's call this selection α). Then, information about the work machine 10 corresponding to the second machine selection unit T2 is displayed in the main display area M. At this time, the content displayed in the main display area M is the content that was last displayed in the main display area M when the second machine selection unit T2 was selected before the above selection (α). However, it is conceivable that a machine selection unit T other than the second machine selection unit T2 (first machine selection unit T1) may be selected when the screen transition is performed automatically (e.g., during communication connection, waiting until the initial posture is reached after the end of automatic operation). In this case, if the above "automatic screen transition state" ends before the selection (α) is performed, the content displayed in the main display area M will be the screen after the screen transition. Furthermore, if the "automatic screen transition state" described above has not ended when selection (α) is made, the content displayed in the main display area M will be the screen in the "automatic screen transition state" (screen transition in progress).

[0054] (Other machine selection section T and other indicators) If there are no work machines 10 connected to the information terminal 23, the machine selection unit T does not need to be displayed. If there is one work machine 10 connected to the information terminal 23, the machine selection unit T may or may not be displayed. If the connection between the information terminal 23 and the work machine 10 is disconnected, the machine selection unit T corresponding to the disconnected work machine 10 may be removed. If the first machine selection unit T1 disappears from the display image G, and there was a second machine selection unit T2 adjacent to the disappearing first machine selection unit T1 (for example, adjacent to the right), the adjacent second machine selection unit T2 may move to the position where the disappearing first machine selection unit T1 was displayed. If the number of connected work machines 10 increases, additional machine selection units T may be added, for example, a machine selection unit T may be added in a position adjacent to (for example, adjacent to the right of) an already displayed machine selection unit T.

[0055] (Not connected tab U) If no work machine 10 exists that corresponds to the machine selection unit T, an image similar to the machine selection unit T (the unconnected tab U (see Figure 14B)) may be displayed. The unconnected tab U is displayed in a way that distinguishes it from the machine selection unit T (the part where the work machine 10 corresponding to the machine selection unit T exists) and the operator. For example, the unconnected tab U may include the words "unconnected". Note that if a work machine 10 is connected to the information terminal 23, the unconnected tab U may or may not be displayed. If the unconnected tab U is selected, the settings to connect the work machine 10 (see Figure 14A) may be made.

[0056] The machine addition section Tadd is selected when a work machine 10 to be connected to the information terminal 23 is added. For example, the machine addition section Tadd is placed in a position adjacent to the machine selection section T (for example, to the right). When the machine addition section Tadd is selected, a new machine selection section T may be added, and settings for connecting the work machine 10 (see Figure 14A) may be made. The machine addition section Tadd may be displayed (or set to active) when it is possible to add a work machine 10 to the information terminal 23, and may not be displayed (or set to inactive) when it is not possible to add one. Specifically, for example, if there is a limit to the number of work machines 10 that can be connected to the information terminal 23, the machine addition section Tadd may be displayed when the number of connected work machines 10 has not reached the limit, and may not be displayed when the limit has been reached.

[0057] In the following sections, referring to Figures 3A to 14D, we will explain the parts (elements) of the displayed image G that differ from those explained primarily with reference to Figure 2, as well as provide specific examples of transitions within the displayed image G. Note that the transitions between screens and the conditions under which each screen is displayed can be modified in various ways.

[0058] Figure 3A shows the home screen. The home screen is displayed when the home selection unit H is selected. The home screen includes a loading operation selection unit F3A1 and a lifting magnet operation selection unit F3A2. The loading operation selection unit F3A1 and the lifting magnet operation selection unit F3A2 are each displayed in the main display area M, for example, as icons. The function selection unit B (see Figure 2) is not displayed on the home screen. In the example shown in Figure 3A, the machine selection unit T (see Figure 2) is not displayed.

[0059] Figure 3B shows the home screen when there are multiple connected work machines 10. This home screen includes a loading work selection unit F3B1, a lifting magnet work selection unit F3B2, and a unit count display unit F3B3.

[0060] The images in the work selection unit A (loading work selection unit F3B1 and lifting magnet work selection unit F3B2) change for each type of work, depending on the number of connected work machines 10 for each type of work. For example, the image in the work selection unit A changes so that the operator can distinguish whether or not there are connected work machines 10 for each type of work (0 or not). Specifically, for example, the brightness of the lifting magnet work selection unit F3B2 is lower (darker, grayed out) when the number of connected work machines 10 associated with lifting magnet work is 0 compared to when there are more than 0. The number display unit F3B3 is the part that shows the number of connected work machines 10 for each type of work. For example, the number display unit F3B3 is positioned to overlap with a part (for example, the upper right part) of the image (for example, the icon) of the loading work selection unit F3B1. In addition, similar to the work selection section A (loading work selection section F3B1, lifting magnet work selection section F3B2) displayed in the main display area M, the work selection section A on the left side of the display image G (application selection area) may also change according to the number of connected work machines 10. If there is a limit to the number of work machines 10 that can be connected to the information terminal 23, the image of the work selection section A may change depending on whether the number of connected work machines 10 has reached the limit (similar to the machine addition section Tadd described above). The all-machine stop selection section F3B4 will be described later.

[0061] The following section will primarily describe the case where the information terminal 23 is connected to the loading machine 10.

[0062] (Teaching function) Figure 4A shows the teaching top screen. The teaching top screen is the screen displayed (for example, the first screen displayed) when the teaching function selection unit B1 is selected. The teaching top screen includes a new teaching selection unit F4A1, a taught operation selection unit F4A2, a current status display unit F4A3, and a machine graphic display unit F4A4. The new teaching selection unit F4A1 is selected by the operator when new teaching is to be performed. The taught operation selection unit F4A2 is selected when the machine 10 is to be made to perform automatic operation based on teaching data (for example, teaching data that has been taught in the past) that has been set in the automatic operation information processing device 20. The current status display unit F4A3 indicates that the status of the machine 10 is being monitored. The machine graphic display unit F4A4 includes a graphic representing the machine 10. The image representing the work machine 10 displayed in the machine graphic display unit F4A4 may be a three-dimensional image (see Figure 9A) or a two-dimensional image, and only one work machine 10 may be displayed, or multiple work machines 10 may be displayed (see Figure 5A). The display mode of the machine graphic display unit F4A4 may be changeable (for example, changeable according to operation). For example, the viewing angle (viewpoint) of the work machine 10 displayed in the machine graphic display unit F4A4 may be changeable (see the rotatable display unit F5B2a in Figure 5B). For example, the work machine 10 displayed in the machine graphic display unit F4A4 may be enlarged and reduced. The machine graphic display unit F4A4 may also include a graphic of the work site where the work machine 10 performs its work (see Figure 9A). This work site graphic may be a three-dimensional image or a two-dimensional image, similar to the graphic of the work machine 10.

[0063] Figure 4B shows the pre-teaching screen. The pre-teaching screen is displayed before teaching is performed. The pre-teaching screen is displayed when the new teaching selection unit F4A1 is selected on the teaching top screen (Figure 4A). The pre-teaching screen comprises a current status display unit F4B1, a teaching start selection unit F4B2, and a cancellation selection unit F4B3. The current status display unit F4B1 displays the status related to teaching and indicates that teaching has not yet begun. The teaching start selection unit F4B2 is the part that is selected when a new teaching is to be started. The cancellation selection unit F4B3 is the part that is selected when a new teaching is to be canceled (not started).

[0064] Figure 4C shows the teaching screen displayed during teaching. The teaching screen includes a current status display unit F4C1, a teaching end selection unit F4C2, a cancel selection unit F4C3, and a machine graphic display unit F4C4. The current status display unit F4C1 indicates that teaching is in progress. The teaching end selection unit F4C2 is selected when teaching is to be terminated. When the teaching end selection unit F4C2 is selected, the teaching data obtained through teaching is set (stored, saved) in the automatic driving information processing device 20. The cancel selection unit F4C3 is selected (cancel button) when teaching is to be canceled (cancelled). For example, the cancel selection unit F4C3 is selected when the work machine 10 has been operated, but the data obtained by the operation is not to be used as teaching data. The machine graphic display unit F4C4 includes a graphic of the work machine 10 (similar to the machine graphic display unit F4A4 in Figure 4A). The graphic representation of the work machine 10 displayed on the machine graphic display unit F4C4 changes according to the movement of the work machine 10 during teaching (such as rotation and movement of the attachment 15). The machine graphic display unit F4C4 may also include a graphic representation (e.g., a curve) of the trajectory (path) of a specific part of the attachment 15 during teaching (see Figure 5A). In this case, the graphic representation of the trajectory of the specific part of the attachment 15 is displayed superimposed on the graphic representation of the work machine 10.

[0065] (Trajectory confirmation function) Figure 5A shows the trajectory confirmation screen (route confirmation screen, teaching trajectory confirmation screen). The trajectory confirmation screen is a screen that allows the operator to confirm the taught trajectory (route). The trajectory confirmation screen is displayed after teaching has been performed. The trajectory confirmation screen is displayed when the trajectory confirmation function selection unit B3 is selected. The trajectory confirmation screen comprises a current status display unit F5A1, a trajectory display unit F5A2, a machine figure display unit F5A3, a teaching completion selection unit F5A4, a trajectory change selection unit F5A5, and a re-teaching selection unit F5A6. The current status display unit F5A1 indicates that teaching has been completed. The trajectory display unit F5A2 displays a figure that shows the taught trajectory. The trajectory display unit F5A2 displays a figure that shows the trajectory superimposed on the machine figure display unit F5A3, which includes a figure representing the work machine 10. The figure that shows the trajectory in the trajectory display unit F5A2 may be multiple points or a line. The teaching completion selection unit F5A4 is selected when teaching is complete. When the teaching completion selection unit F5A4 is selected, the taught trajectory is set as teaching data. The trajectory change selection unit F5A5 is selected when the operator changes the taught trajectory. The re-teaching selection unit F5A6 is selected when the operator performs teaching again.

[0066] Figure 5B shows the trajectory change screen (route change screen). The trajectory change screen is displayed when the trajectory is changed. The trajectory change screen is displayed when the trajectory change selection unit F5A5 (see Figure 5A) is selected. The trajectory change screen comprises a current status display unit F5B1, a trajectory display unit F5B2, a trajectory change unit F5B4, a movement direction selection unit F5B6, a trajectory change selection unit F5B7, a reset selection unit F5B8, and a cancel selection unit F5B9. The current status display unit F5B1 indicates that the trajectory is being changed (screen).

[0067] The trajectory may be modified in various ways. Here, one example of how to modify the trajectory is described. The trajectory display unit F5B2 displays the trajectory in the same way as the trajectory display unit F5A2 shown in Figure 5A. The trajectory display unit F5B2 is displayed from at least one of the following viewpoints: a horizontal view (e.g., directly to the side) of the work machine 10, and a top-down view. In this case, the operator can easily grasp the positional relationship between the work machine 10 and the trajectory. It also makes it easier for the operator to understand which direction to change the trajectory when modifying it. The trajectory modification unit F5B4 shown in Figure 5B is a part of the trajectory that is subject to modification. For example, the trajectory modification unit F5B4 may be one point that constitutes the trajectory, or it may be a part of the area that constitutes the trajectory. When the trajectory modification unit F5B4 is selected, the trajectory modification unit F5B4 becomes movable (operable) in the display image G. The movement direction selection unit F5B6 is the part for moving the trajectory modification unit F5B4. The movement direction selection unit F5B6 allows the trajectory change unit F5B4 to be operated in multiple directions (for example, four directions (up, down, forward, backward, etc.)). The movement direction selection unit F5B6 may include shapes (for example, arrows). The movement direction selection unit F5B6 becomes selectable (active) when the trajectory change unit F5B4 is selected, and may be inactive or hidden when the trajectory change unit F5B4 is not selected. The trajectory change unit F5B4 may also be movable in the displayed image G by operations other than those of the movement direction selection unit F5B6. For example, the trajectory change unit F5B4 may be movable in the displayed image G by operations such as tapping and dragging on the touch panel, or dragging the mouse cursor.

[0068] The trajectory change selection unit F5B7 is selected when the trajectory change is confirmed (determined). When the trajectory change selection unit F5B7 is selected, the display image G returns to the trajectory confirmation screen (see Figure 5A). When the trajectory change is confirmed, the changed trajectory is set in the information terminal 23 (automatic driving information processing device 20) as teaching data used for the automatic driving of the work machine 10. The reset selection unit F5B8 is selected when the trajectory is returned to its original state. For example, if multiple parts of the trajectory (e.g., multiple points) are changed, when the reset selection unit F5B8 is selected, all changed parts may return to their original positions. For example, when the reset selection unit F5B8 is selected, only a part (e.g., one point) of the changed parts (e.g., multiple points) may return to its original position. Specifically, for example, the changed parts (e.g., multiple points) may return to their original positions in order from the last changed part. The cancel selection unit F5B9 is selected when the trajectory change is to be canceled. If the Cancel option F5B9 is selected, the displayed image G returns to the trajectory confirmation screen (see Figure 5A).

[0069] (Target range adjustment function) Figure 6 shows the target range setting screen. The target range setting screen is used to set the target range TA. In the example shown in Figure 6, the target range setting screen is the release range setting screen (soil discharge position adjustment screen), which is used to set the range of positions (target release range TA3) for releasing the captured work object. For example, the release range setting screen is used to set the position (soil discharge position) for loading the excavated soil onto the vehicle. The release range setting screen is displayed when the release range function selection unit B4 (soil discharge position adjustment function selection unit) is selected. The "setting" of the release range may be an adjustment (correction) of a pre-set release range, or it may be the setting of a new release range. For example, teaching may be performed and the release range may be set. Alternatively, for example, no teaching may be performed and the release range may be set. The release range setting screen includes a release range setting method display unit F6A1, a setting information display unit F6A2, registration selection units F6A5 and F6A6, a relative position display unit F6A7, and a setting completion selection unit F6A8.

[0070] The release range setting method display unit F6A1 displays the method for setting the release range (area where soil is to be excavated). The release range setting method display unit F6A1 displays which position within the release range to specify. The release range setting method display unit F6A1 displays how the specified positions F6A1a and F6A1b should be specified (specific examples will be described later). The setting information display unit F6A2 displays information about the set specified positions F6A1a and F6A1b. The setting information display unit F6A2 may display the coordinates of the specified positions F6A1a and F6A1b, or it may display the rotation angle of the upper rotating body 13 relative to the lower traveling body 11. The setting information display unit F6A2 displays information for each of the multiple specified positions F6A1a and F6A1b. The registration selection units F6A5 and F6A6 are selected when a predetermined state for registering the specified position is reached, specifically, for example, when a specific part of the attachment 15 is positioned at the specified positions F6A1a and F6A1b. For example, when registration selection units F6A5 and F6A6 are selected, the positions of specific parts of the attachment 15 are registered (set) in the information terminal 23 as the positions of designated positions F6A1a and F6A1b. Registration selection units F6A5 and F6A6 are displayed for each of the multiple designated positions F6A1a and F6A1b.

[0071] A specific example of how to set the release area (area where soil will be removed) is as follows. In this example, the release area is a rectangle when viewed from above. The release area is set by specifying the positions F6A1a and F6A1b, which are the two corners of the release area that are closer to the viewer from the work machine 10. Alternatively, the release area may be set by specifying three or more points (for example, four points) within the release area. In this example, the specified positions are registered in the order of specified position F6A1a and specified position F6A1b. When registering specified position F6A1a, the release area setting method display unit F6A1 indicates that a specific part of the attachment 15 (the tip of the bucket in Figure 6) should be aligned with specified position F6A1a. The operator operates the work machine 10 to align the specific part of the attachment 15 with specified position F6A1a. In this state, the registration selection unit F6A5 for specified position F6A1a is selected. Then, the position of the specified position F6A1a is registered, the information of the specified position F6A1a is displayed on the setting information display unit F6A2, and the registration selection unit F6A5 for the specified position F6A1a becomes inactive. Then, the registration selection unit F6A6 for the next specified position F6A1b becomes active. Next, the registration of the specified position F6A1b is performed in the same way as when the specified position F6A1a was registered. At this time, the release range setting method display unit F6A1 indicates that the bucket cutting edge should be aligned with the specified position F6A1b.

[0072] The relative position display unit F6A7 displays a figure indicating the relative position between the release range (area where soil will be removed) and the work machine 10. The setting completion selection unit F6A8 is selected when the setting of the release range is completed. For example, the setting completion selection unit F6A8 becomes active when all the information for multiple specified positions F6A1a and F6A1b has been set, and remains inactive until then.

[0073] In addition, similar to the release range, a capture range (target capture range TA1 (see Figure 10D), for example, the area to be excavated) in which the work machine 10 captures the work object may be set. The function selection unit B may include a part (capture position function selection unit) (not shown) for selecting a function to set the capture range.

[0074] (Preparing for autonomous driving) Once the work plan (e.g., teaching data) used in autonomous driving is finalized, preparations are made to change the operating mode of the work machine 10 to the autonomous driving mode (the mode in which the work machine 10 operates autonomously). The operating mode of the work machine 10 is set, for example, in the controller of the work machine 10. The operating mode of the work machine 10 may include, for example, a mode in which an operator is on board the work machine 10 and operates it (onboard operation mode). The operating mode of the work machine 10 may also include a mode in which an operator remotely controls the work machine 10 (remote operation mode).

[0075] The conditions for initiating automatic driving mode (automatic driving mode initiation conditions) can be set in various ways. For example, the automatic driving mode initiation conditions include at least one of the following: the work machine 10 is returned to its initial position, the engine is turned off, the key is turned off, the remote control connection is disconnected if it was remotely controlled, and the emergency stop switch is powered on. Here, we will describe the case where the automatic driving mode initiation conditions include all of the above conditions.

[0076] The conditions for starting the automatic driving mode include the work machine 10 being set to its initial position. The work machine 10 may be set to its initial position by operator operation (onboard operation or remote operation) or it may be set to its initial position automatically. Here, we will describe the case where the work machine 10 is set to its initial position by operator operation.

[0077] Figure 7A shows the initial posture preparation screen. The initial posture preparation screen is displayed when the work machine 10 is set to its initial posture. The initial posture preparation screen is displayed, for example, after the work plan (e.g., teaching data) has been finalized. The initial posture preparation screen comprises a notification unit F7A1, an initial posture display unit F7A2, an initial posture preparation completion notification unit F7A3, and an initial posture confirmation selection unit F7A4. The notification unit F7A1 is the part that notifies the operator that the work machine 10 should be set to its initial posture. The notification unit F7A1 notifies the operator that the "initial posture" is the posture displayed on the initial posture display unit F7A2. The initial posture display unit F7A2 is the part that notifies the operator what the initial posture is. The initial posture display unit F7A2 notifies the initial posture, for example, using a graphic. In the illustrated example, the initial position is such that the boom 15a is raised to approximately 45 degrees relative to the ground, the arm 15b is lowered almost vertically from the boom 15a, and the opening of the tip attachment 15c (bucket) is almost horizontal. Furthermore, the initial position is such that the tip attachment 15c is positioned directly above the area for capturing the work object (the area to be excavated). The initial position preparation completion notification unit F7A3 displays whether the work machine 10 has reached the initial position so that the operator can distinguish whether or not. For example, the initial position preparation completion notification unit F7A3 may include an icon, and the color of the icon may change depending on whether or not the work machine 10 has reached the initial position. The initial position preparation completion notification unit F7A3 may also include text such as "OK" or "NG". The initial position confirmation selection unit F7A4 is the part (OK button) that is selected when the work machine 10 has reached the initial position. The initial posture confirmation selection unit F7A4 is inactive when the work machine 10 is not in its initial posture, and may become active when the work machine 10 is in its initial posture.

[0078] Figure 7B shows the machine state preparation screen. The machine state preparation screen is displayed when the initial posture confirmation selection unit F7A4 (see Figure 7A) is selected. The machine state preparation screen is displayed, for example, after the work plan (teaching data) has been finalized. The machine state preparation screen comprises a condition notification unit F7B1 and a condition confirmation selection unit F7B2. The condition notification unit F7B1 notifies (displays) the conditions for starting the automatic operation mode, other than the machine 10 being placed in its initial posture. The condition notification unit F7B1 notifies that the engine should be turned off (not driven). The condition notification unit F7B1 notifies that the operator needs to be sufficiently far away from the machine 10. The condition notification unit F7B1 notifies that the key should be turned off (power off). The condition notification unit F7B1 notifies that the connection between the remote control device and the machine 10 should be disconnected if the machine 10 is in a state where it can be remotely controlled (for example, when teaching has been performed by remote control). The condition notification unit F7B1 notifies that the emergency stop switch (not shown) should be turned on. The emergency stop switch is held by an operator near the work machine 10 and is used to emergency stop the work machine 10. The condition confirmation selection unit F7B2 is the part (OK button) that is selected when the conditions displayed on the condition notification unit F7B1 are met. The condition confirmation selection unit F7B2 may become inactive when the automatic operation mode start conditions are not met and become active when the automatic operation mode start conditions are met.

[0079] Figure 7C shows the automatic driving mode start notification screen. The automatic driving mode start notification screen is displayed when the automatic driving mode is started. The automatic driving mode start notification screen is displayed when the condition notification unit F7B1 (see Figure 7B) is selected. The automatic driving mode start notification screen comprises a start notification unit F7C1 and a start confirmation selection unit F7C2. The start notification unit F7C1 is the part that notifies that the automatic driving mode will be started. The start confirmation selection unit F7C2 is the part (OK button) that is selected when it is confirmed that the automatic driving mode will be started.

[0080] A notification unit, such as the start notification unit F7C1, which prevents processing from proceeding unless the operator performs a confirmation operation (such as pressing the OK button), may be displayed on the display image G at various times other than when the automatic driving mode is started. Such notification units may be displayed, for example, at the start and end of teaching, the start and end of trajectory changes, and the start and end of various processes described later. Specifically, the confirmation operation in the notification unit may be, for example, pressing the OK button (an operation to select a selection unit), or pressing the OK button with a checkbox checked.

[0081] (Initial settings) Figure 8 shows the automatic operation initial setup screen. The automatic operation initial setup screen is a screen for performing initial settings related to automatic operation. For example, the automatic operation initial setup screen is displayed after the operation mode of the work machine 10 is switched to automatic operation mode (for example, initially). For example, the automatic operation initial setup screen is displayed when the loading function selection unit B2 is selected (for example, initially). The automatic operation initial setup screen comprises an operation setting unit F8A1 and an initial setup completion selection unit F8A2. The operation setting unit F8A1 is the part that allows the operator to set the automatic operation. The operation setting unit F8A1 comprises a work count setting unit C1, a target work amount setting unit C2, and an operating speed setting unit SS. The initial setup completion selection unit F8A2 is the part (OK button) that is selected when the initial setup of automatic operation is completed.

[0082] The work count setting unit C1 is the part that allows the operator to set the number of times to perform work by automatic operation (number of operations, how many cycles of automatic operation to perform). As described above, "setting" is done by the operator operating the input unit 23i, similar to "selection". "Setting" may be done in the same way as the specific example of "selection", or it may be done by numerical input, etc. The work count setting unit C1 includes a part that displays the number of operations (setting value display unit). The work count setting unit C1 may also include a part (input box) into which a numerical value for the number of operations can be entered. This input box may also be used for the setting value display unit (the same applies to the target work amount setting unit C2). The work count setting unit C1 includes a selection unit (+ button) to increase the number of operations and a selection unit (- button) to decrease the number of operations.

[0083] The target workload setting unit C2 is the part that allows the operator to set the amount of work object to be released (target workload) in the operation in which the work machine 10 moves and releases the captured work object. For example, the target workload may be the mass or volume of the work object. The target workload may be, for example, the amount of soil to be loaded onto a vehicle in loading work. The target workload setting unit C2 includes a part that displays the target workload (set value display unit). The target workload setting unit C2 may also include a part that allows input of a numerical value for the target workload (input box). The target workload setting unit C2 includes a selection unit (+ button) to increase the target workload and a selection unit (- button) to decrease the target workload.

[0084] The operating speed setting unit SS is the part that allows the operator to set the operating speed for automatic operation. The operating speed setting unit SS displays the operating speed for automatic operation from multiple operating speed levels. The "multiple operating speed levels" mentioned above include the operating speed of the work machine 10 in the teaching data (operating speed during teaching). In the example shown in Figure 8, the "multiple levels" are three levels (high speed, medium speed, low speed), but there may also be two levels or four or more levels.

[0085] Of the multiple operating speed settings that can be set in the operating speed setting unit SS, the fastest operating speed (third speed setting unit SS3) is preferably the operating speed of the work machine 10 in the teaching data. In this case, the operating speed during automatic operation will never be faster than the taught operating speed. Therefore, the operating speed during automatic operation will be less than or equal to the speed intended by the operator who provided the teaching, and less than or equal to the speed expected by the operator who viewed the teaching.

[0086] Of the multiple operating speed settings that can be set in the operating speed setting unit SS, the fastest operating speed (third speed setting unit SS3) may be faster than the operating speed of the work machine 10 in the teaching data. In this case, the work efficiency of the work machine 10 by automatic operation can be improved. For example, during teaching, the work machine 10 can be reliably operated at a slow operating speed, and during automatic operation, the work machine 10 can be operated efficiently at an operating speed faster than that of the teaching.

[0087] The operating speed setting unit SS can set the operating speed of automatic operation as follows. The operating speed setting unit SS may have multiple selection units that allow the operating speed of automatic operation to be selected from multiple stages. In the example shown in Figure 8, the operating speed setting unit SS includes a first speed setting unit SS1 (low speed), a second speed setting unit SS2 (medium speed), and a third speed setting unit SS3 (high speed). The operating speed setting unit SS may also allow the operating speed of automatic operation to be set by numerical input (not shown, similar to the operation count setting unit C1 and the target work amount setting unit C2). The operating speed setting unit SS may also allow setting the ratio of the operating speed of automatic operation to the operating speed in the teaching data. The operating speed setting unit SS may also allow setting the ratio of the speed in automatic operation to the fastest operating speed among the multiple operating speed stages (an operating speed equal to or greater than the operating speed in the teaching data).

[0088] The operating speed setting unit SS may allow the operating speed to be set for each work phase P. More specifically, if there are multiple work phases P in one cycle of automatic operation, the operating speed setting unit SS is displayed for each work phase P. In the example shown in Figure 8, one cycle of automatic operation comprises four work phases P. Specifically, for example, the work phases P include a capture phase P1 (excavation phase), a lifting and rotating phase P2 (soil removal and rotating phase), a release phase P3 (soil removal phase), and a return and rotating phase P4. For each of the four work phases P, the first speed setting unit SS1, the second speed setting unit SS2, and the third speed setting unit SS3 are displayed.

[0089] The state in which all stages of the multi-stage operating speed of each of the multiple work phases P are displayed on the display image G (as shown in Figure 8) is defined as the first speed display state SP1.

[0090] After the operating speed is set by the operating speed setting unit SS, the control unit 23c commands the work machine 10 to perform automatic operation at the operating speed set by the operating speed setting unit SS. The control unit 23c commands the work machine 10 to perform automatic operation at the operating speed set by the operating speed setting unit SS for each work phase P.

[0091] (Settings before starting autonomous driving) Figure 9A shows the screen before automatic operation starts. The screen before automatic operation starts is displayed after the initial setup of automatic operation (see Figure 8) is completed. The screen before automatic operation starts is displayed when the initial setup completion selection unit F8A2 (see Figure 8) is selected. The screen before automatic operation starts may also be displayed when the start of automatic operation is canceled (when the cancellation selection unit F9D3 in Figure 9D is selected), or when setting changes (see Figures 9B and 9C) are completed. The screen before automatic operation starts includes a current status display unit F9A1, a work count display unit F9A2, a target work amount display unit F9A3, and an operating speed display unit F9A4.

[0092] The current status display unit F9A1 indicates that the system is preparing to start automatic operation. The work count display unit F9A2 displays the set number of work operations. This "set number of work operations" is the number of work operations set in the initial settings (see Figure 8). However, if the number of work operations set in the initial settings is changed (see Figure 9C), the "set number of work operations" will be the changed number of work operations. The target work amount display unit F9A3 displays the set target work amount (target soil volume). This "set target work amount" is the number of work operations set in the initial settings (see Figure 8). However, if the target work amount set in the initial settings is changed, the "set target work amount" will be the changed target work amount.

[0093] The operating speed display unit F9A4 displays the set operating speed. This "set operating speed" is the operating speed for automatic operation set in the initial settings (Figure 8). However, if the operating speed for automatic operation set in the initial settings is changed (see Figure 9B), the changed operating speed will be displayed. The operating speed display unit F9A4 displays the operating speed for each work phase P. Hereafter, unless otherwise specified, the operating speed for automatic operation will also be simply referred to as "operating speed".

[0094] Figure 9B shows the operating speed change screen. The operating speed change screen is displayed when the set operating speed is changed. The operating speed may be changed for each work phase P, or it may be changed for the entire cycle (across multiple work phases P). The following describes the case where the operating speed is changed for each work phase P. The operating speed change screen comprises an operating speed setting unit F9BSS and an operating speed display unit F9B2. For the selected work phase P, the operating speed setting unit F9BSS displays the operating speed for automatic operation from multiple operating speed (candidates) and allows setting (changing). On the other hand, for work phase P that is not selected, the set operating speed is displayed in the operating speed display unit F9B2 (similar to the operating speed display unit F9A4 in Figure 9A).

[0095] This section explains the operation of the operating speed setting unit F9BSS and provides specific examples of images. From the operating speed display unit F9A4 shown in Figure 9A, the work phase P to be changed (for example, the capture phase P1) is selected. At this time, an image is displayed showing the operating speed setting unit F9BSS for the selected work phase P (capture phase P1) being pulled out (for example, pulled out to the right) from the operating speed display unit F9A4 of the selected work phase P (capture phase P1), as shown in Figure 9B. For example, if the operating speed can be set (selected) in three stages, the first speed setting unit F9BSS1 (low speed), the second speed setting unit F9BSS2 (medium speed), and the third speed setting unit F9BSS3 (high speed) are displayed for the selected work phase P (capture phase P1). When the operating speed is set using the operating speed setting unit F9BSS, the display of the operating speed setting unit F9BSS ends (closes), and the operating speed display unit F9A4 shown in Figure 9A is displayed (the display returns).

[0096] As shown in Figure 9B, when the operating speed setting unit F9BSS for a selected work phase P (e.g., capture phase P1) is displayed (open), it is possible that the operating speed display unit F9B2 for a previously unselected work phase P may be selected. In this case, for the selected work phase P (capture phase P1), the operating speed setting unit F9BSS becomes hidden (closed), and the operating speed display unit F9A4 (see Figure 9A) is displayed (the display returns). On the other hand, for the newly selected work phase P, the display of the operating speed display unit F9B2 changes to the display of the operating speed setting unit F9BSS (opens).

[0097] As shown in Figures 9A and 9B, the second speed display state SP2 is defined as the state in which the operating speed is displayed only at the stage set by the operating speed setting unit SS (see Figures 8 and 9B) for at least some of the multiple work phases P. More specifically, in the example shown in Figure 9A, the operating speed display unit F9A4 displays the operating speed only at the stage set by the operating speed setting unit SS for all of the multiple work phases P. In the example shown in Figure 9B, the operating speed display unit F9B2 displays the operating speed only at the stage set by the operating speed setting unit SS for some of the multiple work phases P (excluding the capture phase P1). It may also be possible to transition the display image G back to the first speed display state SP1 (return to the initial setting) after entering the second speed display state SP2.

[0098] Figure 9C shows the operation count change screen. The operation count change screen is displayed when the set operation count is changed. The operation count change screen comprises an operation count change unit F9C1 and a target operation amount display unit F9C2. The operation count change unit F9C1 is displayed when the operation count display unit F9A2 shown in Figure 9A is selected. For example, when the operation count display unit F9A2 is selected, an image is displayed in which the operation count change unit F9C1 shown in Figure 9C is pulled out (for example, pulled down) from the operation count display unit F9A2. The operation count change unit F9C1 is displayed in a way that allows the operation count to be changed. For example, the operation count change unit F9C1 may include a selection unit to increase the operation count (+ button) and a selection unit to decrease the operation count (- button), and may also include a section where the changed operation count can be numerically input. The operation count change unit F9C1 includes a confirmation selection unit (OK button) for confirming the change in the operation count. When this selection unit is selected, the operation count change unit F9C1 is hidden (closed), and the operation count display unit F9A2 (see Figure 9A) is displayed (the display returns to normal). Similarly, if another part of the display image G (for example, the target work amount display unit F9C2) is selected while the operation count change unit F9C1 is displayed, the operation count change unit F9CT1 is hidden, and the operation count display unit F9A2 (see Figure 9A) is displayed (the display returns to normal).

[0099] Similar to changing the number of operations using the operation count change unit F9C1, the target workload may also be changeable. For example, when the target workload display unit F9C2 is selected, the section for changing the target workload (target workload change unit) may be displayed.

[0100] The operating speed, number of operations, and target workload may be changed only when the work machine 10 is not operating automatically (for example, while preparing for automatic operation). Alternatively, the operating speed, number of operations, and target workload may be changed while the work machine 10 is operating automatically (simultaneously with automatic operation).

[0101] The screen shown in Figure 9C includes an initial posture readiness notification unit F9C3, a key state selection unit F9C4, an engine state selection unit F9C5, a lever lock state selection unit F9C6, and a work start selection unit F9C7. The initial posture readiness notification unit F9C3 displays whether the work machine 10 is in the initial posture or not, allowing the operator to distinguish between the two (similar to the initial posture readiness notification unit F7A3 in Figure 7A). It is assumed that the work machine 10 is in the initial posture when entering automatic operation mode. The operator can confirm that the work machine 10 is maintaining the initial posture using the initial posture readiness notification unit F9C3. The key state selection unit F9C4 is the part selected when switching the key state between on and off. The key state selection unit F9C4 is also the part that displays the key state (key state display unit). The engine state selection unit F9C5 is the part selected when switching the engine state between on (RUN) (running state) and off (stopped state). The engine status selection unit F9C5 also serves as the engine status display unit. The lever lock status selection unit F9C6 is selected when switching the lever lock status between the locked state (the state in which the work machine 10 cannot be operated) and the unlocked state (the state in which the work machine 10 can be operated). The lever lock status selection unit F9C6 also serves as the lever lock status display unit.

[0102] The conditions for initiating automatic operation (automatic operation start conditions) can be set in various ways. For example, the automatic operation start conditions include at least one of the following: the operating mode of the work machine 10 is automatic operation mode, the work machine 10 is in its initial position, the key is on, the engine is on, and the lever lock is in the released state. In the example shown in Figure 9C, the key is off, the engine is off, and the lever lock is locked, so the automatic operation start conditions are not met. Therefore, the information terminal 23 is operated by the operator to satisfy the automatic operation start conditions. Specifically, for example, the key state selection unit F9C4 is selected, and the key is changed from off to on. The engine state selection unit F9C5 is selected, and the engine is changed from off to on (RUN). The lever lock state selection unit F9C6 is selected, and the lever lock is changed from the locked state to the released state. A confirmation screen (similar to Figure 7C) may be displayed when at least one of the key state selection unit F9C4, engine state selection unit F9C5, or lever lock state selection unit F9C6 is operated. For example, notifications may be given when the state of the work machine 10 changes, such as when the engine changes from off to on. Notifications may also be given to ensure that there are no people around the work machine 10.

[0103] The relationship between the key state and the engine state can be configured in various ways. For example, the condition for the engine to turn on may include the key being in the "on" position. The engine may also automatically turn on when the key is turned on. When the engine switches from on to off, the key may also turn off, or it may remain in the "on" position.

[0104] The operation start selection unit F9C7 is the part that is selected when automatic operation is started. For example, the operation start selection unit F9C7 becomes active when the automatic operation start conditions are met, and may remain inactive when the automatic operation start conditions are not met.

[0105] Figure 9D shows the pre-automatic driving confirmation screen. The pre-automatic driving confirmation screen is displayed before automatic driving starts (for example, immediately before). The pre-automatic driving confirmation screen is displayed when the operation start selection unit F9C7 (see Figure 9C) is selected. The pre-automatic driving confirmation screen comprises an automatic driving start notification unit F9D1, a start selection unit F9D2, and a cancel selection unit F9D3. The automatic driving start notification unit F9D1 is the part that notifies (displays) that automatic driving will start. The start selection unit F9D2 is the part (OK button) that is selected when automatic driving is to start. The cancel selection unit F9D3 is the part (cancel button) that is selected when automatic driving is not to start.

[0106] (Autonomous driving in progress) Figure 10A shows the automatic operation screen (first screen, automatic operation details screen). The automatic operation screen is displayed during automatic operation. The automatic operation screen may also be displayed when the work machine 10 is in standby mode. The automatic operation screen may also be displayed when the work machine 10 has made an emergency stop (emergency stop). The automatic operation screen includes a current status display unit F10A1, a remaining work count display unit F10A2, a target work amount display unit F10A3a, and a current work amount display unit F10A3b. The automatic operation screen also includes a machine information display unit F10A4, a viewpoint change selection unit F10A5, a simple screen selection unit F10A6, a horn operation unit F10A7, and an intermediate termination selection unit F10A8. The current status display unit F10A1 indicates that the work machine 10 is operating under automatic operation. The remaining number of operations display unit F10A2 displays the remaining number of operations to be performed by automatic operation. This "remaining number of operations" is the set number of operations (cycles) minus the number of operations that have been completed. The remaining time for operations display unit F11A2 (see Figure 11) may also be displayed. The target workload display unit F10A3a is the same as the target workload display unit F9A3 shown in Figure 9A. The current workload display unit F10A3b displays the amount of work completed (amount of work completed) (for example, the amount of soil loaded). The current workload display unit F10A3b may display the amount of work completed, or it may display content corresponding to the amount of work completed. The current workload display unit F10A3b may also display the ratio of the amount of work completed to the target workload.

[0107] The machine information display unit F10A4 is a part that displays information related to the work machine 10. For example, the machine information display unit F10A4 displays information detected by sensors mounted on the work machine 10 or sensors located outside the work machine 10. The machine information display unit F10A4 may also display information related to the work angles (such as the rotation angle and the angle relative to the horizontal). The machine information display unit F10A4 may also display information related to the quantity of the work object, for example, it may display the quantity of the work object that the work machine 10 captures (excavates) in one cycle (such as the amount of excavated soil). The viewpoint change selection unit F10A5 is a part that is selected when changing the viewpoint of the figure of the work machine 10 displayed in the main display area M. Each time the viewpoint change selection unit F10A5 is selected, the viewpoint of the work machine 10 displayed in the main display area M may be switched. The viewpoint change selection unit F10A5 may be displayed for each viewpoint of the work machine 10 displayed in the main display area M (for example, it may have multiple buttons). The viewpoint of the work machine 10 displayed in the main display area M may include a viewpoint of the work machine 10 viewed from diagonally above (diagonal view), a viewpoint viewed from the horizontal (see Figure 5), or a viewpoint viewed from above (see Figure 5).

[0108] The simple screen selection section F10A6 is selected when the displayed image G is switched to the simple screen (second screen) (Figure 11) during automatic operation. The horn operation section F10A7 is selected when the work machine 10 is made to sound its horn. The intermediate termination selection section F10A8 is selected when the automatic operation is terminated midway (details will be described later).

[0109] Figure 10B shows the display settings screen. The display settings screen is displayed, for example, when the menu selection unit F2A1 (see Figure 10A) is selected and a predetermined menu item (not shown) is selected. The display settings screen includes a display settings unit F10B1. The display settings unit F10B1 is the part where settings related to the elements (content) displayed in the main display area M are made. The display settings unit F10B1 is the part for setting the manner (e.g., presence or absence) of the display of elements in the main display area M. The display settings unit F10B1 includes, for example, checkboxes or switches provided for each element whose presence or absence of display is set. Elements whose display manner in the main display area M can be set by the display settings unit F10B1 include, for example, at least one of the taught trajectory (target path PA in Figure 10D), actual work trajectory PB (see Figure 10D), transport vehicle (dump truck) detection area, and video captured by the imaging device. The actual work trajectory PB is the trajectory of a specific part of the attachment 15 when the work machine 10 actually performs work through autonomous operation (described later). The "transport vehicle (dump) detection area" above is the range of positions where the transport vehicle (dump) on which the work object is loaded is detected. The "image captured by the imaging device" above is the image captured by the imaging device installed on or outside the work machine 10 (e.g., at the work site). The image captured by this imaging device may be an image captured by a two-dimensional camera, or a three-dimensional image (e.g., a point cloud image) captured by a device that acquires three-dimensional information (e.g., LiDAR, stereo camera, etc.). In the example shown in Figure 10B, the elements (contents) displayed in the main display area M are a three-dimensional image representing the work machine 10 and a point cloud image.

[0110] Figure 10C shows the sensor status confirmation screen. The sensor status confirmation screen is displayed, for example, when the menu selection unit F2A1 (see Figure 10A) is selected and a predetermined menu item (not shown) is selected. The sensor status confirmation screen includes a sensor status display unit F10C1. The sensor status display unit F10C1 displays the status of the work machine 10 or a sensor provided outside the work machine 10. The sensor status display unit F10C1 may, for example, display whether the sensor is normal or not, or it may display the content acquired by the sensor. The sensor whose status is displayed by the sensor status display unit F10C1 may be the angle sensors of each element of the attachment 15 (boom 15a, arm 15b, tip attachment 15c). The sensor whose status is displayed by the sensor status display unit F10C1 may be an imaging device (LiDAR, camera, etc.), an automatic operation controller of the work machine 10, or a controller other than the automatic operation controller of the work machine 10.

[0111] Figure 10D shows the screen during automatic operation, including the work phase display unit PD. The screen during automatic operation, including the work phase display unit PD, is displayed, for example, when the display setting unit F10B1 shown in Figure 10B selects to display the target route PA (taught trajectory in Figure 10B). The screen during automatic operation, including the work phase display unit PD, comprises the target route PA, the actual work trajectory PB, the target range TA, the work phase display unit PD, the work phase text display unit F10D1, and the legend F10D2.

[0112] The target path PA is the target path for a specific part of the attachment 15 (for example, the tip of the tip attachment 15c), as described above. The target path PA is displayed in the main display area M. The target path PA is displayed as a line. For example, multiple types of target paths PA are set to correspond to multiple work phases P. Specifically, for example, the target path PA includes a capture target path PA1, a lift-and-swivel target path PA2, a release target path PA3, and a return-and-swivel target path PA4. The capture target path PA1 is the target path PA for the capture phase P1. The lift-and-swivel target path PA2 is the target path PA for the lift-and-swivel phase P2. The release target path PA3 is the target path PA for the release phase P3. The return-and-swivel target path PA4 is the target path PA for the return-and-swivel phase P4.

[0113] The display method for multiple types of target paths PA (capture target path PA1, lift-and-turn target path PA2, release target path PA3, and return-and-turn target path PA4) may be varied for each type. For example, the color of the target path PA (e.g., hue, density, lightness, and saturation) may be varied for each type of target path PA. For example, the type of line representing the target path PA (thickness, solid line, dashed line, etc.) may be varied for each type of target path PA.

[0114] The image (specifically, a line) showing the target path PA may also indicate the direction of the target path PA. The direction of the target path PA is the direction of movement of a specific part of the attachment 15 when that part is moved along the target path PA. For example, the image showing the target path PA may include an arrow, or it may include an animation that moves in the direction of the target path PA.

[0115] The actual work trajectory PB is the trajectory of the attachment 15 when the work machine 10 actually performs the work under automatic operation. The actual work trajectory PB is displayed in the main display area M. The actual work trajectory PB is displayed in the main display area M at the position (trajectory) that the attachment 15 has passed through on the work machine 10. The actual work trajectory PB may be, for example, the trajectory (e.g., a line) of a specific part of the attachment 15 (e.g., the tip of the end attachment 15c), or it may be the trajectory of the entire end attachment 15c.

[0116] The actual work trajectory PB and the target path PA are displayed superimposed. This superimposed display allows the operator to confirm whether a specific part of the attachment 15 is moving along the target path PA (movement that matches the target of autonomous driving).

[0117] The actual work trajectory PB is displayed in a way that distinguishes it from the target path PA. The manner in which the actual work trajectory PB is displayed differs from the manner in which the target path PA is displayed. For example, the color (e.g., hue) of the actual work trajectory PB is different from the color (e.g., hue) of the target path PA. The color of the actual work trajectory PB may be a single color or may have multiple colors (e.g., a gradient). The manner in which the actual work trajectory PB is displayed may change depending on the type of work phase P (currently being performed work phase P) being carried out by the work machine 10 (described later).

[0118] The actual work trajectory PB may be displayed as an animation. For example, a portion of the actual work trajectory PB may be removed from the display image G when a predetermined condition (trajectory display condition) is no longer met. Removing a portion of the actual work trajectory PB helps to prevent the display image G from becoming difficult to see due to an excessive amount of the actual work trajectory PB being displayed. Specifically, for example, the actual work trajectory PB may be removed from display at a certain position after a predetermined time (trajectory display time) has elapsed since a specific part of the attachment 15 passed that position. This trajectory display time may be a predetermined fixed time, or it may be set arbitrarily by the operator.

[0119] The target range TA is the target range for work performed by the automated operation of the work machine 10. The target range TA is displayed in the main display area M. Multiple types of target range TA can be set. Specifically, for example, the target range TA includes a target acquisition range TA1 and a target release range TA3. The target acquisition range TA1 is the range in which the work machine 10 (the tip attachment 15c) acquires the work object in the acquisition phase P1 (e.g., the area to be excavated). The target release range TA3 is the range in which the work machine 10 (the tip attachment 15c) releases the work object in the release phase P3 (e.g., the area to be excavated).

[0120] The display manner of multiple types of target ranges TA (target acquisition range TA1, target release range TA3) may be changed for each type. For example, the color (e.g., hue) of the target range TA may be changed for each type of target range TA. The display manner of each type of target range TA may correspond to the display manner of the type of target path PA corresponding to the type of target range TA. Specifically, for example, the display manner (e.g., hue) of target acquisition range TA1 may be the same (or nearly the same) as the display manner (e.g., hue) of acquired target path PA1. Also, for example, the display manner of target release range TA3 may be the same (or nearly the same) as the display manner of released target path PA3. More specifically, suppose acquired target path PA1 is displayed in orange and released target path PA3 is displayed in green. In this case, target acquisition range TA1 may be displayed in orange and target release range TA3 in green. Note that the display manner of each type of target range TA does not have to correspond to the display manner of the type of target path PA corresponding to the type of target range TA.

[0121] The work phase display unit PD indicates which of several types of work phases P the work (automatic operation) being performed by the work machine 10 is currently in. Hereinafter, the type of work phase P indicated by the work phase display unit PD and the work phase P being performed by the work machine 10 will also be referred to as the "currently performed work phase P". The work phase display unit PD is a display that allows the operator to easily understand the currently performed work phase P. The work phase display unit PD displays the currently performed work phase P in a way that allows the operator to distinguish between the currently performed work phase P and other types of work phase P.

[0122] The work phase display unit PD may include a display of the target path PA. Specifically, the display of the target path PA may be changed for the type of target path PA corresponding to the currently performed work phase P (e.g., capture phase P1) (e.g., capture target path PA1) and for other types of target path PAs. The display manner that can be changed may be the color of the target path PA (e.g., at least one of hue, density, lightness, and saturation) or the type of line representing the target path PA. Specifically, for example, the lightness of the type of target path PA corresponding to the currently performed work phase P may be increased, while the lightness of other types of target path PA may be decreased (grayed out).

[0123] The work phase display unit PD may include a display of the actual work trajectory PB. Specifically, the manner of displaying the actual work trajectory PB (e.g., hue) may be changed for each type of work phase P currently in progress. The manner of displaying the actual work trajectory PB may be changed each time the type of work phase P currently in progress changes.

[0124] The work phase display unit PD may include a display of the target range TA. Specifically, the display manner of the target range TA may be changed for the type of target range TA corresponding to the currently performed work phase P (e.g., target acquisition range TA1) and other types of target range TA (e.g., target release range TA3). The display manner that can be changed in this case may be the color inside the target range TA (e.g., hue), the color of the outer frame of the target range TA, or the type of line indicating the outer frame of the target range TA. Specifically, for example, the brightness of the target range TA corresponding to the currently performed work phase P may be increased, and the brightness of other types Target range TA The brightness may be low (grayed out).

[0125] The work phase display unit PD may include an image of the work machine 10. Specifically, the display of part or all of the work machine 10 may be changed for each type of work phase P currently in progress. Each time the type of work phase P currently in progress changes, the display of part or all of the work machine 10 may be changed. If only a part of the display of the work machine 10 is changed for each type of work phase P currently in progress, the part whose display is changed may be only the attachment 15 or only the tip attachment 15c.

[0126] For example, as described above, the color (e.g., hue) of the target path PA may differ for each type of target path PA. In this case, the display color of part or all of the work machine 10 may be the same as or approximately the same as the color of the type of target path PA corresponding to the currently performed work phase P. Specifically, for example, suppose that the capture target path PA1 is displayed in orange, the lifting and slewing target path PA2 is displayed in blue, and the release target path PA3 is displayed in green. In this case, if the currently performed work phase P is the capture phase P1, part or all of the work machine 10 will be displayed in orange. If the currently performed work phase P is the lifting and slewing phase P2, part or all of the work machine 10 will be displayed in blue. If the currently performed work phase P is the release phase P3, part or all of the work machine 10 will be displayed in green.

[0127] The work phase display unit PD may include a work phase character display unit F10D1. The work phase character display unit F10D1 displays characters indicating the currently performed work phase P. The display manner (e.g., color) of the work phase character display unit F10D1 may be the same as or nearly the same as the display manner (e.g., color) of the type of target range TA corresponding to the currently performed work phase P.

[0128] The work phase display unit PD may include the entire display image G, or it may include the entire main display area M. Specifically, the display mode (e.g., color) of the entire display image G may be changed for each type of work phase P currently being performed, or the display mode (e.g., color) of the entire main display area M may be changed. The display mode (e.g., color) of the entire display image G or the entire main display area M may be the same as or approximately the same as the display mode (e.g., color) of the target path PA (or target range TA) of the type corresponding to the work phase P currently being performed.

[0129] The work phase display unit PD may display the type of work phase P that will be performed after the currently performed work phase P, in addition to displaying the type of work phase P that is currently being performed. In this case, the work phase display unit PD may display the type of work phase P that is currently being performed and the type of work phase P that will be performed next in a way that allows them to be distinguished.

[0130] Legend F10D2 indicates which type of work phase P corresponds to which display (e.g., target path PA) is shown in various forms depending on the type of work phase P. For example, in the example shown in Figure 10D, Legend F10D1 shows the relationship between the display forms of multiple types of target path PA and the type of work phase P. For example, Legend F10D2 may show the relationship between the hue of the target path PA and the type of work phase P (which color corresponds to which work phase P). Legend F10D1 may also show the relationship between the display forms (e.g., color) of displays other than target path PA (e.g., target range TA, actual work trajectory PB) and the type of work phase P.

[0131] (Simple screen display during autonomous driving) Figure 11 shows the simple screen (second screen) during automatic operation. The simple screen during automatic operation is a screen displayed during automatic operation. The simple screen during automatic operation may also be displayed when the work machine 10 is in standby mode. The simple screen during automatic operation may also be displayed when the work machine 10 has made an emergency stop (during emergency stop). Display image G can be switched between the automatic operation screen (detailed automatic operation screen) (also simply called the "detailed screen") shown in Figure 10A and the simple screen during automatic operation shown in Figure 11. For example, the simple screen during automatic operation is displayed when the simple screen selection unit F10A6 (see Figure 10A) is selected in the detailed screen (see Figure 10A). The simple screen during automatic operation may also be displayed when the simple screen selection unit F10A6 is selected only once (in one action). The simple screen during automatic operation may also be displayed after the simple screen selection unit F10A6 is selected and a confirmation operation (such as pressing the OK button) is performed.

[0132] It is conceivable that an operator may not be able to focus on the display image G, for example, when performing "other tasks." The "other tasks" mentioned above are tasks different from those performed on the work machine 10, where information is displayed on the display image G. Specifically, these might include tasks performed on machines not connected to the information terminal 23 (such as excavators). "Other tasks" may also include manual or remote operation of the machine. There may be cases where an operator wants to understand the status of the automated operation of the work machine 10 in a short viewing time (for example, periodically) while performing "other tasks." For example, an operator may want to know how far the automated operation has progressed, or how much longer it will take to complete. However, for an operator who cannot focus on the display image G, the amount of information on the detailed screen (see Figure 10A) is large. Therefore, it may be difficult for an operator to confirm the information they want to check in a short viewing time.

[0133] Therefore, display image G displays a simplified screen during autonomous operation. The simplified screen during autonomous operation is displayed in a way that allows the operator to grasp the information even with a shorter viewing time compared to the detailed screen (see Figure 10A). Specifically, the amount of information displayed on the simplified screen during autonomous operation is less than the amount of information displayed on the detailed screen. The information related to autonomous operation displayed on the simplified screen during autonomous operation includes only a portion of the information related to autonomous operation displayed on the detailed screen. Specifically, for example, the display of the work site (point cloud image, etc.) shown in the machine information display unit F10A4 and the main display area M shown in Figure 10A is not displayed on the simplified screen during autonomous operation, as shown in Figure 11. Since the simplified screen during autonomous operation is displayed as display image G, the operator can obtain information related to the autonomous operation of the work machine 10 even with a short viewing time of display image G. As a result, for example, even if the operator views display image G while performing other tasks (e.g., manual or remote operation of the machine), the operator can monitor the work machine 10 without significantly impairing the efficiency of the other tasks (it is possible to balance other tasks and monitoring of autonomous operation).

[0134] The images displayed on the simple screen during autonomous driving are preferably simpler (less elaborate) than the images displayed on the detailed screen (see Figure 10A). For example, the information displayed on the simple screen during autonomous driving may be larger than the information displayed on the detailed screen. The "information displayed" may be text or a graphic (such as an illustration that visually represents information). The difference in brightness (contrast) between the information displayed on the simple screen during autonomous driving and the background color of this information displayed may be greater than the contrast between the information displayed and the background color on the detailed screen. Information that was displayed as text on the detailed screen may be displayed as a graphic on the simple screen during autonomous driving (see, for example, the current workload display unit F11A3 described later). The simple screen during autonomous driving may include parts that are not present on the detailed screen (for example, the remaining work time display unit F11A2 and the detailed screen selection unit F11A4 described later).

[0135] The content (elements, items) displayed on the simple screen during autonomous driving may be configurable by the operator or pre-configured on the information terminal 23. The content displayed on the simple screen during autonomous driving should include information necessary for the operator to monitor autonomous driving, and preferably be as minimal as possible (preferably the bare minimum). For example, the simple screen during autonomous driving preferably includes a status display unit F11A1 and a display showing the progress of the autonomous driving operation (for example, a remaining work time display unit F11A2 and a current work amount display unit F11A3). The simple screen during autonomous driving also preferably includes a detailed screen selection unit F11A4, a key state selection unit F11A5a, an engine state selection unit F11A5b, a lever lock state selection unit F11A5c, a horn operation unit F11A7, and an intermediate termination selection unit F11A8.

[0136] The status display unit F11A1 is a part that displays the status of the work machine 10. The status display unit F11A1 displays the status of the work machine 10 in automatic operation. The status of the work machine 10 displayed by the status display unit F11A1 includes, for example, at least one of the following: in normal operation, in standby (a state in which work by automatic operation has been completed and the machine is in standby mode), and in emergency stop. The status of the work machine 10 displayed by the status display unit F11A1 may also include the following: in detection operation (a state in which the work machine 10 has detected an object such as a work object and is operating automatically).

[0137] When the status display unit F11A1 indicates that the work machine 10 is in standby mode, the operator can easily notice that the work machine 10 is in standby mode. The operator can then instruct the work machine 10 to operate automatically, thereby reducing the time the work machine 10 is not performing work (i.e., in standby mode). This improves the work efficiency of the work machine 10.

[0138] When the status indicator F11A1 indicates that the work machine 10 is in an emergency stop state, the worker can easily become aware that the work machine 10 is in an emergency stop state. Therefore, the worker can take action regarding the work machine 10 in an emergency stop state earlier (for example, in a safer situation).

[0139] The status display unit F11A1 is preferably displayed in such a way that the worker can easily notice changes in the display of the status display unit F11A1 even without intently looking at the display image G (for example, using peripheral vision). In this case, the number of times and the amount of time the worker looks at the display image G can be reduced. If the worker views the display image G while performing other tasks, the number of times and the amount of time the worker looks at the display image G will be reduced, which will improve the efficiency of the other tasks. For example, it is preferable that the proportion of the status display unit F11A1 in the display image G be as large as possible. For example, it is preferable that the contrast between the status display unit F11A1 and the background of the status display unit F11A1 is high.

[0140] The status display unit F11A1 may display text (phrasing) indicating the status of the work machine 10. This text may be, for example, "Normal operation" (or simply "Operating"), "Standby," or "Emergency stop." The status display unit F11A1 may display different shapes (e.g., face shape and color) depending on the status of the work machine 10. Specifically, for example, the shape (e.g., face shape) when the work machine 10 is operating normally and on standby may be different from the shape when the work machine 10 is in an emergency stop. The display (e.g., text, shape) of the status display unit F11A1 may also be in different colors depending on the status of the work machine 10. Specifically, for example, the display of the status display unit F11A1 may be green when operating normally, blue when on standby, and red when in an emergency stop. The background color of the status display unit F11A1 may also be different depending on the status of the work machine 10.

[0141] The display of the portion of the display image G other than the status display section F11A1 may change in accordance with the change in the status of the work machine 10 indicated by the status display section F11A1. This "portion other than the status display section F11A1" may be, for example, the entire main display area M, or the outer periphery (edge) of the main display area M. Specifically, for example, the color (background color or edge color) of the main display area M may change in accordance with the change in the status of the work machine 10 indicated by the status display section F11A1, or the brightness of the main display area M may change (for example, by blinking). The status display section F11A1 may also be displayed on the detail screen (see Figure 10A).

[0142] The remaining work time display unit F11A2 displays the remaining time for work performed by the automated system. Just as the display of the status display unit F11A1 may be changed according to the status of the work machine 10, the display of the remaining work time display unit F11A2 may be changed according to the remaining time for work performed by the automated system (e.g., text color, background color, etc.). In addition, the remaining number of work attempts display unit F10A2 (see Figure 10A) may be displayed in the simple screen during automated operation.

[0143] The current workload display unit F11A3 displays the same content as the current workload display unit F10A3b shown in Figure 10A. The current workload display unit F11A3 shown in Figure 11 is displayed in a way that makes it easier for the worker to grasp the information than the current workload display unit F10A3b shown in Figure 10A. For example, the current workload display unit F11A3 shown in Figure 11 may display larger characters and have a greater contrast between the characters and background color compared to the current workload display unit F10A3b shown in Figure 10A. The current workload display unit F11A3 may change its display (e.g., character color, background color, etc.) according to the current workload. Furthermore, the current workload display unit F11A3 shown in Figure 11 displays a figure that shows the ratio of completed work to the target workload. The current workload display unit F11A3 changes the displayed figure in multiple stages (e.g., 5 stages) according to this ratio. Specifically, for example, the current work amount display unit F11A3 displays a graphic indicating how much soil has been loaded into the vehicle (dump truck). The graphic displayed in the current work amount display unit F11A3 may also be displayed on the detail screen (see Figure 10A). The detail screen selection unit F11A4 is the part that is selected when switching the display image G from the simple screen during automatic driving to the detail screen (Figure 10A).

[0144] The key state selection unit F11A5a, the engine state selection unit F11A5b, and the lever lock state selection unit F11A5c have the same functions as the key state selection unit F9C4, the engine state selection unit F9C5, and the lever lock state selection unit F9C6 shown in Figure 9C. The horn operation unit F11A7 shown in Figure 11 is the part selected when the work machine 10 is made to sound a horn. Thus, the simple screen during automatic operation may also include selection units for operating the work machine 10. The intermediate termination selection unit F11A8 is the part selected when the automatic operation is terminated midway (details will be described later).

[0145] (Autonomous driving operation completed) Figure 12A shows the automated operation completion screen. The automated operation completion screen is displayed when the automated operation is completed successfully. For example, the automated operation completion screen is displayed when the amount of work completed (amount of soil loaded) reaches the target amount of work (target amount of soil). The automated operation completion screen comprises a completion notification unit F12A1, a completion selection unit F12A2, and a cancellation selection unit F12A3. The completion notification unit F12A1 is the part that notifies the operator that the automated operation is ending. The completion notification unit F12A1 notifies that the work on the teaching data in which the automated operation was performed is ending. The completion selection unit F12A2 is the part that is selected when the work on the teaching data in which the automated operation was performed is ending. The cancellation selection unit F12A3 is the part that is selected when the work on the teaching data in which the automated operation was performed is not ending. For example, if the cancellation selection unit F12A3 is selected, the system may return to the screen before the start of automatic operation (see Figure 9A), or automatic operation may be resumed after, for example, the number of operations and the target workload have been changed.

[0146] When the automated operation is completed, the results of the operation (results display unit) may be displayed. The results of the operation may include, for example, the quantity of work completed, the number of operations, and the operation time.

[0147] Figure 12B shows the premature termination screen. The premature termination screen is displayed when automatic operation is terminated prematurely. Premature termination of automatic operation occurs when the amount of work completed (amount of soil loaded) has not reached the target amount of work (target amount of soil). The premature termination screen is displayed, for example, when the premature termination selection unit F10A8 (see Figure 10A) or F11A8 (see Figure 11) is selected. The premature termination screen comprises a premature termination notification unit F12B1, an end selection unit F12B2, and a cancel selection unit F12B3. The premature termination notification unit F12B1 is the part that notifies that automatic operation will be terminated prematurely. The premature termination notification unit F12B1 notifies that work will be completed in the current cycle. The end selection unit F12B2 is the part (OK button) that is selected when premature termination of automatic operation. The cancellation selection section F12B3 is the part (cancel button) selected when you want to continue autonomous driving (not terminate the autonomous driving process midway).

[0148] If automatic operation is terminated prematurely, the work machine 10 continues automatic operation until it returns to its initial position, and then stops in that position. This "initial position" is, for example, the initial position at the start of automatic operation. If multiple work phases P (see Figure 8) are set, this "initial position" is, for example, the position at the start of the first work phase P (for example, the capture phase P1 (see Figure 8)). A specific example of premature termination of automatic operation is as follows: For example, suppose the premature termination selection unit F11A8 (see Figure 11) is selected during the release phase P3 (see Figure 8). In this case, the work machine 10 completes the release phase P3, completes the return rotation phase P4 (see Figure 8), and stops in its initial position. From the time the premature termination selection unit F11A8 (see Figure 11) is selected until the work machine 10 returns to its initial position, a notification that work is being terminated (premature termination display unit) may be displayed.

[0149] (Emergency stop) Figure 13A shows the emergency stop screen. The emergency stop screen is displayed when the work machine 10 comes to an emergency stop.

[0150] The conditions for the emergency stop of the work machine 10 (emergency stop conditions) may include the selection of the emergency stop selection unit F13A1. The emergency stop conditions may also include the selection of the all-machine stop selection unit F13A2. The emergency stop conditions may also include other conditions set in the controller of the work machine 10 or the automatic operation information processing device 20.

[0151] The emergency stop selection section F13A1 is the part (for example, an emergency stop button) selected when an emergency stop is initiated on the work machine 10 (the work machine 10 corresponding to the selected machine selection section T) whose information is displayed in the main display area M. The emergency stop selection section F13A1 is displayed during automatic operation, but may also be displayed at times other than automatic operation.

[0152] The all-machine stop selection unit F13A2 is the part selected when an emergency stop is performed on all work machines 10 connected to (already connected to) the information terminal 23 (for example, the all-machine emergency stop button).

[0153] If a total machine stop selection unit F13A2 is provided, the following effects can be obtained. For example, in order to stop a work machine 10 from the information terminal 23, it is assumed that the worker selects the machine selection unit T corresponding to the work machine 10 to be stopped and then selects the emergency stop selection unit F13A1. In this case, if the work machine 10 that the worker wants to stop is different from the work machine 10 selected in the machine selection unit T, the worker must select the machine selection unit T corresponding to the work machine 10 to be stopped (switch screens) and then select the emergency stop selection unit F13A1. Therefore, the operation of selecting the machine selection unit T is time-consuming and the operation takes time. Also, if multiple work machines 10 are connected to the information terminal 23, the worker must go through multiple steps to stop all connected work machines 10. Specifically, the worker must repeatedly switch the machine selection unit T and select the emergency stop selection unit F13A1. For example, in the event of a disaster, when it is necessary to immediately emergency stop all connected work machines 10, it is undesirable for the operation to take time. Therefore, a total machine stop selection unit F13A2 is provided. By selecting (for example, pressing) the total machine stop selection unit F13A2, the operator can perform an emergency stop on all connected work machines 10 (total machine stop function). In this case, the operator can stop all connected work machines 10 without selecting a machine selection unit T (without switching screens).

[0154] It is preferable that the All Machine Stop Selection Unit F13A2 is always displayed when at least one work machine 10 is connected to the information terminal 23. More specifically, it is preferable that the All Machine Stop Selection Unit F13A2 is always displayed on the display image G, even if the content (state) of the display image G changes (transitions) in various ways, as long as a work machine 10 is connected to the information terminal 23. Specifically, for example, it is preferable that the All Machine Stop Selection Unit F13A2 is displayed when a work machine 10 is selected by the machine selection unit T and information regarding the selected work machine 10 is displayed on the display image G. Also, it is preferable that the All Machine Stop Selection Unit F3B4 is displayed on screens where a work machine 10 is connected to the information terminal 23, but a work machine 10 is not selected by the machine selection unit T, such as the home screen shown in Figure 3B. Note that when no work machine 10 is connected to the information terminal 23, the All Machine Stop Selection Unit F13A2 shown in Figure 13A does not need to be displayed (see Figure 14A).

[0155] It is preferable that the All Machine Stop Selection Unit F13A2 is always displayed at a specific position (always the same position) on the display image G when at least one work machine 10 is connected to the information terminal 23. This "specific position" is a part of the display image G, or a specific location within the display image G. For example, the All Machine Stop Selection Unit F13A2 is displayed in a part of the display image G where the display does not change even when the state of the display image G changes. Specifically, for example, the All Machine Stop Selection Unit F13A2 is displayed in a part where it does not change depending on the selection state of the machine selection unit T. In the example shown in Figure 13A, the All Machine Stop Selection Unit F13A2 is displayed below the home selection unit H, below the work selection unit A, and in the lower left portion of the display image G. For example, the All Machine Stop Selection Unit F13A2 may also be displayed at the top of the display image G, or in an area above the machine selection unit T (top header of the screen) (not shown).

[0156] This all-machine stop selection unit F13A2 is always selectable (operable) when at least one work machine 10 is connected to the information terminal 23. This all-machine stop selection unit F13A2 is always capable of emergency stopping all work machines 10 connected to the information terminal 23 when work machines 10 are connected to the information terminal 23.

[0157] It is preferable that the All-Machine Stop Selection Unit F13A2 can be used to emergency stop all connected work machines 10 with a single operation (one action). In this case, the operator can emergency stop the connected work machines 10 with a simple operation. However, the All-Machine Stop Selection Unit F13A2 may also be capable of emergency stopping all connected work machines 10 with multiple operations. For example, the connected work machines 10 may be emergency stopped after a confirmation operation (such as pressing the OK button) is performed after the All-Machine Stop Selection Unit F13A2 is selected.

[0158] When the all-machine stop selection unit F13A2 is selected, the information terminal 23 simultaneously transmits an emergency stop signal to all connected work machines 10.

[0159] When a work machine 10 (one or more work machines 10) comes to an emergency stop, the machine selection unit T will display, for example, as follows: The machine selection unit T corresponding to the work machine 10 that has come to an emergency stop will display an emergency stop indicator. This emergency stop indicator will be displayed in a way that distinguishes it from the display of the machine selection unit T when the work machine 10 is not in an emergency stop (so that the operator can distinguish it). Specifically, for example, the color of the machine selection unit T will be green when the work machine 10 is not in an emergency stop, and red when the work machine 10 has come to an emergency stop. If there are multiple work machines 10 connected to the information terminal 23 and all of them come to an emergency stop, all machine selection units T (for example, the first machine selection unit T1 and the second machine selection unit T2) will display the emergency stop indicator. As described above, an unselected machine selection unit T will display an indicator that it is unselected (for example, blinking), and a selected machine selection unit T will display an indicator that it is selected (for example, not blinking).

[0160] The emergency stop screen includes an emergency stop notification unit F13A3. The emergency stop notification unit F13A3 is the part that notifies that the work machine 10 has made an emergency stop. The emergency stop notification unit F13A3 may also notify the reason (error details) why the work machine 10 made an emergency stop.

[0161] If the emergency stop selection unit F13A1 or the all-machine stop selection unit F13A2 is selected (hereinafter also referred to as "when emergency stop is selected"), the work machine 10 will perform an emergency stop, for example, as follows. For example, if emergency stop is selected, the work machine 10 may immediately transition to the emergency stop state. In this case, the work machine 10 will come to a stop in the middle of its current operation. Alternatively, for example, if emergency stop is selected, the work machine 10 may transition to the emergency stop state and come to a stop when the currently performed operation (for example, the currently performed operation phase P (see Figure 8) or one cycle of automatic operation) is completed. Alternatively, for example, if emergency stop is selected while the work machine 10 is holding an object to be worked on, the work machine 10 may release the object to be worked on by automatic operation, then transition to the emergency stop state and come to a stop. In this case, it is preferable for the work machine 10 to release the object to be worked on in a position where the object can be stably positioned. If emergency stop is selected while the work machine 10 is holding an object to be worked on, a notification corresponding to the following types of work (see Figure 13B) may be issued.

[0162] After a work machine 10 has made an emergency stop, it can be restored to operation as follows. "Restoration" may also mean that the emergency stop state is released. Specifically, for example, restoration may mean that the emergency stop state is released by turning off the engine of the work machine 10 and disconnecting the connection between the information terminal 23 and the work machine 10. Alternatively, "restoration" may mean that the emergency stop state is released and automatic operation is resumed. For example, the operator selects the machine selection unit T corresponding to the work machine 10 that they want to restore and performs the operation to restore the selected work machine 10 (restoration operation). If multiple work machines 10 (for example, all machines) have made an emergency stop, the operator performs the restoration operation on each machine one by one. The information terminal 23 may be configured so that multiple work machines 10 can be restored simultaneously.

[0163] Figure 13B shows an emergency stop screen corresponding to the type of work being performed. This screen includes an emergency stop notification unit F13B1 and a selection unit F13B2.

[0164] Here, when the work machine 10 makes an emergency stop, it is conceivable that the state of the work machine 10 may be changed, for example, by turning off the engine or disabling the automatic operation mode and switching to a manually operable mode. However, depending on the work content, the configuration of the work machine 10, the type of tip attachment 15c, etc., problems may arise if the state of the work machine 10 changes. Therefore, it is preferable to provide a notification that can help avoid these problems. The following describes a specific example of this notification. The emergency stop notification unit F13B1 is displayed when the work machine 10 performing lifting magnet work makes an emergency stop. The emergency stop notification unit F13B1 notifies the user of the nature of the problem. Specifically, the emergency stop notification unit F13B1 notifies the user that if the engine stops while the magnet is holding the work object (metal), the work object will fall from the magnet. The emergency stop notification unit F13B1 also notifies the user of how to avoid the problem. In this example, the emergency stop notification unit F13B1 notifies the user of how to prevent the work object from falling from the magnet. Specifically, the emergency stop notification unit F13B1 notifies the operator to board the work machine 10, raise the lever lock (to the locked state), and render the work machine 10 inoperable. The emergency stop notification unit F13B1 notifies the operator to turn on the key in the driver's cab 13a and keep the engine on (running). The emergency stop notification unit F13B1 notifies the operator to press the selection unit F13B2 (OK button) with the lever lock raised and the key turned on. When the selection unit F13B2 is selected, a notification (not shown) indicating that the automatic operation mode will be canceled is displayed, and the automatic operation mode is canceled. After that, for example, the lever lock is released, the operator operates the work machine 10, and the work object is lowered. The above example describes the case where the work machine 10 performing lifting magnet work is emergency stopped, but if the work machine 10 performing other types of work is emergency stopped, a notification corresponding to the type of work may also be displayed.

[0165] (setting) Figure 14A shows the connection settings screen. The connection settings screen is displayed when the information terminal 23 is configured to connect to the work machine 10. The connection settings screen is displayed, for example, when the setting unit B5 is selected. The connection settings screen includes a setting item selection unit F14A1, a connection setting selection unit F14A2, a new connection selection unit F14A3, a registered machine selection unit F14A4, and a parameter setting selection unit F14A5.

[0166] The setting item selection section F14A1 is a section that displays the ability to select one setting item from multiple setting items. The connection setting selection section F14A2 is a part of the setting item selection section F14A1. The connection setting selection section F14A2 is selected when settings related to the connection between the information terminal 23 and the work machine 10 are made. The new connection selection section F14A3 is selected when a new work machine 10 is connected to the information terminal 23. When a new work machine 10 is connected, information to identify the work machine 10 to be connected (e.g., IP address, registered machine name, etc.) is set. The above "registered machine name" is the name of the work machine 10 that is displayed, for example, in the machine selection section T (see Figure 14B) and the registered machine selection section F14A4.

[0167] Figure 14B shows the task selection screen for new connections. The task selection screen for new connections includes task selection units F14BA1 and F14BA2. Task selection units F14BA1 and F14BA2 are for selecting which type of task to associate the connected task machine 10 with when the new task machine 10 is connected to the information terminal 23. Note that this task selection screen for new connections is a task selection screen (application selection screen) associated with the newly connected task machine 10, and is not the home screen (see Figures 3A and 3B). The number of units display unit F3B3 (see Figure 3B) is not displayed on this screen.

[0168] The registered machine selection unit F14A4 shown in Figure 14A is a part that displays information about work machines 10 (registered machines) that have been previously connected to the automatic driving information processing device 20 (e.g., information terminal 23). For example, the registered machine selection unit F14A4 displays the registered machine name of the registered machine. If a registered machine is selected from the registered machine selection unit F14A4, information about the registered machine may be notified. The notified information may include the registered machine name, information about the connection between the registered machine and the automatic driving information processing device 20 (e.g., the date and time of the previous connection), and information about the work performed during the previous connection (e.g., the duration of the previous work).

[0169] The way in which registered machines are displayed in the registered machine selection unit F14A4 may change depending on whether or not they are machines corresponding to the work selected in the work selection unit A. For example, if loading work selection unit A1 is selected in work selection unit A, the registered machine selection unit F14A4 may display the registered machines in a way that allows the operator to distinguish whether or not they are machines corresponding to loading work. Specifically, for example, the registered machine selection unit F14A4 may make registered machines corresponding to the type of work selected in work selection unit A active (selectable) and make registered machines that do not correspond to the type of work selected in work selection unit A inactive (not selectable). The registered machine selection unit F14A4 may also hide registered machines that do not correspond to the type of work selected in work selection unit A.

[0170] The parameter setting selection section F14A5 is the part selected when parameter settings (not shown) are to be made. The parameters to be set may be parameters related to the work machine 10, or parameters related to automatic operation.

[0171] Figure 14C shows the sensor status display screen. The sensor status display screen is a screen that displays the status of the sensor. The sensor status display screen comprises a sensor status selection unit F14C1 and a sensor status display unit F14C2. The sensor status selection unit F14C1 is the part selected to display the sensor status display screen (the sensor status display unit F14C2). The sensor status display unit F14C2 displays the status of the work machine 10 or a sensor installed outside the work machine 10, similar to the sensor status display unit F10C1 (see Figure 10).

[0172] Figure 14D shows the accessory operation screen. The accessory operation screen is a screen that displays the status of each device (accessory) of the work machine 10, and is a screen that allows the accessories to be operated (remotely operated). The accessory operation screen comprises an accessory operation selection unit F14D1 and an accessory status display setting unit F14D2. The accessory operation selection unit F14D1 is the part selected to display the accessory operation screen (accessory status display setting unit F14D2). The accessory status display setting unit F14D2 is the part that displays the status of the accessories (accessory status display unit). The accessory status display setting unit F14D2 is the part that displays the status of the accessories in a way that allows them to be set (changed) (accessory status setting unit). The accessory status that can be displayed and set in the accessory status display setting unit F14D2 (displayable and settable accessory status) may include, for example, the on / off status of lights (boom lights, cab lights, etc.). The configurable accessory statuses may include whether the wipers are operating and their operating interval (intermittent, continuous), whether the washer (device for cleaning the windows of the driver's cab 13a) is on or off, and whether the DPF (Diesel particulate filter) regeneration is on or off. The configurable accessory statuses may also include the operating mode of the work machine 10 (high speed, low speed, fuel-efficient (ECO)).

[0173] (Specific examples of operation) Referring to the flowchart shown in Figure 15A, further specific examples of the operation of the automated driving information processing device 20 (mainly the operation of the control unit 23c), the automated driving information processing method, and the automated driving information processing program shown in Figure 1 will be explained. Below, unless otherwise specified, the operation (work, processing) of the automated driving information processing device 20 will be explained in order. Note that the order of operation can be changed in various ways. Furthermore, steps S11 to S72 shown in Figure 15A will be explained with reference to Figure 15A.

[0174] In step S11, the information terminal 23 shown in Figure 1 is connected to the work machine 10.

[0175] In step S21, the target area TA (see Figure 10D) is taught (area teaching is performed) (see Figure 6).

[0176] In step S22, the target route PA (see Figure 10D) is taught (route teaching is performed) (see Figure 4C).

[0177] In step S31, the operating mode of the work machine 10 is changed (transitioned) to automatic operation mode. For example, if the above conditions for starting automatic operation mode are met (see Figures 7A and 7B), the operating mode of the work machine 10 transitions to automatic operation mode (see Figure 7C).

[0178] In step S32, the settings for automatic driving (automatic driving operation settings) are performed. Specifically, as shown in Figure 8, the initial settings for automatic driving are performed. In addition, as shown in Figures 9A, 9B, 9C, and 9D, the operating speed of the automatic driving system and other settings are configured.

[0179] In step S41, preparations are made for the start of automated operation of the work machine 10. For example, the posture of the work machine 10 is changed to a posture that allows for the start of automated operation.

[0180] In step S42, the work machine 10 starts automatic operation (see Figures 10A, 10B, 10C, 10D, and 11).

[0181] In steps S43, S44, and S45, the following operations are performed. If a transport vehicle (dump truck) is detected in the transport vehicle (dump truck) detection area (step S43), a target route PA (see Figure 10D) for performing work on the transport vehicle (e.g., soil removal) is displayed on the display unit 23o (step S44). If the operator selects to have the work machine 10 operate according to this target route PA, the work machine 10 starts working on the transport vehicle (step S45).

[0182] In step S46, the control unit 23c (see Figure 1) monitors the automatic operation of the work machine 10. For example, as shown in Figures 10A and 11, the control unit 23c monitors the status of the automatic operation (progress, whether it is operating normally or not, etc.). The control unit 23c also monitors whether to terminate the automatic operation midway (whether the midway termination selection unit F10A8 in Figure 10A is selected) and whether to perform an emergency stop on the automatic operation (whether the emergency stop selection unit F13A1 in Figure 13 is selected).

[0183] In step S51, the automatic operation of the work machine 10 is completed, and the automatic operation of the work machine 10 stops. If the automatic operation is completed successfully, the automatic operation completion screen shown in Figure 12A is displayed on the display unit 23o. If the automatic operation is terminated prematurely, the premature termination screen shown in Figure 12B is displayed on the display unit 23o.

[0184] In step S52, the results of the automated operation are displayed on the display unit 23o. For example, the amount of work completed is displayed on the current work amount display unit F10A3b shown in Figure 10A, and the current work amount display unit F11A3 shown in Figure 11.

[0185] In step S53, the control unit 23c (see Figure 1) determines whether to perform the same (or nearly the same) automated operation as the completed automated operation. For example, the control unit 23c may make this determination in response to input (selection) by the operator. Alternatively, for example, the control unit 23c may make this determination automatically in response to the work situation (e.g., amount of work, number of times, time, etc.). If the control unit 23c decides to perform the same (or nearly the same) automated operation as the completed automated operation (YES in step S53), it returns the processing flow to step S32. If the control unit 23c decides not to perform the same (or nearly the same) automated operation as the completed automated operation (NO in step S53), it proceeds the processing flow to step S61.

[0186] In step S61, preparations are made for ending the automatic operation mode on the work machine 10 (machine-side termination preparations). For example, the control unit 23c changes the posture of the work machine 10 back to its initial posture.

[0187] In step S62, the control unit 23c terminates the automatic operation mode of the work machine 10.

[0188] In step S63, the control unit 23c determines whether or not to complete the work on the work machine 10. For example, the control unit 23c determines whether or not to complete the work on the work machine 10 using the information terminal 23 (for example, whether or not to complete the work for the day). If the work on the work machine 10 is to be completed (YES in step S63), the control unit 23c proceeds to step S71. If the work on the work machine 10 is not to be completed (NO in step S63), the control unit 23c returns to step S21.

[0189] In step S71, the work machine 10 is shut down. Specifically, the engine of the work machine 10 is turned off and the power is switched off. For example, the information terminal 23 may send a command to the work machine 10 to shut it down. The work machine 10 may also be shut down by manual operation by the worker.

[0190] In step S72, the connection between the information terminal 23 and the work machine 10 is disconnected.

[0191] (Processing to bring all aircraft to an emergency stop) Referring to the flowchart in Figure 15B, a specific example of the processing of the automatic operation information processing device 20 (mainly the control unit 23c) shown in Figure 1 when the all-machine stop selection unit F13A2 shown in Figure 13A is selected will be explained. In the following, unless otherwise specified, the explanation will follow the order of operation (tasks, processes). Note that the order of operation can be changed in various ways.

[0192] When the all-aircraft stop selection unit F13A2 shown in Figure 13A is selected, the all-aircraft emergency stop process shown in Figure 15B is performed (interrupted), regardless of which process shown in Figure 15A is being performed. Steps S91 to S94 shown in Figure 15B will be explained below with reference to Figure 15B.

[0193] In step S91, the information terminal 23 shown in Figure 1 transmits an emergency stop signal to all work machines 10 that are connected to the information terminal 23.

[0194] In step S92, all work machines 10 connected to the information terminal 23 are brought to an emergency stop.

[0195] In step S93, all work machines 10 connected to the information terminal 23 are shut down (the shutdown procedure is the same as in step S71 shown in Figure 15A).

[0196] In step S94, the connection between all work machines 10 that are connected to the information terminal 23 and the information terminal 23 is disconnected.

[0197] Furthermore, each operation of the automatic driving information processing device 20 (display, selection, setting, etc.) may be referred to as a "step" in the method and program. Specifically, for example, selection may be referred to as a selection step. More specifically, the selection of the work machine 10 by the machine selection unit T (see Figure 2) may be referred to as a "work machine selection step."

[0198] (Effects of the first invention) The effects of the automated driving information processing device 20 shown in Figure 1 are as follows:

[0199] [Configuration 1] The automated driving information processing device 20 is connected to a work machine 10 capable of automated driving in a communicative manner. The automated driving information processing device 20 includes a display unit 23o and an input unit 23i. The display unit 23o displays a display image G containing information related to automated driving. The input unit 23i receives information related to automated driving in response to the operator's operation.

[0200] With the above configuration [Configuration 1], the operator can easily check and input information regarding the automatic operation of the work machine 10 using the automatic operation information processing device 20.

[0201] (Effects of the second invention) [Configuration 2] As shown in Figure 2, the display image G includes a work selection unit A. The work selection unit A displays each type of work performed by the work machine 10 (see Figure 1, the same applies to the work machine 10 hereafter) in automatic operation. The work selection unit A displays the types of work that can be selected by the input unit 23i (see Figure 1).

[0202] With the above [Configuration 2], the operator can easily select the type of work to be performed by the work machine 10.

[0203] (Effects of the third invention) [Configuration 3] The display on the work selection unit A changes for each type of work, according to the number of connected work machines 10 for each type of work.

[0204] With the above [Configuration 3], the worker can easily grasp the number of connected work machines 10 for each type of work.

[0205] (Effects of the fourth invention) [Configuration 4] Display image G includes a function selection unit B. The function selection unit B displays multiple functions corresponding to the type of work selected in the work selection unit A. The function selection unit B displays multiple functions that can be selected by the input unit 23i (see Figure 1).

[0206] With the above configuration [4], the operator can easily select a function corresponding to the type of work selected in the work selection unit A.

[0207] (Effects of the fifth invention) [Configuration 5] The display image G comprises a machine selection unit T and a main display area M. The machine selection unit T is displayed for each work machine 10, and the work machine 10 can be selected by the input unit 23i (see Figure 1). The main display area M displays information about the work machine 10 selected by the machine selection unit T. The main display area M is a different area from the machine selection unit T within the display area of ​​the display unit 23o (see Figure 1).

[0208] With the above configuration [5], the operator can easily select the work machine 10 on which information is displayed in the main display area M.

[0209] (Effects of the sixth invention) [Configuration 6] The image on the machine selection unit T changes according to the status of the work machine 10.

[0210] As described in [Configuration 6] above, the operator can easily understand the status of the work machine 10 by checking the machine selection unit T.

[0211] (Effects of the seventh invention) [Configuration 7] The image of the machine selection unit T changes depending on at least one of the following states: automatic operation is in progress, automatic operation has finished successfully, and the work machine 10 has come to an emergency stop.

[0212] With the above configuration [7], the operator can easily determine at least one of the following states by checking the machine selection unit T: that automatic operation is in progress, that automatic operation has completed successfully, and that the work machine 10 has come to an emergency stop.

[0213] (Effects of the 8th Invention) [Configuration 8] The image of the machine selection unit T changes depending on whether the machine selection unit T corresponding to the working machine 10 whose state has changed is selected or not, after the state of the working machine 10 has changed.

[0214] As described in [Configuration 8] above, the operator can easily determine whether or not they have selected the machine selection unit T corresponding to the work machine 10 whose state has changed by checking the machine selection unit T. As a result, the operator can easily determine whether or not they have checked the information of the work machine 10 whose state has changed in the main display area M.

[0215] (Effects of the 9th Invention) Automated driving is performed based on teaching data set by an operator controlling the work machine 10.

[0216] [Configuration 9] As shown in Figure 8, the display image G includes an operating speed setting unit SS. The operating speed setting unit SS displays the operating speed for automatic operation, which can be set by the input unit 23i (see Figure 1) from multiple operating speeds, including the operating speed of the work machine 10 in the teaching data.

[0217] With the above configuration [9], the operator can easily set the operating speed of the work machine 10 in automatic operation mode.

[0218] (Effects of the 10th Invention) [Configuration 10] The fastest operating speed among the multiple operating speeds ([Configuration 9] above) is the operating speed of the work machine 10 in the teaching data.

[0219] In the above configuration

[10] , the operating speed of the work machine 10 during automated operation is less than or equal to the operating speed in the teaching data. Therefore, it is suppressed that the work machine 10 operates automatically at a speed that differs from the operator's intention.

[0220] (Effects of the 11th Invention) [Configuration 11] One cycle of automatic operation includes multiple work phases P. The operating speed setting unit SS displays the operating speed for automatic operation, which can be set for each work phase P.

[0221] With the above configuration

[11] , the operator can easily set the operating speed of the automated work machine 10 for each work phase P.

[0222] (Effects of the 12th Invention) [Configuration 12] The display image G switches between a first speed display state SP1 and a second speed display state SP2 (see Figure 9A). The first speed display state SP1 is a state in which all stages of the operating speed of each of the multiple stages of the work phase P are displayed. As shown in Figure 9A, the second speed display state SP2 is a state in which the operating speed of at least some of the work phases P of the multiple work phases P is displayed only for the stages set by the operating speed setting unit SS.

[0223] As a result of the above [Configuration 12], the display in the second speed display state SP2 shown in Figure 9A is simpler compared to the first speed display state SP1 shown in Figure 8. Therefore, the operator can easily grasp the operating speed set in the operating speed setting unit SS shown in Figure 8.

[0224] (Effects of the 13th Invention) [Configuration 13] As shown in Figure 10D, one cycle of automatic operation includes multiple work phases P. Display image G includes a work phase display unit PD that shows which work phase P the work machine 10 is performing.

[0225] With the above configuration

[13] , the operator can easily understand which work phase P the work machine 10 is performing.

[0226] (Effects of the 14th Invention) [Configuration 14] As shown in Figure 13A, the display image G includes a total machine stop selection unit F13A2 that performs an emergency stop on all connected work machines 10 (connected to the automatic driving information processing device 20).

[0227] With the above configuration

[14] , the operator can perform an emergency stop on all work machines 10 connected to the automatic driving information processing device 20 with simple operation.

[0228] (Effects of the 15th Invention) [Configuration 15] The all-machine stop selection unit F13A2 can perform an emergency stop on all connected work machines 10 (connected to the automatic operation information processing device 20) with a single operation.

[0229] With the above configuration

[15] , the operator can perform an emergency stop on all work machines 10 connected to the automatic driving information processing device 20 with simpler operation.

[0230] (Effects of the 16th Invention) [Configuration 16] The all-machine stop selection unit F13A2 is always displayed at a specific position on a part of the display image G when the work machine 10 is connected (the work machine 10 is connected to the automatic driving information processing device 20).

[0231] With the above configuration

[16] , the operator can easily determine the location of the all-machine stop selection unit F13A2 on the display image G. As a result, the operator can easily perform an emergency stop on all work machines 10 connected to the automatic operation information processing device 20.

[0232] (Effects of the 17th Invention) [Configuration 17] As shown in Figure 2, the display image G includes a machine status display unit F2A3 that displays the status of the work machine 10. The image of the machine status display unit F2A3 is linked to the display of the status display device 13b (see Figure 1). The status display device 13b is mounted on the actual work machine 10 and displays the status of the work machine 10.

[0233] With the above configuration

[17] , the operator can understand the status of the actual work machine 10 by looking at the machine status display unit F2A3 displayed as a display image G, without having to look at the status display device 13b of the actual work machine 10.

[0234] (Effects of the 18th Invention) [Configuration 18] Display image G can be switched between displaying a detailed screen (first screen) (see Figure 10A) and a simple screen during autonomous driving (second screen) (see Figure 11). As shown in Figure 11, the information on autonomous driving displayed on the simple screen during autonomous driving includes only a portion of the information on autonomous driving displayed on the detailed screen (see Figure 10A).

[0235] As a result of the above [Configuration 18], when an operator views the simple screen during autonomous driving, compared to when an operator views the detailed screen (see Figure 10A), the operator can easily obtain information related to autonomous driving (for example, in a short viewing time).

[0236] (Effects of the 19th Invention) [Configuration 19] The simple screen during autonomous driving includes a status display unit F11A1 that displays the status related to autonomous driving. The status displayed by the status display unit F11A1 includes that the work machine 10 is working under autonomous driving conditions, that the work machine 10 is in standby mode, and that the work machine 10 is in emergency stop mode.

[0237] With the above configuration

[19] , the worker can easily grasp the status of the work machine 10 (for example, with a short viewing time).

[0238] (Effects of the 20th Invention) [Configuration 20] The automatic driving information processing method comprises a connection step, a display step, and an input step. The connection step connects to a work machine 10 capable of automatic driving in a communicative manner. The display step displays a display image G containing information related to automatic driving on the display unit 23o. The input step acquires information related to automatic driving that has been input by the input unit 23i shown in Figure 1 in response to the operator's operation.

[0239] With the above configuration

[20] , the operator can easily check and input information regarding the automatic operation of the work machine 10.

[0240] (Effects of the 21st Invention) [Configuration 21] The automated driving information processing program consists of a connection step, a display step, an input step, and is executed by a computer (control unit 23c). The connection step connects to the work machine 10 capable of automated driving in a communicative manner. The display step displays a display image G containing information about automated driving on the display unit 23o. The input step acquires information about automated driving that has been input by the input unit 23i in response to the operator's operation.

[0241] With the above configuration

[21] , the operator can easily check and input information regarding the automatic operation of the work machine 10.

[0242] (modified version) The above embodiments may be modified in various ways. For example, the number of components of the above embodiments (such as each part of the display image G) (including modified versions) may be changed, and some components may not be provided. For example, modified versions of the above embodiments may be combined in various ways. For example, the arrangement of components may be changed. For example, the inclusion relationships of components may be changed in various ways. For example, something described as a subordinate component included in a higher-level component may not be included in this higher-level component, but may be included in other components. For example, something described as multiple different members or parts may be treated as a single member or part. For example, something described as a single member or part may be divided and provided as multiple different members or parts. For example, the order of transitions of the display image G may be changed. For example, elements described as components of different screens may be displayed on the same screen. For example, each component may have only a part of each feature (function, arrangement, shape, operation, etc.). [Explanation of Symbols]

[0243] 10 Working Machines 13b Status display device 15c tip attachment 20 Automated Driving Information Processing System 23c Control Unit 23i Input Section 23o Display section A. Work Selection Section B Function Selection Section F2A3 Machine Status Display Unit F3B4, F13A2 All-Aircraft Stop Selection Section F11A1 Status Display Unit G Display Image M Main display area P Work Phase P1 Capture Phase P2 Lifting and rotating phase (capture and movement phase) P3 Liberation Phase P4 Recovery turning phase (recovery movement phase) PD Work Phase Display Unit SS operating speed setting section SP1 First Speed ​​Display State SP2 Second Speed ​​Display Status T Machine Selection Section

Claims

1. An automated driving information processing device that is communicatively connected to a work machine capable of performing automated driving based on a work plan, A display unit that displays an image containing information related to the aforementioned autonomous driving, An input unit into which information regarding the automated driving is input in response to the operator's actions, Equipped with, The aforementioned work plan is information for causing the work machine to perform a cycle of work, which includes multiple work phases, for multiple cycles through the automated operation. The aforementioned automatic operation for one cycle is The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, Including multiple of the aforementioned work phases, Autonomous driving information processing device.

2. An automated driving information processing device that is communicatively connected to a work machine capable of autonomous driving, A display unit that displays an image containing information related to the aforementioned autonomous driving, An input unit into which information regarding the automated driving is input in response to the operator's actions, Equipped with, The aforementioned display image is A work selection unit displays the types of work performed by the aforementioned work machine in the automatic operation mode, and allows the type of work to be selected by the input unit. Equipped with, The type of work is determined by at least one of the following: the type of tip attachment of the work machine, the capacity of the work machine, the type of object to be worked on by the work machine, the destination location of the object to be moved, and the type of capture of the object. Autonomous driving information processing device.

3. An automated driving information processing device that is communicatively connected to a work machine capable of autonomous driving, A display unit that displays an image containing information related to the aforementioned autonomous driving, An input unit into which information regarding the automated driving is input in response to the operator's actions, Equipped with, The aforementioned automated driving is performed based on teaching data set by an operator operating the work machine. The aforementioned display image is The operating speed setting unit displays the operating speed for automatic operation, which can be set by the input unit, from multiple operating speeds, including the operating speed of the work machine in the teaching data. Equipped with, The aforementioned automatic driving cycle is, The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, This includes multiple work phases, The operating speed setting unit displays the operating speed for automatic operation so that it can be set for each work phase. Autonomous driving information processing device.

4. An automated driving information processing device that is communicatively connected to a work machine capable of autonomous driving, A display unit that displays an image containing information related to the aforementioned autonomous driving, An input unit into which information regarding the automated driving is input in response to the operator's actions, Equipped with, The aforementioned automated driving is performed based on teaching data set by an operator operating the work machine. One cycle of the aforementioned automated operation includes multiple work phases, The aforementioned display image is The operating speed setting unit displays the operating speed for automatic operation, which can be set by the input unit for each work phase, from multiple operating speeds, including the operating speed of the work machine in the teaching data. Equipped with, The aforementioned display image is A first speed display state that displays all stages of the operating speed of each of the multiple stages of the aforementioned multiple work phases, In at least some of the aforementioned work phases, a second speed display state is provided which displays the operating speed only at the stage set by the operating speed setting unit, Switches to Autonomous driving information processing device.

5. An automated driving information processing device according to claim 3 or 4, The fastest operating speed among the multiple operating speeds is the operating speed of the work machine in the teaching data. Autonomous driving information processing device.

6. An automated driving information processing device that is communicatively connected to a work machine capable of autonomous driving, A display unit that displays an image containing information related to the aforementioned autonomous driving, An input unit into which information regarding the automated driving is input in response to the operator's actions, Equipped with, The aforementioned automatic driving cycle is, The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, This includes multiple work phases, The display image includes a work phase display unit that indicates which work phase the work being performed by the work machine is in. Autonomous driving information processing device.

7. A connection step that enables communication with a work machine capable of performing automated operation based on a work plan, A display step of displaying a display image containing the information related to the autonomous driving on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, Equipped with, The aforementioned work plan is information for causing the work machine to perform a cycle of work, which includes multiple work phases, for multiple cycles through the automated operation. The aforementioned automatic operation for one cycle is The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, Including multiple of the aforementioned work phases, Automated driving information processing method.

8. A connection step that enables communication with a work machine capable of autonomous operation, A display step of displaying a display image containing the information related to the autonomous driving on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, Equipped with, The aforementioned display image is A work selection unit displays the types of work performed by the aforementioned work machine in the automatic operation mode, and allows the type of work to be selected by the input unit. Equipped with, The type of work is determined by at least one of the following: the type of tip attachment of the work machine, the capacity of the work machine, the type of object to be worked on by the work machine, the destination location of the object to be moved, and the type of capture of the object. Automated driving information processing method.

9. A connection step that enables communication with a work machine capable of autonomous operation, A display step of displaying a display image containing the information related to the autonomous driving on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, Equipped with, The aforementioned automated driving is performed based on teaching data set by an operator operating the work machine. The aforementioned display image is The operating speed setting unit displays the operating speed for automatic operation, which can be set by the input unit, from multiple operating speeds, including the operating speed of the work machine in the teaching data. Equipped with, The aforementioned automatic driving cycle is, The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, This includes multiple work phases, The operating speed setting unit displays the operating speed for automatic operation so that it can be set for each work phase. Automated driving information processing method.

10. A connection step that enables communication with a work machine capable of autonomous operation, A display step of displaying a display image containing the information related to the autonomous driving on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, Equipped with, The aforementioned automated driving is performed based on teaching data set by an operator operating the work machine. One cycle of the aforementioned automated operation includes multiple work phases, The aforementioned display image is The operating speed setting unit displays the operating speed for automatic operation, which can be set by the input unit for each work phase, from multiple operating speeds, including the operating speed of the work machine in the teaching data. Equipped with, The aforementioned display image is A first speed display state that displays all stages of the operating speed of each of the multiple stages of the aforementioned multiple work phases, In at least some of the aforementioned work phases, a second speed display state is provided which displays the operating speed only at the stage set by the operating speed setting unit, Switches to Automated driving information processing method.

11. A connection step that enables communication with a work machine capable of autonomous operation, A display step of displaying a display image containing the information related to the autonomous driving on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, Equipped with, The aforementioned automatic driving cycle is, The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, This includes multiple work phases, The display image includes a work phase display unit that indicates which work phase the work being performed by the work machine is in. Automated driving information processing method.

12. A connection step that enables communication with a work machine capable of performing automated operation based on a work plan, A display step in which a display image containing the information related to the autonomous driving is displayed on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, An automated driving information processing program that causes a computer to execute, The aforementioned work plan is information for causing the work machine to perform a cycle of work, which includes multiple work phases, for multiple cycles through the automated operation. The aforementioned automatic operation for one cycle is The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, Including multiple of the aforementioned work phases, Autonomous driving information processing program.

13. A connection step that enables communication with a work machine capable of autonomous operation, A display step in which a display image containing the information related to the autonomous driving is displayed on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, An automated driving information processing program that causes a computer to execute, The aforementioned display image is A work selection unit displays the types of work performed by the aforementioned work machine in the automatic operation mode, and allows the type of work to be selected by the input unit. Equipped with, The type of work is determined by at least one of the following: the type of tip attachment of the work machine, the capacity of the work machine, the type of object to be worked on by the work machine, the destination location of the object to be moved, and the type of capture of the object. Autonomous driving information processing program.

14. A connection step that enables communication with a work machine capable of autonomous operation, A display step in which a display image containing the information related to the autonomous driving is displayed on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, An automated driving information processing program that causes a computer to execute, The aforementioned automated driving is performed based on teaching data set by an operator operating the work machine. The aforementioned display image is The operating speed setting unit displays the operating speed for automatic operation, which can be set by the input unit, from multiple operating speeds, including the operating speed of the work machine in the teaching data. Equipped with, The aforementioned automatic driving cycle is, The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, This includes multiple work phases, The operating speed setting unit displays the operating speed for automatic operation so that it can be set for each work phase. Autonomous driving information processing program.

15. A connection step that enables communication with a work machine capable of autonomous operation, A display step in which a display image containing the information related to the autonomous driving is displayed on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, An automated driving information processing program that causes a computer to execute, The aforementioned automated driving is performed based on teaching data set by an operator operating the work machine. One cycle of the aforementioned automated operation includes multiple work phases, The aforementioned display image is The operating speed setting unit displays the operating speed for automatic operation, which can be set by the input unit for each work phase, from multiple operating speeds, including the operating speed of the work machine in the teaching data. Equipped with, The aforementioned display image is A first speed display state that displays all stages of the operating speed of each of the multiple stages of the aforementioned multiple work phases, In at least some of the aforementioned work phases, a second speed display state is provided which displays the operating speed only at the stage set by the operating speed setting unit, Switches to Autonomous driving information processing program.

16. A connection step that enables communication with a work machine capable of autonomous operation, A display step in which a display image containing the information related to the autonomous driving is displayed on the display unit, An input step to acquire information related to the automatic operation that is input at the input unit in response to the operator's operation, An automated driving information processing program that causes a computer to execute, The aforementioned automatic driving cycle is, The aforementioned work machine's tip attachment captures the work object in a capture phase, The capture and movement phase involves the tip attachment moving while it has captured the work object, The release phase involves the tip attachment releasing the workpiece, A return movement phase in which the tip attachment moves to the position where the capture phase is performed, This includes multiple work phases, The display image includes a work phase display unit that indicates which work phase the work being performed by the work machine is in. Autonomous driving information processing program.

Images