Automatic operation information processing device, automatic operation information processing method, and automatic operation information processing program

Abstract

The present invention facilitates the confirmation and input of information pertaining to an automatic operation of a work machine. This automatic operation information processing device 20 is connected in a communicable manner to a work machine 10 capable of an automatic operation. The automatic operation information processing device 20 comprises a display unit 23o and an input unit 23i. The display unit 23o displays information pertaining to an automatic operation. The input unit 23i receives the information pertaining to the automatic operation according to an operation of an operator.

Description

Autonomous driving information processing device, autonomous driving information processing method, and autonomous driving information processing program

[0001] The present invention relates to an automatic driving information processing device, an automatic driving information processing method, and an automatic driving information processing program that process information related to the automatic driving of a work machine.

[0002] For example, Patent Document 1 describes a work machine that performs automatic driving.

[0003] Japanese Patent Application Laid-Open No. 2018-193704

[0004] It is desirable for workers to be able to easily check and input information relating to the automatic operation of work machines.

[0005] Therefore, an object of the present invention is to provide an automatic driving information processing device, an automatic driving information processing method, and an automatic driving information processing program that allow easy confirmation and input of information related to the automatic driving of a work machine.

[0006] The autonomous driving information processing device of the first means is communicably connected to a work machine capable of autonomous driving. The autonomous driving information processing device includes a display unit and an input unit. The display unit displays a display image including information related to the autonomous driving. The information related to the autonomous driving is input to the input unit in response to operation by an operator.

[0007] The autonomous driving information processing method of the second means includes a connection step, a display step, and an input step. The connection step communicatively connects to a work machine capable of autonomous driving. The display step displays a display image including information related to the autonomous driving on a display unit. The input step acquires information related to the autonomous driving input by an input unit in response to an operator's operation.

[0008] The third means of the automatic driving information processing program causes a computer to execute a connection step, a display step, and an input step. The connection step connects to a work machine capable of automatic driving so as to be able to communicate with the work machine. The display step displays a display image including information related to the automatic driving on a display unit. The input step acquires information related to the automatic driving inputted by an input unit in response to an operator's operation.

[0009] Each of the above means 1 to 3 makes it easy to check and input information relating to the automatic operation of the work machine.

[0010] 2 is a diagram showing an autonomous driving system 1 including an autonomous driving information processing device 20 and the like.

[0023] FIG. 1 is a diagram showing a display image G displayed on the display unit 23o shown in FIG. 1, and is a diagram showing a basic screen.

[0024] FIG. 2 is a diagram showing a home screen of the display image G shown in FIG. 2.

[0025] FIG. 3 is a diagram showing a home screen of the display image G shown in FIG. 2 when a plurality of work machines 10 are connected.

[0026] FIG. 4 is a diagram showing a teaching top screen of the display image G shown in FIG. 2.

[0027] FIG. 5 is a diagram showing a pre-teaching screen of the display image G shown in FIG. 2.

[0028] FIG. 6 is a diagram showing a teaching screen of the display image G shown in FIG. 2.

[0029] FIG. 7 is a diagram showing a trajectory confirmation screen of the display image G shown in FIG. 2.

[0030] FIG. 8 is a diagram showing a trajectory change screen of the display image G shown in FIG. 2.

[0031] FIG. 9 is a diagram showing a target range setting screen of the display image G shown in FIG. 2.

[0032] FIG. 10 is a diagram showing an initial attitude preparation screen of the display image G shown in FIG. 2.

[0033] FIG. 11 is a diagram showing a work machine state preparation screen of the display image G shown in FIG. 2.

[0034] FIG. 12 is a diagram showing an autonomous driving mode start notification screen of the display image G shown in FIG. 2.

[0035] FIG. 13 is a diagram showing an autonomous driving initial setting screen of the display image G shown in FIG. 2.

[0036] FIG. 14 is a diagram showing a pre-start screen of the autonomous driving of the display image G shown in FIG. 2.

[0037] FIG. 15 is a diagram showing an operation 13A is a diagram showing a confirmation screen before the start of automatic driving of the display image G shown in FIG. 2. FIG. 13B is a diagram showing a screen during automatic driving of the display image G shown in FIG. 2. FIG. 13C is a diagram showing a display setting screen for the display image G shown in FIG. 2. FIG. 13D is a diagram showing a sensor status confirmation screen for the display image G shown in FIG. 2. FIG. 13E is a diagram showing a screen during automatic driving including a work phase display section PD and the like of the display image G shown in FIG. 2. FIG. 13F is a diagram showing a simple screen during automatic driving of the display image G shown in FIG. 2. FIG. 13G is a diagram showing a screen for ending automatic driving work of the display image G shown in FIG. 2. FIG. 13H is a diagram showing a screen for ending automatic driving work according to the work content of the display image G shown in FIG. 2. FIG. 13I is a diagram showing a connection setting screen for the display image G shown in FIG. 2. FIG. 13I is a diagram showing a work selection screen for new connection of the display image G shown in FIG. 2. FIG. 13I is a diagram showing a sensor status display screen for the display image G shown in FIG. 2. FIG. 13I is a diagram showing an accessory operation screen for the display image G shown in FIG. 2. FIG. 13I is a diagram showing an operation of the automatic driving system 1 shown in FIG. 1. FIG. 13I is a diagram showing an operation of the automatic driving system 1 shown in FIG. 1 when the all-machine stop selection section F13A2 shown in FIG. 13A is selected.

[0011] The autonomous driving system 1 will be described with reference to FIGS. 1 to 15B.

[0012] 1 , the automatic driving system 1 is a system related to the automatic driving of a construction machine 10. The automatic driving system 1 includes the construction machine 10 and an automatic driving information processing device 20.

[0013] The work machine 10 is a machine that performs work, such as a construction machine that performs construction work, and may be, for example, a shovel or a crane. The following describes a case where the work machine 10 is a shovel. The work machine 10 is configured to be capable of automatic operation. The work machine 10 may be operated by a worker (operator) in a cab 13a (described below), or may be remotely operated. The number of work machines 10 included in the automatic operation system 1 may be one or more. The work machine 10 comprises a machine main body 10a and an attachment 15.

[0014] The machine body 10a is the main body of the work machine 10. The machine body 10a includes a lower traveling body 11 and an upper rotating body 13. The lower traveling body 11 is capable of traveling on a traveling surface (such as the ground). The lower traveling body 11 may be equipped with crawlers or wheels. The upper rotating body 13 is rotatably mounted on the lower traveling body 11. The upper rotating body 13 includes a cab 13a and a status display device 13b. The cab 13a is a section where an operator can operate the work machine 10. The status display device 13b is a device that displays the status (described below) of the work machine 10. The status display device 13b is, for example, a rotating beacon, a screen, a light, etc. The status display device 13b is provided, for example, on the upper surface of the upper rotating body 13 (for example, the upper surface of the cab 13a).

[0015] The attachment 15 is a part that performs work and includes, for example, a boom 15a, an arm 15b, and a tip attachment 15c. The boom 15a is attached to the upper rotating body 13 so that it can be raised and lowered. The arm 15b is attached to the boom 15a so that it can rotate.

[0016] The tip attachment 15c is provided at the tip of the attachment 15 and rotatably attached to the arm 15b. The tip attachment 15c performs work on a work object. The work object may be soil, sand, stone, wood, metal, resin, waste, or a structure (such as a block). The tip attachment 15c is set to a type corresponding to the work object. For example, the tip attachment 15c may be a bucket used to scoop or excavate soil-like, granular, chip-like, or powder-like work objects. The tip attachment 15c may be a lifting magnet that magnetically attracts and lifts metal (the work object). The tip attachment 15c may be a device that clamps the work object (such as a grapple or nibbler) or a device that crushes the work object (such as a breaker).

[0017] (Types of Work Performed by the Work Machine 10) For example, the work machine 10 performs work of capturing a work object and moving the captured work object to a predetermined position. Specific examples of work performed by the work machine 10 are as follows. The work machine 10 may perform loading work. Loading work is work in which a work machine 10 equipped with a bucket (an example of the tip attachment 15c) excavates (captures) soil (the work object), unloads the excavated soil onto a transport vehicle (e.g., a dump truck), and loads it onto the vehicle. The work machine 10 may perform lifting magnet work. Lifting magnet work is work in which a work machine 10 equipped with a lifting magnet (an example of the tip attachment 15c) attracts (captures) metal (the work object) and moves the captured metal to a predetermined position. The work machine 10 may perform turning-over work. Turning-over work is work in which a work machine 10 equipped with a bucket or the like turns soil (the work object) upside down. The work machine 10 may perform work to transport structures. Work to transport structures is work in which a work machine 10 equipped with an end attachment 15c capable of grasping a structure (work target) grasps the structure and places the grasped structure in a predetermined position. Note that a single work machine 10 may be capable of performing multiple types of work. Whether or not a work machine 10 is capable of performing a certain type of work depends on, for example, the type of end attachment 15c, the capabilities of the work machine 10, etc.

[0018] The autonomous driving information processing device 20 is a device that processes information related to the autonomous driving of the work machine 10. The autonomous driving information processing device 20 may be equipped with a function for operating an autonomously operable work machine 10. The autonomous driving information processing device 20 may be equipped with a function for assisting in the management of an autonomously operable work machine 10. The autonomous driving information processing device 20 is communicatively connected to the work machine 10. Hereinafter, a communicative connection will also be simply referred to as a "connection". The autonomous driving information processing device 20 may be equipped with various information processing devices (devices). Specifically, for example, the autonomous driving information processing device 20 is equipped with a server 21 (computer) and an information terminal 23.

[0019] The server 21 is a computer (controller) that inputs and outputs information, performs calculations (processing), stores information, etc. For example, the functions of the server 21 are realized by a calculation unit executing a program stored in a storage unit of the server 21. The server 21 is disposed outside the work machine 10 and the information terminal 23. The server 21 is connected to the work machine 10.

[0020] The information terminal 23 is connected to the work machine 10. The information terminal 23 may be connected to the work machine 10 via the server 21, or may be connected to the work machine 10 without via the server 21. Below, a case where the information terminal 23 is connected to the work machine 10 will mainly be described as an example of connecting the autonomous driving information processing device 20 to the work machine 10. The information terminal 23 is a device that is capable of inputting and outputting information. The information terminal 23 may be portable or not. The information terminal 23 may be a tablet, a smartphone, or a personal computer. The information terminal 23 may also serve as the server 21. The information terminal 23 may be located outside the work machine 10, or may be located inside the work machine 10 (for example, it may be brought in). The information terminal 23 comprises a control unit 23c (computer), an input unit 23i, and a display unit 23o.

[0021] The control unit 23c is a computer (controller) that performs signal input / output, calculations (processing), information storage, etc. For example, the functions of the control unit 23c are realized by the calculation unit executing a program stored in the storage unit of the control unit 23c. The control unit 23c controls information related to the display of the display unit 23o, for example, causing the display unit 23o to display. The control unit 23c controls information related to the input of the input unit 23i, for example, processing information input to the input unit 23i. The control unit 23c may also perform other control functions, such as control of information communication. Note that the control unit 23c may be distributed across multiple components (a distributed system may be configured). The control unit 23c may be integrated with one or both of the input unit 23i and the display unit 23o, or may not be integrated with them. For example, the input unit 23i and the display unit 23o may be provided in the information terminal 23, and part or all of the control unit 23c may be provided in the server 21, etc.

[0022] Information related to autonomous driving is input to the input unit 23i in response to an operation by an operator. Specifically, for example, the input unit 23i may be a touch panel, a mouse, a keyboard, a device that inputs information based on information about the operator's line of sight, or a device that inputs information based on information about the position of the operator's finger in space.

[0023] The display unit 23o displays information related to autonomous driving (display output unit). Specifically, for example, the display unit 23o may be a monitor (display), a device that projects an image onto an object, or one that uses AR (Augmented Reality) technology. The image displayed by the display unit 23o (the image that the control unit 23c causes the display unit 23o to display) is referred to as a display image G. The display image G includes information related to autonomous driving. The display image G includes shapes, patterns, colors, characters, symbols, etc. The display image G includes a GUI (Graphical User Interface). Note that the information terminal 23 may also include a unit (output unit) that performs output other than display, such as an audio output unit and a vibration output unit.

[0024] (Automatic operation of the work machine 10) The work machine 10 is configured to be able to perform automatic operation based on a work plan. The work plan is information related to the work targets of the work machine 10, and is set in advance (before automatic operation) in the automatic operation information processing device 20. The work plan may include information on the target route for travel of the work machine 10. The work plan may include information on the target range TA (for example, the target capture range TA1 and target release range TA3 shown in FIG. 10D) in which the tip attachment 15c will perform work.

[0025] (Target path PA, target trajectory) The work plan may include information on a target path PA (see FIG. 10D ) (also referred to as a target trajectory) of a specific portion of the attachment 15. The specific portion of the attachment 15 may be, for example, the tip of the tip attachment 15c or the base end of the tip attachment 15c (tip of the arm 15b). The specific portion of the attachment 15 may be set in only one location, or in multiple locations. The target path PA is information that includes, for example, information on the positions (coordinates) of multiple target points and information on the order of each target point. The work plan may also include information on a target trajectory of a specific portion of the attachment 15. The above-mentioned "target trajectory" is information in which time information is added to information on the target path PA.

[0026] (Cycle, Work Phase P) A work plan may include information for causing the work machine 10 to perform a certain task (for example, loading work, lifting magnet work, etc.) over multiple cycles. One cycle may include multiple work phases P (see Figures 8 and 10D) (operation phases). Specific examples of the work phases P are as follows. Here, a specific example of the work phases P when the work machine 10 performs the task of capturing a work object and moving the captured work object to a predetermined position by automatic operation will be described. In this case, as shown in Figure 10D, the work phases P include a capture phase P1 (for example, an excavation phase), a lifting swing phase P2 (earth unloading swing phase), a release phase P3 (earth unloading phase), and a return swing phase P4. The capture phase P1 is a phase in which the tip attachment 15c captures the work object within the target capture range TA1 (for example, excavating earth). For example, the target capture range TA1 is set to a location where work objects are collected (e.g., a pile of earth and sand). The lifting and swinging phase P2 (see FIG. 8 ) is a phase in which the tip attachment 15c moves from the target capture range TA1 toward the target release range TA3 while the tip attachment 15c has captured the work object. The release phase P3 is a phase in which the tip attachment 15c releases (e.g., dumps) the work object in the target release range TA3. The target release range TA3 may be set to a predetermined range above the bed of a transport vehicle (e.g., a dump truck), a predetermined range above a container other than the bed, or a predetermined range above the ground. The return swinging phase P4 is a phase in which the tip attachment 15c moves from the target release range TA3 toward the target capture range TA1. For example, in automatic operation, a series of work phases P (one cycle of automatic operation work) consisting of a capture phase P1, a lifting and turning phase P2, a release phase P3, and a return turning phase P4 is repeatedly performed.

[0027] (Teaching) At least a part of the work plan may be set by teaching, or may be set by a method other than teaching (for example, numerical input, etc.). Information set by teaching is called teaching data. The teaching data is set by an operator operating the work machine 10 shown in FIG. 1. Specifically, for example, the operator operates the work machine 10 from aboard the work machine 10, or the operator remotely operates the work machine 10. For example, the operator operates the work machine 10 to place a specific part of the attachment 15 at a position (route, range, etc.) that is desired to be set as teaching data. Then, the teaching data is set in the autonomous driving information processing device 20 based on the position where the specific part of the attachment 15 is placed. For example, the operator operates the work machine 10 to place a specific part of the attachment 15 at a specific position in the range that is desired to be set as the target range TA (target capture range TA1, target release range TA3) shown in FIG. 10D (see the description of FIG. 6 below). The position where the specific part of the attachment 15 shown in FIG. 1 is located is calculated based on the detection results of an attitude sensor (not shown) that detects the attitude of the work machine 10. Then, a target range TA shown in FIG. 10D is set based on the position where the specific part of the attachment 15 is located. For example, the operator operates the work machine 10 to move the specific part of the attachment 15 along the route that is desired to be set as the target route PA (see the explanation of FIGS. 4A, 4B, and 4C described below). For example, the operator operates the work machine 10 to move the specific part of the attachment 15 along the route that is desired to be set as the target trajectory at a speed that is desired to be set as the target trajectory. Then, the autonomous driving information processing device 20 sets the route (or trajectory) along which the specific part of the attachment 15 has moved as the target route PA (or trajectory).

[0028] (Operation) The operation of the autonomous driving information processing device 20 shown in FIG. 1, the autonomous driving information processing method, and the autonomous driving information processing program will be described below. The following mainly describes the display image G (see FIGS. 2 to 14D) that the control unit 23c causes to be displayed on the display unit 23o. The components shown in FIG. 1 (the work machine 10, the autonomous driving information processing device 20, etc.) will be described with reference to FIG. 1. The display image G shown in FIG. 2 changes in various ways depending on the situation. The display image G has various parts (elements such as various selection units, various display units, and various setting units). Whether or not each part of the display image G is displayed, the display mode, and whether or not selection or operation is possible using the input unit 23i (active or inactive) will change in various ways depending on the situation. The following first describes the parts of the display image G that are commonly displayed in many situations, and then describes the other parts and screen transitions, etc.

[0029] 2 shows the basic screen. The basic screen includes a task selection section A, a task addition section Aadd, a function selection section B, a menu selection section F2A1, an information terminal operation content display section F2A2, a machine status display section F2A3, and a communication status display section F2A4. The basic screen also includes a home selection section H, a main display area M, a machine selection section T, and a machine addition section Tadd.

[0030] The work selection section A is a section that allows the operator to select the type of work that will be performed automatically by the work machine 10. The work selection section A displays the type of work that can be selected using the input section 23i.

[0031] This "selection" is a selection made by the worker operating the input unit 23i. Specifically, for example, if the input unit 23i is a touch panel, the selection is made by touching the work selection section A. If the input unit 23i is a mouse or a keyboard, the selection may be made by placing a cursor on the work selection section A and performing an operation to confirm the selection (clicking, pressing a predetermined key, etc.). The selection may also be made by performing a specific mouse operation or key operation (such as a shortcut key operation) to select the work selection section A. Hereinafter, the selection made by the worker operating the input unit 23i will also be simply referred to as "selection" (the same applies to "selection" and "setting" of parts other than the work selection section A).

[0032] The results of the selection of the type of work by the work selection unit A can be used for various processes. For example, if some or all of the application software used for autonomous driving differs depending on the type of work, the results of the work selection by the work selection unit A may be used to select which application software to use. In this case, the work selection unit A is an application selection unit (app selection unit). Also, for example, there are cases where the type of work is associated with the work machine 10. In this case, when narrowing down the multiple work machines 10 to those 10 that are associated with a specific type of work, the results of the work selection by the work selection unit A may be used to select the type of work to be narrowed down. Also, for example, when newly connecting a work machine 10 to the information terminal 23, the results of the work selection by the work selection unit A may be used to select the type of work to associate the connected work machine 10 with (see the explanation of FIG. 14B ).

[0033] The position of the work selection section A in the display image G can be set in various ways. The position of the work selection section A in the display image G may be the position shown in the figure or a position close to the position shown in the figure (the same applies to other parts). In the example shown in FIG. 2, the work selection section A is located in the leftmost area (application selection area) of the display image G. The work selection section A may also be located in the main display area M (see the loading work selection section F3A1 and the lifting magnet work selection section F3A2 in FIG. 3A). Note that the position of the work selection section A may be located in a position different from the position shown in the figure (the same applies to other parts).

[0034] The work selection unit A may include at least one of graphics (shape, pattern, color, etc.), text, symbols, and image changes (movement, etc.). Specifically, for example, the work selection unit A may include a graphic in a button-like shape (button shape), or may include an icon. The work selection unit A may include text (such as "loading" or "lifting magnet") that explains the function of the work selection unit A. Unless otherwise specified, the same applies to other parts, including graphics, text, symbols, etc.

[0035] When a task selection section A (a certain portion of the display image G) is selected, the selected task selection section A is displayed so that the worker can distinguish between the selected task selection section A and the unselected portions (the same applies to other selectable portions). For example, at least one of changes in figures, characters, symbols, and images may differ between the selected portion and the unselected portions. Specifically, for example, the selected portion may be brighter (highlighted) than the unselected portions.

[0036] The work selection section A is displayed for each type of work that the work machine 10 will perform in automatic operation. Specifically, for example, if the work types are loading work and lifting magnet work, the work selection section A will include a loading work selection section A1 and a lifting magnet work selection section A2. The loading work selection section A1 is a section that allows the worker to select loading work as the "work type." The lifting magnet work selection section A2 is a section that allows the worker to select lifting magnet work as the "work type."

[0037] The task addition section Aadd is a section that is selected when a task type that can be selected in the task selection section A is added. When the task addition section Aadd is selected, the control section 23c executes a function of adding a task type. For example, the task addition section Aadd is located in a position continuous with the task selection section A, and more specifically, it is located below and adjacent to the task selection section A.

[0038] The function selection section B allows the operator to select a function related to autonomous driving. For example, the function selection section B is located adjacent to the task selection section A (e.g., to the right). The function selection section B is displayed for each of a plurality of functions. The function selection section B can select a function corresponding to the type of task selected in the task selection section A. Specifically, for example, if the type of task selected in the task selection section A is loading, the function selection section B includes a teaching function selection section B1, a loading function selection section B2, a trajectory confirmation function selection section B3, a release range function selection section B4, and a setting section B5. The operation of the autonomous driving information processing device 20 when each selection section is selected will be described later. Note that a section for adding a new selection section to the function selection section B (e.g., a section similar to the task addition section Aadd) may be provided in the display image G. For example, the function selection section B may include multiple button-shaped figures.

[0039] The menu selection section F2A1 is a section that is selected when a menu screen (not shown) is displayed. The "menu screen" is a screen that allows the operator to select menu items to be displayed as the display image G (see, for example, the explanations of FIGS. 10B and 10C).

[0040] The information terminal operation content display unit F2A2 displays information indicating what is currently being done (such as an operation or setting) on ​​the information terminal 23. Specifically, for example, the information terminal operation content display unit F2A2 displays information such as which function is selected in the function selection unit B, and what is being done in the function selected in the function selection unit B (operation setting in FIG. 2).

[0041] The machine state display unit F2A3 displays the state of the work machine 10. The machine state display unit F2A3 displays the state of the work machine 10 in a manner that is distinguishable by the operator. For example, the state of the work machine 10 displayed on the machine state display unit F2A3 may include a state related to automatic operation, or may include a state other than automatic operation. For example, the state of the work machine 10 displayed on the machine state display unit F2A3 may include a state of whether the work machine 10 is normal or not. Specifically, for example, the state of the work machine 10 displayed on the machine state display unit F2A3 includes at least one of emergency stop, a state in which the lever lock is engaged (a state in which the work machine 10 cannot operate), automatic operation, and remote operation. In this case, if the state of the work machine 10 displayed on the machine state display unit F2A3 is "automatic operation," "remote manual," or "hydraulic lock," it may indicate that the work machine 10 is normal. Furthermore, if the state of the work machine 10 displayed on the machine state display unit F2A3 is "emergency stop," it may indicate that the work machine 10 is not normal. Specifically, for example, the machine state display section F2A3 may include an image in which a light corresponding to the state of the work machine 10 flashes.

[0042] For example, a status display device 13b (see Figure 1) that displays the status of the work machine 10 may be mounted on the actual work machine 10. In this case, the image of the machine status display unit F2A3 shown in Figure 2 and the display of the status display unit 13b of the actual work machine 10 are linked. Specifically, for example, the on / off state of a rotating light (an example of the status display unit 13b) of the actual work machine 10 is linked to the on / off state in the image of the machine status display unit F2A3. For example, the color of the rotating light that is lit on the actual work machine 10 is linked to the color of the graphic of the rotating light that is lit in the image of the machine status display unit F2A3.

[0043] Here, depending on the relative position (e.g., angle) between the status display device 13b of the actual work machine 10 and the worker, it may be difficult for the worker to visually check the status display device 13b. Furthermore, if the actual work machine 10 is located far away from the worker, it may be impossible or difficult for the worker to visually check the status display device 13b. In such cases, it is impossible or difficult for the worker to distinguish (determine, understand) the status of the work machine 10 by visually checking the status display device 13b. On the other hand, in this embodiment, the image of the machine status display unit F2A3 is linked to the display of the status display device 13b of the actual work machine 10. In this case, the worker can distinguish (determine, understand) the status of the work machine 10 by looking at the image of the machine status display unit F2A3 (for example, at hand) in the display image G, without looking at the status display device 13b of the actual work machine 10. As a result, a worker who looks at the image of the machine status display unit F2A3 may be able to call attention to the area around the work machine 10 depending on the status of the work machine 10, or may be able to take action, such as operating the information terminal 23.

[0044] The status of the work machine 10 may be displayed on the machine selection unit T, which will be described later. In this case, the machine selection unit T may be a "machine status display unit."

[0045] The communication status display unit F2A4 displays the communication status of the information terminal 23. The communication status display unit F2A4 may display the communication status between the information terminal 23 and the work machine 10, or may display the communication status between the information terminal 23 and the server 21. The communication status includes, for example, the states of "good," "unstable," and "not connected." The communication status display unit F2A4 displays the communication status in a manner that allows the worker to distinguish it. For example, the communication status display unit F2A4 changes at least one of the color and shape of the icon depending on the communication status.

[0046] The home selection section H is a section that is selected when displaying a home screen (for example, FIG. 3A or FIG. 3B).

[0047] The main display area M occupies the majority (e.g., more than half) of the display image G. For example, when a work machine 10 is selected in the machine selection section T, the main display area M displays information about the work machine 10 selected in the machine selection section T. Also, for example, when a function is selected in the function selection section B, the main display area M displays information about the function selected in the function selection section B. Specific examples of the main display area M will be described later.

[0048] The machine selection section T is a section that allows the worker to select one work machine 10 from a plurality of work machines 10. For example, the machine selection section T is a section that allows the worker to select a work machine 10 for which information will be displayed in the main display area M. For example, the machine selection section T is displayed at the top of the display image G, etc. The machine selection section T may include a graphic in a shape like a tab (tab shape). The machine selection section T may include an icon, etc. (not shown). The machine selection section T may include text. The machine selection section T displays information for identifying the work machine 10. For example, the machine selection section T may display the name of the work machine 10, or may display the number of the work machine 10, etc. The machine selection section T is displayed for each work machine 10. More specifically, the machine selection section T has one tab-shaped graphic for each work machine 10.

[0049] Similar to the above-described task selection section A, the machine selection section T is displayed so that the worker can distinguish between selected and unselected machine selection sections T. Specifically, for example, the selected machine selection section T is displayed (highlighted) with a higher (brighter) brightness than the unselected machine selection section T. In the example shown in FIG. 2 , two machine selection sections T are displayed, comprising a first machine selection section T1 and a second machine selection section T2. ​​The number of machine selection sections T may be three or more. There may be cases where only one machine selection section T is displayed. There may be cases where no machine selection section T is displayed in the display image G.

[0050] (Changes in machine selection section T according to the state of the work machine 10) The image of the machine selection section T changes according to the state of the work machine 10. For example, the image of the machine selection section T changes according to at least one of a state in which automatic driving is being performed, a state in which automatic driving has ended normally, and a state in which automatic driving has made an emergency stop. The image of the machine selection section T may also change according to a state of the work machine 10 other than those described above. The image of the machine selection section T is displayed in a way that allows the worker to distinguish the state of the work machine 10. Specifically, for example, the color of the machine selection section T may differ depending on the state of the work machine 10.

[0051] (Changes in machine selection section T depending on whether or not a selection has been made) After the state of a work machine 10 has changed, the image of the machine selection section T changes depending on whether the machine selection section T corresponding to the work machine 10 whose state has changed has been selected (if selected) or not (if not selected). The machine selection section T displays in a way that allows the worker to distinguish whether or not the machine selection section T corresponding to the work machine 10 whose state has changed has been selected. Specifically, for example, an unselected machine selection section T may be displayed so as to flash (alternating between high- and low-brightness images). A selected machine selection section T may be displayed so as not to flash. Even more specifically, for example, suppose that the state of the work machine 10 corresponding to an unselected second machine selection section T2 has changed (for example, automatic operation has ended normally). At this time, the second machine selection section T2 will display (for example, turn blue) an indication that the state has changed (automatic operation has ended normally) and will also display (for example, flash) an indication that the work machine has not been selected. Then, when the second machine selection unit T2 becomes selected, the display (e.g., blue) indicating that the state has changed (automatic operation has ended normally) is maintained, the display (e.g., blinking) indicating that it is not selected ends, and the display changes to indicate that it has been selected (e.g., the blinking stops).

[0052] (Display of machine selection section T according to type of work) The image of the machine selection section T changes according to the type of work to be done by autonomous driving. The image of the machine selection section T is displayed so that the worker can distinguish the type of work. Specifically, for example, the image of the frame of the machine selection section T (e.g., the color of the frame) may be changed according to the type of work associated with the work machine 10 corresponding to the machine selection section T.

[0053] (Switching of machine selection section T) A specific example of the transition of the display image G when the selected machine selection section T (selected work machine 10) is switched is as follows. For example, assume that the first machine selection section T1 is selected, and information about the work machine 10 corresponding to the first machine selection section T1 is displayed in the main display area M. At this time, the second machine selection section T2 is selected (this selection is designated as α). Then, information about the work machine 10 corresponding to the second machine selection section T2 is displayed in the main display area M. At this time, the content displayed in the main display area M is the content that was last displayed in the main display area M when the second machine selection section T2 was selected prior to the above selection (α). However, it is conceivable that in a state in which the screen transition occurs automatically (during communication connection, while waiting for the machine to assume the initial attitude after an intermediate termination of autonomous driving, etc.), a machine selection section T other than the second machine selection section T2 (first machine selection section T1) may be selected. In this case, if the "automatic screen transition state" ends before the selection (α) is made, the content displayed in the main display area M is the screen after the screen transition. Also, if the "automatic screen transition state" has not ended when the selection (α) is made, the content displayed in the main display area M is the screen in the "automatic screen transition state" (during screen transition).

[0054] (Display of other machine selection sections T, etc.) If there are zero work machines 10 connected to the information terminal 23, the machine selection section T does not have to be displayed. If there is one work machine 10 connected to the information terminal 23, the machine selection section T may or may not be displayed. If the connection between the information terminal 23 and a work machine 10 is disconnected, the machine selection section T corresponding to the work machine 10 whose connection has been disconnected may be erased. If a first machine selection section T1 disappears from the display image G, and there is a second machine selection section T2 that was adjacent (for example, adjacent to the right) to the first machine selection section T1 that has disappeared, the adjacent second machine selection section T2 may be moved to the position where the first machine selection section T1 that has disappeared was displayed. If the number of connected work machines 10 increases, a machine selection section T may be added; for example, a machine selection section T may be added to a position adjacent (for example, adjacent to the right) to a machine selection section T that is already displayed.

[0055] (Not connected tab U) When there is no work machine 10 that corresponds to the machine selection section T, an image similar to the machine selection section T (not connected tab U (see FIG. 14B)) may be displayed. The not connected tab U is displayed so that the worker can distinguish it from the machine selection section T (the part where there is a work machine 10 that corresponds to the machine selection section T). For example, the not connected tab U may include the words "not connected." Note that when there is a work machine 10 that is connected to the information terminal 23, the not connected tab U may or may not be displayed. When the not connected tab U is selected, a setting to connect the work machine 10 (see FIG. 14A) may be performed.

[0056] The machine addition section Tadd is a section that is selected when a work machine 10 to be connected to the information terminal 23 is added. For example, the machine addition section Tadd is located in a position continuous with the machine selection section T (for example, immediately to the right). When the machine addition section Tadd is selected, a new machine selection section T may be added, or settings for connecting a work machine 10 (see FIG. 14A ) may be made. The machine addition section Tadd may be displayed (or made active) when a work machine 10 to be connected to the information terminal 23 can be added, and may not be displayed (or may be made inactive) when addition is not possible. Specifically, for example, if there is a limit on the number of work machines 10 that can be connected to the information terminal 23, the machine addition section Tadd may be displayed when the number of connected work machines 10 has not reached the limit, and may not be displayed when the limit has been reached.

[0057] 3A to 14D, a description will be given below of the various parts (elements) of the display image G that differ mainly from those described with reference to Fig. 2, as well as specific examples of transitions in the display image G. Note that the transitions between screens, the conditions under which each screen is displayed, and the like can be changed in various ways.

[0058] FIG. 3A is a diagram showing a home screen. The home screen is displayed when the home selection section H is selected. The home screen includes a loading operation selection section F3A1 and a lifting magnet operation selection section F3A2. The loading operation selection section F3A1 and the lifting magnet operation selection section F3A2 are each displayed in the main display area M as, for example, an icon. The function selection section B (see FIG. 2) is not displayed on the home screen. In the example shown in FIG. 3A, the machine selection section T (see FIG. 2) is not displayed.

[0059] 3B is a diagram showing the home screen when there are multiple connected work machines 10. This home screen includes a loading work selection section F3B1, a lifting magnet work selection section F3B2, and a number display section F3B3.

[0060] The images in the work selection section A (loading work selection section F3B1 and lifting magnet work selection section F3B2) change for each work type, depending on the number of connected work machines 10 for each work type. For example, the image in the work selection section A changes so that the worker can distinguish whether or not there are connected work machines 10 for each work type (whether there are zero or not). Specifically, for example, the brightness of the lifting magnet work selection section F3B2 is lower (darker, grayed out) when the number of connected work machines 10 associated with a lifting magnet work is zero, compared to when the number is greater than zero. The number display section F3B3 is a section that shows the number of connected work machines 10 for each work type. For example, the number display section F3B3 is positioned so that it overlaps with part (e.g., the upper right part) of the image (e.g., an icon) of the loading work selection section F3B1. Like the work selection section A (loading work selection section F3B1, lifting magnet work selection section F3B2) displayed in the main display area M, the work selection section A in the left portion (app selection area) of the display image G may also change depending on the number of connected work machines 10. If there is a limit to the number of work machines 10 that can be connected to the information terminal 23, the image of the work selection section A may change depending on whether the number of connected work machines 10 has reached the limit (similar to the machine addition section Tadd described above). The all-machine stop selection section F3B4 will be described later.

[0061] The following mainly describes the case where the information terminal 23 is connected to the work machine 10 that is performing loading work.

[0062] (Teaching Function) FIG. 4A is a diagram showing the teaching top screen (teaching top screen). The teaching top screen is a screen that is displayed (for example, displayed initially) when the teaching function selection section B1 is selected. The teaching top screen includes a new teaching selection section F4A1, a taught operation selection section F4A2, a current status display section F4A3, and a machine graphic display section F4A4. The new teaching selection section F4A1 is selected by the operator when a new teaching is to be performed. The taught operation selection section F4A2 is selected when the work machine 10 is to perform autonomous driving based on teaching data that has been preset in the autonomous driving information processing device 20 (for example, teaching data that has been taught in the past). The current status display section F4A3 displays that the state of the work machine 10 is being monitored. The machine graphic display section F4A4 includes a graphic representing the work machine 10. The image representing the work machine 10 displayed in the machine graphic display section F4A4 may be a three-dimensional image (see FIG. 9A) or a two-dimensional image, and only one work machine 10 may be displayed, or multiple work machines 10 may be displayed (see FIG. 5A). The display mode of the machine graphic display section F4A4 may be changeable (for example, changeable in response to operation). For example, the angle (viewpoint) from which the work machine 10 displayed in the machine graphic display section F4A4 is viewed may be changeable (see the rotatable display section F5B2a in FIG. 5B). For example, the work machine 10 displayed in the machine graphic display section F4A4 may be enlarged and reduced. The machine graphic display section F4A4 may include a graphic of the work site where the work machine 10 will be performing work (see FIG. 9A). This graphic of the work site may be a three-dimensional image or a two-dimensional image, like the graphic of the work machine 10.

[0063] FIG. 4B is a diagram showing a pre-teach screen. The pre-teach screen is a screen that is displayed before teaching is performed. The pre-teach screen is a screen that is displayed when a new teach selection section F4A1 is selected on the teach top screen (FIG. 4A). The pre-teach screen includes a current status display section F4B1, a teach start selection section F4B2, and a cancel selection section F4B3. The current status display section F4B1 displays the status related to teaching and indicates that teaching has not yet begun. The teach start selection section F4B2 is a section that is selected when a new teaching is to be started. The cancel selection section F4B3 is a section that is selected when a new teaching is to be canceled (not started).

[0064] FIG. 4C is a diagram showing a teaching in progress screen that is displayed during teaching. The teaching in progress screen includes a current status display section F4C1, a teaching end selection section F4C2, a cancel selection section F4C3, and a machine graphic display section F4C4. The current status display section F4C1 displays that teaching is in progress. The teaching end selection section F4C2 is a section that is selected when teaching is to be ended. When the teaching end selection section F4C2 is selected, the teaching data obtained by the teaching is set (stored, saved) in the autonomous driving information processing device 20. The cancel selection section F4C3 is a section (cancel button) that is selected when the teaching is to be canceled (canceled). For example, the cancel selection section F4C3 is selected when the work machine 10 has been operated, but the data obtained by the operation is not to be used as teaching data. The machine graphic display section F4C4 includes a graphic of the work machine 10 (similar to the machine graphic display section F4A4 in FIG. 4A). The graphic of the work machine 10 displayed in the machine graphic display section F4C4 changes according to the movement of the work machine 10 during teaching (such as turning or movement of the attachment 15). The machine graphic display section F4C4 may include a graphic (for example, a curve) representing the trajectory (path) of a specific part of the attachment 15 during teaching (see FIG. 5A). In this case, the graphic representing the trajectory of the specific part of the attachment 15 is displayed superimposed on the graphic representing the work machine 10.

[0065] (Trajectory Confirmation Function) FIG. 5A is a diagram showing a trajectory confirmation screen (route confirmation screen, teaching trajectory confirmation screen). The trajectory confirmation screen is a screen that allows the operator to confirm the taught trajectory (route). The trajectory confirmation screen is displayed after teaching has been performed. The trajectory confirmation screen is displayed when the trajectory confirmation function selection unit B3 is selected. The trajectory confirmation screen comprises a current status display unit F5A1, a trajectory display unit F5A2, a machine graphic display unit F5A3, a teaching completion selection unit F5A4, a trajectory change selection unit F5A5, and a re-teach selection unit F5A6. The current status display unit F5A1 displays that teaching has been completed. The trajectory display unit F5A2 displays a graphic that indicates the taught trajectory. The trajectory display unit F5A2 displays a graphic that indicates the trajectory, superimposed on the machine graphic display unit F5A3 that includes a graphic representing the work machine 10. The graphic showing the trajectory in the trajectory display section F5A2 may be a plurality of points or a line. The teaching completion selection section F5A4 is selected when teaching is to be completed. When the teaching completion selection section F5A4 is selected, the taught trajectory is set as teaching data. The trajectory change selection section F5A5 is a section selected when the operator wants to change the taught trajectory. The re-teach selection section F5A6 is a section selected when the operator wants to teach again.

[0066] FIG. 5B is a diagram showing a trajectory change screen (route change screen). The trajectory change screen is a screen that is displayed when the trajectory is changed. The trajectory change screen is a screen that is displayed when the trajectory change selection section F5A5 (see FIG. 5A) is selected. The trajectory change screen includes a current status display section F5B1, a trajectory display section F5B2, a trajectory change section F5B4, a movement direction selection section F5B6, a trajectory change selection section F5B7, a reset selection section F5B8, and a cancel selection section F5B9. The current status display section F5B1 indicates that the screen is in a state (screen) for changing the trajectory.

[0067] The trajectory may be changed by various methods. Here, an example of a trajectory change method will be described. The trajectory display unit F5B2 displays the trajectory in the same manner as the trajectory display unit F5A2 shown in FIG. 5A. The trajectory display unit F5B2 is displayed from at least one of a viewpoint viewing the work machine 10 from a horizontal direction (for example, directly beside) and a viewpoint viewing the work machine 10 from directly above. In this case, the operator can easily grasp the positional relationship between the work machine 10 and the trajectory. Furthermore, when the operator changes the trajectory, the operator can easily understand in which direction the change should be made. The trajectory change unit F5B4 shown in FIG. 5B is a part of the trajectory that is the target of change. For example, the trajectory change unit F5B4 may be a single point that constitutes the trajectory, or may be a partial area that constitutes the trajectory. When the trajectory change unit F5B4 is selected, the trajectory change unit F5B4 becomes movable (operable) on the display image G. The movement direction selection unit F5B6 is a unit for moving the trajectory change unit F5B4. The movement direction selection unit F5B6 can operate the trajectory change unit F5B4 in multiple directions (for example, four directions (up, down, front, back, etc.)). The movement direction selection unit F5B6 may include a graphic (for example, an arrow). The movement direction selection unit F5B6 may become selectable (active) when the trajectory change unit F5B4 is selected, and may be unselectable (inactive) or hidden when the trajectory change unit F5B4 is not selected. Note that the trajectory change unit F5B4 may be movable in the display image G by an operation different from the operation of the movement direction selection unit F5B6. For example, the trajectory change unit F5B4 may be movable in the display image G by tapping and dragging the touch panel, dragging the mouse cursor, or the like.

[0068] The trajectory change selection section F5B7 is selected when the trajectory change is confirmed (determined). When the trajectory change selection section F5B7 is selected, the display image G returns to the trajectory confirmation screen (see FIG. 5A ). When the trajectory change is confirmed, the changed trajectory is set in the information terminal 23 (in the automatic driving information processing device 20) as teaching data to be used for automatic driving of the work machine 10. The reset selection section F5B8 is selected when the trajectory is to be restored to its pre-change state. For example, if multiple portions (e.g., multiple points) of the trajectory have been changed, when the reset selection section F5B8 is selected, all of the changed portions may be returned to their pre-change positions. For example, when the reset selection section F5B8 is selected, only a portion (e.g., one point) of the changed portions (e.g., multiple points) may be returned to their pre-change positions. Specifically, for example, the changed portions (e.g., multiple points) may be returned to their pre-change positions in order, starting with the last changed portion. The cancel selection section F5B9 is selected when the trajectory change is to be canceled. When the cancel selection section F5B9 is selected, the display image G returns to the trajectory confirmation screen (see FIG. 5A).

[0069] (Target Range Adjustment Function) FIG. 6 is a diagram showing a target range setting screen. The target range setting screen is a screen for setting a target range TA. In the example shown in FIG. 6, the target range setting screen is a release range setting screen (discharge position adjustment screen) that is a screen for setting a range of positions (target release range TA3) for releasing the captured work object. For example, the release range setting screen is a screen for setting a position (discharge position) for loading the excavated soil into the vehicle. The release range setting screen is displayed when the release range function selection section B4 (discharge position adjustment function selection section) is selected. The "setting" of the release range may be an adjustment (correction) of a previously set release range, or may be the setting of a new release range. For example, teaching may be performed, and then a new release range may be set. Furthermore, for example, the release range may be set without teaching. The release range setting screen includes a release range setting method display section F6A1, a setting information display section F6A2, registration selection sections F6A5 and F6A6, a relative position display section F6A7, and a setting completion selection section F6A8.

[0070] The release range setting method display unit F6A1 displays a method for setting the release range (planned unloading area). The release range setting method display unit F6A1 displays which position within the release range is to be specified. The release range setting method display unit F6A1 displays how the specified positions F6A1a and F6A1b should be specified (specific examples will be described later). The setting information display unit F6A2 displays information about the set specified positions F6A1a and F6A1b. The setting information display unit F6A2 may display the coordinates of the specified positions F6A1a and F6A1b, or may display the rotation angle of the upper rotating body 13 relative to the lower traveling body 11. The setting information display unit F6A2 displays information about each of the multiple specified positions F6A1a and F6A1b. The registration selection units F6A5 and F6A6 are selected in a predetermined state for registering the specified positions, specifically, for example, when a specific part of the attachment 15 is positioned at the specified positions F6A1a and F6A1b. For example, when the registration selection sections F6A5 and F6A6 are selected, the positions of the specific parts of the attachment 15 are registered (set) as the positions of the designated positions F6A1a and F6A1b in the information terminal 23. The registration selection sections F6A5 and F6A6 are displayed for each of the multiple designated positions F6A1a and F6A1b.

[0071] A specific example of a method for setting the release range (planned soil unloading area) is as follows. In this example, the release range is a square (e.g., a rectangle) when viewed from above. The release range is set by specifying designated positions F6A1a and F6A1b, which are the two corners of the release range that are closest to the work machine 10. Note that the release range may also be set by specifying three or more designated positions (e.g., four points) within the release range. In this example, the designated positions are registered in the order of designated position F6A1a and designated position F6A1b. When the designated position F6A1a is registered, the release range setting method display unit F6A1 displays that a specific part of the attachment 15 (the tip of the bucket in FIG. 6) is to be aligned with the designated position F6A1a. The operator operates the work machine 10 and aligns the specific part of the attachment 15 with the designated position F6A1a. In this state, the registration selection unit F6A5 for the designated position F6A1a is selected. The designated position F6A1a is then registered, information about the designated position F6A1a is displayed in the setting information display section F6A2, and the registration selection section F6A5 for the designated position F6A1a becomes inactive. The registration selection section F6A6 for the next designated position F6A1b then becomes active. Next, the designated position F6A1b is registered in the same way as when the designated position F6A1a was registered. At this time, the release range setting method display section F6A1 indicates that the bucket cutting edge should be aligned with the designated position F6A1b.

[0072] The relative position display section F6A7 displays a graphic that indicates the relative position of the release range (planned earth unloading area) and the work machine 10. The setting completion selection section F6A8 is selected when setting of the release range is completed. For example, the setting completion selection section F6A8 becomes active when all of the information for the multiple specified positions F6A1a, F6A1b has been set, and is inactive until then.

[0073] Similar to the release range, a capture range (target capture range TA1 (see FIG. 10D), for example, a planned excavation area) in which the work machine 10 captures the work object may be set. The function selection unit B may include a section (capture position function selection unit) (not shown) for selecting a function for setting the capture range.

[0074] (Preparation for Automatic Driving) Once the work plan (e.g., teaching data) to be used in automatic driving has been finalized, preparations are made to change the driving mode of the work machine 10 to an automatic driving mode (a mode in which the work machine 10 drives automatically). The driving mode of the work machine 10 is set, for example, in the controller of the work machine 10. The driving mode of the work machine 10 may include, for example, a mode in which an operator boards the work machine 10 and operates the work machine 10 (board operation mode). The driving mode of the work machine 10 may also include a mode in which an operator remotely operates the work machine 10 (remote operation mode).

[0075] The conditions for starting the automatic driving mode (automatic driving mode start conditions) can be set in various ways. For example, the automatic driving mode start conditions include at least one of the following: the work machine 10 is put into its initial position, the engine is turned off, the key is turned off, if remotely controlled, the remote control is disconnected, and the emergency stop switch is turned on. Here, a case will be described in which the automatic driving mode start conditions include all of the above conditions.

[0076] The conditions for starting the automatic driving mode include the work machine 10 being put into an initial position. The work machine 10 may be put into the initial position by operation by an operator (onboard operation or remote operation), or may be put into the initial position automatically. Here, a case where the work machine 10 is put into the initial position by operation by an operator will be described.

[0077] FIG. 7A is a diagram showing the initial attitude preparation screen. The initial attitude preparation screen is a screen that is displayed when the work machine 10 is put into the initial attitude. The initial attitude preparation screen is displayed, for example, after a work plan (e.g., teaching data) has been finalized. The initial attitude preparation screen comprises a notification section F7A1, an initial attitude display section F7A2, an initial attitude preparation completion notification section F7A3, and an initial attitude confirmation selection section F7A4. The notification section F7A1 is a section that notifies the operator that the work machine 10 should be put into the initial attitude. The notification section F7A1 notifies the operator that the "initial attitude" is the attitude displayed in the initial attitude display section F7A2. The initial attitude display section F7A2 is a section that notifies the operator of what kind of attitude the initial attitude is. The initial attitude display section F7A2 notifies the operator of the initial attitude, for example, by using a graphic. In the illustrated example, the initial posture is a state in which the boom 15a is raised at approximately 45 degrees relative to the ground, the arm 15b is lowered from the boom 15a approximately vertically, and the opening of the tip attachment 15c (bucket) is approximately horizontal. Furthermore, the initial posture is a state in which the tip attachment 15c is positioned directly above the range in which the work object is to be captured (the area to be excavated). The initial posture preparation completion notification section F7A3 displays information so that the operator can distinguish whether or not the work machine 10 has reached the initial posture. For example, the initial posture preparation completion notification section F7A3 may include an icon, and the color of the icon may change depending on whether or not the work machine 10 has reached the initial posture. The initial posture preparation completion notification section F7A3 may include text such as "OK" or "NG." The initial posture confirmation selection section F7A4 is a section (OK button) that is selected when the work machine 10 has reached the initial posture. The initial attitude confirmation selection section F7A4 may be inactive when the work machine 10 is not in the initial attitude, and may become active when the work machine 10 is in the initial attitude.

[0078] FIG. 7B is a diagram showing the work machine state preparation screen. The work machine state preparation screen is a screen that is displayed when the initial attitude confirmation selection section F7A4 (see FIG. 7A ) is selected. The work machine state preparation screen is displayed, for example, after a work plan (teaching data) has been finalized. The work machine state preparation screen comprises a condition notification section F7B1 and a condition confirmation selection section F7B2. The condition notification section F7B1 notifies (displays) the conditions for starting the automatic driving mode other than the work machine 10 being placed in its initial attitude. The condition notification section F7B1 notifies that the engine should be turned off (not running). The condition notification section F7B1 notifies that the operator needs to move sufficiently away from the work machine 10. The condition notification section F7B1 notifies that the key should be turned off (power off). The condition notification section F7B1 notifies that the connection between the remotely operating device and the work machine 10 should be disconnected when the work machine 10 is in a state where it can be remotely operated (for example, when teaching has been performed by remote operation). The condition notification unit F7B1 notifies the operator that an emergency stop switch (not shown) should be powered on. The emergency stop switch is held by an operator near the work machine 10 and is used to bring the work machine 10 to an emergency stop. The condition confirmation selection unit F7B2 is a section (OK button) that is selected when the conditions displayed in the condition notification unit F7B1 are met. The condition confirmation selection unit F7B2 may become inactive when the automatic driving mode start conditions are not met, and may become active when the automatic driving mode start conditions are met.

[0079] FIG. 7C is a diagram showing an autonomous driving mode start notification screen. The autonomous driving mode start notification screen is a screen that is displayed when the autonomous driving mode is started. The autonomous driving mode start notification screen is a screen that is displayed when the condition notification section F7B1 (see FIG. 7B) is selected. The autonomous driving mode start notification screen includes a start notification section F7C1 and a start confirmation selection section F7C2. The start notification section F7C1 is a section that notifies the user that the autonomous driving mode will be started. The start confirmation selection section F7C2 is a section (OK button) that is selected when it is confirmed that the autonomous driving mode will be started.

[0080] A notification section such as the start notification section F7C1, which requires a confirmation operation (such as pressing an OK button) by the operator before processing can proceed, may be displayed on the display image G at various times other than when the autonomous driving mode is started. Such a notification section may be displayed, for example, at the start and end of teaching, the start and end of trajectory change, and the start and end of various processes described below. Specifically, the confirmation operation in the notification section may be, for example, the operation of pressing the OK button (the operation of selecting a selection section), or the operation of pressing the OK button with a check mark in a checkbox.

[0081] (Initial Settings) FIG. 8 is a diagram showing the automatic driving initial setting screen. The automatic driving initial setting screen is a screen for performing initial settings related to automatic driving. For example, the automatic driving initial setting screen is displayed (for example, initially) after the operation mode of the work machine 10 is switched to the automatic driving mode. For example, the automatic driving initial setting screen is displayed (for example, initially) when the loading function selection section B2 is selected. The automatic driving initial setting screen comprises an operation setting section F8A1 and an initial setting completion selection section F8A2. The operation setting section F8A1 is a section that allows the operator to set automatic driving. The operation setting section F8A1 comprises a work count setting section C1, a target work volume setting section C2, and an operating speed setting section SS. The initial setting completion selection section F8A2 is a section (OK button) that is selected when the initial settings of automatic driving are completed.

[0082] The task count setting unit C1 is a unit that allows the worker to set the number of times a task will be performed by automatic operation (the number of tasks, the number of cycles of automatic operation). As described above, "setting," like "selection," is a setting that the worker performs by operating the input unit 23i. "Setting" may be performed in the same manner as the specific example of "selection," or may be set by inputting a numerical value. The task count setting unit C1 includes a unit that displays the number of tasks (a set value display unit). The task count setting unit C1 may also include a unit (an input box) into which the numerical value of the task count can be input. This input box may also serve as the set value display unit (the same applies to the target task amount setting unit C2). The task count setting unit C1 includes a selection unit (a + button) for increasing the number of tasks and a selection unit (a - button) for decreasing the number of tasks.

[0083] The target workload setting unit C2 is a unit that allows the operator to set the amount of work object to be released (target workload) when the work machine 10 moves and releases a captured work object. For example, the target workload may be the mass or volume of the work object. The target workload is, for example, the amount of earth to be loaded onto a vehicle in a loading operation. The target workload setting unit C2 includes a unit that displays the target workload (a set value display unit). The target workload setting unit C2 may also include a unit that allows the numerical value of the target workload to be input (an input box). The target workload setting unit C2 includes a selection unit (a + button) for increasing the target workload and a selection unit (a - button) for decreasing the target workload.

[0084] The operating speed setting unit SS is a section that allows the operator to set the operating speed during automatic operation. The operating speed setting unit SS displays the operating speed during automatic operation so that it can be set from multiple operating speed levels. The "multiple operating speed levels" include the operating speed of the work machine 10 in the teaching data (the operating speed during teaching). In the example shown in Figure 8, the "multiple levels" are three levels (high speed, medium speed, low speed), but may also be two levels, or four or more levels.

[0085] Of the multiple operating speed levels that can be set by the operating speed setting unit SS, the fastest operating speed (third speed setting unit SS3) is preferably the operating speed of the work machine 10 in the teaching data. In this case, the operating speed during automatic driving will never be faster than the taught operating speed. Therefore, the operating speed during automatic driving will be lower than the speed intended by the operator who gave the teaching, and lower than the speed assumed by the operator who viewed the teaching.

[0086] Of the multiple operating speed stages that can be set by the operating speed setting unit SS, the fastest operating speed (third speed setting unit SS3) may be faster than the operating speed of the work machine 10 in the taught data. In this case, it is possible to improve the work efficiency of work performed by the automatic operation of the work machine 10. For example, when taught, the work machine 10 can be operated reliably at a slow operating speed, and when automatically operated, the work machine 10 can be operated efficiently at an operating speed faster than the taught operating speed.

[0087] The operation speed setting unit SS can set the operation speed of the automatic driving as follows. The operation speed setting unit SS may include multiple selection units that can select the operation speed of the automatic driving from multiple levels. In the example shown in FIG. 8 , the operation speed setting unit SS includes a first speed setting unit SS1 (low speed), a second speed setting unit SS2 (medium speed), and a third speed setting unit SS3 (high speed). The operation speed setting unit SS may be able to set the operation speed of the automatic driving by inputting a numerical value (similar to the operation count setting unit C1 and the target work volume setting unit C2, not shown). The operation speed setting unit SS may be able to set the ratio (proportion) of the operation speed of the automatic driving to the operation speed in the teaching data. The operation speed setting unit SS may be able to set the ratio (proportion) of the speed of the automatic driving to the operation speed of the fastest level among the multiple operation speed levels (an operation speed equal to or higher than the operation speed in the teaching data).

[0088] The operating speed setting unit SS may be capable of setting an operating speed for each work phase P. More specifically, if one cycle of automatic operation has multiple work phases P, the operating speed setting unit SS is displayed for each work phase P. In the example shown in FIG. 8 , one cycle of automatic operation includes four work phases P. Specifically, for example, the work phases P include a capture phase P1 (digging phase), a lifting swing phase P2 (earth unloading swing phase), a release phase P3 (earth unloading phase), and a return swing phase P4. Then, a first speed setting unit SS1, a second speed setting unit SS2, and a third speed setting unit SS3 are displayed for each of the four work phases P.

[0089] A state in which all of the multiple stages of operating speeds for each of the multiple work phases P are displayed in the display image G (as shown in FIG. 8) is referred to as a first speed display state SP1.

[0090] After the operating speed has been set by the operating speed setting unit SS, the control unit 23c commands the work machine 10 to perform automatic driving at the operating speed set by the operating speed setting unit SS. The control unit 23c commands the work machine 10 to perform automatic steering at the operating speed set for each work phase P by the operating speed setting unit SS.

[0091] (Settings before starting autonomous driving) FIG. 9A is a diagram showing a screen before starting autonomous driving. The screen before starting autonomous driving is a screen that is displayed after the initial settings of autonomous driving (see FIG. 8) are completed. The screen before starting autonomous driving is displayed when the initial setting completion selection unit F8A2 (see FIG. 8) is selected. The screen before starting autonomous driving may be displayed when the start of autonomous driving is canceled (when the cancel selection unit F9D3 in FIG. 9D is selected), or may be displayed when the setting change (see FIGS. 9B and 9C) is completed. The screen before starting autonomous driving includes a current status display unit F9A1, a task count display unit F9A2, a target task amount display unit F9A3, and an operation speed display unit F9A4.

[0092] The current status display section F9A1 indicates that preparations are being made to start automatic operation. The task count display section F9A2 displays the set task count. This "set task count" is the task count set in the initial settings (see FIG. 8). However, if the task count set in the initial settings is changed (see FIG. 9C), the "set task count" is the changed task count. The target task volume display section F9A3 displays the set target task volume (target soil volume). This "set target task volume" is the task count set in the initial settings (see FIG. 8). However, if the target task volume set in the initial settings is changed, the "set target task volume" is the changed target task volume.

[0093] The operating speed display unit F9A4 displays the set operating speed. This "set operating speed" is the operating speed in automatic operation that was set in the initial settings (FIG. 8). However, if the operating speed in automatic operation that was set in the initial settings has been changed (see FIG. 9B), the operating speed is the operating speed after the change. The operating speed display unit F9A4 displays the operating speed for each work phase P. Hereinafter, unless otherwise specified, the operating speed in automatic operation will also be simply referred to as the "operating speed."

[0094] FIG. 9B is a diagram showing the operating speed change screen. The operating speed change screen is a screen that is displayed when the set operating speed is changed. The operating speed may be changed for each work phase P or for the entire cycle (collectively for multiple work phases P). The following describes a case where the operating speed is changed for each work phase P. The operating speed change screen includes an operating speed setting unit F9BSS and an operating speed display unit F9B2. The operating speed setting unit F9BSS displays the operating speed for automatic operation for the selected work phase P from multiple operating speed candidates. On the other hand, for work phases P that are not selected, the set operating speed is displayed in the operating speed display unit F9B2 (similar to the operating speed display unit F9A4 in FIG. 9A).

[0095] Specific examples of the operation and images of the actuation speed setting unit F9BSS will be described. The work phase P to be changed (e.g., the capture phase P1) is selected from the actuation speed display unit F9A4 shown in FIG. 9A. At this time, an image is displayed in which the actuation speed setting unit F9BSS for the selected work phase P is pulled out (e.g., pulled to the right) from the actuation speed display unit F9A4 for the selected work phase P (the capture phase P1), as shown in FIG. 9B. For example, if the actuation speed can be set (selected) in three stages, a first speed setting unit F9BSS1 (low speed), a second speed setting unit F9BSS2 (medium speed), and a third speed setting unit F9BSS3 (high speed) are displayed for the selected work phase P (the capture phase P1). When the actuation speed is set in the actuation speed setting unit F9BSS, the display of the actuation speed setting unit F9BSS ends (closes), and the actuation speed display unit F9A4 shown in FIG. 9A is displayed (the display returns).

[0096] As shown in Fig. 9B, when the actuation speed setting section F9BSS of a selected work phase P (e.g., capture phase P1) is displayed (open), the actuation speed display section F9B2 of a previously unselected work phase P may be selected. In this case, the actuation speed setting section F9BSS for the selected work phase P (capture phase P1) is hidden (closed), and the actuation speed display section F9A4 (see Fig. 9A) is displayed (returned to display). Meanwhile, for the newly selected work phase P, the display of the actuation speed display section F9B2 changes to the display of the actuation speed setting section F9BSS (open).

[0097] 9A and 9B , a state in which only the actuation speeds set by the actuation speed setting unit SS (see FIGS. 8 and 9B ) are displayed in at least some of the multiple work phases P is referred to as a second speed display state SP2. More specifically, in the example shown in FIG. 9A , the actuation speed display unit F9A4 displays only the actuation speeds set by the actuation speed setting unit SS in all of the multiple work phases P. In the example shown in FIG. 9B , the actuation speed display unit F9B2 displays only the actuation speeds set by the actuation speed setting unit SS in some of the multiple work phases P (other than the capture phase P1). Note that after entering the second speed display state SP2, the display image G may be transitioned to the first speed display state SP1 (returning to the initial setting).

[0098] FIG. 9C illustrates a task count change screen. The task count change screen is displayed when the set task count is changed. The task count change screen includes a task count change section F9C1 and a target task volume display section F9C2. The task count change section F9C1 is displayed when the task count display section F9A2 shown in FIG. 9A is selected. For example, when the task count display section F9A2 is selected, an image is displayed in which the task count change section F9C1 shown in FIG. 9C appears to be pulled out (e.g., pulled down) from the task count display section F9A2. The task count change section F9C1 displays the task count changeable. For example, the task count change section F9C1 may include a selection section (+ button) for increasing the task count and a selection section (- button) for decreasing the task count, or may include a section for inputting the changed task count. The task count change section F9C1 includes a confirmation selection section (OK button) for confirming the change to the task count. When this decision selection section is selected, the task count change section F9C1 is hidden (closed), and the task count display section F9A2 (see FIG. 9A) is displayed (the display returns). Similarly, if another section of the display image G (for example, the target task amount display section F9C2) is selected while the task count change section F9C1 is displayed, the task count change section F9CT1 is hidden, and the task count display section F9A2 (see FIG. 9A) is displayed (the display returns).

[0099] The target workload may be changed in the same way as the workload change unit F9C1 changes the number of tasks. For example, when the target workload display unit F9C2 is selected, a section for changing the target workload (target workload change unit) may be displayed.

[0100] The operating speed, the number of operations, and the target work volume may be changed only when the work machine 10 is not operating automatically (for example, while preparing for automatic operation). The operating speed, the number of operations, and the target work volume may be changed when the work machine 10 is operating automatically (simultaneously with automatic operation).

[0101] The screen shown in Figure 9C includes an initial position preparation completion notification section F9C3, a key state selection section F9C4, an engine state selection section F9C5, a lever lock state selection section F9C6, and a work start selection section F9C7. The initial position preparation completion notification section F9C3 displays whether the work machine 10 is in the initial position so that the operator can distinguish (similar to the initial position preparation completion notification section F7A3 in Figure 7A). It is assumed that the position of the work machine 10 is in the initial position when the automatic driving mode is switched to the automatic driving mode. The operator can confirm that the work machine 10 is maintaining the initial position from the initial position preparation completion notification section F9C3. The key state selection section F9C4 is a section that is selected when switching the key state between on and off. The key state selection section F9C4 is also a section that displays the key state (key state display section). The engine state selection section F9C5 is a section that is selected when switching the engine state between on (RUN) (driving state) and off (stopped state). The engine state selection section F9C5 is also a section that displays the state of the engine (engine state display section). The lever lock state selection section F9C6 is selected when switching the lever lock state between a locked state (a state in which the work machine 10 cannot operate) and an unlocked state (a state in which the work machine 10 can operate). The lever lock state selection section F9C6 is also a section that displays the state of the lever lock (lever lock state display section).

[0102] The conditions for starting automatic driving (automatic driving start conditions) can be set in a variety of ways. For example, the automatic driving start conditions include at least one of the following: the driving mode of the work machine 10 is the automatic driving mode; the work machine 10 is in an initial posture; the key is on; the engine is on; and the lever lock is in a released state. In the example shown in FIG. 9C , the key is off, the engine is off, and the lever lock is in a locked state, and the automatic driving start conditions are not satisfied. Therefore, the information terminal 23 is operated by the operator so that the automatic driving start conditions are satisfied. Specifically, for example, the key state selection unit F9C4 is selected and the key is changed from off to on. The engine state selection unit F9C5 is selected and the engine is changed from off to on (RUN). The lever lock state selection unit F9C6 is selected and the lever lock is changed from a locked state to a released state. A confirmation screen (similar to FIG. 7C ) may be displayed when at least one of the key state selection unit F9C4, the engine state selection unit F9C5, and the lever lock state selection unit F9C6 is operated. For example, a notification may be given that the state of the work machine 10 is changing, such as a notification to start the engine when the engine changes from off to on. Also, a notification may be given that it is necessary to check that there are no people around the work machine 10.

[0103] The relationship between the key state and the engine state can be set in various ways. For example, the condition for turning on the engine may include the key being on. The engine may be turned on automatically when the key is on. When the engine switches from on to off, the key may also be turned off, or the key may remain on.

[0104] The work start selection unit F9C7 is a part that is selected when automatic driving is started. For example, the work start selection unit F9C7 may be active when an automatic driving start condition is satisfied and inactive when the automatic driving start condition is not satisfied.

[0105] FIG. 9D is a diagram showing a confirmation screen before autonomous driving starts. The confirmation screen before autonomous driving starts is a screen that is displayed before (for example, immediately before) autonomous driving starts. The confirmation screen before autonomous driving starts is a screen that is displayed when the work start selection section F9C7 (see FIG. 9C) is selected. The confirmation screen before autonomous driving starts includes an autonomous driving start notification section F9D1, a start selection section F9D2, and a cancel selection section F9D3. The autonomous driving start notification section F9D1 is a section that notifies (displays) that autonomous driving will start. The start selection section F9D2 is a section (OK button) that is selected when autonomous driving starts. The cancel selection section F9D3 is a section (cancel button) that is selected when autonomous driving will not start.

[0106] (During automatic operation) Fig. 10A is a diagram showing an automatic operation screen (first screen, automatic operation detail screen). The automatic operation screen is displayed during automatic operation. The automatic operation screen may be displayed when the work machine 10 is on standby (on standby). The automatic operation screen may be displayed when the work machine 10 has made an emergency stop (during emergency stop). The automatic operation screen comprises a current status display section F10A1, a remaining task count display section F10A2, a target workload display section F10A3a, and a current workload display section F10A3b. The automatic operation screen comprises a machine information display section F10A4, a viewpoint change selection section F10A5, a simple screen selection section F10A6, a horn operation section F10A7, and an interim termination selection section F10A8. The current status display section F10A1 displays that the work machine 10 is working by automatic operation. The remaining task count display section F10A2 displays the remaining number of tasks to be performed by automatic operation. This "remaining task count" is the set number of tasks (number of cycles) minus the number of tasks completed. The remaining task time display section F11A2 (see FIG. 11 ) may also be displayed. The target task amount display section F10A3a is similar to the target task amount display section F9A3 shown in FIG. 9A. The current task amount display section F10A3b displays the amount of work that has been completed (the amount of completed work) (e.g., the amount of earth loaded). The current task amount display section F10A3b may display the amount of completed work, or may display content corresponding to the amount of completed work. The current task amount display section F10A3b may also display the ratio (proportion) of the amount of completed work to the target amount of work.

[0107] The machine information display section F10A4 is a section that displays information related to the work machine 10. For example, the machine information display section F10A4 displays information detected by a sensor mounted on the work machine 10 or a sensor located external to the work machine 10. The machine information display section F10A4 may display, for example, angle information related to the work (such as a swing angle or an angle relative to the horizontal direction). The machine information display section F10A4 may display information related to the amount of work object, for example, the amount of work object (such as the amount of excavated soil) that the work machine 10 captures (excavates) in one cycle. The viewpoint change selection section F10A5 is a section that is selected when changing the viewpoint of, for example, the graphic of the work machine 10 that is displayed in the main display area M. Each time the viewpoint change selection section F10A5 is selected, the viewpoint of the work machine 10 displayed in the main display area M may be switched. The viewpoint change selection section F10A5 may be displayed for each viewpoint of the work machine 10 displayed in the main display area M (for example, it may be provided with multiple buttons). The viewpoint of the work machine 10 displayed in the main display area M may include a viewpoint from diagonally above the work machine 10 (oblique view), a viewpoint from the horizontal direction (see Figure 5), or a viewpoint from above (see Figure 5).

[0108] The simple screen selection section F10A6 is a section that is selected when the display image G is switched to the simple screen (second screen) during automatic operation (FIG. 11). The horn operation section F10A7 is a section that is selected when causing the work machine 10 to sound the horn. The mid-termination selection section F10A8 is a section that is selected when mid-terminating automatic operation (details will be described later).

[0109] FIG. 10B is a diagram illustrating a display setting screen. The display setting screen is displayed, for example, when the menu selection unit F2A1 (see FIG. 10A ) is selected and a predetermined menu item (not shown) is selected. The display setting screen includes a display setting unit F10B1. The display setting unit F10B1 is a section for configuring settings related to elements (content) to be displayed in the main display area M. The display setting unit F10B1 is a section for configuring the manner in which elements are displayed in the main display area M (e.g., whether or not to display them). The display setting unit F10B1 includes, for example, a check box or switch provided for each element for configuring whether or not to display them. Elements whose display manner in the main display area M can be configured using the display setting unit F10B1 include, for example, at least one of the taught trajectory (target route PA in FIG. 10D ), the actual work trajectory PB (see FIG. 10D ), the transport vehicle (dump truck) detection area, and the image captured by the imaging device. The actual work trajectory PB is the trajectory of a specific part of the attachment 15 when the work machine 10 actually performs work by autonomous driving (described below). The "transport vehicle (dump truck) detection area" is the range of positions where a transport vehicle (dump truck) onto which a work object is loaded is detected. The "image captured by an imaging device" is image captured by an imaging device provided on the work machine 10 or outside the work machine 10 (such as at a work site). The image captured by this imaging device may be image captured by a two-dimensional camera, or may be three-dimensional image (e.g., point cloud image) captured by a device that acquires three-dimensional information (e.g., LiDAR, stereo camera, etc.). In the example shown in FIG. 10B , the elements (content) displayed in the main display area M are a three-dimensional image representing the work machine 10 and point cloud image.

[0110] FIG. 10C is a diagram showing a sensor status check screen. The sensor status check screen is displayed, for example, when the menu selection unit F2A1 (see FIG. 10A ) is selected and a predetermined menu item (not shown) is selected. The sensor status check screen includes a sensor status display unit F10C1. The sensor status display unit F10C1 displays the status of the work machine 10 or a sensor provided external to the work machine 10. The sensor status display unit F10C1 may, for example, display whether the sensor is normal or not, or may display the content acquired by the sensor. The sensor whose status is displayed by the sensor status display unit F10C1 may be an angle sensor of each element of the attachment 15 (boom 15a, arm 15b, tip attachment 15c). The sensor whose status is displayed by the sensor status display unit F10C1 may be an imaging device (LiDAR, camera, etc.), the automatic driving controller of the work machine 10, or a controller other than the automatic driving controller of the work machine 10.

[0111] 10D is a diagram showing an automatic driving screen including a work phase display section PD, etc. The automatic driving screen including the work phase display section PD, etc. is displayed, for example, when displaying a target route PA (a taught trajectory in FIG. 10B ) is selected in the display setting section F10B1 shown in FIG. 10B . The automatic driving screen including the work phase display section PD, etc., includes a target route PA, an actual work trajectory PB, a target range TA, a work phase display section PD, a work phase character display section F10D1, and a legend F10D2.

[0112] As described above, the target path PA is a path that is a target for a specific portion of the attachment 15 (for example, the tip of the tip attachment 15c). The target path PA is displayed in the main display area M. The target path PA is displayed as a line. For example, multiple types of target paths PA are set to correspond to multiple work phases P. Specifically, for example, the target path PA includes a capture target path PA1, a lifting and swinging target path PA2, a release target path PA3, and a return swinging target path PA4. The capture target path PA1 is the target path PA for the capture phase P1. The lifting and swinging target path PA2 is the target path PA for the lifting and swinging phase P2. The release target path PA3 is the target path PA for the release phase P3. The return swinging target path PA4 is the target path PA for the return swing phase P4.

[0113] The display manner of the multiple types of target paths PA (capture target path PA1, lift-up turning target path PA2, release target path PA3, and return turning target path PA4) may be changed for each type. For example, the color of the target path PA (e.g., at least one of hue, density, brightness, and saturation) may be changed for each type of target path PA. For example, the type of line representing the target path PA (thickness, solid line, dashed line, etc.) may be changed for each type of target path PA.

[0114] The image (specifically, a line) indicating the target path PA may indicate the direction of the target path PA. The direction of the target path PA is the direction of movement of a specific part of the attachment 15 when the specific part is moved along the target path PA. For example, the image indicating the target path PA may include an arrow, or may include an animation that moves in the direction of the target path PA.

[0115] The actual work trajectory PB is the trajectory of the attachment 15 when the work machine 10 actually performs work by automatic operation. The actual work trajectory PB is displayed in the main display area M. The actual work trajectory PB is displayed at the position (trajectory) through which the attachment 15 of the work machine 10 displayed in the main display area M has passed. The actual work trajectory PB may be, for example, the trajectory (e.g., a line) of a specific portion of the attachment 15 (e.g., the tip of the tip attachment 15c), or the trajectory of the entire tip attachment 15c.

[0116] The actual work trajectory PB and the target route PA are displayed in a superimposed manner. This superimposed display allows the worker to confirm whether a specific part of the attachment 15 is operating along the target route PA (operating in accordance with the target of automatic driving).

[0117] The actual work trajectory PB is displayed so as to be distinguishable from the target path PA. The display mode of the actual work trajectory PB differs from the display mode of the target path PA. For example, the color (e.g., hue) of the display of the actual work trajectory PB differs from the color (e.g., hue) of the display of the target path PA. The color of the actual work trajectory PB may be a single color or may have multiple colors (e.g., a gradation). The display mode of the actual work trajectory PB may change depending on the type of work phase P (currently being performed work phase P) being performed by the work machine 10 (described below).

[0118] The actual work trajectory PB may be displayed using animation. For example, a portion of the actual work trajectory PB may be erased from the display image G when a predetermined condition (trajectory display condition) is no longer satisfied. By erasing a portion of the actual work trajectory PB, the problem of the actual work trajectory PB being displayed too much and making the display image G difficult to see is suppressed. Specifically, for example, when a predetermined time (trajectory display time) has elapsed since a specific portion of the attachment 15 passed a certain position, the display of the actual work trajectory PB at this "certain position" may be erased. This trajectory display time may be a fixed time set in advance, or may be arbitrarily set by the worker.

[0119] The target range TA is the range that is the target for work performed by the automatic operation of the work machine 10. The target range TA is displayed in the main display area M. Multiple types of target ranges TA are set. Specifically, for example, the target range TA comprises a target capture range TA1 and a target release range TA3. The target capture range TA1 is the range (e.g., the area to be excavated) within which the work machine 10 (the tip attachment 15c) will capture the work object in the capture phase P1. The target release range TA3 is the range (e.g., the area to be unloaded) within which the work machine 10 (the tip attachment 15c) will release the work object in the release phase P3.

[0120] The display manner of the multiple types of target ranges TA (target capture range TA1, target release range TA3) may be different for each type. For example, the color (e.g., hue) of the target range TA may be different for each type of target range TA. The display manner of each type of target range TA may correspond to the display manner of the type of target route PA corresponding to the type of target range TA. Specifically, for example, the display manner (e.g., hue) of the target capture range TA1 may be the same (or substantially the same) display manner (e.g., hue) as the capture target route PA1. Furthermore, for example, the display manner of the target release range TA3 may be the same (or substantially the same) display manner as the release target route PA3. More specifically, suppose the capture target route PA1 is displayed in orange and the release target route PA3 is displayed in green. In this case, the target capture range TA1 may be displayed in orange and the target release range TA3 may be displayed in green. Note that the display manner of each type of target range TA does not have to correspond to the display manner of the type of target route PA corresponding to the type of target range TA.

[0121] The work phase display unit PD is a display that indicates which type of work phase P, out of multiple types of work phases P, the work (automated driving work) being performed by the work machine 10 is in. Hereinafter, the type of work phase P indicated by the work phase display unit PD, that is, the work phase P of the work being performed by the work machine 10, will also be referred to as the "currently being performed work phase P", etc. The work phase display unit PD is a display that allows the worker to easily understand the currently being performed work phase P. The work phase display unit PD displays the currently being performed work phase P in a manner that allows the worker to distinguish between the currently being performed work phase P and other types of work phases P.

[0122] The work phase display unit PD may include a display of the target path PA. Specifically, the display mode of the target path PA may be different between a type of target path PA (e.g., a captured target path PA1) corresponding to the currently performed work phase P (e.g., a capture phase P1) and other types of target paths PA. The display mode that can be changed in this case may be the color of the target path PA (e.g., at least one of hue, density, brightness, and saturation) or the type of line representing the target path PA. Specifically, for example, the brightness of the type of target path PA corresponding to the currently performed work phase P may be increased, and the brightness of other types of target paths PA may be decreased (grayed out).

[0123] The work phase display unit PD may include a display of the actual work trajectory PB. Specifically, the manner of displaying the actual work trajectory PB (e.g., hue) may be changed for each type of currently performed work phase P. Each time the type of currently performed work phase P changes, the manner of displaying the actual work trajectory PB may be changed.

[0124] The work phase display unit PD may include a display of the target range TA. Specifically, the display manner of the target range TA may be different between a type of target range TA corresponding to the currently performed work phase P (e.g., a target capture range TA1) and other types of target ranges TA (e.g., a target release range TA3). The display manner that can be changed in this case may be the color (e.g., hue) of the interior of the target range TA, the color of the outer frame of the target range TA, or the type of line indicating the outer frame of the target range TA. Specifically, for example, the brightness of the type of target range TA corresponding to the currently performed work phase P may be increased, and the brightness of other types of target routes PA may be decreased (grayed out).

[0125] The work phase display unit PD may include an image of the work machine 10. Specifically, the display mode of part or the whole of the work machine 10 may be changed for each type of work phase P currently being performed. Each time the type of work phase P currently being performed changes, the display mode of part or the whole of the work machine 10 may be changed. When the display mode of only part of the work machine 10 is changed for each type of work phase P currently being performed, the part whose display mode is changed may be only the attachment 15 or only the tip attachment 15c.

[0126] For example, as described above, the color (e.g., hue, etc.) of the target route PA may differ for each type of target route PA. In this case, the color of the display of part or all of the work machine 10 may be the same or approximately the same as the color of the target route PA of the type corresponding to the currently performed work phase P. Specifically, for example, assume that the capture target route PA1 is displayed in orange, the lifting and swinging target route PA2 is displayed in blue, and the release target route PA3 is displayed in green. In this case, if the currently performed work phase P is the capture phase P1, part or all of the work machine 10 is displayed in orange. If the currently performed work phase P is the lifting and swinging phase P2, part or all of the work machine 10 is displayed in blue. If the currently performed work phase P is the release phase P3, part or all of the work machine 10 is displayed in green.

[0127] The work phase display unit PD may include a work phase character display unit F10D1. The work phase character display unit F10D1 displays characters indicating the currently performed work phase P. The display mode (e.g., color) of the work phase character display unit F10D1 may be the same as or approximately the same as the display mode (e.g., color) of the target range TA of the type corresponding to the currently performed work phase P.

[0128] The work phase display unit PD may include the entire display image G, or may include the entire main display area M. Specifically, the display manner (e.g., color) of the entire display image G may be changed for each type of work phase P currently being performed, or the display manner (e.g., color) of the entire main display area M may be changed. The display manner (e.g., color) of the entire display image G or the entire main display area M may be the same or substantially the same as the display manner (e.g., color) of the target route PA (or target range TA) of the type corresponding to the currently being performed work phase P.

[0129] The work phase display unit PD may display the type of work phase P to be performed next to the currently performed work phase P, in addition to displaying the type of currently performed work phase P. In this case, the work phase display unit PD may display the type of currently performed work phase P and the type of next performed work phase P in a manner that allows them to be distinguished from each other.

[0130] Legend F10D2 indicates which type of work phase P corresponds to a display (e.g., target route PA) that is displayed in various modes depending on the type of work phase P. For example, in the example shown in FIG. 10D , legend F10D1 indicates the relationship between the display modes of multiple types of target routes PA and the types of work phase P. For example, legend F10D2 may indicate the relationship between the hue of the target route PA and the type of work phase P (which color corresponds to which work phase P). Note that legend F10D1 may also indicate the relationship between the display mode (e.g., color) of a display other than the target route PA (e.g., target range TA, actual work trajectory PB, etc.) and the type of work phase P.

[0131] 10A 。 (Simple screen display during automatic operation) FIG. 11 is a diagram showing the simple screen during automatic operation (second screen). The simple screen during automatic operation is a screen that is displayed during automatic operation. The simple screen during automatic operation may be displayed when the work machine 10 is on standby (during standby). The simple screen during automatic operation may be displayed when the work machine 10 has made an emergency stop (during emergency stop). The display image G can switch between the screen during automatic operation (detail screen during automatic operation) (also simply referred to as the "detail screen") shown in FIG. 10A and the simple screen during automatic operation shown in FIG. 11. For example, the simple screen during automatic operation is displayed when the simple screen selection section F10A6 (see FIG. 10A) is selected on the detail screen (see FIG. 10A). The simple screen during automatic operation may be displayed by simply selecting the simple screen selection section F10A6 once (with one action). The simple screen during automatic operation may be displayed when a confirmation operation (such as pressing the OK button) is performed after the simple screen selection section F10A6 is selected.

[0132] It is conceivable that the worker may be unable to focus on the display image G, for example, when the worker is performing "other work." The "other work" is work different from the work performed on the work machine 10 for which information is displayed in the display image G, and specifically, for example, work performed on a machine (such as an excavator) that is not connected to the information terminal 23. The "other work" may be manual or remote operation of a machine. There are cases where the worker wants to understand the work status of the autonomous driving of the work machine 10 in a short visual confirmation time (for example, periodically) while performing "other work." For example, there are cases where the worker wants to understand how much progress has been made in the autonomous driving work, how much time is left until completion, etc. However, for a worker who cannot focus on the display image G, the amount of information on the details screen (see FIG. 10A ) is large. Therefore, it may be difficult for the worker to confirm the information they want to confirm in a short visual confirmation time.

[0133] Therefore, the display image G displays a simple screen during automatic driving. Compared to the detailed screen (see FIG. 10A ), the simple screen during automatic driving is displayed so that the worker can grasp information even with a short viewing time. Specifically, the amount of information displayed on the simple screen during automatic driving is less than the amount of information displayed on the detailed screen. The information related to automatic driving displayed on the simple screen during automatic driving includes only a portion of the information related to automatic driving displayed on the detailed screen. Specifically, for example, the display of the work site (such as point cloud video) displayed in the machine information display section F10A4 and main display area M shown in FIG. 10A is not displayed on the simple screen during automatic driving, as shown in FIG. 11 . Because the simple screen during automatic driving is displayed as the display image G, the worker can obtain information related to the automatic driving of the work machine 10 even by viewing the display image G for a short time. As a result, for example, even if the worker views the display image G while performing other work (such as manual or remote operation of the machine), the worker can monitor the work machine 10 without significantly impairing the efficiency of the other work (the worker can perform both other work and monitoring automatic driving).

[0134] Images displayed on the simple screen during autonomous driving are preferably simpler (less cluttered) than images displayed on the detailed screen (see FIG. 10A ). For example, information display objects displayed on the simple screen during autonomous driving may be larger than information display objects displayed on the detailed screen. The "information display objects" may be text or graphics (such as illustrations that visually represent information). The difference in brightness (contrast) between the information display object and the background color of the information display object on the simple screen during autonomous driving may be greater than the contrast between the information display object and the background color on the detailed screen. Information displayed as text on the detailed screen may be displayed as graphics on the simple screen during autonomous driving (see, for example, the current work amount display section F11A3 described below). The simple screen during autonomous driving may include parts not present on the detailed screen (e.g., the remaining work time display section F11A2 and the detailed screen selection section F11A4 described below).

[0135] The content (elements, items) displayed on the simple screen during automatic driving may be set by the operator or may be set in advance in the information terminal 23. The content displayed on the simple screen during automatic driving preferably includes information necessary for the operator to monitor automatic driving, and is as minimal as possible (preferably the bare minimum). For example, the simple screen during automatic driving preferably includes a status display section F11A1 and a display showing the progress of the automatic driving work (e.g., a remaining work time display section F11A2 and a current work amount display section F11A3). The simple screen during automatic driving also includes a detail screen selection section F11A4, a key status selection section F11A5a, an engine status selection section F11A5b, a lever lock status selection section F11A5c, a horn operation section F11A7, and an interim termination selection section F11A8.

[0136] The status display unit F11A1 is a section that displays the status of the work machine 10. The status display unit F11A1 displays the status related to the automatic operation of the work machine 10. The status of the work machine 10 displayed by the status display unit F11A1 includes, for example, at least one of: operating normally, waiting (a state in which work by automatic operation has ended and the work machine is waiting), and emergency stop. The status of the work machine 10 displayed by the status display unit F11A1 may also include detection operation in progress (a state in which the work machine 10 has detected an object such as a work target and is operating automatically), etc.

[0137] When the status display unit F11A1 displays that the work machine 10 is on standby, the worker can easily notice that the work machine 10 is on standby. Then, by the worker instructing the work machine 10 to perform automatic operation, the time that the work machine 10 is not working (i.e., is on standby) can be reduced. Therefore, the work efficiency of the work machine 10 can be improved.

[0138] When the status display unit F11A1 displays that the work machine 10 is undergoing an emergency stop, the worker can easily notice that the work machine 10 is undergoing an emergency stop. Therefore, the worker can take action earlier (for example, in a safer situation) regarding the work machine 10 that is undergoing an emergency stop.

[0139] It is preferable that the status display section F11A1 is displayed so that the worker can easily notice changes in the display of the status display section F11A1 even when the worker is not gazing at the display image G (for example, even with peripheral vision). In this case, the number of times and the amount of time the worker looks at the display image G can be reduced. When the worker views the display image G while performing other tasks, the number of times and the amount of time the worker looks at the display image G can be reduced, thereby improving the work efficiency of the other tasks. For example, it is preferable that the proportion of the status display section F11A1 in the display image G be as large as possible. For example, it is preferable that there is a high contrast between the status display section F11A1 and the background of the status display section F11A1.

[0140] The status display unit F11A1 may display characters (wording) that indicate the status of the work machine 10. This wording may be, for example, "operating normally" (or simply "operating"), "standby", or "emergency stop". The status display unit F11A1 may display a different graphic (for example, a face shape and color) for each status of the work machine 10. Specifically, for example, the graphic (for example, a face graphic) displayed when the work machine 10 is operating normally and on standby may be different from the graphic displayed when the work machine 10 is on emergency stop. The display (for example, characters, graphics) of the status display unit F11A1 may be a different color for each status of the work machine 10. Specifically, for example, the display of the status display unit F11A1 may be green for normal operation, blue for standby, and red for emergency stop. The background color of the status display unit F11A1 may be a different color for each status of the work machine 10.

[0141] The display of the portion of the display image G other than the status display section F11A1 may change in accordance with changes in the status of the work machine 10 indicated by the status display section F11A1. This "portion other than the status display section F11A1" may be, for example, the entire main display area M, or the outer periphery (edge) of the main display area M. Specifically, for example, in accordance with changes in the status of the work machine 10 indicated by the status display section F11A1, the color (background color or edge color) of the main display area M may change, or the brightness of the main display area M may change (for example, blink). The status display section F11A1 may be displayed on a details screen (see FIG. 10A).

[0142] The remaining task time display section F11A2 is a section that displays the remaining time for task execution by automatic operation. Just as the display of the status display section F11A1 may be changed according to the status of the work machine 10, the remaining task time display section F11A2 may change its display (e.g., text color, background color, etc.) according to the remaining time for task execution by automatic operation. Note that the remaining task count display section F10A2 (see FIG. 10A) may also be displayed on the simple screen during automatic operation.

[0143] The current work volume display section F11A3 displays the same content as the current work volume display section F10A3b shown in FIG. 10A . The current work volume display section F11A3 shown in FIG. 11 displays information that is easier for the worker to grasp than the current work volume display section F10A3b shown in FIG. 10A . For example, the current work volume display section F11A3 shown in FIG. 11 may display larger text and a greater contrast between the text and the background color compared to the current work volume display section F10A3b shown in FIG. 10A . The current work volume display section F11A3 may change its display (e.g., text color, background color, etc.) depending on the current work volume. Furthermore, the current work volume display section F11A3 shown in FIG. 11 displays a graphic indicating the ratio (proportion) of the completed work volume to the target work volume. The current work volume display section F11A3 changes the displayed graphic in multiple stages (e.g., five stages) depending on the ratio (proportion). Specifically, for example, the current work amount display section F11A3 displays a graphic indicating how much soil has been loaded onto the vehicle (dump truck). The graphic in the current work amount display section F11A3 may be displayed on the detail screen (see FIG. 10A). The detail screen selection section F11A4 is a section that is selected when the display image G is switched from the simple screen during automatic driving to the detail screen (FIG. 10A).

[0144] The key state selection section F11A5a, engine state selection section F11A5b, and lever lock state selection section F11A5c have the same functions as the key state selection section F9C4, engine state selection section F9C5, and lever lock state selection section F9C6 shown in Fig. 9C, respectively. The horn operation section F11A7 shown in Fig. 11 is a section selected when causing the work machine 10 to sound its horn. In this way, the simple screen during automatic driving may be provided with selection sections for operating the work machine 10. The mid-termination selection section F11A8 is a section selected when mid-terminating automatic driving (details will be described later).

[0145] (End of automatic driving work) FIG. 12A is a diagram showing an automatic driving work end screen. The automatic driving work end screen is displayed when work performed by automatic driving has ended normally. For example, the automatic driving work end screen is displayed when the amount of work completed (loaded earth volume) has reached the target amount of work (target earth volume). The automatic driving work end screen includes an end notification section F12A1, an end selection section F12A2, and a cancel selection section F12A3. The end notification section F12A1 is a section that notifies the operator that automatic driving will end. The end notification section F12A1 notifies the operator that work using the teaching data for which automatic driving was performed will end. The end selection section F12A2 is a section that is selected when work using the teaching data for which automatic driving was performed will end. The cancel selection section F12A3 is a section that is selected when work using the teaching data for which automatic driving was performed will not end. For example, if the cancel selection section F12A3 is selected, the screen may return to the screen before the start of automatic driving (see Figure 9A), or automatic driving may be resumed after, for example, the number of tasks and the target amount of work are changed.

[0146] When the work is completed by the automated driving, the results of the work may be displayed (on the result display unit). The results of the work may include, for example, at least one of the amount of work object that has been completed, the number of times the work has been performed, and the time required for the work.

[0147] FIG. 12B is a diagram showing the mid-way termination screen. The mid-way termination screen is a screen displayed when automatic operation is terminated. Mid-way termination of automatic operation occurs when the amount of work completed (loaded earth volume) has not reached the target amount of work (target earth volume). The mid-way termination screen is displayed, for example, when the mid-way termination selection section F10A8 (see FIG. 10A) or F11A8 (see FIG. 11) is selected. The mid-way termination screen includes a mid-way termination notification section F12B1, a termination selection section F12B2, and a cancel selection section F12B3. The mid-way termination notification section F12B1 notifies the user that automatic operation will be terminated. The mid-way termination notification section F12B1 notifies the user that work will be terminated in the current cycle. The termination selection section F12B2 is a section (OK button) selected when automatic operation is terminated. The cancel selection section F12B3 is a section (cancel button) that is selected when automatic driving is not to be terminated midway (automatic driving is to be continued).

[0148] When automatic driving is terminated midway, the work machine 10 continues automatic driving until it assumes its initial attitude, and then stops in the initial attitude. This "initial attitude" is, for example, the initial attitude at the start of automatic driving. If multiple work phases P (see FIG. 8) are set, this "initial attitude" is the attitude at the start of the first work phase P (for example, capture phase P1 (see FIG. 8)). A specific example of midway termination of automatic driving is as follows. For example, suppose that the midway termination selection unit F11A8 (see FIG. 11) is selected during the release phase P3 (see FIG. 8). At this time, the work machine 10 completes the release phase P3, completes the return swing phase P4 (see FIG. 8), and stops in a state where it assumes its initial attitude. A notification that work is ending (midway termination display unit) may be displayed from the time the midway termination selection unit F11A8 (see FIG. 11) is selected until the work machine 10 assumes its initial attitude.

[0149] (Emergency Stop) Fig. 13A is a diagram showing an emergency stop screen. The emergency stop screen is a screen that is displayed when the work machine 10 comes to an emergency stop.

[0150] The conditions under which the work machine 10 makes an emergency stop (emergency stop conditions) may include selection of the emergency stop selection section F13A1. The emergency stop conditions may include selection of the all-machine stop selection section F13A2. The emergency stop conditions may be conditions other than these, and may include conditions set in the controller of the work machine 10 or the automatic driving information processing device 20.

[0151] The emergency stop selection section F13A1 is a section (for example, an emergency stop button) that is selected when making an emergency stop of the work machine 10 (the work machine 10 corresponding to the selected machine selection section T) whose information is displayed in the main display area M. The emergency stop selection section F13A1 is displayed during automatic driving, and may also be displayed at times other than automatic driving.

[0152] The all-machine stop selection section F13A2 is a section (for example, an all-machine emergency stop button) that is selected when bringing all of the work machines 10 connected (already connected) to the information terminal 23 to an emergency stop.

[0153] When the all-machine stop selection unit F13A2 is provided, the following effects can be obtained. For example, it is assumed that, in order to stop the work machines 10 from the information terminal 23, the worker selects the machine selection unit T corresponding to the work machine 10 that the worker wants to stop, and then selects the emergency stop selection unit F13A1. In this case, if the work machine 10 that the worker wants to stop is different from the work machine 10 selected in the machine selection unit T, the worker needs to select the machine selection unit T corresponding to the work machine 10 that the worker wants to stop (switch screens), and then select the emergency stop selection unit F13A1. This requires the worker to perform the operation of selecting the machine selection unit T, which takes time. Furthermore, when multiple work machines 10 are connected to the information terminal 23, the worker must take multiple steps to stop all of the connected work machines 10. Specifically, the worker needs to repeatedly switch the machine selection unit T and select the emergency stop selection unit F13A1. For example, when a disaster occurs and it is necessary to immediately perform an emergency stop on all connected work machines 10, it is undesirable for the operation to perform the emergency stop to take time. For this reason, an all-machine stop selection section F13A2 is provided. By selecting (for example, pressing) the all-machine stop selection section F13A2, the operator can bring all connected work machines 10 to an emergency stop (all-machine stop function). At this time, the operator can stop all connected work machines 10 without selecting the machine selection section T (without switching screens).

[0154] It is preferable that this all-machine stop selection section F13A2 is always displayed when even one work machine 10 is connected to the information terminal 23. More specifically, even if the content (state) of the display image G changes (transitions) in various ways, it is preferable that the all-machine stop selection section F13A2 is always displayed in the display image G when a work machine 10 is connected to the information terminal 23. Specifically, for example, it is preferable that the all-machine stop selection section F13A2 is displayed when a work machine 10 is selected in the machine selection section T and information about the selected work machine 10 is displayed in the display image G. It is also preferable that the all-machine stop selection section F3B4 is displayed even on a screen where a work machine 10 is connected to the information terminal 23 but no work machine 10 is selected in the machine selection section T, such as the home screen shown in Figure 3B. It is to be noted that when no work machine 10 is connected to the information terminal 23, the all-machine stop selection section F13A2 shown in Figure 13A does not have to be displayed (see Figure 14A).

[0155] It is preferable that this stop all machines selection section F13A2 is always displayed at a specific position (always at the same position) on the display image G when at least one work machine 10 is connected to the information terminal 23. This "specific position" is the position of a part of the display image G, that is, one specific position within the display image G. For example, the stop all machines selection section F13A2 is displayed in a part where the display does not change even when the state of the display image G changes. Specifically, for example, the stop all machines selection section F13A2 is displayed in a part where the display does not change depending on the selection state of the machine selection section T. In the example shown in FIG. 13A , the stop all machines selection section F13A2 is displayed below the home selection section H, below the work selection section A, and in the lower left portion of the display image G. For example, the stop all machines selection section F13A2 may be displayed at the top of the display image G, or in an area above the machine selection section T (top screen header) (not shown).

[0156] This all-machine stop selection section F13A2 is always in a selectable (operable) state when even one work machine 10 is connected to the information terminal 23. When a work machine 10 is connected to the information terminal 23, this all-machine stop selection section F13A2 is always in a state where it is possible to bring all of the work machines 10 connected to the information terminal 23 to an emergency stop.

[0157] It is preferable that this all-machine stop selection unit F13A2 bring all connected work machines 10 to an emergency stop with just one operation of the all-machine stop selection unit F13A2 (one action). In this case, the operator can bring all connected work machines 10 to an emergency stop with a simple operation. Note that the all-machine stop selection unit F13A2 may also be capable of bringing all connected work machines 10 to an emergency stop with multiple operations. For example, the connected work machines 10 may be brought to an emergency stop when a confirmation operation (such as pressing the OK button) is performed after the all-machine stop selection unit F13A2 is selected.

[0158] When this all-machine stop selection section F13A2 is selected, the information terminal 23 simultaneously transmits an emergency stop signal to all of the work machines 10 that are already connected.

[0159] When a work machine 10 (one or more work machines 10) undergoes an emergency stop, the machine selection unit T is displayed, for example, as follows: The machine selection unit T corresponding to the work machine 10 that has undergone an emergency stop displays a display indicating that an emergency stop has occurred (emergency stop display). This emergency stop display is displayed so as to be distinguishable (so that the operator can distinguish it) from the display of the machine selection unit T when the work machine 10 is not emergency stopped. Specifically, for example, the color of the machine selection unit T is green when the work machine 10 is not emergency stopped, and red when the work machine 10 has emergency stopped. When there are multiple work machines 10 connected to the information terminal 23 and all of them have undergone an emergency stop, all of the machine selection units T (for example, the first machine selection unit T1 and the second machine selection unit T2) display an emergency stop. As described above, unselected machine selection units T display an indication that they are not selected (for example, blinking), and selected machine selection units T display an indication that they are selected (for example, not blinking).

[0160] The emergency stop screen includes an emergency stop notification section F13A3. The emergency stop notification section F13A3 is a section that notifies that an emergency stop has occurred in the work machine 10. The emergency stop notification section F13A3 may also notify the reason (error content) that the work machine 10 has come to an emergency stop.

[0161] When the emergency stop selection section F13A1 or the all-machine stop selection section F13A2 is selected (hereinafter also referred to as "when an emergency stop is selected"), the work machine 10 will make an emergency stop, for example, as follows. For example, when an emergency stop is selected, the work machine 10 may immediately transition to an emergency stop state. In this case, the work machine 10 will stop in the middle of the current work. Also, for example, when an emergency stop is selected, the work machine 10 may transition to an emergency stop state and stop when the current work (for example, the currently ongoing work phase P (see FIG. 8) or one cycle of automatic operation) is completed. Also, for example, when an emergency stop is selected while the work machine 10 is holding a work object, the work machine 10 may release the work object through automatic operation, then transition to an emergency stop state and stop. In this case, it is preferable that the work machine 10 releases the work object in a position where it can be placed stably. When an emergency stop is selected while the work machine 10 is holding a work object, a notification corresponding to the type of work listed below (see FIG. 13B) may be made.

[0162] After an emergency stop has been made on a work machine 10, it can be restored, for example, as follows. "Restore" may also mean that the emergency stop state is released. Specifically, for example, restoration may mean that the emergency stop state is released by turning off the engine of the work machine 10 and severing the connection between the information terminal 23 and the work machine 10. "Restore" may also mean that the emergency stop state is released and automatic operation is resumed. For example, the worker selects the machine selection section T corresponding to the work machine 10 that is to be restored, and performs an operation to restore the selected work machine 10 (restore operation). If multiple (for example, all) work machines 10 have made an emergency stop, the worker performs the restore operation on each one. Note that the information terminal 23 may be configured so that multiple work machines 10 can be restored simultaneously.

[0163] 13B is a diagram showing an emergency stop screen according to the work content. This screen includes an emergency stop notification section F13B1 and a selection section F13B2.

[0164] Here, when the work machine 10 makes an emergency stop, it is assumed that the state of the work machine 10 may be changed, for example, by turning off the engine or disengaging the automatic operation mode and switching to an operating mode allowing manual operation. However, depending on the work content, the configuration of the work machine 10, the type of end attachment 15c, and other circumstances, problems may arise when the state of the work machine 10 changes. For this reason, it is preferable to provide a notification that allows problems to be avoided. A specific example of this notification is described below. The emergency stop notification unit F13B1 is displayed when the work machine 10 performing lifting magnet work makes an emergency stop. The emergency stop notification unit F13B1 notifies the user of the nature of the problem. Specifically, the emergency stop notification unit F13B1 notifies the user that if the engine stops while the magnet is attracting a work object (metal), the work object will fall from the magnet. The emergency stop notification unit F13B1 also notifies the user of how to avoid the problem. In this example, the emergency stop notification unit F13B1 notifies the user of how to prevent the work object from falling from the magnet. Specifically, the emergency stop notification unit F13B1 notifies the operator to board the work machine 10, raise the lever lock (to lock the work machine 10), and make the work machine 10 inoperable. The emergency stop notification unit F13B1 notifies the operator to turn on the key in the cab 13a and keep the engine on (running). The emergency stop notification unit F13B1 notifies the operator to raise the lever lock, turn the key on, and press the selection unit F13B2 (OK button). When the selection unit F13B2 is selected, a notification (not shown) indicating that the automatic driving mode will be canceled is displayed, and the automatic driving mode is canceled. Thereafter, for example, the lever lock is released, the operator operates the work machine 10, and the work object is lowered. In the above example, a case where an emergency stop occurs on a work machine 10 performing lifting magnet work has been described, but a notification corresponding to the type of work may also be displayed when an emergency stop occurs on a work machine 10 performing other types of work.

[0165] (Settings) Fig. 14A is a diagram showing a connection setting screen. The connection setting screen is a screen that is displayed when settings are made for connecting the information terminal 23 to the work machine 10. The connection setting screen is displayed, for example, when the setting section B5 is selected. The connection setting screen includes a setting item selection section F14A1, a connection setting selection section F14A2, a new connection selection section F14A3, a registered machine selection section F14A4, and a parameter setting selection section F14A5.

[0166] The setting item selection section F14A1 is a section that displays one setting item selectably from a plurality of setting items. The connection setting selection section F14A2 is part of the setting item selection section F14A1. The connection setting selection section F14A2 is a section that is selected when settings related to the connection between the information terminal 23 and the work machine 10 are made. The new connection selection section F14A3 is a section that is selected when a new work machine 10 is connected to the information terminal 23. When a new work machine 10 is connected, for example, information for identifying the work machine 10 to be connected to (for example, IP address, registered machine name, etc.) is set. The above-mentioned "registered machine name" is the name of the work machine 10 that is displayed, for example, in the machine selection section T (see Figure 14B) and the registered machine selection section F14A4, etc.

[0167] Figure 14B is a diagram showing the work selection screen at new connection. The work selection screen at new connection comprises work selection sections F14BA1 and F14BA2. The work selection sections F14BA1 and F14BA2 are sections for selecting the type of work to associate with the connected work machine 10 when a new work machine 10 is connected to the information terminal 23. Note that this work selection screen at new connection is a work selection screen (app selection screen) to be associated with the newly connected work machine 10, and is not the home screen (see Figures 3A and 3B). The unit number display section F3B3 (see Figure 3B) is not displayed on this screen.

[0168] The registered machine selection unit F14A4 shown in FIG. 14A is a section that displays information about a work machine 10 (registered machine) that has previously been connected to the automatic driving information processing device 20 (e.g., information terminal 23). For example, the registered machine selection unit F14A4 displays the registered machine name of the registered machine. When a registered machine is selected from the registered machine selection unit F14A4, information about the registered machine may be notified. The notified information may include the registered machine name, or may include information about the connection between the registered machine and the automatic driving information processing device 20 (e.g., the date and time of the last connection), or may include information about the work performed at the time of the last connection (e.g., the last work time).

[0169] The display mode of the registered machine displayed in the registered machine selection unit F14A4 may change depending on whether the machine corresponds to the work selected in the work selection unit A. For example, when the loading work selection unit A1 is selected in the work selection unit A, the registered machine selection unit F14A4 may display the registered machines in a manner that allows the worker to distinguish whether the machine is compatible with loading work. Specifically, for example, the registered machine selection unit F14A4 may activate (selectable) registered machines that correspond to the type of work selected in the work selection unit A, and inactivate (unselectable) registered machines that do not correspond to the type of work selected in the work selection unit A. The registered machine selection unit F14A4 may not display registered machines that do not correspond to the type of work selected in the work selection unit A.

[0170] The parameter setting selection section F14A5 is a section that is selected when parameter setting (not shown) is to be performed. The parameters to be set may be parameters related to the work machine 10 or parameters related to automatic operation.

[0171] Figure 14C is a diagram showing a sensor status display screen. The sensor status display screen is a screen that displays the status of the sensors. The sensor status display screen comprises a sensor status selection section F14C1 and a sensor status display section F14C2. The sensor status selection section F14C1 is a section that is selected to display the sensor status display screen (sensor status display section F14C2). The sensor status display section F14C2, like the sensor status display section F10C1 (see Figure 10), displays the status of the work machine 10 or sensors provided external to the work machine 10.

[0172] FIG. 14D is a diagram showing an accessory operation screen. The accessory operation screen is a screen that displays the status of each device (accessory) of the work machine 10 and is a screen that is displayed so that the accessories can be operated (remotely controlled). The accessory operation screen includes an accessory operation selection section F14D1 and an accessory status display setting section F14D2. The accessory operation selection section F14D1 is a section that is selected to display the accessory operation screen (accessory status display setting section F14D2). The accessory status display setting section F14D2 is a section that displays the status of the accessories (accessory status display section). The accessory status display setting section F14D2 is a section that displays the status of the accessories so that the status can be set (changed) (accessory status setting section). The accessory status that can be displayed and set in the accessory status display setting section F14D2 (display-settable accessory status) may include, for example, the on / off status of lights (boom lights, cab lights, etc.). The displayable accessory status may include whether the wipers are operating and the operation interval (intermittent, continuous), whether the washer (a device that washes the windows of the operator's cab 13a) is on or off, and whether DPF (Diesel particulate filter) regeneration is on or off. The displayable accessory status may also include the work mode of the work machine 10 (high speed, low speed, fuel-saving (ECO)).

[0173] (Specific example of operation) With reference to the flowchart shown in Figure 15A, further specific examples of the operation of the autonomous driving information processing device 20 shown in Figure 1 (mainly the operation of the control unit 23c), the autonomous driving information processing method, and the autonomous driving information processing program will be described. Unless otherwise specified, the following will be described in the order of the operation (tasks, processes) of the autonomous driving information processing device 20. Note that the order of operation can be changed in various ways. Furthermore, steps S11 to S72 shown in Figure 15A will be described with reference to Figure 15A.

[0174] In step S11, the information terminal 23 shown in FIG.

[0175] In step S21, the target area TA (see FIG. 10D) is taught (area teaching is performed) (see FIG. 6).

[0176] In step S22, the target path PA (see FIG. 10D) is taught (path teaching is performed) (see FIG. 4C).

[0177] In step S31, the operation mode of the work machine 10 is changed (transitioned) to the automatic operation mode. For example, when the automatic operation mode start conditions described above are satisfied (see FIGS. 7A and 7B), the operation mode of the work machine 10 transitions to the automatic operation mode (see FIG. 7C).

[0178] In step S32, automatic driving settings (automatic driving operation settings) are performed. Specifically, as shown in Fig. 8, initial settings related to automatic driving are performed. In addition, as shown in Fig. 9A, Fig. 9B, Fig. 9C, and Fig. 9D, the operating speed of automatic driving and the like are set.

[0179] In step S41, preparations are made for the start of automatic driving in the work machine 10. For example, the attitude of the work machine 10 is changed to an attitude that allows automatic driving to begin.

[0180] In step S42, the work machine 10 starts automatic operation (see FIGS. 10A, 10B, 10C, 10D, and 11).

[0181] In steps S43, S44, and S45, the following operations are performed. When a transport vehicle (dump truck) is detected in the transport vehicle (dump truck) detection area (step S43), a target route PA (see FIG. 10D) for performing work on the transport vehicle (e.g., dumping) is displayed on the display unit 23o (step S44). When the operator selects to have the work machine 10 work in accordance with this target route PA, the work machine 10 begins work on the transport vehicle (step S45).

[0182] In step S46, the control unit 23c (see FIG. 1) monitors the automatic operation of the work machine 10. For example, as shown in FIGS. 10A and 11, the control unit 23c monitors the status of the automatic operation (progress, whether the operation is normal, etc.). The control unit 23c also monitors whether the automatic operation is to be terminated midway (whether the midway termination selection unit F10A8 in FIG. 10A has been selected), whether the automatic operation is to be stopped in an emergency (whether the emergency stop selection unit F13A1 in FIG. 13 has been selected), etc.

[0183] In step S51, the automatic driving work of the work machine 10 is completed, and the automatic driving operation of the work machine 10 is stopped. If the automatic driving has ended normally, the automatic driving work end screen shown in Fig. 12A is displayed on the display unit 23o. If the automatic driving has ended midway, the midway end screen shown in Fig. 12B is displayed on the display unit 23o.

[0184] In step S52, the results of the work performed during automatic driving are displayed on the display unit 23o. For example, the amount of work after completion of the work is displayed in the current work amount display unit F10A3b shown in Fig. 10A and the current work amount display unit F11A3 shown in Fig. 11.

[0185] In step S53, the control unit 23c (see FIG. 1) determines whether to perform the same (or substantially the same) automatic driving as the completed automatic driving. For example, the control unit 23c may make this determination based on an input (selection) by the worker. Alternatively, for example, the control unit 23c may automatically make this determination based on the work situation (e.g., the amount, number of times, and time of work). If the control unit 23c determines that the same (or substantially the same) automatic driving as the completed automatic driving will be performed (YES in step S53), the control unit 23c returns the processing flow to step S32. If the control unit 23c determines that the same (or substantially the same) automatic driving as the completed automatic driving will not be performed (NO in step S53), the control unit 23c advances the processing flow to step S61.

[0186] In step S61, preparations for ending the automatic driving mode (machine-side preparations for ending) are made in the work machine 10. For example, the control unit 23c changes the posture of the work machine 10 to the initial posture.

[0187] In step S62, the control unit 23c ends the automatic driving mode of the work machine 10.

[0188] In step S63, the control unit 23c determines whether or not to complete work on the work machine 10. For example, the control unit 23c determines whether or not to complete work under automatic operation of the work machine 10 using the information terminal 23 (for example, whether or not to complete work for that day). If the control unit 23c determines that work on the work machine 10 is to be completed (YES in step S63), the control unit 23c advances the processing flow to step S71. If the control unit 23c determines that work on the work machine 10 is not to be completed (NO in step S63), the control unit 23c returns the processing flow to step S21.

[0189] In step S71, the work machine 10 is shut down. Specifically, the engine of the work machine 10 is turned off and the power is turned off. For example, the information terminal 23 may send a command to the work machine 10 to shut down the work machine 10. Note that the work machine 10 may also be shut down by manual operation by the operator.

[0190] In step S72, the connection between the information terminal 23 and the work machine 10 is cut off.

[0191] (Processing of emergency stop of all machines) With reference to the flowchart shown in Fig. 15B, a specific example of the processing of the automatic driving information processing device 20 (mainly the control unit 23c) shown in Fig. 1 when the all-machine stop selection unit F13A2 shown in Fig. 13A is selected will be described. Unless otherwise specified, the following will be described in the order of operations (tasks, processing). Note that the order of operations can be changed in various ways.

[0192] When the all-machine stop selection section F13A2 shown in Fig. 13A is selected, the all-machine emergency stop process shown in Fig. 15B is performed (interrupted) regardless of which process shown in Fig. 15A is being performed. Steps S91 to S94 shown in Fig. 15B will be described below with reference to Fig. 15B.

[0193] In step S91, the information terminal 23 shown in FIG. 1 transmits an emergency stop signal to all of the work machines 10 that are connected to the information terminal 23 (already connected).

[0194] In step S92, all of the work machines 10 connected to the information terminal 23 come to an emergency stop.

[0195] In step S93, all of the work machines 10 that are connected to the information terminal 23 are shut down (shutdown is the same as in step S71 shown in FIG. 15A).

[0196] In step S94, the connection between the information terminal 23 and all of the work machines 10 that have already been connected to the information terminal 23 is cut off.

[0197] Note that each operation (display, selection, setting, etc.) of the autonomous driving information processing device 20 may be considered a "step" in the method and program. Specifically, for example, selection may be considered a selection step. Even more specifically, the selection of a work machine 10 by the machine selection unit T (see FIG. 2) may be considered a "work machine selection step."

[0198] (Effects of the First Invention) The effects of the automatic driving information processing device 20 shown in FIG. 1 are as follows.

[0199] [Configuration 1] The autonomous driving information processing device 20 is communicatively connected to a work machine 10 capable of autonomous driving. The autonomous driving information processing device 20 includes a display unit 23o and an input unit 23i. The display unit 23o displays a display image G including information related to autonomous driving. The input unit 23i inputs information related to autonomous driving in accordance with operation by an operator.

[0200] The above [Configuration 1] enables the worker to easily check and input information relating to the automatic driving of the work machine 10 using the automatic driving information processing device 20.

[0201] (Effect of the second invention) [Configuration 2] As shown in Fig. 2, the display image G has a work selection section A. The work selection section A is displayed for each type of work that the work machine 10 (see Fig. 1; the same applies below to the work machine 10) will perform during automatic operation. The work selection section A displays the type of work so that it can be selected using the input section 23i (see Fig. 1).

[0202] The above [Configuration 2] allows the operator to easily select the type of work to be performed by the work machine 10.

[0203] (Effect of the Third Invention) [Configuration 3] The display in the work selection section A changes for each type of work, depending on the number of work machines 10 already connected for each type of work.

[0204] The above [Configuration 3] allows the worker to easily grasp the number of connected work machines 10 for each type of work.

[0205] (Effect of the Fourth Invention) [Configuration 4] The display image G includes a function selection section B. The function selection section B displays a plurality of functions corresponding to the type of work selected in the work selection section A. The function selection section B displays a plurality of functions selectable by the input section 23i (see FIG. 1).

[0206] The above [Configuration 4] allows the worker to easily select a function according to the type of work selected in the work selection section A.

[0207] (Effect of the Fifth Invention) [Configuration 5] The display image G comprises a machine selection section T and a main display area M. The machine selection section T is displayed for each work machine 10, and is displayed so that the work machine 10 can be selected using the input unit 23i (see Figure 1). The main display area M displays information related to the work machine 10 selected in the machine selection section T. The main display area M is an area of ​​the display area of ​​the display unit 23o (see Figure 1) that is different from the machine selection section T.

[0208] The above [Configuration 5] allows the operator to easily select the work machine 10 for which information is displayed in the main display area M.

[0209] (Effect of the Sixth Invention) [Configuration 6] The image of the machine selection section T changes depending on the state of the work machine 10.

[0210] With the above [Configuration 6], the operator can easily understand the state of the work machine 10 by checking the machine selection section T.

[0211] (Effect of the Seventh Invention) [Configuration 7] The image of the machine selection section T changes depending on at least one of the following states: automatic operation is in progress; automatic operation has ended normally; and the work machine 10 has made an emergency stop.

[0212] With the above [Configuration 7], by checking the machine selection unit T, the worker can easily understand at least one of the following states: that automatic operation is in progress, that automatic operation has ended normally, and that the work machine 10 has made an emergency stop.

[0213] (Effect of the Eighth Invention) [Configuration 8] After the state of a work machine 10 has changed, the image of the machine selection section T changes depending on whether the machine selection section T corresponding to the work machine 10 whose state has changed is selected or not.

[0214] With the above [Configuration 8], the worker can easily know whether or not he has selected the machine selection section T corresponding to the work machine 10 whose status has changed, by checking the machine selection section T. As a result, the worker can easily know whether or not he has checked the information on the work machine 10 whose status has changed in the main display area M.

[0215] (Effect of the Ninth Invention) Automatic operation is performed based on teaching data that is set by the operator operating the work machine 10 .

[0216] [Configuration 9] As shown in Fig. 8, the display image G has an operating speed setting section SS. The operating speed setting section SS displays the operating speed for automatic operation so that it can be set by the input section 23i (see Fig. 1) from a plurality of operating speed stages including the operating speed of the work machine 10 in the teaching data.

[0217] The above [Configuration 9] allows the operator to easily set the operating speed of the work machine 10 in automatic operation.

[0218] (Effect of the Tenth Aspect) [Configuration 10] The highest operating speed among the multiple stages of operating speed (above [Configuration 9]) is the operating speed of the work machine 10 in the teaching data.

[0219] In the above [Configuration 10], the operating speed of the work machine 10 during automatic operation is equal to or lower than the operating speed in the teaching data. Therefore, the work machine 10 is prevented from operating automatically at a high operating speed that is different from the operator's intention.

[0220] (Effect of the Eleventh Invention) [Configuration 11] One cycle of automatic operation includes a plurality of work phases P. The operating speed setting unit SS displays the operating speed in automatic operation so that it can be set for each work phase P.

[0221] With the above [Configuration 11], the operator can easily set the operating speed of the work machine 10 during automatic operation for each work phase P.

[0222] (Effect of the 12th Invention) [Configuration 12] The display image G switches between a first speed display state SP1 and a second speed display state SP2 (see FIG. 9A ). The first speed display state SP1 is a state in which all stages of the multiple operating speeds for each of the multiple work phases P are displayed. As shown in FIG. 9A , the second speed display state SP2 is a state in which only the operating speeds set by the operating speed setting unit SS are displayed for at least some of the multiple work phases P.

[0223] The above [Configuration 12] simplifies the display in the second speed display state SP2 shown in Fig. 9A compared to the first speed display state SP1 shown in Fig. 8. Therefore, the operator can easily understand the operating speed set in the operating speed setting unit SS shown in Fig. 8.

[0224] (Effect of the thirteenth invention) [Configuration 13] As shown in Fig. 10D, one cycle of automatic operation includes multiple work phases P. The display image G is provided with a work phase display section PD that displays in which work phase P the work being performed by the work machine 10 is.

[0225] By the above [Configuration 13], the operator can easily understand which work phase P the work being performed by the work machine 10 is in.

[0226] (Effect of the 14th invention) [Configuration 14] As shown in Figure 13A, the display image G has an all-machine stop selection section F13A2 that brings all connected work machines 10 (connected to the automatic driving information processing device 20) to an emergency stop.

[0227] The above [Configuration 14] enables the worker to bring all of the work machines 10 connected to the automatic driving information processing device 20 to an emergency stop with a simple operation.

[0228] (Effect of the Fifteenth Invention) [Configuration 15] The all-machine stop selection unit F13A2 brings all connected work machines 10 (connected to the automatic driving information processing device 20) to an emergency stop with just one operation.

[0229] The above [Configuration 15] enables the operator to bring all of the work machines 10 connected to the automatic driving information processing device 20 to an emergency stop with a simpler operation.

[0230] (Effect of the 16th invention) [Configuration 16] The all-machine stop selection unit F13A2 is always displayed at a specific position in a part of the display image G when the work machine 10 is connected (when the work machine 10 is connected to the automatic driving information processing device 20).

[0231] The above [Configuration 16] allows the operator to easily grasp the position of the all-machine stop selection section F13A2 on the display image G. As a result, the operator can easily bring all of the work machines 10 connected to the automatic driving information processing device 20 to an emergency stop.

[0232] (Effect of the Seventeenth Invention) [Configuration 17] As shown in Figure 2, the display image G is provided with a machine status display section F2A3 that displays the status of the work machine 10. The image of the machine status display section F2A3 is linked to the display on the status display device 13b (see Figure 1). The status display device 13b is mounted on the actual work machine 10 and displays the status of the work machine 10.

[0233] With the above [Configuration 17], the worker can understand the status of the actual work machine 10 by looking at the machine status display section F2A3 displayed as display image G, without having to look at the status display device 13b of the actual work machine 10.

[0234] (Effect of the 18th invention) [Configuration 18] The display image G can be switched between a detailed screen (first screen) (see FIG. 10A) and a simple screen during automatic driving (second screen) (see FIG. 11). As shown in FIG. 11, the information about automatic driving displayed on the simple screen during automatic driving includes only a portion of the information about automatic driving displayed on the detailed screen (see FIG. 10A).

[0235] With the above [Configuration 18], when an operator views the simple screen during automatic driving, the operator can easily obtain information about automatic driving (e.g., in a short viewing time) compared to when the operator views the detailed screen (see Figure 10A).

[0236] (Effects of the 19th invention) [Configuration 19] The simple screen during automatic operation is provided with a status display section F11A1 that displays a status related to automatic operation. The status displayed by the status display section F11A1 includes the fact that the work machine 10 is working by automatic operation, the work machine 10 is on standby, and the work machine 10 is in an emergency stop.

[0237] The above [Configuration 19] allows the operator to easily grasp the status of the work machine 10 (for example, in a short visual confirmation time).

[0238] (Effect of the 20th Invention) [Configuration 20] An autonomous driving information processing method includes a connection step, a display step, and an input step. The connection step involves connecting to a work machine 10 capable of autonomous driving so that it can communicate with the work machine 10. The display step involves displaying a display image G including information related to autonomous driving on the display unit 23o. The input step involves acquiring information related to autonomous driving that has been input by the input unit 23i shown in FIG. 1 in response to an operator's operation.

[0239] The above [Configuration 20] allows the worker to easily check and input information relating to the automatic operation of the work machine 10.

[0240] (Effect of 21st Invention) [Configuration 21] The automatic driving information processing program causes a computer (control unit 23c) to execute a connection step, a display step, and an input step. The connection step connects to a work machine 10 capable of automatic driving so that it can communicate with the work machine 10. The display step displays a display image G including information related to automatic driving on the display unit 23o. The input step acquires information related to automatic driving that is input by the input unit 23i in response to operation by the worker.

[0241] The above [Configuration 21] allows the worker to easily check and input information relating to the automatic operation of the work machine 10.

[0242] (Modifications) The above embodiment may be modified in various ways. For example, the number of components (such as parts of the display image G) (including modifications) of the above embodiment may be changed, or some of the components may not be provided. For example, modifications of the above embodiment may be combined in various ways. For example, the arrangement of the components may be changed. For example, the inclusion relationship of the components may be changed in various ways. For example, a component described as a lower-level component included in a higher-level component may not be included in this higher-level component, but may be included in another component. For example, what is described as multiple different components or parts may be combined into a single component or part. For example, what is described as a single component or part may be provided as multiple different components or parts. For example, the order of transitions in the display image G may be changed. For example, elements described as components of different screens may be displayed on the same screen. For example, each component may have only a portion of its respective features (function, arrangement, shape, operation, etc.).

[0243] 10 Work machine 13b Status display device 20 Automatic driving information processing device 23c Control unit 23i Input unit 23o Display unit A Work selection unit B Function selection unit F2A3 Machine status display unit F3B4, F13A2 All-machine stop selection unit F11A1 Status display unit G Display image M Main display area P Work phase PD Work phase display unit SS Operation speed setting unit SP1 First speed display state SP2 Second speed display state T Machine selection unit

Claims

1. An automatic driving information processing device communicably connected to a work machine capable of performing automatic driving, a display unit that displays a display image including information related to the automatic driving, an input unit into which information related to the automatic driving is input according to an operation of an operator, comprising, an automatic driving information processing device.

2. The automatic driving information processing device according to claim 1, wherein the display image is displayed for each type of work performed by the work machine in the automatic driving, and a work selection unit for selectively displaying the type of work by the input unit is provided, an automatic driving information processing device.

3. The automatic driving information processing device according to claim 2, wherein the display of the work selection unit changes for each type of work according to the number of connected work machines for each type of work, an automatic driving information processing device.

4. The automatic driving information processing device according to claim 2 or 3, wherein the display image is displayed for each of a plurality of functions corresponding to the type of work selected in the work selection unit, and a function selection unit for selectively displaying the plurality of functions by the input unit is provided, an automatic driving information processing device.

5. The automatic driving information processing device according to claim 1, wherein the display image is displayed for each work machine, and a machine selection unit for selectively displaying the work machine by the input unit, and information related to the work machine selected in the machine selection unit is displayed, and a main display area that is an area different from the machine selection unit in the display area of the display unit are provided, an automatic driving information processing device.

6. The automatic driving information processing device according to claim 5, wherein the image of the machine selection unit changes according to the state of the work machine, an automatic driving information processing device.

7. The automatic driving information processing device according to claim 6, wherein the image of the machine selection unit changes according to at least one of a state in which the automatic driving is in progress, a state in which the automatic driving has ended normally, and a state in which the work machine has stopped abnormally, an automatic driving information processing device.

8. The automatic driving information processing device according to claim 6 or 7, wherein the image of the machine selection unit changes depending on whether the machine selection unit corresponding to the work machine whose state has changed is not selected or is selected after the state of the work machine has changed, an automatic driving information processing device.

9. The automatic driving information processing device according to claim 1, The automatic operation is performed based on teaching data set by an operator operating the work machine. The display image includes an operating speed setting unit that can be set by the input unit to display the operating speed in the automatic operation from a plurality of levels of operating speeds including the operating speed of the work machine in the teaching data. It is provided with an automatic operation information processing device.

10. The automatic operation information processing device according to claim 9, wherein the operating speed of the fastest stage among the plurality of levels of operating speeds is the operating speed of the work machine in the teaching data. an automatic operation information processing device.

11. The automatic operation information processing device according to claim 9 or 10, wherein one cycle of the automatic operation includes a plurality of work phases, and the operating speed setting unit is displayed so that the operating speed in the automatic operation can be set for each work phase. an automatic operation information processing device.

12. The automatic operation information processing device according to claim 11, wherein the display image has a first speed display state that displays all stages of the operating speeds of the plurality of levels for each of the plurality of work phases, and a second speed display state that displays only the operating speed of the stage set by the operating speed setting unit in at least some of the plurality of work phases. It switches to an automatic operation information processing device.

13. The automatic operation information processing device according to claim 1, wherein one cycle of the automatic operation includes a plurality of work phases, and the display image includes a work phase display unit that displays which work phase the work being performed by the work machine is in. an automatic operation information processing device.

14. The automatic operation information processing device according to claim 1, wherein the display image includes an all-machine stop selection unit that immediately stops all connected work machines. an automatic operation information processing device.

15. The automatic operation information processing device according to claim 14, wherein the all-machine stop selection unit immediately stops all connected work machines with a single operation. an automatic operation information processing device.

16. The automatic operation information processing device according to claim 14 or 15, wherein the all-machine stop selection unit is always displayed at a specific position in a part of the display image when the work machines are connected. an automatic operation information processing device.

17. The automatic operation information processing device according to claim 1, wherein the display image includes a machine state display unit that displays the state of the work machine. The image of the machine state display unit is mounted on the actual work machine and is linked to the display of a state display device that displays the state of the work machine. Automatic driving information processing device.

18. The automatic driving information processing device according to claim 1, The display image can be switched between a first screen and a second screen and displayed, The information related to the automatic driving displayed on the second screen includes only a part of the information related to the automatic driving displayed on the first screen. Automatic driving information processing device.

19. The automatic driving information processing device according to claim 18, The second screen is provided with a situation display unit that displays the situation related to the automatic driving, The situation displayed by the situation display unit is The work machine is in the process of working by automatic driving, The work machine is on standby, The work machine is in an emergency stop, including Automatic driving information processing device.

20. A connection step of communicably connecting to a work machine capable of performing automatic driving, A display step of displaying a display image including information related to the automatic driving on a display unit, An input step of acquiring information related to the automatic driving input by an operator at an input unit according to the operation of the operator, comprising Automatic driving information processing method.

21. A connection step of communicably connecting to a work machine capable of performing automatic driving, A display step of causing a display unit to display a display image including information related to the automatic driving, An input step of acquiring information related to the automatic driving input by an operator at an input unit according to the operation of the operator, to be executed by a computer, Automatic driving information processing program.