Automatic operation information processing device, automatic operation information processing method, and automatic operation information processing program

Abstract

The present invention facilitates confirmation and input of information relating to automatic operation of a work machine. The automatic operation information processing device (20) is communicably connected to a work machine (10) capable of automatic operation. The automatic operation information processing device (20) includes a display unit (23o) and an input unit (23i). A display unit (23o) displays information relating to an automatic operation. An input unit (23i) receives information relating to an automatic operation in accordance with an operation by an operator.

Description

Technical Field

[0001] The present invention relates to an automatic operation information processing device, an automatic operation information processing method and an automatic operation information processing program for processing information related to the automatic operation of a working machine. Background Art

[0002] For example, Patent Document 1 describes an automatically operated working machine.

[0003] Reference List

[0004] Patent Literature

[0005] [Patent Document 1] Japanese Patent Publication No. 2018-193704 Summary of the invention

[0006] Technical issues

[0007] It is desirable to allow an operator to easily confirm and input information related to the automatic operation of a work machine.

[0008] Under such circumstances, an object of the present invention is to provide an automatic operation information processing device, an automatic operation information processing method, and an automatic operation information processing program that enable easy confirmation and input of information related to automatic operation of a working machine.

[0009] Solution to the problem

[0010] The automatic operation information processing device according to the first means is connected to a working machine capable of performing automatic operation in a communicable manner. The automatic operation information processing device includes a display unit and an input unit. The display unit is configured to display a display image including information related to the automatic operation. The input unit receives information related to the automatic operation in response to an operation of an operator.

[0011] The automatic operation information processing method according to the second means includes a connection step, a display step, and an input step. In the connection step, a connection with a working machine capable of performing automatic operation is made in a communicable manner. In the display step, a display image including information related to the automatic operation is displayed on a display unit. In the input step, information on the automatic operation input by an operator through an input unit is acquired.

[0012] The automatic operation information processing program according to the third means causes a computer to execute a connection step, a display step, and an input step. In the connection step, a connection with a working machine capable of performing automatic operation is performed in a communicable manner. In the display step, a display image including information related to the automatic operation is displayed on a display unit. In the input step, information on the automatic operation input by an operator through an input unit is acquired.

[0013] Advantageous Effects of the Invention

[0014] By each of the first to third means, information on the automatic operation of the working machine can be easily checked and input. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An automatic operation system 1 including components such as an automatic operation information processing device 20 is shown.

[0016] Figure 2 Shows Figure 1 The display image G is displayed on the display unit 23o shown in , and shows a basic screen.

[0017] Figure 3A Shows Figure 2 The main screen of the display image G shown in .

[0018] Figure 3B Shows Figure 2 0 is a main screen of a display image shown in , which is shown when a plurality of working machines 10 are connected.

[0019] Figure 4A Shows Figure 2 The instruction top screen of the display image G shown in FIG.

[0020] Figure 4B Shows Figure 2 The pre-instruction screen of the display image G shown in .

[0021] Figure 4C Shows Figure 2 The instruction period screen of the display image G shown in FIG.

[0022] Figure 5A Shows Figure 2 The trajectory confirmation screen of the display image G shown in FIG.

[0023] Figure 5B Shows Figure 2 The trajectory change screen of the display image G shown in .

[0024] Figure 6 Shows Figure 2 The target range setting screen of the display image G shown in FIG.

[0025] Fig. 7A Shows Figure 2 The initial posture preparation screen of the display image G shown in FIG.

[0026] Figure 7B Shows Figure 2 The working machine state preparation screen of the display image G shown in FIG.

[0027] Figure 7C Shows Figure 2 The automatic operation mode start notification screen of the display image G shown in FIG.

[0028] Figure 8 Shows Figure 2 The automatic operation initial setting screen of the display image G shown in FIG.

[0029] Fig. 9A Shows Figure 2 The automatic operation start-up screen of the display image G shown in FIG.

[0030] Fig. 9B Shows Figure 2 The operation speed change screen of the display image G shown in FIG.

[0031] Fig. 9C Shows Figure 2 The operation count change screen of the display image G shown in FIG.

[0032] Fig.9D Shows Figure 2 The automatic operation start pre-confirmation screen of the display image G shown in FIG.

[0033] Fig. 10A Shows Figure 2 The automatic operation period screen of the display image G shown in FIG.

[0034] Fig. 10B Shows Figure 2 The display setting screen of the display image G shown in FIG.

[0035] Fig. 10C Shows Figure 2 The sensor status confirmation screen of the display image G shown in FIG.

[0036] Fig. 10D Shows Figure 2 The automatic operation period screen of the display image G shown in FIG. 1 includes a work phase display portion PD.

[0037] Fig.11 Shows Figure 2 A simple screen during automatic operation of the display image G shown in FIG.

[0038] Fig. 12A Shows Figure 2 The automatic operation end screen of the display image G shown in FIG.

[0039] Fig. 12B Shows Figure 2 The pause screen of the display image G shown in FIG.

[0040] Fig.13A Shows Figure 2 The emergency stop screen of the display image G shown in FIG.

[0041] Fig. 13B Shows Figure 2 The emergency stop screen of the display image G shown in FIG. 1 corresponds to the work content.

[0042] Fig.14A Shows Figure 2 The connection setting screen shown in the display image G.

[0043] Fig. 14B Shows Figure 2 The new connection task selection screen shown in the display image G.

[0044] Fig. 14C Shows Figure 2 The sensor status display screen of the display image G shown in FIG.

[0045] Fig.14D Shows Figure 2 The accessory operation screen of the display image G shown in FIG.

[0046] Fig.15A It shows Figure 1 A flow chart of how the automatic operating system 1 operates is shown in FIG.

[0047] Fig. 15B It shows that when you select Fig.13A When all machines shown in the stop selection section F13A2 are Figure 1 A flow chart of how the automatic operating system 1 operates is shown in FIG. DETAILED DESCRIPTION

[0048] The following will refer to Figures 1 to 15B Description of the automatic operating system 1.

[0049] The automatic operating system 1 is Figure 1 The automatic operation system 1 includes the working machine 10 and the automatic operation information processing device 20.

[0050] The work machine 10 is a machine that performs a task. Examples of the work machine 10 include a construction machine that performs a construction task, such as a forklift or a crane. The following description assumes that the work machine 10 is a forklift. The work machine 10 can be automatically operated. The work machine 10 can be operated by an operator in a cab cabin 13a (described later), or can be remotely operated. The number of work machines 10 included in the automatic operation system 1 can be one or more than one. The work machine 10 includes a machine body 10a and an attachment device 15.

[0051] The machine body 10a is the main part of the working machine 10. The machine body 10a includes a lower traveling body 11 and an upper rotating body 13. The lower traveling body 11 is capable of traveling on a traveling surface (e.g., the ground). The lower traveling body 11 may be equipped with tracks or wheels. The upper rotating body 13 is rotatably attached to the lower traveling body 11. The upper rotating body 13 includes a driver's cabin 13a and a status display device 13b. The driver's cabin 13a is a part where an operator can operate the working machine 10. The status display device 13b is a device configured to display the status of the working machine 10 (described later). For example, the status display device 13b is a rotating light, a screen, or a light. The status display device 13b is, for example, arranged on the upper surface of the upper rotating body 13 (e.g., the upper surface of the driver's cabin 13a).

[0052] The attachment device 15 is a part configured to perform a task, and includes, for example, an arm 15a, an arm 15b, and a front end attachment device 15c. The arm 15a is attached to the upper rotating body 13 in a raisable manner. The arm 15b is rotatably attached to the arm 15a.

[0053] The front end attachment device 15c is arranged at the front end portion of the attachment device 15 and is rotatably attached to the arm 15b. The front end attachment device 15c is used to perform tasks on the operating target. The operating target may be soil, sand, stone, wood, metal, resin, waste or structure (e.g., block). The front end attachment device 15c is arranged to be a type corresponding to the operating target. For example, the front end attachment device 15c may be a bucket that performs tasks such as shoveling and drilling of operating targets in the form of soil, particles, debris, powder, etc. The front end attachment device 15c may be a lifting magnet that magnetically attracts and lifts metal (operating target). The front end attachment device 15c may be a device for clamping the operating target (e.g., a grab and gnaw machine), or a device for crushing the operating target (e.g., a crusher).

[0054] (Task Type Performed by Work Machine 10)

[0055] For example, the working machine 10 performs a task of capturing an operating target and moving the captured operating target to a predetermined position. A specific example of the task performed by the working machine 10 will be described. The working machine 10 can perform a loading task. The loading task is a task in which the working machine 10 equipped with a bucket (an example of the front end attachment device 15c) digs (captures) soil (operation target) and loads the excavated soil onto a transport vehicle (such as a dump truck) by discharging the soil on the vehicle. The working machine 10 can perform a lifting magnet task. The lifting magnet task is a task in which the working machine 10 equipped with a lifting magnet (an example of the front end attachment device 15c) attracts (captures) metal (operation target) and moves the captured metal to a specified position. The working machine 10 can perform a flipping task. The flipping task is a task in which the working machine 10 equipped with a bucket, etc. flips the soil (operation target). The working machine 10 can perform a structure transport task. The structure transport task is a task in which a work machine 10 equipped with a front end attachment device 15c capable of gripping a structure (operation target) grips the structure and places the gripped structure at a predetermined position. Note that a single work machine 10 can perform multiple types of tasks. Whether the work machine 10 can perform a certain type of task depends on factors such as the type of the front end attachment device 15c and the capabilities of the work machine 10.

[0056] The automatic operation information processing device 20 is a device configured to process information related to the automatic operation of the working machine 10. The automatic operation information processing device 20 may have a function of operating the working machine 10 that can be automatically operated. The automatic operation information processing device 20 may have a function of assisting in the management of the working machine 10 that can be automatically operated. The automatic operation information processing device 20 is connected to the working machine 10 in a communicative manner. The connection in a communicative manner may be simply referred to as "connection". The automatic operation information processing device 20 may be equipped with various information processors (devices). Specifically, for example, the automatic operation information processing device 20 includes a server 21 (computer) and an information terminal 23.

[0057] The server 21 is a computer (controller) configured to perform tasks such as input and output of information, calculation (processing), and information storage. For example, the function of the server 21 is implemented in such a manner that a program stored in a storage unit of the server 21 is executed by a processing unit. The server 21 is placed outside the working machine 10 and the information terminal 23. The server 21 is connected to the working machine 10.

[0058] The information terminal 23 is connected to the working machine 10. The information terminal 23 may be connected to the working machine 10 via the server 21, or may be directly connected to the working machine 10 without the intervention of the server 21. As an example of a case where the automatic operation information processing device 20 is connected to the working machine 10, the case where the information terminal 23 is connected to the working machine 10 will be mainly described below. The information terminal 23 is a device capable of inputting and outputting information. The information terminal 23 may or may not be portable. The information terminal 23 may be a tablet computer, a smart phone, or a personal computer. The information terminal 23 may act as a server 21. The information terminal 23 may be placed outside the working machine 10 or inside the working machine 10 (for example, the information terminal 23 may be brought into the working machine 10). The information terminal 23 includes a control unit 23c (computer), an input unit 23i, and a display unit 23o.

[0059] The control unit 23c is a computer (controller) configured to perform tasks such as input and output of signals, calculation (processing), and information storage. For example, the function of the control unit 23c is implemented in such a way that a program stored in the storage unit of the control unit 23c is executed by the processing unit. The control unit 23c controls information related to the image display of the display unit 23o, for example, causing the display unit 23o to display an image. The control unit 23c controls information related to the input of the input unit 23i, for example, processing information input to the input unit 23i. The control unit 23c can perform control of information communication and other controls. The control unit 23c can be arranged at multiple parts in a distributed manner. (In other words, the control unit 23c can be a distributed system.) The control unit 23c may or may not be provided integrally with one or both of the input unit 23i and the display unit 23o. For example, the input unit 23i and the display unit 23o are arranged in the information terminal 23, and the control unit 23c may be partially or entirely arranged in the server 21, etc.

[0060] The input unit 23i receives information related to the automatic operation according to the operation of the operator. Specifically, for example, the input unit 23i can be a touch panel, a mouse, a keyboard, or a device for inputting information based on the operator's line of sight information. In addition, the input unit 23i can be a device for inputting information based on the position information of the operator's finger in space.

[0061] The display unit 23o is configured to display information related to the automatic operation (display output unit). Specifically, for example, the display unit 23o can be a monitor (display), a device configured to project an image onto an object, or a device using AR (augmented reality) technology. The image displayed on the display unit 23o (that is, the image displayed by the control unit 23c on the display unit 23o) is called a display image G. The display image G includes information related to the automatic operation. The display image G includes shapes, patterns, colors, text, symbols, etc. The display image G includes a GUI (graphical user interface). In addition, the information terminal 23 may have a unit (output unit) for performing outputs other than image display. Examples of such units include an audio output unit and a vibration output unit.

[0062] (Automatic Operation of Working Machine 10)

[0063] The work machine 10 is configured to be able to perform automatic operation based on a work plan. The work plan is information related to the target of the task to be performed by the work machine 10, and is pre-set in the automatic operation information processing device 20 (before the automatic operation). The work plan may include information on the target route of travel of the work machine 10. The work plan may include information on the target range TA (e.g., the target range TA) for the front end attachment device 15c to perform the task. Fig. 10D Information of the target capture range TA1 and target release range TA3) shown in FIG.

[0064] (Target route PA and target track)

[0065] The work plan may include information about a target route PA (also referred to as a target path) for a particular portion of the attachment device 15 (see Fig. 10D ). For example, the specific part of the attachment device 15 may be the front end part of the front end attachment device 15c or the base end part of the front end attachment device 15c (i.e., the front end part of the arm 15b). The specific part of the attachment device 15 may be set in one place or multiple places. For example, the target route PA is information including position information (coordinates) of the target point and sequence information of the target point. The work plan may include information about the target track of the specific part of the attachment device 15. The target track is information in which time information is added to the target route PA.

[0066] (Cycle and Operation Phase P)

[0067] The work plan may include information for causing the work machine 10 to perform a task (eg, a loading task or a magnet lifting task) in a plurality of cycles. One cycle may include a plurality of work phases P (operation phases) (see Figure 8 and Fig. 10D). A specific example of the operation phase P is as follows. A specific example of the operation phase P when the operation machine 10 automatically performs the task of capturing the operation target and moving the captured operation target to a predetermined position will be described below. In this case, Fig. 10D As shown in FIG. 1 , the operation phase P includes a capture phase P1 (e.g., excavation phase), a lifting and rotating phase P2 (earth discharge rotating phase), a release phase P3 (earth discharge phase), and a return rotating phase P4. The capture phase P1 is a phase in which the front end attachment device 15c captures the operation target (e.g., excavated soil) in the target capture range TA1. For example, the target capture range TA1 is set at a place where the operation targets gather (e.g., a soil pile). The lifting and rotating phase P2 (see FIG. 1 ) is a phase in which the front end attachment device 15c captures the operation target (e.g., excavated soil) in the target capture range TA1. For example, the target capture range TA1 is set at a place where the operation targets gather (e.g., a soil pile). Figure 8 ) is a stage in which the front end attachment device 15c moves from the target capture range TA1 toward the target release range TA3 while the front end attachment device 15c captures the operation target. The release stage P3 is a stage in which the front end attachment device 15c releases the operation target (e.g., soil discharge) in the target release range TA3. For example, the target release range TA3 may be set at a predetermined range on the cargo bed of a transport vehicle (e.g., a dump truck), at a predetermined range on a container outside the cargo bed, or on a predetermined range on the ground. The return rotation stage P4 is a stage in which the front end attachment device 15c moves from the target release range TA3 toward the target capture range TA1. For example, in automatic operation, a series of operation stages P (one cycle of automatic operation) including the capture stage P1, the lifting and rotation stage P2, the release stage P3, and the return rotation stage P4 are repeated.

[0068] (instruction)

[0069] At least a portion of the work plan may be set by instructions (teaching), or may be set by methods other than instructions (e.g., numerical input). Information set by instructions is called instruction data. Instruction data is set in the following manner: Figure 1 The work machine 10 shown in FIG. More specifically, for example, the operator gets on the work machine 10 and operates the work machine 10, or the operator remotely controls the work machine 10. For example, the operator places a specific part of the attachment device 15 at a desired position (path, range, etc.) according to the instruction data by operating the work machine 10. The instruction data is set in the automatic operation information processing device 20 based on the position where the specific part of the attachment device 15 is placed. For example, the operator places the specific part of the attachment device 15 at a desired position (path, range, etc.) according to the instruction data by operating the work machine 10. Fig. 10D The desired position shown in FIG. 1 is a specific position within the range of the target range TA (target capture range TA1 and target release range TA3) (see the later described Figure 6 ). Figure 1The position where the specific portion of the attachment device 15 shown in FIG. is located is calculated based on the detection result of a posture sensor (not shown) configured to detect the posture of the working machine 10. Based on the position where the specific portion of the attachment device 15 is placed, the setting Fig. 10D For example, the operator moves a specific portion of the attachment device 15 along a desired path that is desired as the target route PA by operating the work machine 10 (see the following description). Figure 4A , 4B 4C). For example, the operator moves the specific part of the attachment device 15 along a path desired as a target track at a speed desired for the target track by operating the working machine 10. Then, the automatic operation information processing device 20 sets the path (or track) along which the specific part of the attachment device 15 moves as the target route PA (or track).

[0070] (operate)

[0071] Described below Figure 1 The operation of the automatic operation information processing device 20, the automatic operation information processing method and the automatic operation information processing program shown in FIG. Figures 2 to 14D ). Figure 1 To illustrate Figure 1 Components shown in (eg, the working machine 10 and the automatic operation information processing device 20). Figure 2 The display image G shown in the figure changes variously depending on the situation. The display image G has various parts (e.g., a selection part, a display part, and a setting part). Whether each part of the display image G is displayed, how the image is displayed, and whether it can be selected or operated by the input unit 23i (whether the unit is active or inactive) changes according to the situation. Hereinafter, among the parts of the display image G, the part that is usually displayed in most cases is described, and then other parts, screen transitions, etc. are described.

[0072] Figure 2 The basic screen is shown. The basic screen includes a task selection section A, a task adding section Aadd, a function selection section B, a menu selection section F2A1, an information terminal operation content display section F2A2, a machine status display section F2A3, and a communication status display section F2A4. The basic screen also includes a homepage selection section H, a main display area M, a machine selection section T, and a machine adding section Tadd.

[0073] The task selection portion A is a portion that allows the operator to select the type of task to be automatically performed by the working machine 10. The task selection portion A displays the type of task that can be selected through the input unit 23i.

[0074] This "selection" is performed by the operator by operating the input unit 23i. Specifically, for example, when the input unit 23i is a touch panel, the selection is performed by touching the task selection part A. When the input unit 23i is a mouse or a keyboard, the selection can be performed by performing an operation (for example, clicking or pressing a specific key) when the cursor is positioned on the task selection part A. In addition, the selection can also be performed by performing a specific mouse or keyboard operation (for example, pressing a shortcut key) for selecting the task selection part A. In the following, the selection performed by the operator by operating the input unit 23i will be referred to as "selection" (including "selection" and "setting" of parts other than the task selection part A).

[0075] The selection result of the task type by the task selection unit A can be used in various processes. For example, when part or all of the application software used for automatic operation differs according to the task type, the selection result of the task by the task selection unit A can be used to select which application software to use. In this case, the task selection unit A acts as an application software selection unit (application selection unit). In addition, for example, there may be a case where a task type is associated with the working machine 10. In this case, when the working machines 10 are narrowed down to the working machines 10 associated with a specific task type, the selection result of the task by the task selection unit A can be used to select the type of task to perform the narrowed down range. In addition, for example, when the working machine 10 is newly connected to the information terminal 23, the task selection result by the task selection unit A can be used to determine which type of task the working machine 10 to be connected is associated with (see Fig. 14B ).

[0076] The position of the task selection part A in the display image G can be set in a variety of ways. The position of the task selection part A in the display image G can be at the position shown in the figure, or a position close to the position shown in the figure (the same applies to other parts). Figure 2 In the example shown in , the task selection section A is placed in the left end area (application selection area) of the display image G. The task selection section A may be set in the main display area M (see Figure 3A The loading task selection part F3A1 and the lifting magnet task selection part F3A2 shown in the figure). The position of the task selection part A may be different from the position shown in the figure. (The same applies to other parts.)

[0077] The task selection part A may include at least one of a shape (form, pattern, color), a text, a symbol, or an image change (e.g., movement). More specifically, for example, the task selection part A may include a button-like shape (button shape), or may include an icon. The task selection part A may include text explaining the function of the task selection part A, such as "loading" or "lifting magnet". Unless otherwise specified, other parts also include shapes, text, symbols, etc.

[0078] When the task selection part A (a part of the display image G) is selected, the selected task selection part A and another part that is not selected are displayed in a manner that they are distinguishable for the operator. (This also applies to other parts.) For example, the selected part and the unselected part may differ in at least one of shape, text, symbol, or image change. Specifically, for example, the brightness of the selected part may be higher (brighter) than the brightness of the unselected part. (In other words, the selected part may be highlighted.)

[0079] The task selection section A is displayed for each task type automatically performed by the working machine 10. Specifically, for example, when there are two types of tasks, namely, a loading task and a lifting magnet task, the task selection section A has a loading task selection section A1 and a lifting magnet task selection section A2. The loading task selection section A1 is a section that allows the operator to select a loading task as a task type. The lifting magnet task selection section A2 is a section that allows the operator to select a lifting magnet task as a task type.

[0080] The task adding part Aadd is a part selected when adding a task type that can be selected by the task selecting part A. When the task adding part Add is selected, the control unit 23c performs a function of adding a task type. For example, the task adding part Aadd is set at a position continuous with the task selecting part A. More specifically, the task adding part Aadd is set at a position, for example, below the task selecting part A and adjacent to the task selecting part A.

[0081] Function selection section B is a part that allows the operator to select functions related to automatic operation. For example, function selection section B is set at a position adjacent to task selection section A (for example, immediately to its right). Function selection section B is displayed for each function. Through function selection section B, a function corresponding to the task type selected by task selection section A can be selected. Specifically, for example, when the task type selected by task selection section A is a loading task, function selection section B includes instruction function selection section B1, loading function selection section B2, trajectory confirmation function selection section B3, release range function selection section B4 and setting section B5. The operation of the automatic operation information processing device 20 when each selection section is selected will be described later. A part (for example, a part similar to the task adding section Aadd) can be provided in the display image G to add a new selection section to function selection section B. For example, function selection section B may include a graphic formed as a plurality of buttons.

[0082] The menu selection section F2A1 is a portion selected when a menu screen (not shown) is displayed. The above-mentioned “menu screen” is a screen that allows the operator to select menu items to be displayed as the display image G (for example, see Fig. 10B and Fig. 10C ).

[0083] The information terminal operation content display section F2A2 displays information indicating what is currently being done (e.g., operation or setting) at the information terminal 23. Specifically, for example, the information terminal operation content display section F2A2 displays information indicating which function is selected by the function selection section B and what the function selected by the function selection section B is doing ( Figure 2 , in the Operation Settings).

[0084] The machine state display unit F2A3 displays the state of the working machine 10. The machine state display unit F2A3 displays the state of the working machine 10 in a manner that the state is distinguishable for the operator. For example, the state of the working machine 10 displayed on the machine state display unit F2A3 may include a state related to automatic operation and a state other than automatic operation. For example, the state of the working machine 10 displayed on the machine state display unit F2A3 may include a state indicating whether the working machine 10 is normal. More specifically, for example, the state of the working machine 10 displayed on the machine state display unit F2A3 may include at least one of an emergency stop, a state in which a lever lock is locked (i.e., the working machine 10 cannot be operated), a state in which automatic operation is being performed, or a state in which remote control is being performed. In this case, when the state of the working machine 10 displayed on the machine state display unit F2A3 is "automatic operation", "remote manual operation", or "hydraulic lock", the state of the working machine 10 may be indicated as normal. When the state of the working machine 10 displayed on the machine state display unit F2A3 is "automatic operation", the state of the working machine 10 may be indicated as abnormal. Specifically, for example, the machine state display unit F2A3 may include an image in which a light corresponding to the state of the working machine 10 is turned on.

[0085] For example, there is a case where a state display device 13b (see FIG. 1 ) for displaying the state of the working machine 10 is installed on the actual working machine 10. Figure 1 ). In this case, Figure 2 The image of the machine state display unit F2A3 shown in the figure is synchronized with the image display of the state display device 13b of the actual working machine 10. Specifically, for example, the on or off state of the rotating light (an example of the state display device 13b) of the actual working machine 10 is synchronized with the on or off state in the image of the machine state display unit F2A3. For example, the color of the turned-on rotating light in the actual working machine 10 is synchronized with the color of the image of the turned-on rotating light in the image of the machine state display unit F2A3.

[0086] In connection with the above, depending on the relative position (e.g., angle) between the status display device 13b of the actual working machine 10 and the operator, it may be difficult for the operator to see the status display device 13b. In addition, when the actual working machine 10 is located at a position far from the operator, it may be difficult for the operator to see the status display device 13b. In this case, it is impossible or difficult for the operator to intuitively distinguish (judge, grasp) the state of the working machine 10 by viewing the status display device 13b. On the other hand, in the present embodiment, the image of the machine status display part F2A3 is synchronized with the image display of the status display device 13b of the actual working machine 10. In this case, the operator is able to distinguish (judge, grasp) the state of the working machine 10 by viewing the image of the machine status display part F2A3 displaying the image G (e.g., the image at hand) without viewing the status display device 13b of the actual working machine 10. As a result, according to the state of the working machine 10, the operator viewing the image of the machine status display part F2A3 can issue a warning to the surrounding environment of the working machine 10, and can perform actions such as operation of the information terminal 23.

[0087] The state of the working machine 10 may be displayed in a machine selection portion T described later. In this case, the machine selection portion T may be a "machine state display portion".

[0088] The communication status display unit F2A4 displays the communication status of the information terminal 23. The communication status display unit F2A4 can display the communication status between the information terminal 23 and the working machine 10, or the communication status between the information terminal 23 and the server 21. For example, the communication status includes various states such as "good", "unstable" and "disconnected". The communication status display unit F2A4 displays the communication status in a manner that the status is distinguishable for the operator. For example, the communication status display unit F2A4 changes at least one of the color and shape of the icon according to the communication status.

[0089] The home page selection section H is a section selected when the home screen is displayed (see, for example, Figure 3A and Figure 3B ).

[0090] The main display area M is a portion that occupies most (e.g., more than half) of the display image G. For example, when the work machine 10 is selected in the machine selection portion T, the main display area M displays information about the work machine 10 selected in the machine selection portion T. Furthermore, for example, when a function is selected in the function selection portion B, the main display area M displays information about the function selected in the function selection portion B. A specific example of the main display area M will be described later.

[0091] The machine selection part T is a part that allows the operator to select a working machine 10 from a plurality of working machines 10. For example, the machine selection part T is a part that allows the operator to select a working machine 10 whose information is to be displayed in the main display area M. For example, the machine selection part T is displayed in, for example, the upper part of the display image G. The machine selection part T may include a label-like graphic (label shape). The machine selection part T may include icons and the like (not shown). The machine selection part T may include characters. The machine selection part T displays information for identifying the working machine 10. For example, the machine selection part T may display the name of the working machine 10, or may display the number of the working machine 10. The machine selection part T is displayed for each working machine 10. More specifically, the machine selection part T has a label-like graphic for each working machine 10.

[0092] In the same manner as the task selection section A described above, the mechanical selection section T displays the selected mechanical selection section T and the unselected mechanical selection section T in such a manner that these sections are distinguishable for the operator. Specifically, for example, the brightness of the selected mechanical selection section T is arranged to be higher (brighter) than the brightness of the unselected mechanical selection section T. (In other words, the selected portion is highlighted.) Figure 2 In the example shown in , two mechanical selection parts T, namely, the first mechanical selection part T1 and the second mechanical selection part T2 are displayed. The number of mechanical selection parts T may be three or more. There may be a case where only one mechanical selection part T is displayed. There may be a case where the mechanical selection part T is not displayed in the display image G.

[0093] (Changes in the Machine Selector T According to the State of the Working Machine 10)

[0094] The image of the machine selection part T changes according to the state of the working machine 10. For example, the image of the machine selection part T changes at least according to whether the current state is a state in which the automatic operation is in progress, a state in which the automatic operation is completed normally, or a state in which the automatic operation is stopped in an emergency. The image of the machine selection part T may change according to a state of the working machine 10 different from the above. The image of the machine selection part T is displayed in a manner that the state of the working machine 10 is distinguishable for the operator. Specifically, for example, the color of the machine selection part T may be different according to the state of the working machine 10.

[0095] (Changes in mechanical selection part T according to selection)

[0096] After the state of the working machine 10 changes, the image of the machine selection part T changes between the case where the machine selection part T corresponding to the working machine 10 whose state has changed is selected (i.e., selected) and the case where the machine selection part T is not selected (i.e., not selected). The machine selection part T displays whether the machine selection part T corresponding to the working machine 10 whose state has changed is selected so that the operator can distinguish between selection and non-selection. Specifically, for example, the unselected machine selection part T can be displayed in a flashing manner (i.e., an image with high brightness and an image with low brightness are displayed alternately). The selected machine selection part T can be displayed in a non-flashing manner. In addition, for example, it is assumed that the state of the working machine 10 corresponding to the unselected second machine selection part T2 has changed (e.g., the automatic operation has been completed normally). At this stage, the second machine selection part T2 displays an image indicating a state change (e.g., the automatic operation has been completed normally) (e.g., a blue image) and an image indicating non-selection (e.g., flashing). When the second mechanical selection unit T2 is selected, the image indicating a state change (e.g., automatic operation has been completed normally) is maintained (e.g., a blue image), and the image display indicating non-selection (e.g., flashing) is changed to an image display indicating selection (e.g., non-flashing).

[0097] (Display image in the machine selection section T according to the task type)

[0098] The image of the machine selection unit T changes according to the type of task to be automatically performed. The image of the machine selection unit T is displayed in a manner that the task type is distinguishable for the operator. Specifically, for example, according to the task type associated with the working machine 10 corresponding to the machine selection unit T, the image of the frame of the machine selection unit T (for example, the color of the frame) can be changed.

[0099] (Switching of Mechanical Selector T)

[0100] A specific example of the conversion of the display image G when the selected machine selection unit T (i.e., the selected working machine 10) is switched will be described below. For example, it is assumed that the first machine selection unit T1 has been selected, and information about the working machine 10 corresponding to the first machine selection unit T1 has been displayed in the main display area M. In this case, the second machine selection unit T2 is selected (this selection will be referred to as α). As a result, information about the working machine 10 corresponding to the second machine selection unit T2 is displayed in the main display area M. At this stage, the content displayed in the main display area M is the same as the content last displayed in the main display area M when the second machine selection unit T2 was selected before the above-mentioned selection (α). However, for example, in the state of automatic screen conversion (for example, after the pause of automatic operation, the state of attempting communication connection and the state of waiting for the establishment of the initial posture), there may be a situation where a machine selection unit T (first machine selection unit T1) different from the second machine selection unit T2 is selected. In this case, when the state of the above-mentioned automatic screen conversion is completed before the selection (α) is made, the content displayed in the main display area M is the screen after the screen conversion. In this case, when the automatic screen transition state is not completed when (α) is selected, the content displayed in the main display area M is the screen in the automatic screen transition state (during screen transition).

[0101] (Image display in other mechanical selection section T, etc.)

[0102] If there is no working machine 10 connected to the information terminal 23, there is no need to display the machine selection part T. If there is a working machine 10 connected to the information terminal 23, the machine selection part T may be displayed or not. If the information terminal 23 is disconnected from the working machine 10, the machine selection part T corresponding to the disconnected working machine 10 may disappear. When the first machine selection part T1 disappears from the display image G, if there is a second machine selection part T2 adjacent to the disappeared first machine selection part T1 (for example, immediately to the right thereof), the second machine selection part T2 may be moved to the position where the disappeared first machine selection part T1 is displayed. If the number of connected working machines 10 increases, a machine selection part T may be added. For example, a machine selection part T may be added at a position adjacent to the already displayed machine selection part T (for example, immediately to the right thereof).

[0103] (Tag U not connected)

[0104] If there is no working machine 10 corresponding to the machine selection part T, an image similar to the machine selection part T may be displayed (ie, the tag U is not connected (see Fig. 14B)). The unconnected label U is displayed so that the operator can distinguish it from the machine selection part T (i.e., the part corresponding to the machine selection part T and in which the working machine 10 exists). For example, the unconnected label U may include the text "unconnected". If there is a working machine 10 connected to the information terminal 23, the unconnected label U may be displayed or may not be displayed. When the unconnected label U is selected, the setting for connecting the working machine 10 may be performed (see Fig.14A ).

[0105] The machine adding part Tadd is a part selected when adding a working machine 10 to be connected to the information terminal 23. For example, the machine adding part Tadd is set at a position continuous with the machine selection part T (for example, immediately to the right thereof). When the machine adding part Tadd is selected, a new machine selection part T can be added, or settings for connecting the working machine 10 can be performed (see Fig.14A ). The machine adding part Tadd may be displayed (or activated) when it is possible to add the working machine 10 to be added to the information terminal 23, and may not be displayed (or deactivated) when it is impossible to add. Specifically, for example, when the number of working machines 10 that can be connected to the information terminal 23 is limited, the machine adding part Tadd may be displayed when the number of working machines 10 that have been connected has not reached the maximum number, and may not be displayed when the number of working machines 10 has reached the maximum number.

[0106] refer to FIG. 3A to FIG. 14D , the following will describe the main reference Figure 2 The parts (elements) of the display image G that have been described are different, and a specific example of the transition of the display image G will be described. Note that the transition of each screen and the conditions for display of each screen can be variously changed.

[0107] Figure 3A The main screen is shown. When the home selection part H is selected, the main screen is displayed. The main screen includes a loading task selection part F3A1 and a lifting magnet task selection part F3A2. Each of the loading task selection part F3A1 and the lifting magnet task selection part F3A2 is displayed in the main display area M and is, for example, an icon. In the main screen, the function selection part B (see Figure 2 ).exist Figure 3A In the example shown in FIG. 1 , the mechanical selection portion T is not shown (see FIG. 1 ). Figure 2 ).

[0108] Figure 3B The main screen is shown when there are a plurality of connected working machines 10. The main screen includes a loading task selection portion F3B1, a lifting magnet task selection portion F3B2, and a machine quantity display portion F3B3.

[0109] The image of the task selection section A (loading task selection section F3B1 and lifting magnet task selection section F3B2) changes according to the number of connected working machines 10 of each task type. For example, the image of the task selection section A is changed so that the operator can distinguish the existence of the connected working machines 10 of each task type (that is, whether such working machines 10 do not exist). Specifically, for example, when the number of connected working machines 10 associated with the lifting magnet task is 0, the brightness of the lifting magnet task selection section F3B2 is low (dark, grayed out) compared to the case where the number of connected working machines 10 is greater than 0. The machine quantity display section F3B3 displays the number of connected working machines 10 of each task type. For example, the machine quantity display section F3B3 is set at a position where the portion overlaps with a portion (e.g., the upper right portion) of the image (e.g., icon) of the loading task selection section F3B1. Similar to the image of the task selection section A (loading task selection section F3B1 and lifting magnet task selection section F3B2) displayed in the main display area M, the task selection section A in the left portion (application selection area) of the display image G may change according to the number of connected working machines 10. When the number of working machines 10 that can be connected to the information terminal 23 is limited, the image of the task selection section A may change according to whether the number of connected working machines 10 has reached the limit (in the same manner as the above-mentioned machine addition section Tadd). The all machine stop selection section F3B4 will be described later.

[0110] The following will mainly describe a case where the information terminal 23 is connected to the working machine 10 that performs a loading operation.

[0111] (Command function)

[0112] Figure 4A The instruction top screen (teaching top screen) is shown. When the instruction function selection part B1 is selected, the instruction top screen is displayed (for example, the screen is initially displayed). The instruction top screen includes a new instruction selection part F4A1, an instruction operation selection part F4A2, a current status display part F4A3, and a machine graphic display part F4A4. When making a new instruction, the operator selects the new instruction selection part F4A1. When the working machine 10 is automatically operated based on the instruction data preset in the automatic operation information processing device 20 (for example, instruction data that has been formed in the past), the instruction operation selection part F4A2 is selected. The current status display part F4A3 displays information indicating the status of the monitoring working machine 10. The machine graphic display part F4A4 includes a graphic representing the working machine 10. The image representing the working machine 10 displayed in the machine graphic display part F4A4 may be a three-dimensional image (see Fig. 9A ) or a two-dimensional image, and the number of work machines 10 shown in the image may be one or more (see Figure 5A). The display state of the machine graphic display portion F4A4 may be changeable (eg, may be changed in response to an operation). For example, the angle (ie, the viewpoint) from which the working machine 10 displayed in the machine graphic display portion F4A4 is viewed may be changed (see Figure 5B For example, the working machine 10 displayed in the machine graphic display unit F4A4 can be enlarged or reduced. The machine graphic display unit F4A4 may include a graphic representation of the working site where the working machine 10 performs a task (see Fig. 9A ). In the same manner as the graphic of the working machine 10, the graphic representation of the operation site may be a three-dimensional image or a two-dimensional image.

[0113] Figure 4B The pre-instruction screen is shown. The pre-instruction screen is a screen displayed before an instruction is made. The pre-instruction screen is a screen displayed when the new instruction selection part F4A1 is selected in the instruction top screen (see Figure 4A ). The pre-instruction screen includes a current status display portion F4B1, an instruction start selection portion F4B2, and a cancel selection portion F4B3. The current status display portion F4B1 displays the status related to the instruction and indicates that the instruction has not yet been made. The instruction start selection portion F4B2 is a portion selected when a new instruction is started. The cancel selection portion F4B3 is a portion selected when a new instruction is canceled (i.e., not started).

[0114] Figure 4C The instruction-period screen displayed when an instruction is made is shown. The instruction-period screen includes a current status display section F4C1, an instruction end selection section F4C2, a cancel selection section F4C3, and a machine graphic display section F4C4. The current status display section F4C1 displays information indicating that an instruction is being made. The instruction end selection section F4C2 is a portion selected when the instruction is completed. When the instruction end selection section F4C2 is selected, the instruction data obtained by the instruction is set (stored) in the automatic operation information processing device 20. The cancel selection section F4C3 is a portion (cancel button) selected when the instruction is canceled (undone). For example, when the operating machine 10 is operated but the data obtained by the operation is not used as the instruction data, the cancel selection section F4C3 is selected. The machine graphic display section F4C4 shows a graphic of the operating machine 10 (in a manner similar to that of the machine 10). Figure 4A The graphic of the working machine 10 displayed in the mechanical graphic display section F4C4 changes according to the movement of the working machine 10 during the instruction (e.g., the rotation and movement of the attachment device 15). The mechanical graphic display section F4C4 may include a graphic (e.g., a curve) representing the trajectory (path) of a specific part of the attachment device 15 during the instruction (see Figure 5A ). In this case, a graph representing the trajectory of a specific portion of the attachment device 15 is superimposed and displayed on a graphic representing the work machine 10.

[0115] (Trajectory confirmation function)

[0116] Figure 5A The trajectory confirmation screen (path confirmation screen, teaching trajectory confirmation screen) is shown. The trajectory confirmation screen is a screen that allows the operator to check the command trajectory (path). The trajectory confirmation screen is displayed after the instruction is made. The trajectory confirmation screen is displayed when the trajectory confirmation function selection part B3 is selected. The trajectory confirmation screen includes a current state display part F5A1, a trajectory display part F5A2, a mechanical graphic display part F5A3, an instruction completion selection part F5A4, a trajectory change selection part F5A5, and a re-instruction selection part F5A6. The current state display part F5A1 displays information that the instruction has been made. The trajectory display part F5A2 displays a graphic representing the instruction trajectory. The trajectory display part F5A2 displays a graphic of the instruction trajectory in a manner overlapping with the mechanical graphic display part F5A3 including a graphic representing the working machine 10. The graphic representing the trajectory in the trajectory display part F5A2 can be a plurality of points or a line. When the instruction is completed, the instruction completion selection part F5A4 is selected. When the instruction completion selection part F5A4 is selected, the instruction trajectory is set as the instruction data. The trajectory change selection unit F5A5 is a portion selected when the operator changes the command trajectory. The re-command selection unit F5A6 is a portion selected when the operator executes the command again.

[0117] Figure 5B The track change screen (path change screen) is shown. The track change screen is a screen displayed when the track is changed. The track change screen is displayed when the track change selection part F5A5 (see Figure 5A ). The track change screen includes a current state display section F5B1, a track display section F5B2, a track change section F5B4, a moving direction selection section F5B6, a track change selection section F5B7, a reset selection section F5B8, and a cancel selection section F5B9. The current state display section F5B1 displays information indicating the state (screen) that has been set for changing the track.

[0118] The trajectory can be changed in various ways. An example of a method of changing the trajectory will be described below. The trajectory display unit F5B2 is Figure 5A The trajectory is displayed in the same manner as the trajectory display unit F5A2 shown in FIG. The trajectory display unit F5B2 displays the working machine 10 as viewed from at least a viewpoint (e.g., a side view) of the machine in the horizontal direction and a viewpoint (e.g., a side view) of the machine from directly above. This arrangement allows the operator to easily grasp the positional relationship between the working machine 10 and the trajectory. In addition, when the operator changes the trajectory, the arrangement allows the operator to easily grasp in which direction the trajectory should be changed. Figure 5BThe trajectory changing unit F5B4 shown in is a part of the trajectory and is the target of the change. For example, the trajectory changing unit F5B4 can be a point that is a part of the trajectory, or an area that is a part of the trajectory. When the trajectory changing unit F5B4 is selected, the trajectory changing unit F5B4 becomes movable (operable) in the display image G. The moving direction selection unit F5B6 is a part for moving the trajectory changing unit F5B4. The moving direction selection unit F5B6 makes it possible to move the trajectory changing unit F5B4 in multiple directions (for example, four directions such as upward, downward, forward and backward). The moving direction selection unit F5B6 may include a graphic (for example, an arrow). The moving direction selection unit F5B6 may be arranged to be selectable (valid) when the trajectory changing unit F5B4 is selected, and may be arranged to be unselectable (invalid) when the trajectory changing unit F5B4 is not selected. The trajectory changing unit F5B4 can be moved in the display image G by an operation different from the operation of the moving direction selection unit F5B6. For example, the trajectory changing portion F5B4 can be moved in the display image G by an operation of clicking and dragging the touch panel or an operation of dragging a mouse cursor.

[0119] When the track change is confirmed (determined), the track change selection part F5B7 is selected. When the track change selection part F5B7 is selected, the display image G returns to the track confirmation screen (see Figure 5A ). When the change of the trajectory is confirmed, the changed trajectory is set in the information terminal 23 (in the automatic operation information processing device 20) as instruction data for the automatic operation of the working machine 10. When restoring the trajectory before the change, select the reset selection part F5B8. For example, when multiple parts (e.g., multiple points) of the trajectory are changed, when the reset selection part F5B8 is selected, all the changed parts can be returned to the position before the change. For example, when the reset selection part F5B8 is selected, only at least one (e.g., one point) of the changed parts (e.g., multiple points) can be returned to the position before the change. Specifically, for example, the changed parts (e.g., multiple points) can be returned to the position before the change in the order of the last changed part returning first, the second to last changed part returning second, and so on. When canceling the trajectory change, select the cancel selection part F5B9. When the cancel selection part F5B9 is selected, the display image G returns to the trajectory confirmation screen (see Figure 5A ).

[0120] (Target range adjustment function)

[0121] Figure 6 1 shows a target range setting screen. The target range setting screen is a screen for setting a target range TA. Figure 6In the example shown in , the target range setting screen is a release range setting screen (earth discharge position adjustment screen), which is a screen for setting the position range (i.e., target release range TA3) for releasing the captured operation target. For example, the release range setting screen is a screen for setting the position (earth discharge position) at which the excavated soil is loaded onto the vehicle. When the release range function selection part B4 (earth discharge position adjustment function selection part) is selected, the release range setting screen is displayed. The "setting" of the release range may be an adjustment (correction) to a pre-set release range, or may be setting a new release range. For example, the release range may be set after an instruction is made. Alternatively, for example, the release range may be set without an instruction. The release range setting screen includes a release range setting method display part F6A1, a setting information display part F6A2, registration selection parts F6A5 and F6A6, a relative position display part F6A7, and a setting completion selection part F6A8.

[0122] The release range setting method display section F6A1 shows a method for setting a release range (planned dumping area). The release range setting method display section F6A1 shows which position of the release range is specified. The release range setting method display section F6A1 shows how the specified positions F6A1a and F6A1b should be specified. (Specific examples will be given later.) The setting information display section F6A2 displays information about the specified positions F6A1a and F6A1b that have been set. The setting information display section F6A2 can display the coordinates of the specified positions F6A1a and F6A1b, or can display the rotation angle of the upper rotating body 13 relative to the lower traveling body 11. The setting information display section F6A2 displays a set of information of the corresponding specified positions F6A1a and F6A1b. The registration selection section F6A5, F6A6 is selected in a predetermined state for registering the specified position. More specifically, for example, a specific part of the attachment device 15 is selected in a state where the part is placed at the specified position F6A1a or F6A1b. For example, when the registration selection part F6A5, F6A6 is selected, the position of the specific part of the attachment device 15 is registered (set) as the position of the designated position F6A1a, F6A1b in the information terminal 23. The registration selection part F6A5, F6A6 is displayed for each designated position F6A1a, F6A1b.

[0123] A specific example of a method for setting a release range (planned soil discharge area) will be described below. In this example, the release range is quadrilateral (e.g., rectangular) in shape when viewed from above. The release range is set in such a manner that designated positions F6A1a and F6A1b at the corresponding two corners of the release range and close to the working machine 10 are specified. Alternatively, the release range may be set in such a manner that designated positions at three or more (e.g., four) points of the release range are specified. In this example, the designated position F6A1a is registered, and then the designated position F6A1b is registered. When the designated position F6A1a is registered, the release range setting method display section F6A1 is displayed to indicate a specific portion of the attachment device 15 ( Figure 6 The operator operates the working machine 10 to place a specific part of the attachment device 15 at the specified position F6A1a. In this state, the registration selection part F6A5 corresponding to the specified position F6A1a is selected. As a result, the position of the specified position F6A1a is registered, the information of the specified position F6A1a is displayed in the setting information display part F6A2, and the registration selection part F6A5 of the specified position F6A1a becomes inactive. Subsequently, the registration selection part F6A6 of the next specified position F6A1b becomes active. Then, the specified position F6A1b is registered in the same manner as the registration of the specified position F6A1a. At this stage, the release range setting method display part F6A1 is displayed to indicate that the bucket blade edge is placed at the specified position F6A1b.

[0124] The relative position display part F6A7 displays a graphic indicating the relative position of the release range (planned dumping area) and the work machine 10. When the setting of the release range is completed, the setting completion selection part F6A8 is selected. For example, the setting completion selection part F6A8 is usually inactive, and becomes active when all the information sets of the designated positions F6A1a, F6A1b are set.

[0125] Similar to the release range, the capture range (target capture range TA1 (see Fig. 10D ), for example, a planned excavation area). The function selection section B may include a portion (a capture position function selection section, not shown) by which a function of setting a capture range is selected.

[0126] (Preparation for automatic operation)

[0127] When the work plan (e.g., instruction data) for automatic operation is confirmed, preparation is performed to change the operation mode of the work machine 10 to the automatic operation mode (a mode in which the work machine 10 operates automatically). For example, the operation mode of the work machine 10 is set by a controller of the work machine 10. For example, the operation mode of the work machine 10 may include a mode in which an operator boards the work machine 10 and operates the work machine 10 (boarding operation mode). The operation mode of the work machine 10 may include a mode in which an operator remotely operates the work machine 10 (remote operation mode).

[0128] The conditions for starting the automatic operation mode (automatic operation mode start conditions) may be variously set. For example, the automatic operation mode start conditions include at least one of the following conditions: the work machine 10 changes to the initial posture; the engine is turned off; the key is turned off; if the machine is remotely controlled, the connection for remote control is cut off; and the emergency stop switch is turned on. The case where the automatic operation mode start conditions include all of these conditions will be described below.

[0129] The automatic operation mode start condition includes a condition that the work machine 10 changes to the initial posture. The work machine 10 may change to the initial posture due to the operation of an operator (who is on the machine or performs remote control), or may automatically change to the initial posture. The following will describe a case where the work machine 10 is manually changed to the initial posture by the operator's operation.

[0130] Fig. 7AAn initial posture preparation screen is shown. The initial posture preparation screen is a screen displayed when the working machine 10 changes to the initial posture. For example, after completing the work plan (e.g., instruction data), the initial posture preparation screen is displayed. The initial posture preparation screen includes a notification section F7A1, an initial posture display section F7A2, an initial posture preparation completion notification section F7A3, and an initial posture confirmation selection section F7A4. The notification section F7A1 is a portion that notifies the operator that the working machine 10 should be changed to the initial posture. The notification section F7A1 notifies the operator that the initial posture is the posture displayed in the initial posture display section F7A2. The initial posture display section F7A2 is a portion that notifies the operator what posture is the initial posture. For example, the initial posture display section F7A2 notifies the initial posture through a graphic. In the illustrated example, in the initial posture, the arm 15a forms an angle of approximately 45 degrees with the ground, the arm 15b extends substantially vertically from the arm 15a, and the opening portion of the front end attachment 15c (bucket) extends substantially horizontally. In addition, in the initial posture, the front end attachment device 15c is set directly above the range (planned excavation area) for capturing the operation target. The initial posture preparation completion notification part F7A3 displays whether the working machine 10 has changed to the initial posture in a manner that is distinguishable to the operator. For example, the initial posture preparation completion notification part F7A3 may include an icon, and the color of the icon may change depending on whether the working machine 10 has changed to the initial posture. The initial posture preparation completion notification part F7A3 may include characters such as "OK" or "NG". The initial posture confirmation selection part F7A4 is a part (OK button) that is selected when the working machine 10 takes the initial posture. The initial posture confirmation selection part F7A4 may be in an inactive state when the working machine 10 has not taken the initial posture, and may become active when the working machine 10 takes the initial posture.

[0131] Figure 7B The work machine state preparation screen is shown. When the initial posture confirmation selection part F7A4 (see Fig. 7A), the working machine state preparation screen is displayed. The working machine state preparation screen is displayed, for example, after the work plan (instruction data) is completed. The working machine state preparation screen includes a condition notification unit F7B1 and a condition confirmation selection unit F7B2. The condition notification unit F7B1 notifies (displays) the automatic operation mode start-up conditions other than the conditions for the working machine 10 to change to the initial posture. The condition notification unit F7B1 notifies to turn off the engine (i.e., switch the engine to a non-operating state). The condition notification unit F7B1 notifies the operator that he needs to be far enough away from the working machine 10. The condition notification unit F7B1 notifies to turn off the key (i.e., turn off the power). When the working machine 10 is in a remotely controllable state (for example, when an instruction is made by remote operation), the condition notification unit F7B1 notifies to disconnect the remote control device from the working machine 10. The condition notification unit F7B1 notifies to turn on the emergency stop switch (not shown). The emergency stop switch is a switch held by an operator around the working machine 10, and is used to stop the working machine 10 in an emergency. Condition confirmation selection part F7B2 is a part (OK button) selected when the condition displayed in condition notification part F7B1 is satisfied. Condition confirmation selection part F7B2 may be inactive when the automatic operation mode start condition is not satisfied, and may be active when the automatic operation mode start condition is satisfied.

[0132] Figure 7C The automatic operation mode start notification screen is shown. The automatic operation mode start notification screen is displayed when the automatic operation mode is started. When the condition notification part F7B1 (see Figure 7B ), the automatic operation mode startup notification screen is displayed. The automatic operation mode startup notification screen includes a startup notification portion F7C1 and a startup confirmation selection portion F7C2. The startup notification portion F7C1 is a portion for notifying the startup of the automatic operation mode. The startup confirmation selection portion F7C2 is a portion (OK button) selected when confirming the startup of the automatic operation mode.

[0133] As with the activation notification portion F7C1, a notification portion that requires a confirmation operation (e.g., pressing an OK button) by the operator to continue the process may be displayed in the display image G at various times other than the activation of the automatic operation mode. For example, such a notification portion may be displayed at the start or end of an instruction, at the start or end of a trajectory change, and at the start or end of each process described later. Specific examples of the confirmation operation performed with respect to the notification portion may be an operation of pressing an OK button (an operation of selecting a selection portion) and an operation of pressing an OK button after checking a check box.

[0134] (Initial Settings)

[0135] Figure 8The automatic operation initial setting screen is shown. The automatic operation initial setting screen is a screen for performing initial settings related to automatic operation. For example, after the operation mode of the working machine 10 is switched to the automatic operation mode, the automatic operation initial setting screen is displayed. (For example, this screen is the first screen displayed after the switch.) For example, when the loading function selection part B2 is selected, the automatic operation initial setting screen is displayed. (For example, this screen is the first screen displayed after the selection.) The automatic operation initial setting screen includes an operation setting part F8A1 and an initial setting completion selection part F8A2. The operation setting part F8A1 is a part for allowing the operator to set the automatic operation. The operation setting part F8A1 includes an operation count setting part C1, a target operation amount setting part C2, and an operation speed setting part SS. The initial setting completion selection part F8A2 is a part (OK button) selected when the initial setting of the automatic operation is completed.

[0136] The operation count setting unit C1 allows the operator to set the number of times the automatic operation is performed (operation count, how many cycles the automatic operation is performed). As described above, similar to the selection, the setting is performed by the operator through the operation input unit 23i. The setting can be performed in the same manner as the specific example of the selection, for example, by inputting a number. The operation count setting unit C1 has a portion (a setting value display portion) that displays the operation count. The operation count setting unit C1 may have a portion (an input box) in which the operation count can be numerically input. A single portion can be used as both an input box and a setting value display portion. (This also applies to the target operation amount setting unit C2.) The operation count setting unit C1 includes a selection portion (a + button) for increasing the operation count and a selection portion (a - button) for reducing the operation count.

[0137] The target operation amount setting part C2 is a part that allows the operator to set the amount of the operation target to be released in the operation of moving the working machine 10 and releasing the captured operation target (i.e., the target operation amount). For example, the target operation amount can be the mass or volume of the operation target. The target operation amount is, for example, the amount of soil loaded onto the vehicle in a loading task. The target operation amount setting part C2 has a part (set value display part) that displays the target operation amount. The target operation amount setting part C2 may include a part (input box) in which a numerical value of the target operation amount can be input. The target operation amount setting part C2 includes a selection part (+ button) for increasing the target operation amount and a selection part (- button) for reducing the target operation amount.

[0138] The operating speed setting section SS is a section where the operator sets the operating speed in the automatic operation. The operating speed setting section SS displays a plurality of operating speed levels so that one of them can be selected as the operating speed in the automatic operation. These operating speed levels include the operating speed of the working machine 10 in the instruction data (i.e., the operating speed in the teaching). The plurality of levels can be as follows: Figure 8Three levels (high speed, medium speed, and low speed) in the example shown in , two levels, or four or more levels.

[0139] Among the various levels of operation speeds selectable in the operation speed setting section SS, the fastest operation speed (third speed setting section SS3) is preferably the operation speed of the working machine 10 in the instruction data. In this case, the operation speed in the automatic operation is never faster than the instruction operation speed. Therefore, the operation speed in the automatic operation is equal to or lower than the speed expected by the operator who executes the instruction, and is equal to or lower than the speed assumed by the operator who sees the instruction.

[0140] Among the various stages of operating speeds selectable in the operating speed setting section SS, the fastest operating speed (third speed setting section SS3) may be higher than the operating speed of the working machine 10 in the instruction data. This makes it possible to improve the working efficiency of the automatic operation of the working machine 10. For example, an arrangement may be achieved in which the working machine 10 is reliably operated at a slow operating speed in the instruction, and the working machine 10 is efficiently operated in the automatic operation at an operating speed faster than the operating speed in the instruction.

[0141] By using the operation speed setting section SS, the operation speed in the automatic operation can be set in the following manner. The operation speed setting section SS may include a plurality of selection sections to allow the operator to select the operation speed in the automatic operation from a plurality of operation speeds. Figure 8 In the example shown in , the operating speed setting section SS includes a first speed setting section SS1 (low speed), a second speed setting section SS2 (medium speed) and a third speed setting section SS3 (high speed). The operating speed setting section SS allows the operator to set the operating speed in automatic operation by inputting a number (not shown; this part is similar to the operation count setting section C1 and the target operation amount setting section C2). The operating speed setting section SS allows the operator to set the ratio (percentage) of the operating speed in automatic operation to the operating speed in the instruction data. The operating speed setting section SS allows the operator to set the ratio (percentage) of the operating speed in automatic operation to the fastest operating speed in the multi-stage operating speed (i.e., an operating speed equal to or higher than the operating speed in the instruction data).

[0142] The operation speed setting part SS may allow the operator to set the operation speed in each work phase P. More specifically, when there are a plurality of work phases P in one cycle of automatic operation, the operation speed setting part SS is displayed for each work phase P. Figure 8In the example shown in , one cycle of automatic operation consists of four work phases P. Specifically, for example, the work phase P includes a capture phase P1 (excavation phase), a lifting rotation phase P2 (earth discharge rotation phase), a release phase P3 (earth discharge phase), and a return rotation phase P4. For each of the four work phases P, a first speed setting portion SS1, a second speed setting portion SS2, and a third speed setting portion SS3 are displayed.

[0143] The state in which all operation speed levels in each operation stage P are displayed in the display image G (i.e., Figure 8 The state shown in ) will be regarded as the first speed display state SP1.

[0144] After the operating speed is set by the operating speed setting part SS, the control unit 23c instructs the working machine 10 to perform automatic operation at the operating speed set by the operating speed setting part SS. The control unit 23c instructs the working machine 10 to perform automatic operation at the operating speed set for each working phase P by the operating speed setting part SS.

[0145] (Settings before automatic operation starts)

[0146] Fig. 9A The following figure shows the screen before the automatic operation starts. Figure 8 ) is completed, the screen before the automatic operation starts is displayed. When the initial setting completion selection part F8A2 (see Figure 8 ), the automatic operation start screen is displayed. The automatic operation start screen can be displayed when the start of automatic operation is canceled (that is, when the automatic operation is canceled). Fig.9D The cancel selection shown in F9D3) is displayed, or you can change the setting (see Fig. 9B and Fig. 9C The screen before the automatic operation starts includes a current state display section F9A1, an operation count setting display section F9A2, a target operation amount display section F9A3, and an operation speed display section F9A4.

[0147] The current state display section F9A1 indicates that preparation for starting the automatic operation is in progress. The operation count setting display section F9A2 is a section that displays the set operation count. The set operation count is the operation count set in the initial setting (see Figure 8 ). However, if the operation count set in the initial settings is changed (see Fig. 9C ), the set operation count is the changed operation count. The target operation amount display section F9A3 is a section that displays the set target operation amount (target soil amount). The set target operation amount is the operation count set in the initial setting (see Figure 8). However, if the target operation amount set in the initial setting is changed, the target operation amount that has been set will be the changed target operation amount.

[0148] The operation speed display section F9A4 shows the set operation speed. The set operation speed is the operation speed in automatic operation, which is set in the initial setting (see Figure 8 ). However, if the operation speed in the automatic operation set in the initial setting is changed (see Fig. 9B ), the set operation speed is the changed operation speed. The operation speed display section F9A4 displays the operation speed in each work phase P. Hereinafter, the operation speed in the automatic operation is simply referred to as the operation speed unless otherwise specified.

[0149] Fig. 9B An operation speed change screen is shown. This operation speed change screen is displayed when changing the set operation speed. This change of the operation speed can be performed for each work stage P, or can be performed for the entire cycle (that is, for multiple work stages P together). The case of changing the operation speed for each work stage P will be described below. The operation speed change screen includes an operation speed setting section F9BSS and an operation speed display section F9B2. The operation speed setting section F9BSS displays a multi-level (candidate) operation speed for the selected work stage P, so that the operation speed in the automatic operation can be set (changed). On the other hand, for the unselected work stage P, the set operation speed (in a manner consistent with the operation speed) is displayed in the operation speed display section F9B2. Fig. 9A The same manner as in the operation speed display section F9A4 shown in FIG.

[0150] A specific example of the manipulation and image of the operation speed setting section F9BSS will be described below. Fig. 9A In the operation speed display section F9A4 shown in FIG. 1 , the operation phase P (for example, the capture phase P1) is selected as the change target. In doing so, the following may be displayed: Fig. 9B , an image of pulling out (e.g., pulling out to the right) the operation speed setting section F9BSS of the selected operation stage P (capturing stage P1) from the operation speed display section F9A4 of the selected operation stage P. For example, when one of three levels of operation speeds can be set (can be selected), a first speed setting section F9BSS1 (low speed), a second speed setting section F9BSS2 (medium speed), and a third speed setting section F9BSS3 (high speed) are displayed for the selected operation stage P (capturing stage P1). When the operation speed is set by the operation speed setting section F9BSS, the operation speed setting section F9BSS is no longer displayed (i.e., is turned off), and is displayed (again) Fig. 9A The operation speed display section F9A4 shown in FIG.

[0151] like Fig. 9B As shown in FIG. 1 , when the operation speed setting section F9BSS of the selected work stage P (for example, the capture stage P1) is displayed (i.e., turned on), there is a case where the operation speed display section F9B2 of the unselected work stage P is selected. In this case, for the selected work stage P (capture stage P1), the operation speed setting section F9BSS is no longer displayed (turned off), and the operation speed display section F9A4 is displayed (again) (see FIG. 1 ). Fig. 9A ). On the other hand, for the newly selected work phase P, the displayed operation speed display portion F9B2 is switched to the operation speed setting portion F9BSS (ie, the operation speed setting portion F9BSS is turned on).

[0152] like Fig. 9A and Fig. 9B As shown in FIG. 1 , only the operation speed setting section SS (see FIG. 1 ) in at least one of the operation stages P is shown. Figure 8 and Fig. 9B ) is regarded as the second speed display state SP2. More specifically, in Fig. 9A In the example shown in FIG. 1 , the operation speed display section F9A4 displays only the operation speed of the level set by the operation speed setting section SS in all the work phases P. Fig. 9B In the example shown in , the operation speed display section F9B2 displays only the operation speed of the level set by the operation speed setting section SS in the operation stage P (which is not the capture stage P1) among the operation stages P. Note that after the second speed display state SP2 is established, the display image G may be converted to the first speed display state SP1 (that is, may be returned to the initial setting).

[0153] Fig. 9C The operation count change screen is shown. When changing the set operation count, the operation count change screen is displayed. The operation count change screen includes an operation count change section F9C1 and a target operation amount display section F9C2. Fig. 9A When the operation count setting display section F9A2 shown in FIG. 8 is selected, the operation count change section F9C1 is displayed. For example, when the operation count setting display section F9A2 is selected, a display section pulled out from the operation count setting display section F9A2 (for example, pulled down) is displayed. Fig. 9C. The operation count change section F9C1 is displayed to allow the operator to change the operation count. For example, the operation count change section F9C1 may include a selection section (+ button) for increasing the operation count and a selection section (- button) for decreasing the operation count, and may include a portion capable of digitally inputting the operation count after the change. The operation count change section F9C1 includes a determination selection section (OK button) by which the change in the operation count is determined. When the determination selection section is selected, the operation count change section F9C1 is no longer displayed (i.e., turned off), and the operation count setting display section F9A2 (see Fig. 9A ). Similarly, when another part of the display image G (for example, the target operation amount display part F9C2) is selected while the operation count change part F9C1 is displayed, the operation count change part F9CT1 is no longer displayed, and the operation count setting display part F9A2 is displayed (again) (see Fig. 9A ).

[0154] Similar to the change of the operation count by the operation count change portion F9C1, the target operation amount can be changed. For example, when the target operation amount display portion F9C2 is selected, a portion for changing the target operation amount (target operation amount change portion) can be displayed.

[0155] The operation speed, operation count and target operation amount can be changed only when the work machine 10 is not performing the automatic operation (e.g., the automatic operation is in preparation). The operation speed, operation count and target operation amount can be changed when the work machine 10 is performing the automatic operation (e.g., simultaneously with the automatic operation).

[0156] Fig. 9C The screen shown in FIG. 1 includes an initial posture preparation completion notification unit F9C3, a key state selection unit F9C4, an engine state selection unit F9C5, a lever lock state selection unit F9C6, and an operation start selection unit F9C7. The initial posture preparation completion notification unit F9C3 displays whether the working machine 10 is in the initial posture (in a manner that is distinguishable to the operator) Fig. 7AThe initial posture preparation completion notification unit F7A3 shown in FIG. 1 is the same as the initial posture preparation completion notification unit F7A3 shown in FIG. 1 ). When switching to the automatic operation mode, the working machine 10 is assumed to be in the initial posture. The operator is allowed to check whether the working machine 10 is maintained in the initial posture through the initial posture preparation completion notification unit F9C3. The key state selection unit F9C4 is a portion selected when the state of the key is switched between on and off. The key state selection unit F9C4 is also a portion that displays the state of the key (i.e., a key state display unit). When the state of the engine is switched between on (RUN) (operating state) and off (stop state), the engine state selection unit F9C5 is selected. The engine state selection unit F9C5 is also a portion that displays the state of the engine (i.e., an engine state display unit). When the state of the lever lock is switched between a locked state (a state in which the working machine 10 cannot be operated) and a released state (a state in which the working machine 10 can be operated), the lever lock state selection unit F9C6 is selected. The lever lock state selection unit F9C6 is also a portion that displays the state of the lever lock (i.e., a lever lock state display unit).

[0157] The conditions for starting the automatic operation (automatic operation start condition) can be set in various ways. For example, the automatic operation start condition includes at least one of the following conditions: the operation mode of the work machine 10 is the automatic operation mode; the work machine 10 is in the initial posture; the key is turned on; the engine is turned on; and the lever lock is in the released state. Fig. 9C In the example shown in , the key is turned off, the engine is turned off, and the lever lock is in a locked state, and therefore the automatic operation start condition is not satisfied. Therefore, the information terminal 23 is operated by the operator to satisfy the automatic operation start condition. More specifically, for example, the key state selection part F9C4 is selected, and the key is switched from off to on. The engine state selection part F9C5 is selected, and the engine is switched from off to on (RUN). The lever lock state selection part F9C6 is selected, and the lever lock is changed from a locked state to a released state. When at least one of the key state selection part F9C4, the engine state selection part F9C5, or the lever lock state selection part F9C6 is operated, a confirmation screen (similar to Figure 7C For example, when the engine is switched from off to on, the state change of the working machine 10, such as turning on the engine, can be notified. In addition, it can also be notified that it is necessary to confirm that there is no one around the working machine 10.

[0158] The relationship between the key state and the engine state can be set in a variety of ways. For example, the condition for the engine to be turned on may include the condition for the key to be turned on. When the key is turned on, the engine can be automatically turned on. When the engine is switched from on to off, the key can be turned off, or it can be kept on.

[0159] The operation start selection part F9C7 is a part selected when the automatic operation is started. For example, the operation start selection part F9C7 may be in an active state when the automatic operation start condition is satisfied, and may be in an inactive state when the automatic operation start condition is not satisfied.

[0160] Fig.9D The automatic operation start confirmation screen is shown. The automatic operation start confirmation screen is displayed before the automatic operation is started (for example, immediately before the start). Fig. 9C ), the automatic operation start confirmation screen is displayed. The automatic operation start confirmation screen includes an automatic operation start notification section F9D1, a start selection section F9D2, and a cancel selection section F9D3. The automatic operation start notification section F9D1 is a section that notifies (displays) that the automatic operation is started. The start selection section F9D2 is a section that is selected when the automatic operation is started (OK button). The cancel selection section F9D3 is a section that is selected when the automatic operation is not started (Cancel button).

[0161] (During automatic operation)

[0162] Fig. 10A The screen during automatic operation (first screen, detailed screen during automatic operation) is shown. The screen during automatic operation is displayed during automatic operation. The screen during automatic operation can be displayed when the working machine 10 is waiting (on standby). The screen during automatic operation can be displayed when the working machine 10 stops in an emergency (in an emergency stop state). The screen during automatic operation includes a current state display section F10A1, a remaining operation count display section F10A2, a target operation amount display section F10A3a, and a current operation amount display section F10A3b. The screen during automatic operation includes a machine information display section F10A4, a viewpoint change selection section F10A5, a simple screen selection section F10A6, an alarm operation section F10A7, and a pause selection section F10A8. The current state display section F10A1 displays information indicating that the working machine 10 is operating automatically. The remaining operation count display section F10A2 is a section that displays the remaining number of automatically executed operations. The remaining number of executed operations is calculated by subtracting the completed operation count from the set operation count (number of cycles). The remaining time of the task can be displayed in the display unit F11A2 (see Fig.11 ). The target operation amount display unit F10A3a and Fig. 9AThe current operation amount display section F10A3b is similar to the target operation amount display section F9A3 shown in . The current operation amount display section F10A3b is a section that displays information about the amount of the operation target that has been processed (processed operation amount) (for example, the amount of soil that has been loaded). The current operation amount display section F10A3b can display the processed operation amount or content corresponding to the processed operation amount. The current operation amount display section F10A3b can display the ratio (percentage) of the processed operation amount to the target operation amount.

[0163] The mechanical information display unit F10A4 is a portion that displays information about the working machine 10. For example, the mechanical information display unit F10A4 displays information detected by a sensor mounted on the working machine 10 or by a sensor provided outside the working machine 10. For example, the mechanical information display unit F10A4 may display information about the angle of operation (e.g., the rotation angle or the angle relative to the horizontal direction). The mechanical information display unit F10A4 may display information about the amount of the operating target, for example, the amount of the operating target captured (excavated) by the working machine 10 in one cycle (e.g., the amount of excavated soil). The viewpoint change selection unit F10A5 is a portion that is selected when changing the viewpoint of, for example, a graphic indicating the working machine 10 displayed in the main display area M. Each time the viewpoint change selection unit F10A5 is selected, the viewpoint of the working machine 10 displayed in the main display area M may be switched. The viewpoint change selection unit F10A5 may be displayed for each viewpoint of the working machine 10 displayed in the main display area M. (For example, multiple buttons may be provided.) The viewpoints of the working machine 10 displayed in the main display area M may include a viewpoint for observing the working machine 10 from obliquely above (oblique viewpoint), a viewpoint for observing the machine in a horizontal direction (see FIG. 5 ), and a viewpoint for observing the machine from above (see FIG. 5 ).

[0164] The simple screen selection section F10A6 is a portion selected when the display image G is switched to the simple screen (second screen) during automatic operation (see Fig.11 The alarm operation part F10A7 is a part selected when causing the working machine 10 to sound an alarm. The pause selection part F10A8 is a part selected when suspending the automatic operation (details will be given below).

[0165] Fig. 10B For example, when the menu selection part F2A1 (see Fig. 10A) and a predetermined menu item not shown in the figure is selected, the display setting screen is shown. The display setting screen includes a display setting section F10B1. The display setting section F10B1 is a part that performs settings related to the elements (contents) displayed in the main display area M. The display setting section F10B1 is a part that sets the display state (for example, presence or absence) of the elements in the main display area M. For example, the display setting section F10B1 includes a check box or a switch provided for each element to determine whether to display the element. The elements whose display state in the main display area M can be set by the display setting section F10B1 include, for example, instruction tracks ( Fig. 10D The target route PA shown in FIG. 1 ), the actual operation trajectory PB (see FIG. Fig. 10D ), a transport vehicle (dump truck) detection area, or an image captured by an imaging device. The actual operation trajectory PB is the trajectory of a specific part of the attachment device 15 when the work machine 10 actually performs the task automatically. The transport vehicle (dump truck) detection area is the position range of the transport vehicle (dump truck) that detects the target of the loading operation. The above-mentioned image captured by the imaging device is an image captured by an imaging device set at the work machine 10 or outside the work machine 10 (for example, at the work site). The image captured by the imaging device may be an image captured by a two-dimensional camera, or a three-dimensional image (for example, a point cloud image) captured by a device configured to obtain three-dimensional information (for example, LiDAR or a stereo camera). Fig. 10B In the example shown in , the elements (contents) displayed in the main display area M are a three-dimensional image and a point cloud image representing the working machine 10 .

[0166] Fig. 10C The sensor status confirmation screen is shown. For example, when the menu selection part F2A1 (see Fig. 10A ) and a predetermined menu item not shown in the figure is selected, a sensor status confirmation screen is shown. The sensor status confirmation screen includes a sensor status display unit F10C1. The sensor status display unit F10C1 displays the status of a sensor provided at the working machine 10 or outside the working machine 10. For example, the sensor status display unit F10C1 may display whether the sensor is normal, or may display the content obtained by the sensor. The sensor whose status is displayed by the sensor status display unit F10C1 may be an angle sensor of each element (arm 15a, arm 15b, and front end attachment 15c) of the attachment device 15. The sensor whose status is displayed by the sensor status display unit F10C1 may be an imaging device (e.g., LiDAR or a camera), an automatic operation controller of the working machine 10, or a controller different from the automatic operation controller of the working machine 10.

[0167] Fig. 10D 1 shows a screen during automatic operation, which includes, for example, a work stage display section PD. Fig. 10B In the display setting section F10B1 shown in FIG. 10B, the target route PA is selected to be displayed. Fig. 10B When the option of the command trajectory in the operation is selected, the automatic operation screen including the operation phase display part PD is displayed. The automatic operation screen including the operation phase display part PD includes the target route PA, the actual operation trajectory PB, the target range TA, the operation phase display part PD, the operation phase text display part F10D1 and the explanatory comment F10D2.

[0168] As described above, the target route PA is a target route of a specific part of the attachment device 15 (for example, the front end part of the front end attachment device 15c). The target route PA is displayed in the main display area M. The target route PA is displayed as a line. For example, multiple types of target routes PA are set to correspond to the operation stages P accordingly. Specifically, for example, the target route PA includes a capture target route PA1, a lifting and turning target route PA2, a release target route PA3, and a return and turn target route PA4. The capture target route PA1 is the target route PA in the capture stage P1. The lifting and turning target route PA2 is the target route PA in the lifting and turning stage P2. The release target route PA3 is the target route PA in the release stage P3. The return and turn target route PA4 is the target route PA in the return and turn stage P4.

[0169] The display state of each type of target route PA (capture target route PA1, lift rotation target route PA2, release target route PA3, and return rotation target route PA4) may be different between each type. For example, the color (e.g., at least one of hue, concentration, brightness saturation) of the target route PA may be different between each type of target route PA. For example, the type of line representing the target route PA (thickness, solid line, dotted line, etc.) may be different between each type of target route PA.

[0170] The image (specifically, line) showing the target route PA may indicate the orientation of the target route PA. The orientation of the target route PA is the orientation of movement of a specific part of the attachment device 15 when the specific part moves along the target route PA. For example, the image showing the target route PA may include an arrow, a moving image moving along the orientation of the target route PA, etc.

[0171] The actual operation trajectory PB is the trajectory of the attachment device 15 when the working machine 10 actually performs the task automatically. The actual operation trajectory PB is displayed in the main display area M. The actual operation trajectory PB is displayed at the position (trajectory) passed by the attachment device 15 of the working machine 10 displayed in the main display area M. For example, the actual operation trajectory PB may be a trajectory (e.g., a line) of a specific portion of the attachment device 15 (e.g., the front end portion of the front end attachment device 15c), or a trajectory of the entire front end attachment device 15c.

[0172] The actual operation trajectory PB is displayed in an overlapping manner with the target route PA. This overlapping display allows the operator to confirm whether a specific part of the attachment device 15 is performing an operation along the target route PA (ie, an operation corresponding to the target of the automatic operation).

[0173] The actual operation trajectory PB is displayed to be distinguished from the target route PA. The display state of the actual operation trajectory PB is different from the display state of the target route PA. For example, the color (e.g., hue) of the displayed actual operation trajectory PB is different from the color (e.g., hue) of the displayed target route PA. The color of the actual operation trajectory PB may be monochrome or multiple colors (e.g., color gradient). The display state of the actual operation trajectory PB may be different according to each type of operation stage P performed by the working machine 10 (i.e., the currently performed operation stage P).

[0174] The actual operation trajectory PB can be displayed with animation. For example, when a predetermined condition (trajectory display condition) is no longer satisfied, a portion of the actual operation trajectory PB can be deleted from the display image G. Since this portion of the actual operation trajectory PB is deleted, it is possible to suppress the problem of insufficient visibility of the display image G due to excessive image display of the actual operation trajectory PB. Specifically, for example, the actual operation trajectory PB can be arranged so that when a predetermined time (trajectory display time) passes from a specific part of the attachment device 15 to a specific position, a portion of the actual operation trajectory PB at the specific position is deleted. The trajectory display time may be a fixed time period set in advance, or may be optionally set by the operator.

[0175] The target range TA is a target range of an operation automatically performed by the working machine 10. The target range TA is displayed in the main display area M. A plurality of sets of target ranges TA are set. Specifically, for example, the target range TA includes a target capture range TA1 and a target release range TA3. The target capture range TA1 is a range in which the working machine 10 (front end attachment 15c) captures the operation target (e.g., a planned excavation area) in the capture phase P1. The target release range TA3 is a range in which the working machine 10 (front end attachment 15c) releases the operation target (e.g., a planned earth removal area) in the release phase P3.

[0176] The display state of each type of target range TA (target capture range TA1 and target release range TA3) may be different between each type. For example, the color (e.g., hue) of the target range TA may be different between each type of the target range TA. The display state of each type of target distance TA may correspond to the display state of the target route PA of the type corresponding to the target distance TA of the type. Specifically, for example, the display state (e.g., hue) of the target capture range TA1 may be the same (or substantially the same) as the display state (e.g., hue) of the capture target route PA1. In addition, for example, the display state of the target release range TA3 may be the same (or substantially the same) as the display state of the release target route PA3. More specifically, it is assumed that the color of the capture target route PA1 is orange, and the color of the release target route PA3 is green. In this case, the target capture range TA1 may be orange in color, and the target release range TA3 may be green in color. The display state of each type of target distance TA may not correspond to the display state of the target route PA of the type corresponding to the target distance TA of the type.

[0177] The work phase display section PD displays information indicating to which type of work phase P among a plurality of types of work phases P the task being (automatically) performed by the work machine 10 belongs. Hereinafter, the work phase P displayed by the work phase display section PD and corresponding to the task performed by the work machine 10 may be referred to as the currently performed work phase P. The work phase display section PD displays information allowing the operator to easily grasp the currently performed work phase P. The work phase display section PD displays the currently performed work phase P and other types of work phases P in a manner distinguishable for the operator.

[0178] The operation stage display section PD may include an image of the target route PA. More specifically, the display state of the target route PA may be different between the type of target route PA (e.g., capture target route PA1) corresponding to the currently executed operation stage P (e.g., capture stage P1) and other types of target routes OA. The display state that is changed in this case may be the color of the target route PA (e.g., at least one of hue, concentration, brightness, and saturation), or the type of line indicating the target route PA. Specifically, for example, the brightness of the type of target route PA corresponding to the currently executed operation stage P may be high, and the brightness of other types of target routes PA may be low (grayed out).

[0179] The work phase display part PD may include an image of the actual operation trajectory PB. More specifically, for example, the display state (e.g., color tone) of the actual operation trajectory PB may be different between the types of the currently executed work phase P. Each time the type of the currently executed work phase P is changed, the display state of the actual operation trajectory PB may be changed.

[0180] The operation stage display part PD may include an image of the target range PA. More specifically, the display state of the target range TA may be different between the type of target range TA corresponding to the currently executed operation stage P (e.g., target capture range TA1) and the other type of target range TA (e.g., target release range TA3). The display state to be distinguished may be the color (e.g., hue) within the target range TA, the color of the outer frame of the target range TA, or the type of line indicating the outer frame of the target range TA. Specifically, for example, the brightness of the type of target range TA corresponding to the currently executed operation stage P may be high, and the brightness of the other type of target route PA may be low (grayed out).

[0181] The work phase display portion PD may include an image of the work machine 10. More specifically, for example, the display state of a part or all of the work machine 10 may be different between the types of the work phase P currently being executed. Each time the type of the work phase P currently being executed is changed, the display state of a part or all of the work machine 10 may be changed. When the display state of only a part of the work machine 10 is different between the types of the work phase P currently being executed, the part whose display state is distinguished may be only the attachment device 15 or only the front end attachment device 15c.

[0182] For example, as described above, the color (e.g., hue) of the target route PA may be differentiated between the types of target routes PA. In this case, the display color of a portion or all of the working machine 10 may be the same or substantially the same as the color of the target route PA of the type corresponding to the currently executed working stage P. More specifically, for example, assume that the color of the capture target route PA1 is orange, the color of the lifting and turning target route PA2 is blue, and the color of the release target route PA3 is green. In this case, when the currently executed working stage P is the capture stage P1, a portion or all of the working machine 10 is displayed in orange. When the currently executed working stage P is the lifting and turning stage P2, a portion or all of the working machine 10 is displayed in blue. When the currently executed working stage P is the release stage P3, a portion or all of the working machine 10 is displayed in green.

[0183] The work phase display portion PD may include a work phase text display portion F10D1. The work phase text display portion F10D1 displays text indicating the currently executed work phase P. The display state (e.g., color) of the work phase text display portion F10D1 may be arranged to be the same or substantially the same as the display state (e.g., color) of the target range TA corresponding to the type of the currently executed work phase P.

[0184] The work phase display portion PD may include the entire display image G, or may include the entire main display area M. More specifically, the display state (e.g., color) of the entire display image G or the display state (e.g., color) of the entire main display area M may be different between the types of the currently executed work phase P. The display state (e.g., color) of the entire display image G or the entire main display area M may be arranged to be the same or substantially the same as the display state (e.g., color) of the target route PA (or target range TA) corresponding to the type of the currently executed work phase P.

[0185] The work phase display section PD may display information indicating the type of the work phase P executed after the currently executed work phase P in the same manner as the information indicating the type of the currently executed work phase P. In this case, the work phase display section PD may display the currently executed work phase P and the subsequent work phase P in such a manner that they are distinguishable in type.

[0186] The explanatory note F10D2 indicates what type of work phase P the image (for example, of the target route PA) shown in various states according to the type of the work phase P corresponds to. Fig. 10D In the example shown in , the explanatory annotation F10D1 indicates the relationship between the display status of multiple types of target routes PA and the type of the work stage P. For example, the explanatory annotation F10D2 may indicate the relationship between the color tone of the target route PA and the type of the work stage P (i.e., indicating which color corresponds to which work stage P). The explanatory annotation F10D1 may indicate the relationship between the display status (e.g., color) of an item different from the target route PA (e.g., a target range TA or an actual operation trajectory PB) and the type of the work stage P.

[0187] (Simple screen display during automatic operation)

[0188] Fig.11 The simple screen during automatic operation (second screen) is shown. The simple screen during automatic operation is displayed during automatic operation. The simple screen during automatic operation can be displayed when the working machine 10 is waiting (standby). The simple screen during automatic operation can be displayed when the working machine 10 stops in an emergency (is in an emergency stop state). The display image G can be displayed in a switchable manner. Fig. 10A The automatic operation period screen shown in FIG. 1 (the automatic operation period detailed screen, which may be simply referred to as the detailed screen) and Fig.11 For example, when in the detailed screen (see Fig. 10A ) Select the simple screen selection part F10A6 (see Fig. 10A), the simple screen during automatic operation is displayed. The simple screen during automatic operation can be displayed when the simple screen selection part F10A6 is selected only once (i.e., displayed by one action). When a confirmation operation (e.g., pressing the OK button) is performed after selecting the simple screen selection part F10A6, the simple screen during automatic operation can be displayed.

[0189] For example, there may be a situation where the operator cannot observe the display image G while the operator is performing another task. The "other task" is a task different from the task performed by the working machine 10 whose information is displayed in the display image G. A specific example of the other task is a task performed by using a machine that is not connected to the information terminal 23 (for example, a forklift). The other task may be manual or remote operation of the machine. There is a situation where the operator wants to grasp the status of the task automatically performed by the working machine 10 for a short observation time (for example, periodically) while performing another task. For example, the operator may want to grasp the progress of the automatically performed task and the estimated time to completion. However, when the operator cannot approach and observe the display image G, the detailed screen (see Fig. 10A ) has a large amount of information. Therefore, it may be difficult for the operator to check the necessary information with a short observation time.

[0190] For this reason, the display image G shows a simple screen during automatic operation. Fig. 10A ), the simple screen during automatic operation is arranged to allow the operator to grasp information in a short observation time. More specifically, the amount of information displayed in the simple screen during automatic operation is less than the amount of information displayed in the detailed screen. The information about the automatic operation displayed in the simple screen during automatic operation includes only a part of the information about the automatic operation displayed in the detailed screen. More specifically, for example, Fig.11 As shown in FIG. 1 , the machine information display section F10A4 and the simple screen are not displayed during automatic operation. Fig. 10A . Since a simple screen is displayed as the display image G during the automatic operation, the operator can obtain information about the automatic operation of the working machine 10 even by glancing at the display image G. As a result, for example, even when the operator observes the display image G while performing another task (for example, manually operating or remotely operating a machine), the operator can monitor the operation of the working machine 10 without significantly affecting the efficiency of the other task (that is, can perform both the other task and monitor the automatic operation).

[0191] The image displayed in the simple screen during automatic operation is preferably simpler (more basic) than the image displayed in the detailed screen (see Fig. 10A). For example, the information display shown in the simple screen during automatic operation may be larger than the information display shown in the detailed screen. The information display may be text or a graphic (a diagram that visually presents information). In the simple screen during automatic operation, the brightness difference (contrast) between the information display and the background color of the information display may be greater than the contrast between the information display and the background color in the detailed screen. Information displayed in text form in the detailed screen may be displayed in graphic form in the simple screen during automatic operation (for example, see the current operation amount display part F11A3 described later). The simple screen during automatic operation may include portions that are not included in the detailed screen (for example, the task remaining time display part F11A2 and the detailed screen selection part F11A4 to be described later).

[0192] The contents (elements and items) displayed in the simple screen during automatic operation can be set by the operator, or can be set in advance in the information terminal 23. The contents displayed in the simple screen during automatic operation preferably include the information required for the operator to monitor the automatic operation, and the amount is as small as possible (that is, preferably the minimum required content). For example, the simple screen during automatic operation preferably includes a status display part F11A1 and an image indicating the progress of the automatic operation (for example, a task remaining time display part F11A2 and a current operation amount display part F11A3). The simple screen during automatic operation also includes a detailed screen selection part F11A4, a key status selection part F11A5a, an engine status selection part F11A5b, a lever lock status selection part F11A5c, an alarm operation part F11A7, and a pause selection part F11A8.

[0193] The status display portion F11A1 is a portion that displays the status of the working machine 10. The status display portion F11A1 displays the status of the automatic operation of the working machine 10. For example, the status of the working machine 10 displayed in the status display portion F11A1 includes at least one of a normal operation in progress, a standby state (a standby state after the automatically executed task is completed), or an emergency stop state. The status of the working machine 10 displayed in the status display portion F11A1 may include a detection operation in progress (i.e., a state in which the working machine 10 detects an object such as an operation target and operates automatically).

[0194] When the status display unit F11A1 displays information indicating that the working machine 10 is in the standby state, the operator can easily notice that the working machine 10 is in the standby state. Since the operator instructs the working machine 10 to perform automatic operation, the time during which the working machine 10 does not perform any task (i.e., is in the standby state) can be reduced. Therefore, the working efficiency of the working machine 10 can be improved.

[0195] When the state display F11A1 displays information indicating that the working machine 10 is in an emergency stop, the operator can easily notice that the working machine 10 is in an emergency stop. Therefore, the operator can handle the working machine 10 in an emergency stop more quickly (eg, more safely).

[0196] The status display portion F11A1 is preferably displayed to allow the operator to easily notice changes in the status display portion F11A1 even when the operator is not close to observe the display image G (for example, peripheral vision). In this case, the operator can reduce the number of times and duration of observing the display image G. When the operator observes the display image G while performing another task, the operating efficiency of the other task is improved because the number of times and duration of the operator observing the display image G are reduced. For example, the occupancy rate of the status display portion F11A1 in the display image G is preferably as large as possible. For example, the contrast between the status display portion F11A1 and the background of the status display portion F11A1 is preferably large.

[0197] The status display unit F11A1 may display text (expression) indicating the status of the working machine 10. For example, the text is "in normal operation" (or simply "in operation"), "in standby state", or "in emergency stop". The status display unit F11A1 may display different graphics (e.g., the shape and color of a face) according to the status of the working machine 10. More specifically, for example, the graphics (e.g., a face image) when the working machine 10 is in normal operation or on standby may be different from the graphics when the working machine 10 is in emergency stop. The image display (e.g., text or graphics) of the status display unit F11A1 may differ in color according to the status of the working machine 10. More specifically, for example, the image display of the status display unit F11A1 may be green when the machine is in normal operation, blue when the machine is on standby, and red when the machine is in emergency stop. The background color of the status display unit F11A1 may differ according to the status of the working machine 10.

[0198] The image display of the portion other than the status display portion F11A1 in the display image G may change according to the change in the state of the working machine 10 indicated by the status display portion F11A1. These "portions other than the status display portion F11A1" may refer to, for example, the entire main display area M or the outer periphery (edge) of the main display area M. More specifically, for example, according to the change in the state of the working machine 10 indicated in the status display portion F11A1, the color of the main display area M (the background color or the color of the edge) may change, or the brightness of the main display area M may change (for example, the main display area M flashes). The status display portion F11A1 may be displayed in the detailed screen (see Fig. 10A ).

[0199] The task remaining time display part F11A2 is a part that displays the remaining time of the automatically executed task. In the same manner as the image display of the status display part F11A1 changes according to the status of the working machine 10, the image display (for example, the text color or the background color) of the task remaining time display part F11A2 can be changed according to the remaining time of the automatically executed task. In the simple screen during automatic operation, the remaining operation count display part F10A2 (see Fig. 10A ).

[0200] The current operation amount display part F11A3 displays Fig. 10A The content displayed in the current operation amount display unit F10A3b shown in FIG. Fig. 10A Compared with the current operation amount display part F10A3b shown in Fig.11 The current operation amount display part F11A3 shown in FIG. 1 is displayed so that the operator can easily grasp the information. Fig.11 In the current operation amount display part F11A3 shown in Fig. 10A Compared with the current operation amount display unit F10A3b shown in FIG, the displayed text may be larger, and the contrast between the text and its background color may be greater. The current operation amount display unit F11A3 may change the image display (for example, the text color or the background color) according to the current operation amount. In addition, Fig.11 The current operation amount display unit F11A3 shown in the figure can display a graph showing the ratio (percentage) of the processed operation amount to the target operation amount. According to the ratio (percentage), the current operation amount display unit F11A3 changes the displayed image in multiple stages (for example, 5 stages). Specifically, for example, the current operation amount display unit F11A3 displays a graph indicating the amount of soil loaded into the vehicle (dump truck). The graph of the current operation amount display unit F11A3 can be displayed in the detailed screen (see Fig. 10A ). The detailed screen selection unit F11A4 is used when the display image G is switched from the simple screen during automatic operation to the detailed screen ( Fig. 10A ) is selected.

[0201] The key state selection unit F11A5a, the engine state selection unit F11A5b and the lever lock state selection unit F11A5c have Fig. 9C The key state selection unit F9C4, the engine state selection unit F9C5 and the lever lock state selection unit F9C6 shown in FIG. Fig.11 The alarm operation part F11A7 shown in FIG. 1 is a part selected when causing the work machine 10 to sound an alarm. In this way, the simple screen during automatic operation may include a selection part for operating the work machine 10. The pause selection part F11A8 is a part selected when pausing the automatic operation (details will be given below).

[0202] (End automatic operation)

[0203] Fig. 12A The automatic operation end screen is shown. When the automatically executed task is completed normally, the automatic operation end screen is displayed. For example, when the completed operation amount (loaded soil amount) reaches the target operation amount (target soil amount), the automatic operation end screen is displayed. The automatic operation end screen includes an end notification part F12A1, an end selection part F12A2, and a cancel selection part F12A3. The end notification part F12A1 is a part for notifying the operator of the end of the automatic operation. The end notification part F12A1 notifies the operator of the end of the task automatically executed based on the instruction data. The end selection part F12A2 is a part for selecting to end the task automatically executed based on the instruction data. The cancel selection part F12A3 is a part for selecting not to end the task automatically executed based on the instruction data. For example, when the cancel selection part F12A3 is selected, the screen before the automatic operation was started can be displayed again (see Fig. 9A ), or, for example, the automatic operation may be started again after changing the operation count and the target operation amount.

[0204] When the automatically executed task is finished, the result of the task may be displayed (result display unit). For example, the result of the task may include at least: the amount of completed operation goals, operation count and operation time.

[0205] Fig. 12B The pause screen is shown. The pause screen is a screen displayed when the automatic operation is paused. The pause of the automatic operation means that the automatic operation is paused when the completed operation amount (soil loading amount) has not yet reached the target operation amount (target soil amount). For example, when the pause selection part F10A8 (see Fig. 10A )、F11A8(See Fig.11 ), the pause screen is displayed. The pause screen includes a pause notification portion F12B1, an end selection portion F12B2, and a cancel selection portion F12B3. The pause notification portion F12B1 is a portion by which the automatic operation is notified of the pause. The pause notification portion F12B1 notifies that the task is completed in the current cycle. The end selection portion F12B2 is a portion (OK button) selected when the automatic operation is paused. The cancel selection portion F12B3 is a portion (Cancel button) selected when the automatic operation is not paused (i.e., the automatic operation continues).

[0206] When the automatic operation is suspended, the working machine 10 continues to operate automatically until the machine takes the initial posture, and stops when the machine takes the initial posture. For example, the initial posture is the initial posture when the automatic operation is started. For example, the initial posture is when setting a plurality of working stages P (see Figure 8 ) in the initial operation phase P (e.g., capture phase P1 (see Figure 8)) The following will describe a specific example of the pause of the automatic operation. For example, assuming that in the release phase P3 (see Figure 8 ) during the period, select the pause selection unit F11A8 (see Fig.11 In this case, the working machine 10 completes the release phase P3 and completes the return rotation phase P4 (see Figure 8 ), and stops when the machine takes the initial posture. Fig.11 ) thereafter, a notification indicating that a pause is being executed (a pause period display unit) may be displayed until the working machine 10 assumes the initial posture.

[0207] (Emergency Stop)

[0208] Fig.13A The emergency stop screen is displayed. The emergency stop screen is a screen displayed when the working machine 10 stops in an emergency.

[0209] The condition for emergency stop of the working machine 10 (emergency stop condition) may include selecting the emergency stop selection part F13A1. The emergency stop condition may include selecting the all machine stop selection part F13A2. The emergency stop condition may be another condition and may include a condition set in the controller of the working machine 10 or the automatic operation information processing device 20.

[0210] The emergency stop selection part F13A1 is a part (e.g., an emergency stop button) selected when stopping the working machine 10 whose information is displayed in the main display area M (i.e., the working machine 10 corresponding to the selected machine selection part T) in an emergency. The emergency stop selection part F13A1 may be displayed in automatic operation, and not displayed when the automatic operation is not performed.

[0211] The all-machine stop selection portion F13A2 is a portion (for example, an all-machine emergency stop button) that is selected when all the working machines 10 (already connected to the information terminal 23) are stopped in an emergency.

[0212] When all machine stop selection parts F13A2 are provided, the following effects can be achieved. For example, there is a case where, in order to stop the working machine 10 from the information terminal 23, the operator selects the machine selection part T corresponding to the working machine 10 to be stopped, and then selects the emergency stop selection part F13A1. In this case, if the working machine 10 that the operator wants to stop is different from the working machine 10 selected by the machine selection part T, the operator needs to select the machine selection part T corresponding to the working machine 10 to be stopped (switch screen), and then select the emergency stop selection part F13A1. Therefore, it is necessary to select the machine selection part T, and this operation takes time. In addition, when a plurality of working machines 10 are connected to the information terminal 23, the operator needs to perform a plurality of steps in order to stop all connected working machines 10. Specifically, the operator needs to repeatedly switch the machine selection part T, and select the emergency stop selection part F13A1. For example, when it is necessary to stop all connected working machines 10 in an emergency, such as due to a disaster, it is not desirable to spend time on the operation of emergency stop. For this reason, all machine stop selection parts F13A2 are provided. The operator is allowed to stop all connected working machines 10 in an emergency by selecting (e.g., pressing) the all machine stop selection part F13A2 (all machine stop function). Therefore, the operator can stop all connected working machines 10 without selecting the machine selection part T (i.e., without switching the screen).

[0213] The all machine stop selection section F13A2 is preferably always displayed when at least one work machine 10 is connected to the information terminal 23. More specifically, even if the content (state) of the display image G is variously changed (varied), when the work machine 10 is connected to the information terminal 23, the all machine stop selection section F13A2 is preferably always displayed in the display image G. Specifically, for example, the all machine stop selection section F13A2 is preferably displayed in a state where the work machine 10 is selected in the machine selection section T, and information about the selected work machine 10 is displayed in the display image G. In addition, even in a screen where the work machine 10 is connected to the information terminal 23 but the work machine 10 is not selected in the machine selection section T, the all machine stop selection section F3B4 is preferably displayed, for example, as Figure 3B When no work machine 10 is connected to the information terminal 23, the main screen may not be displayed. Fig.13A All mechanical stop selection parts shown in F13A2 (see Fig.14A ).

[0214] The all machine stop selection unit F13A2 is preferably always displayed at a specific position (always at the same position) in the display image G when at least one working machine 10 is connected to the information terminal 23. The specific position is the position of a part of the display image G, and is the position of a specific part of the display image G. For example, the all machine stop selection unit F13A2 is displayed at a portion where the image display does not change even if the state of the display image G changes. Specifically, for example, the all machine stop selection unit F13A2 is displayed in a portion that does not change according to the selection state of the machine selection unit T. Fig.13A In the example shown in , the all machine stop selection part F13A2 is displayed below the main selection part H, below the task selection part A, and in the lower left portion of the display image G. For example, the all machine stop selection part F13A2 may be displayed in the upper portion of the display image G, or may be displayed in an area above the machine selection part T (i.e., in the header at the upper portion of the screen) (not shown).

[0215] The all-machine stop selection unit F13A2 is always selectable (operable) when at least one working machine 10 is connected to the information terminal 23. The all-machine stop selection unit F13A2 is in a state where all working machines 10 connected to the information terminal 23 can always be stopped in an emergency when at least one working machine 10 is connected to the information terminal 23.

[0216] The all-machine stop selection section F13A2 is preferably arranged so that all connected working machines 10 are stopped in an emergency by only one operation (one action) performed on the all-machine stop selection section F13A2. In this case, the operator is allowed to stop the connected working machines 10 in an emergency by performing a simple operation. The all-machine stop selection section F13A2 may be arranged so that all connected working machines 10 are stopped in an emergency by performing multiple operations. For example, when a confirmation operation (e.g., pressing an OK button) is performed after selecting the all-machine stop selection section F13A2, the connected working machines 10 may be stopped in an emergency.

[0217] When the all-machine stop selection unit F13A2 is selected, the information terminal 23 transmits an emergency stop signal to all connected working machines 10 at the same time.

[0218] When the working machine 10 (one or more working machines 10) stops in an emergency, the machine selection unit T is displayed, for example, as follows. The machine selection unit T corresponding to the working machine 10 that stops in an emergency indicates that an emergency stop has occurred (emergency stop display). The emergency stop display is displayed so as to be distinguishable (for the operator) from the image display of the machine selection unit T when the working machine 10 is not stopped in an emergency. Specifically, for example, the color of the machine selection unit T is green when the working machine 10 is not stopped in an emergency, and is red when the working machine 10 stops in an emergency. When there are multiple working machines 10 connected to the information terminal 23 and all of these machines are stopped in an emergency, all machine selection units T (for example, the first machine selection unit T1 and the second machine selection unit T2) are in the emergency stop display. As described above, the unselected machine selection unit T is displayed in a state indicating that it is not selected (for example, flashing), and the selected machine selection unit T is displayed in a state indicating that it is selected (for example, non-flashing).

[0219] The emergency stop screen includes an emergency stop notification unit F13A3. The emergency stop notification unit F13A3 is a unit for notifying the emergency stop of the working machine 10. The emergency stop notification unit F13A3 can notify the cause of the emergency stop of the working machine 10 (error content).

[0220] When the emergency stop selection part F13A1 or the all machine stop selection part F13A2 is selected (hereinafter, this case may be referred to as "when the emergency stop is selected"), the working machine 10 stops in an emergency, for example, in the following manner. For example, when the emergency stop is selected, the working machine 10 may be immediately converted to the emergency stop state. In this case, the working machine 10 becomes stationary in the middle of the task currently being performed. Alternatively, for example, when the emergency stop is selected, the working machine 10 may be in the middle of the task currently being performed (for example, the work phase P currently being performed (see Figure 8 ) or one cycle of automatic operation) is completed, the work machine 10 switches to the emergency stop state and becomes stationary. In addition, for example, when the work machine 10 is in a state of maintaining the operation target, after automatically releasing the operation target, the work machine 10 can switch to the emergency stop state and become stationary. In this case, the work machine 10 preferably releases the operation target at a position where the operation target can be stably provided. When the work machine 10 selects an emergency stop while maintaining the operation target, a notification corresponding to each task type described below can be made (see Fig. 13B ).

[0221] After the working machine 10 stops in an emergency, recovery is performed, for example, in the following manner. The recovery may indicate the cancellation of the emergency stop state. Specifically, for example, the recovery may indicate the cancellation of the emergency stop state by turning off the engine of the working machine 10 and disconnecting the information terminal 23 from the working machine 10. The recovery may indicate the cancellation of the emergency stop state and the restart of automatic operation. For example, the operator selects the machine selection part T corresponding to the working machine 10 to be restored, and performs an operation (recovery operation) to restore the selected working machine 10. When multiple (for example, all) working machines 10 stop in an emergency, the operator performs a recovery operation for each machine. The information terminal 23 can be arranged to allow multiple working machines 10 to be restored at the same time.

[0222] Fig. 13B An emergency stop screen corresponding to the work content is shown. This screen includes an emergency stop notification part F13B1 and a selection part F13B2.

[0223] In this regard, when the working machine 10 stops in an emergency, the state of the working machine 10 can be changed, for example, by shutting down the engine or canceling the automatic operation mode and setting a manually operable operation mode. However, depending on the operation content, the arrangement of the working machine 10, the type of the front-end attachment device 15c, etc., problems may arise due to the change in the state of the working machine 10. Therefore, it is preferable to make a notification that enables the problem to be avoided. A specific example of such a notification will be described below. The emergency stop notification unit F13B1 is displayed when the working machine 10 performing the lifting magnet task stops in an emergency. The emergency stop notification unit F13B1 notifies the content of the problem. More specifically, the emergency stop notification unit F13B1 notifies that if the engine is stopped while the magnet adsorbs the operating target (metal), the operating target may fall from the magnet. In addition, the emergency stop notification unit F13B1 notifies a method to avoid the problem. In this example, the emergency stop notification unit F13B1 notifies a method to avoid the operating target from falling from the magnet. Specifically, the emergency stop notification unit F13B1 notifies the operator to board the working machine 10 and lift the lever lock (so as to be in a locked state) to make the working machine 10 inoperable. The emergency stop notification unit F13B1 notifies the operator to turn on the key in the driver's cabin 13a and maintain the engine on (driving) state. The emergency stop notification unit F13B1 notifies that the selection unit F13B2 (OK button) is pressed in the state where the lever lock is lifted and the key is turned. When the selection unit F13B2 is selected, a notification (not shown) to cancel the automatic operation mode is displayed, and the automatic operation mode is canceled. Thereafter, for example, the lever lock is released, and the operator operates the working machine 10 to unload the operation target. The above example illustrates a case where the working machine 10 performing the task of lifting the magnet stops in an emergency. In addition to this, when the working machine 10 performing another type of task stops in an emergency, a notification corresponding to that type of task may be displayed.

[0224] (set up)

[0225] Fig.14A The connection setting screen is shown. The connection setting screen is a screen displayed when performing settings for connecting the information terminal 23 with the working machine 10. For example, when the setting part B5 is selected, the connection setting screen is displayed. The connection setting screen includes a setting item selection part F14A1, a connection setting selection part F14A2, a new connection selection part F14A3, a registered machine selection part F14A4, and a parameter setting selection part F14A5.

[0226] The setting item selection section F14A1 is a portion that is displayed to allow selection of one of the setting items. The connection setting selection section F14A2 is a portion of the setting item selection section F14A1. The connection setting selection section F14A2 is a portion that is selected when performing settings related to the connection between the information terminal 23 and the work machine 10. The new connection selection section F14A3 is a portion that is selected when a new work machine 10 is connected to the information terminal 23. When connecting a new work machine 10, for example, information for specifying the work machine 10 to be connected (for example, an IP address and a registered machine name) is set. For example, the registered machine name is the name of the work machine 10 that is specified in the machine selection section T (see Fig. 14B ), register the name of the working machine 10 displayed in the machine selection section F14A4.

[0227] Fig. 14B The newly connected task selection screen is shown. The newly connected task selection screen includes task selection sections F14BA1 and F14BA2. Task selection sections F14BA1 and F14BA2 are sections for selecting which type of task is associated with the work machine 10 to be connected when a new work machine information terminal is connected to the information terminal 23. Note that this newly connected task selection screen is a task selection screen (application selection screen) associated with the newly connected work machine 10, not the main screen (see Figure 3A and Figure 3B ). In this screen, the machine quantity display part F3B3 (see Figure 3B ).

[0228] Fig.14A The registered machine selection part F14A4 shown in is a part that displays information of the working machine 10 (registered machine) that was connected to the automatic operation information processing device 20 (e.g., the information terminal 23) in the past. For example, the registered machine selection part F14A4 displays the registered machine name of the registered machine. When a registered machine is selected through the registered machine selection part F14A4, information about the registered machine can be notified. The information to be notified may include the registered machine name, connection information between the registered machine and the automatic operation information processing device 20 (e.g., the date and time of the previous connection), and information about the tasks performed at the time of the previous connection (e.g., the previous operation time).

[0229] The display state of the registered machine displayed in the registered machine selection part F14A4 may change according to whether the machine corresponds to the task selected in the task selection part A. For example, if the loading task selection part A1 is selected in the task selection part A, the registered machine selection part F14A4 may display the registered machine to allow the operator to distinguish whether the machine corresponds to the loading task. More specifically, for example, the registered machine selection part F14A4 may make the registered machine corresponding to the task type selected in the task selection part A active (selectable), and may make the registered machine that does not correspond to the task type selected in the task selection part A inactive (unselectable). The registered machine selection part F14A4 may not display the registered machine that does not correspond to the task type selected in the task selection part A.

[0230] The parameter setting selection part F14A5 is a part (not shown) for selecting when setting parameters. The parameters to be set may be parameters related to the working machine 10 or parameters related to automatic operation.

[0231] Fig. 14C The sensor status display screen is shown. The sensor status display screen is a screen that displays the status of the sensor. The sensor status display screen includes a sensor status selection section F14C1 and a sensor status display section F14C2. The sensor status selection section F14C1 is a portion that is selected to display the sensor status display screen (sensor status display section F14C2). The sensor status display section F14C2 displays the status of a sensor provided at or outside the working machine 10 in the same manner as the sensor status display section F10C1 (see FIG. 10 ).

[0232] Fig.14DAn attachment operation screen is shown. The attachment operation screen is a screen that displays the status of each device (attachment) of the working machine 10 in a manner that each attachment can be (remotely) operated. The attachment operation screen includes an attachment operation selection section F14D1 and an attachment status display setting section F14D2. The attachment operation selection section F14D1 is a portion through which the attachment operation screen (attachment status display setting section F14D2) is displayed. The attachment status display setting section F14D2 is a portion that displays the status of the attachment (attachment status display section). The attachment status display setting section F14D2 is a portion that displays the status of a settable (changeable) attachment (attachment status setting section). In the attachment status display setting section F14D2, for example, the status of a displayable and settable attachment (displayable and settable attachment status) may include turning on and off lights (arm lights, cabin lights, etc.). The displayable settable accessory states may include activation and deactivation of wipers, an operating interval of wipers (intermittent or continuous), turning washers (devices for cleaning windows of the driver's cabin 13a) on and off, and turning DPF (diesel particulate filter) regeneration on and off. The displayable settable accessory states may include operating modes of the work machine 10 (high speed, low speed, and fuel economy (ECO)).

[0233] (Specific example of operation)

[0234] refer to Fig.15A The flowchart shown in Figure 1 The specific examples of the operation of the automatic operation information processing device 20 (mainly the operation of the control unit 23c), the automatic operation information processing method, and the automatic operation information processing program shown in FIG. 1 are shown in FIG. 2. In the following, unless otherwise specified, the description will be given in the order of steps of the operation (task, process) of the automatic operation information processing device 20. The order of steps of the operation can be changed in various ways. In the following description, reference will be made to Fig.15A explain Fig.15A Steps S11 to S72 shown in FIG.

[0235] In step S11, Figure 1 The information terminal 23 shown in FIG. 1 is connected to the working machine 10 .

[0236] In step S21, the target range TA (see Fig. 10D )(i.e., perform area teaching)(See Figure 6 ).

[0237] In step S22, the target route PA (see Fig. 10D )(i.e., executing route teaching)(See Figure 4C ).

[0238] In step S31, the operation mode of the working machine 10 is changed (converted) to the automatic operation mode. For example, when the above-mentioned automatic operation mode start condition is satisfied (see Fig. 7A and Figure 7B ), the operation mode of the working machine 10 is switched to the automatic operation mode (see Figure 7C ).

[0239] In step S32, the setting of automatic operation (automatic operation setting) is performed. More specifically, Figure 8 As shown in , perform the initial settings for automatic operation. Fig. 9A , Fig. 9B , Fig. 9C and Fig.9D As shown in , set the operation speed of automatic operation, etc.

[0240] In step S41, preparations are made for starting the automatic operation of the working machine 10. For example, the posture of the working machine 10 is changed to a posture that enables starting the automatic operation.

[0241] In step S42, the working machine 10 starts automatic operation (see Fig. 10A , Fig. 10B , Fig. 10C , Fig. 10D and Fig.11 ).

[0242] In steps S43, S44 and S45, the following operations are performed. When a transport vehicle (dump truck) is detected in the transport vehicle (dump truck) detection area (step S43), a target route PA (see FIG. 14 ) provided for the transport vehicle to perform a task (e.g., dumping soil) is displayed in the display unit 23o. Fig. 10D ) (Step S44). When the operator selects the option to cause the working machine 10 to perform a mission along the target route PA, the working machine 10 starts the mission for the transport vehicle (Step S45).

[0243] In step S46, the control unit 23c (see Figure 1 ) monitors the automatic operation of the working machine 10. For example, the control unit 23c monitors the status of the automatically executed task (progress, whether the task is completed normally, etc.), such as Fig. 10A and Fig.11 In addition, for example, the control unit 23c also monitors whether to suspend the automatic operation (whether to select Fig. 10A 13A1 ) and whether to stop the automatic operation in an emergency (whether to select the pause selection part F10A8 shown in FIG. 13 ).

[0244] In step S51, the task automatically performed by the working machine 10 is completed, and the automatic operation of the working machine 10 is stopped. When the automatic operation is normally completed, the display unit 23o displays Fig. 12A When the automatic operation is suspended, the display unit 23o displays Fig. 12B The pause screen shown in .

[0245] In step S52, the result of the automatically executed task is displayed on the display unit 23o. Fig. 10A The current operation amount display part F10A3b and Fig.11 The current operation amount display part F11A3 shown in FIG. 1 displays the operation amount after the task is completed.

[0246] In step S53, the control unit 23c (see Figure 1 ) determines whether to perform an automatic operation that is the same as (or substantially the same as) the automatic operation that has been completed. For example, the control unit 23c may perform the determination based on the input (selection) of the operator. In addition, for example, the control unit 23c may automatically perform the determination based on the state of the task (e.g., the amount of operation, the number of times, and the time). When the automatic operation that is the same as (or substantially the same as) the automatic operation that has been completed is performed (i.e., "Yes" in step S53), the control unit 23c returns the process to step S32. When the automatic operation that is the same as (or substantially the same as) the automatic operation that has been completed is not performed (i.e., "No" in step S53), the control unit 23c returns the process to step S61.

[0247] In step S61, preparations (machine-side end preparations) are performed for ending the automatic operation mode in the working machine 10. For example, the control unit 23c changes the posture of the working machine 10 to the initial posture.

[0248] In step S62 , the control unit 23 c ends the automatic operation mode of the working machine 10 .

[0249] In step S63, the control unit 23c determines whether the task performed by the working machine 10 is completed. For example, the control unit 23c determines whether the task automatically performed by the working machine 10 is completed (for example, whether the task of the day is completed) by using the information terminal 23. When the task performed by the working machine 10 is completed ("Yes" in step S63), the control unit 23c causes the flow to proceed to step S71. When the task performed by the working machine 10 is not completed ("No" in step S63), the control unit 23c causes the flow to return to step S21.

[0250] In step S71, the working machine 10 is turned off. Specifically, the engine of the working machine 10 is turned off, and the power is turned off. For example, the information terminal 23 may send a command to turn off the working machine 10 to the working machine 10. Alternatively, the working machine 10 may be turned off manually by the operator.

[0251] In step S72 , the information terminal 23 and the working machine 10 are disconnected.

[0252] (All machinery emergency stop procedures)

[0253] refer to Fig. 15B The following describes the flowchart shown in Fig.13A When all the machines shown in the stop selection section F13A2 are Figure 1 Specific examples of the processes performed by the automatic operation information processing device 20 (mainly the control unit 23c) shown in FIG. 1 are shown in FIG. 2. In the following, unless otherwise specified, the description will be given in the order of steps of the operation (task, process). The order of steps of the operation can be changed in various ways.

[0254] When choosing Fig.13A When all machines are stopped as shown in the F13A2 selection section, regardless of the Fig.15A Whichever step is shown in the table is executed (in interrupt mode) Fig. 15B All the emergency stop procedures for machinery are shown in the. Note that reference will be made to Fig. 15B To illustrate Fig. 15B Steps S91 to S94 shown in FIG.

[0255] In step S91, Figure 1 The information terminal 23 shown in FIG. 2 sends an emergency stop signal to all the working machines 10 that are connected (have been connected) to the information terminal 23 .

[0256] In step S92, all the working machines 10 connected to the information terminal 23 are stopped in an emergency.

[0257] In step S93, all the working machines 10 connected to the information terminal 23 are shut down. Fig.15A is the same as step S71 shown in . )

[0258] In step S94 , the connection between the information terminal 23 and all the working machines 10 that have been connected to the information terminal 23 is cut off.

[0259] Note that each operation (e.g., display, selection, and setting) of the automatic operation information processing device 20 can be regarded as each step in the method and the program. Specifically, for example, selection can be regarded as a selection step. Further specifically, for example, in the mechanical selection section T (see Figure 2) can be regarded as a work machine selection step.

[0260] (Effects of the first aspect of the invention)

[0261] Figure 1 The effects of the automatic operation information processing device 20 shown in are as follows.

[0262] [Feature 1] The automatic operation information processing device 20 is connected to the working machine 10 capable of performing automatic operation in a communicable manner. The automatic operation information processing device 20 includes a display unit 23o and an input unit 23i. The display unit 23o is configured to display a display image G including information related to the automatic operation. The input unit 23i receives information related to the automatic operation in response to an operation by an operator.

[0263] With the above-described [Feature 1], the operator can easily check and input information related to the automatic operation of the working machine 10 by using the automatic operation information processing device 20 .

[0264] (Effect of the Second Aspect of the Invention)

[0265] [Feature 2] Figure 2 As shown in FIG. 1 , the display image G has a task selection portion A. The task selection portion A is displayed for each type of task automatically performed by the working machine 10 (see FIG. 1 ). Figure 1 ; hereinafter, this also applies to the working machine 10). The task selection section A displays the task type that can be selected through the input unit 23i (see Figure 1 ).

[0266] With the above-described [Feature 2], the operator can easily select the type of task to be performed by the working machine 10 .

[0267] (Effects of the third aspect of the invention)

[0268] [Feature 3] The image of the task selection portion A changes according to the number of connected working machines 10 for each task type.

[0269] With the above-mentioned [Feature 3], the operator can easily grasp the number of connected working machines 10 for each task type.

[0270] (Effects of the fourth aspect of the invention)

[0271] [Feature 4] The display image G includes a function selection section B. The function selection section B is displayed for each function corresponding to the task type selected by the task selection section A. The function selection section B displays the functions that can be selected by the input unit 23i (see Figure 1 ).

[0272] With the above-mentioned [Feature 4], the operator can easily select a function corresponding to the task type selected by the task selection section A.

[0273] (Effects of the Fifth Aspect of the Invention)

[0274] [Feature 5] The display image G includes a machine selection portion T and a main display area M. The machine selection portion T is displayed for each working machine 10 so that each working machine 10 can be selected by the input unit 23i (see Figure 1 ). The main display area M displays information about the working machine 10 selected by the machine selection section T. The main display area M is an area different from the machine selection section T in the display area of ​​the display unit 23o (see Figure 1 ).

[0275] With the above-mentioned [Feature 5], the operator can easily select the working machine 10 whose information is displayed in the main display area M.

[0276] (Effects of the Sixth Aspect of the Invention)

[0277] [Feature 6] The image of the machine selection unit T changes according to the state of the working machine 10.

[0278] With the above-mentioned [Feature 6], the operator can easily grasp the state of the working machine 10 by checking the machine selection part T.

[0279] (Effect of the Seventh Aspect of the Invention)

[0280] [Feature 7] The image of the machine selection unit T changes depending on at least whether the current state is a state in which the automatic operation is in progress, a state in which the automatic operation has been completed normally, or a state in which the working machine 10 is stopped in an emergency.

[0281] By utilizing the above-mentioned [Feature 7], by checking the machine selection part T, the operator can easily grasp at least the state in which the automatic operation is in progress, the state in which the automatic operation has been completed correctly, or the state in which the working machine 10 has stopped in an emergency.

[0282] (Effects of the Eighth Aspect of the Invention)

[0283] [Feature 8] The image of the machine selection portion T changes between a case where the machine selection portion T corresponding to the working machine 10 whose state has changed is selected and a case where the machine selection portion T is not selected after the state of the working machine 10 has changed.

[0284] With the above-mentioned [Feature 8], the operator can easily understand whether the machine selection part T corresponding to the working machine 10 whose state has been changed is selected by checking the machine selection part T. As a result, the operator can easily understand whether the information of the working machine 10 whose state has been changed has been confirmed in the main display area M.

[0285] (Effects of the Ninth Aspect of the Invention)

[0286] The automatic operation is performed based on instruction data set by the operator by operating the working machine 10 .

[0287] [Feature 9] Figure 8 As shown in FIG. 1 , the display image G has an operation speed setting section SS. The operation speed setting section SS is displayed to allow the operation speed to be set by the input unit 23i (see FIG. 1 ). Figure 1 ) Select one of the multiple operating speeds including the operating speed of the working machine 10 in the instruction data as the operating speed in the automatic operation.

[0288] With the above-mentioned [Feature 9], the operator can easily set the operating speed of the working machine 10 in the automatic operation.

[0289] (Effect of the Tenth Aspect of the Invention)

[0290] [Feature 10] The fastest operating speed among the multiple operating speeds (in the above-mentioned [Feature 9]) is the operating speed of the working machine 10 in the command data.

[0291] In the above-mentioned [Feature 10], the operating speed of the working machine 10 in the automatic operation is equal to or lower than the operating speed in the instruction data. Therefore, it is possible to suppress the working machine 10 from automatically operating at a high operating speed that is not intended by the operator.

[0292] (Effect of the eleventh aspect of the invention)

[0293] [Feature 11] One cycle of the automatic operation includes a plurality of work phases P. The operation speed setting section SS is displayed so that the operation speed in the automatic operation can be set for each work phase P.

[0294] With the above-described [Feature 11], the operator can easily set the operating speed of the working machine 10 in automatic operation for each working phase P.

[0295] (Effect of the twelfth aspect of the invention)

[0296] [Feature 12] The display image G switches between the first speed display state SP1 and the second speed display state SP2 (see Fig. 9A ). The first speed display state SP1 is a state in which the operation speeds of all levels in each operation stage P are displayed. Fig. 9A As shown in FIG. 1 , the second speed display state SP2 is a state in which only the operation speed of the level set by the operation speed setting section SS in at least one work phase P is displayed.

[0297] According to the above [Feature 12], Fig. 9AThe image display ratio in the second speed display state SP2 shown in FIG. Figure 8 The image display in the first speed display state SP1 shown in FIG. 1 is simple. Therefore, the operator can easily grasp the speed of the vehicle by Figure 8 The operation speed setting section SS shown in FIG. 1 sets the operation speed.

[0298] (Effects of the Thirteenth Aspect of the Invention)

[0299] [Feature 13] Fig. 10D As shown in , one cycle of the automatic operation includes a plurality of work phases P. The display image G includes a work phase display portion PD that displays information indicating in which work phase P the task performed by the work machine 10 is.

[0300] By utilizing the above-mentioned [Feature 13], the operator can easily understand in which work phase P the task executed by the working machine 10 is.

[0301] (Effect of the Fourteenth Aspect of the Invention)

[0302] [Feature 14] Fig.13A As shown in FIG. 1 , the display image G includes an all-machine stop selection portion F13A2 by which all connected working machines 10 (which have been connected to the automatic operation information processing device 20 ) are stopped in an emergency.

[0303] Utilizing the above-described [Feature 14], the operator is allowed to stop all the working machines 10 connected to the automatic operation information processing device 20 by performing a simple operation in an emergency.

[0304] (Effects of the Fifteenth Aspect of the Invention)

[0305] [Feature 15] The all-machine stop selection unit F13A2 stops all connected working machines 10 (which have been connected to the automatic operation information processing device 20) by only a single operation in an emergency.

[0306] Utilizing the above-described [Feature 15], the operator is allowed to stop all the working machines 10 connected to the automatic operation information processing device 20 by performing simpler operations in an emergency.

[0307] (Effect of the Sixteenth Aspect of the Invention)

[0308] [Feature 16] When the working machine 10 is connected (ie, when the working machine 10 is connected to the automatic operation information processing device 20), the all machine stop selection parts F13A2 are always displayed at a specific position as a part of the display image G.

[0309] With the above-described [Feature 16], the operator can easily grasp where the all-machine stop selection parts F13A2 are located on the display image G. Therefore, the operator is allowed to easily stop all the working machines 10 connected to the automatic operation information processing device 20 in an emergency.

[0310] (Effect of the Seventeenth Aspect of the Invention)

[0311] [Feature 17] Figure 2 As shown in FIG. 1 , the display image G includes a machine state display unit F2A3 that displays the state of the working machine 10. The image of the machine state display unit F2A3 is synchronized with the image display of the state display device 13b (see FIG. 14 ). Figure 1 ). The status display device 13 b is mounted on the actual working machine 10 , and is used to display the status of the working machine 10 .

[0312] With the above-mentioned [Feature 17], the operator can grasp the state of the actual working machine 10 by viewing the machine state display portion F2A3 displayed in the display image G without viewing the state display device 13 b of the actual working machine 10 .

[0313] (Effects of the Eighteenth Aspect of the Invention)

[0314] [Feature 18] The display image G can be displayed in the detail screen (first screen) (see Fig. 10A ) and the simple screen (second screen) during automatic operation (see Fig.11 ). Fig.11 As shown in , the information about automatic operation displayed in the simple screen during automatic operation includes only the detailed screen (see Fig. 10A ) about automatic operations.

[0315] According to the above [Feature 18], compared with the case where the operator views the detailed screen (see Fig. 10A ), when the operator views a simple screen during automatic operation, the operator can easily obtain information about the automatic operation (for example, with a short viewing time).

[0316] (Effects of the Nineteenth Aspect of the Invention)

[0317] [Feature 19] The simple screen during automatic operation includes a status display unit F11A1 that displays status related to automatic operation. The status displayed by the status display unit F11A1 includes a status in which the working machine 10 is automatically performing a task, a status in which the working machine 10 is on standby, and a status in which the working machine 10 is stopped in an emergency.

[0318] By utilizing the above-mentioned [Feature 19], the operator can easily grasp the state of the working machine 10 (for example, in a short observation time).

[0319] (Effect of the 20th Aspect of the Invention)

[0320] [Feature 20] The automatic operation information processing method includes a connection step, a display step, and an input step. In the connection step, a connection is made with a working machine 10 capable of performing automatic operation in a communicative manner. In the display step, a display image G including information about the automatic operation is displayed on the display unit 23o. In the input step, an operator's input information is obtained. Figure 1 Information on automatic operation is inputted through the input unit 23i shown in FIG.

[0321] With the above-described [Feature 20], the operator can easily check and input information on the automatic operation of the working machine 10 .

[0322] (Effect of the twenty-first aspect of the present invention)

[0323] [Feature 21] The automatic operation information processing program causes the computer (control unit 23c) to execute the connection step, display step, and input step. In the connection step, a connection is made with the working machine 10 capable of performing automatic operation in a communicable manner. In the display step, a display image G including information about the automatic operation is displayed in the display unit 23o. In the input step, information about the automatic operation input by the operator through the input unit 23i is obtained.

[0324] With the above-described [Feature 21], the operator can easily check and input information on the automatic operation of the working machine 10 .

[0325] (Modification plan)

[0326] The above-described embodiments may be modified in various ways. For example, the number of components (e.g., the parts of the display image G) in the above-described embodiments (including the modification scheme) may be changed, and one or some of the components may not be provided. For example, the modification schemes of the above-described embodiments may be combined in various ways. For example, the configuration of the components may be changed. For example, the inclusion relationship of the components may be changed. For example, a component described as a component that is secondary to another component may not be included in the other component, and may be included in different components. For example, what is described as different components or different parts may be a single component or part. For example, what is described as a single component or part may be provided as multiple components or parts in a separate manner. For example, the conversion order of the display image G may be changed. For example, components described as different screens may be displayed in the same screen. For example, each component may have only certain characteristics (operating function, arrangement, shape, action, etc.).

[0327] Reference numerals list

[0328] 10 Operating machinery

[0329] 13b Status display device

[0330] 20 Automatic information processing device

[0331] 23c Control Unit

[0332] 23i Input Unit

[0333] 23o Display unit

[0334] A. Task Selection Department

[0335] B Function selection section

[0336] F2A3 mechanical status display

[0337] F3B4, F13A2 all mechanical stop selection parts

[0338] F11A1 Status display

[0339] G Display image

[0340] M Main display area

[0341] P Operation Phase

[0342] PD Operation stage display

[0343] SS Operation speed setting section

[0344] SP1 First speed display status

[0345] SP2 Second speed display status

[0346] T Mechanical Selection Department

Claims

1. An automatic operation information processing device, which is connected to an automatically operable working machine in a communicable manner, the automatic operation information processing device comprising: a display unit configured to display a display image including information related to the automatic operation; as well as An input unit is configured to receive information related to the automatic operation in response to an operation by an operator.

2. The automatic operation information processing device according to claim 1, wherein: The displayed image includes: A task selection unit is displayed for each task type automatically executed by the working machine, and displays the task type in a manner that allows selection via the input unit.

3. The automatic operation information processing device according to claim 2, wherein: The image display in the task selection portion changes for each of the task types according to the number of connected working machines corresponding to each of the task types.

4. The automatic operation information processing device according to claim 2 or 3, wherein: The displayed image includes: A function selection section is displayed for each function corresponding to the task type selected in the task selection section, and displays the function in a manner that can be selected by the input unit.

5. The automatic operation information processing device according to any one of claims 1 to 4, wherein: The displayed image includes: a machine selection unit that is displayed for each working machine so that each working machine can be selected by the input unit; and A main display area displays information on the working machine selected by the machine selection section and is an area different from the machine selection section among display areas on the display unit.

6. The automatic operation information processing device according to claim 5, wherein: The image of the machine selection unit changes according to the state of the working machine.

7. The automatic operation information processing device according to claim 6, wherein: The image of the machine selection unit changes according to at least one of a state in which the automatic operation is in progress, a state in which the automatic operation has been completed correctly, or a state in which the working machine is stopped in an emergency.

8. The automatic operation information processing device according to claim 6 or 7, wherein: After the state of the working machine is changed, the image of the machine selection section changes between a case where the machine selection section corresponding to the working machine whose state has been changed is selected and a case where the machine selection section whose state has been changed is not selected.

9. The automatic operation information processing device according to any one of claims 1 to 8, wherein: The automatic operation is performed based on instruction data set by the operator by operating the working machine, and The displayed image includes: An operating speed setting section is displayed to allow one of a plurality of operating speeds including the operating speed of the working machine in the instruction data to be selected through the input unit as the operating speed in the automatic operation.

10. The automatic operation information processing device according to claim 9, wherein: The fastest operating speed among the multiple operating speeds is the operating speed of the working machine in the command data.

11. The automatic operation information processing device according to claim 9 or 10, wherein: One cycle of the automatic operation includes multiple operation stages, and The operation speed setting section is displayed so that the operation speed in the automatic operation can be set for each of the work stages.

12. The automatic operation information processing device according to claim 11, wherein: The display image switches between: a first speed display state in which the operating speeds of all levels in each of the working stages are displayed; and In a second speed display state, only the operation speed of the level set by the operation speed setting section in at least one of the work stages is displayed.

13. The automatic operation information processing device according to any one of claims 1 to 12, wherein: One cycle of the automatic operation includes multiple operation stages, and The display image includes a work stage display portion that displays information indicating in which work stage a task performed by the work machine is located.

14. The automatic operation information processing device according to any one of claims 1 to 13, wherein: The display image includes an all-machine stop selection section, by which all connected working machines are stopped in an emergency.

15. The automatic operation information processing device according to claim 14, wherein: The all-machine stop selection unit stops all connected working machines by only a single operation in an emergency.

16. The automatic operation information processing device according to claim 14 or 15, wherein: When the working machine is connected, the all machine stop selection section is always displayed at a specific position as a part of the display image.

17. The automatic operation information processing device according to any one of claims 1 to 16, wherein: The display image includes a machine state display unit that displays the state of the working machine, and The image of the machine state display section is synchronized with the image display of a state display device mounted on the actual working machine, and is used to display the state of the working machine.

18. The automatic operation information processing device according to any one of claims 1 to 17, wherein: The display image can be switched between a first screen and a second screen, and The information about the automatic operation displayed in the second screen includes only a part of the information about the automatic operation displayed in the first screen.

19. The automatic operation information processing device according to claim 18, wherein: The second screen includes a status display portion that displays a status of the automatic operation, and The status displayed by the status display unit includes: The working machine is in a state of automatically performing a task; The working machine is in a standby state; and The working machine is stopped in an emergency.

20. An automatic operation information processing method, comprising: A step of connecting to a working machine capable of performing an automatic operation in a communicable manner; a display step of displaying a display image including information related to the automatic operation on a display unit; as well as The step of acquiring information on the automatic operation input by an operator through an input unit.

21. An automatic information processing program causing a computer to execute: A step of connecting to a working machine capable of performing an automatic operation in a communicable manner; a display step of displaying a display image including information related to the automatic operation on a display unit; and The step of acquiring information on the automatic operation input by an operator through an input unit.

Images