Autonomous driving system

Abstract

This system allows for easy recovery of work machinery that has become unstable during autonomous operation. [Solution] The automated driving system 1 comprises a work machine 10 and an information processing device 40. The work machine 10 is capable of automated driving. The information processing device 40 is connected to the work machine 10 in a communicative manner. The information processing device 40 comprises a display unit 42, an input unit 41, and a control unit 43. The display unit 42 displays information related to the work machine 10. The input unit 41 receives operations from the operator. The control unit 43 controls the display image G to be displayed on the display unit 42 and the information related to the input of the input unit 41. The control unit 43 performs automated driving processing and remote operation processing. In the automated driving processing, information related to automated driving is displayed on the display unit 42. In the automated driving processing, the input unit 41 receives input related to automated driving. In the remote operation processing, the work machine 10 is remotely operated based on the operations received by the input unit 41.

Description

Technical Field

[0001] The present invention relates to an automatic driving system related to automatic driving.

Background Art

[0002] For example, Patent Document 1 etc. describes a working machine that performs automatic driving.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the working machine becomes unstable during automatic driving, such as when the vehicle body floats, it is desired that it can easily return to a stable state.

[0005] Therefore, an object of the present invention is to provide an automatic driving system that can easily return a working machine that has become unstable during automatic driving.

Means for Solving the Problems

[0006] The automated driving system comprises a work machine and an information processing device. The work machine is capable of automated driving. The information processing device is communicatively connected to the work machine. The information processing device includes a display unit, an input unit, and a control unit. The display unit displays information related to the work machine. The input unit receives operations from an operator. The control unit controls the display image to be displayed on the display unit and the information related to the input from the input unit. The control unit performs automated driving processing and remote operation processing. The automated driving processing displays information related to the automated driving on the display unit. The automated driving processing involves the input unit receiving input related to the automated driving. The remote operation processing remotely operates the work machine based on the operations received by the input unit. [Effects of the Invention]

[0007] The above-described automated driving system makes it easy to recover from an unstable state in the work machine during automated operation. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an automated driving system 1, which includes a work machine 10 and an information processing device 40. [Figure 2] Figure 1 is a block diagram of the autonomous driving system 1. [Figure 3] Figure 2 shows the screen of the automatic driving mode, which is displayed in the display unit 42 shown in Figure 2. [Figure 4] Figure 3 shows the vehicle body lift detection screen, display image G. [Figure 5] Figure 3 shows the normal operation screen for display image G. [Figure 6] This is the command signal table used for the driving control unit C1C on the normal operation screen shown in Figure 5. [Figure 7] Figure 3 shows the driving operation screen for display image G. [Figure 8] Figure 3 shows the attachment operation screen for display image G. [Figure 9] Figure 8 is an explanatory diagram illustrating the selection of operation for attachment 15 on the attachment operation screen. [Figure 10] Figure 8 shows the attachment operation screen when the work machine 10, as shown in display image G, is in an unstable position. [Figure 11] This figure shows a modified version of the normal operation screen shown in Figure 5. [Modes for carrying out the invention]

[0009] The automated driving system 1 will be described with reference to the drawings.

[0010] As shown in Figure 1, the automatic driving system 1 is a system for the automatic driving of the work machine 10. The automatic driving system 1 is a system for restoring the work machine 10 when it has become unstable. The automatic driving system 1 comprises the work machine 10, an information processing device 40, and a detection unit 31 (see Figure 2).

[0011] The work machine 10 is a machine that performs work. The work machine 10 may be a construction machine that performs construction work, or a material handling machine that performs material handling work. The work machine 10 may be, for example, an excavator or a crane. The work machine 10 may be a bulldozer or a wheel loader. The following describes the case where the work machine 10 is an excavator. The work machine 10 is configured to be automatically operated. The work machine 10 may also operate in response to the operation of a worker (operator) without the use of automatic control. The work machine 10 may be operated by a worker (operator) in the driver's cab 13a (described later). The work machine 10 is configured to be remotely operated from outside the work machine 10. The number of work machines 10 included in the automatic driving system 1 may be one or multiple. The work machine 10 comprises a machine body 10a, an attachment 15, a drive control unit 17, and an actuator 21.

[0012] The machine body 10a is the main body of the work machine 10. The machine body 10a comprises a lower traveling body 11, a slewing device 12, and an upper slewing body 13. The lower traveling body 11 is capable of traveling on a traveling surface (such as the ground). The lower traveling body 11 may be equipped with crawlers or wheels. As shown in Figure 1, in this embodiment, the lower traveling body 11 comprises a traveling body base 11a, a left traveling body 11bL as one of the drive units, and a right traveling body 11bR as one of the drive units. The traveling body base 11a is a frame (structure) that supports the upper slewing body 13 via the slewing device 12 (described later). The traveling body base 11a connects the left traveling body 11bL and the right traveling body 11bR. The left traveling body 11bL is located on the left side of the lower traveling body 11. The right traveling body 11bR is located on the right side of the lower traveling body 11. The left travel body 11bL is capable of traveling in both forward and reverse directions (the same applies to the right travel body 11bR). The "forward side" of the left travel body 11bL is one side in the longitudinal direction of the left travel body 11bL (the same applies to the right travel body 11bR). The "reverse side" of the left travel body 11bL is the opposite side of the left travel body 11bL from the forward side (the same applies to the right travel body 11bR). The right travel body 11bR is attached to the travel body base 11a on the side opposite to the side to which the left travel body 11bL is attached. The slewing device 12 is a device (e.g., a slewing bearing) that supports the upper slewing body 13 so that it can slewing relative to the lower travel body 11. The upper slewing body 13 is slewingly supported by the lower travel body 11. The upper slewing body 13 includes an operator's cab 13a. The operator's cab 13a is the part from which an operator can operate the work machine 10. Furthermore, if the work machine 10 operates in response to the operator's operation, the work machine 10 may be operated (operated while on board) by the operator inside the driver's cab 13a, or it may be remotely operated from outside the work machine 10.

[0013] Attachment 15 is a work device attached to the machine body 10a. Attachment 15 is the work unit and comprises, for example, a boom 15a, an arm 15b, and a tip attachment 15c. The boom 15a is rotatably attached to the upper slewing body 13. The arm 15b is rotatably attached to the boom 15a.

[0014] The tip attachment 15c is provided at the tip of the attachment 15. The tip attachment 15c is rotatably attached to the arm 15b. The tip attachment 15c performs work on the work object. The tip attachment 15c may be a bucket capable of performing operations such as scooping up the work object and excavation. The tip attachment 15c may be provided with a device for clamping the work object (such as a grapple, nibbler, rotating fork, etc.), may be provided with a device for crushing the work object (such as a breaker, etc.), or may be provided with a magnet for adsorbing a metallic work object.

[0015] The work object is an object to be worked on by the work machine 10. The work object may be earth and sand, rock, a magnetic substance (such as metal), resin, waste, wood (such as a log), or a structure (such as a block). When the work object is earth and sand, the work object may be in a soil-like, granular, chipped, powdery, or other form.

[0016] Note that the work machine 10 may have a dozer as a working device for performing work. The dozer is a working device attached to the lower traveling body 11 (for example, the traveling body base 11a). The dozer includes, for example, a plate-like member (such as a spoil board) extending in the width direction and the vertical direction of the lower traveling body 11. The dozer may be movable in the vertical direction with respect to the lower traveling body 11.

[0017] The actuator 21 is a device for moving the work machine 10. The actuator 21 may include a hydraulic actuator that operates by hydraulic pressure, or may include an electric actuator that operates by electric power. The actuator 21 may include a motor that rotates and drives, or may include a cylinder (a telescopic cylinder) that expands and contracts and drives.

[0018] The actuator 21 includes a travel motor 26, a slewing motor 21a, a boom cylinder 21b, an arm cylinder 21c, and a tip attachment cylinder 21d. The travel motor 26 drives the lower travel body 11. For example, as shown in Figure 1, if the lower travel body 11 includes a left travel body 11bL and a right travel body 11bR, a motor is provided to drive the lower travel body 11 and a motor is provided to drive the right travel body 11bR. The travel motor 26 may be a hydraulic motor or an electric motor (the same applies to the slewing motor 21a). The slewing motor 21a rotates the upper slewing body 13 relative to the lower travel body 11. The boom cylinder 21b raises and lowers the boom 15a relative to the upper slewing body 13. The boom cylinder 21b is, for example, a hydraulic cylinder (the same applies to the arm cylinder 21c and the tip attachment cylinder 21d). The arm cylinder 21c rotates the arm 15b relative to the boom 15a. The tip attachment cylinder 21d rotates the tip attachment 15c relative to the arm 15b. If the tip attachment 15c itself is drivable, for example, as a device for gripping objects, an actuator 21 for driving the tip attachment 15c may be provided.

[0019] The drive control unit 17 (see Figure 2) controls the actuator 21 that moves the work machine 10. The drive control unit 17 may also include a hydraulic circuit to control a hydraulic actuator that operates using hydraulic pressure. The drive control unit 17 may also include an electrical circuit to control an electric actuator that operates using electric power.

[0020] The drive control unit 17 (see Figure 2) controls the travel motor 26 that drives the lower travel body 11. The drive control unit 17 controls the slewing motor 21a that slewing the upper slewing body 13 relative to the lower travel body 11. The drive control unit 17 controls the boom cylinder 21b that rotates (raises and lowers) the boom 15a relative to the upper slewing body 13. The drive control unit 17 controls the arm cylinder 21c that rotates the arm 15b relative to the boom 15a. The drive control unit 17 controls the tip attachment cylinder 21d that rotates the tip attachment 15c relative to the arm 15b.

[0021] The detection unit 31 (see Figure 2) detects various states. Part or all of the detection unit 31 may be mounted on the work machine 10 or located outside the work machine 10. The same applies to the information processing device 40, which will be described later, which may be mounted on the work machine 10 or located outside the work machine 10.

[0022] As shown in Figure 2, the detection unit 31 includes a position detection unit 311, a direction detection unit 312, an imaging device 313, and an attitude detection unit 315. The position detection unit 311 detects the position of the object to be measured. The position detection unit 311 detects the position of a specific part of the work machine 10 (see Figure 1). For example, the position detection unit 311 may detect the position of a specific part of the upper rotating body 13 (see Figure 1), or the position of a specific part of the attachment 15 (see Figure 1). The position detection unit 311 may include a device that detects position using electromagnetic waves (light, radio waves, etc.). The position detection unit 311 may include a device that uses a satellite positioning system, for example, a device that uses GNSS (Global Navigation Satellite System). The position detection unit 311 may include a device that detects position without using satellites, a device that detects position using a ground transmitter and receiver, or a device that detects position using the reflection of light (e.g., laser light) (e.g., a total station). The position detection unit 311 may calculate the position of the object to be measured based on position information detected by multiple types of devices.

[0023] The direction detection unit 312 detects the direction (orientation, posture) of the object to be measured. The direction detection unit 312 detects the direction of a specific part of the work machine 10 (see Figure 1). For example, the direction detection unit 312 may detect the direction of a specific part of the upper rotating body 13, or it may detect the direction of a specific part of the attachment 15 (see Figure 1). The direction detection unit 312 may be equipped with a device that uses the Earth's magnetic field to detect the orientation of the object to be measured. The direction detection unit 312 may also detect the direction of the object to be measured based on the positions of multiple parts of the object to be measured relative to the work site (for example, positions detected by the position detection unit 311).

[0024] The imaging device 313 images the object to be imaged. The imaging device 313 may image part or all of the work machine 10 (see Figure 1), or it may image objects around the work machine 10. The imaging device 313 may detect two-dimensional information (two-dimensional image), or it may detect three-dimensional information (three-dimensional image, distance image) that includes depth information. The imaging device 313 may be passive or active. Specifically, the imaging device 313 may be equipped with a camera (monocular camera) that detects two-dimensional information. The imaging device 313 may be equipped with a stereo camera that detects three-dimensional information. The imaging device 313 may detect three-dimensional information of the object to be imaged by irradiating the object with waves such as electromagnetic waves and detecting the reflected waves. The imaging device 313 may be equipped with a TOF (Time Of Flight) sensor that detects distance based on the time from wave irradiation to the return of the reflected wave, or it may be equipped with a sensor that detects distance based on the frequency of the reflected wave. The imaging device 313 may include a device for detecting three-dimensional information using light (e.g., laser light), such as LiDAR (Light Detection and Ranging). The imaging device 313 may also include a device for detecting three-dimensional information using radio waves (e.g., millimeter-wave radar).

[0025] The imaging device 313 may be provided as a single unit or as a group. If multiple imaging devices 313 are provided, the types of imaging devices 313 (type of imaging method, whether two-dimensional or three-dimensional, etc.) may be the same or different. The imaging device 313 may detect information about the object to be imaged (e.g., three-dimensional information) by combining multiple types of information (e.g., two-dimensional information and three-dimensional information).

[0026] The posture detection unit 315 detects the posture of the work machine 10 (see Figure 1). The posture detection unit 315 may also detect the position and orientation of the work machine 10 relative to the work site. The posture detection unit 315 may also detect the position and orientation of a reference position of the work machine 10 relative to the work site. The reference position of the work machine 10 may be, for example, a specific position of the upper slewing body 13 (see Figure 1) or the lower traveling body 11 (see Figure 1). The reference position of the work machine 10 may be the attachment point (boom foot) of the boom 15a (see Figure 1) to the upper slewing body 13, or a specific position on the pivot axis of the upper slewing body 13 relative to the lower traveling body 11. The posture detection unit 315 may also detect the inclination of the work machine 10 with respect to the horizontal plane. The posture detection unit 315 may also detect information (angle, angular velocity, angular acceleration, etc.) of the slewing of the upper slewing body 13 relative to the lower traveling body 11. The attitude detection unit 315 may detect information about the rotation of the boom 15a relative to the upper slewing body 13 (angle, angular velocity, angular acceleration, etc.). The attitude detection unit 315 may also detect information about the rotation of the arm 15b (see Figure 1) relative to the boom 15a. The attitude detection unit 315 may also detect information about the rotation of the tip attachment 15c (see Figure 1) relative to the arm 15b.

[0027] The information processing device 40 is a device that processes information related to the work machine 10 (see Figure 1). The information processing device 40 is a device that processes information related to the automatic operation of the work machine 10. The information processing device 40 has a function to remotely control the work machine 10 which is capable of automatic operation. The information processing device 40 may also have a function to support the management of the work machine 10 which is capable of automatic operation. The information processing device 40 is connected to the work machine 10 in a communicative manner. Hereinafter, a communicative connection will also be simply referred to as "connection". The information processing device 40 may be equipped with various information processing devices. The functions of the information processing device 40 may be distributed and arranged across multiple information processing devices (a distributed system may be configured). Specifically, for example, the information processing device 40 includes a server 40a (computer) and an information terminal 40b. The server 40a and the information terminal 40b may be connected by wireless communication or by wired communication. For example, communication may be performed by means of communication such as a mobile phone line, optical fiber line, wireless LAN (Local Area Network), or wired LAN.

[0028] For example, server 40a is a computer (controller) that performs information input / output, calculations (processing), and information storage. For example, the functions of server 40a are realized by the execution of a program stored in the storage unit of server 40a by the calculation unit. Server 40a is located outside the work machine 10 (see Figure 1) and the information terminal 40b. Server 40a is connected to the work machine 10. Information terminal 40b is connected to the work machine 10. Information terminal 40b may be connected to the work machine 10 via server 40a, or it may be connected to the work machine 10 without going through server 40a.

[0029] The information terminal 40b is a computer capable of inputting and outputting information. The information terminal 40b may or may not be portable. As shown in Figure 1, in this embodiment, the information terminal 40b is a tablet, but it may also be a smartphone or a personal computer. The information terminal 40b may also be used in conjunction with the server 40a. The information terminal 40b may be located outside the work machine 10 (see Figure 1) or inside the work machine 10 (for example, it may be brought in).

[0030] As shown in Figure 2, the information processing device 40 includes an input unit 41, a display unit 42, and a control unit 43. The input unit 41 is for inputting information (input device). The input unit 41 accepts operations from the operator. For example, the input unit 41 is operated by the operator and outputs a signal corresponding to the operation. The input unit 41 outputs information to the control unit 43. In this embodiment, the input unit 41 is a touch panel provided on the information terminal 40b (see Figure 1). The input unit 41 may be, for example, a mouse, a keyboard, a device that inputs information based on the operator's gaze information, or a device that inputs information based on the position information of the operator's fingers in space. The input unit 41 may also be equipped with a device for voice input (specifically a microphone). The input unit 41 may be provided on a smartphone or on a personal computer. The input unit 41 receives information related to automatic driving in response to the operator's operations. The input unit 41 receives information for remotely controlling the work machine 10 (see Figure 1) in response to the operator's operations.

[0031] The display unit 42 is a device that outputs information. The display unit 42 displays information related to the work machine 10 (see Figure 1). The display unit 42 displays information related to automatic operation. The display unit 42 outputs information based on signals output from the control unit 43. As shown in Figure 1, in this embodiment, the display unit 42 is a monitor (display) provided on the information terminal 40b, but it may be provided on a smartphone or a personal computer. The display unit 42 may be equipped with a projection device that projects onto an object such as the ground. The display unit 42 may be equipped with a light-emitting device (light). The display unit 42 may be equipped with a device that utilizes VR (Virtual Reality) technology (VR device) or a device that utilizes AR (Augmented Reality) technology (AR device). The display unit 42 may change at least one of the hue, density (transparency), brightness, and saturation of the light it outputs. The image displayed by the display unit 42 (the image that the control unit 43 causes the display unit 42 to display) is called the display image G. The display image G includes shapes, patterns, colors, characters, symbols, etc. The display image G includes a GUI (Graphical User Interface). The information terminal 40b may also be equipped with parts that perform output other than display (output units), such as an audio output unit and a vibration output unit.

[0032] The control unit 43 is a computer (controller) that performs signal input / output, calculations (processing), and information storage. For example, the functions of the control unit 43 are realized by the execution of a program stored in the memory unit of the control unit 43 by the calculation unit. The control unit 43 controls information related to the display unit 42, for example, causing the display unit 42 to display information. The control unit 43 controls information related to the input unit 41, for example, processing the information input to the input unit 41. The control unit 43 may also perform other controls, such as information communication control. The control unit 43 may be provided on, for example, the server 40a, or on the information terminal 40b, or its functions may be distributed between the server 40a and the information terminal 40b. The control unit 43 includes an automatic driving control unit 431, a remote control control unit 432, a mode switching control unit 433, and an ambient recognition unit 434.

[0033] The automatic driving control unit 431 causes the display unit 42 to display information related to automatic driving. The automatic driving control unit 431 causes the input unit 41 to accept input related to automatic driving. An example of the display on the display unit 42 in automatic driving mode, which displays information for automatic driving processing, will be described later. The remote control control unit 432 remotely operates the work machine 10 (see Figure 1) based on the operation received by the input unit 41. The mode switching control unit 433 switches between automatic driving mode and remote control mode. An example of the display on the display unit 42 in remote control mode, which displays information for remote control processing, will be described later. The surrounding recognition unit 434 acquires the work machine 10, which is to be operated automatically or remotely, and the surrounding conditions of the work machine 10, based on the information acquired from the detection unit 31. When acquiring the work machine 10 and the surrounding conditions, the surrounding recognition unit 434 may combine the information from the detection unit 31 with map information of the work site where the work machine 10 is located.

[0034] Furthermore, the operating performance of the work machine 10 (see Figure 1) may be limited when remotely operated compared to when it is in automatic operation mode. For example, even when the same command signal is received, the operating speed of the work machine 10 may be set slower in remote operation mode than in automatic operation mode. Also, for example, an upper limit may be set on the operating speed in remote operation mode. For example, the operation of the work machine 10 when remotely operated may be slower than the operation of the work machine 10 when operated by the operating lever of the operating device in the operator's cab 13a or an operating device that mimics the operating device in the operator's cab 13a.

[0035] (Automatic operation of work machine 10) The work machine 10 (see Figure 1) is configured to perform automatic operation based on a work plan. The work plan is information regarding the objectives of the work machine 10's operation and is pre-set in the information processing device 40 (before automatic operation). The work plan may also be set by the automatic operation control unit 431. The work plan may include information on the target route for the work machine 10's travel. The work plan may also include information on the target range (e.g., target acquisition range, target release range) in which the tip attachment 15c (see Figure 1) will perform its work.

[0036] (Target path, target trajectory) The work plan may include information on the target path (also called the target trajectory) of a specific part of attachment 15 (see Figure 1). The specific part of attachment 15 may be, for example, the tip of the tip attachment 15c (see Figure 1), or the base end of the tip attachment 15c (the tip of the arm 15b (see Figure 1)). The specific part of attachment 15 may be set in only one location, or in multiple locations. The "target path" above is information that includes, for example, information on the positions (coordinates) of multiple target points and information on the order of each target point. The work plan may also include information on the target trajectory of a specific part of attachment 15. The "target trajectory" above is information that adds time information to the target path information.

[0037] (Cycle, work phase) The work plan may include information for having the work machine 10 (see Figure 1) perform a certain task (e.g., loading, lifting, etc.) in multiple cycles. One cycle may include multiple work phases. A specific example of a work phase is as follows. Here, a specific example of a work phase is described when the work machine 10 automatically performs the task of capturing a work object and moving the captured work object to a predetermined position. In this case, the work phases include a capture phase (e.g., excavation phase), a lifting and rotating phase (soil removal and rotating phase), a release phase (soil removal phase), and a return and rotating phase. The capture phase is the phase in which the tip attachment 15c shown in Figure 1 captures the work object within the target capture range (e.g., excavating soil). For example, the "target capture range" is set to a place where the work objects are collected (e.g., a pile of soil). The lifting and turning phase is the phase in which the tip attachment 15c moves from the target acquisition range to the target release range while the tip attachment 15c has acquired the work object. The release phase is the phase in which the tip attachment 15c releases the work object (e.g., excavates soil) within the target release range. The above "target release range" is set to a predetermined range, for example, on the bed of a transport vehicle (such as a dump truck). The return and turning phase is the phase in which the tip attachment 15c moves from the target release range to the target acquisition range. For example, in automated driving, the acquisition phase, lifting and turning phase, release phase, and return and turning phase are repeated in a series of work phases (one cycle of automated driving work).

[0038] (teaching) At least a portion of the work plan may be set by teaching, or by other methods (e.g., numerical input). Information set by teaching is called teaching data. Teaching data is set by an operator operating the work machine 10 (see Figure 1). Specifically, for example, the operator may be on board the work machine 10 and operate it, or the operator may remotely operate the work machine 10. For example, by operating the work machine 10, the operator places a specific part of the attachment 15 (see Figure 1) at a location (path, range, etc.) that they want to set as teaching data. Then, based on the position where the specific part of the attachment 15 is placed, the teaching data is set in the information processing device 40. For example, by operating the work machine 10, the operator places a specific part of the attachment 15 at a specific position within a range that they want to set as a target range (target acquisition range or target release range). The position where a specific part of the attachment 15 is positioned is calculated based on the detection result of the posture detection unit 315, which detects the posture of the work machine 10. Then, the target range is set based on the position where the specific part of the attachment 15 is positioned. For example, the operator moves the specific part of the attachment 15 along the path that they want to set as the target path by operating the work machine 10. For example, the operator moves the specific part of the attachment 15 along the path that they want to set as the target trajectory at the speed they want to set as the target trajectory by operating the work machine 10. Then, the information processing device 40 sets the path (trajectory) along which the specific part of the attachment 15 has moved as the target path (trajectory).

[0039] (Operation) The following describes the operation of the information processing device 40, the automatic driving information processing method, and the automatic driving information processing program. The following mainly describes the display image G that the control unit 43 displays on the display unit 42. The display image G shown in Figure 3 changes in various ways depending on the situation. The display image G comprises various parts (elements such as various selection units, various display units, and various setting units). The presence or absence of each part of the display image G, the manner of display, and whether or not it can be selected or operated by the input unit 41 (active or inactive) all change in various ways depending on the situation.

[0040] (Automatic driving mode) First, let's explain the display image G that the control unit 43 (specifically, the automatic operation control unit 431) displays on the display unit 42 in automatic operation mode. Figure 3 is a diagram showing an example of the display image G in automatic operation mode when the work machine 10 (see Figure 1) is in automatic operation mode. In automatic operation mode, the display image G displays information for performing automatic operation processing. For example, in automatic operation mode, the display image G displays a screen that allows the operator to set up automatic operation. Specifically, for example, in automatic operation mode, the display image G displays a screen that allows the operator to select the type of work to be performed in automatic operation mode. Also specifically, for example, in automatic operation mode, the display image G displays a screen that allows the operator to check and modify the settings set for automatic operation. Also specifically, for example, in automatic operation mode, the display image G displays a screen that allows the operator to start, interrupt, and end automatic operation. Also specifically, for example, in automatic operation mode, the display image G displays a screen that shows the status of the work machine 10 during automatic operation. Furthermore, specifically, for example, in the automatic driving mode, display image G shows a screen indicating the work results of the automatic operation performed by the work machine 10.

[0041] In such a screen, the display image G shows the mode switching unit A. In automatic driving mode, the mode switching unit A is the part that allows the operator to select whether to switch from automatic driving mode to remote control mode. As shown in the example in Figure 3, in automatic driving mode, the mode switching unit A may be marked with something (for example, text) to indicate that it is switching to remote control mode. For example, when the operator selects the mode switching unit A, the control unit 43 (specifically the mode switching control unit 433) switches the user interface of the display unit 42 from automatic driving mode to remote control mode. Thus, in the display image G, the mode switching unit A is a switching reception area for the input unit 41 to accept the switching between automatic driving mode and remote control mode.

[0042] This "selection" is a selection made by the operator operating the input unit 41. Specifically, for example, if the input unit 41 is a touch panel, the selection is made by touching the type of task to be performed in automatic operation. If the input unit 41 is a mouse or keyboard, the selection may also be made by placing the cursor over the type of task to be performed in automatic operation and then performing an operation to confirm the selection (click, pressing a designated key, etc.). Alternatively, the selection may also be made by performing a specific mouse operation or keyboard operation (shortcut key operation, etc.) to select the type of task to be performed in automatic operation. Hereafter, the selection made by the operator operating the input unit 41 will also be simply referred to as "selection" (the same applies to "selection" and "setting" in parts other than the mode switching unit A).

[0043] Figure 4 shows the vehicle lift detection screen in automatic driving mode. The vehicle lift detection screen is displayed when vehicle lift, which is an unstable posture for the work machine 10 (see Figure 1), is detected and an emergency stop is performed. For example, if the control unit 43 (specifically the surrounding recognition unit 434) determines that vehicle lift has occurred, the vehicle lift detection screen is displayed on the display unit 42. For example, the conditions for determining that vehicle lift has occurred include the tilt of the work machine 10 relative to the ground detected by the posture detection unit 315 being greater than or equal to a predetermined value. The ground may have coordinate axes on a horizontal plane, coordinates from pre-set map information may be used, or coordinates may be obtained from three-dimensional information of the ground detected by the imaging device 313.

[0044] The vehicle lift detection screen has a vehicle lift notification unit B. The vehicle lift notification unit B is the part that notifies the operator of vehicle lift of the work machine 10 (see Figure 1). The vehicle lift notification unit B has a mode switching unit B1. The mode switching unit B1, like the mode switching unit A, is the part that allows the operator to select to switch from automatic driving mode to remote control mode. As shown in the example in Figure 3, in automatic driving mode, the mode switching unit B1 may be marked with something (e.g., text) to indicate that it is switching to remote control mode. For example, when the operator selects the mode switching unit B1, the control unit 43 (specifically the mode switching control unit 433) switches the user interface of the display unit 42 from automatic driving mode to remote control mode. In this way, the mode switching unit B1 is a switching reception area in the display image G for the input unit 41 to accept the switching between automatic driving mode and remote control mode. In this way, the control unit 43 detects and notifies that the posture of the work machine 10 is unstable in automatic driving mode. Furthermore, if the control unit 43 detects that the posture of the work machine 10 is unstable in automatic driving mode, it presents the mode switching unit B1 as a switching acceptance area. Note that the unstable posture is not limited to the vehicle body floating.

[0045] (Remote control mode) The display image G in remote control mode will now be described. Remote control mode is presented when switched from automatic driving mode. Remote control mode is a mode in which the input unit 41 and the display unit 42 present an interface for directly operating the work machine 10 (see Figure 1). In remote control mode, three operation screens are displayed in sequence: a normal operation screen, a driving operation screen, and an attachment operation screen. In remote control mode, the work machine 10 to be operated may be automatically selected from the work machine 10 selected in automatic driving mode, or it may be selected by the operator when switching to remote control mode. When switching to remote control mode by the mode switching unit B1, it is preferable that the work machine 10 to be operated is automatically selected from the work machine 10 selected in automatic driving mode. In remote control mode, the control unit 43 (specifically, the remote control control unit 432) operates the work machine 10 in response to the operator's operation received by the input unit 41.

[0046] Furthermore, the working status of the work machine 10 may be displayed at the time of switching from automatic driving mode to remote control mode. The working status may only be displayed if the work machine 10 has made an emergency stop and has been switched to remote control mode. The working status of the work machine 10 indicates the progress of automatic driving by the work machine 10 up to the above timing. For example, the type of work that the work machine 10 is performing (specifically, loading work, etc.) may be displayed as the working status. Also, for example, the number of completed work phases (number of cycles) may be displayed as the working status. Also, for example, the number of times a predetermined phase has been performed may be displayed as the working status. More specifically, the number of release phases performed (for example, the number of times soil has been loaded onto the cargo bed of the transport vehicle, etc.) may be displayed as the working status. Also, for example, the amount of work completed may be displayed as the working status. More specifically, the total amount of work objects released into the target release range (for example, the total amount of soil loaded onto the cargo bed of the transport vehicle, etc.) may be displayed as the working status. The work status may be displayed at the time when it is possible to switch to remote control mode. For example, the work status may be displayed on the vehicle body lifting notification unit B (see Figure 4) on the vehicle body lifting detection screen.

[0047] Figure 5 shows an example of a normal operation screen in remote control mode. The normal operation screen includes a left lever operation unit C1L, a right lever operation unit C1R, a travel operation unit C1C, a mode switching unit C11, and an operation screen switching unit C12. The left lever operation unit C1L is located on the left side of the normal operation screen. The right lever operation unit C1R is located on the right side of the normal operation screen. The travel operation unit C1C is located between the left lever operation unit C1L and the right lever operation unit C1R. The left lever operation unit C1L and the right lever operation unit C1R are the parts that allow the operator to select the operation of the work machine 10 (see Figure 1). Specifically, the operations performed are the rotation of the upper slewing body 13 (see Figure 1) of the work machine 10, the operation of the boom 15a (see Figure 1), the operation of the arm 15b (see Figure 1), and the tip attachment 15c (see Figure 1).

[0048] For example, as shown in Figure 5, the left lever operating section C1L has the upper part of the left lever C1L1, the lower part of the left lever C1L2, the left side of the left lever C1L3, and the right side of the left lever C1L4. Specifically, the upper part of the left lever C1L1 is the part that allows the operator to select the operation to rotate the arm 15b (see Figure 1) in a direction that brings it closer to the boom 15a (see Figure 1). The lower part of the left lever C1L2 is the part that allows the operator to select the operation to rotate the arm 15b in a direction that moves it away from the boom 15a. The left side of the left lever C1L3 is the part that allows the operator to select the operation to rotate the upper slewing body 13 (see Figure 1) to the left when viewed from the operator's cab 13a (see Figure 1) towards the boom 15a. The right side of the left lever C1L4 is the part that allows the operator to select the operation to rotate the upper slewing body 13 to the right when viewed from the operator's cab 13a towards the boom 15a.

[0049] Furthermore, as shown in Figure 5, for example, the right lever operating section C1R has the upper part C1R1 of the right lever, the lower part C1R2 of the right lever, the left part C1R3 of the right lever, and the right part C1R4 of the right lever. Specifically, for example, the upper part C1R1 of the right lever is the part that allows the operator to select an operation to rotate the boom 15a (see Figure 1) in a direction that brings it closer to the upper slewing body 13 (see Figure 1). For example, the lower part C1R2 of the right lever is the part that allows the operator to select an operation to rotate the boom 15a in a direction that moves it away from the upper slewing body 13. For example, the left part C1R3 of the right lever is the part that allows the operator to select an operation to rotate the tip attachment 15c (see Figure 1) in a direction that brings it closer to the arm 15b (see Figure 1). For example, the right part C1R4 of the right lever is the part that allows the operator to select an operation to rotate the tip attachment 15c in a direction that moves it away from the arm 15b. The correspondence between the up, down, left, and right parts of the left lever operating section C1L and the right lever operating section C1R and their respective operations may be changed in various ways.

[0050] Furthermore, the travel control unit C1C is the part that allows the operator to select whether to drive the left travel body 11bL (see Figure 1) or the right travel body 11bR (see Figure 1). The travel control unit C1C is the part that allows the operator to select whether to output a composite operation signal in a single operation, which includes a signal for controlling the drive of the left travel body 11bL and a signal for controlling the drive of the right travel body 11bR. For example, as shown in Figure 5, the travel control unit C1C has a travel control upper part C1C1, a travel control lower part C1C2, a travel control left part C1C3, a travel control right part C1C4, a travel control upper left part C1C5, a travel control upper right part C1C6, a travel control lower left part C1C7, and a travel control lower right part C1C8.

[0051] Figure 6 is a command signal table showing the command signals output when the travel control unit C1C is selected. For example, as shown in Figure 6, each of the control units C1C1 to C1C8 of the travel control unit C1C is assigned a command signal for controlling the drive of the left travel body 11bL (see Figure 1) and the right travel body 11bR (see Figure 1). That is, when an operator selects each of the control units C1C1 to C1C8, the command signals shown in the example in Figure 6 are transmitted to the travel motors 26 (see Figure 1) that control the left travel body 11bL and the right travel body 11bR, respectively.

[0052] For example, as shown in Figure 6, the upper travel control section C1C1 allows the operator to select between moving the left travel body 11bL (see Figure 1) forward and moving the right travel body 11bR (see Figure 1) forward. As a result, the lower travel body 11 (see Figure 1) moves the work machine 10 (see Figure 1) forward. The lower travel control section C1C2 allows the operator to select between moving the left travel body 11bL backward and moving the right travel body 11bR backward. As a result, the lower travel body 11 moves the work machine 10 backward. The left travel control section C1C3 allows the operator to select between moving the left travel body 11bL backward and moving the right travel body 11bR forward. As a result, the lower travel body 11 makes the work machine 10 spin turn to the left. The right travel control section C1C4 allows the operator to select between moving the left travel body 11bL backward and moving the right travel body 11bR forward. As a result, the lower travel unit 11 causes the work machine 10 to spin turn to the right. Also, the upper left travel control points C1 and C5 allow the operator to choose between stopping the left travel unit 11bL and moving the right travel unit 11bR forward. As a result, the lower travel unit 11 causes the work machine 10 to pivot turn to the left around the left travel unit 11bL as the axis. Also, the upper right travel control points C1 and C6 allow the operator to choose between moving the left travel unit 11bL forward and stopping the right travel unit 11bR. As a result, the lower travel unit 11 causes the work machine 10 to pivot turn to the right around the right travel unit 11bR as the axis. Also, the lower left travel control points C1 and C7 allow the operator to choose between moving the left travel unit 11bL backward and stopping the right travel unit 11bR. As a result, the lower travel unit 11 causes the work machine 10 to pivot turn to the left around the right travel unit 11bR as the axis. Furthermore, the lower right travel control C1C8 is the part that allows the operator to choose between stopping the left travel body 11bL and reversing the right travel body 11bR. As a result, the lower travel body 11 pivots the work machine 10 to the right around the left travel body 11bL as the axis. Thus, the work machine 10 is equipped with a left travel body 11bL and a right travel body 11bR, which are driven according to command signals. The control unit 43 then outputs command signals to the work machine 10 for the left travel body 11bL and the right travel body 11bR based on a single operation to the travel control unit C1C received by the input unit 41.

[0053] The mode switching unit C11 is the part that allows the operator to select whether to switch from remote control mode to automatic operation mode. As shown in the example in Figure 6, the mode switching unit C11 may be marked with something (for example, text) to indicate that it is switching to automatic operation mode. For example, when the operator selects the mode switching unit C11, the control unit 43 (specifically the mode switching control unit 433) switches the user interface of the display unit 42 from remote control mode to automatic operation mode. In this way, the mode switching unit C11 is a switching reception area in the display image G for the input unit 41 to accept the switching between automatic operation mode and remote control mode.

[0054] The operation screen switching unit C12 is the part that allows the operator to select which operation screen to switch. For example, if the operation screen switching unit C12 is selected on the normal operation screen, the operation screen will switch to the driving operation screen.

[0055] Figure 7 shows an example of a screen for driving operations in remote control mode. The normal operation screen has a driving operation unit C1C, a mode switching unit C11, an operation screen switching unit C12, and a work machine top display unit C13. The driving operation unit C1C and the mode switching unit C11 are the same as those on the normal operation screen. For example, if the operation screen switching unit C12 is selected on the driving operation screen, the operation screen switches to the attachment operation screen.

[0056] The top display section C13 of the work machine is a part for displaying the rotation angle of the upper slewing body 13 (see Figure 1) relative to the lower traveling body 11 (see Figure 1) of the work machine 10 (see Figure 1). For example, as shown in Figure 7, the top display section C13 of the work machine may be a diagram simulating the work machine 10. Specifically, the top display section C13 of the work machine has a lower traveling body section C131 that simulates the lower traveling body 11 and an upper slewing body section C132 that simulates the upper slewing body 13. The top display section C13 of the work machine simulates a top view of the work machine 10 and shows the rotation position of the upper slewing body 13 relative to the lower traveling body section C131. Thus, the top display section C13 of the work machine shows the rotation angle of the upper slewing body 13 relative to the lower traveling body 11. The rotation angle of the upper slewing body 13 relative to the lower traveling body 11 may also be shown numerically. Furthermore, the image representing the slewing angle may be a diagram other than a top view of the work machine 10. For example, the image representing the slewing angle may be one that shows the slewing angle as a percentage of data in a pie chart.

[0057] The top display unit C13 of the work machine may also be a part that allows the operator to select the rotation of the upper rotating body 13 (see Figure 1) of the work machine 10 (see Figure 1). For example, the position in which the upper rotating body 13 rotates may be determined by dragging the upper rotating body unit C132.

[0058] Figure 8 shows an example of an attachment operation screen in remote control mode. The attachment operation screen includes a left lever operation section C1L, a right lever operation section C1R, a mode switching section C11, an operation screen switching section C12, and a work machine side display section C14. The left lever operation section C1L, the right lever operation section C1R, and the mode switching section C11 are the same as those on the normal operation screen. For example, if the operation screen switching section C12 is selected on the attachment operation screen, the operation screen switches to the attachment operation screen.

[0059] The side display unit C14 of the work machine is for displaying the current posture of the work machine 10 (see Figure 1). The side display unit C14 of the work machine also allows the operator to select the operation of the attachment 15 (see Figure 1) of the work machine 10. This is a diagram simulating the side display unit C14 of the work machine and the work machine 10. The side display unit C14 of the work machine simulates a side view of the work machine 10. Specifically, the top display unit C13 of the work machine has a machine body section C141, an arm section C143, and a tip attachment section C144. The machine body section C141 simulates the machine body 10a. The boom section C142 simulates the boom 15a (see Figure 1). The arm section C143 simulates the arm 15b (see Figure 1). The tip attachment section C144 simulates the tip attachment 15c (see Figure 1).

[0060] Referring to Figure 9, a specific example of selecting the operation of attachment 15 (see Figure 1) is shown below. In the example in Figure 9, an example of operating the arm 15b of attachment 15 (see Figure 1) is described. First, the control unit 43 allows the operator to select the component of attachment 15 to be operated. For example, the components of attachment 15 that the operator is allowed to select are the boom section C142, the arm section C143, and the tip attachment section C144. If any of the boom section C142, the arm section C143, or the tip attachment section C144 is selected, the control unit 43 identifies it as the component to be operated. At this time, the control unit 43 may also use the condition that the selection is a specific operation to identify the component to be operated. For example, the control unit 43 may identify the component to be operated when any of the boom section C142, the arm section C143, or the tip attachment section C144 is pressed and held. The specific operation is not limited to this and may also be a drag operation, for example. The requirement that the work machine 10 (see Figure 1) be remotely controlled only when the input unit 41 accepts a specific operation may also apply to the left lever operation unit C1L, the right lever operation unit C1R, the travel operation unit C1C, and the work machine top display unit C13.

[0061] Once the components of attachment 15 (see Figure 1) are identified, the identified components may be highlighted. The highlighting of identified components may be indicated by shape, pattern, color, letters, symbols, etc. For example, in the example in Figure 9, the color of the identified arm section C143 area is changed to be different from the other components. The control unit 43 then allows the operator to select the destination of the component. For example, the destination of the component is selected by dragging the component identified by touch operation. For example, in the example in Figure 9, the arm section C143 is displayed in a rotated state by drag operation. Specifically, the arm section C143 is displayed in a rotated state relative to the boom section C142 to which the arm section C143 is attached. At this time, the tip attachment section C144 is displayed in a rotated state together with the arm section C143 while maintaining its positional relationship with the arm section C143. For example, although not shown, if the boom section C142 is displayed in a rotated state, the boom section C142 will be displayed in a rotated state relative to the machine body section C141 to which the boom section C142 is attached. Then, the arm section C143 and the tip attachment section C144 are displayed rotating together with the boom section C142 while maintaining their relative position to the boom section C142. In this way, one of the components of the attachment 15 is identified in response to a touch operation received by the input section 41. Then, in response to a drag operation received by the input section 41 after the touch operation, the identified component is displayed rotating on the machine body section 141c or the component on the side to which the identified component is attached. The control unit 43 then remotely controls the work machine 10 (see Figure 1) by sending a command signal to the work machine 10 to move the identified component to its destination.

[0062] Figure 10 shows an example of a screen for attachment operation in remote control mode. As shown in Figure 10, for example, if the work machine 10 (see Figure 1) is in an unstable position, the control unit 43 may display the stable position section C15 on the display unit 42. The stable position section C15 is the part that displays the desired position of the work machine 10. As shown in Figure 10, it is preferable for the control unit 43 to display the stable position section C15 and the work machine side display section C14 superimposed. Also, as shown in Figure 10, it is preferable for the control unit 43 to display the stable position section C15 and the work machine side display section C14 in different ways. For example, the stable position section C15, which is the desired position, may be displayed in a transparent manner, and the work machine side display section C14 may be displayed in an opaque manner. The control unit 43 determines the desired position of the work machine 10 based on the tilt of the work machine 10 detected by the position detection unit 315 and the position of each component of the attachment 15 (see Figure 1).

[0063] Furthermore, if the working machine 10 (see Figure 1) is in an unstable position, the control unit 43 displays on the display unit 42 the necessary operations for the input unit 41 to bring the working machine 10 into a desirable position. The operations necessary to bring the working machine 10 into a desirable position may be indicated by shapes, patterns, colors, characters, symbols, etc. For example, in the example in Figure 10, the upper part C1R1 of the right lever, which allows the operator to select the operation to rotate the boom 15a (see Figure 1) in a direction that brings it closer to the upper slewing body 13 (see Figure 1), is changed to a different color from the other parts.

[0064] Furthermore, the control unit 43 allows remote operation if the working machine 10 (see Figure 1) is in an unstable position and the input unit 41 receives an operation necessary to bring the working machine 10 into a desirable position. For example, in the example shown in Figure 10, only operation on the upper part C1R1 of the right lever, which is indicated as an operation necessary to bring the working machine 10 into a desirable position, is permitted.

[0065] Each of the above operations (display, selection, setting, etc.) may also be referred to as a "step" in the method and program. Specifically, for example, selection may be referred to as a selection step. More specifically, the remote operation of the work machine 10 (see Figure 1) may be referred to as a "remote operation step."

[0066] (Effects of the first invention) The effects of the autonomous driving system 1 shown in Figure 1 are as follows:

[0067] [Configuration 1] The automatic driving system 1 comprises a work machine 10 and an information processing device 40. The work machine 10 is capable of automatic driving. The information processing device 40 is connected to the work machine 10 in a communicative manner. The information processing device 40 comprises a display unit 42 (see Figure 2), an input unit 41 (see Figure 2), and a control unit 43 (see Figure 2). The display unit 42 displays information related to the work machine 10. The input unit 41 receives operations from the operator. The control unit 43 controls the display image G to be displayed on the display unit 42 and the information related to the input of the input unit 41. The control unit 43 performs automatic driving processing and remote operation processing. In the automatic driving processing, information related to automatic driving is displayed on the display unit 42. In the automatic driving processing, the input unit 41 receives input related to automatic driving. In the remote operation processing, the work machine 10 is remotely operated based on the operations received by the input unit 41.

[0068] With the above configuration [Configuration 1], the operator can remotely control the work machine 10 using the input unit 41 (see Figure 2) which receives input information related to autonomous driving. For example, if the work machine 10 becomes unstable during autonomous driving, it can be easily restored to working order.

[0069] (Effects of the second invention) [Configuration 2] In the automatic driving system 1, the control unit 43 (see Figure 2) switches between an automatic driving mode that displays information for performing automatic driving processing and a remote control mode that displays information for performing remote control processing.

[0070] With the above [Configuration 2], it is possible to switch between displaying the automated driving process and the remote control process.

[0071] (Effects of the third invention) [Configuration 3] As shown in Figure 1, in the automatic driving system 1, the display unit 42 is a display. The control unit 43 (see Figure 2) displays the mode switching unit A on the display unit 42 as a switching reception area for the input unit 41 to accept switching between the automatic driving mode and the remote control mode.

[0072] With the above [Configuration 3], the operator can switch between displaying the automated driving process and the remote control process.

[0073] (Effects of the fourth invention) [Configuration 4] As shown in Figure 1, in the automatic driving system 1, the work machine 10 comprises a left travel body 11bL and a right travel body 11bR. As shown in Figure 6, the left travel body 11bL and the right travel body 11bR are each driven according to command signals. The control unit 43 (see Figure 2) outputs multiple command signals to the work machine 10 based on a single operation received by the input unit 41 (see Figure 2).

[0074] With the above [Configuration 4], the operator can perform a complex operation with a single operation.

[0075] (Effects of the fifth invention) [Configuration 5] As shown in Figure 1, in the automatic driving system 1, the work machine 10 comprises an upper slewing body 13 and a lower traveling body 11 that supports the upper slewing body 13 so as to be rotatable. As shown in Figure 7, the control unit 43 (see Figure 2) displays the slewing angle of the upper slewing body 13 relative to the lower traveling body 11 on the display unit 42.

[0076] With the above configuration [5], the rotation angle of the upper rotating body 13 relative to the lower traveling body 11 can be displayed to the operator, reducing the possibility of misjudging the direction of travel.

[0077] (Effects of the sixth invention) [Configuration 6] As shown in Figure 8, in the automatic driving system 1, the control unit 43 (see Figure 2) displays the current posture of the work machine 10 (see Figure 1) on the work machine side display unit C14 on the display unit 42 (see Figure 2).

[0078] As described in [Configuration 6] above, the operator can easily grasp the orientation of the work machine 10 (see Figure 1) using the display unit 42 (see Figure 2).

[0079] (Effects of the seventh invention) [Configuration 7] As shown in Figure 10, in the automatic driving system 1, if the posture of the work machine 10 (see Figure 1) is unstable, the control unit 43 (see Figure 2) displays the stable posture section C15, which is the desired posture of the work machine 10, on the display unit 42 (see Figure 2).

[0080] With the above configuration [7], the operator can easily understand the current position and desired position of the work machine 10 using the display unit 42.

[0081] (Effects of the 8th Invention) [Configuration 8] As shown in Figure 10, in the automatic driving system 1, the control unit 43 (see Figure 2) displays the stable posture section C15, which is the desired posture, and the work machine side display section C14, which is the current posture of the work machine 10, superimposed on the display unit 42 (see Figure 2).

[0082] As described in [Configuration 8] above, the operator can easily grasp the desired posture state of the work machine 10 (see Figure 1) relative to its current posture using the display unit 42.

[0083] (Effects of the 9th Invention) [Configuration 9] As shown in Figure 10, in the automatic driving system 1, the control unit 43 (see Figure 2) displays the stable posture section C15, which is the desired posture, and the work machine side display section C14, which is the current posture, in different ways on the display unit 42 (see Figure 2).

[0084] With the above configuration [9], the operator can more easily grasp the desired position of the work machine 10 relative to its current position using the display unit 42.

[0085] (Effects of the 10th Invention) [Configuration 10] As shown in Figure 10, in the automatic driving system 1, if the posture of the work machine 10 (see Figure 1) is unstable, the control unit 43 (see Figure 2) displays on the display unit 42 (see Figure 2) the operations required for the input unit 41 (see Figure 2) to bring the work machine 10 into a stable posture unit C15, which is the desired posture.

[0086] In the above configuration

[10] , the operator can easily understand the operations required to move the work machine 10 from its current position to a desired position using the display unit 42.

[0087] (Effects of the 11th Invention) [Configuration 11] As shown in Figure 10, in the automatic driving system 1, the control unit 43 (see Figure 2) allows remote operation when the working machine 10 (see Figure 1) is in an unstable position and the input unit 41 (see Figure 2) receives the necessary operation to bring the working machine 10 into a stable position C15, which is a desirable position.

[0088] With the above configuration

[11] , the operator can input the necessary operations to transition the work machine 10 from its current position to a desired position via the display unit 42 (see Figure 2).

[0089] (Effects of the 12th Invention) [Configuration 12] In the autonomous driving system 1, the control unit 43 (see Figure 2) allows remote operation when the input unit 41 (see Figure 2) receives a specific operation (for example, a long press operation, a drag operation).

[0090] The above [Configuration 12] reduces the possibility of the system being remotely controlled due to operator error.

[0091] (Effects of the 13th Invention) [Configuration 13] As shown in Figure 4, in the automatic driving system 1, the control unit 43 (see Figure 2) detects and notifies that the posture of the work machine 10 (see Figure 1) is unstable in the automatic driving mode.

[0092] The above [Configuration 13] makes it possible to notify the worker that it is necessary to return the work machine 10 to a stable position.

[0093] (Effects of the 14th Invention) [Configuration 14] As shown in Figure 4, in the automatic driving system 1, if the control unit 43 (see Figure 2) detects that the posture of the work machine 10 (see Figure 1) is unstable in the automatic driving mode, it presents the mode switching unit B1 as a switching acceptance area.

[0094] With the above configuration

[14] , it is possible to guide the operator to a remote control mode that returns the work machine 10 to a stable position.

[0095] (Effects of the 15th Invention) [Configuration 15] In the automated driving system 1, the operational performance of the work machine 10 (see Figure 1) is limited when it is remotely controlled compared to when it is automated.

[0096] With the above configuration

[15] , the operator can make fine adjustments to the posture of the work machine 10 via remote control.

[0097] (Effects of the 16th Invention) [Configuration 16] As shown in Figure 1, in the automatic driving system 1, the work machine 10 comprises a machine body 10a and an attachment 15. The attachment 15 comprises a boom 15a, an arm 15b, and a tip attachment 15c as its components. The boom 15a is rotatably attached to the machine body 10a. The arm 15b is rotatably attached to the boom 15a. The tip attachment 15c is rotatably attached to the arm 15b. The display unit 42 is a display. The input unit 41 is a touch panel provided on the display. The control unit 43 (see Figure 2) identifies one of the components in response to a touch operation received by the input unit 41. In response to a drag operation received by the input unit 41 after the touch operation, the control unit 43 rotates and displays the identified component on the machine body 10a or component to which the identified component is attached. The control unit 43 remotely controls the identified component in response to the drag operation.

[0098] With the above configuration

[16] , the attachment 15 of the work machine 10 can be intuitively operated remotely.

[0099] (modified version) The above embodiments may be modified in various ways. For example, the number of components of the above embodiments (such as each part of the display image G) (including modified versions) may be changed, and some components may not be provided. For example, modified versions of the above embodiments may be combined in various ways. For example, the arrangement of components may be changed. For example, the inclusion relationships of components may be changed in various ways. For example, something described as a subordinate component included in a higher-level component may not be included in this higher-level component, but may be included in other components. For example, something described as multiple different members or parts may be treated as a single member or part. For example, something described as a single member or part may be divided and provided as multiple different members or parts. For example, the order of transitions of the display image G may be changed. For example, elements described as components of different screens may be displayed on the same screen. For example, each component may have only a part of each feature (function, arrangement, shape, operation, etc.).

[0100] For example, in remote control mode, the left lever operating section C1L, the right lever operating section C1R, and the travel operating section C1C may be in any configuration. Specifically, modified examples of the left lever operating section C1L, the right lever operating section C1R, and the travel operating section C1C will be described with reference to Figure 11.

[0101] For example, as shown in Figure 11, the normal operation drawing for remote control mode may have a left lever operation section C2L, a right lever operation section C2R, and a travel operation section C2C. The left lever operation section C2L simulates the left lever (not shown) mounted on the work machine 10 and is the part that allows the operator to select the operation of the left lever. The left lever operation section C2L has a lever section C2L1, a lever play section C2L2, and a lever operation range section C2L3. The lever section C2L1 is the part that allows the operator to select the operation of the left lever. When the selected state of the lever section C2L1 is released, it returns to the neutral position (center position). The lever operation range section C2L3 is the part that displays the operable range of the lever section C2L1. The lever play section C2L2 is part of the lever operation range section C2L3 and is the part that displays the range in which the work machine 10 is not operated. In other words, even if the operator operates the lever C2L1 within the range of the lever play C2L2, the work machine 10 will not be activated.

[0102] The right lever operating section C2R simulates the right lever (not shown) mounted on the work machine 10 and allows the operator to select which lever to operate. For example, as shown in Figure 11, the right lever operating section C2R has a lever section C2R1 similar to the left lever operating section C2L, a lever play section C2R2, and a lever operating range section C2R3.

[0103] The travel control unit C2C simulates the left and right travel pedals (not shown) mounted on the work machine 10 and is the part that allows the operator to select which travel pedal to operate. For example, as shown in Figure 11, the travel control unit C2C has a left pedal control unit C2CL and a right pedal control unit C2CR. The left pedal control unit C2CL is the part that allows the operator to select which travel pedal to operate. The left pedal control unit C2CL has a left pedal section C2CL1, a left pedal play section C2CL2, and a left pedal operating range section C2CL3. The left pedal section C2CL1 is the part that allows the operator to select which left pedal to operate. The left pedal operating range section C2CL3 is the part that displays the operable range of the left pedal section C2CL1. The left pedal play section C2CL2 is part of the left pedal operating range section C2CL3 and is the part that displays the range in which the work machine 10 is not operated. In other words, even if the operator operates the left pedal C2CL1 within the range of the left pedal play C2CL2, the work machine 10 will not be activated.

[0104] The right pedal operation unit C2CR is the part that allows the operator to select which pedal to operate. For example, as shown in Figure 11, the right pedal operation unit C2CR has a right pedal section C2CR1 similar to the left pedal operation unit C2CL, a right pedal play section C2CR2, and a right pedal operating range section C2CR3. [Explanation of Symbols]

[0105] 1: Autonomous driving system 10: Working Machinery 10a: Machine body 11: Lower running body 13: Upper rotating body 15: Attachment 15a: Boom 15b: Arm 15c: Tip attachment 22: Input section 40: Information Processing Device 41: Input section 42:Display section 43: Control Unit

Claims

1. An automated driving system comprising a work machine capable of autonomous driving and an information processing device that is communicatively connected to the work machine, The aforementioned information processing device is A display unit that displays information related to the aforementioned work machine, An input unit that accepts operator input, A control unit that controls the display image to be displayed on the display unit and the information related to the input of the input unit, Equipped with, The control unit, The automatic driving process involves displaying information related to the automatic driving on the display unit and receiving input information related to the automatic driving on the input unit. Based on the operation received by the input unit, a remote operation process is performed to remotely control the work machine, Execute Autonomous driving system.

2. An automated driving system according to claim 1, The control unit switches between an automatic driving mode that displays information for performing the automatic driving process and a remote operation mode that displays information for performing the remote operation process. Autonomous driving system.

3. An automated driving system according to claim 2, The aforementioned display unit is a display, The control unit displays a switching reception area on the display unit for the input unit to accept a switch between the automatic operation mode and the remote operation mode. Autonomous driving system.

4. An automated driving system according to claim 1, The aforementioned work machine is equipped with multiple drive units, Each of the aforementioned multiple drive units is driven according to a command signal. The control unit outputs a plurality of command signals to the work machine based on a single operation received by the input unit. Autonomous driving system.

5. An automated driving system according to claim 1, The aforementioned work machine is The upper rotating body and A lower traveling body that supports the upper rotating body so that it can rotate, Equipped with, The control unit displays the rotation angle of the upper rotating body relative to the lower traveling body on the display unit. Autonomous driving system.

6. An automated driving system according to claim 1, The control unit displays the current position of the work machine on the display unit. Autonomous driving system.

7. An automated driving system according to claim 1, The control unit, when the working machine is in an unstable position, displays the desired position of the working machine on the display unit. Autonomous driving system.

8. An automated driving system according to claim 7, The control unit displays the desired posture and the current posture of the work machine superimposed on the display unit. Autonomous driving system.

9. An automated driving system according to claim 8, The control unit displays the desired posture and the current posture in different ways on the display unit. Autonomous driving system.

10. An automated driving system according to claim 1, If the working machine is in an unstable position, the control unit displays the necessary operations for the input unit to bring the working machine into a desirable position on the display unit. Autonomous driving system.

11. An automated driving system according to claim 1, The control unit, when the working machine is in an unstable position and the input unit receives an operation necessary to bring the working machine into a desirable position, allows the remote operation. Autonomous driving system.

12. An automated driving system according to claim 1, The control unit allows the remote operation when the input unit receives a specific operation. Autonomous driving system.

13. An automated driving system according to claim 2, The control unit detects and notifies that the posture of the work machine is unstable in the automatic operation mode. Autonomous driving system.

14. An automated driving system according to claim 3, If the control unit detects that the posture of the work machine is unstable in the automatic operation mode, it presents the switching acceptance area. Autonomous driving system.

15. An automated driving system according to claim 1, The operating performance of the aforementioned work machine is limited when it is remotely controlled compared to when it is automatically operated. Autonomous driving system.

16. An automated driving system according to claim 6, The aforementioned work machine is The machine body and Attachments and, Equipped with, The aforementioned attachment has the following components: A boom rotatably attached to the machine body, An arm rotatably attached to the boom, A tip attachment rotatably mounted on the aforementioned arm, Equipped with, The aforementioned display unit is a display, The input unit is a touch panel provided on the display, The control unit, In response to the touch operation received by the input unit, one of the components is identified, Following the aforementioned touch operation, in response to the drag operation received by the input unit, the identified component is rotated and displayed on the machine body or the component on the side to which the identified component is attached. In response to the aforementioned drag operation, the identified component is remotely controlled. Autonomous driving system.

Images