Automatic operation information processor, automatic operation system, automatic operation method, and automatic operation program

Abstract

To provide an automatic operation information processor that can change a viewpoint of a work site image according to an operational step during automatic operation without workers viewing a work site image having to perform complicated input operations when a task performed by automatic operation of a work machine includes a plurality of operational steps.SOLUTION: An automatic operation information processor (200) for automatic operation of a work machine (100) includes a display (223) for displaying a work site image, which is an image of a work site, and a controller (222) for controlling the display (223). The controller (222) executes a process for changing a viewpoint of the work site image in accordance with a plurality of preset operation steps included in a task at the work site.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to technologies related to working machines such as hydraulic excavators.

Background Art

[0002] Patent Document 1 discloses an automatic driving excavator composed of an excavator and an automatic driving controller provided on the excavator. The automatic driving controller causes the excavator to perform a series of repetitive operations based on at least the instructed excavation position and dumping position that the excavator should take during automatic driving. Patent Document 2 discloses a position information setting system for setting position information related to a working machine. Patent Document 3 discloses a remote operation system that displays an image such that the sense of distance to a work object can be accurately grasped even when the work object is viewed from an arbitrary angle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] The first problem in the present disclosure is as follows. That is, when the work performed by the automatic driving of a working machine includes a plurality of operation steps, each of the plurality of operation steps has a perspective that is easy for a worker to view. Therefore, the worker needs to perform a complicated input operation to change the perspective of the work site image according to the operation steps during automatic driving.

Means for Solving the Problems

[0005] A first object of the present disclosure is to provide a technology that can change the viewpoint of a work site image according to operation steps during automatic driving even when the work performed by automatic driving of a work machine includes a plurality of operation steps, without the need for a worker viewing the work site image to perform complicated input operations.

[0006] The first invention for solving the first problem includes an automatic driving information processing device, an automatic driving system, an automatic driving method, and an automatic driving program.

[0007] An automatic driving information processing device according to an aspect of the first invention is an information processing device for automatic driving of a work machine, and includes a display unit for displaying a work site image that is an image of a work site, and a controller for controlling the display unit. The controller executes a process of changing the viewpoint of the work site image according to a plurality of preset operation steps included in the work at the work site.

[0008] An automatic driving system according to another aspect of the first invention includes the automatic driving information processing device and the work machine.

[0009] An automatic driving method according to still another aspect of the first invention includes changing, by a controller, the viewpoint of a work site image that is an image of a work site in automatic driving of a work machine according to a plurality of preset operation steps included in the work at the work site.

[0010] An automatic driving program according to still another aspect of the first invention causes a computer to execute a process of changing the viewpoint of a work site image that is an image of a work site in automatic driving of a work machine according to a plurality of preset operation steps included in the work at the work site.

Advantages of the Invention

[0011] According to the first invention, a technology is provided that can change the viewpoint of a work site image according to operation steps during automatic driving without the need for a worker viewing the work site image to perform complicated input operations.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0013] The present disclosure includes the first invention for solving the first problem, the second invention for solving the second problem described later, the third invention for solving the third problem described later, the fourth invention for solving the fourth problem described later, and the fifth invention for solving the fifth problem described later.

[0014] [Embodiment of the First Invention] First, an embodiment of the first invention will be described. The first invention includes a plurality of embodiments, and the plurality of embodiments include an automatic driving information processing device 200, an automatic driving system 300, an automatic driving method, and an automatic driving program described below.

[0015] FIG. 1 is a diagram showing an automatic driving system 300 according to an embodiment. The automatic driving system 300 is a system for automatic driving of the work machine 100. As shown in FIG. 1, the automatic driving system 300 includes an automatic driving information processing device 200 and at least one work machine 100. The automatic driving information processing device 200 is a device for processing information related to the automatic driving of the work machine 100. In the specific example shown in FIG. 1, the automatic driving system 300 includes a plurality of work machines 100. The plurality of work machines 100 may be arranged at the same work site or at different work sites.

[0016] The working machine 100 includes a lower traveling body 1 including a traveling device, an upper revolving body 2 supported by the lower traveling body 1 so as to be rotatable about a vertically extending swivel axis Z, and a working device 3 supported by the upper revolving body 2. In the specific example shown in FIG. 1, the working machine 100 is a hydraulic excavator, but the working machine in the present disclosure is not limited to a hydraulic excavator, and may be other working machines such as a crane and a bulldozer. The traveling device may be the crawler traveling device shown in FIG. 1, or may be a traveling device having tires (not shown). The upper revolving body 2 includes a swivel frame rotatably supported by the lower traveling body 1, and a cab 9 supported by the swivel frame. Various components such as a driver's seat and an actual machine operating device are arranged in the cab 9.

[0017] The working device 3 includes a boom 4 attached to the upper revolving body 2 so as to be able to rise and fall, an arm 5 attached to the boom 4 so as to be rotatable, and a tip attachment 6 attached to the arm 5 so as to be rotatable. The tip attachment 6 may be a bucket 6A, may be a lifting magnet 6B, or may be other tip attachments (not shown) such as a grapple, a fork, and a crusher.

[0018] The working machine 100 includes a power source 7, a hydraulic pump 8, and a plurality of actuators. The hydraulic pump 8 is driven by the power source 7. The power source 7 may be, for example, an engine or an electric motor. The electric motor may be driven by, for example, at least one of a fuel cell and a battery.

[0019] Each of the plurality of actuators operates by receiving the supply of hydraulic oil discharged from the hydraulic pump 8. The plurality of actuators includes a boom cylinder 35 which is a cylinder for raising and lowering the boom 4, an arm cylinder 36 which is a cylinder for rotating the arm 5, a tip attachment cylinder 37 which is a cylinder for rotating the tip attachment 6, a swing motor 38 which is a hydraulic motor for swinging the upper swing body 2 with respect to the lower traveling body 1, and a traveling motor 39 which is a hydraulic motor for traveling the lower traveling body 1.

[0020] Each of the plurality of work machines 100 may be provided with a communication device for wirelessly or wiredly transmitting and receiving data between the work machine 100 and the automatic driving information processing device 200.

[0021] The automatic driving information processing device 200 has a computer including an arithmetic processing unit and a memory. The functions of the automatic driving information processing device 200 are realized by the arithmetic processing unit executing a program stored in the memory. The automatic driving information processing device 200 may be arranged at a location (for example, a remote location) away from the work machine 100. The automatic driving information processing device 200 can communicate with each of the plurality of work machines 100.

[0022] In the present embodiment, the automatic driving information processing device 200 may include a first external device 210 and a second external device 220.

[0023] The first external device 210 may be, for example, a management device such as a server, or may be a remote operation device such as a cockpit. The remote operation device may include at least one remote operator that receives an operation by an operator, and a display device that displays an image of a work site including the work machine 100. The at least one remote operator may include, for example, a remote operation lever that receives a lever operation by an operator, and a remote operation pedal that receives a pedal operation by an operator. The operator remotely operates the work machine 100 by operating the at least one remote operator while viewing the image displayed on the display device of the remote operation device.

[0024] The second external device 220 may be, for example, a work management information terminal which is an information terminal such as a tablet type computer (so-called tablet), a smart phone, a laptop personal computer, or a desktop personal computer, or may be a management device such as a server. Further, the second external device 220 may be, for example, a computer in a cloud service provided as a service on a network such as the Internet. Further, the second external device 220 may be a computer that controls a head-up display, or may be a wearable device such as a head-mounted display or smart glasses.

[0025] In addition, when the second external device 220 also has the function of the first external device 210, the first external device 210 may be omitted in the automatic driving information processing device 200. That is, the automatic driving information processing device 200 may include the second external device 220 and may not include the first external device 210. Further, when the function for remote operation is not required, the automatic driving information processing device 200 may include the second external device 220 and may not include the first external device 210.

[0026] The first external device 210 may have a communication device for wirelessly or wiredly transmitting and receiving data with each of the plurality of working machines 100, and the second external device 220 may have a communication device for wirelessly or wiredly transmitting and receiving data with each of the plurality of working machines 100. Further, the first external device 210 and the second external device 220 may be capable of wirelessly or wiredly transmitting and receiving data with each other.

[0027] The first external device 210 includes a computer having an arithmetic processing unit and a memory. The functions of the first external device 210 are realized by the arithmetic processing unit executing a program stored in the memory.

[0028] The automatic driving system 300 includes an imaging device 400. The imaging device 400 may be attached to the working machine 100 as shown in FIG. 1, for example, or may be disposed at a position separated from the working machine 100 at the work site.

[0029] The imaging device 400 may be a camera that acquires two-dimensional information, or may be a camera that acquires three-dimensional information. The camera that acquires three-dimensional information may be, for example, a LiDAR (Light Detection and Ranging), a stereo camera, or other imaging device capable of acquiring three-dimensional information. The LiDAR acquires point cloud data corresponding to the three-dimensional information of the work site. The information acquired by the imaging device 400 may be transmitted from the imaging device 400 to the automatic driving information processing device 200 via the communication device of the working machine 100, or may be directly transmitted from the imaging device 400 to the automatic driving information processing device 200.

[0030] [Features of the Embodiment of the First Invention] Next, the features of the embodiment of the first invention for solving the first problem will be described. The first invention includes the following first to tenth aspects.

[0031] The automatic driving information processing device 200 according to the first aspect is an information processing device for the automatic driving of the work machine 100, and includes a display unit 223 for displaying a work site image which is an image of the work site, and a controller 221 for controlling the display unit 223. The controller 221 executes a process of changing the viewpoint of the work site image according to a plurality of preset operation steps included in the work at the work site.

[0032] In this first aspect, even when the work performed by the automatic driving of the work machine 100 includes a plurality of operation steps, the controller 221 changes the viewpoint of the work site image according to the operation steps during the automatic driving. Therefore, the worker who views the work site image does not need to perform a complicated input operation for changing the viewpoint of the work site image.

[0033] The worker is a person related to the work by the work machine 100. Specifically, the worker may be an operator who boards the work machine 100, an operator who remotely operates the work machine 100 using the remote operation device, a manager who manages the work by the work machine 100, an assistant who assists the operator and / or the manager, or other related persons.

[0034] The controller 221 includes a computer having an arithmetic processing unit and a memory. The functions of the controller 221 are realized by the arithmetic processing unit executing a program stored in the memory.

[0035] The controller 221 controls the display unit 223 by performing arithmetic processing using at least one of the information input to the automatic driving information processing device 200 (for example, the second external device 220) via the communication device and the information corresponding to the input operation received by the input unit 222 described later. The information input to the automatic driving information processing device 200 via the communication device may include, for example, at least one of the information transmitted by the work machine 100, the information including the image data of the work site transmitted by the imaging device 400, and the information transmitted by the first external device 210 to the second external device 220.

[0036] The display unit 223 displays information related to autonomous driving. The display unit 223 may be a monitor (display), a device that projects an image onto an object, or one that uses AR (Augmented Reality) technology. Also, the display unit 223 may be a function included in, for example, the touch panel 224 described later. The display unit 223 displays a work site image that is an image of the work site. The controller 221 generates the work site image using the image data input from the imaging device 400, and causes the generated work site image to be displayed on the display unit 223.

[0037] In this embodiment, the autonomous driving information processing device 200 further includes an input unit 222. The input unit 222 receives an input operation by a worker. The input unit 222 may be a keyboard, a mouse, a joystick, an input switch, a device that inputs information based on the line-of-sight information of the worker, a device that inputs information based on the position information of the worker's finger in space, or other input devices. Also, the input unit 222 may be a function included in the touch panel 224 described later.

[0038] In this embodiment, the second external device 220 of the autonomous driving information processing device 200 includes a controller 221, an input unit 222, and a display unit 223. However, the first external device 210 of the autonomous driving information processing device 200 may include a controller 221, an input unit 222, and a display unit 223. Also, the first external device 210 may include a part of the controller 221, the input unit 222, and the display unit 223, and the second external device 220 may include the rest of the controller 221, the input unit 222, and the display unit 223.

[0039] In this embodiment, the second external device 220 is a tablet-type computer (so-called tablet) having a touch panel 224. The touch panel 224 includes the function of the input unit 222 and the function of the display unit 223. That is, the touch panel 224 includes the function of a display for displaying an image and the function of a pointing device for receiving an input operation by a worker. In this case, the touch panel 224 including the input unit 222 receives a touch operation as an input operation by a worker, and the touch panel 224 including the display unit 223 displays various images based on the control by the controller 221.

[0040] FIG. 2 is a diagram showing a display image displayed on the display unit 223 of the second external device 220, and shows an example of a basic screen. The basic screen may include, for example, a work selection section A, a work addition section B, a function selection section C, a menu selection section D, an operation content display section E, a machine state display section F, and a communication state display section G. Each of the work selection section A, the work addition section B, the function selection section C, and the menu selection section D receives a touch operation (input operation) of a worker. That is, on the basic screen, the input unit 222 includes the work selection section A, the work addition section B, the function selection section C, and the menu selection section D. The content of the input unit 222 changes according to the image displayed on the display unit 223 (touch panel 224).

[0041] In addition, when the input unit 222 includes a mouse, the input operation on the input unit 222 may include an operation using the mouse, and when the input unit 222 includes a keyboard, the input operation on the input unit 222 may include an operation using the keyboard.

[0042] The work selection section A is a part that receives an input operation for selecting the type of work performed by the work machine 100 in automatic operation. The work selection section A includes options for the work performed by the work machine 100 in automatic operation. In the specific example shown in FIG. 2, the work selection section A includes a part A1 for selecting a loading work and a part A2 for selecting a work using a lifting magnet (lifting magnet work).

[0043] When the operation selection unit A receives an input operation, the controller 221 outputs an operation instruction for instructing the automatic operation of an operation (for example, a loading operation) corresponding to the input operation. Among the plurality of working machines 100, the working machine 100 that has established communication access with the automatic driving information processing device 200 receives the operation instruction and executes the operation (for example, a loading operation) corresponding to the operation instruction by automatic driving.

[0044] The operation addition unit B is a part that receives an input operation for adding an operation other than the operations selectable in the operation selection unit A. That is, the operation addition unit B is a part that receives an input operation for adding options in the operation selection unit A. When the operation addition unit B receives an input operation, the controller 221 executes a process for adding operation options.

[0045] The function selection unit C is a part that receives an input operation for adding a function related to automatic driving. Specifically, for example, the function selection unit C may include a teaching function selection unit C1, a loading function selection unit C2, a trajectory confirmation function selection unit C3, a loading position adjustment function selection unit C4, and a setting unit C5.

[0046] The teaching function selection unit C1 is a part that receives an input operation related to teaching. Specifically, for example, when the teaching function selection unit C1 receives an input operation, the controller 221 executes a process related to teaching for automatic driving. Teaching may be performed, for example, by an operator remotely operating the working machine 100 using the remote operation device (the first external device 210), and the storage of the working machine 100 storing teaching data corresponding to the remote operation.

[0047] The loading function selection unit C2 is a part that receives an input operation for setting working conditions. Specifically, for example, the working conditions may include at least one of the operating speed of the working device 3 in the loading operation, the turning speed of the upper slewing body 2, the target number of times of the loading operation, and the target loading earthwork volume.

[0048] The locus confirmation function selection unit C3 is a part that receives an input operation for confirming the taught content. Specifically, for example, when the locus confirmation function selection unit C3 receives an input operation, the controller 221 performs a process of switching the image on the display unit 223 to a screen for confirming the taught content.

[0049] The loading position adjustment function selection unit C4 is a part that receives an input operation for setting a target position (target range) for discharging the earth and sand held in the bucket 6A. Specifically, for example, when the loading position adjustment function selection unit C4 receives an input operation, the controller 221 performs a process of switching the image on the display unit 223 to a screen for setting the target position (target range).

[0050] The setting unit C5 is a part that receives an input operation for setting the connection between the second external device 220 and the working machine 100. Specifically, for example, when the setting unit C5 receives an input operation, the controller 221 performs a process of switching the image on the display unit 223 to a screen for setting the connection between the second external device 220 and the working machine 100.

[0051] The menu selection unit D is a part that receives an input operation for displaying a menu screen (not shown).

[0052] The operation content display unit E is a part that displays the state of the second external device 220 at that time. The machine state display unit F is a part that displays the state of the working machine 100 at that time. The communication state display unit G is a part that displays the communication state of the second external device 220. The communication state may be, for example, the communication state between the second external device 220 and the working machine 100, or the communication state between the second external device 220 and the first external device 210.

[0053] A specific example will be given and described for the features of the first aspect. As described above, in the first aspect, the controller 221 executes processing for changing the viewpoint of the work site image according to a plurality of preset operation steps included in the work at the work site. In the following specific example, the work at the work site (work by automatic driving) is a loading work.

[0054] FIG. 3 is a perspective view for explaining a plurality of operation steps included in the loading work. As shown in FIG. 3, the loading work includes an excavation step (an example of an operation step), a lifting and turning step (an example of an operation step), an earth discharge step (an example of an operation step), and a return turning step (an example of an operation step). The excavation step, the lifting and turning step, the earth discharge step, and the return turning step are performed in this order. The excavation step is an operation step for excavating an object to be excavated such as the ground. The lifting and turning step is an operation step for moving the bucket 6A that holds the excavated earth and sand from the excavation target directly above the earth discharge area. The earth discharge area may be, for example, the loading platform of a dump truck or an area for discharging earth formed at the work site. The earth discharge step is an operation step for discharging earth and sand from the bucket 6A to the earth discharge area. The return turning step is an operation step for returning the bucket 6A from directly above the earth discharge area to the excavation target.

[0055] The teaching data corresponding to the loading work including a plurality of operation steps is stored in advance in at least one of the storage of the work machine 100 and the storage of the automatic driving information processing device 200 (for example, the storage of the second external device 220).

[0056] Each of FIGS. 4 to 7 is a diagram showing a display image displayed on the display unit 223. FIG. 4 is an example of a work site image (plan view) from the first viewpoint for the excavation step. FIG. 5 is an example of a work site image (perspective view) from the second viewpoint for the lifting and turning step. FIG. 6 is an example of a work site image (plane) from the third viewpoint for the earth discharge step. FIG. 7 is an example of a work site image (perspective view) from the fourth viewpoint for the return turning step.

[0057] The first perspective shown in FIG. 4 is a perspective from directly above the work site including the work machine 100, and the work site image from the first perspective may be a plan view as shown in FIG. 4. The second perspective shown in FIG. 5 is a perspective from diagonally above the work site including the work machine 100, and the work site image from the second perspective may be a perspective view as shown in FIG. 5. The third perspective shown in FIG. 6 is a perspective from directly above the work site including the work machine 100, and the work site image from the third perspective may be a plan view as shown in FIG. 6. The fourth perspective shown in FIG. 7 is a perspective from diagonally above the work site including the work machine 100, and the work site image from the fourth perspective may be a perspective view as shown in FIG. 7.

[0058] The imaging device 400 acquires, for example, image data including point cloud data corresponding to the three-dimensional information of the work site, and the acquired image data is transmitted to the automatic driving information processing device 200 (for example, the second external device 220). Since the point cloud data includes data of the three-dimensional coordinates of each point, the controller 221 can generate a work site image viewed from an arbitrary perspective using the point cloud data. Each of the first to fourth perspectives may be preset and stored in the controller 221. Each of the first to fourth perspectives may be set when the input unit 222 receives an input operation by the operator.

[0059] The controller 221 causes the display unit 223 to display a work site image from the first perspective shown in FIG. 4 in the excavation step, causes the display unit 223 to display a work site image from the second perspective shown in FIG. 5 in the lifting and turning step, causes the display unit 223 to display a work site image from the third perspective shown in FIG. 6 in the soil discharge step, and causes the display unit 223 to display a work site image from the fourth perspective shown in FIG. 7 in the return turning step. In this way, since the controller 221 changes the perspective of the work site image according to the operation step during automatic driving, the worker viewing the work site image does not need to perform a complicated input operation for changing the perspective of the work site image.

[0060] The range of each operation step may be set using, for example, three-dimensional coordinates. Specifically, each of the start time point and the end time point of each operation step may be set using, for example, three-dimensional coordinates. The controller 221 can determine the operation step at that time by comparing the range of each operation step with the three-dimensional coordinates of a specific part of the machine tool 100. In the present embodiment, the specific part is a part of the bucket 6A (for example, the tip of the bucket 6A). The specific part is not limited to a part of the bucket 6A, and may be a part of the arm 5 or another part of the working device 3.

[0061] Note that the viewpoints corresponding to the plurality of operation steps are not limited to the specific examples shown in FIGS. 4 to 7. For example, when the dumping area in the dumping step is, for example, the loading platform of a dump truck, in the dumping step, a work site image from the viewpoint as shown in FIG. 8 may be displayed on the display unit 223 so that the worker can easily determine the vertical relative position between the bucket 6A and the loading platform. The viewpoint shown in FIG. 8 is a viewpoint from which the work site including the machine tool 100 can be seen from the side, and the work site image from this viewpoint may be a side view as shown in FIG. 8.

[0062] Also, although not shown, in the excavation step, a work site image from a viewpoint from which the excavation target can be seen from the side may be displayed on the display unit 223 so that the worker can easily determine the unevenness (ground surface) of the surface of the excavation target such as the ground. The work site image from this viewpoint may be a side view (or cross-sectional view) including the uneven shape (surface shape) of the ground.

[0063] FIG. 14 is a flowchart showing an example of arithmetic processing performed by the automatic driving information processing device 200. In the first operation step (for example, the excavation step), the automatic driving information processing device 200 displays, on the display unit 223, a work site image from a first viewpoint preset for the first operation step (step S1). When the first operation step is completed (YES in step S2), the automatic driving information processing device 200 displays, on the display unit 223, a work site image from a second viewpoint preset for the second operation step (for example, the lifting and turning step) in the second operation step (step S3). When the second operation step is completed (YES in step S4), the automatic driving information processing device 200 displays, on the display unit 223, a work site image from a third viewpoint preset for the third operation step (for example, the dumping step) in the third operation step (step S5). When the third operation step is completed (YES in step S6), the automatic driving information processing device 200 displays, on the display unit 223, a work site image from a fourth viewpoint preset for the fourth operation step (for example, the return turning step) in the fourth operation step (step S7). When the fourth operation step is completed (YES in step S8), the automatic driving information processing device 200 repeats the processing after step S1.

[0064] The automatic driving information processing device 200 according to the second aspect preferably further includes the following configuration in the automatic driving information processing device 200 according to the first aspect. That is, in the automatic driving information processing device 200 according to the second aspect, it is preferable that the controller 221 causes the display unit 223 to display a viewpoint switching input unit I that receives an input operation by a worker for switching to a work site image from a specific viewpoint that is a preset viewpoint.

[0065] In this second aspect, the work site image can be adjusted to an image from a desired viewpoint with a small number of operations by the worker. Specifically, it is as follows.

[0066] For example, in the automatic driving screen displayed on the display unit 223 during automatic driving, such as in FIGS. 4 to 7, the controller 221 may display the viewpoint switching input unit I on the display unit 223. When this viewpoint switching input unit I receives an input operation, the controller 221 may execute a process of switching to a work site image from a preset specific viewpoint. The specific viewpoint may be, for example, the viewpoint shown in FIG. 8, a viewpoint that can view the excavation target from the side, or another viewpoint.

[0067] In the default setting, even when the display unit 223 displays a work site image from the third viewpoint shown in FIG. 6 in the dumping step, if a worker gives an input operation to the viewpoint switching input unit I in the dumping step, the controller 221 may execute a process of switching from the third viewpoint shown in FIG. 6 to, for example, the viewpoint shown in FIG. 8. As a result, in the dumping step, the work site image switches from the work site image from the viewpoint shown in FIG. 6 to the work site image from the viewpoint shown in FIG. 8. Also, in the default setting, even when the display unit 223 displays a work site image from the first viewpoint shown in FIG. 4 in the excavation step, if a worker gives an input operation to the viewpoint switching input unit I in the excavation step, the controller 221 may execute a process of switching from the first viewpoint shown in FIG. 4 to, for example, a viewpoint that can view the excavation target from the side. As a result, in the excavation step, the work site image switches from the work site image from the viewpoint shown in FIG. 4 to a work site image of a side view (or cross-sectional view) including the uneven shape (surface shape) of the ground.

[0068] Also, in this second aspect, when the controller 221 stores a plurality of different specific viewpoints, the controller 221 may execute a process of switching the viewpoint of the work site image so that the plurality of specific viewpoints are sequentially switched in response to repeated input operations being given to the viewpoint switching input unit I. Thereby, the worker can select a preferred viewpoint from among the plurality of specific viewpoints with a small number of operations.

[0069] The automatic driving information processing device 200 according to the third aspect preferably further includes the following configuration in the automatic driving information processing device 200 according to the second aspect. That is, in the automatic driving information processing device 200 according to the third aspect, the input operation for switching to the work site image from a specific viewpoint is preferably a single operation by the worker, that is, a single operation given to the viewpoint switching input unit I.

[0070] In this third aspect, the work site image can be adjusted by a single operation by the worker to an image from a desired viewpoint, for example, an image from a specific viewpoint shown in any of FIGS. 4 to 10.

[0071] The automatic driving information processing device 200 according to the fourth aspect preferably further includes the following configuration in the automatic driving information processing device 200 according to any one of the first to third aspects. That is, in the automatic driving information processing device 200 according to the fourth aspect, the controller 221 preferably controls the display unit 223 so that the work site image is switched to an image from a specific viewpoint preset for checking the deviation of the work target at the work site before the start of the automatic driving.

[0072] In this fourth aspect, before the start of the automatic driving, the controller 221 switches to the work site image from a viewpoint suitable for the worker to check the deviation of the work target, so the worker does not need to perform a complicated input operation for changing the viewpoint of the work site image before the start of the automatic driving. A specific example of the fourth aspect is as follows. The work site image that the controller 221 causes the display unit 223 to display before the start of the automatic driving may be, for example, the image shown in FIG. 9, the image shown in FIG. 10, or other images.

[0073] In the present embodiment, the automatic driving information processing device 200 has a function for checking the deviation of the work target at the work site before the start of the automatic driving of the work machine 100.

[0074] The operation target may be, for example, a target excavation area (an example of a target work area) that is the target of the excavation area in the excavation step, or a target dumping area (an example of a target work area) that is the target of the dumping area in the dumping step, or a target path that is the target of the path along which a specific part of the work machine 100 (for example, the tip of the bucket 6A) moves in the lifting and slewing step, or a target path that is the target of the path along which a specific part of the work machine 100 (for example, the tip of the bucket 6A) moves in the return slewing step.

[0075] FIG. 9 is a diagram showing a display image displayed on the display unit 223, and is an example of a target confirmation screen which is a screen for confirming the deviation of the operation target at the work site. In this target confirmation screen, the work site image is shown as a perspective view seen obliquely from above the excavation area. In FIG. 9, the target excavation area TA is indicated by a rectangular (for example, parallelogram) frame. Since this target excavation area TA may deviate from the actual excavation area AA1 before the start of the automatic operation, the worker confirms the deviation between the target excavation area TA and the actual excavation area AA1 before the start of the automatic operation on the target confirmation screen.

[0076] The target confirmation screen shown in FIG. 9 includes a correction input unit H for correcting the deviation. The correction input unit H includes a first correction input unit H1 for rotating the work site image in various directions, a second correction input unit H2 for moving the work site image vertically and horizontally, a third correction input unit H3 for zooming in and zooming out, and a fourth correction input unit H4 for returning the work site image to the default state. The worker can finely adjust the viewing point of the work site image on the target confirmation screen by giving an input operation to the correction input unit H.

[0077] When the target confirmation screen is a perspective view as shown in FIG. 9 and the work site image is represented by a point cloud, this work site image is not necessarily easy for the workers to view, and it is difficult to accurately distinguish the deviation between the target excavation area TA and the actual excavation area AA1. Further, when adjusting the viewpoint by giving an input operation to the correction input unit H, the worker needs to perform many input operations.

[0078] Therefore, in this aspect, the controller 221 may cause the display unit 223 to display a viewpoint switching input unit I (frame display switching button) for receiving an input operation by the worker to switch to a work site image from a specific viewpoint which is a preset viewpoint. Then, when the worker gives an input operation to the viewpoint switching input unit I, the controller 221 may execute a process of switching the work site image from the viewpoint shown in FIG. 9 to a work site image from a specific viewpoint shown in FIG. 10, for example. The specific viewpoint shown in FIG. 10 is a viewpoint from which it is possible to view the work site including the excavation area AA1, the target excavation area TA, and the working machine 100 from directly above, and the work site image from this specific viewpoint may be a plan view as shown in FIG. 10. In this aspect, the work site image can be adjusted to an image from a specific viewpoint where the deviation is easy to distinguish with a small number of operations by the worker. Note that, in this aspect, the input operation for switching to the work site image from the specific viewpoint does not necessarily have to be a single operation. However, for further improvement in operability, the input operation for switching to the work site image from the specific viewpoint is preferably a single operation by the worker, that is, a single operation given to the viewpoint switching input unit I.

[0079] Note that the work site image shown in FIG. 9 is displayed by a point cloud, and the work site image shown in FIG. 10 is displayed by a schematic diagram, but the work site image shown in FIG. 10 may also be displayed by a point cloud. Also in FIGS. 3 to 8, the work site image may be displayed by a point cloud as shown in FIG. 9.

[0080] The automatic driving information processing device 200 according to the fifth aspect preferably further includes the following configuration in the automatic driving information processing device 200 according to any one of the first to fourth aspects. That is, in the automatic driving information processing device 200 according to the fifth aspect, the controller 221 is configured to control the work machine 100 so that a specific operation preset at the work site is performed, and the controller 221 preferably controls the work machine 100 so that a simple emergency stop confirmation operation is performed before the start of the automatic driving, compared to the specific operation.

[0081] In this fifth aspect, it is possible to confirm an emergency stop even before teaching a specific operation.

[0082] When the specific operation is, for example, the operation of the loading work, as described above, the loading work includes the excavation step, the lifting and turning step, the earth discharge step, and the return turning step. Therefore, in order to confirm an emergency stop while performing the operation of this loading work, it is necessary to previously teach the loading work before this emergency stop confirmation work. On the other hand, in the fifth aspect, it is possible to confirm an emergency stop even before teaching the loading work.

[0083] The automatic driving information processing device 200 according to the sixth aspect preferably further includes the following configuration in the automatic driving information processing device 200 according to the fifth aspect. That is, in the automatic driving information processing device 200 according to the sixth aspect, the work machine 100 includes an upper swing body 2 (an example of a machine body), a boom 4 supported by the upper swing body 2 so as to be able to rise and fall, an arm 5 supported by the boom 4, and a bucket 6A (an example of the tip attachment 6) rotatably supported at the tip of the arm 5. The emergency stop confirmation operation is preferably a rotation operation of the bucket 6A. The boom 4 and the arm 5 are examples of lifting members.

[0084] In this sixth aspect, it is possible to confirm an emergency stop by using a simple operation such as the rotation operation of the bucket 6A. A specific example of the sixth aspect is as follows.

[0085] FIG. 11 is a diagram showing a display image displayed on the display unit 223 of the second external device 220, and is a diagram for explaining an emergency stop confirmation operation that is simpler than the operation (specific operation) of the loading operation by the automatic driving. The emergency stop confirmation screen shown in this FIG. 11 has an emergency stop input unit J.

[0086] As shown in FIG. 11, the controller 221 may display a display for prompting a worker to perform a predetermined operation on the display unit 223. This display may, for example, prompt the worker to adjust the posture of the working device 3 so that the bucket 6A does not interfere with the surroundings.

[0087] When the worker gives an input operation to the start input unit K that receives an input operation for starting the emergency stop confirmation operation, the controller 221 inputs an instruction to the working machine 100 so that the bucket 6A performs a rotation operation. Then, when the bucket 6A is performing a rotation operation, the worker gives an input operation to the emergency stop input unit J. In this case, if the emergency stop function is normal, the rotation operation of the bucket 6A stops. On the other hand, if the emergency stop function is not normal, the rotation operation of the bucket 6A does not stop. Therefore, in this sixth aspect, it is possible to confirm the emergency stop by using a simple operation of the rotation operation of the bucket 6A.

[0088] Note that the emergency stop may be performed by blocking the supply of pilot pressure to the pilot port of the control valve. The control valve is, for example, a valve that controls the supply of hydraulic oil to the boom cylinder 35, the arm cylinder 36, the tip attachment cylinder 37 (bucket cylinder), the slewing motor 38, and the traveling motor 39.

[0089] The automatic driving information processing device 200 according to the seventh aspect preferably further includes the following configuration in the automatic driving information processing device 200 according to any one of the first to sixth aspects. That is, in the automatic driving information processing device 200 according to the seventh aspect, the plurality of operation steps include a first operation step and a second operation step performed next to the first operation step, and the controller 221 preferably determines that the transition from the first operation step to the second operation step has occurred when a preset specific part in the working device 3 exits from the boundary of the working area corresponding to the first operation step.

[0090] In this seventh aspect, the accuracy of the transition determination can be improved as compared with the case where the transition determination of the operation steps is performed based on the operation speed of the movable part (for example, the turning speed of the upper slewing body 2).

[0091] The first operation step may be, for example, an excavation step, and the second operation step may be, for example, a lifting and turning step. In this case, when a specific part (for example, the tip of the bucket 6A) of the working machine 100 passes through a position corresponding to the boundary of the excavation area AA1 shown in FIGS. 3 and 4 (for example, the edge of the rectangular frame shown in FIG. 4) and the specific part exits the excavation area AA1, the controller 221 may determine that the transition from the excavation step to the lifting and turning step has occurred. The controller 221 can perform the above determination by comparing the coordinates of the boundary of the excavation area AA1 and the coordinates of the specific part.

[0092] Also, the first operation step may be, for example, a lifting and turning step, and the second operation step may be, for example, a soil discharge step. In this case, when a specific part (for example, the tip of the bucket 6A) of the working machine 100 passes through a position corresponding to the boundary of the soil discharge area AA2 shown in FIGS. 3 and 4 (for example, the edge of the rectangular frame shown in FIG. 4) and the specific part enters the soil discharge area AA2, the controller 221 may determine that the transition from the lifting and turning step to the soil discharge step has occurred. The controller 221 can perform the above determination by comparing the coordinates of the boundary of the soil discharge area AA2 and the coordinates of the specific part.

[0093] Also, the first operation step may be, for example, an earth discharging step, and the second operation step may be, for example, a return turning step. In this case, when a specific part (for example, the tip of the bucket 6A) of the working machine 100 passes through a position corresponding to the boundary of the earth discharging area AA2 shown in FIGS. 3 and 4 (for example, the edge of the rectangular frame shown in FIG. 4) and the specific part exits the earth discharging area AA2, the controller 221 may determine that the process has shifted from the earth discharging step to the return turning step. The controller 221 can make the above determination by comparing the coordinates of the boundary of the earth discharging area AA2 with the coordinates of the specific part.

[0094] Also, the first operation step may be, for example, a return turning step, and the second operation step may be, for example, an excavation step. In this case, when a specific part (for example, the tip of the bucket 6A) of the working machine 100 passes through a position corresponding to the boundary of the excavation area AA1 shown in FIGS. 3 and 4 (for example, the edge of the rectangular frame shown in FIG. 4) and the specific part enters the excavation area AA1, the controller 221 may determine that the process has shifted from the return turning step to the excavation step. The controller 221 can make the above determination by comparing the coordinates of the boundary of the excavation area AA1 with the coordinates of the specific part.

[0095] Note that the position corresponding to the boundary of the excavation area AA1 may be coordinates corresponding to the X coordinate and the Y coordinate of the excavation area AA1, and the position corresponding to the boundary of the earth discharging area AA2 may be coordinates corresponding to the X coordinate and the Y coordinate of the earth discharging area AA2. The X coordinate may be, for example, a coordinate extending in the horizontal direction, and the Y coordinate may be a coordinate extending in the horizontal direction and orthogonal to the X coordinate.

[0096] The automatic driving system 300 according to the eighth aspect includes an automatic driving information processing device 200 and at least one working machine 100.

[0097] In this eighth aspect, even when the work performed by the automatic operation of the working machine 100 includes a plurality of operation steps, since the viewpoint of the work site image changes according to the operation steps during the automatic operation, the worker who views the work site image does not need to perform complicated input operations.

[0098] The automatic driving system 300 according to this eighth aspect may further include at least one feature of the first to seventh aspects described above.

[0099] The automatic driving method according to the ninth aspect includes changing, by the controller 221, the viewpoint of the work site image, which is an image of the work site in the automatic driving of the working machine, according to a plurality of preset operation steps included in the work at the work site.

[0100] In this ninth aspect, even when the work performed by the automatic operation of the working machine 100 includes a plurality of operation steps, since the viewpoint of the work site image changes according to the operation steps during the automatic operation, the worker who views the work site image does not need to perform complicated input operations.

[0101] The automatic driving method according to this ninth aspect may further include at least one feature of the first to eighth aspects described above.

[0102] The automatic driving program according to the tenth aspect causes a computer (for example, the controller 221) to execute a process of changing the viewpoint of the work site image, which is an image of the work site in the automatic driving of the working machine, according to a plurality of preset operation steps included in the work at the work site.

[0103] In this tenth aspect, even when the work performed by the automatic operation of the working machine 100 includes a plurality of operation steps, since the viewpoint of the work site image changes according to the operation steps during the automatic operation, the worker who views the work site image does not need to perform complicated input operations. The automatic driving program may be stored in a storage medium.

[0104] The automatic driving program according to this tenth aspect may further include at least one feature of the above-described first to ninth aspects.

[0105] [Modification Example] As described above, the embodiments of the first invention for solving the first problem have been described. However, the first invention is not limited to the embodiments. Each of the automatic driving information processing device 200, the automatic driving system 300, the automatic driving method, and the automatic driving program according to the embodiments of the first invention includes, for example, the following modification examples.

[0106] (A) Modification Example 1: Regarding the operation confirmation of the excavation step The problem of Modification Example 1 of the embodiment of the first invention is that even when the dump truck is not present at a predetermined position at the work site, it is desired to provide a technique for performing the operation confirmation of the excavation step.

[0107] In the technique according to Modification Example 1 for solving this problem, the controller 221 outputs an excavation operation confirmation instruction for the work machine 100 to perform an excavation step of excavating an excavation target such as the ground with the bucket 6A in the excavation area AA1 and a soil discharge step of discharging the earth and sand held by the bucket 6A from the bucket 6A in the excavation area AA1. Thereby, even when the dump truck is not present at a predetermined position at the work site, the operation confirmation of the excavation step can be performed without discharging the soil to an area other than the excavation area AA1.

[0108] In this Modification Example 1, it is preferable that the controller 221 outputs a plurality of the excavation operation confirmation instructions so that the work machine 100 continuously repeats a set of operations including the above-described excavation step and the above-described soil discharge step a plurality of times. Thereby, a plurality of operation confirmations for the excavation step can be continuously performed.

[0109] In this first modification example, the controller 221 may output a boring operation confirmation instruction such that the operation speed of the work device 3 in the operation confirmation of the boring step is lower than the operation speed of the work device 3 in the boring step of the actual loading operation, or may output a boring operation confirmation instruction such that the operation speed is higher than the operation speed of the work device 3 in the boring step of the actual loading operation. Also, in the operation confirmation of the boring step, it is preferable that the slewing operation of the upper slewing body 2 is not performed.

[0110] The controller 221 may, for example, determine the quality of the operation in the boring step based on the amount of soil held in the bucket 6A in the operation confirmation of the boring step. If the operation in the boring step does not meet a preset standard, the controller 221 may perform processing to prompt the worker to change the parameters in the boring step, that is, the second modification example described below. The controller 221 may determine the amount of soil held in the bucket 6A based on image data from an imaging device 400 such as LiDAR in the operation confirmation, or may determine it based on the value of a sensor that measures the amount of soil. Also, the controller 221 may, for example, determine the quality of the operation in the boring step based on the distribution of the earth and sand in the bucket 6A in the operation confirmation of the boring step. For example, it is not preferable for the earth and sand to be biased toward the tip side of the bucket 6A. When the distribution of the earth and sand in the bucket 6A is not within a preset allowable range, the controller 221 may perform processing to prompt the worker to change the parameters related to the distribution of the earth and sand in the bucket 6A. The controller 221 may, for example, determine the distribution of the earth and sand in the bucket 6A based on image data from an imaging device 400 such as LiDAR in the operation confirmation. The parameter related to the distribution of the earth and sand in the bucket 6A may be, for example, the angle of the bucket 6A with respect to the ground (or horizontal plane) in the boring step (bucket-to-ground angle). Based on the input of the parameter change by the worker, the controller 221 may, for example, change the setting of the bucket-to-ground angle during the boring operation, or may change the setting of the bucket-to-ground angle at the completion of the boring operation.

[0111] (B) Variant Example 2: Regarding Changes in Parameters of the Excavation Step At the work site, it is assumed that the quality of the earth and sand varies for each work area, and the quality of the earth and sand fluctuates depending on the weather. Even if the apparent volume of only the earth and sand held in the bucket 6A is the same, the weight of the earth and sand held in the bucket 6A may be different. In this case, in order to prevent the lower traveling body 1 of the work machine 100 from floating, it is preferable for the worker to adjust the amount of earth and sand (target earth amount) excavated by one excavation step in the automatic operation. In the automatic operation of the loading work, it is desirable to appropriately adjust the excavation operation according to the daily change in soil quality and the change in the excavation shape to ensure a stable excavation earth amount.

[0112] FIG. 12 is a diagram showing a display image displayed on the display unit 223, and shows an example of an excavation operation setting screen for setting the excavation operation. As shown in FIG. 12, in this variant example 2, the excavation operation setting screen includes at least one excavation operation setting input unit for adjusting the target earth amount. In the specific example shown in FIG. 12, the at least one excavation operation setting input unit L includes a penetration depth setting input unit L1 and a target earth amount in bucket setting input unit L2.

[0113] The penetration depth setting input unit L1 is an input unit for setting the penetration depth of the bucket with respect to the ground in the excavation area AA1. When the worker performs an input operation on the penetration depth setting input unit L1, the controller 221 changes the setting of the penetration depth.

[0114] The target earth amount in bucket setting input unit L2 is an input unit for setting the target earth amount, which is the target amount of earth and sand held in the bucket 6A in the excavation step. When the worker performs an input operation on the target earth amount in bucket setting input unit L2, the controller 221 changes the setting of the target earth amount.

[0115] In this Modification Example 2, the worker determines whether a setting change is necessary based on the situation of the excavation step, and the worker can adjust the setting of the excavation operation by performing an input operation on the excavation operation setting input section L of the excavation operation setting screen as described above. Also, in this Modification Example 2, the worker can adjust the setting of the excavation operation using an external device (external terminal) such as a tablet. In the specific example of FIG. 12, the parameters for setting the excavation operation are limited to two important items, and moreover, these parameters are easy for the worker to understand.

[0116] (C) Modification Example 3: Regarding the air operation mode The problem of Modification Example 3 of the embodiment of the first invention is that even when the dump truck is not present at a predetermined position at the work site, it is desired to provide a technique for performing an operation check of a series of operation steps included in the loading operation.

[0117] In the technique according to Modification Example 3 for solving this problem, the controller 221 outputs an air operation instruction to the work machine 100 so that a series of operation steps included in a preset air operation are performed. The series of operation steps includes an air excavation step in which an excavation operation is performed while the bucket 6A is kept away from the ground in the excavation area AA1, a lifting and slewing step for moving the bucket 6A that is not holding earth and sand directly above the dumping area from the excavation target, a dumping step for causing an operation to dump earth and sand from the bucket 6A to the dumping area, and a return slewing step for returning the bucket 6A from directly above the dumping area to the excavation target.

[0118] In this Modification 3, the air excavation step is an operation step in which the excavation operation, that is, the excavation operation of the bucket 6A in the air, is performed while the bucket 6A is kept away from the ground in the excavation area AA1. Therefore, even when the dump truck is not present at a predetermined position at the work site, it is possible to check the operation of a series of operation steps included in the loading operation. The controller 221 may set the distance between the bucket 6A and the ground in the air excavation step as follows, for example. That is, the controller 221 identifies the height position of the ground in the excavation area AA1 based on the image data from the imaging device 400 such as LiDAR, and adds a reference height preset to the ground height. The obtained value may be set as the operating height of the bucket 6A in the air excavation step. In the soil discharging step, the soil discharging area may be preset, and the soil discharging step may be performed in a state where the bucket 6A is arranged near the center of the soil discharging area in plan view.

[0119] (D) Modification 4: Reference setting for determination of displacement of lower traveling body The problem of Modification 4 of the embodiment of the first invention is that it is assumed that the position of the lower traveling body 1 of the work machine 100 shifts due to the lower traveling body 1 slipping during automatic driving. If the automatic driving is continued with this displacement occurring, the relative position between the operation path of the work machine 100 based on the teaching data and the actual work site will shift.

[0120] FIG. 13 is a diagram showing a display image displayed on the display unit 223, and shows an example of a reference setting screen for determining the displacement of the lower traveling body 1 of the work machine 100. In this Modification 4, a reference mark MK is arranged in advance at the work site. The mark MK may be, for example, a sphere, and preferably has a conspicuous color (for example, yellow). The controller 221 stores an initial value of the relative position (relative coordinates) between the lower traveling body 1 of the work machine 100 and the mark MK before the start of work. The storage of the relative coordinates is performed, for example, by adjusting the posture of the work machine 100 so that the tip of the bucket 6A contacts or approaches the mark MK as in the work machine 100 in the reference setting screen of FIG. 13. That is, the controller 221 can acquire information regarding the posture of the work machine 100, specifically, information regarding postures such as the posture of the boom 4, the posture of the arm 5, the posture of the bucket 6A, and the turning angle of the upper swing body 2 with respect to the lower traveling body 1, based on the detection values by the posture sensors 40 (see FIG. 1) such as angle sensors. The controller 221 may acquire the position of the mark MK based on, for example, image data from an imaging device 400 such as LiDAR.

[0121] When the relative coordinates of the lower traveling body 1 with respect to the mark MK change from the initial value during automatic operation, the controller 221 may determine that a displacement of the lower traveling body 1 has occurred and stop the operation of the work machine 100. In addition, when the controller 221 determines that the displacement has occurred, the controller 221 may automatically adjust the position of the lower traveling body 1 by automatic operation so that the relative coordinates become or approach the initial value. Further, the worker may install the mark MK at the work site and register the front-rear direction, height, and turning angle in the reference setting screen shown in FIG. 13.

[0122] (E) Modification 5: Regarding the sensor check function In the problem of Modification 5 of the embodiment of the first invention, it is assumed that due to long-term use, the detection value of the posture sensor 40 such as an angle sensor may deviate from the normal value or the posture sensor 40 may malfunction.

[0123] In Modification 5, the controller 221 may determine the presence or absence of a sensor abnormality by comparing the current sensor detection values when the movable parts such as the working device 3 and the upper swing body 2 of the working machine 100 are arranged in a predetermined posture with the initial values (for example, the sensor detection values at the time of factory shipment) in the same posture. It is preferable that the determination of the presence or absence of the sensor abnormality is performed before the start of work for each day. The predetermined posture preferably corresponds to, for example, a state in which the rod is maximally extended with respect to the head or a state in which the rod is maximally contracted with respect to the head in a cylinder such as the boom cylinder 35.

[0124] (F) Modification 6: Regarding the display of function information In Modification 6 of the embodiment of the first invention, since there is no operator in the cab 9 of the working machine 100 during automatic driving, it is impossible to visually confirm the state of the working machine 100 such as the presence or absence of fuel and the presence or absence of urea in the working machine 100.

[0125] In Modification 6, information regarding the state of the working machine 100 is transmitted to the automatic driving information processing device 200 (for example, the second external device 220), and the controller 221 preferably controls the display unit 223 so that information regarding the state of the working machine 100 such as the presence or absence of fuel and the presence or absence of urea is displayed on the display unit 223. Thereby, even at a remote location, the worker can confirm the state of the working machine 100.

[0126] [Second Invention] Next, the second invention will be described.

[0127] [Problems to be Solved by the Second Invention] The second problem in the present disclosure is as follows. That is, it is necessary for a worker such as an operator to repeatedly operate the fine adjustment button to finely adjust the viewpoint of the work site image little by little, which is time-consuming.

[0128] [Means for Solving the Second Problem] The second invention for solving the second problem includes the following automatic driving information processing device, automatic driving system, and automatic driving method.

[0129] [The first aspect of the second invention] An automatic driving information processing device according to the first aspect of the second invention is an automatic driving information processing device for the automatic driving of a working machine, a display unit for displaying a work site image which is an image of the work site, and a controller for controlling the display unit. The controller causes the display unit to display a viewpoint switching input unit that receives an input operation by a worker for switching to the work site image from a specific viewpoint which is a preset viewpoint.

[0130] [The second aspect of the second invention] A work management system according to the second aspect of the second invention includes the automatic driving information processing device, and a working machine.

[0131] [The third aspect of the second invention] A work management method according to the third aspect of the second invention is an automatic driving method for the automatic driving of a working machine, and includes causing a display unit to display a viewpoint switching input unit that receives an input operation by a worker for switching to a work site image from a specific viewpoint which is a preset viewpoint. [Effects according to the first to third aspects of the second invention]

[0132] In the first to third aspects of the second invention, the work site image can be adjusted with a small number of operations by the worker.

[0133] [The third invention] Next, the third invention will be described.

[0134] [Problems to be solved by the third invention] The third problem in the present disclosure is as follows. That is, before the start of autonomous driving, a worker such as an operator needs to perform a complicated input operation to change the on-site image to an image from a perspective where the deviation is easy to check in order to check the deviation of work targets such as the target work area, the target dumping area, and the target route.

[0135] [Means for Solving the Third Problem] The third invention for solving the third problem includes the following autonomous driving information processing device, autonomous driving system, and autonomous driving method.

[0136] [The First Aspect of the Third Invention] The autonomous driving information processing device according to the first aspect of the third invention is an autonomous driving information processing device for the autonomous driving of a work machine, a display unit for displaying a work site image that is an image of the work site, and a controller for controlling the display unit, wherein the controller controls the display unit so that the work site image is switched to an image from a specific perspective preset for checking the deviation of work targets at the work site before the start of the autonomous driving.

[0137] [The Second Aspect of the Third Invention] The autonomous driving system according to the second aspect of the third invention includes the autonomous driving information processing device, and a work machine.

[0138] [The Third Aspect of the Third Invention] The autonomous driving method according to the third aspect of the third invention is an autonomous driving method for the autonomous driving of a work machine, including controlling, by a controller, a display unit so that a work site image is switched to an image from a specific perspective preset for checking the deviation of work targets at the work site before the start of the autonomous driving. [Effects of the First to Third Aspects of the Third Invention] In the first to third aspects of the third invention, before the start of the automatic driving, the controller switches to a work site image from a perspective suitable for a worker to check for a deviation in the work target. Therefore, the worker does not need to perform a complicated input operation to change the perspective of the work site image.

[0139] [Fourth Invention] Next, the fourth invention will be described.

[0140] [Problems to be Solved by the Fourth Invention] The fourth problem in the present disclosure is as follows. That is, when performing a confirmation of an emergency stop while actually executing a specific operation (for example, a specific operation including excavation, lifting and turning, earth discharge, and return turning) preset for performing work by automatic driving of a work machine, before performing this confirmation of the emergency stop, it is necessary to perform teaching of the specific operation.

[0141] [Means for Solving the Fourth Problem] The fourth invention for solving the fourth problem includes the following automatic driving information processing device, automatic driving system, and automatic driving method.

[0142] [First Aspect of the Fourth Invention] An automatic driving information processing device according to the first aspect of the fourth invention is an automatic driving information processing device for automatic driving of a work machine, comprising a controller that controls the work machine so that a specific operation preset at the work site is performed, wherein the controller controls the work machine so that a simple emergency stop confirmation operation is performed before the start of the automatic driving, as compared with the specific operation.

[0143] [Second Aspect of the Fourth Invention] An automatic driving system according to the second aspect of the fourth invention, comprises the automatic driving information processing device, and a work machine.

[0144] [Third Aspect of the Fourth Invention] The automatic driving method according to the third aspect of the fourth invention is an automatic driving method for an operating machine, including the controller controlling the operating machine so that a simple emergency stop confirmation operation is performed before the start of the automatic driving, compared with a predetermined specific operation. [Effects according to the first to third aspects of the fourth invention] In the first to third aspects of the fourth invention, it is possible to confirm an emergency stop even before teaching a specific operation.

[0145] [Fifth invention] Next, the fifth invention will be described.

[0146] [Problems to be solved by the fifth invention] The fifth problem in the present disclosure is as follows. That is, for example, when an operation including a plurality of operation steps such as an excavation step, a lifting and turning step, a soil discharge step, and a return turning step is performed, the controller needs to determine, for example, whether or not the operation has shifted from the excavation step to the lifting and turning step. When the determination of the shift of the operation step is performed based on the operation speed of a movable part such as the turning speed, the accuracy of the determination of the shift of the operation step is not necessarily high.

[0147] [Means for solving the fifth problem] The fifth invention for solving the fifth problem includes the following automatic driving information processing device, automatic driving system, and automatic driving method.

[0148] [The first aspect of the fifth invention] The automatic driving information processing device according to the first aspect of the fifth invention is an automatic driving information processing device for an operating machine, comprising a controller that controls the operating machine so that a plurality of operation steps preset at a work site are performed, wherein the operating machine includes a machine body and a working device supported by the machine body, The plurality of operation steps include a first operation step and a second operation step performed after the first operation step. When a preset specific part in the working device exits from the boundary of the working area corresponding to the first operation step, the controller determines that the process has shifted from the first operation step to the second operation step.

[0149] [Second Aspect of the Fifth Invention] An automatic driving system according to the second aspect of the fifth invention is equipped with the automatic driving information processing device and a working machine.

[0150] [Third Aspect of the Fifth Invention] An automatic driving method according to the third aspect of the fifth invention is an automatic driving method for a working machine, wherein a plurality of operation steps preset at a work site include a first operation step and a second operation step performed after the first operation step. When a preset specific part in the working device of the working machine exits from the boundary of the working area corresponding to the first operation step, the controller determines that the process has shifted from the first operation step to the second operation step. [Effects of the First to Third Aspects of the Fifth Invention] In this aspect, the accuracy of the transition determination can be improved as compared with the case where the transition determination of the operation steps is performed based on the operation speed of the movable part.

Description of Reference Numerals

[0151] 1: Lower Traveling Body 2: Upper Swing Body 3: Working Device 4: Boom 5: Arm 6: Tip Attachment 7: Power Source 8: Hydraulic Pump 9: Cab 10: Actual Machine Operating Device 35: Boom cylinder 36: Arm cylinder 37: Tip attachment cylinder 38: Swing motor 39: Travel motor 100: Working machine 200: Automatic driving information processing device 210: Management device 220: Information terminal 221: Input unit 222: Controller 223: Display unit 300: Automatic driving system 400: Imaging device Z: Swivel axis

Claims

1. An automatic driving information processing device for the automatic driving of a working machine, comprising: a display unit for displaying a work site image which is an image of the work site; a controller for controlling the display unit, wherein the controller executes a process of changing the viewpoint of the work site image according to a plurality of preset operation steps included in the work at the work site. The automatic driving information processing device.

2. The automatic driving information processing device according to claim 1, wherein the controller causes the display unit to display a viewpoint switching input unit that receives an input operation by a worker for switching to the work site image from a specific viewpoint which is a preset viewpoint.

3. The automatic driving information processing device according to claim 2, wherein the input operation is a single operation by the worker.

4. The automatic driving information processing device according to claim 1, wherein the controller controls the display unit so that the work site image is switched to an image from a specific viewpoint preset for checking the deviation of the work target at the work site before the start of the automatic driving.

5. The controller is configured to control the working machine so that a preset specific operation is performed at the work site, The automatic driving information processing device according to claim 1, wherein the controller controls the working machine so that a simple emergency stop confirmation operation is performed before the start of the automatic driving as compared with the specific operation.

6. The working machine includes a machine body, a lifting member supported by the machine body so as to be able to lift, and a tip attachment rotatably supported at the tip of the lifting member. The automatic driving information processing device according to claim 5, wherein the emergency stop confirmation operation is a rotation operation of the tip attachment.

7. The working machine includes a machine body and a working device supported by the machine body. The plurality of operation steps include a first operation step and a second operation step performed next to the first operation step. The automatic driving information processing device according to claim 1, wherein the controller determines that a shift has occurred from the first operation step to the second operation step when a preset specific part of the working device has exited from the boundary of the work area corresponding to the first operation step.

8. An automatic driving system comprising the automatic driving information processing device according to any one of claims 1 to 7, and the working machine.

9. An automatic driving method including changing, by a controller, a viewpoint of a work site image, which is an image of a work site in the automatic driving of a work machine, according to a plurality of preset operation steps included in the work of the work site.

10. An automatic driving program for causing a computer to execute a process of changing a viewpoint of a work site image, which is an image of a work site in the automatic driving of a work machine, according to a plurality of preset operation steps included in the work of the work site.

Images