Work machine, remote operation support device and support system

Abstract

To provide a technique capable of appropriately supporting operation of a work machine by a user.SOLUTION: A shovel 100 according to one embodiment of the present disclosure includes: an attachment AT which is operated to change the shape of the object in the work area of the shovel 100; and a display device 50A for visually proposing a work place where the attachment AT changes the shape of sediment in the work area of the shovel 100, on the basis of information indicating the current shape of the object of the work area of the shovel 100.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present disclosure relates to work machines and the like.

Background Art

[0002] Conventionally, a function for assisting an operator in operating a work machine has been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the machine guidance function of Patent Document 1, only the distance between the target construction surface serving as a standard for work when performing excavation work of an excavator and the working part of the attachment (for example, a bucket) is notified. Therefore, for example, an inexperienced operator may not be able to appropriately determine the location of the work target, such as from which location in the work area to start the excavation work and which location to excavate next.

[0005] Therefore, in view of the above problems, an object is to provide a technology capable of appropriately assisting an operator in operating a work machine.

Means for Solving the Problems

[0006] To achieve the above object, in one embodiment of the present disclosure, a working device that operates to change the shape of an object in the working area of the work machine; a display device that visually proposes a location of a work target where the working device changes the shape of the object in the working area of the work machine based on information representing the current shape of the object in the working area of the work machine, A working machine is provided.

[0007] In another embodiment of the present disclosure, an operation unit for a user to remotely operate a working machine including a working device that operates to change the shape of an object in a working area, a communication unit that transmits information representing the operation state of the operation unit to the working machine, and a display unit that visually proposes a work target location where the working device changes the shape of an object in the working area of the working machine based on information representing the current shape of the object in the working area of the working machine. A remote operation support device is provided.

[0008] In still another embodiment of the present disclosure, a support system for assisting an operation by a user of a working machine including a working device that operates to change the shape of an object in a working area, and a display unit that visually proposes to the user a work target location where the working device changes the shape of an object in the working area of the working machine based on information representing the current shape of the object in the working area of the working machine. A support system is provided.

Advantages of the Invention

[0009] According to the above-described embodiment, it is possible to appropriately support the operation of the working machine by the user.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

[0011] Hereinafter, embodiments will be described with reference to the drawings.

[0012] [Overview of the Operation Support System] With reference to FIGS. 1 to 3, an overview of the operation support system SYS according to this embodiment will be described.

[0013] FIG. 1 is a diagram showing an example of the operation support system SYS. In FIG. 1, the left side view of the excavator 100 is shown. FIG. 2 is a top view showing an example of the excavator 100. FIG. 3 is a diagram showing an example of the configuration related to the remote operation of the excavator 100. Hereinafter, when explaining the direction in the excavator 100 or the direction viewed from the excavator 100, the direction in which the attachment AT extends in the top view of the excavator 100 (the upward direction in FIG. 2) is defined as "front".

[0014] As shown in FIG. 1, the operation support system SYS includes an excavator 100, an information processing device 200, and a sensor group 300.

[0015] The operation support system SYS uses the information processing device 200 to cooperate with the excavator 100 and provide support for the operation of the excavator 100.

[0016] The excavator 100 included in the operation support system SYS may be one or a plurality.

[0017] The excavator 100 is a working machine that is the object of support for operation in the operation support system SYS.

[0018] As shown in FIGS. 1 and 2, the excavator 100 includes a lower traveling body 1, an upper revolving body 3, an attachment AT including a boom 4, an arm 5, and a bucket 6, and a cabin 10.

[0019] The lower traveling body 1 uses the crawlers 1C to move the excavator 100. The crawlers 1C include a left crawler 1CL and a right crawler 1CR. The crawler 1CL is hydraulically driven by a traveling hydraulic motor 1ML. Similarly, the crawler 1CR is hydraulically driven by a traveling hydraulic motor 1MR. Thereby, the lower traveling body 1 can travel by itself.

[0020] The upper revolving body 3 is mounted on the lower traveling body 1 via a slewing mechanism 2 so as to be slewing-capable (slewing-free). For example, the upper revolving body 3 slews with respect to the lower traveling body 1 when the slewing mechanism 2 is hydraulically driven by a slewing hydraulic motor 2M.

[0021] The boom 4 is attached to the center of the front part of the upper revolving body 3 so as to be capable of pitching about a rotation axis along the left-right direction. The arm 5 is attached to the tip of the boom 4 so as to be rotatable about a rotation axis along the left-right direction. The bucket 6 is attached to the tip of the arm 5 so as to be rotatable about a rotation axis along the left-right direction.

[0022] The bucket 6 is an example of an end attachment and is used, for example, in excavation work, slope work, leveling work, etc.

[0023] The bucket 6 is attached to the tip of the arm 5 in a manner that can be appropriately replaced according to the work content of the excavator 100. That is, instead of the bucket 6, a bucket of a different type from the bucket 6, for example, a relatively large large bucket, a slope bucket, a dredging bucket, etc. may be attached to the tip of the arm 5. Also, an end attachment of a type other than the bucket, for example, a stirrer, a breaker, a crusher, etc. may be attached to the tip of the arm 5. Further, a preliminary attachment such as a quick coupler or a tilt rotator may be provided between the arm 5 and the end attachment.

[0024] The boom 4, the arm 5, and the bucket 6 are each hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0025] The cab 10 is an operator's cab for an operator to board and operate the excavator 100. The cab 10 is mounted, for example, on the left front side of the upper revolving body 3.

[0026] The excavator 100 is equipped with a communication device 60 and can communicate with external devices such as an information processing device 200 and a sensor group 300 through a predetermined communication line NW.

[0027] The communication line NW includes, for example, a local area network (LAN) at the work site. The communication line NW may also include a wide area network (WAN). The wide area network includes, for example, a mobile communication network with a base station as an end point, a satellite communication network using communication satellites, the Internet, etc. The communication line NW may also include a short-distance communication line based on a wireless communication standard such as WiFi or Bluetooth (registered trademark).

[0028] For example, the excavator 100 operates driven elements such as the lower traveling body 1 (i.e., a pair of left and right crawlers 1CL, 1CR), the upper slewing body 3, the boom 4, the arm 5, and the bucket 6 according to the operation of an operator boarding the cab 10.

[0029] Alternatively, or in addition to being configured to be operable by an operator boarding the cab 10, the excavator 100 may be configured to be remotely operable from outside the excavator 100. When the excavator 100 is remotely operated, the inside of the cab 10 may be unmanned. Also, when the excavator 100 is dedicated to remote operation, the cab 10 may be omitted. Hereinafter, the description will proceed on the premise that the operator's operation includes at least one of the operation on the operation device 26 of the operator in the cab 10 and the remote operation of an external operator.

[0030] For example, as shown in FIG. 3, the remote operation includes a mode in which the excavator 100 is operated by an operation input regarding an actuator of the excavator 100 performed by a remote operation support device 400 capable of communicating with the excavator 100 through the communication line NW. The remote operation support device 400 may be provided separately from the information processing device 200 or may be the information processing device 200.

[0031] The remote operation support device 400 is provided, for example, in a management center or the like that manages the operation of the excavator 100 from the outside. Further, the remote operation support device 400 may be a portable operation terminal. In this case, the operator can remotely operate the excavator 100 while directly checking the working status of the excavator 100 from around the excavator 100.

[0032] The excavator 100 may transmit, for example, an image (hereinafter, "surrounding image") representing the state of the surroundings including the front of the excavator 100 based on the captured image output by the imaging device mounted on itself to the remote operation support device 400 through the communication device 60. Further, the excavator 100 may transmit the captured image output by the imaging device to the remote operation support device 400 through the communication device 60, and the remote operation support device 400 may process the captured image received from the excavator 100 to generate a surrounding image. Then, the remote operation support device 400 may display a surrounding image representing the state of the surroundings including the front of the excavator 100 on its own display device. Also, various information images (information screens) displayed on the output device 50 (display device 50A) inside the cab 10 of the excavator 100 may be similarly displayed on the display device of the remote operation support device 400. Thereby, the operator using the remote operation support device 400 can remotely operate the excavator 100 while checking the display contents such as the image and information screen representing the state of the surroundings of the excavator 100 displayed on the display device. Then, the excavator 100 may operate the actuator according to the remote operation signal representing the content of the remote operation received from the remote operation support device 400 by the communication device 60, and drive driven elements such as the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, and the bucket 6.

[0033] Further, remote operation may include a mode in which the excavator 100 is operated, for example, by external voice input or gesture input from a person (e.g., an operator) around the excavator 100 to the excavator 100. Specifically, the excavator 100 recognizes voices spoken by surrounding workers or gestures made by workers, etc. through a voice input device (e.g., a microphone) or a gesture input device (e.g., an imaging device) mounted on itself. Then, the excavator 100 may operate an actuator according to the content of the recognized voice, gesture, etc., and drive driven elements such as the lower traveling body 1 (i.e., crawlers 1CL, 1CR), the upper slewing body 3, the boom 4, the arm 5, and the bucket 6.

[0034] Further, the excavator 100 may automatically operate the actuator regardless of the content of the operator's operation. Thereby, the excavator 100 can realize a function of automatically operating at least a part of driven elements such as the lower traveling body 1, the upper slewing body 3, and the attachment AT, that is, a so-called "automatic driving function" or "Machine Control (MC) function".

[0035] The automatic driving function includes, for example, a semi-automatic driving function (operation support type MC function). The semi-automatic driving function is a function that automatically operates a driven element (actuator) other than the driven element (actuator) of the operation target in response to the operation of the operator. Further, the automatic driving function may include a fully automatic driving function (fully automatic type MC function). The fully automatic driving function is a function that automatically operates at least a part of a plurality of driven elements (hydraulic actuators) on the premise that there is no operation by the operator. In the excavator 100, when the fully automatic driving function is valid, the inside of the cab 10 may be unmanned. Further, when the excavator 100 is dedicated to fully automatic driving, the cab 10 may be omitted. Further, the semi-automatic driving function, the fully automatic driving function, etc. include, for example, a rule-based automatic driving function. The rule-based automatic driving function is an automatic driving function in a mode in which the operation content of the driven element (actuator) of the automatic driving target is automatically determined according to a rule defined in advance. Further, the semi-automatic driving function, the fully automatic driving function, etc. may include an autonomous driving function. The autonomous driving function is an automatic driving function in a mode in which the excavator 100 autonomously makes various judgments, and the operation content of the driven element (hydraulic actuator) of the automatic driving target is determined according to the judgment result.

[0036] Further, the work of the excavator 100 may be remotely monitored. In this case, a remote monitoring support device having the same function as the remote operation support device 400 may be provided. The remote monitoring support device is, for example, the information processing device 200. Thereby, a monitor who is a user of the remote monitoring support device can monitor the work situation of the excavator 100 while checking the surrounding image displayed on the display device of the remote monitoring support device. Further, for example, when the monitor determines that it is necessary from the viewpoint of safety, the monitor can intervene in the operation or automatic driving by the operator of the excavator 100 and stop the excavator 100 urgently by making a predetermined input using the input device of the remote monitoring support device.

[0037] The information processing device 200 communicates with the excavator 100 to cooperate with each other and supports the operation of the excavator 100.

[0038] The information processing device 200 is, for example, a server device or a management terminal device installed in a management office within the work site of the excavator 100, or in a management center or the like that manages the operating status of the excavator 100 at a location different from the work site of the excavator 100. The server device may be an on-premises server, a cloud server, or an edge server. The management terminal device may be, for example, a stationary terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile terminal) such as a tablet terminal, a smartphone, or a laptop PC. In the latter case, workers at the work site, supervisors who supervise the work, managers who manage the work site, etc. can carry the portable information processing device 200 and move within the work site. Also, in the latter case, the operator can, for example, bring the portable information processing device 200 into the cab of the excavator 100.

[0039] The information processing device 200 acquires data regarding the operating state from the excavator 100, for example. Thereby, the information processing device 200 can grasp the operating state of the excavator 100 and monitor the presence or absence of abnormalities in the excavator 100. Also, the information processing device 200 can display data regarding the operating state of the excavator 100 through, for example, the display device 208 described later, and allow the user to confirm it. Also, the information processing device 200 can, for example, cause the operating state of the excavator 100 to be learned by a learning model and generate a learned model for assisting the operation of the excavator 100.

[0040] Also, the information processing device 200 may transmit various data such as programs and reference data used in the processing of the excavator 100, such as the controller 30, to the excavator 100. Thereby, the excavator 100 can perform various processes related to the operation of the excavator 100 using the various data downloaded from the information processing device 200.

[0041] The sensor group 300 is provided at the work site of the excavator 100.

[0042] For example, when a plurality of excavators 100 are included in the operation support system SYS, a sensor group 300 is provided for each excavator 100. Further, when the plurality of excavators 100 included in the operation support system SYS work at the same work site, one sensor group 300 may be shared for the plurality of excavators 100.

[0043] The sensor group 300 includes sensors 300-1 to 300-M (M: an integer of 2 or more). The sensors 300-1 to 300-M measure the state of an object at the work site around the excavator 100 and acquire measurement data regarding the state. The objects at the work site include the objects in the work area around the excavator 100. The objects in the work area of the excavator 100 are, for example, earth and sand in the work area around the excavator 100. Further, the objects at the work site include, in addition to the objects (i.e., earth and sand in the work area) in the work area around the excavator 100, obstacles around the excavator 100. The obstacles around the excavator 100 include, for example, other excavators around the excavator 100, work machines such as bulldozers, and work vehicles such as trucks for transporting earth and sand. The state of the object includes the shape and characteristics of the object.

[0044] The sensors 300-1 to 300-M include, for example, distance measuring sensors (range sensors). The distance measuring sensors include, for example, LIDAR (Light Detecting and Ranging), millimeter wave radar, ultrasonic sensors, infrared sensors, etc. Further, the sensors 300-1 to 300-M may include a 3D camera capable of acquiring data regarding distance (depth) in addition to a two-dimensional image, such as a stereo camera, a TOF (Time Of Flight) camera, etc. Also, the sensors 300-1 to 300-M may include a mixture of distance measuring sensors and 3D cameras. Thereby, the sensor group 300 can acquire measurement data representing the shape of an object at the work site around the excavator 100. Hereinafter, sensors capable of acquiring measurement data representing the shape of an object, such as distance measuring sensors and 3D cameras, may be referred to as "shape sensors" for convenience.

[0045] Further, the sensors 300-1 to 300-M may include a multi-wavelength spectroscopic camera. The multi-wavelength spectroscopic camera includes, for example, a multispectral camera, a hyperspectral camera, and the like. Thereby, for example, the sensor group 300 can acquire measurement data representing the characteristics of an object at the work site around the excavator 100, such as the hardness and moisture content of earth and sand. Hereinafter, a sensor capable of acquiring measurement data representing the characteristics of an object, such as a multi-wavelength spectroscopic camera, may be conveniently referred to as a "characteristic sensor".

[0046] For example, the sensors 300-1 to 300-M include a plurality of shape sensors. The plurality of shape sensors may be provided at different locations at the work site around the excavator 100, and the sensing range of each may overlap at least with the sensing range of one other shape sensor. Thereby, for example, even if occlusion occurs in the measurement data of one shape sensor and measurement data representing the shape of an object in a part of the sensing range cannot be acquired, the other shape sensor may be able to acquire measurement data representing the shape of the object in that range. Therefore, the sensor group 300 can more reliably acquire measurement data representing the shape of an object at the work site around the excavator 100.

[0047] Further, the sensors 300-1 to 300-M may include a plurality of characteristic sensors. The plurality of characteristic sensors may be provided at different locations at the work site around the excavator 100, and the sensing range of each may overlap at least with that of one other characteristic sensor. Thereby, for example, even if occlusion occurs in the measurement data of one characteristic sensor and measurement data representing the characteristics of an object in a part of the sensing range cannot be acquired, the other shape sensor may be able to acquire measurement data representing the characteristics of the object in that range. Therefore, the sensor group 300 can more reliably acquire measurement data representing the characteristics of an object at the work site around the excavator 100.

[0048] Further, the sensors 300-1 to 300-M may include sensors having both the functions of the shape sensor and the characteristic sensor (hereinafter, "integrated sensors"). In this case, the sensors 300-1 to 300-M may include a plurality of integrated sensors. And the plurality of characteristic sensors may be provided at different locations around the work site of the excavator 100 and with the sensing range of each overlapping at least one other characteristic sensor.

[0049] In addition, the sensor group 300 may simply include only one shape sensor or characteristic sensor. Also, the operation support system SYS may simply include only one sensor capable of acquiring measurement data regarding the state of an object at the work site around the excavator 100 instead of the sensor group 300.

[0050] The sensors 300-1 to 300-M may be fixed at the work site around the excavator 100, or may be mounted on a moving body movable within the work site around the excavator 100. The moving body includes, for example, a working machine or a work vehicle that moves within the work site. Also, the moving body movable within the work site may include, for example, a flying body such as a drone that flies over the work site.

[0051] The output (measurement data) of the sensors 300-1 to 300-M is taken into the information processing device 200 through the communication line NW. The output of the sensors 300-1 to 300-M is directly taken into the information processing device 200 through the communication line NW, for example. Also, the output of the sensors 300-1 to 300-M may be taken into the excavator 100 once through the communication line NW and then taken into the information processing device 200 via the excavator 100. Further, when the sensors 300-1 to 300-M are mounted on a predetermined device such as the above-mentioned moving body, the output of the sensors 300-1 to 300-M may be taken into the predetermined device once and then taken into the information processing device 200 from the device.

[0052] In addition, the sensor group 300 may be omitted.

[0053] [Configuration of the Operation Support System] Next, in addition to FIGS. 1 to 3, with reference to FIGS. 4 and 5, the hardware configuration of the operation support system SYS will be described.

[0054] [Configuration of the Excavator] FIG. 4 is a diagram showing an example of the configuration of the excavator 100.

[0055] In FIG. 4, the path through which mechanical power is transmitted is shown by a double line, the path through which high-pressure hydraulic oil for driving the hydraulic actuator flows is shown by a solid line, the path through which the pilot pressure is transmitted is shown by a broken line, and the path through which the electric signal is transmitted is shown by a dotted line, respectively.

[0056] The excavator 100 includes respective components such as a hydraulic drive system related to the hydraulic drive of the driven elements, an operation system related to the operation of the driven elements, a user interface system related to the information exchange with the user, a communication system related to the communication with the outside, and a control system related to various controls.

[0057] ≪Hydraulic Drive System≫ As shown in FIG. 4, the hydraulic drive system of the excavator 100 includes, as described above, hydraulic actuators HA that hydraulically drive each of the driven elements such as the lower traveling body 1 (left, i.e., crawlers 1CL, 1CR), the upper revolving body 3, the boom 4, the arm 5, and the bucket 6. Further, the hydraulic drive system of the excavator 100 according to the present embodiment includes an engine 11, a regulator 13, a main pump 14, and a control valve 17.

[0058] The hydraulic actuators HA include traveling hydraulic motors 1ML, 1MR, a swing hydraulic motor 2M, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, etc.

[0059] Note that in the excavator 100, part or all of the hydraulic actuators HA may be replaced with electric actuators. That is, the excavator 100 may be a hybrid excavator or an electric excavator.

[0060] The engine 11 is the prime mover of the excavator 100 and is the main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, at the rear of the upper swing body 3. The engine 11 rotates at a constant speed at a preset target rotational speed under the direct or indirect control of a controller 30 described later, and drives the main pump 14 and the pilot pump 15.

[0061] Note that instead of or in addition to the engine 11, other prime movers (for example, an electric motor) etc. may be mounted on the excavator 100.

[0062] The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate (hereinafter, "tilt angle") of the main pump 14 in response to a control command from the controller 30.

[0063] The main pump 14 supplies hydraulic oil to the control valve 17 through the high-pressure hydraulic line. The main pump 14 is mounted, for example, at the rear of the upper swing body 3, similar to the engine 11. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump. As described above, the stroke length of the piston is adjusted by adjusting the tilt angle of the swash plate by the regulator 13 under the control of the controller 30, and the discharge flow rate and discharge pressure are controlled.

[0064] The control valve 17 drives the hydraulic actuator HA according to the operations on the operator's operating device 26, the content of remote operations, or the operation commands corresponding to the automatic driving function. The control valve 17 is mounted, for example, at the center of the upper swing body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line, and selectively supplies the hydraulic oil supplied from the main pump 14 to each hydraulic actuator according to the operations of the operator or the operation commands corresponding to the automatic driving function. Specifically, the control valve 17 includes a plurality of control valves (also referred to as "direction change valves") that control the flow rate and flow direction of the hydraulic oil supplied from the main pump 14 to each of the hydraulic actuators HA.

[0065] <<Operation system>> As shown in FIG. 4, the operation system of the excavator 100 includes a pilot pump 15, an operating device 26, a hydraulic control valve 31, a shuttle valve 32, and a hydraulic control valve 33.

[0066] The pilot pump 15 supplies pilot pressure to various hydraulic devices via the pilot line 25. The pilot pump 15 is mounted, for example, at the rear of the upper swing body 3, similar to the engine 11. The pilot pump 15 is, for example, a fixed-displacement hydraulic pump and is driven by the engine 11 as described above.

[0067] Incidentally, the pilot pump 15 may be omitted. In this case, the relatively low-pressure hydraulic oil after the relatively high-pressure hydraulic oil discharged from the main pump 14 is decompressed by a predetermined decompression valve may be supplied as the pilot pressure to various hydraulic devices.

[0068] The operating device 26 is provided near the driver's seat in the cabin 10 and is used for the operator to operate various driven elements. Specifically, the operating device 26 is used for the operator to operate the hydraulic actuators HA that drive the respective driven elements. As a result, the operator can operate the driven elements driven by the hydraulic actuators HA. The operating device 26 includes a pedal device and a lever device for operating the respective driven elements (hydraulic actuators HA).

[0069] For example, as shown in FIG. 4, the operating device 26 is of a hydraulic pilot type. Specifically, the operating device 26 utilizes the hydraulic oil supplied from the pilot pump 15 through the pilot line 25 and the pilot line 25A branched therefrom, and outputs a pilot pressure corresponding to the operation content to the secondary pilot line 27A. The pilot line 27A is connected to one inlet port of the shuttle valve 32 and is connected to the control valve 17 via the pilot line 27 connected to the outlet port of the shuttle valve 32. Thereby, a pilot pressure corresponding to the operation content regarding various driven elements (hydraulic actuators HA) in the operating device 26 can be input to the control valve 17 via the shuttle valve 32. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26 by the operator or the like.

[0070] Also, the operating device 26 may be electric. In this case, the pilot line 27A, the shuttle valve 32, and the hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electric signal (hereinafter, "operation signal") corresponding to the operation content, and the operation signal is taken into the controller 30. Then, the controller 30 outputs a control command corresponding to the content of the operation signal, that is, a control signal corresponding to the operation content of the operating device 26, to the hydraulic control valve 31. Thereby, a pilot pressure corresponding to the operation content of the operating device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26.

[0071] Also, the control valve (direction switching valve) that drives each hydraulic actuator HA incorporated in the control valve 17 may be an electromagnetic solenoid type. In this case, the operation signal output from the operation device 26 may be directly input to the control valve 17 (that is, to the electromagnetic solenoid type control valve).

[0072] Also, as described above, part or all of the hydraulic actuator HA may be replaced with an electric actuator. In this case, the controller 30 may output a control command corresponding to the operation content of the operation device 26 or the content of the remote operation defined by the remote operation signal to the electric actuator or a driver that drives the electric actuator. Further, when the excavator 100 is remotely operated, the operation device 26 may be omitted.

[0073] The hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) to be operated by the operation device 26 and for each driving direction (for example, the raising direction and the lowering direction of the boom 4) of the driven element (hydraulic actuator HA). For example, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA for driving the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, the bucket 6, and the like. The hydraulic control valve 31 is provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and may be configured to be able to change its flow passage area (that is, the cross-sectional area through which the hydraulic oil can flow). Thereby, the hydraulic control valve 31 can output a predetermined pilot pressure to the secondary pilot line 27B by using the hydraulic oil of the pilot pump 15 supplied through the pilot line 25B. Therefore, the hydraulic control valve 31 can indirectly apply a predetermined pilot pressure corresponding to the control signal from the controller 30 to the control valve 17 through the shuttle valve 32 between the pilot line 27B and the pilot line 27. Thus, for example, the controller 30 can supply a pilot pressure corresponding to an operation command corresponding to the automatic operation function from the hydraulic control valve 31 to the control valve 17, and realize the operation of the excavator 100 by the automatic operation function.

[0074] Further, the controller 30 may control the hydraulic control valve 31 to realize remote operation of the excavator 100. Specifically, the controller 30 outputs, to the hydraulic control valve 31, a control signal corresponding to the content of the remote operation specified by the remote operation signal received from the remote operation support device 400 by the communication device 60. Thereby, the controller 30 can cause the hydraulic control valve 31 to supply a pilot pressure corresponding to the content of the remote operation to the control valve 17, and realize the operation of the excavator 100 based on the remote operation of the operator.

[0075] Also, when the operation device 26 is electric, the controller 30 can directly supply a pilot pressure corresponding to the operation content (operation signal) of the operation device 26 from the hydraulic control valve 31 to the control valve 17, and realize the operation of the excavator 100 based on the operation of the operator.

[0076] The shuttle valve 32 has two inlet ports and one outlet port, and outputs the hydraulic oil having the higher pilot pressure among the pilot pressures input to the two inlet ports to the outlet port. Similar to the hydraulic control valve 31, the shuttle valve 32 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 and for each driving direction of the driven element (hydraulic actuator HA). For example, two shuttle valves 32 are provided for each double-acting hydraulic actuator HA for driving the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, the bucket 6, and the like. One of the two inlet ports of the shuttle valve 32 is connected to the secondary pilot line 27A of the operating device 26 (specifically, the above-mentioned lever device or pedal device included in the operating device 26), and the other is connected to the secondary pilot line 27B of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of the corresponding control valve of the control valve 17 through the pilot line 27. The corresponding control valve is the control valve that drives the hydraulic actuator HA which is the operation target of the above-mentioned lever device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, each of these shuttle valves 32 can apply the higher one of the pilot pressure of the secondary pilot line 27A of the operating device 26 and the pilot pressure of the secondary pilot line 27B of the hydraulic control valve 31 to the pilot port of the corresponding control valve. That is, the controller 30 can control the corresponding control valve without depending on the operation of the operator on the operating device 26 by outputting a pilot pressure higher than the secondary pilot pressure of the operating device 26 from the hydraulic control valve 31. Therefore, the controller 30 can control the operation of the driven elements (lower traveling body 1, upper slewing body 3, boom 4, arm 5, bucket 6) regardless of the operation state of the operator on the operating device 26, and can realize the automatic operation function and the remote operation function.

[0077] The hydraulic control valve 33 is provided in a pilot line 27A that connects the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured, for example, to be able to change its flow passage area. The hydraulic control valve 33 operates in response to a control signal input from the controller 30. Thereby, when the operating device 26 is being operated by the operator, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator HA corresponding to the operation of the operating device 26. Further, the controller 30 can, for example, reduce the pilot pressure output from the operating device 26 even when the operating device 26 is being operated, and make it lower than the pilot pressure output from the hydraulic control valve 31. Therefore, by controlling the hydraulic control valve 31 and the hydraulic control valve 33, the controller 30 can surely cause a desired pilot pressure to act on the pilot port of the control valve in the control valve 17, for example, regardless of the operation content of the operating device 26. Thus, the controller 30 can more appropriately realize the automatic operation function and the remote operation function of the excavator 100, for example, by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31.

[0078] ≪User Interface System≫ As shown in FIG. 4, the user interface system of the excavator 100 includes an operating device 26, an output device 50, and an input device 52.

[0079] The output device 50 outputs various information to the user of the excavator 100 (for example, the operator in the cab 10 or the operator of the external remote operation) and the people around the excavator 100 (for example, the worker or the driver of the work vehicle).

[0080] For example, the output device 50 includes lighting equipment and a display device 50A (see FIG. 6) that output various types of information in a visual manner. The lighting equipment is, for example, a warning light (indicator lamp) or the like. The display device 50A is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like. For example, as shown in FIG. 2, the lighting equipment and the display device 50A may be provided inside the cabin 10 and output various types of information to an operator or the like inside the cabin 10 in a visual manner. Further, the lighting equipment and the display device 50A may be provided, for example, on the side surface of the upper swing body 3 or the like and output various types of information to workers or the like around the excavator 100 in a visual manner.

[0081] In addition, the output device 50 may include a sound output device that outputs various types of information in an auditory manner. The sound output device includes, for example, a buzzer, a speaker, or the like. The sound output device is provided, for example, on at least one of the inside and outside of the cabin 10 and may output various types of information to an operator inside the cabin 10 or a person (such as a worker) around the excavator 100 in an auditory manner.

[0082] In addition, the output device 50 may include a device that outputs various types of information in a tactile manner such as vibration of the operator's seat.

[0083] The input device 52 receives various inputs from the user of the excavator 100, and a signal corresponding to the received input is taken into the controller 30. For example, as shown in FIG. 2, the input device 52 is provided inside the cabin 10 and receives inputs from an operator or the like inside the cabin 10. Further, the input device 52 may be provided, for example, on the side surface of the upper swing body 3 or the like and receive inputs from workers or the like around the excavator 100.

[0084] For example, the input device 52 includes a mechanical input device that receives an input by a mechanical operation from the user. The mechanical input device may include a touch panel mounted on the display device 50A, a touch pad installed around the display device 50A, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), or the like.

[0085] In addition, the input device 52 may include a voice input device that receives a user's voice input. The voice input device includes, for example, a microphone.

[0086] In addition, the input device 52 may include a gesture input device that receives a user's gesture input. The gesture input device includes, for example, an imaging device that images the state of the gesture performed by the user.

[0087] In addition, the input device 52 may include a biological input device that receives a user's biological input. The biological input includes, for example, the input of biological information such as the user's fingerprint and iris.

[0088] <<Communication system>> As shown in FIG. 4, the communication system of the excavator 100 according to the present embodiment includes a communication device 60.

[0089] The communication device 60 is connected to an external communication line NW and communicates with a device provided separately from the excavator 100. The device provided separately from the excavator 100 includes, in addition to the devices outside the excavator 100, a portable terminal device (portable terminal) brought into the cab 10 by the user of the excavator 100. The communication device 60 includes, for example, a mobile communication module compliant with standards such as 4G (4 th Generation) and 5G (5 th Generation). In addition, the communication device 60 may include, for example, a satellite communication module. In addition, the communication device 60 may include, for example, a WiFi communication module, a Bluetooth (registered trademark) communication module, or the like. In addition, when there are a plurality of connectable communication lines NW, the communication device 60 may include a plurality of communication devices according to the type of the communication line NW.

[0090] For example, the communication device 60 communicates with external devices such as the information processing device 200 and the remote operation support device 400 within the work site through a local communication line constructed at the work site. The local communication line is, for example, a mobile communication line based on local 5G (so-called local 5G) constructed at the work site or a local network based on WiFi6.

[0091] In addition, the communication device 60 may communicate with the information processing device 200, the sensor group 300, the remote operation support device 400, etc. outside the work site through a wide-area communication line including the work site, that is, a wide-area network.

[0092] ≪Control system≫ As shown in FIG. 4, the control system of the excavator 100 includes a controller 30. In addition, the control system of the excavator 100 according to the present embodiment includes an operation pressure sensor 29, a sensor 40, and sensors S1 to S9.

[0093] The controller 30 performs various controls related to the excavator 100.

[0094] The functions of the controller 30 may be realized by any hardware, or any combination of hardware and software, etc. For example, as shown in FIG. 3, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D connected by a bus BS1.

[0095] The auxiliary storage device 30A is a non-volatile storage means, stores the installed programs, and stores necessary files, data, etc. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, or the like.

[0096] When there is an instruction to start a program, for example, the memory device 30B loads the program in the auxiliary storage device 30A so that the CPU 30C can read it. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

[0097] The CPU 30C executes, for example, the program loaded in the memory device 30B and realizes various functions of the controller 30 according to the instructions of the program.

[0098] The interface device 30D functions as a communication interface for connecting to the internal communication line of the excavator 100, for example. The interface device 30D may include a plurality of different types of communication interfaces according to the type of the communication line to be connected.

[0099] Also, the interface device 30D functions as an external interface for reading data from a recording medium and writing data to the recording medium. The recording medium is, for example, a dedicated tool connected by a cable detachable from a connector installed inside the cab 10. Also, the recording medium may be a general-purpose recording medium such as an SD memory card or a USB (Universal Serial Bus) memory. Thereby, the program that realizes various functions of the controller 30 can be provided by a portable recording medium, for example, and installed in the auxiliary storage device 30A of the controller 30. Also, the program may be downloaded from another computer (for example, the information processing device 200) outside the excavator 100 through the communication device 60 and installed in the auxiliary storage device 30A.

[0100] Note that part of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers mounted on the excavator 100.

[0101] The operation pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot-operated device 26, that is, the pilot pressure corresponding to the operation state of each driven element (hydraulic actuator) in the operating device 26. The detection signal of the pilot pressure corresponding to the operation state regarding each driven element (hydraulic actuator HA) in the operating device 26 by the operation pressure sensor 29 is taken into the controller 30.

[0102] Incidentally, when the operating device 26 is electric, the operation pressure sensor 29 is omitted. This is because the controller 30 can grasp the operation state of each driven element through the operating device 26 based on the operation signal taken in from the operating device 26.

[0103] The sensor 40 acquires measurement data regarding the state of an object around the excavator 100, for example.

[0104] For example, the sensor 40 is a shape sensor capable of acquiring measurement data representing the shape of an object around the excavator 100, such as a distance measuring sensor or an imaging device. Further, in addition to the function of the shape sensor, the sensor 40 may be an integrated sensor having the function of a characteristic sensor capable of acquiring measurement data representing the characteristics of an object around the excavator 100, such as a multi-wavelength spectroscopic camera.

[0105] For example, as shown in FIG. 2, the sensor 40 includes sensors 40F, 40B, 40L, and 40R. The sensor 40F measures the state (shape and characteristics) of an object in front of the upper swing body 3. The sensor 40B measures the state of an object behind the upper swing body 3. The sensor 40L measures the state of an object to the left of the upper swing body 3. The sensor 40R measures the state of an object to the right of the upper swing body 3. Thereby, the sensor 40 can measure the state of an object in the entire circumference, that is, in the range of 360 degrees in the angular direction centering on the excavator 100 in the top view of the excavator 100. Hereinafter, any one of the sensors 40F, 40B, 40L, and 40R may be individually referred to as "sensor 40X".

[0106] The output data of sensor 40X (i.e., the measurement data regarding the state of the objects around the excavator 100) is taken into the controller 30 through a one-to-one communication line or an in-vehicle network. Thereby, for example, the controller 30 can grasp the state such as the shape and characteristics of the objects around the excavator 100 based on the output data of the sensor 40X.

[0107] Note that some or all of the sensors 40B, 40L, and 40R may be omitted.

[0108] Sensor S1 is attached to the boom 4 and measures the attitude state of the boom 4. Sensor S1 outputs measurement data representing the attitude state of the boom 4. The attitude state of the boom 4 is, for example, the attitude angle (hereinafter, "boom angle") around the rotation axis of the base end corresponding to the connecting portion of the boom 4 with the upper swing body 3. Sensor S1 includes, for example, a rotary potentiometer, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), etc. The same may apply to sensors S2 to S4 below. Further, sensor S1 may include a cylinder sensor that detects the extension and retraction position of the boom cylinder 7. The same may apply to sensors S2 and S3 below. The output of sensor S1 (measurement data representing the attitude state of the boom 4) is taken into the controller 30. Thereby, the controller 30 can grasp the attitude state of the boom 4.

[0109] Sensor S2 is attached to the arm 5 and measures the attitude state of the arm 5. Sensor S2 outputs measurement data representing the attitude state of the arm 5. The attitude state of the arm 5 is, for example, the attitude angle (hereinafter, "arm angle") around the rotation axis of the base end corresponding to the connecting portion of the arm 5 with the boom 4. The output of sensor S2 (measurement data representing the attitude state of the arm 5) is taken into the controller 30. Thereby, the controller 30 can grasp the attitude state of the arm 5.

[0110] Sensor S3 is attached to the bucket 6 and measures the attitude state of the bucket 6. Sensor S3 outputs measurement data representing the attitude state of the bucket 6. The attitude state of the bucket 6 is, for example, the attitude angle (hereinafter, "arm angle") around the rotation axis of the base end corresponding to the connection part of the bucket 6 with the arm 5. The output of sensor S3 (measurement data representing the attitude state of the bucket 6) is taken into the controller 30. Thereby, the controller 30 can grasp the attitude state of the bucket 6.

[0111] Sensor S4 measures the attitude state of the body of the excavator 100 (for example, the upper swing body 3). Sensor S4 outputs measurement data representing the attitude state of the body of the excavator 100. The attitude state of the body of the excavator 100 is, for example, the inclination state of the body with respect to a predetermined reference plane (for example, the horizontal plane). For example, sensor S4 is attached to the upper swing body 3 and measures the inclination angles (hereinafter, "front - rear inclination angle" and "left - right inclination angle") around two axes in the front - rear direction and the left - right direction of the excavator 100. The output of sensor S4 (measurement data representing the attitude state of the body of the excavator 100) is taken into the controller 30. Thereby, the controller 30 can grasp the attitude state (inclination state) of the body (upper swing body 3).

[0112] Sensor S5 is attached to the upper swing body 3 and measures the turning state of the upper swing body 3. Sensor S5 outputs measurement data representing the turning state of the upper swing body 3. Sensor S5 measures, for example, the turning angular velocity and the turning angle of the upper swing body 3. Sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, etc. The output of sensor S5 (measurement data representing the turning state of the upper swing body 3) is taken into the controller 30. Thereby, the controller 30 can grasp the turning state such as the turning angle of the upper swing body 3.

[0113] Based on the outputs of sensors S1 to S5, the controller 30 can grasp (estimate) the position of the tip (bucket 6) of the attachment AT.

[0114] Further, when the sensor S4 includes a gyro sensor, a six-axis sensor, an IMU, etc. capable of detecting the angular velocity around three axes, the turning state (e.g., turning angular velocity) of the upper slewing body 3 may be detected based on the detection signal of the sensor S4. In this case, the sensor S5 may be omitted.

[0115] The sensor S6 measures the position of the excavator 100. The sensor S6 may measure the position in world (global) coordinates or may measure the position in local coordinates at the work site. In the former case, the sensor S6 is, for example, a GNSS (Global Navigation Satellite System) sensor. In the latter case, the sensor S6 is a transceiver capable of communicating with a device serving as a reference for the position at the work site and outputting a signal corresponding to the position relative to the reference. The output of the sensor S6 is taken into the controller 30.

[0116] The sensor S7 measures the pressure (cylinder pressure) in the oil chamber of the boom cylinder 7. The sensor S7 includes, for example, a sensor that measures the cylinder pressure (rod pressure) in the rod-side oil chamber of the boom cylinder 7 and a sensor that measures the cylinder pressure (bottom pressure) in the bottom-side oil chamber. The output of the sensor S7 (measurement data of the cylinder pressure of the boom cylinder 7) is taken into the controller 30.

[0117] The sensor S8 measures the pressure (cylinder pressure) in the oil chamber of the arm cylinder 8. The sensor S8 includes, for example, a sensor that measures the cylinder pressure (rod pressure) in the rod-side oil chamber of the arm cylinder 8 and a sensor that measures the cylinder pressure (bottom pressure) in the bottom-side oil chamber. The output of the sensor S8 (measurement data of the cylinder pressure of the arm cylinder 8) is taken into the controller 30.

[0118] The sensor S9 measures the pressure (cylinder pressure) in the oil chamber of the bucket cylinder 9. The sensor S9 includes, for example, a sensor that measures the cylinder pressure (rod pressure) in the oil chamber on the rod side of the bucket cylinder 9 and a sensor that measures the cylinder pressure (bottom pressure) in the oil chamber on the bottom side of the bucket cylinder 9. The output of the sensor S9 (measurement data of the cylinder pressure of the bucket cylinder 9) is taken into the controller 30.

[0119] Based on the outputs of the sensors S7 to S9, the controller 30 can grasp the load state acting on the attachment AT. The load acting on the attachment AT includes, for example, the reaction force acting on the bucket 6 from an object in the work area (e.g., soil and sand on the ground) and the weight of the soil and sand contained in the bucket 6.

[0120] Note that some or all of the sensors S1 to S9 may be omitted depending on necessity. Further, in addition to the sensors S1 to S9, the excavator 100 may be equipped with other sensors capable of grasping the state of the excavator 100. For example, the excavator 100 may be provided with an orientation sensor capable of detecting its own orientation. The orientation sensor is, for example, an electronic compass including a geomagnetic sensor.

[0121] <Configuration of the information processing device> FIG. 5 is a diagram showing an example of the configuration of the information processing device 200.

[0122] The functions of the information processing device 200 are realized by any hardware or any combination of hardware and software, etc. For example, as shown in FIG. 5, the information processing device 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a high-speed arithmetic device 205, a communication interface 206, an input device 207, a display device 208, and an audio output device 209. These are connected by a bus BS2.

[0123] The external interface 201 functions as an interface for reading data from the recording medium 201A and writing data to the recording medium 201A. The recording medium 201A includes, for example, a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, a USB memory, and the like. Thereby, the information processing apparatus 200 can read various data used in processing through the recording medium 201A, store them in the auxiliary storage device 202, or install a program for realizing various functions.

[0124] In addition, the information processing apparatus 200 may acquire various data and programs used in processing from an external device through the communication interface 206.

[0125] The auxiliary storage device 202 stores various installed programs and also stores files, data, and the like necessary for various processes. The auxiliary storage device 202 includes, for example, an HDD (Hard Disc Drive), an SSD (Solid State Disc), a flash memory, and the like.

[0126] When there is an instruction to start a program, the memory device 203 reads the program from the auxiliary storage device 202 and stores it. The memory device 203 includes, for example, a DRAM (Dynamic Random Access Memory) and an SRAM.

[0127] The CPU 204 executes various programs loaded from the auxiliary storage device 202 to the memory device 203 and realizes various functions related to the information processing apparatus 200 according to the programs.

[0128] The high-speed arithmetic unit 205 operates in conjunction with the CPU 204 and performs arithmetic processing at a relatively high speed. The high-speed arithmetic unit 205 includes, for example, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), and the like.

[0129] Note that the high-speed arithmetic unit 205 may be omitted depending on the speed of the necessary arithmetic processing.

[0130] The communication interface 206 is used as an interface for communicably connecting to an external device. Thereby, the information processing apparatus 200 can communicate with an external device such as the excavator 100 through the communication interface 206. Further, the communication interface 206 may have a plurality of types of communication interfaces depending on the communication method with the connected device and the like.

[0131] The input device 207 receives various inputs from the user. The input device 207 includes an operating device for remote operation for remotely operating the excavator 100.

[0132] The input device 207 includes, for example, an input device (hereinafter, "mechanical input device") in a form that receives a mechanical operation input from the user. When remote operation of the excavator 100 is performed, the operating device for the remote operation may be a mechanical input device. The mechanical input device includes, for example, buttons, toggles, levers, keyboards, mice, touch panels mounted on the display device 208, touch pads provided separately from the display device 208, and the like.

[0133] Further, the input device 207 may include a voice input device capable of receiving a voice input from the user. The voice input device includes, for example, a microphone capable of collecting the user's voice.

[0134] In addition, the input device 207 may include a gesture input device capable of receiving gesture inputs from the user. The gesture input device includes, for example, a camera capable of imaging the state of the user's gestures.

[0135] In addition, the input device 207 may include a biometric input device capable of receiving biometric inputs from the user. The biometric input device includes, for example, a camera capable of acquiring image data containing information about the user's fingerprint or iris.

[0136] The display device 208 displays an information screen or an operation screen for the user of the information processing device 200. The display device 208 is, for example, a liquid crystal display, an organic EL display, or the like.

[0137] The sound output device 209 conveys various information to the user of the information processing device 200 by sound. The sound output device 209 is, for example, a buzzer, an alarm, a speaker, or the like.

[0138] <Configuration of Remote Operation Support Device> The hardware configuration of the remote operation support device 400 may be the same as that of the information processing device 200. Therefore, the illustration and description of the hardware configuration of the remote operation support device 400 are omitted.

[0139] Hereinafter, each of the communication interface, the input device, and the display device of the remote operation support device 400 may be referred to as "communication interface 406", "input device 407", and "display device 408" for convenience.

[0140] [First Example of Functional Configuration of Operation Support System] Next, with reference to FIGS. 6 to 10 in addition to FIGS. 1 to 5, a first example of the functional configuration of the operation support system SYS will be described.

[0141] FIG. 6 is a functional block diagram showing a first example of the functional configuration of the operation support system SYS. FIG. 7 is a diagram showing an example of a work area. FIGS. 8 to 10 are diagrams showing first to third examples of a display method regarding proposal of a location of a work target of the excavator 100.

[0142] In FIGS. 8 to 10, a heat map using color differences is expediently replaced by pattern differences for representation.

[0143] Hereinafter, the "trajectory of the working part of the excavator 100" is used with the intention of including both the path (i.e., the locus) that the working part of the excavator 100 has already moved and the path that may move in the future. The working part corresponds to the tip of the attachment AT used to apply a change to the work object. Specifically, the working part is the bucket 6.

[0144] The excavator 100 includes a support device 150. In this example, the support device 150 provides support to a user who operates the excavator 100 to perform work (i.e., the operator of the excavator 100).

[0145] As shown in FIG. 6, the support device 150 includes a controller 30, a sensor 40, a display device 50A, and sensors S1 to S9.

[0146] The controller 30 includes, as functional parts, an operation log providing part 301 and a work support part 302.

[0147] In addition, when there are a plurality of excavators 100 included in the operation support system SYS, there may be an excavator 100 that includes only the former of the operation log providing part 301 and the work support part 302 in the controller 30, and an excavator 100 that includes only the latter. In this case, the former excavator 100 has only the function of acquiring the operation log of the excavator 100 and providing it to the information processing device 200, which is used for the work support function of the latter excavator 100. The same may apply to the second example (FIG. 13) and the third example (FIG. 15) of the functional configuration described later.

[0148] The information processing device 200 includes, as functional parts, a log acquisition part 2001, a simulator part 2002, a log storage part 2003, a teacher data generation part 2004, a machine learning part 2005, a learned model storage part 2006, and a distribution part 2007.

[0149] The operation log providing unit 301 is a functional unit that acquires the operation log during a predetermined operation of the excavator 100 and provides it to the information processing apparatus 200.

[0150] The predetermined operations include, for example, excavation operations, boom raising and slewing operations, boom lowering and slewing operations, earth discharging operations, sweeping operations, etc., which are utilized during excavation work. Further, the predetermined operations may include excavation operations, earth discharging operations, sweeping operations, horizontal pulling operations, compaction operations, sweeping operations, etc., which are utilized during land leveling work. Further, the predetermined operations may include cutting operations, compaction operations, etc., which are utilized during slope work. The sweeping operation is, for example, an operation of operating the attachment AT and pushing the bucket 6 forward along the ground to sweep the earth and sand forward with the back surface of the bucket 6. In the sweeping operation, for example, the attachment AT performs a lowering operation of the boom 4 and an opening operation of the arm 5. The horizontal pulling operation is, for example, an operation of operating the attachment AT and moving the tip of the bucket 6 along the ground in a substantially horizontal manner toward the front to level the unevenness of the ground (the surface of the terrain). In the horizontal pulling operation, for example, the attachment AT performs a raising operation of the boom 4 and a closing operation of the arm 5. The compaction operation is, for example, an operation of operating the attachment AT and pressing the ground with the back surface of the bucket 6. Further, the compaction operation may be an operation of pressing the ground by hitting the back surface of the bucket 6 against the ground while moving the bucket 6 up and down. Further, the compaction operation may be an operation of pushing the bucket 6 forward along the ground to sweep the earth and sand forward to a predetermined position with the back surface of the bucket 6 and then pressing the ground at the predetermined position with the back surface of the bucket 6. In the compaction operation, for example, the attachment AT performs a lowering operation of the boom 4 when pressing the ground. The sweeping operation is, for example, an operation of operating the upper slewing body 3 and slewing the bucket 6 left and right while keeping the bucket 6 along the ground. Further, the sweeping operation may be an operation of operating the attachment AT and the upper slewing body 3, and alternately slewing the bucket 6 left and right while keeping the bucket 6 along the ground and pushing the bucket 6 forward. In the sweeping operation, for example, the upper slewing body 3 repeatedly performs left and right slewing operations alternately. Further, in the sweeping operation, for example, in addition to the left and right alternating slewing operations of the upper slewing body 3, the attachment AT may perform a lowering operation of the boom 4 and an opening operation of the arm 5 as in the case of the sweeping operation.

[0151] The operation log of the excavator 100 is time-series data representing the operating state of the excavator 100. For example, the operation log of the excavator 100 includes time-series data representing the operation content of the operator. The time-series data representing the operation content of the operator is, for example, the time-series output data of the operation pressure sensor 29 corresponding to the hydraulic pilot type operation device 26 or the time-series output data (operation signal data) of the operation device 26 corresponding to the electric operation device 26. Further, the operation log of the excavator 100 may be the time-series output data of the sensors S1 to S5 or the time-series data representing the posture state of the excavator 100 obtained from the output data of the sensors S1 to S5.

[0152] For example, the operation log providing unit 301 acquires the operation log when an operator (hereinafter, for convenience, referred to as a "skilled operator") with a long operation history of the excavator 100 and relatively experienced with respect to a predetermined standard operates the excavator 100, and provides it to the information processing device 200. Thereby, as will be described later, learning models LM1 and LM2 that reflect the operation of the excavator 100 by the operation of the skilled operator can be generated by machine learning based on the operation log of the excavator 100.

[0153] The operation log providing unit 301 includes an operation log recording unit 301A, an operation log storage unit 301B, and an operation log transmission unit 301C.

[0154] The operation log recording unit 301A acquires the operation log during a predetermined operation of the excavator 100 and records it in the operation log storage unit 301B. For example, each time a predetermined operation of the excavator 100 is executed, the operation log recording unit 301A records the operation log during that operation in the operation log storage unit 301B.

[0155] The operation log storage unit 301B stores the operation logs of the excavator 100. For example, in the operation log storage unit 301B, for each predetermined operation executed by the excavator 100, the operation log and the data of the time (date and time) when the predetermined operation was executed are associated and stored. The data of the time when the predetermined operation was executed includes the data of both the start and end times of the predetermined operation of the excavator 100. Also, when a plurality of predetermined operations are defined, in the operation log storage unit 301B, for each predetermined operation executed by the excavator 100, the operation log, the data of the time when the predetermined operation was executed, and the data of the identification information of the executed predetermined operation are associated and stored. Hereinafter, the data associated with the operation log of the excavator 100 may be conveniently referred to as "accompanying data". For example, in the operation log storage unit 301B, for each predetermined operation executed by the excavator 100, record data representing the correspondence relationship between the operation log and the accompanying data is accumulated, thereby constructing a database of the operation log data at the time of execution of the predetermined operation of the excavator 100.

[0156] In addition, the operation logs in the operation log storage unit 301B that have already been transmitted to the information processing device 200 by the operation log transmission unit 301C described later may be erased afterwards.

[0157] The operation log transmission unit 301C transmits the operation log when the excavator 100 executes a predetermined operation, which is stored in the operation log storage unit 301B, and the accompanying data associated with the operation log to the information processing device 200 through the communication device 60. Also, the operation log transmission unit 301C may also transmit the record data representing the correspondence relationship between the operation log and the accompanying data of the excavator 100 for each predetermined operation executed by the excavator 100 to the information processing device 200.

[0158] For example, in response to a transmission request for the operation log of the excavator 100 received from the information processing apparatus 200, the operation log transmission unit 301C transmits the untransmitted operation log of the excavator 100 and the accompanying data stored in the operation log storage unit 301B to the information processing apparatus 200. Further, the operation log transmission unit 301C may automatically transmit the untransmitted operation log of the excavator 100 and the accompanying data stored in the operation log storage unit 301B to the information processing apparatus 200 at a predetermined timing. The predetermined timing is, for example, when the excavator 100 stops operating (key switch off) or starts operating (key switch on).

[0159] The log acquisition unit 2001 acquires a log when the excavator 100 executes a predetermined operation.

[0160] The log when the excavator 100 executes a predetermined operation includes the operation log when the excavator 100 executes a predetermined operation and the state log of the objects in the work area. The state log of the objects in the work area includes data representing the state of the objects in the work area before and after the execution of the predetermined operation of the excavator 100. The state of the objects in the work area includes the shape of the objects in the work area (for example, the terrain shape of the ground surface in the work area) and the characteristics of the earth and sand in the work area. The operation log when the excavator 100 executes a predetermined operation is uploaded from the excavator 100. The state log of the objects in the work area when the excavator 100 executes a predetermined operation is acquired based on the measurement data uploaded from the sensor group 300 and the accompanying data (data at the time when the predetermined operation is executed) uploaded from the excavator 100.

[0161] The simulator unit 2002 performs a computer simulation related to a predetermined operation of the excavator 100 using the virtual models of the excavator 100 and the objects (i.e., earth and sand) in the work area.

[0162] For example, using the distinct element method (DEM), the earth and sand in the work area are modeled as aggregates of fine particles. Thereby, the simulator unit 2002 causes the virtual model of the excavator 100 to execute a predetermined operation such as an excavation operation, and analyzes the individual movements of the fine particles to virtually reproduce the overall behavior of the earth and sand in the work area as an aggregate, the reaction force from the earth and sand, etc.

[0163] The simulator unit 2002 acquires data on the trajectory of the working part of the excavator 100 and data on the state of the earth and sand in the work area before and after the execution of a predetermined operation as a log when the excavator 100 executes a predetermined operation by computer simulation. The former data corresponds to the operation log when the excavator 100 executes a predetermined operation by computer simulation, and the latter data corresponds to the state log of the object in the work area when the excavator 100 executes a predetermined operation by computer simulation.

[0164] The simulator unit 2002 performs a large number of patterns of computer simulations regarding a predetermined operation of the excavator 100 using the states of various work objects (earth and sand) and the trajectories of the working parts of various excavators 100. Thereby, the simulator unit 2002 can accumulate in the log storage unit 2003 the logs when the excavator 100 executes a predetermined operation by computer simulation under different conditions.

[0165] In the log storage unit 2003, the logs when the excavator 100 executes a predetermined operation, acquired by the log acquisition unit 2001 and the simulator unit 2002, are stored in a form in which they are accumulated. For example, in the log storage unit 2003, the operation log, the state log of the object in the work area, and the accompanying data for each predetermined operation actually executed by the excavator 100 or by computer simulation are stored in a linked form. In the log storage unit 2003, the log acquired by the log acquisition unit 2001 and the log acquired by the simulator unit 2002 may be stored in a distinguishable manner or may be stored mixed in an indistinguishable manner.

[0166] The teacher data generation unit 2004 generates teacher data for machine learning based on the logs stored in the log storage unit 2003 when the excavator 100 executes a predetermined operation, and outputs a teacher data set which is a collection of a large number of teacher data. The teacher data generation unit 2004 may automatically generate teacher data by batch processing, or may generate teacher data in response to an input from the user of the information processing apparatus 200. The teacher data generation unit 2004 includes a teacher data generation unit 2004A.

[0167] The teacher data generation unit 2004A generates a teacher dataset for generating the learned model LM1. Based on the data representing the state of the objects in the working area of the excavator 100, the learned model LM1 infers the distribution of the degree of recommendation as the working location (hereinafter, for convenience, referred to as the "working location") where the excavator 100 changes the shape of the object by a predetermined operation in the working area. For example, as shown in FIG. 7, the working area TA of the excavator 100 is divided into a plurality of small rectangular areas, and the degree of recommendation as the working location for each of the plurality of small areas is inferred. Further, the learned model LM1 may infer the distribution of the degree of recommendation as the working location of the excavator 100 in the working area based on the data representing the state of the objects in the working area of the excavator 100 and the data representing the target shape of the objects in the working area. For example, the data representing the target shape of the objects in the working area of the excavator 100 is the data representing the target construction surface of the working area. Further, the learned model LM1 may infer the distribution of the degree of recommendation as the working location of the excavator 100 for each of the first to Nth times in the working area on the premise that a predetermined operation of the excavator 100 is performed N times (N: an integer of 2 or more). The objects in the working area of the excavator 100 include the earth and sand in the working area as the objects to be worked on by the excavator 100. Further, the objects in the working area of the excavator 100 may include obstacles other than the objects to be worked on by the excavator 100. "The excavator 100 changes the shape of the object by a predetermined operation" includes, for example, the case where the shape of the object is changed by a predetermined operation of bringing the working part of the attachment AT (for example, the bucket 6) into contact with the object. The predetermined operation of bringing the working part of the attachment AT into contact with the object is, for example, an excavation operation of excavating the earth and sand in the working area with the bucket 6. Further, "the excavator 100 changes the shape of the object by a predetermined operation" may include the case where the shape of the object in the working area is changed by discharging the contents such as the earth and sand contained in the bucket 6 as the working part of the attachment AT into the working area. Discharging the contents such as the earth and sand contained in the bucket 6 as the working part of the attachment AT into the working area is, for example, a soil discharging operation.For example, when a predetermined operation of the excavator 100 is an excavation operation, the learned model LM1 infers a distribution of recommendation degrees as excavation locations in the work area by the bucket 6 of the excavator 100 based on data representing the shapes of objects in the work area of the excavator 100.

[0168] The input data corresponding to the learned model LM1 includes data representing the state of the objects in the work area before the execution of a predetermined operation of the excavator 100. The state of the objects in the work area includes, for example, the shapes of the objects in the work area. Also, the state of the objects in the work area may include the characteristics of the objects (e.g., earth and sand) in the work area. For example, data representing the characteristics of earth and sand includes data on the angle of repose of the earth and sand. Thereby, the learned model LM1 can infer a more appropriate distribution of recommendation degrees as work locations considering the angle of repose of the earth and sand. Also, the input data corresponding to the learned model LM1 may include data representing the target shapes of the objects in the work area of the excavator 100.

[0169] The teacher data is a combination of input data of a type defined for the learned model LM1 and data representing the correct inference result corresponding to the input data (i.e., correct data). The correct data is data representing the locations where the excavator 100 changes the shape of the object by a predetermined operation in the work area corresponding to the input data included in the teacher data. Also, when the distribution of recommendation degrees as work locations of the excavator 100 in the work area is inferred for each of N times of a predetermined operation of the excavator 100, the correct data includes data representing the locations where the shape of the object is changed by each of the N times of the predetermined operation of the excavator 100 in the work area. Also, when a plurality of types of predetermined operations are defined, the learned model LM1 may be generated for each type of predetermined operation. In this case, the teacher data generation unit 2004A generates a teacher data set for each type of predetermined operation.

[0170] The teacher dataset for generating the learned model LM1 is generated based on, for example, the logs acquired by the log acquisition unit 2001. Specifically, the teacher data may be a combination of data representing the state of an object in the work area before the excavator 100 executes a predetermined operation by the operation of a skilled person, and data representing the location where the shape of the object has changed during the execution of the predetermined operation. Thereby, the information processing apparatus 200 can generate the learned model LM1 in which the location where the shape of the object in the work area has changed when the excavator 100 executes a predetermined operation by the operation of a skilled person is reflected. Therefore, the learned model LM1 can infer the distribution of the recommendation degrees as the work location of the excavator 100, in which the experience of the skilled person is reflected, with the data representing the state of the object in the work area of the excavator 100 as the input. In other words, the distribution of the recommendation degrees as the work location of the excavator 100 corresponds to the distribution of the reliability as the work location of the excavator 100 from the viewpoints of work efficiency, safety, etc. based on the experience of the skilled person. For example, the shorter the travel distance of the lower traveling body 1 required to perform the work of changing the shape of the object at the target location, and the smaller the turning angle of the upper slewing body 3, the easier it is to reach the working part of the attachment AT at that location. On the other hand, the longer the travel distance of the lower traveling body 1 required to perform the work of changing the shape of the object at the target location, and the larger the turning angle of the upper slewing body 3, the more difficult it is to reach the working part of the attachment AT at that location. Therefore, a skilled person may tend to select, as the work location, a location where it is easy to reach the working part of the attachment AT of the excavator 100 from the viewpoint of work efficiency and the like. In this case, the learned model LM1 estimates the distribution of the recommendation degrees as the work location of the excavator 100 in the work area so that the location where it is easier to reach the attachment AT of the excavator 100 has a higher recommendation degree as the work location than the location where it is difficult to reach the attachment AT. Further, the teacher dataset for generating the learned model LM1 may include a base teacher dataset and a teacher dataset for final adjustment (fine-tuning).

[0171] The machine learning unit 2005 generates a trained model by performing machine learning on the base learning model based on the teacher data set generated by the teacher data generating unit 2004. The trained model (and the base learning model) includes, for example, a neural network such as a DNN (Deep Neural Network).

[0172] The machine learning unit 2005 includes a machine learning unit 2005A.

[0173] The machine learning unit 2005A causes the base learning model M1 to perform machine learning based on the teacher data set output from the teacher data generation unit 2004A. As a result, the machine learning unit 2005A can generate a learned model LM1 that can output (infer) a distribution of recommendation degrees for the shovel 100 as a work location in the work area by using data on the state of objects in the work area of ​​the shovel 100 as input. For example, the machine learning unit 2005A can optimize the learning model M1 using an error backpropagation algorithm based on the error between the output data of the learning model M1 for the input data included in the teacher data and the correct answer data, thereby generating the learned model LM1.

[0174] Furthermore, the machine learning unit 2005A may generate the learned model LM1 by applying reinforcement learning to the learning model M1 instead of supervised learning. In this case, the teacher data generating unit 2004A is omitted. For example, the machine learning unit 2005A causes the learning model M1 to perform reinforcement learning based on logs acquired by the log acquiring unit 2001 and the simulator unit 2002 so as to maximize a predetermined reward related to the work efficiency or safety of the shovel 100. At this time, the machine learning unit 2005A works in conjunction with the simulator unit 2002 to cause the simulator unit 2002 to try out many operation patterns, thereby enabling the learning model M1 to perform reinforcement learning more efficiently.

[0175] The learned model storage unit 2006 stores the learned model LM1 output by the machine learning unit 2005. Also, when the learned model LM1 is relearned or additional learned by the machine learning unit 2005A, the learned model LM1 in the learned model storage unit 2006 is updated. Further, when the learned model LM1 is updated, the learned model LM1 before the update may be stored in the learned model storage unit 2006 or other storage units in a reusable form. Thereby, for example, when there is a problem with the learned model LM1 after the update, the learned model LM1 before the update can be restored and reused.

[0176] The distribution unit 2007 distributes the data of the learned model LM1 to the excavator 100.

[0177] For example, when the learned model LM1 is generated or updated by the machine learning unit 2005A, the distribution unit 2007 distributes the most recently generated or updated learned model LM1 to the excavator 100. Also, the distribution unit 2007 may distribute the latest learned model LM1 in the learned model storage unit 2006 to the excavator 100 in response to a signal requesting the distribution of the learned model LM1 received from the excavator 100.

[0178] The work support unit 302 is a functional unit for providing support to a user (i.e., an operator of the excavator 100) who operates the excavator 100 to perform work.

[0179] The work support unit 302 includes a learned model storage unit 302A, a state acquisition unit 302B, a recommendation degree distribution estimation unit 302C, and a proposal unit 302D.

[0180] The learned model storage unit 302A stores the learned model LM1 distributed from the information processing device 200 and received through the communication device 60.

[0181] The state acquisition unit 302B acquires data representing the state of an object in the working area of the excavator 100. For example, based on the output of the sensor 40, the state acquisition unit 302B acquires data representing the state of an object in the working area of the excavator 100, such as the shape and characteristics of the object. Further, the state acquisition unit 302B may acquire data representing the state of an object in the working area of the excavator 100 based on the output of the sensor 300-X acquired through the communication device 60. In this case, the sensor 40 may be omitted. Also, there may be a place where occlusion occurs in the sensor 40 or the sensor 300-X and data representing the state of the object cannot be observed by the sensor 40 or the sensor 300-X. In this case, the state acquisition unit 302B estimates the change in the shape of the object (i.e., earth and sand) from before the execution of the predetermined operation based on the trajectory (i.e., locus) of the working part at the time of the previous predetermined operation of the excavator 100, and may acquire data representing the shape of the object in the working area of the excavator 100. Further, the state acquisition unit 302B estimates the reaction force from the object in the working area to the working part at the time of the previous predetermined operation of the excavator 100 based on the measurement data of S7 to S9, and may acquire data representing the state of the object in the working area of the excavator 100 based on the estimation result.

[0182] Regardless of the outputs of sensors 40, sensor 300-X, etc., data representing the state of an object in the working area of the excavator 100 may be obtained by estimating the state of the object in the working area of the excavator 100. For example, the information processing device 200 may construct a digital twin representing the working conditions at the work site of the excavator 100 using the simulator unit 2002. In this case, the information processing device 200 updates the digital twin based on the log data uploaded in real time from the excavator 100, and can output the estimated state of the working area, such as the objects and characteristics in the current working area of the excavator 100, from the digital twin. Therefore, by transmitting the output of the digital twin from the information processing device 200 to the excavator 100 through the communication interface 206, the state acquisition unit 302B can acquire data representing the estimated state of the object in the working area of the excavator 100. Also, the machine learning unit 2005 of the information processing device 200 may generate a learned model LM4 capable of inferring the state of an object in the working area of the excavator 100 for a given input. For example, the learned model LM4 inputs the state of the object in the working area before the execution of a predetermined operation of the excavator 100 and the trajectory (i.e., the locus) of the working part during the execution of the predetermined operation of the excavator 100, and estimates the shape of the object in the working area after the execution of the predetermined operation of the excavator 100. In this case, the teacher data set for the learned model LM4 is generated by the teacher data generation unit 2004 based on the log data acquired by the log acquisition unit 2001 and the log data based on the simulation in the simulator unit 2002. Thereby, every time a predetermined operation of the excavator 100 is executed, the state acquisition unit 302B can estimate the state of the current working area after the execution of the predetermined operation using the learned model LM4 distributed from the distribution unit 2007 of the information processing device 200.

[0183] Based on the data representing the state of the objects in the working area of the excavator 100 acquired by the state acquisition unit 302B, the recommendation degree distribution estimation unit 302C estimates the distribution of the recommendation degree as the working location of the excavator 100 in the working area. Specifically, the recommendation degree distribution estimation unit 302C applies the learned model LM1 to the input data including the data representing the state of the objects in the state of the working area of the excavator 100, and estimates the distribution of the recommendation degree as the working location of the excavator 100 in the working area. Further, based on the data representing the state of the objects in the working area of the excavator 100, the recommendation degree distribution estimation unit 302C may use the learned model LM1 to estimate the distribution of the recommendation degree as the working location for each of the predetermined operations of the excavator 100 from the first time to the Nth time in the working area.

[0184] Based on the estimation result of the recommendation degree distribution estimation unit 302C, the proposal unit 302D visually proposes to the user in the cab 10 the working location of the excavator 100 in the working area through the display device 50A. Further, when the excavator 100 is remotely operated, the proposal unit 302D visually proposes to the user the working location of the excavator 100 in the working area through the display device 408 of the remote operation support device 400 based on the estimation result of the recommendation degree distribution estimation unit 302C. In this case, for example, the proposal unit 302D can control the display device 408 by transmitting a control command including information regarding the proposal of the working location of the excavator 100 in the working area to the remote operation support device 400 through the communication device 60. Thereby, the user can relatively easily make a judgment on which location of the objects in the working area to change the shape of by a predetermined operation of the excavator 100, such as the location where excavation is performed with the bucket 6 by the excavation operation of the excavator 100.

[0185] For example, the proposal unit 302D causes the display device 50A or the display device 408 to display the distribution of the recommendation degree as the working location of the excavator 100 in the working area, which is estimated by the recommendation degree distribution estimation unit 302C. Thereby, the proposal unit 302D can propose to the user as the working location a location with a relatively high recommendation degree in the working area of the excavator 100 through the distribution of the recommendation degree as the working location of the excavator 100 displayed on the display device 50A or the like.

[0186] For example, as shown in FIG. 8, the proposal unit 302D proposes the working location of the excavator 100 in the working area through the screen 800 of the display device 50A.

[0187] The screen 800 includes a working area image 801, an excavator image 802, a target shape image 803, and a recommendation degree distribution image 804.

[0188] The working area image 801 is an image that simulates a two-dimensional cross-section of the working area of the excavator 100.

[0189] The excavator image 802 is an image that simulates a left side view of the excavator 100.

[0190] The target shape image 803 is an image that simulates the target shape of the working area, specifically, the target construction surface in two dimensions.

[0191] The recommendation degree distribution image 804 is an image that represents the distribution of the recommendation degree of the working location of the excavator 100 in the working area. In this example, the recommendation degree distribution image 804 is an image that extracts and displays only the distribution of the recommendation degree around the location where the recommendation degree is maximized among the distribution of the recommendation degree as the working location of the excavator 100 in the working area. The recommendation degree distribution image 804 includes recommendation degree distribution images 804A to 804C corresponding to the locations where the recommendation degree as the working location is maximized in the working area of the excavator 100. Thereby, the controller 30 can propose, through the display device 50A, three locations corresponding to each of the recommendation degree distribution images 804A to 804C in the working area as the working locations of the excavator 100.

[0192] The recommended degree distribution image 804A is an image representing the distribution of the recommended degree as a working place near the apex of the slope in the working area. The recommended degree distribution image 804B is an image representing the distribution of the recommended degree as a working place near the middle of the slope in the working area. The recommended degree distribution image 804C is an image representing the distribution of the recommended degree as a working place near the foot of the slope in the working area.

[0193] In this example, the recommended degree distribution image 804B takes a maximum value with a relatively large value of the recommended degree as a working place. On the other hand, the recommended degree distribution images 804A and 804C take a maximum value with a relatively small value of the recommended degree as a working place. Therefore, the user can easily recognize that the recommended degree as a working place at the location corresponding to the recommended degree distribution image 804B is higher than the recommended degree as a working place at the locations corresponding to the recommended degree distribution images 804A and 804C.

[0194] As described above, in this example, the proposal unit 302D can propose the working place of the excavator 100 to the user by causing the display device 50A to display the working area represented in two dimensions and the distribution of the recommended degree as the working place of the excavator 100 in that working area.

[0195] Further, as shown in FIG. 9, the proposal unit 302D may propose the working place of the excavator 100 in the working area through the screen 900 of the display device 50A.

[0196] The screen 900 includes a working area image 901 and a recommended degree distribution image 904.

[0197] The working area image 901 is an image that simulates and represents the working area of the excavator 100 in three dimensions. In this example, the working area image 901 simulates and represents the working area as seen from the viewpoint of the operator of the cab 10 of the excavator 100 in three dimensions.

[0198] Note that the working area image 901 may be replaced with a viewpoint conversion image generated based on a captured image captured by an imaging device (camera) as the sensor 40.

[0199] The recommendation degree distribution image 904 is an image representing the distribution of the recommendation degree as the working location of the excavator 100 in the working area. In this example, similar to the above-described recommendation degree distribution image 804, the recommendation degree distribution image 904 is an image that extracts and displays only the distribution of the recommendation degree around the location where the recommendation degree as the working location is maximized among the distribution of the recommendation degree as the working location of the excavator 100 in the working area. The recommendation degree distribution image 904 includes recommendation degree distribution images 904A to 904C corresponding to the locations where the recommendation degree as the working location is maximized in the working area of the excavator 100. Thereby, the controller 30 can propose three locations corresponding to each of the recommendation degree distribution images 904A to 904C as the working locations of the excavator 100 to the user through the display device 50A.

[0200] In this example, the recommendation degree distribution image 904A takes a maximum value with a relatively large value of the recommendation degree as the working location. On the other hand, the recommendation degree distribution images 904B and 904C take a maximum value with a relatively small value of the recommendation degree as the working location. Therefore, the user can easily recognize that the recommendation degree as the working location of the location corresponding to the recommendation degree distribution image 904A is higher than the recommendation degree as the working location of the locations corresponding to the recommendation degree distribution images 904B and 904C.

[0201] In this way, in this example, the proposal unit 302D can propose a working location to the user by causing the display device 50A to display the working area viewed from the cab of the excavator 100 represented in three dimensions and the distribution of the recommendation degree as the working location of the excavator 100 in the working area. Therefore, for example, a user (operator) in the cab 10 of the excavator 100 can visually recognize the distribution of the recommendation degree as the working location in the working area in the same view as the actual working area viewed from the cab 10.

[0202] In addition, the screen 900 may include an image representing the target shape of the work object in three dimensions in addition to the work area image 901 and the recommendation degree distribution image 904.

[0203] Further, as shown in FIG. 10, the proposal unit 302D may propose the working location of the excavator 100 within the working area through the screen 1000 of the display device 50A.

[0204] The screen 1000 includes a working area image 1001, an excavator image 1002, and a recommendation degree distribution image 1004.

[0205] The working area image 1001 is an image that simulates and represents the working area of the excavator 100 in three dimensions. In this example, the working area image 1001 simulates and represents the working area in three dimensions as seen from the viewpoint in front of the excavator 100 and facing the excavator 100.

[0206] Note that the working area image 1001 may be replaced with a viewpoint conversion image generated based on an imaging image captured by an imaging device (camera) as the sensor 40.

[0207] The excavator image 1002 is an image that simulates and represents the excavator 100 seen from the front in three dimensions.

[0208] The recommendation degree distribution image 1004 is an image that represents the distribution of the recommendation degree of the working location of the excavator 100 within the working area. In this example, the recommendation degree distribution image 1004 is an image that extracts and displays only the distribution of the recommendation degree around the location where the recommendation degree is maximized among the distribution of the recommendation degree of the working location of the excavator 100 in the working area. Thereby, the controller 30 can propose, through the display device 50A, three locations corresponding to each of the recommendation degree distribution images 1004A to 1004C as the working locations of the excavator 100.

[0209] In this example, the recommendation degree distribution image 1004B takes a maximum value with a relatively large value. On the other hand, the recommendation degree distribution images 1004A and 1004C take a maximum value with a relatively small value of the recommendation degree of the working location. Therefore, the user can easily recognize that the recommendation degree of the location corresponding to the recommendation degree distribution image 1004B is higher than the recommendation degree of the locations corresponding to the recommendation degree distribution images 1004A and 1004C.

[0210] Thus, in this example, the proposal unit 302D causes the display device 50A to display the working area of the excavator 100 represented in three dimensions as viewed from the front and the distribution of the recommended degrees of the working positions of the excavator 100 in that working area, so that the working positions can be proposed to the user. Therefore, the user can visually recognize the distribution of the recommended degrees while checking both the working area viewed from the excavator 100 side and the working area viewed from the front of the excavator 100, that is, from the opposite side.

[0211] In addition, the screen 1000 may include an image representing the target shape of the work object in three dimensions in addition to the work area image 1001, the excavator image 1002, and the recommended degree distribution image 1004. Further, when the excavator 100 is remotely operated, the proposal unit 302D may cause the display device 408 of the remote operation support device 400 to display the screen 800, the screen 900, or the screen 1000.

[0212] Alternatively, instead of the difference in color for each location proposed as the working position, the proposal unit 302D may express the difference in the recommended degree as a working position for each proposed working position by representing each recommended degree numerically.

[0213] In addition, when the proposal unit 302D displays the distribution of the recommended degree as the working location in the working area, it may display the distribution of the recommended degree limited to a predetermined range in the working area where the excavator 100 can easily reach the working part of the attachment AT. The range where the excavator 100 can easily reach the working part of the attachment AT is, for example, a range where the traveling distance of the lower traveling body 1 required to change the shape of the object at that location is relatively small with respect to a predetermined reference distance, and the swing angle of the upper swing body 3 is relatively small with respect to a predetermined reference angle. Relatively small with respect to the reference distance may mean less than or shorter than the reference distance. Similarly, relatively small with respect to the reference angle may mean less than or smaller than the reference angle. The reference distance may be zero. Similarly, the reference angle may be zero. Thereby, the proposal unit 302D can propose, as the working target location, a location in the working area where the excavator 100 can relatively easily reach the working part of the attachment AT.

[0214] In addition, when there is no location where the recommended degree as the working location is relatively high with respect to a predetermined standard in the working target of the excavator 100, the proposal unit 302D may not propose a working location. In this case, the proposal unit 302D causes, for example, a display device 50A or a display device 408 to display that there is no proposed working location.

[0215] In addition, for the predetermined operations of the excavator 100 from the first to the Nth times in the future, the proposal unit 302D may visually propose the working location of the excavator 100 in each working area through the display device 50A or the display device 408. In this case, the proposal unit 302D causes, for example, an image representing the working area to be displayed on the display device 50A or the display device 408, and an image representing the location with the highest recommended degree in each working area to be superimposed and displayed.

[0216] Part or all of the functions of the state acquisition unit 302B, the recommendation degree distribution estimation unit 302C, and the proposal unit 302D may be transferred outside the excavator 100. For example, the functions of the state acquisition unit 302B, the recommendation degree distribution estimation unit 302C, and the proposal unit 302D may be transferred to the information processing device 200, and the display content of the display device 50A may be controlled from the information processing device 200. Also, for example, when the excavator 100 is remotely operated, the functions of the state acquisition unit 302B, the recommendation degree distribution estimation unit 302C, and the proposal unit 302D may be transferred to the remote operation support device 400. In these cases, for example, the outputs of the sensors 40 and S1 to S9 are uploaded in real time to the information processing device 200 or the remote operation support device 400 through the communication device 60.

[0217] [First Example of Processing for Proposing a Workplace] Next, with reference to FIG. 11, a first example of the processing for proposing a workplace of the excavator 100 will be described.

[0218] FIG. 11 is a flowchart schematically showing a first example of the processing for proposing a workplace of the excavator 100.

[0219] The flowchart of FIG. 11 is executed, for example, when the function for proposing a workplace of the excavator 100 is valid and the work by a predetermined operation of the excavator 100 is being executed. The same may apply to the flowcharts of FIGS. 12, 14, and 16 described later.

[0220] As shown in FIG. 11, in step S10, the state acquisition unit 302B acquires data representing the shape of an object in the work area of the excavator 100 based on the output of the sensor 40 or the like.

[0221] When the processing of step S10 is completed, the controller 30 proceeds to step S11.

[0222] In step S11, based on the data acquired in step S10, the recommendation degree distribution estimation unit 302C estimates the distribution of the recommendation degree as the working location of the excavator 100 in the work area using the learned model LM1.

[0223] When the process of step S11 is completed, the controller 30 proceeds to step S12.

[0224] In step S12, based on the estimation result of the distribution of the recommendation degree as the working location of the excavator 100 in the work area acquired in step S11, the proposal unit 302D proposes the working location of the excavator 100 through the display device 50A or the display device 408.

[0225] When the process of step S12 is completed, the controller 30 proceeds to step S13.

[0226] In step S13, the controller 30 starts a predetermined operation of the excavator 100, and then determines whether the operation is completed. That is, after the proposal in step S12, the controller 30 determines whether a predetermined operation of the excavator 100 has been executed and completed. The controller 30 determines the start of execution and the completion of a predetermined operation of the excavator 100, for example, based on the outputs of the sensors S1 to S9 and using the pre-specified operation conditions. When a predetermined operation of the excavator 100 is started and then the operation is completed, the controller 30 proceeds to step S14; otherwise, the process of step S13 is repeated.

[0227] In step S14, the controller 30 determines whether or not a predetermined condition (hereinafter referred to as "operation end condition") indicating the end of the operation of the excavator 100 is satisfied. The operation end condition is, for example, that a predetermined input indicating the end of the operation is received through the input device 52 or the input device 407 from the user. Also, the operation end condition may be, for example, a condition related to time. Further, the operation end condition may be a condition indicating that the shape of the object in the work area has reached the target shape. If the operation end condition is satisfied, the controller 30 ends the processing of the current flowchart. If the operation end condition is not satisfied, the controller 30 returns to step S10 and repeats the processing after step S10.

[0228] [Second Example of Processing for Proposal of Work Location] Next, with reference to FIG. 12, a second example of the processing for proposing the work location of the excavator 100 will be described.

[0229] As shown in FIG. 12, since the processing in step S20 is the same as the processing in step S10 of FIG. 11, the description thereof will be omitted.

[0230] When the processing in step S20 is completed, the controller 30 proceeds to step S21.

[0231] In step S21, based on the data acquired in step S20, the recommendation degree distribution estimation unit 302C uses the learned model LM1 to estimate the distribution of the recommendation degree as the work location of the excavator 100 in the work area corresponding to the predetermined operations of the excavator 100 from the first time to the Nth time.

[0232] When the processing in step S21 is completed, the controller 30 proceeds to step S22.

[0233] In step S22, based on the estimation result of the distribution of the recommended degree as the working location of the excavator 100 in the working area obtained in step S21, the proposal unit 302D makes proposals for the working locations of the excavator 100 corresponding to the predetermined operations of the excavator 100 from the first time to the Nth time in the future through the display device 50A or the display device 408.

[0234] When the processing of step S22 is completed, the controller 30 proceeds to step S23.

[0235] Since the processing of steps S23 and S24 is the same as the processing of steps S13 and S14 in FIG. 11, the description thereof is omitted.

[0236] Note that in step S23, instead of the completion of one predetermined operation of the excavator 100, the presence or absence of the completion of N predetermined operations of the excavator 100 may be determined.

[0237] [Second Example of Functional Configuration of Operation Support System] Next, in addition to FIGS. 1 to 5, with reference to FIG. 13, a second example of the functional configuration of the operation support system SYS will be described.

[0238] Hereinafter, in this example, the same reference numerals are assigned to the same or corresponding configurations as those in the above-described first example (FIG. 6), and the description will focus on the parts different from the above-described first example, and the description of the same or corresponding content as the above-described first example may be omitted.

[0239] FIG. 13 is a functional block diagram showing a second example of the functional configuration of the operation support system SYS.

[0240] In this example, mainly, the learned model LM2 is generated by the information processing device 200, and the controller 30 includes the trajectory generation unit 302E, which is different from the above-described first example.

[0241] The controller 30 of the excavator 100 includes, as functional units, an operation log providing unit 301 and an operation support unit 302, similar to the above-described first example.

[0242] As a functional unit, the information processing apparatus 200 includes, as in the first example described above, a log acquisition unit 2001, a simulator unit 2002, a log storage unit 2003, a teacher data generation unit 2004, a machine learning unit 2005, a learned model storage unit 2006, and a distribution unit 2007.

[0243] The teacher data generation unit 2004 includes a teacher data generation unit 2004B in addition to the teacher data generation unit 2004A.

[0244] The teacher data generation unit 2004B generates teacher data for generating the learned model LM2. The learned model LM2 infers the trajectory of the working part in a predetermined operation of the excavator 100 based on data representing the state of the object in the working area of the excavator 100. Further, the learned model LM2 may infer the trajectory of the working part in a predetermined operation of the excavator 100 based on data representing the state of the object in the working area of the excavator 100 and data representing the target shape of the object in the working area. Further, as a constraint condition, data specifying the working location of the excavator 100 is input, and under this constraint condition, the learned model LM2 may infer the trajectory of the working part in a predetermined operation of the excavator 100.

[0245] The input data corresponding to the learned model LM2 includes, for example, data representing the state of the object in the working area of the excavator 100. The data representing the state of the object (i.e., earth and sand) in the working area includes, for example, data representing the shape of the object in the working area as described above. Further, the data representing the state of the object in the working area may include data representing the characteristics of the object (i.e., earth and sand) in the working area as described above. Further, the input data corresponding to the learned model LM2 may include data representing the target shape of the work object. Further, when the predetermined operation of the excavator 100 is a dumping operation, the input data corresponding to the learned model LM2 may include data representing the weight or volume of the earth and sand contained in the bucket 6 before dumping.

[0246] The teacher data is a combination of input data of a type defined for the learned model LM2 and data representing the correct inference result corresponding to the input data (correct answer data). Specifically, the input data included in the teacher data includes data representing the state of the work area before the execution of a predetermined operation of the excavator 100. Further, the input data included in the teacher data may include data representing the target shape of an object in the work area of the excavator 100. Further, when the predetermined operation of the excavator 100 is an earthmoving operation, the input data included in the teacher data may include data representing the weight or volume of the earth and sand contained in the bucket 6 before the earthmoving. Further, the correct answer data included in the teacher data includes data representing the trajectory of the working part when the excavator 100 executes a predetermined operation by the operation of a skilled person on the premise of the state of the object in the work area corresponding to the input data. That is, the teacher data generation unit 2004B generates a teacher data set based on the log obtained by the log acquisition unit 2001 when the excavator 100 performs a predetermined operation by the operation of a skilled person. Further, when a plurality of types of predetermined operations are defined, the learned model LM2 may be generated for each type of predetermined operation. In this case, the teacher data generation unit 2004B generates a teacher data set for each type of predetermined operation.

[0247] The machine learning unit 2005 includes a machine learning unit 2005B in addition to the machine learning unit 2005A.

[0248] Based on the teacher data set output from the teacher data generation unit 2004B, the machine learning unit 2005B causes the base learning model M2 to perform machine learning. As a result, the machine learning unit 2005B can generate a learned model LM2 capable of inferring the trajectory of the working part in a predetermined operation of the excavator 100 by using, as input, data such as the state of the object in the work area of the excavator 100. For example, the machine learning unit 2005B can optimize the learning model M2 using the algorithm of the error backpropagation method based on the error between the output data of the learning model M2 for the input data included in the teacher data and the correct answer data, and generate the learned model LM2.

[0249] Further, the machine learning unit 2005 may generate a learned model LM2 by applying reinforcement learning to the learning model M2 instead of supervised learning. In this case, the teacher data generation unit 2004B is omitted. For example, the machine learning unit 2005B performs reinforcement learning on the learning model M2 so as to maximize a predetermined reward related to work efficiency or safety based on the logs acquired by the log acquisition unit 2001 and the simulator unit 2002. At this time, in conjunction with the simulator unit 2002, by causing the simulator unit 2002 to try a number of operation patterns, the learning model M2 can be more efficiently reinforced.

[0250] In addition, instead of the learned models LM1 and LM2, the machine learning unit 2005 may generate a learned model LM3 that can infer both the distribution of the recommended degree as the working location of the excavator 100 in the working area and the trajectory of the working part in a predetermined operation of the excavator 100, using data representing the shape of an object in the working area of the excavator 100 as input. In this case, instead of the machine learning units 2005A and 2005B, a machine learning unit for generating the learned model LM3 is provided. Also, in this case, instead of the teacher data generation units 2004A and 2004B, a teacher data generation unit for generating a teacher data set for generating the learned model LM3 is provided.

[0251] The learned model storage unit 2006 stores the learned models LM1 and LM2 output by the machine learning unit 2005. Also, when the machine learning unit 2005B performs relearning or additional learning on the learned model LM2, the learned model LM2 in the learned model storage unit 2006 is updated. Further, when the learned model LM2 is updated, the learned model LM2 before the update may be stored in the learned model storage unit 2006 or another storage unit in a reusable form. Thereby, for example, when there is a problem with the learned model LM2, the learned model LM2 before the update can be restored and reused.

[0252] When the learned model LM3 is generated, the learned model storage unit 2006 stores the learned model LM3 instead of the learned models LM1 and LM2.

[0253] In addition to the learned model LM1, the distribution unit 2007 distributes the data of the learned model LM2 to the shovel 100.

[0254] For example, when the learned model LM2 is generated or updated by the machine learning unit 2005B, the distribution unit 2007 distributes the most recently generated or updated learned model LM2 to the shovel 100. Further, the distribution unit 2007 may distribute the latest learned model LM2 in the learned model storage unit 2006 to the shovel 100 in response to a signal requesting the distribution of the learned model LM2 received from the shovel 100.

[0255] When the learned model LM3 is generated, the distribution unit 2007 distributes the learned model LM3 to the shovel 100 instead of the learned models LM1 and LM2.

[0256] The work support unit 302 includes an orbit generation unit 302E in addition to a learned model storage unit 302A, a state acquisition unit 302B, a recommendation degree distribution estimation unit 302C, and a proposal unit 302D as functional units.

[0257] The learned model storage unit 302A stores the learned models LM1 and LM2 distributed from the information processing device 200.

[0258] Based on the data representing the state of the objects in the current working area of the excavator 100 obtained by the state acquisition unit 302B, the trajectory generation unit 302E generates a trajectory of the working part in a predetermined operation of the excavator 100 (hereinafter, for convenience, referred to as the "recommended trajectory"). Specifically, the trajectory generation unit 302E applies the learned model LM2 based on the data representing the state of the objects in the current working area of the excavator 100 to generate the recommended trajectory of the working part in a predetermined operation of the excavator 100. For example, based on the estimation result of the recommendation degree distribution estimation unit 302C, the trajectory generation unit 302E generates a recommended trajectory for changing the shape of the object at a location with a relatively high recommendation degree as the working location of the excavator 100 in the working area by a predetermined operation. The location with a relatively high recommendation degree as the working location of the excavator 100 in the working area corresponds to the location proposed as the working location by the proposal unit 302D. The locations with a relatively high recommendation degree as the working location of the excavator 100 include, for example, the location with the highest recommendation degree as the working location of the excavator 100. Also, the locations with a relatively high recommendation degree as the working location of the excavator 100 may include the locations where the recommendation degree as the working location of the excavator 100 reaches a maximum.

[0259] In addition, when the learned model LM3 is generated, the functions of the recommendation degree distribution estimation unit 302C and the trajectory generation unit 302E may be integrated into one functional unit. In this case, the integrated functional unit uses the learned model LM3 to estimate the distribution of the recommendation degree as the working location of the excavator 100 in the working area and the recommended trajectory of the working part in a predetermined operation of the excavator 100.

[0260] In addition to proposing the working location of the excavator 100 in the working area, the proposal unit 302D proposes the recommended trajectory of the working part in a predetermined operation of the excavator 100 at that working location generated by the trajectory generation unit 302E.

[0261] For example, in the screen 800 of FIG. 8 described above, the proposal unit 302D displays, at the corresponding positions instead of the recommendation degree distribution images 804A to 804C, images that two-dimensionally represent the recommended trajectories of the working parts of the shovel 100. Thereby, the proposal unit 302D can propose the working location of the shovel 100 in the working area based on the position of the image in the working area image 801, and can propose the recommended trajectory for a predetermined operation of the shovel 100 based on the shape of the image.

[0262] Also, in the screen 900 of FIG. 9 described above, the proposal unit 302D displays, at the corresponding positions instead of the recommendation degree distribution images 904A to 904C, images that three-dimensionally represent the recommended trajectories of the working parts of the shovel 100 as viewed from the cab 10. Thereby, the proposal unit 302D can propose the working location of the shovel 100 in the working area based on the position of the image in the working area image 901, and can propose the recommended trajectory for a predetermined operation of the shovel 100 based on the shape of the image. Similarly, in the screen 1000 of FIG. 10 described above, the proposal unit 302D displays, at the corresponding positions instead of the recommendation degree distribution images 1004A to 1004C, images that three-dimensionally represent the recommended trajectories of the working parts of the shovel 100 as viewed from the front of the shovel 100. Thereby, the proposal unit 302D can propose the working location of the shovel 100 in the working area based on the position of the image in the working area image 1001, and can propose the recommended trajectory for a predetermined operation of the shovel 100 based on the shape of the image.

[0263] Further, in any of the screens 800, 900, and 1000 of FIGS. 8 to 10 described above, the proposal unit 302D makes the colors of the images representing the recommended trajectories of the working parts of the shovel 100 at the three locations different according to the recommendation degree as the working location of that location. Thereby, the proposal unit 302D can notify the user of the recommendation degree as the working location of the location in the working area corresponding to the position where the image representing the recommended trajectory of the working part of the shovel 100 is displayed, based on the color of the image.

[0264] Further, in any of the screens 800, 900, and 1000 of FIGS. 8 to 10 described above, the proposal unit 302D may display numerically the recommended degree of the work location at each of the three locations as associated with an image representing the recommended trajectory of the working part of each of the excavators 100 at that location. Thereby, the proposal unit 302D can represent the difference in the recommended degree of the work location for each location in the work area corresponding to the position representing the recommended trajectory of the working part of the excavator 100 by the difference in numerical values.

[0265] In addition, part or all of the functions of the state acquisition unit 302B, the recommended degree distribution estimation unit 302C, the proposal unit 302D, and the trajectory generation unit 302E may be transferred outside the excavator 100. For example, the functions of the state acquisition unit 302B, the recommended degree distribution estimation unit 302C, the proposal unit 302D, and the trajectory generation unit 302E may be transferred to the information processing device 200, and the display content of the display device 50A may be controlled from the information processing device 200. Further, for example, when the excavator 100 is remotely operated, the functions of the state acquisition unit 302B, the recommended degree distribution estimation unit 302C, the proposal unit 302D, and the trajectory generation unit 302E may be transferred to the remote operation support device 400. In these cases, for example, the outputs of the sensors 40 and S1 to S9 are uploaded in real time to the information processing device 200 or the remote operation support device 400 through the communication device 60.

[0266] [Third Example of Processing for Proposal of Work Location] Next, with reference to FIG. 14, a third example of the processing for the proposal of the work location of the excavator 100 will be described.

[0267] FIG. 14 is a flowchart schematically showing a third example of the processing for the proposal of the work location of the excavator 100.

[0268] As shown in FIG. 14, since the processes in steps S30 and S31 are the same as the processes in steps S10 and S11 in FIG. 11, the description thereof is omitted.

[0269] When the process in step S31 is completed, the controller 30 proceeds to step S32.

[0270] In step S32, based on the estimation result of the recommendation degree distribution estimation unit 302C, the trajectory generation unit 302E generates a recommended trajectory for changing the shape of an object at a location with a relatively high recommendation degree as the working location of the excavator 100 in the work area by a predetermined operation.

[0271] When the process of step S32 is completed, the controller 30 proceeds to step S33.

[0272] In step S33, based on the results of the processes in steps S31 and S32, the proposal unit 302D visually proposes, through the display device 50A or the display device 408, the working location of the excavator 100 in the work area and the recommended trajectory of the working part by a predetermined operation of the excavator 100 at that working location.

[0273] When the process of step S33 is completed, the controller 30 proceeds to step S34.

[0274] Since the processes in steps S34 and S35 are the same as the processes in steps S13 and S14 in FIG. 11, the description is terminated.

[0275] [Third Example of the Functional Configuration of the Operation Support System] Next, in addition to FIGS. 1 to 5, with reference to FIG. 15, a third example of the functional configuration of the operation support system SYS will be described.

[0276] Hereinafter, in this example, the same or corresponding components as those in the above-described first example (FIG. 6) and second example (FIG. 13) are denoted by the same reference numerals, and the description will focus on the parts different from the above-described first example and second example, and the description of the same or corresponding content as the above-described first example and second example may be omitted.

[0277] In this example, mainly, the controller 30 includes an operation control unit 302F, which is different from the second example described above.

[0278] Similar to the first example described above, the controller 30 of the excavator 100 includes, as functional units, an operation log providing unit 301 and a work support unit 302.

[0279] As a functional unit, the operation support unit 302 includes an operation control unit 302F in addition to a learned model storage unit 302A, a state acquisition unit 302B, a recommendation degree distribution estimation unit 302C, a proposal unit 302D, and a trajectory generation unit 302E.

[0280] The operation control unit 302F causes the excavator 100 to automatically perform an operation according to a predetermined input in response to a predetermined input from the user through the input device 52 or the input device 407.

[0281] For example, when one work location is selected by the user through the input device 52 or the input device 407 from the work locations proposed by the proposal unit 302D, the operation control unit 302F causes the excavator 100 to automatically perform a predetermined operation. Specifically, the operation control unit 302F controls the hydraulic actuator HA so that the working part moves along the recommended trajectory corresponding to the selected work location. Thereby, the user can cause the excavator 100 to perform a predetermined operation so that the working part moves along the recommended trajectory only by selecting a work location with the input device 52 or the like, without operating the operation device 26 or the operation device for remote operation. Therefore, even a user with little experience in operating the excavator 100 can efficiently proceed with the work. As a result, the controller 30 can improve the work efficiency of the excavator 100.

[0282] For example, the user can perform an operation of selecting one of the images representing the recommended trajectory of the working part on the screen displayed on the display device 50A or the like through the input device 52 or the like. Then, the operation control unit 302F causes the excavator 100 to perform a predetermined operation so that the working part moves along the recommended trajectory corresponding to the selected image representing the recommended trajectory.

[0283] Furthermore, part or all of the functions of the state acquisition unit 302B, the degree-of-recommendation distribution estimation unit 302C, the proposal unit 302D, the trajectory generation unit 302E, and the operation control unit 302F may be transferred outside the excavator 100. For example, the functions of the state acquisition unit 302B, the degree-of-recommendation distribution estimation unit 302C, the proposal unit 302D, the trajectory generation unit 302E, and the operation control unit 302F may be transferred to the information processing device 200, and the display content of the display device 50A and the operation of the excavator 100 may be controlled from the information processing device 200. Also, for example, when the excavator 100 is remotely operated, the functions of the state acquisition unit 302B, the degree-of-recommendation distribution estimation unit 302C, the proposal unit 302D, the trajectory generation unit 302E, and the operation control unit 302F may be transferred to the remote operation support device 400. In these cases, for example, the outputs of the sensors 40 and S1 to S9 are uploaded in real time to the information processing device 200 and the remote operation support device 400 through the communication device 60.

[0284] [Fourth Example of Processing Related to Proposal of Work Location] Next, with reference to FIG. 16, a fourth example of the processing related to the proposal of the work location of the excavator 100 will be described.

[0285] FIG. 16 is a flowchart schematically showing a fourth example of the processing related to the proposal of the work location of the excavator 100.

[0286] As shown in FIG. 16, the processing in steps S40 to S43 is the same as the processing in steps S30 to S33 in FIG. 14, and thus the description thereof is omitted.

[0287] When the processing in step S43 is completed, the controller 30 proceeds to step S44.

[0288] In step S44, the controller 30 determines whether or not an operation has been performed to select and confirm one work location from among the work locations visually proposed through the display device 50A via the input device 52 or the input device 407. If the target operation has been performed, the controller 30 proceeds to step S45; otherwise, the processing in step S44 is repeated.

[0289] In step S45, the operation control unit 302F performs operation control to cause the excavator 100 to perform a predetermined operation so that the working part moves along the recommended function corresponding to one working place selected in step S44.

[0290] When the process of step S45 is completed, the controller 30 proceeds to step S46.

[0291] Since the process of step S46 is the same as the process of step S14 in FIG. 11, the description thereof is omitted.

[0292] [Other Embodiments] In the above-described embodiments, appropriate modifications and changes may be made as appropriate, such as by combining or replacing their contents.

[0293] For example, in the above-described embodiment, the display device 50A may be a head-up display. In this case, under the control of the proposal unit 302D, the display device 50A displays an image (for example, the above-described recommendation degree distribution images 904A to 904C) that proposes the working place in the working area within the field of view when the user views the working area around the excavator 100 from the operator's seat.

[0294] Also, in the above-described embodiment, instead of the display device 50A or the display device 408, the proposal unit 302D may visually propose the working place of the excavator 100, the recommended trajectory, etc. through smart glasses or AR (Augmented Reality) glasses worn by the user.

[0295] Also, in the above-described embodiment, the display device 50A may be a projection device that projects an image onto the ground in the working area around the excavator 100 and displays information to the user in the cabin 10 or the remotely operated user by applying a technique such as projection mapping. In this case, the proposal unit 302D may control the projection device to project an image that proposes the working place of the excavator 100 in the working area onto the very place corresponding to the working place in the working area.

[0296] In addition, the proposed methods for workplaces and recommended trajectories in the above-described embodiments and their variations and modifications may be applied to other construction machines different from the excavator 100. For example, another construction machine is a continuous unloader. In this case, the operation support system SYS may propose a workplace for scraping the bulk cargo (e.g., iron ore, coal, etc.) in the ship's hold by the lower part (scraping part) of the bucket elevator of the continuous unloader, and may also propose a recommended trajectory for the scraping part as a working part.

[0297] [Function] Next, the functions of the construction machine, remote operation support device, and operation support system according to this embodiment will be described.

[0298] In the first aspect of this embodiment, the construction machine includes a working device and a display device. The construction machine is the above-described excavator 100. Alternatively, the construction machine may be the above-described continuous unloader. The working device is, for example, the above-described attachment AT. Alternatively, the working device may be, for example, the bucket elevator of the continuous unloader. The display device is, for example, the above-described display device 50A. Specifically, the working device operates to change the shape of an object in the working area of the construction machine. Then, based on the information representing the current shape of the object in the working area of the construction machine, the display device visually proposes a location of the working object in the working area of the construction machine where the working device changes the shape of the object.

[0299] Thereby, the user can determine the location of the working object from the working area using the display content of the display device as a reference. Therefore, the construction machine can appropriately assist the operation by the user.

[0300] In addition, in the second aspect of this embodiment, on the premise of the first aspect described above, the display device may display in the working area of the construction machine in a distinguishable manner a location with a relatively high recommendation degree as the working object location and a location with a relatively low recommendation degree.

[0301] As a result, the user can determine the location of the work target from within the work area using the degree of recommendation of the location of the work target as a judgment material.

[0302] In addition, in the third aspect of the present embodiment, on the premise of the above-described second aspect, the display device may display the difference in the degree of recommendation by a difference in color or a difference in numerical value.

[0303] As a result, the user can easily grasp the difference in the degree of recommendation of the location of the work target for each location within the work area.

[0304] In addition, in the fourth aspect of the present embodiment, on the premise of the above-described third aspect, the display device may display the difference in the degree of recommendation by a heat map.

[0305] As a result, the user can easily grasp the difference in the degree of recommendation of the location of the work target for each location within the work area.

[0306] In addition, in the fifth aspect of the present embodiment, on the premise of any one of the above-described second to fourth aspects, the display device may display the location where the degree of recommendation takes a maximum value within the work area of the work machine.

[0307] As a result, the work machine can propose to the user the location where the degree of recommendation of the location of the work target takes a maximum value within the work area as the location of the work target.

[0308] In addition, in the sixth aspect of the present embodiment, on the premise of any one of the above-described second to fifth aspects, when there is no work location where the degree of recommendation is relatively high with respect to a predetermined standard within the work area of the work machine, the display device may display that there is no proposed work location.

[0309] As a result, the work machine can suppress a situation in which, for example, the work efficiency and safety are deteriorated by proposing a location with a relatively low degree of recommendation as the location of the work target.

[0310] Further, in the seventh aspect of the present embodiment, on the premise of any one of the above-described second to sixth aspects, the level of the recommendation degree may be defined so as to be associated with the level of the working efficiency or safety of the working machine.

[0311] Thereby, the working machine can propose a location of a work target so that the working efficiency and safety of the working machine are improved.

[0312] Further, in the eighth aspect of the present embodiment, on the premise of the above-described seventh aspect, the working machine may include a processing device. The processing device is, for example, the above-described controller 30. Specifically, the processing device uses a learned model that has been machine-learned by teacher data regarding the operation of the working machine by an operator with a relatively high proficiency level with respect to a predetermined standard, which is associated with the shape of an object in the working area of the working machine, and based on information representing the current shape of the object in the working area of the working machine, may output a distribution of the recommendation degree in the working area of the working machine. An operator with a relatively high proficiency level with respect to a predetermined standard is, for example, the above-described skilled person. The learned model is, for example, the above-described learned model LM1.

[0313] Thereby, the working machine can propose to the user a location of a work target that reflects the working efficiency and safety viewpoints in the operation of an operator with a high proficiency level.

[0314] Further, in the ninth aspect of the present embodiment, on the premise of any one of the above-described first to eighth aspects, the display device may visually propose a plurality of the work target locations in time series based on information representing the current shape of the object in the working area of the working machine.

[0315] Thereby, the working machine can, for example, collectively propose the locations of the work target corresponding to a plurality of operations. Therefore, the working machine can improve the convenience and working efficiency of the user.

[0316] Further, in the tenth aspect of the present embodiment, on the premise of any one of the above-described first to ninth aspects, the display device may visually propose a trajectory of a working part of the working device when changing the location of the work object and the shape of the object at the location of the work object in the working area of the working machine based on information representing the current shape of the object in the working area of the working machine. The working part is, for example, the above-described bucket 6. Further, the working part may be the scraping part of the above-described continuous unloader.

[0317] Thereby, the working machine can propose not only the location of the work object but also the trajectory of the working part for changing the shape of the object at that location.

[0318] Further, in the eleventh aspect of the present embodiment, on the premise of any one of the above-described first to tenth aspects, the display device may display an image for proposing the location of the work object in the working area of the working machine, superimposed on an image representing the shape of the object in the working area of the working machine. The image representing the shape of the object in the working area of the working machine is, for example, the above-described working area images 801, 901, 1001. The image for proposing the location of the work object is, for example, the above-described recommendation degree distribution images 804A to 804C, recommendation degree distribution images 904A to 904C, and recommendation degree distribution images 1004A to 1004C.

[0319] Thereby, the working machine can propose the location of the work object in the working area in association with the shape of the object in the working area.

[0320] Further, in the twelfth aspect of the present embodiment, on the premise of any one of the above-described first to tenth aspects, the display device may display an image for proposing the location of the work object in the working area of the working machine within the field of view when the user views the working area of the working machine from the driver's seat.

[0321] Thereby, the working machine can propose the location of the work object in the working area in association with the shape of the object in the working area.

[0322] Further, in the 13th aspect of the present embodiment, on the premise of any one of the above-described 1st to 10th aspects, the display device may display by projecting an image that proposes a location of the work object in the work area of the work machine onto the work area of the work machine.

[0323] Thereby, the work machine can propose the location of the work object in the work area in association with the shape of the object in the work area.

[0324] Further, in the 14th aspect of the present embodiment, the remote operation support device includes an operation unit, a communication unit, and a display unit. The remote operation support device is, for example, the above-described remote operation support device 400. The operation unit is, for example, a remote operation device as the above-described input device 407. The communication unit is, for example, the above-described communication interface 406. The display unit is, for example, the above-described display device 408. Specifically, the operation unit is used for a user to remotely operate a work machine including a work device that operates to change the shape of an object in the work area. The communication unit transmits information representing the operation state of the operation unit to the work machine. Then, the display unit visually proposes a location of a work object where the work device changes the shape of the object in the work area of the work machine based on information representing the shape of the object in the work area of the work machine.

[0325] Thereby, the user performing the remote operation can determine the location of the work object from the work area using the display content of the display unit as a reference. Therefore, the remote operation support device can appropriately support the operation of the work machine by the user.

[0326] In addition, in the 15th aspect of the present embodiment, the support system supports the operation by the user of a working machine including a working device that operates to change the shape of an object in the working area. The support system is, for example, the operation support system SYS described above. Specifically, the support system includes a display unit. The display unit is, for example, the display device 50A or the display device 408 described above. Further, the display unit may be, for example, the smart glasses or the AR goggles described above. More specifically, the display unit visually proposes to the user the location of the work target where the working device changes the shape of the object in the working area of the working machine based on the information representing the shape of the object in the working area of the working machine.

[0327] As a result, the user can determine the location of the work target from the working area using the display content of the display unit as a reference. Therefore, the support system can appropriately support the operation of the working machine by the user.

[0328] Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist described in the claims.

Description of Reference Numerals

[0329] 10 Cabin 26 Operating Device 30 Controller 31 Hydraulic Control Valve 40 Sensor 40B, 40F, 40L, 40R Sensors 50 Output Device 50A Display Device 52 Input Device 60 Communication Device 100 Excavator 200 Information Processing Device 300 Sensor Group 300-1 to 300-M Sensors 301 Operation Log Providing Unit 301A Operation Log Recording Unit 301B Operation Log Storage Unit 301C Action Log Sending Unit 302 Work Support Unit 302A Trained Model Memory Unit 302B State Acquisition Unit 302C Degree of Recommendation Distribution Estimation Unit 302D Proposal Unit 302E Orbit Generation Unit 302F Action Control Unit 400 Remote Operation Support Device 406 Communication Interface 407 Input Device 408 Display Device 800 Screen 801 Work Area Image 802 Excavator Image 803 Target Shape Image 804 Degree of Recommendation Distribution Image 804A~804C Degree of Recommendation Distribution Image 900 Screen 901 Work Area Image 904 Degree of Recommendation Distribution Image 904A~904C Degree of Recommendation Distribution Image 1000 Screen 1001 Work Area Image 1002 Excavator Image 1004 Degree of Recommendation Distribution Image 1004A~1004C Degree of Recommendation Distribution Image 2001 Log Acquisition Unit 2002 Simulator Unit 2003 Log Memory Unit 2004 Teacher Data Generation Unit 2004A,2004B Teacher Data Generation Unit 2005 Machine Learning Unit 2005A,2005B Machine Learning Unit 2006 Trained Model Memory Unit 2007 Distribution Unit AT Attachment LM1~LM3 Trained Model S1~S9 Sensor SYS Operation Support System

Claims

1. A working device that operates to change the shape of an object in the working area of a working machine, and a display device that visually proposes a location of a working target where the working device changes the shape of the object in the working area of the working machine based on information representing the current shape of the object in the working area of the working machine. A working machine comprising: A working machine.

2. The display device displays in the working area of the working machine so as to be able to distinguish a location with a relatively high degree of recommendation as the location of the working target from a location with a relatively low degree of recommendation. The working machine according to claim 1.

3. The display device displays the difference in the degree of recommendation by a difference in color or a difference in numerical value. The working machine according to claim 2.

4. The display device displays the difference in the degree of recommendation by a heat map. The working machine according to claim 3.

5. The display device displays a location where the degree of recommendation takes a maximum value in the working area of the working machine. The working machine according to any one of claims 2 to 4.

6. When there is no working place in the working area of the working machine where the degree of recommendation is relatively high with respect to a predetermined standard, the display device displays that there is no proposed working place. The working machine according to any one of claims 2 to 4.

7. The high or low degree of recommendation is defined to be associated with the high or low working efficiency or safety of the working machine. The working machine according to any one of claims 2 to 4.

8. Using a learned model that has been machine-learned by teacher data regarding the operation of the working machine by an operator with a relatively high level of proficiency corresponding to the shape of the object in the working area of the working machine, based on information representing the current shape of the object in the working area of the working machine, a processing device that outputs the distribution of the degree of recommendation in the working area of the working machine is provided. The working machine according to claim 7.

9. The display device visually proposes a plurality of the locations of the working target in time series based on information representing the current shape of the object in the working area of the working machine. The working machine according to any one of claims 1 to 4.

10. The display device visually proposes the location of the working target in the working area of the working machine and the trajectory of the working part of the working device when changing the shape of the object at the location of the working target based on information representing the current shape of the object in the working area of the working machine. The working machine according to any one of claims 1 to 4.

11. The display device displays an image that proposes the location of the work target in the work area of the work machine, superimposed on an image representing the shape of an object in the work area of the work machine. The work machine according to any one of claims 1 to 4.

12. The display device displays an image that proposes the location of the work target in the work area of the work machine within the field of view when the user views the work area of the work machine from the operator's seat. The work machine according to any one of claims 1 to 4.

13. The display device is configured to display by projecting an image that proposes the location of the work target in the work area of the work machine onto the work area of the work machine. The work machine according to any one of claims 1 to 4.

14. An operation unit for a user to remotely operate a work machine provided with a work device that operates to change the shape of an object in the work area, A communication unit that transmits information representing the operation state of the operation unit to the work machine, A display unit that visually proposes a work target location where the work device changes the shape of an object in the work area of the work machine based on information representing the current shape of the object in the work area of the work machine. Remote operation support device.

15. A support system for assisting an operation by a user of a work machine provided with a work device that operates to change the shape of an object in the work area, A display unit that visually proposes to the user a work target location where the work device changes the shape of an object in the work area of the work machine based on information representing the current shape of the object in the work area of the work machine. Support system.

Images