Support devices, work machines, and programs

Abstract

To provide a technique capable of more easily generating a locus of a work portion of a work machine.SOLUTION: A support device 150 according to one embodiment of the present disclosure comprises: a work object shape acquisition unit 302B which acquires data about a topographic shape of a construction object in the periphery of a shovel 100; a display device 50A which displays an image expressing the topographic shape in the periphery of the shovel 100 on the basis of the data acquired by the work object shape acquisition unit 302B; a condition setting unit 302D which sets a point being a target in the time of the operation of the shovel 100 in the topographic shape in the periphery of the shovel 100 according to the input from a user; and a locus generation unit 302E which generates a locus of a bucket 6 of the shovel 100 on the basis of the data acquired by the work object shape acquisition unit 302B and the point set by the condition setting unit 302D.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to an assistance device for a work machine. [Background technology]

[0002] For example, a technique has been disclosed in which teaching points are set and a trajectory of a working portion of a work machine is generated based on the teaching points (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-50576 Summary of the Invention [Problem to be solved by the invention]

[0004] However, for example, in Patent Document 1, it is necessary to set teaching points by actually operating a shovel and operating the work machine, which may result in a lot of time and effort being required to generate the target trajectory for the work area.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a technique that can more easily generate a trajectory of a working portion of a work machine. [Means for solving the problem]

[0006] In order to achieve the above object, in one embodiment of the present disclosure, an acquisition unit that acquires data relating to the shape of a work target in the vicinity of the work machine; a display unit that displays an image representing the shape of the work object based on the data acquired by the acquisition unit; a setting unit that sets a point on the shape of the work object that will be a target when the work machine is operating; a generation unit that generates a trajectory of a working portion of the work machine based on the data acquired by the acquisition unit, the target shape of the work object, and the points set by the setting unit, Support devices are provided.

[0007] In another embodiment of the present disclosure, The above-mentioned support device is provided. A work machine is provided.

[0008] In still another embodiment of the present disclosure, Support devices include: an acquisition step of acquiring data relating to the shape of a work object around the work machine; a display step of displaying an image representing the shape of the work object on a display unit based on the data acquired in the acquisition step; a setting step of setting a point on the shape of the work object that will be a target when the work machine is operating; a generating step of generating a trajectory of a working portion of the work machine based on the data acquired in the acquiring step, the target shape of the work object, and the points set in the setting step; Programs are offered. [Effects of the Invention]

[0009] According to the above-described embodiment, the trajectory of the working portion of the work machine can be generated more easily. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of an operation support system. [Figure 2] FIG. 1 is a top view showing an example of a shovel. [Figure 3] FIG. 2 is a diagram illustrating an example of a configuration for remotely controlling a shovel. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of the shovel. [Figure 5] FIG. 2 illustrates an example of a hardware configuration of an information processing device. [Figure 6] FIG. 2 is a functional block diagram showing an example of a functional configuration related to generation of a target trajectory of a working part of a shovel. [Figure 7] FIG. 10 is a diagram showing an example of a screen related to generation of a target trajectory of a working part of a shovel. [Figure 8] FIG. 10 is a diagram showing another example of a screen relating to generation of a target trajectory of a working part of a shovel. [Figure 9] FIG. 10 is a diagram showing yet another example of a screen relating to generation of a target trajectory of a working part of a shovel. [Figure 10] 10 is a flowchart illustrating an example of a process for generating a target trajectory of a working portion of a shovel. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment will be described with reference to the drawings.

[0012] [Outline of the operation support system] First, an overview of the operation support system SYS according to this embodiment will be described with reference to FIGS.

[0013] FIG. 1 is a diagram showing an example of an operation support system SYS. In FIG. 1, a left side view of a shovel 100 is shown. FIG. 2 is a top view showing an example of the shovel 100. FIG. 3 is a diagram showing an example of a configuration related to remote operation of a shovel. Hereinafter, the direction in which the attachment AT extends when viewed from above the shovel 100 (the upward direction in FIG. 2) will be defined as "front," and directions on the shovel 100 or directions seen from the shovel 100 may be described.

[0014] As shown in FIG. 1, the operation support system SYS includes an excavator 100 and an information processing device 200.

[0015] The operation support system SYS uses the information processing device 200 to cooperate with the shovel 100 and provide support regarding the operation of the shovel 100.

[0016] The operation support system SYS may include one or more excavators 100.

[0017] The excavator 100 is a work machine that receives support for operation in the operation support system SYS.

[0018] As shown in FIGS. 1 and 2, the excavator 100 includes a lower traveling body 1, an upper rotating body 3, an attachment AT including a boom 4, an arm 5, and a bucket 6, and a cabin 10.

[0019] The lower traveling body 1 uses crawlers 1C to travel the excavator 100. The crawlers 1C include a left crawler 1CL and a right crawler 1CR. The crawlers 1CL are hydraulically driven by a traveling hydraulic motor 1ML. Similarly, the crawlers 1CL are hydraulically driven by a traveling hydraulic motor 1MR. This allows the lower traveling body 1 to travel independently.

[0020] The upper rotating body 3 is rotatably mounted on the lower traveling body 1 via a rotating mechanism 2. For example, the upper rotating body 3 rotates relative to the lower traveling body 1 when the rotating mechanism 2 is hydraulically driven by a rotating hydraulic motor 2M.

[0021] Boom 4 is attached to the front center of upper rotating body 3 so as to be able to tilt up and down about a rotation axis that runs along the left-right direction. Arm 5 is attached to the tip of boom 4 so as to be able to rotate about a rotation axis that runs along the left-right direction. Bucket 6 is attached to the tip of arm 5 so as to be able to rotate about a rotation axis that runs along the left-right direction.

[0022] The bucket 6 is an example of an end attachment, and is used, for example, for excavation work.

[0023] The bucket 6 is attached to the tip of the arm 5 in a manner that allows it to be appropriately replaced depending on the work content of the excavator 100. In other words, instead of the bucket 6, a bucket of a different type from the bucket 6, such as a relatively large bucket, a slope bucket, or a dredging bucket, may be attached to the tip of the arm 5. Also, a type of end attachment other than a bucket, such as a mixer, breaker, or crusher, may be attached to the tip of the arm 5. Also, a spare attachment such as a quick coupling or a tiltrotator may be provided between the arm 5 and the end attachment.

[0024] The boom 4, arm 5, and bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0025] The cabin 10 is a control room where an operator sits and operates the excavator 100. The cabin 10 is mounted on the front left side of the upper rotating body 3, for example.

[0026] For example, the excavator 100 operates driven elements such as the lower traveling body 1 (i.e., a pair of left and right crawlers 1CL, 1CR), upper rotating body 3, boom 4, arm 5, and bucket 6 in response to operations by an operator seated in the cabin 10.

[0027] Furthermore, instead of or in addition to being configured to be operable by an operator inside the cabin 10, the shovel 100 may be configured to be remotely operable from outside the shovel 100. When the shovel 100 is remotely operated, the inside of the cabin 10 may be unmanned. The following description will be given on the assumption that the operation of the operator includes at least one of operation of the operating device 26 by the operator inside the cabin 10 and remote operation by an external operator.

[0028] For example, as shown in FIG. 3, remote operation includes a mode in which the shovel 100 is operated by operation input relating to the actuator of the shovel 100 performed by a remote operation support device 300.

[0029] The remote operation support device 300 is provided, for example, in a management center that externally manages the work of the shovel 100. The remote operation support device 300 may also be a portable operation terminal, in which case the operator can remotely operate the shovel 100 while directly checking the work status of the shovel 100 from the vicinity of the shovel 100.

[0030] The shovel 100 may transmit, for example, via a communication device 60 described below, an image (hereinafter referred to as a "peripheral image") showing the surroundings including the area in front of the shovel 100, based on an image captured by an imaging device 40 described below. The remote operation support device 300 may then display the image (peripheral image) received from the shovel 100 on a display device. Furthermore, various information images (information screens) displayed on the output device 50 (display device 50A) inside the cabin 10 of the shovel 100 may also be displayed on the display device of the remote operation support device 300. This allows the operator using the remote operation support device 300 to remotely operate the shovel 100 while checking, for example, the contents of the image, information screen, or the like showing the surroundings of the shovel 100 displayed on the display device. The excavator 100 may operate actuators and drive driven elements such as the lower running body 1, upper rotating body 3, boom 4, arm 5, and bucket 6 in accordance with a remote control signal indicating the content of the remote control, which is received by the communication device 60 from the remote control support device 300.

[0031] Remote control may also include, for example, a mode in which the shovel 100 is operated by an external voice input or gesture input to the shovel 100 by a person (e.g., a worker) around the shovel 100. Specifically, the shovel 100 recognizes voices uttered by surrounding workers or gestures made by the workers through a voice input device (e.g., a microphone) or a gesture input device (e.g., an imaging device) mounted on the shovel 100. Then, the shovel 100 may operate actuators in accordance with the content of the recognized voices or gestures to drive driven elements such as the lower traveling body 1 (left and right crawlers 1C), upper rotating body 3, boom 4, arm 5, and bucket 6.

[0032] Furthermore, the work of the shovel 100 may be remotely monitored. In this case, a remote monitoring support device having the same functions as the remote operation support device 300 may be provided. The remote monitoring support device is, for example, the information processing device 200. This allows a monitor, who is a user of the remote monitoring support device, to monitor the status of the work of the shovel 100 while checking a peripheral image displayed on a display device of the remote monitoring support device. Furthermore, for example, if the monitor determines it is necessary from a safety standpoint, the monitor can intervene in the operation by the operator of the shovel 100 and bring about an emergency stop by making a predetermined input using an input device of the remote monitoring support device.

[0033] The information processing device 200 communicates with the shovel 100 to cooperate with each other and provide support regarding the operation of the shovel 100.

[0034] The information processing device 200 is, for example, a server or a management terminal device installed in an administration office within the work site of the shovel 100, or in a management center that manages the operating status of the shovel 100 and is located in a place different from the work site of the shovel 100. The management terminal device may be, for example, a fixed terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile terminal) such as a tablet terminal, a smartphone, or a laptop PC. In the latter case, workers at the work site, supervisors who supervise the work, managers who manage the work site, etc. can move around the work site carrying the portable information processing device 200. In addition, in the latter case, an operator can, for example, bring the portable information processing device 200 into the cabin of the shovel 100.

[0035] The information processing device 200 acquires, for example, data relating to the operating state from the shovel 100. This enables the information processing device 200 to grasp the operating state of the shovel 100 and monitor the presence or absence of abnormalities in the shovel 100. Furthermore, the information processing device 200 displays data relating to the operating state of the shovel 100 via a display device 208, which will be described later, and allows the user to confirm the data.

[0036] Furthermore, the information processing device 200 transmits to the shovel 100, for example, various data such as programs and reference data used in the processing of the controller 30, etc., to the shovel 100. This allows the shovel 100 to perform various processes related to the operation of the shovel 100 using the various data downloaded from the information processing device 200.

[0037] [Hardware configuration of the operation support system] Next, the hardware configuration of the operation support system SYS will be described with reference to FIGS. 4 and 5 in addition to FIGS.

[0038] <Excavator hardware configuration> FIG. 4 is a block diagram showing an example of the hardware configuration of the shovel 100.

[0039] In Figure 4, the paths through which mechanical power is transmitted are indicated by double lines, the paths through which high-pressure hydraulic oil that drives the hydraulic actuator flows are indicated by solid lines, the paths through which pilot pressure is transmitted are indicated by dashed lines, and the paths through which electrical signals are transmitted are indicated by dotted lines.

[0040] The shovel 100 includes various components, such as a hydraulic drive system for hydraulically driving the driven elements, an operation system for operating the driven elements, a user interface system for exchanging information with the user, a communication system for communicating with the outside world, and a control system for various controls.

[0041] <Hydraulic drive system> 4, the hydraulic drive system of the excavator 100 includes the hydraulic actuator HA that hydraulically drives each of the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, and the attachment AT, as described above. The hydraulic drive system of the excavator 100 according to this embodiment also includes the engine 11, a regulator 13, a main pump 14, and a control valve 17.

[0042] The hydraulic actuator HA includes traveling hydraulic motors 1ML, 1MR, a swing hydraulic motor 2M, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and the like.

[0043] Note that the hydraulic actuators HA of the shovel 100 may be partially or entirely replaced with electric actuators. In other words, the shovel 100 may be a hybrid shovel or an electric shovel.

[0044] The engine 11 is the prime mover of the excavator 100 and the main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, on the rear of the upper rotating body 3. The engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15.

[0045] It should be noted that instead of or in addition to the engine 11, the excavator 100 may be equipped with another prime mover (for example, an electric motor).

[0046] The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter referred to as the "tilting angle") in response to a control command from the controller 30.

[0047] The main pump 14 supplies hydraulic oil to the control valve 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13, thereby adjusting the stroke length of the piston and controlling the discharge flow rate and discharge pressure.

[0048] The control valve 17 drives the hydraulic actuators HA in response to an operator's operation of the operating device 26, the details of remote operation, or an operation command corresponding to the automatic operation function. The control valve 17 is mounted, for example, in the center of the upper rotating body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line, and selectively supplies hydraulic oil supplied from the main pump 14 to each hydraulic actuator in response to an operator's operation or an operation command corresponding to the automatic operation function. Specifically, the control valve 17 includes a plurality of control valves (also referred to as "directional control valves") that control the flow rate and flow direction of hydraulic oil supplied from the main pump 14 to each hydraulic actuator HA.

[0049] ≪Operation system≫ As shown in FIG. 4, the operating system of the excavator 100 includes a pilot pump 15, an operating device 26, a hydraulic control valve 31, a shuttle valve 32, and a hydraulic control valve 33.

[0050] The pilot pump 15 supplies pilot pressure to various hydraulic devices via a pilot line 25. The pilot pump 15 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11. The pilot pump 15 is, for example, a fixed displacement hydraulic pump, and is driven by the engine 11 as described above.

[0051] The pilot pump 15 may be omitted. In this case, the relatively high-pressure hydraulic oil discharged from the main pump 14 may be reduced in pressure by a predetermined pressure reducing valve, and the resulting relatively low-pressure hydraulic oil may be supplied to various hydraulic devices as pilot pressure.

[0052] The operating device 26 is provided near the cockpit of the cabin 10 and is used by the operator to operate the various driven elements. Specifically, the operating device 26 is used by the operator to operate the hydraulic actuators HA that drive the respective driven elements, thereby enabling the operator to operate the driven elements that are the targets of the hydraulic actuators HA. The operating device 26 includes pedal devices and lever devices for operating the respective driven elements (hydraulic actuators HA).

[0053] For example, as shown in Fig. 4, the operating device 26 is of a hydraulic pilot type. Specifically, the operating device 26 uses hydraulic oil supplied from the pilot pump 15 through a pilot line 25 and a pilot line 25A branching from the pilot line 25, and outputs a pilot pressure corresponding to the operation to a secondary pilot line 27A. The pilot line 27A is connected to one inlet port of a shuttle valve 32, and is connected to the control valve 17 via a pilot line 27 connected to an outlet port of the shuttle valve 32. This allows pilot pressure corresponding to the operation of various driven elements (hydraulic actuators HA) in the operating device 26 to be input to the control valve 17 via the shuttle valve 32. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation of the operating device 26 by an operator or the like.

[0054] Furthermore, the operating device 26 may be electric. In this case, the pilot line 27A, the shuttle valve 32, and the hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electric signal (hereinafter referred to as an "operation signal") corresponding to the operation content, and the operation signal is input to the controller 30. The controller 30 then outputs a control command corresponding to the operation signal, that is, a control signal corresponding to the operation content of the operating device 26, to the hydraulic control valve 31. As a result, a pilot pressure corresponding to the operation content of the operating device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26.

[0055] Furthermore, the control valves (directional control valves) that are built into the control valve 17 and drive the hydraulic actuators HA may be of an electromagnetic solenoid type. In this case, an operation signal output from the operating device 26 may be input directly to the control valve 17, i.e., the electromagnetic solenoid type control valve.

[0056] Furthermore, as described above, some or all of the hydraulic actuators HA may be replaced with electric actuators. In this case, the controller 30 may output a control command to the electric actuator or a driver that drives the electric actuator, depending on the operation content of the operation device 26 or the content of remote operation specified by the remote operation signal. Furthermore, when the excavator 100 is remotely operated, the operation device 26 may be omitted.

[0057] A hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) operated by the operating device 26 and for each drive direction of the driven element (hydraulic actuator HA) (e.g., the raising and lowering directions of the boom 4). In other words, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA. The hydraulic control valve 31 may be provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17 and configured to be able to change its flow path area (i.e., the cross-sectional area through which hydraulic oil can flow). This allows the hydraulic control valve 31 to output a predetermined pilot pressure to the secondary pilot line 27B using the hydraulic oil from the pilot pump 15 supplied through the pilot line 25B. Therefore, the hydraulic control valve 31 can indirectly apply a predetermined pilot pressure to the control valve 17 in response to a control signal from the controller 30 via a shuttle valve 32 between the pilot line 27B and the pilot line 27B. Therefore, the controller 30 can supply pilot pressure according to the operation command corresponding to the automatic operation function from the hydraulic control valve 31 to the control valve 17, thereby realizing the operation of the excavator 100 using the automatic operation function.

[0058] Furthermore, the controller 30 may, for example, control the hydraulic control valve 31 to realize remote operation of the shovel 100. Specifically, the controller 30 outputs a control signal corresponding to the content of remote operation specified in a remote operation signal received from the remote operation support device 300 to the hydraulic control valve 31 via the communication device 60. As a result, the controller 30 causes the hydraulic control valve 31 to supply a pilot pressure corresponding to the content of remote operation to the control valve 17, thereby realizing operation of the shovel 100 based on remote operation by the operator.

[0059] In addition, if the operating device 26 is electric, the controller 30 can supply pilot pressure corresponding to the operation content (operation signal) of the operating device 26 directly to the control valve 17 from the hydraulic control valve 31, thereby realizing operation of the shovel 100 based on the operation of the operator.

[0060] The shuttle valve 32 has two inlet ports and one outlet port, and outputs hydraulic oil having a higher pilot pressure of the two pilot pressures input to the two inlet ports to the outlet port. A shuttle valve 32 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 and for each drive direction of the driven element (hydraulic actuator HA). One of the two inlet ports of the shuttle valve 32 is connected to a pilot line 27A on the secondary side of the operating device 26 (specifically, the lever device or pedal device included in the operating device 26), and the other is connected to a pilot line 27B on the secondary side of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of a corresponding control valve of the control valve 17 via the pilot line 27. The corresponding control valve is a control valve that drives the hydraulic actuator that is to be operated by the lever device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, each of these shuttle valves 32 can apply the higher of the pilot pressure in pilot line 27A on the secondary side of the operating device 26 and the pilot pressure in pilot line 27B on the secondary side of the hydraulic control valve 31 to the pilot port of the corresponding control valve. In other words, by outputting a pilot pressure higher than the pilot pressure on the secondary side of the operating device 26 from the hydraulic control valve 31, the controller 30 can control the corresponding control valve regardless of the operation of the operating device 26 by the operator. Therefore, the controller 30 can control the operation of the driven elements (undercarriage 1, upper revolving body 3, attachment AT) regardless of the operating state of the operating device 26 by the operator, thereby realizing a remote operation function and an automatic driving function.

[0061] The hydraulic control valve 33 is provided in a pilot line 27A connecting the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured, for example, to be able to change its flow path area. The hydraulic control valve 33 operates in response to a control signal input from the controller 30. As a result, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is operated by an operator. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. Furthermore, for example, even when the operating device 26 is being operated, the controller 30 can reduce the pilot pressure output from the operating device 26 to make it lower than the pilot pressure output from the hydraulic control valve 31. Therefore, by controlling the hydraulic control valves 31 and 33, the controller 30 can reliably apply a desired pilot pressure to the pilot port of the control valve in the control valve 17, for example, regardless of the operation of the operating device 26. Therefore, the controller 30 can more appropriately realize the remote control function and automatic operation function of the excavator 100 by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31, for example.

[0062] <User Interface> As shown in FIG. 4, the user interface system of the excavator 100 includes an operation device 26, an output device 50, and an input device 52.

[0063] The output device 50 outputs various information to a user of the shovel 100 (for example, an operator of the cabin 10 or an external remote control operator) or to people in the vicinity of the shovel 100 (for example, a worker or a driver of a work vehicle).

[0064] For example, the output device 50 includes lighting equipment and a display device 50A (see FIG. 5) that output various types of information visually. The lighting equipment is, for example, a warning light (indicator lamp). The display device 50A is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. For example, as shown in FIG. 2, the lighting equipment and the display device 50A may be provided inside the cabin 10 and output various types of information visually to an operator or the like inside the cabin 10. Furthermore, the lighting equipment and the display device 50A may be provided, for example, on the side of the upper rotating body 3 and output various types of information visually to workers or the like around the excavator 100.

[0065] Furthermore, for example, the output device 50 includes a sound output device that outputs various types of information by auditory means. Sound output devices include, for example, buzzers, speakers, etc. The sound output device may be provided, for example, inside or outside the cabin 10, and may output various types of information by auditory means to an operator inside the cabin 10 or to people (workers, etc.) around the excavator 100.

[0066] Furthermore, for example, the output device 50 may include a device that outputs various types of information in a tactile manner, such as by vibrating the cockpit.

[0067] The input device 52 receives various inputs from the user of the excavator 100, and signals corresponding to the received inputs are taken into the controller 30. The input device 52 is provided, for example, inside the cabin 10, and receives inputs from an operator or the like inside the cabin 10. The input device 52 may also be provided, for example, on the side of the upper rotating body 3, and receive inputs from workers or the like around the excavator 100.

[0068] For example, the input device 52 includes an operation input device that receives input from a user through mechanical operation. The operation input device may include a touch panel mounted on the display device, a touch pad installed around the display device, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), and the like.

[0069] Furthermore, for example, the input device 52 may include an audio input device that accepts audio input from the user. The audio input device includes, for example, a microphone.

[0070] Furthermore, for example, the input device 52 may include a gesture input device that accepts gesture inputs from the user. The gesture input device includes, for example, an imaging device that captures an image of a gesture made by the user.

[0071] Furthermore, for example, the input device 52 may include a biometric input device that accepts biometric input from the user. The biometric input includes input of biometric information such as the user's fingerprint or iris.

[0072] <Communications> As shown in FIG. 4, the communication system of the shovel 100 according to this embodiment includes a communication device 60.

[0073] The communication device 60 connects to an external communication line and communicates with a device provided separately from the shovel 100. The device provided separately from the shovel 100 may include a device external to the shovel 100, as well as a portable terminal device (mobile terminal) brought into the cabin 10 by the user of the shovel 100. The communication device 60 may be, for example, a 4G (4 th Generation) and 5G (5 th The communication device 60 may include a mobile communication module conforming to standards such as the IEEE 802.11 Generation. The communication device 60 may also include, for example, a satellite communication module. The communication device 60 may also include, for example, a WiFi communication module or a Bluetooth (registered trademark) communication module. The communication device 60 may also include multiple communication devices in accordance with the communication lines to be connected.

[0074] For example, the communication device 60 communicates with external devices such as the information processing device 200 and the remote operation support device 300 within the work site through a local communication line established at the work site. The local communication line is, for example, a local 5G (so-called local 5G) mobile communication line established at the work site or a local network (LAN: Local Area Network) using WiFi6.

[0075] Furthermore, for example, the communication device 60 communicates with the information processing device 200 and the remote operation support device 300 outside the work site through a wide area communication line that includes the work site, i.e., a wide area network (WAN). The wide area network includes, for example, a wide area mobile communication network, a satellite communication network, the Internet network, etc.

[0076] <Control System> 4, the control system of the shovel 100 includes a controller 30. The control system of the shovel 100 according to this embodiment also includes an operating pressure sensor 29, an imaging device 40, and sensors S1 to S5.

[0077] The controller 30 performs various controls related to the shovel 100 .

[0078] The functions of the controller 30 may be realized by any hardware or any combination of hardware and software, etc. For example, as shown in Fig. 4, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D, which are connected by a bus B1.

[0079] The auxiliary storage device 30A is a non-volatile storage means that stores the programs to be installed as well as necessary files, data, etc. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory.

[0080] For example, when an instruction to start a program is received, the memory device 30B loads the program from the auxiliary storage device 30A so that it can be read by the CPU 30C. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

[0081] The CPU 30C executes, for example, a program loaded into the memory device 30B, and realizes various functions of the controller 30 according to instructions from the program.

[0082] The interface device 30D functions as, for example, a communication interface for connecting to a communication line inside the shovel 100. The interface device 30D may include a plurality of different types of communication interfaces in accordance with the types of communication lines to be connected.

[0083] The interface device 30D also functions as an external interface for reading data from a recording medium and writing data to a recording medium. The recording medium is, for example, a dedicated tool connected to a connector installed inside the cabin 10 via a detachable cable. The recording medium may also be a general-purpose recording medium, such as an SD memory card or a USB (Universal Serial Bus) memory. As a result, a program that realizes various functions of the controller 30 may be provided by, for example, a portable recording medium and installed in the auxiliary storage device 30A of the controller 30. The program may also be downloaded from another computer external to the excavator 100 via the communication device 60 and installed in the auxiliary storage device 30A.

[0084] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

[0085] The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot type operating device 26, i.e., the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. A detection signal of the pilot pressure by the operating pressure sensor 29 corresponding to the operating state of each driven element (hydraulic actuator HA) in the operating device 26 is taken into the controller 30.

[0086] If the operating device 26 is an electric type, the operating pressure sensor 29 is omitted. This is because the controller 30 can grasp the operating state of each driven element through the operating device 26 based on the operating signal received from the operating device 26.

[0087] The imaging device 40 acquires an image of the periphery of the shovel 100. Furthermore, the imaging device 40 may acquire (generate) three-dimensional data (hereinafter simply referred to as "three-dimensional data of objects") representing the positions and outer shapes of objects around the shovel 100 within the imaging range (angle of view) based on the acquired image and data related to distance, which will be described later. The three-dimensional data of objects around the shovel 100 is, for example, data of coordinate information of a point cloud representing the surface of the object, range image data, etc.

[0088] For example, as shown in FIG. 2 , the imaging device 40 includes a camera 40F that images the area in front of the upper rotating body 3, a camera 40B that images the area behind the upper rotating body 3, a camera 40L that images the area to the left of the upper rotating body 3, and a camera 40R that images the area to the right of the upper rotating body 3. As a result, the imaging device 40 can capture images of the entire circumference of the shovel 100, i.e., a range covering an angular direction of 360 degrees, when viewed from above. Furthermore, the operator can visually recognize images captured by the cameras 40B, 40L, and 40R and peripheral images such as processing images generated based on the captured images via the output device 50 (display device 50A) and the remote control display device, and can check the conditions to the left, right, and rear of the upper rotating body 3. Furthermore, by visually recognizing images captured by the camera 40F and peripheral images such as processing images generated based on the captured images via the remote control display device, the operator can remotely operate the shovel 100 while checking the operation of the attachment AT including the bucket 6. Hereinafter, the cameras 40F, 40B, 40L, and 40R may be collectively or individually referred to as "camera 40X."

[0089] Camera 40X is, for example, a monocular camera. Camera 40X may also be, for example, a stereo camera, a TOF (Time Of Flight) camera, or the like (hereinafter collectively referred to as a "3D camera") that can acquire data related to distance (depth) in addition to two-dimensional images.

[0090] The output data of the imaging device 40 (camera 40X) (for example, image data or three-dimensional data of objects around the shovel 100) is taken into the controller 30 via a one-to-one communication line or an in-vehicle network. This allows the controller 30 to monitor objects around the shovel 100 based on the output data of the camera 40X. Also, for example, the controller 30 can determine the surrounding environment of the shovel 100 based on the output data of the camera 40X. Also, for example, the controller 30 can determine the attitude state of the attachment AT shown in the captured image based on the output data of the camera 40X (camera 40F). Also, for example, the controller 30 can determine the attitude state of the body (upper rotating body 3) of the shovel 100 based on the output data of the camera 40X, using objects around the shovel 100 as a reference.

[0091] Note that some of the cameras 40F, 40B, 40L, and 40R may be omitted. For example, if the shovel 100 is not remotely operated, the cameras 40F and 40L may be omitted. This is because it is relatively easy for the operator in the cabin 10 to confirm the conditions in front of and to the left of the shovel 100. Also, instead of or in addition to the imaging device 40 (camera 40X), a distance sensor may be provided on the upper rotating body 3. The distance sensor is attached, for example, to the top of the upper rotating body 3, and acquires data regarding the distance and direction of surrounding objects relative to the shovel 100. Furthermore, the distance sensor may acquire (generate) three-dimensional data (for example, point cloud coordinate information data) of objects around the shovel 100 within its sensing range based on the acquired data. The distance sensor is, for example, a LIDAR (Light Detection and Ranging). Also, for example, the distance sensor may be, for example, a millimeter-wave radar, an ultrasonic sensor, an infrared sensor, or the like.

[0092] Sensor S1 is attached to the boom 4 and detects the attitude angle (hereinafter referred to as "boom angle") around the rotation axis of the base end of the boom 4, which corresponds to the connection part between the boom 4 and the upper rotating body 3. Sensor S1 includes, for example, a rotary potentiometer, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), etc. The same may be true for sensors S2 and S4 below. Sensor S1 may also include a cylinder sensor that detects the extension / retraction position of the boom cylinder 7. The same may be true for sensor S2 below. The boom angle detection signal from sensor S1 is input to controller 30. This allows controller 30 to grasp the attitude state of the boom 4.

[0093] The sensor S2 is attached to the arm 5 and detects the attitude angle (hereinafter referred to as "arm angle") around the rotation axis of the base end of the arm 5, which corresponds to the connection part between the arm 5 and the boom 4. The detection signal of the arm angle by the sensor S2 is input to the controller 30. This enables the controller 30 to grasp the attitude state of the arm 5.

[0094] Sensor S3 is attached to bucket 6 and detects the attitude angle (hereinafter referred to as "arm angle") around the rotation axis of the base end of bucket 6, which corresponds to the connection part between bucket 6 and arm 5. The arm angle detection signal from sensor S3 is input to controller 30. This enables controller 30 to grasp the attitude state of bucket 6.

[0095] The sensor S4 detects the inclination state of the machine body (for example, the upper rotating body 3) relative to a predetermined reference plane (for example, a horizontal plane). The sensor S4 is attached to the upper rotating body 3, for example, and detects the inclination angles (hereinafter referred to as "fore-aft inclination angle" and "left-right inclination angle") of the excavator 100 (i.e., the upper rotating body 3) about two axes in the front-rear and left-right directions. The detection signals corresponding to the inclination angles (fore-aft inclination angle and left-right inclination angle) detected by the sensor S4 are input to the controller 30. This allows the controller 30 to grasp the inclination state of the machine body (upper rotating body 3).

[0096] The sensor S5 is attached to the upper rotating body 3 and outputs detection information related to the rotation state of the upper rotating body 3. The sensor S5 detects, for example, the rotation angular velocity and rotation angle of the upper rotating body 3. The sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, etc. The detection information related to the rotation state detected by the sensor S5 is input into the controller 30. This allows the controller 30 to grasp the rotation state of the upper rotating body 3, such as the rotation angle.

[0097] If sensor S4 includes a gyro sensor, a six-axis sensor, an IMU, or the like that can detect angular velocity around three axes, the rotation state (e.g., rotation angular velocity) of the upper rotating body 3 may be detected based on the detection signal of sensor S4. In this case, sensor S5 may be omitted. Also, if it is possible to grasp the attitude state of the upper rotating body 3, attachment AT, etc. based on the output of image capture device 40 or a distance sensor, at least some of sensors S1 to S5 may be omitted.

[0098] <Hardware configuration of information processing device> FIG. 5 is a block diagram showing an example of the hardware configuration of the information processing device 200. As shown in FIG.

[0099] The functions of the information processing device 200 are realized by any hardware or any combination of hardware and software, etc. For example, as shown in Fig. 5, the information processing device 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a high-speed calculation device 205, a communication interface 206, an input device 207, and a display device 208, which are connected via a bus B2.

[0100] The external interface 201 functions as an interface for reading data from the recording medium 201A and writing data to the recording medium 201A. Examples of the recording medium 201A include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, a USB memory, etc. This allows the information processing device 200 to read various data used in processing through the recording medium 201A, store the data in the auxiliary storage device 202, and install programs that realize various functions.

[0101] The information processing device 200 may acquire various data and programs used in processing from an external device via the communication interface 206.

[0102] The auxiliary storage device 202 stores various installed programs as well as files and data necessary for various processes. The auxiliary storage device 202 includes, for example, a hard disk drive (HDD), a solid state disk (SSD), a flash memory, etc.

[0103] When an instruction to start a program is received, the memory device 203 reads and stores the program from the auxiliary storage device 202. The memory device 203 includes, for example, a dynamic random access memory (DRAM) or an SRAM.

[0104] The CPU 204 executes various programs loaded from the auxiliary storage device 202 to the memory device 203, and realizes various functions related to the information processing device 200 in accordance with the programs.

[0105] The high-speed arithmetic unit 205 performs arithmetic processing at a relatively high speed in cooperation with the CPU 204. The high-speed arithmetic unit 205 includes, for example, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0106] The high speed calculation device 205 may be omitted depending on the required calculation processing speed.

[0107] The communication interface 206 is used as an interface for communicatively connecting with an external device. This allows the information processing device 200 to communicate with an external device such as the shovel 100 through the communication interface 206. The communication interface 206 may have multiple types of communication interfaces depending on the communication method between the connected device and the like.

[0108] The input device 207 accepts various inputs from the user.

[0109] The input device 207 includes, for example, an operation input device that accepts mechanical operation input from a user. The operation input device includes, for example, a button, a toggle, a lever, etc. The operation input device also includes, for example, a touch panel mounted on the display device 208, a touch pad provided separately from the display device 208, etc.

[0110] The input device 207 also includes, for example, a voice input device capable of receiving voice input from the user. The voice input device includes, for example, a microphone capable of collecting the user's voice.

[0111] The input device 207 includes, for example, a gesture input device capable of receiving a gesture input from a user. The gesture input device includes, for example, a camera capable of capturing an image of a user's gesture.

[0112] The input device 207 includes, for example, a biometric input device capable of accepting biometric input from a user. The biometric input device includes, for example, a camera capable of acquiring image data containing information about a user's fingerprint or iris.

[0113] The display device 208 displays an information screen or an operation screen for the user. For example, the display device 208 includes the above-mentioned remote control display device. The display device 208 is, for example, a liquid crystal display or an organic EL (Electroluminescence) display.

[0114] Note that, like the information processing device 200, the remote operation support device 300 may be realized by any hardware or any combination of hardware and software, and may have a similar hardware configuration. For example, like the information processing device 200 ( FIG. 6 ), the remote operation support device 300 is primarily configured with a computer including a CPU, a memory device, an auxiliary storage device, an interface device, an input device, and a display device. The memory device is, for example, an SRAM or DRAM. The auxiliary storage device is, for example, an HDD, an SSD, an EEPROM, or a flash memory. The interface device includes an external interface for connecting to an external recording medium and a communication interface for communicating with the outside, such as the shovel 100. The input device includes, for example, a lever-type operation input device. This allows the operator to use the operation input device to input operations related to the actuator of the shovel 100, and the remote operation support device 300 can transmit a signal corresponding to the operation input to the shovel 100 using the communication interface. Therefore, the operator can remotely control the shovel 100 using the remote operation support device.

[0115] [Functional configuration for generating the trajectory of the working part] Next, with reference to FIGS. 6 to 9 in addition to FIGS. 1 to 5, a functional configuration relating to generation of a trajectory of a working portion of the shovel 100 (hereinafter referred to as a "target trajectory") will be described.

[0116] Fig. 6 is a block diagram showing a first example of a functional configuration related to generation of a target trajectory for the working part of the shovel 100. Fig. 7 is a diagram showing an example of a screen (screen 700) related to generation of a target trajectory for the working part of the shovel 100, displayed on the display device 50A. Fig. 8 is a diagram showing another example of a screen (screen 800) related to generation of a target trajectory for the working part of the shovel 100, displayed on the display device 50A. Fig. 9 is a diagram showing yet another example of a screen (screen 900) related to generation of a target trajectory for the working part of the shovel 100, displayed on the display device 50A.

[0117] When the excavator 100 is remotely operated, the same screens as those shown in FIGS. 7 and 8 are displayed on the remote operation support device 300 (display device).

[0118] The working part of the excavator 100 is, for example, the tip or back of the bucket 6.

[0119] The shovel 100 includes a support device 150. The support device 150 provides support for the work of the shovel 100.

[0120] 6, the support device 150 includes an operation device 26, a controller 30, an imaging device 40, and an output device 50. In addition, when the shovel 100 is remotely operated, the support device 150 may include a communication device 60.

[0121] The controller 30 includes an operation log providing unit 301 and a work support unit 302 as functional units.

[0122] When the operation support system SYS includes a plurality of shovels 100, there may be an shovel 100 in which the controller 30 includes only the operation log providing unit 301 and the work support unit 302, and an shovel 100 in which the controller 30 includes only the latter. In this case, the former shovel 100 only has the function of acquiring the operation log of the shovel 100 and providing it to the information processing device 200, which is used for the operation support function (operation suggestion function) of the latter shovel 100 for the operator.

[0123] The information processing device 200 includes, as functional units, an operation log acquisition unit 2001, an operation log storage unit 2002, a teacher data generation unit 2003, a machine learning unit 2004, a trained model storage unit 2005, and a distribution unit 2006.

[0124] The operation log providing unit 301 is a functional unit for acquiring an operation log of the shovel 100, which is the original data for realizing the function of generating a target trajectory of the working part of the shovel 100, and providing it to the information processing device 200. Specifically, the operation log acquired when the shovel 100 was operated by an operator who has a long history of operating the shovel 100 and is relatively experienced (hereinafter referred to as an "expert" for convenience), is provided to the information processing device 200.

[0125] The operation log of the shovel 100 includes data on the shape of a work object around the shovel 100 and data on the operation of the shovel 100 performed on the shape of the work object. The data on the shape of the work object around the shovel 100 is, for example, data on the topographical shape of the ground at the work site as the work object of the shovel 100. The data on the shape of the work object of the shovel 100 is, for example, image data from the imaging device 40 or three-dimensional data of the work object obtained from the image data. The data on the operation of the shovel 100 is, for example, data representing the operation content of the operator. The data representing the operation content of the operator is, for example, output data from the operating pressure sensor 29 in the case of a hydraulic pilot type operating device 26, or output data (operation signal data) of the operating device 26 in the case of an electric operating device 26. The data on the operation of the shovel 100 may also be data representing the operating state of the shovel 100 actually performed in response to the operation of the operator. The data representing the operating state of the shovel 100 is, for example, output data of the sensors S1 to S5, or data relating to the posture state of the shovel 100 acquired from the output data of the sensors S1 to S5.

[0126] Operation log provider 301 includes operation log recording unit 301A, operation log storage unit 301B, and operation log transmission unit 301C.

[0127] The operation log recording unit 301A acquires an operation log of the shovel 100 and records it in the operation log storage unit 301B. For example, every time an operation of the shovel 100 is executed, the operation log recording unit 301A records data on the shape of the work target around the shovel 100 at the start of or immediately before the execution of the operation, and data on the operation of the shovel 100 in the operation log storage unit 301B.

[0128] The operation log storage unit 301B stores an operation log of the shovel 100 in an accumulated form. For example, the operation log storage unit 301B stores data relating to the shape of a work target around the shovel 100 for each operation of the shovel 100 and data relating to the operation of the shovel 100 in a linked form. Specifically, the operation log storage unit 301B may accumulate record data indicating a correspondence between the data relating to the shape of a work target around the shovel 100 for each operation of the shovel 100 and the data relating to the operation of the shovel 100, and a database of operation logs may be constructed.

[0129] The operation log stored in operation log storage unit 301B that has already been transmitted to information processing device 200 by operation log transmission unit 301C (described later) may be deleted afterward.

[0130] The operation log transmission unit 301C transmits the operation log of the shovel 100 stored in the operation log storage unit 301B to the information processing device 200 via the communication device 60. The operation log transmission unit 301C may also transmit to the information processing device 200 data relating to the shape of the work target around the shovel 100 for each operation of the shovel 100, and record data indicating the correspondence between the data relating to the operation of the shovel 100.

[0131] For example, the operation log transmission unit 301C transmits an untransmitted operation log of the shovel 100 stored in the operation log storage unit 301B to the information processing device 200 in response to a signal (hereinafter referred to as a "transmission request signal") requesting transmission of the operation log of the shovel 100, which is received from the information processing device 200. Furthermore, the operation log transmission unit 301C may automatically transmit an untransmitted operation log of the shovel 100 stored in the operation log storage unit 301B to the information processing device 200 at a predetermined timing. The predetermined timing is, for example, when the shovel 100 stops operating (when the key switch is turned off) or when it starts operating (when the key switch is turned on).

[0132] The operation log acquisition unit 2001 acquires the operation log of the shovel 100 received from the shovel 100 .

[0133] The operation log acquisition unit 2001 acquires the operation log of the shovel 100 by transmitting a transmission request signal to the shovel 100 in response to an operation by a user of the information processing device 200 or automatically at a predetermined timing. The operation log acquisition unit 2001 may also acquire the operation log of the shovel 100 transmitted from the shovel 100 at a predetermined timing.

[0134] The operation log storage unit 2002 stores an accumulated operation log of the shovel 100 acquired by the operation log acquisition unit 2001. For example, similar to the case of the operation log storage unit 301B, the operation log storage unit 2002 stores data relating to the shape of the work target around the shovel 100 for each operation of the shovel 100 and data relating to the operation of the shovel 100 in a linked manner.

[0135] The training data generation unit 2003 generates training data for machine learning based on the operation log of the shovel 100 stored in the operation log storage unit 2002. The training data generation unit 2003 may generate training data automatically by batch processing, or may generate training data in response to input from a user of the information processing device 200. The training data is a combination of data relating to the shape of a work target around the shovel 100 as input data and data representing the trajectory (track) of the working part of the shovel 100 corresponding to the input data as correct output data (hereinafter referred to as "correct data").

[0136] The data representing the trajectory of the working part of the shovel 100 is generated based on the output data of the sensors S1 to S5 included in the data relating to the operation of the shovel 100, for example.

[0137] The machine learning unit 2004 generates a trained model LM by performing machine learning on the base learning model based on the set of training data generated by the training data generation unit 2003. The trained model LM (base learning model) includes, for example, a neural network such as a DNN (Deep Neural Network).

[0138] The trained model LM uses, for example, data relating to the type of operation of the shovel 100 and the shape of the work target around the shovel 100 as input conditions, and outputs data representing the target trajectory of the working part of the shovel 100 and a prediction probability. The trained model LM also outputs data representing the target trajectory of the working part of the shovel 100 and the type of operation of the shovel 100, including, for example, an excavation operation, a sweeping operation, a horizontal pulling operation, a rolling operation, a broom operation, etc. A sweeping operation is, for example, an operation in which the attachment AT is operated to push the bucket 6 forward along the ground, thereby sweeping earth and sand forward with the back of the bucket 6. In a sweeping operation, for example, the attachment AT moves the boom 4 ofThe attachment AT lowers the boom 4 and opens the arm 5. The horizontal towing operation is, for example, an operation in which the attachment AT operates to move the tip of the bucket 6 approximately horizontally along the ground surface, pulling it toward the user, thereby smoothing out unevenness on the ground surface. In the horizontal towing operation, the attachment AT raises the boom 4 and closes the arm 5. The compaction operation is, for example, an operation in which the attachment AT operates to press the ground with the back surface of the bucket 6. The compaction operation may also be an operation in which the bucket 6 is pushed forward along the ground surface, the back surface of the bucket 6 sweeps out soil and sand to a predetermined position forward, and then the back surface of the bucket 6 presses the ground at the predetermined position against the ground. In the compaction operation, for example, the attachment AT lowers the boom 4 while pressing the ground. The broom operation is, for example, an operation in which the upper rotating body 3 is operated to rotate the bucket 6 left and right while keeping it along the ground surface. The broom operation may be, for example, an operation in which the attachment AT and the upper rotating body 3 are operated to alternately rotate left and right while keeping the bucket 6 along the ground, while pushing the bucket 6 forward. In the broom operation, for example, the upper rotating body 3 alternately rotates left and right. In the broom operation, for example, in addition to the alternately rotating left and right of the upper rotating body 3, the boom 4 may be lowered and the arm 5 may be opened, as in the sweeping operation. The prediction probability represents the reliability of the target trajectory of the working part. As described above, the trained model LM reflects the operation log of the excavator 100 operated by an expert, and the higher the prediction probability, the higher the reliability of the target trajectory of the working part. The prediction probability also represents the degree of suitability of the target trajectory of the working part with respect to the shape of the work object around the excavator 100, which is an input condition. This is because the higher the prediction probability, the more likely an expert will determine that the candidate operation is appropriate for the shape of the work object. For example, a learned model LM is generated for each task such as leveling work, slope construction work, and embankment work.

[0139] The trained model storage unit 2005 stores the trained model LM output by the machine learning unit 2004.

[0140] The distribution unit 2006 distributes the learned model LM to the excavator 100.

[0141] For example, when a trained model LM is generated by the machine learning unit 2004, the distribution unit 2006 distributes the most recently generated trained model LM to the shovel 100. In addition, the distribution unit 2006 may distribute the latest trained model LM in the trained model storage unit 2005 to the shovel 100 in response to a signal received from the shovel 100 requesting distribution of the trained model LM.

[0142] The work support unit 302 is a functional unit for supporting the work of the excavator 100 operated by the operator.

[0143] The work support unit 302 includes a learned model storage unit 302A, a work object shape acquisition unit 302B, an action selection unit 302C, a condition setting unit 302D, a trajectory generation unit 302E, a display processing unit 302F, and an action control unit 302G.

[0144] The trained model storage unit 302A stores the trained model LM that is distributed from the information processing device 200 and received via the communication device 60.

[0145] The work object shape acquisition unit 302B acquires data relating to the shape of the work object (terrain shape) around the shovel 100 based on the output of the imaging device 40 and the distance sensor.

[0146] The action selection unit 302C selects an action (type) of the shovel 100 from among a plurality of action candidates, in accordance with an input from a user (operator) received via the input device 52. Furthermore, when the shovel 100 is remotely operated, the action selection unit 302C may select an action of the shovel 100 from among a plurality of action candidates, in accordance with an input from a user (operator) using the remote operation support device 300, in accordance with an input received via the communication device 60.

[0147] The condition setting unit 302D sets preconditions for generating a target trajectory of the working part of the shovel 100 in response to input from a user (operator) received through the input device 52. When the shovel 100 is remotely operated, the condition setting unit 302D may set preconditions for the target trajectory of the shovel 100 in response to input from a user (operator) using the remote operation support device 300 received through the communication device 60. The condition setting unit 302D may also set preconditions automatically, without relying on input from the user. For example, the condition setting unit 302D may automatically set preconditions based on a trained model generated using, as a training data set, a data history of combinations of data related to the shape of the work object and preconditions set for the shape of the work object. In this case, the condition setting unit 302D may modify the automatically set preconditions in response to input from the user.

[0148] The precondition is, for example, a point (hereinafter referred to as "target point") that serves as a target when the shovel 100 operates on the topographical shape around the shovel 100. Target points include, for example, a target point through which a working part passes when the shovel 100 operates, or a point corresponding to a location where the bucket 6 unloads earth and sand when the shovel 100 operates, etc. The precondition may also include the attitude state of the bucket 6 at the target point (the attitude angle of the bucket 6).

[0149] The trajectory generation unit 302E generates a target trajectory for the working portion of the shovel 100 based on the data acquired by the work object shape acquisition unit 302B, the target shape of the work object, the actions selected by the action selection unit 302C, and the preconditions set by the condition setting unit 302D. The target shape of the work object is, for example, a target construction surface representing a flat or curved surface as a construction object formed by work on the work object (the ground at the work site). The target shape of the work object is set by inputting parameters representing a flat or curved surface from the user via, for example, the input device 52 or the remote operation support device 300 (input device). The target shape of the work object may also be distributed to the shovel 100 from an external device such as the information processing device 200. The trajectory generation unit 302E applies the learned model LM to the data acquired by the work object shape acquisition unit 302B, the target shape of the work object, the actions selected by the action selection unit 302C, and the preconditions set by the condition setting unit 302D as input data. Furthermore, the trajectory generation unit 302E may output a target trajectory of the working part from the trained model LM using the target shape of the work object, the action selected by the action selection unit 302C, and the data acquired by the work object shape acquisition unit 302B as input data. The trajectory generation unit 302E may then generate a target trajectory of the working part by optimizing the output target trajectory of the working part using the preconditions set by the condition setting unit 302D.

[0150] The display processing unit 302F causes the display device 50A to display a screen relating to the generation of a target trajectory of the working part of the shovel 100 (see Figs. 7 and 8). The screen relating to the generation of a target trajectory of the working part of the shovel 100 includes, for example, an operation screen for the user (operator) to perform operation input relating to the operation of the shovel 100 selected by the operation selection unit 302C and the prerequisites set by the condition setting unit 302D. do.Furthermore, the screen related to the generation of the target trajectory of the working part of the shovel 100 includes a screen that displays the target trajectory of the working part of the shovel 100, which is generated by the trajectory generating unit 302E. Furthermore, when the shovel 100 is remotely operated, the display processing unit 302F may transmit data related to the screen related to the generation of the target trajectory of the working part of the shovel 100 to the remote operation support device 300 via the communication device 60. This allows the display processing unit 302F to display the screen related to the generation of the target trajectory of the working part of the shovel 100 on the remote operation support device 300 (display device).

[0151] For example, as shown in FIGS. 7 and 8, the display processing unit 302F causes the display device 50A to display screens 700 and 800.

[0152] As shown in FIG. 7, a screen 700 includes images TG, CG, SB, and PB1.

[0153] The image TG is an image that represents the shape of the terrain around the shovel 100. The image TG is generated based on data acquired by the work object shape acquisition unit 302B. In this example, the image TG is an image that represents the shape of the terrain around the shovel 100 as seen from a predetermined viewpoint outside the shovel 100. The predetermined viewpoint can be changed, for example, in response to input from the user (operator) via the input device 52 or the remote operation support device 300 (input device).

[0154] The image CG is an image representing the excavator 100.

[0155] The positional relationship between the images TG and CG is set so as to be the same as the actual positional relationship between the topographical shape around the shovel 100 and the shovel 100 .

[0156] The images SB are images representing candidate actions that can be selected by the action selection unit 302 C. In this example, the images SB include images SB1 to SB5 representing candidate actions of the shovel 100 that can be performed in ground leveling work.

[0157] The image SB1 is an operation icon for the user to select a combination of the excavation operation and the earth removal operation of the shovel 100.

[0158] The image SB2 is an operation icon for the user to select the sweeping operation of the shovel 100.

[0159] The image SB3 is an operation icon for the user to select the horizontal pulling operation of the shovel 100.

[0160] Image SB4 is an operation icon for the user to select the broom operation of the shovel 100.

[0161] Image SB5 is an operation icon for the user to select the rolling operation of the shovel 100.

[0162] The user can specify one of images SB1 to SB5 via input device 52 or remote operation support device 300 (input device), and select an action for shovel 100 via action selection unit 302C. In this example, there is a cursor (matt finish in the drawing) on ​​image SB1, which represents a state in which an action combination of excavation action and earth removal action of shovel 100 has been selected.

[0163] In addition to the images SB1 to SB5, the image SB may display an operation icon for the user to select an action other than the actions corresponding to the images SB1 to SB5. Furthermore, the image SB may display an operation icon for the user to select an action other than the actions corresponding to the images SB1 to SB5, instead of at least one of the images SB1 to SB5.

[0164] In this example, the image TG includes image regions TG1 and TG2.

[0165] Image region TG1 represents a protrusion on the ground around (in front of) the shovel 100.

[0166] Image region TG2 represents a depression in the ground around (in front of) the shovel 100.

[0167] In this example, the screen 700 also includes images P1 and P2 corresponding to the target points.

[0168] Image P1 is displayed superimposed on image region TG1.

[0169] Image P2 is displayed superimposed on image region TG2.

[0170] For example, the user can set target points corresponding to images P1 and P2 via the condition setting unit 302D by specifying image regions TG1 and TG2 via the input device 52 or the remote operation support device 300 (input device). The user may be able to set a target point within the entire range of the image TG via the input device 52 or the remote operation support device 300 (input device), or may be able to set a target point limited to a range within the entire range of the image TG that the working portion of the bucket 6 can reach. In the former case, when a target point is set within a range within the entire range of the image TG that the working portion of the bucket 6 can reach, a display content indicating an error (warning) may be displayed on the screen 700. In the latter case, an image indicating the range within the entire range of the image TG that the working portion of the bucket 6 can reach may be displayed superimposed on the image TG. The user may also be able to delete a set target point via the input device 52 or the remote operation support device 300 (input device).

[0171] In this example, images RC1 and RC2 are also displayed on the screen 700 so as to accompany the images P1 and P2, respectively.

[0172] Image RC1 is an image that represents the preconditions for the attitude angle of the bucket 6 corresponding to the target point corresponding to image P1.

[0173] Image RC2 is an image that represents the preconditions for the attitude angle of the bucket 6 corresponding to the target point corresponding to image P2.

[0174] For example, the user can specify images P1 and P2 via the input device 52 or the remote operation support device 300 (input device) to set preconditions for the attitude angle of the bucket 6 corresponding to images RC1 and RC2 via the condition setting unit 302D.

[0175] The image PB1 is an icon for operation to cause the trajectory generating unit 302E to generate a trajectory of the working portion of the bucket 6 in accordance with the operation selected on the screen 800 and the preconditions set on the screen 800.

[0176] For example, the user can generate a target trajectory of the bucket 6 through the trajectory generating unit 302E by operating the image PB1 through the input device 52 or the remote operation support device 300 (input device).

[0177] When image PB1 is operated, the display content of display device 50A transitions from screen 700 to screen 800.

[0178] The screen 800 includes images TG, CG, and SB, similar to the screen 700. The screen 800 also includes images P1 and P2, similar to the screen 700. The screen 800 also includes images OG, CG1, and PB2.

[0179] The image OG is an image representing the target trajectory.

[0180] The image CG1 is an image showing the bucket 6, and is displayed in association with the image OG corresponding to the target trajectory.

[0181] In this example, image OG represents a target trajectory that realizes an operation of scooping up earth and sand at a target point corresponding to image P1 through an excavation operation and discharging the earth and sand at a target point corresponding to image P2. Image OG may be expressed so that the trajectory portion where the working part of the bucket 6 comes into contact with earth and the other trajectory portions can be distinguished. For example, image OG displays the trajectory portion where the working part of the bucket 6 comes into contact with earth and the other trajectory portions in different colors.

[0182] Image PB2 is an icon for operation to reproduce on screen 800, as a moving image (animation), the operation of moving the working part of bucket 6 along a target trajectory corresponding to image OG.

[0183] For example, the user can operate image PB2 via input device 52 or remote operation support device 300 (input device) to display a moving image on screen 800, in which image CG1 corresponding to bucket 6 moves along image OG corresponding to the target trajectory. Therefore, by checking the moving image, the user can determine whether the target trajectory is appropriate.

[0184] The moving image may also display the shape of the work target (terrain shape) after the operation of the shovel 100 to move the bucket 6 along the target trajectory. In other words, the screen 800 may display the predicted shape of the work target (terrain shape) around the shovel 100 after the shovel 100 is operated to move the bucket 6 along the target trajectory corresponding to the image OG. This allows the user to more appropriately determine whether the target trajectory is appropriate by checking the moving image and the predicted changes in the terrain shape.

[0185] When image PB1 is operated, the display content of display device 50A transitions from screen 800 to screen 900.

[0186] The screen 900 includes images TG, CG, and SB, just like the screen 800. The screen 900 also includes images P1 and P2, just like the screen 800. The screen 900 also includes images OG and CG1. The screen 900 also includes an image PB3.

[0187] Image PB3 is an operation icon for automatically operating the excavator 100 so as to move the working portion of the bucket 6 along a target trajectory corresponding to image OG.

[0188] For example, by operating image PB3 via input device 52 or remote operation support device 300 (input device), the user can automatically operate shovel 100 via operation control unit 302G so that bucket 6 moves along a target trajectory corresponding to image OG.

[0189] It may be possible to automatically operate the shovel 100 so that the bucket 6 moves along the target trajectory corresponding to the image OG without the user having to check the above-described moving image. In this case, an operation icon corresponding to the image PB3 is displayed on the screen 800 in addition to the image PB2.

[0190] 7, the operation control unit 302G operates the shovel 100 so that the working portion of the bucket 6 moves along the target trajectory generated by the trajectory generating unit 302E, in accordance with input from the user (operator) received through the input device 52. Specifically, the operation control unit 302G can operate the shovel 100 so that the working portion of the bucket 6 moves along the target trajectory by controlling the hydraulic control valve 31 while determining the position of the working portion of the bucket 6 from the outputs of the sensors S1 to S5, etc.

[0191] For example, in response to an input of an instruction to execute a movement from a user, the movement control unit 302G operates the shovel 100 so that the working portion of the bucket 6 moves along the target trajectory generated by the trajectory generating unit 302E.

[0192] In addition, the operation control unit 302G may operate the shovel 100 in response to the operation of the operating device 26 or a remote control signal, thereby assisting the operator in operating the shovel 100 so that the working part of the bucket 6 moves along the target trajectory generated by the trajectory generating unit 302E.

[0193] [Processing for generating the trajectory of the working part] Next, with reference to FIG. 10, a process for generating a target trajectory for the working part of the shovel 100 will be described.

[0194] FIG. 10 is a flowchart showing an example of a process for generating a target trajectory of a working portion of the shovel 100.

[0195] 10 is repeatedly executed during operation of a function related to generation of a target trajectory for a working part of the shovel 100. The function related to generation of a target trajectory for a working part of the shovel 100 is operated (activated) by input of an instruction from the user, which is received via the input device 52 or the remote operation support device 300 (input device).

[0196] As shown in FIG. 10, in step S102 (an example of an acquisition step), the work object shape acquisition unit 302B acquires data relating to the shape of the work object around the shovel 100 from the imaging device 40.

[0197] When the process of step S102 is completed, the controller 30 proceeds to step S104.

[0198] In step S104 (an example of a display step), the display processing unit 302F displays a setting screen (e.g., screen 700) including an image representing the terrain shape on the display device 50A or the remote operation support device 300 (display device) based on the data acquired in step S102.

[0199] When the process of step S104 is completed, the controller 30 proceeds to step S106.

[0200] In step S106, the operation selection unit 302C selects one operation from among a plurality of candidate operations of the shovel 100 in response to an input from the user.

[0201] When the process of step S106 is completed, the controller 30 proceeds to step S108.

[0202] In step S108, the condition setting unit 302D (an example of a setting step) sets preconditions for generating a target trajectory of the working part of the shovel 100 in response to an input from the user.

[0203] When the process of step S108 is completed, the controller 30 proceeds to step S110.

[0204] The order of steps S106 and S108 may be changed depending on the input from the user.

[0205] In step S110 (an example of a generating step), the trajectory generating unit 302E generates a target trajectory of the working part of the shovel 100 for the operation selected in step S106 under the preconditions set in step S108.

[0206] When the process of step S110 is completed, the controller 30 proceeds to step S112.

[0207] In step S112, the display processing unit 302F displays an image representing the target trajectory generated in step S110 on the display device 50A and the remote operation support device 300 (display device).

[0208] When the process of step S112 is completed, the controller 30 proceeds to step S114.

[0209] In step S114, controller 30 determines whether or not an operation input has been received that instructs execution of an operation of shovel 100 to move the working portion of bucket 6 along the target trajectory generated in step S112. If an operation input instructing execution of an operation of shovel 100 has been received, controller 30 proceeds to step S116, and if any other operation, specifically, an operation to generate a target trajectory again, has been received, controller 30 returns to step S106.

[0210] In step S116, the operation control unit 302G controls the hydraulic control valve 31 to automatically operate the excavator 100 so that the working portion of the bucket 6 moves along the target trajectory generated in the processing of the most recent step S110.

[0211] When the process of step S116 is completed, the controller 30 ends the process of this flowchart.

[0212] At the completion of the processing of step S116, the excavator 100 (attachment AT) may be in a state where the working portion of the bucket 6 is at the end point of the target trajectory, or may have returned to the posture state before the processing of step S114 was started.

[0213] In this way, in this example, the support device 150 (controller 30) can generate a target trajectory that matches the shape of the work object, thereby improving the work efficiency of the shovel 100.

[0214] Furthermore, in this example, the support device 150 can generate a target trajectory that conforms to preconditions such as the target point and the attitude angle of the bucket 6. Therefore, a more appropriate target trajectory can be generated by reflecting the judgment and intention of the user, which is based on the shape of the work target, etc.

[0215] Furthermore, in this example, the support device 150 can automatically operate the shovel 100 so that the working portion of the bucket 6 moves along the generated target trajectory. Therefore, even an inexperienced operator can cause the shovel 100 to perform appropriate operations, and as a result, the work efficiency of the shovel 100 can be further improved.

[0216] [Other embodiments] Next, another embodiment will be described.

[0217] The above-described embodiments may be combined, modified, or changed as appropriate.

[0218] For example, in the above-described embodiment, the trajectory generating unit 302E may generate a target trajectory without using the learned model LM. For example, a reference trajectory of the working part may be defined in advance for each of a plurality of candidate actions, and the trajectory generating unit 302E may generate a target trajectory of the working part by optimizing the reference trajectory of the action selected by the action selecting unit 302C in accordance with data related to the shape of the work target (terrain shape) around the shovel 100 and preconditions.

[0219] Furthermore, in the above-described embodiment and its modified examples, data regarding the shape of the work target around the shovel 100 may be acquired based on data from an imaging device, a distance sensor, or the like installed outside the shovel 100. For example, data from an imaging device or a distance sensor installed at the work site is received by the shovel 100 via the communication device 60, allowing the work target shape acquisition unit 302B to acquire data regarding the shape of the work target around the shovel 100. Furthermore, for example, data from an imaging device or a distance sensor mounted on a drone flying above the work site is received by the shovel 100 via the communication device 60, allowing the work target shape acquisition unit 302B to acquire data regarding the shape of the work target around the shovel 100.

[0220] In the above-described embodiment and its modifications, some or all of the functions of the assistance device 150 may be transferred to the remote operation assistance device 300.

[0221] In the above-described embodiment and its modifications, some or all of the functions of the assistance device 150 may be transferred to the information processing device 200.

[0222] [Effect] Next, the operation of the operation support system SYS (operation support device) according to this embodiment will be described.

[0223] In this embodiment, the support device includes an acquisition unit, a display unit, a setting unit, and a generation unit. The support device is, for example, support device 150. The acquisition unit is, for example, work object shape acquisition unit 302B. The display unit is, for example, display device 50A. The setting unit is, for example, condition setting unit 302D. The generation unit is, for example, trajectory generation unit 302E. Specifically, the acquisition unit acquires data related to the terrain shape of the construction object around the work machine. The work machine is, for example, an excavator 100. The display unit displays an image representing the terrain shape of the construction object based on the data acquired by the acquisition unit. The setting unit sets a point (target point) on the terrain shape of the construction object that will serve as a target for the work machine when it operates. The generation unit generates a trajectory for the work portion of the work machine based on the data acquired by the acquisition unit, the target shape of the construction object, and the point set by the setting unit.

[0224] This makes it easier to generate a trajectory for the working part of the work machine. Also, because a target point is set, it is possible to generate a more appropriate trajectory for the working part that reflects, for example, the judgment and intention of the user who has visually recognized the shape of the work object around the work machine. As a result, the work efficiency of the work machine can be improved.

[0225] In this embodiment, the assistance device may also include a selection unit. Specifically, the selection unit may select one operation from a plurality of candidate operations of the work machine in response to input from the user. The generation unit may then generate a trajectory of the working part resulting from the one operation of the work machine based on the data acquired by the acquisition unit and the points set by the setting unit.

[0226] This allows the support device to define the operation of the work machine and generate the trajectory of the working part, thereby further improving the work efficiency of the work machine.

[0227] In addition, in this embodiment, the setting unit may set a point (target point) that will be a target when the work machine is operating on the terrain shape around the work machine, and the posture of the work part that corresponds to that point, in response to input from the user.

[0228] This allows the assistance device to generate a more appropriate trajectory for the working part that reflects the user's judgment and intention regarding the posture of the working part, thereby further improving the work efficiency of the work machine.

[0229] Furthermore, in this embodiment, the display unit may display an image representing the trajectory generated by the generation unit, superimposed on an image representing the shape of the terrain around the work machine.

[0230] This allows the user to visually check the generated image, and by simultaneously visually checking the topographical shape around the work machine and the generated trajectory, the user can more appropriately determine the validity of the generated trajectory.

[0231] In addition, in this embodiment, the display unit may display a moving image of the working part moving along the trajectory generated by the generation unit, superimposed on an image showing the shape of the terrain around the work machine.

[0232] This allows the user to more appropriately determine the validity of the generated trajectory by checking the moving image.

[0233] In addition, in this embodiment, the display unit may display an image representing the shape of the work object around the work machine that is predicted after the work part operates along the trajectory generated by the generation unit.

[0234] This allows the validity of the generated trajectory to be more appropriately determined by checking the shape of the work object after the work part has moved along the generated trajectory.

[0235] In this embodiment, the assistance device may also include a control unit that automatically operates the work machine based on the trajectory generated by the generation unit in response to user input. The control unit is, for example, the operation control unit 302G.

[0236] This allows even a relatively inexperienced operator to operate the working part along the target trajectory, thereby improving work efficiency and user convenience.

[0237] In this embodiment, the work machine may also be equipped with the above-mentioned operation assistance device.

[0238] This allows the work machine to generate a target trajectory more easily and improves work efficiency.

[0239] Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist described in the claims. [Explanation of symbols]

[0240] 1 Undercarriage 1C, 1CL, 1CR Crawler 1ML, 1MR Travel Hydraulic Motor 2. Swivel mechanism 2M Swing Hydraulic Motor 3 Upper rotating body 4. Boom 5 Arm 6 buckets 7 Boom cylinder 8 Arm Cylinder 9 Bucket cylinder 10 Cabins 11 Engine 13 Regulator 14 Main pump 15 Pilot pump 17 Control valve 26 Operating device 29 Operating pressure sensor 30 Controllers 31 Hydraulic control valve 32 Shuttle valve 33 Hydraulic control valve 40 Imaging device 50 Output Device 50A display device 52 Input Device 60 Communication Equipment 100 Shovel 150 Support equipment 200 Information processing device 300 Remote operation support device 301 Operation log provider 301A Operation log recording section 301B Operation log storage unit 301C Operation log transmission unit 302 Work Support Department 302A Trained model memory unit 302B Work object shape acquisition unit 302C Operation selection section 302D Condition setting section 302E Trajectory generation part 302F Display processing unit 302G Motion control unit 700, 800, 900 screen 2001 Operation log acquisition unit 2002 Operation log storage unit 2003 Teacher Data Generation Department 2004 Machine Learning Department 2005 Trained model memory 2006 Distribution Department AT Attachment HA Hydraulic Actuator LM pre-trained model S1~S5 sensors SYS Operation Support System

Claims

1. an acquisition unit that acquires data relating to the shape of a work target in the vicinity of the work machine; a display unit that displays an image representing the shape of the work object based on the data acquired by the acquisition unit; a setting unit that sets a point on the shape of the work object that will be a target when the work machine is operating; a generation unit that generates a trajectory of a working portion of the work machine based on the data acquired by the acquisition unit, the target shape of the work object, and the points set by the setting unit, Support equipment.

2. a selection unit that selects one operation from a plurality of candidate operations of the work machine in response to an input from a user; the generation unit generates a trajectory of the working portion resulting from the one operation of the work machine based on the data acquired by the acquisition unit and the points set by the setting unit. The support device according to claim 1 .

3. the setting unit sets a target point on the topographical shape around the work machine during operation of the work machine, and an attitude of the work part corresponding to that point, in response to input from a user. The support device according to claim 1 or 2.

4. the display unit displays an image representing the trajectory generated by the generation unit, superimposed on an image representing the shape of the terrain around the work machine.

4. The assistance device according to claim 1.

5. the display unit displays a moving image of the working part moving along the trajectory generated by the generation unit, superimposed on an image representing the topographical shape around the work machine. The support device according to claim 4.

6. the display unit displays an image representing a shape of the work object around the work machine that is predicted after the work part moves along the trajectory generated by the generation unit.

6. The support device according to claim 4 or 5.

7. a control unit that automatically operates the work machine based on the trajectory generated by the generation unit in response to a user input; 7. An assistance device according to any one of claims 1 to 6.

8. The setting unit sets a target point on the shape of the work object during operation of the work machine based on the data acquired by the acquisition unit.

8. An assistance device according to any one of claims 1 to 7.

9. The generating unit uses a trained model that has been machine-learned using training data related to the operation of the work machine as operated by a relatively skilled operator, which is associated with the shape of the work object, to generate a trajectory of the work part of the work machine based on the data acquired by the acquiring unit, the target shape of the work object, and the points set by the setting unit.

9. An assistance device according to any one of claims 1 to 8.

10. A support device according to any one of claims 1 to 9, Work machinery.

11. Support devices include: an acquisition step of acquiring data relating to the shape of a work object around the work machine; a display step of displaying an image representing the shape of the work object on a display unit based on the data acquired in the acquisition step; a setting step of setting a point on the shape of the work object that will be a target when the work machine is operating; a generating step of generating a trajectory of a working portion of the work machine based on the data acquired in the acquiring step, the target shape of the work object, and the points set in the setting step; program.

Images