Support device, construction machine, support system, and program

Abstract

The invention relates to a support device, a construction machine, a support system and a program, and provides a technology capable of improving work efficiency when a plurality of actions of the construction machine are combined for work. A support device (150) according to one embodiment of the present invention is provided with: an input device (52) that receives an input from a user; and a display device (50A) which, on the basis of an input from the input device (52), specifies a specification relating to a combination of a plurality of basic operations of different types of the shovel (100), and displays setting screens (1300, 1400) for generating a trajectory of a work site of the shovel (100) on the basis of a composite operation in which the plurality of basic operations are combined.

Description

Technical Field

[0001] The present invention relates to a support device and the like. Background Art

[0002] For example, there is disclosed a technology for generating a trajectory of a working area of ​​a construction machine when performing a predetermined operation based on the surrounding environment of the construction machine, a target shape, and the like (see Patent Document 1).

[0003] In Patent Document 1, a target trajectory of a blade tip of a shovel bucket during excavation is generated.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2020 / 032267 Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, in the above-mentioned technology, only the track of the working part based on the basic action (e.g., excavation action) of the construction machine can be generated. Therefore, for example, when a plurality of actions different from each other are combined to perform an operation, it is necessary to repeat the process of generating the track of the working part based on the basic action of the object corresponding to the surrounding environment that changes successively and making the working part perform the action on the track. As a result, the working efficiency may be reduced.

[0009] Therefore, in view of the above-mentioned problems, an object of the present invention is to provide a technology capable of improving the working efficiency when working by combining a plurality of actions of a construction machine.

[0010] Means for solving problems

[0011] In order to achieve the above object, in one embodiment of the present invention, a support device is provided, which comprises:

[0012] An input unit for receiving input from a user; and

[0013] The display unit determines specifications related to the combination of multiple different types of actions of the construction machine based on the input from the input unit, and displays a first operation screen, wherein the first operation screen is used to generate a trajectory of the working part of the construction machine based on a composite action formed by combining the multiple actions.

[0014] Furthermore, in another embodiment of the present invention, there is provided a construction machine comprising:

[0015] An input unit for receiving input from a user; and

[0016] The display unit determines specifications related to the combination of multiple different types of actions of the construction machine based on the input from the input unit, and displays a first operation screen, wherein the first operation screen is used to generate a trajectory of the working part of the construction machine based on a composite action composed of the multiple actions.

[0017] Furthermore, in another embodiment of the present invention, there is provided a support system including a construction machine and a support device capable of communicating with the construction machine, wherein:

[0018] The supporting device comprises:

[0019] An input unit for receiving input from a user; and

[0020] The display unit determines specifications related to the combination of multiple different types of actions of the construction machine based on the input from the input unit, and displays a first operation screen, wherein the first operation screen is used to generate a track of the working part of the construction machine based on a composite action formed by combining the multiple actions.

[0021] In another embodiment of the present invention, there is provided a program for causing an information processing device having an input unit and a display unit to perform the following processing:

[0022] Specifications related to the combination of multiple actions of different types of the construction machine are determined based on input from the input unit, and a first operation screen is displayed on the display unit. The first operation screen is used to generate a trajectory of the working part of the construction machine based on a composite action composed of the multiple actions.

[0023] Effects of the Invention

[0024] According to the above-described embodiment, it is possible to improve the work efficiency when working by combining a plurality of operations of the construction machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a diagram showing an example of an operation support system.

[0026] Figure 2 It is a top view showing an example of a shovel.

[0027] Figure 3 This is a diagram showing an example of a configuration related to remote control of a shovel.

[0028] Figure 4 This is a block diagram showing an example of the hardware configuration of a shovel.

[0029] Figure 5 This is a diagram showing an example of the hardware configuration of the remote operation support device.

[0030] Figure 6 This is a functional block diagram showing an example of a functional configuration related to generation of a target trajectory of a working location of a shovel in the operation support system.

[0031] Figure 7 1 is a diagram showing an example of the configuration of a model for extracting feature quantities of a trajectory of a working area of ​​a shovel.

[0032] Figure 8 1 is a diagram showing an example of the configuration of a model for extracting feature quantities of a trajectory of a working area of ​​a shovel.

[0033] Fig. 9 This is a diagram showing an example of a configuration of a model for generating a trajectory of a working area in a complex operation of a shovel.

[0034] Fig.10 This is a diagram showing an example of a configuration of a model for generating a trajectory of a working area in a complex operation of a shovel.

[0035] Fig.11 This is a diagram showing an example of a track of a working area during a leveling operation of an excavator.

[0036] Fig.12 This is a diagram showing an example of a screen showing the topographic shape around a shovel.

[0037] Fig.13 This is a diagram showing a first example of a setting screen for specifications related to a combination of a plurality of basic operations in a compound operation of a shovel.

[0038] Fig.14 This is a diagram showing a second example of a setting screen for specifications related to a combination of a plurality of basic operations in a compound operation of a shovel.

[0039] Fig.15 This is a flowchart schematically showing an example of processing related to the generation of a trajectory of a working area of ​​a shovel. DETAILED DESCRIPTION

[0040] Hereinafter, embodiments will be described with reference to the drawings.

[0041] [Overview of Operation Support System]

[0042] refer to Figure 1 to Figure 3 The outline of the operation support system SYS according to the present embodiment will be described.

[0043] Figure 1 FIG. 1 is a diagram showing an example of the operation support system SYS. Figure 1 A left side view of the excavator 100 is shown in FIG. Figure 2 It is a plan view showing an example of the shovel 100 . Figure 3 1 is a diagram showing an example of a structure related to the remote control of the shovel 100. In the following, the direction in which the attachment AT extends when the shovel 100 is viewed from above ( Figure 2 The upper direction of the excavator 100 is defined as “front” to describe a direction in the excavator 100 or a direction viewed from the excavator 100.

[0044] like Figure 1 As shown, the operation support system SYS includes a shovel 100 and an information processing device 200 .

[0045] The operation support system SYS cooperates with the shovel 100 using the information processing device 200 to perform support related to the operation of the shovel 100 .

[0046] The shovel 100 included in the operation support system SYS may be one or a plurality of shovels.

[0047] In the operation support system SYS, the shovel 100 is a construction machine that is a target of support related to operation.

[0048] In addition, the construction machine included in the operation support system SYS and supported by the operation support system may be another construction machine different from the excavator 100. For example, the other construction machine is a construction machine with construction attachments, specifically a crane, a forklift, a road machine, etc. The road machine is, for example, an asphalt roller.

[0049] like Figure 1 , Figure 2 As shown, the shovel 100 includes a lower traveling body 1 , an upper swing body 3 , an attachment AT including a boom 4 , an arm 5 , and a bucket 6 , and a cab 10 .

[0050] The lower traveling body 1 uses crawler tracks 1C to allow the excavator 100 to travel. The crawler tracks 1C include a left crawler track 1CL and a right crawler track 1CR. The crawler tracks 1CL are hydraulically driven by a traveling hydraulic motor 1ML. Similarly, the crawler tracks 1CL are hydraulically driven by a traveling hydraulic motor 1MR. Thus, the lower traveling body 1 can move by itself.

[0051] The upper revolving body 3 is rotatably mounted on the lower traveling body 1 via the revolving mechanism 2. For example, the upper revolving body 3 is revolved relative to the lower traveling body 1 by hydraulically driving the revolving mechanism 2 by a revolving hydraulic motor 2M.

[0052] The boom 4 is mounted at the front center of the upper revolving body 3 so as to be able to pitch about a rotation axis in the left-right direction. The arm 5 is mounted at the front end of the boom 4 so as to be able to rotate about a rotation axis in the left-right direction. The bucket 6 is mounted at the front end of the arm 5 so as to be able to rotate about a rotation axis in the left-right direction.

[0053] The bucket 6 is an example of an end attachment and is used for excavation work or leveling work, for example.

[0054] The bucket 6 is mounted on the front end of the arm 5 in an appropriate replaceable manner according to the work content of the excavator 100. That is, a bucket different from the bucket 6 can be mounted on the front end of the arm 5 to replace the bucket 6, for example, a relatively large bucket, a bucket for inclined surfaces, a bucket for dredging, etc. In addition, a terminal attachment of a type other than a bucket, such as a mixer, a crusher, a crusher, etc., can be mounted on the front end of the arm 5. In addition, a preliminary attachment such as a quick coupling, a tilt rotator, etc. can be provided between the arm 5 and the terminal attachment.

[0055] The boom 4, the arm 5 and the bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8 and a bucket cylinder 9, respectively.

[0056] The cab 10 is a control room for an operator to get on and operate the shovel 100. The cab 10 is mounted on the front left side of the upper swing body 3, for example.

[0057] For example, the shovel 100 operates driven elements such as the lower traveling body 1 (ie, a pair of left and right crawler tracks 1CL and 1CR), the upper swing body 3, the boom 4, the arm 5, and the bucket 6 according to operations of an operator riding in the cab 10.

[0058] Furthermore, the shovel 100 may be configured to be operated by an operator riding in the cab 10, or alternatively or in addition thereto, may be configured to be remotely operated from outside the shovel 100. When the shovel 100 is remotely operated, the interior of the cab 10 may be unmanned. The following description is based on the premise that the operator's operation includes at least one of the operation of the operating device 26 by the operator in the cab 10 and the remote operation by an external operator.

[0059] For example, Figure 3 As shown, the remote operation includes a method of operating the shovel 100 by inputting operation related to the actuator of the shovel 100 by the remote operation support device 300. The remote operation support device 300 may be provided separately from the information processing device 200, or may be the information processing device 200.

[0060] The remote operation support device 300 is, for example, provided in a management center that externally manages the operation of the shovel 100. Furthermore, the remote operation support device 300 may be a portable operation terminal, in which case the operator can remotely operate the shovel 100 while directly checking the operation status of the shovel 100 from the periphery of the shovel 100.

[0061] The shovel 100 can transmit an image (hereinafter referred to as a "surrounding image") representing the surrounding state including the front of the shovel 100 based on the camera image output by the camera device 40 to the remote operation support device 300, for example, through the communication device 60 to be described later. Furthermore, the shovel 100 can transmit the camera image output by the camera device 40 to the remote operation support device 300 through the communication device 60, and the remote operation support device 300 processes the camera image received from the shovel 100 and generates the surrounding image. Then, the remote operation support device 300 can display the surrounding image representing the surrounding state including the front of the shovel 100 on the display device. Moreover, various information images (information screens) displayed on the output device 50 (display device 50A) inside the cab 10 of the shovel 100 can also be displayed on the display device of the remote operation support device 300 in the same manner. Thus, the operator using the remote operation support device 300 can remotely operate the shovel 100 while checking the display contents such as images and information screens showing the state of the shovel 100 surrounding the shovel displayed on the display device. In addition, the shovel 100 can operate the actuators according to the remote operation signal indicating the contents of the remote operation received from the remote operation support device 300 by the communication device 60, and drive the driven elements such as the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, and the bucket 6.

[0062] Furthermore, in the remote operation, for example, a method of operating the shovel 100 by inputting voice or gesture from outside to the shovel 100 by people around the shovel 100 (for example, workers), etc. Specifically, the shovel 100 recognizes voices made by workers around the shovel 100, gestures made by workers, etc., through a voice input device (for example, a microphone) or a gesture input device (for example, a camera) mounted on the shovel 100. Furthermore, the shovel 100 can operate the actuator according to the content of the recognized voice or gesture, and drive the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper swing body 3, the boom 4, the arm 5, and the bucket 6.

[0063] Furthermore, the operation of the shovel 100 can be monitored remotely. In this case, a remote monitoring support device having the same function as the remote operation support device 300 can be provided. The remote monitoring support device is, for example, the information processing device 200. Thus, a monitor who is a user of the remote monitoring support device can monitor the operation status of the shovel 100 while confirming the surrounding image displayed on the display device of the remote monitoring support device. Furthermore, for example, when the monitor determines that it is necessary from the perspective of safety, he or she can intervene in the operation of the operator of the shovel 100 and make the shovel 100 stop urgently by using the input device of the remote monitoring support device to make a predetermined input.

[0064] The information processing device 200 communicates with the shovel 100 to cooperate with each other and provides support related to the operation of the shovel 100 .

[0065] The information processing device 200 is, for example, a server device or a management terminal device, such as a management office set up in the construction site of the excavator 100, or a management center that manages the operating status of the excavator 100 at a location different from the construction site of the excavator 100. The server device can be a local server or a cloud server, or it can be an edge server. The management terminal device can be, for example, a fixed terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile terminal) such as a tablet terminal, a smart phone, or a notebook PC. In the latter case, workers at the construction site, supervisors who supervise the work, managers who manage the construction site, etc. can carry the portable information processing device 200 and move around the construction site. And, in the latter case, the operator can, for example, bring the portable information processing device 200 into the cockpit of the excavator 100.

[0066] The information processing device 200 acquires data related to the operating state from the shovel 100, for example. Thus, the information processing device 200 can grasp the operating state of the shovel 100 and monitor whether the shovel 100 has abnormalities, etc. Furthermore, the information processing device 200 can display the data related to the operating state of the shovel 100 on the display device 208 described later for the user to confirm.

[0067] The information processing device 200 can transmit various data such as programs and reference data used in the processing of the controller 30 to the shovel 100. The shovel 100 can thereby perform various processes related to the operation of the shovel 100 using various data downloaded from the information processing device 200.

[0068] [Hardware structure of the operation support system]

[0069] Then, except Figure 1 to Figure 3 In addition, refer to Figure 4 , Figure 5 , the hardware structure of the operation support system SYS is explained.

[0070] The hardware configuration of the remote operation support device 300 may be the same as that of the information processing device 200. Therefore, illustration and description of the hardware configuration of the remote operation support device 300 are omitted.

[0071] <Hardware structure of excavator>

[0072] Figure 4 This is a block diagram showing an example of the hardware configuration of the shovel 100 .

[0073] In addition, Figure 4 In the figure, a double line represents a path for transmitting mechanical power, a solid line represents a path for high-pressure working oil to drive a hydraulic actuator, a dashed line represents a path for transmitting pilot pressure, and a dotted line represents a path for transmitting electrical signals.

[0074] The excavator 100 includes various components such as a hydraulic drive system related to hydraulic drive of driven elements, an operating system related to operation of driven elements, a user interface system related to information exchange with users, a communication system related to communication with the outside, and a control system related to various controls.

[0075] Hydraulic drive system

[0076] like Figure 4 As shown, the hydraulic drive system of the excavator 100 includes a hydraulic actuator HA that hydraulically drives the driven elements such as the lower traveling body 1 (left and right crawler tracks 1C), the upper swing body 3 and the attachment AT, etc. In addition, the hydraulic drive system of the excavator 100 involved in this embodiment includes an engine 11, a regulator 13, a main pump 14 and a control valve 17.

[0077] The hydraulic actuator HA includes travel hydraulic motors 1ML and 1MR, a swing hydraulic motor 2M, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and the like.

[0078] In addition, in the shovel 100, a part or all of the hydraulic actuator HA may be replaced with an electric actuator. That is, the shovel 100 may be a hybrid shovel or an electric shovel.

[0079] The engine 11 is a prime mover of the excavator 100 and a main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses diesel as fuel. The engine 11 is, for example, mounted on the rear of the upper slewing body 3. The engine 11 rotates at a preset target speed under direct or indirect control of a controller 30 described later, and drives the main pump 14 and the pilot pump 15.

[0080] In addition, other prime movers (for example, electric motors) or the like may be mounted on the shovel 100 instead of or in addition to the engine 11 .

[0081] The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter referred to as "tilt angle") according to a control command from the controller 30.

[0082] The main pump 14 supplies hydraulic oil to the control valve 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, at the rear of the upper slewing body 3, similarly to the engine 11. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the stroke length of the piston is adjusted by adjusting the deflection angle of the swash plate by the regulator 13, thereby controlling the discharge flow rate or discharge pressure.

[0083] The control valve 17 drives the hydraulic actuator HA according to the operator's operation of the operating device 26, the content of the remote operation, or the operation instruction corresponding to the automatic operation function. The control valve 17 is mounted, for example, on the central part of the upper slewing body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic pipeline, and selectively supplies the working oil supplied from the main pump 14 to each hydraulic actuator according to the operator's operation or the operation instruction corresponding to the automatic operation function. Specifically, the control valve 17 includes a plurality of control valves (also called "reversing valves") that control the flow rate and flow direction of the working oil supplied from the main pump 14 to each hydraulic actuator HA.

[0084] "operating system"

[0085] like Figure 4 As shown, the operating system of the excavator 100 includes a pilot pump 15 , an operating device 26 , a hydraulic control valve 31 , a shuttle valve 32 , and a hydraulic control valve 33 .

[0086] The pilot pump 15 supplies pilot pressure to various hydraulic devices via the pilot line 25. The pilot pump 15 is mounted on the rear of the upper swing body 3, for example, similarly to the engine 11. The pilot pump 15 is, for example, a fixed displacement hydraulic pump, and is driven by the engine 11 as described above.

[0087] In addition, the pilot pump 15 may be omitted. In this case, the relatively high-pressure hydraulic oil discharged from the main pump 14 is reduced in pressure by a predetermined pressure reducing valve to a relatively low-pressure hydraulic oil as a pilot pressure and is supplied to various hydraulic devices.

[0088] The operating device 26 is provided near the operator's seat of the cab 10 and is used by the operator to operate various driven elements. Specifically, the operating device 26 is used by the operator to operate the hydraulic actuator HA that drives each driven element, and as a result, the operator can operate the driven element that is the driving target of the hydraulic actuator HA. The operating device 26 includes a pedal device or a joystick device for operating each driven element (hydraulic actuator HA).

[0089] For example, Figure 4As shown, the operating device 26 is a hydraulic pilot type. Specifically, the operating device 26 uses the working oil supplied from the pilot pump 15 through the pilot line 25 and the pilot line 25A branched therefrom to output the pilot pressure corresponding to the operation content to the secondary side pilot line 27A. The pilot line 27A is connected to one inlet port of the shuttle valve 32, and is connected to the control valve 17 via the pilot line 27 connected to the outlet port of the shuttle valve 32. As a result, the pilot pressure corresponding to the operation content related to the various driven elements (hydraulic actuator HA) in the operating device 26 can be input to the control valve 17 via the shuttle valve 32. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26 by the operator or the like.

[0090] Furthermore, the operating device 26 may be an electrical type. In this case, the pilot line 27A, the reciprocating valve 32, and the hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electrical signal (hereinafter referred to as an "operation signal") corresponding to the operation content, and the operation signal is input to the controller 30. Furthermore, the controller 30 outputs a control command corresponding to the content of the operation signal, that is, a control signal corresponding to the operation content of the operating device 26, to the hydraulic control valve 31. As a result, a pilot pressure corresponding to the operation content of the operating device 26 can be input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 drives each hydraulic actuator HA according to the operation content of the operating device 26.

[0091] Furthermore, the control valve (direction change valve) built into the control valve 17 for driving each hydraulic actuator HA may be of an electromagnetic solenoid type. In this case, the operation signal output from the operation device 26 may be directly input to the control valve 17, that is, the electromagnetic solenoid control valve.

[0092] Furthermore, as described above, a part or all of the hydraulic actuator HA may be replaced with an electric actuator. In this case, the controller 30 may output a control command corresponding to the operation content of the operating device 26 or the content of the remote operation specified by the remote operation signal to the electric actuator or a driver driving the electric actuator. Furthermore, when the shovel 100 is remotely operated, the operating device 26 may be omitted.

[0093] The hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) of the operation object of the operating device 26 and for each driving direction of the driven element (hydraulic actuator HA) (for example, the lifting direction and the lowering direction of the boom 4). For example, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA for driving the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, the bucket 6, etc. The hydraulic control valve 31 is provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and can be configured to be able to change its flow area (that is, the cross-sectional area through which the working oil can flow). Thus, the hydraulic control valve 31 can output a prescribed pilot pressure to the secondary side pilot line 27B using the working oil of the pilot pump 15 supplied through the pilot line 25B. Therefore, the hydraulic control valve 31 can indirectly cause a prescribed pilot pressure corresponding to a control signal from the controller 30 to act on the control valve 17 through the reciprocating valve 32 between the pilot line 27B and the pilot line 27. Therefore, for example, the controller 30 supplies a pilot pressure corresponding to an operation command corresponding to the automatic operation function from the hydraulic control valve 31 to the control valve 17 , thereby realizing the operation of the shovel 100 based on the automatic operation function.

[0094] Furthermore, the controller 30 controls the hydraulic control valve 31 to realize the remote operation of the shovel 100. Specifically, the controller 30 outputs a control signal corresponding to the content of the remote operation specified by the remote operation signal received from the remote operation support device 300 to the hydraulic control valve 31 through the communication device 60. As a result, the controller 30 supplies the pilot pressure corresponding to the remote operation content from the hydraulic control valve 31 to the control valve 17, and the operation of the shovel 100 based on the remote operation of the operator can be realized.

[0095] Furthermore, when the operating device 26 is an electrical device, the controller 30 supplies a pilot pressure corresponding to the operation content (operation signal) of the operating device 26 directly from the hydraulic control valve 31 to the control valve 17, thereby enabling the operation of the shovel 100 based on the operator's operation.

[0096] The shuttle valve 32 has two inlet ports and one outlet port, and outputs the working oil having the higher pilot pressure among the pilot pressures input to the two inlet ports to the outlet port. Similar to the hydraulic control valve 31, the shuttle valve 32 is provided for each driven element (hydraulic actuator HA) of the operation object of the operating device 26 and for each driving direction of the driven element (hydraulic actuator HA). For example, two shuttle valves 32 are provided for each double-acting hydraulic actuator HA for driving the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, the bucket 6, etc. One of the two inlet ports of the shuttle valve 32 is connected to the secondary side pilot line 27A of the operating device 26 (specifically, the above-mentioned joystick device or pedal device included in the operating device 26), and the other is connected to the secondary side pilot line 27B of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of the corresponding control valve of the control valve 17 through the pilot line 27. The corresponding control valve is a control valve that drives the hydraulic actuator HA, which is the operating object of the above-mentioned joystick device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, these shuttle valves 32 can respectively allow the higher pilot pressure of the pilot pressure of the secondary side pilot line 27A of the operating device 26 and the pilot pressure of the secondary side pilot line 27B of the hydraulic control valve 31 to act on the pilot port of the corresponding control valve. That is, the controller 30 can control the corresponding control valve independently of the operator's operation of the operating device 26 by outputting a pilot pressure higher than the secondary side pilot pressure of the operating device 26 from the hydraulic control valve 31. As a result, the controller 30 can control the operation of the driven elements (lower traveling body 1, upper swing body 3, attachment AT) independently of the operator's operation state of the operating device 26, and realize the automatic operation function or remote operation function.

[0097] The hydraulic control valve 33 is provided in the pilot line 27A connecting the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured to be able to change its flow path area, for example. The hydraulic control valve 33 operates according to the control signal input from the controller 30. Thus, when the operator operates the operating device 26, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26. Therefore, even when the operating device 26 is operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. Moreover, even when the operating device 26 is operated, the controller 30 can reduce the pilot pressure output from the operating device 26 to be lower than the pilot pressure output from the hydraulic control valve 31. Therefore, the controller 30 can control the hydraulic control valve 31 and the hydraulic control valve 33, for example, regardless of the operation content of the operating device 26, so that the desired pilot pressure can be reliably applied to the pilot port of the control valve in the control valve 17. Thus, the controller 30 controls the hydraulic control valve 33 in addition to the hydraulic control valve 31 , for example, and thus the automatic operation function or the remote control function of the shovel 100 can be more appropriately realized.

[0098] User Interface System

[0099] like Figure 4 As shown, the user interface system of the excavator 100 includes an operating device 26 , an output device 50 , and an input device 52 .

[0100] The output device 50 outputs various information to the user of the shovel 100 (for example, an operator of the cab 10 or an external remote operator), people around the shovel 100 (for example, workers, drivers of construction vehicles), and the like.

[0101] For example, the output device 50 includes a lighting device that outputs various information by visual means, a display device 50A (see Figure 6 ) etc. The lighting device is, for example, a warning light (indicator light) etc. The display device 50A is, for example, a liquid crystal display, an organic EL (Electroluminescence: electroluminescence) display etc. For example, Figure 2 As shown, the lighting device and the display device 50A can be installed inside the cab 10, and various information can be outputted visually to the operator inside the cab 10. In addition, the lighting device and the display device 50A can also be installed on the side of the upper rotating body 3, and various information can be outputted visually to the workers around the excavator 100.

[0102] Furthermore, the output device 50 may include a sound output device that outputs various information in an auditory manner. The sound output device includes, for example, a buzzer, a speaker, etc. The sound output device may be provided, for example, at least one of the inside and outside of the cab 10, and outputs various information in an auditory manner to the operator in the cab 10 and the people (operators, etc.) around the excavator 100.

[0103] Furthermore, the output device 50 may include a device that outputs various information by a tactile method such as vibration of the operator's seat.

[0104] The input device 52 receives various inputs from the user of the shovel 100, and inputs signals corresponding to the received inputs to the controller 30. For example, Figure 2 As shown, the input device 52 is provided inside the cab 10 and receives input from an operator or the like inside the cab 10. Alternatively, the input device 52 may be provided on the side of the upper swing body 3 and receive input from a worker or the like around the shovel 100.

[0105] For example, the input device 52 includes an operation input device that receives input from a user's mechanical operation. The operation input device may include a touch panel mounted on the display device, a touch pad provided around the display device, a button switch, a joystick, a toggle key, a knob switch provided on the operation device 26 (joystick device), and the like.

[0106] Furthermore, the input device 52 may include a voice input device for receiving a voice input from a user. The voice input device includes, for example, a microphone.

[0107] Furthermore, the input device 52 may include a gesture input device that receives a gesture input by the user. The gesture input device includes, for example, an imaging device that captures a state of a gesture performed by the user.

[0108] Furthermore, the input device 52 may include a biometric input device for receiving a user's biometric input, which includes, for example, input of biometric information such as a user's fingerprint or iris.

[0109] Communication Systems

[0110] like Figure 4 As shown, the communication system of the shovel 100 according to the present embodiment includes a communication device 60 .

[0111] The communication device 60 is connected to an external communication line and communicates with a device installed separately from the shovel 100. The device installed separately from the shovel 100 may include, in addition to the device located outside the shovel 100, a portable terminal device (mobile terminal) brought into the cockpit 10 by the user of the shovel 100. The communication device 60 may include, for example, a device compliant with 4G (4 th Generation: fourth generation), 5G (5 th The communication device 60 may include a mobile communication module of a specification such as the fifth generation. Furthermore, the communication device 60 may include, for example, a satellite communication module. Furthermore, the communication device 60 may include, for example, a WiFi communication module, a Bluetooth (registered trademark) communication module, etc. Furthermore, the communication device 60 may include a plurality of communication devices corresponding to the communication line of the connection object.

[0112] For example, the communication device 60 communicates with an external device such as the information processing device 200 or the remote operation support device 300 in the construction site through a local communication line constructed at the construction site. The local communication line is, for example, a mobile communication line based on a local 5G (so-called local 5G) constructed at the construction site or a local area network (LAN: Local Area Network) based on WiFi6.

[0113] Furthermore, the communication device 60 can communicate with the information processing device 200 or the remote operation support device 300 located outside the construction site through a wide area communication line including the construction site, i.e., a wide area network (WAN). For example, the wide area network includes a wide area mobile communication network, a satellite communication network, the Internet, and the like.

[0114] Control Systems

[0115] like Figure 4 As shown, the control system of the shovel 100 includes a controller 30. Furthermore, the control system of the shovel 100 according to the present embodiment includes an operation pressure sensor 29, an imaging device 40, and sensors S1 to S5.

[0116] The controller 30 performs various controls related to the shovel 100 .

[0117] The functions of the controller 30 may be implemented by any hardware, or any combination of hardware and software. Figure 3 As shown, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D connected by a bus BS1.

[0118] The auxiliary storage device 30A is a nonvolatile storage means, and stores installed programs, necessary files and data, etc. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory.

[0119] For example, when there is an instruction to start the program, the memory device 30B loads the program of the auxiliary storage device 30A so that the CPU 30C can read it. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

[0120] The CPU 30C executes, for example, a program loaded into the memory device 30B, and implements various functions of the controller 30 according to the instructions of the program.

[0121] The interface device 30D functions as, for example, a communication interface for connecting to a communication line inside the shovel 100. The interface device 30D may include a plurality of different types of communication interfaces corresponding to the types of communication lines to be connected.

[0122] Furthermore, the interface device 30D functions as an external interface for reading data from a recording medium or writing data to a recording medium. The recording medium is, for example, a dedicated tool connected to a connector provided inside the cab 10 by a detachable cable. Furthermore, the recording medium may be, for example, a general-purpose recording medium such as an SD memory card or a USB (Universal Serial Bus) memory. Thus, a program for realizing various functions of the controller 30 may be provided, for example, via a portable recording medium, and may be installed in the auxiliary storage device 30A of the controller 30. Furthermore, the program may be downloaded from another computer outside the shovel 100 via the communication device 60 and installed in the auxiliary storage device 30A.

[0123] In addition, part of the functions of the controller 30 may be realized by other controllers (control devices). In other words, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

[0124] The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot operating device 26, that is, the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. The detection signal of the pilot pressure corresponding to the operating state related to each driven element (hydraulic actuator HA) in the operating device 26 from the operating pressure sensor 29 is input to the controller 30.

[0125] In addition, when the operating device 26 is an electrical type, the operating pressure sensor 29 is omitted. This is because the controller 30 can grasp the operating state of each driven element by the operating device 26 based on the operating signal input from the operating device 26.

[0126] The camera device 40 acquires an image of the vicinity of the shovel 100. Furthermore, the camera device 40 can acquire (generate) three-dimensional data (hereinafter referred to as "three-dimensional data of the object") representing the position and shape of the object around the shovel 100 within the camera range (angle of view) based on the acquired image and data related to the distance described later. The three-dimensional data of the object around the shovel 100 is, for example, data representing coordinate information of a point group on the surface of the object, distance image data, and the like.

[0127] For example, Figure 2 As shown, the camera device 40 includes a camera 40F for photographing the front of the upper swing body 3, a camera 40B for photographing the rear of the upper swing body 3, a camera 40L for photographing the left side of the upper swing body 3, and a camera 40R for photographing the right side of the upper swing body 3. Thus, the camera device 40 can photograph the entire circumference of the shovel 100, that is, the range of the angle direction of 360 degrees when the shovel 100 is viewed from above. In addition, the operator can visually recognize the camera images of the cameras 40B, 40L, and 40R, the processed images generated from the camera images, and other peripheral images through the display device 50A and the remote operation support device 300, and confirm the situation to the left, right, and rear of the upper swing body 3. In addition, the operator can visually recognize the camera images of the camera 40F, the processed images generated from the camera images, and other peripheral images through the remote operation support device 300, thereby remotely operating the shovel 100 while confirming the operation of the attachment AT. Hereinafter, the cameras 40F, 40B, 40L, and 40R may be collectively or individually referred to as “camera 40X”.

[0128] The camera 40X is, for example, a monocular camera and can acquire data related to distance (depth) in addition to two-dimensional images, such as a stereo camera or a TOF (Time Of Flight) camera (hereinafter collectively referred to as a “3D camera”).

[0129] The output data of the camera device 40 (camera 40X) (for example, image data, three-dimensional data of objects around the shovel 100, etc.) is input to the controller 30 through a one-to-one communication line or an in-vehicle network. Thus, for example, the controller 30 can monitor objects around the shovel 100 based on the output data of the camera 40X. And, for example, the controller 30 can determine the surrounding environment of the shovel 100 based on the output data of the camera 40X. And, for example, the controller 30 can determine the posture state of the attachment AT shown in the camera image based on the output data of the camera 40X (camera 40F). And, for example, the controller 30 can determine the posture state of the shovel body 100 (upper swing body 3) based on the objects around the shovel 100 based on the output data of the camera 40X.

[0130] In addition, part or all of the cameras 40B, 40L, and 40R may be omitted. Furthermore, a distance sensor may be provided on the upper rotating body 3 instead of or in addition to the camera device 40 (camera 40X). The distance sensor is, for example, installed on the upper part of the upper rotating body 3 to obtain data related to the distance and direction of surrounding objects with the excavator 100 as a reference. Furthermore, the distance sensor can obtain (generate) three-dimensional data (for example, data of coordinate information of a point group) of objects around the excavator 100 within the sensing range based on the obtained data. The distance sensor is, for example, LIDAR (Light Detection and Ranging). Furthermore, for example, the distance sensor may be a millimeter wave radar, an ultrasonic sensor, an infrared sensor, or the like.

[0131] The sensor S1 is mounted on the boom 4, and detects the posture angle of the base end of the connection portion of the boom 4 with the upper rotating body 3 around the rotation axis (hereinafter referred to as the "boom angle"). The sensor S1 includes, for example, a rotary potentiometer, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), etc. The following description may also be the same for sensors S2 to S4. Furthermore, the sensor S1 may include a cylinder sensor that detects the telescopic position of the boom cylinder 7. The following description may also be the same for sensors S2 and S3. The detection signal of the boom angle of the sensor S1 is input to the controller 30. Thus, the controller 30 can grasp the posture state of the boom 4.

[0132] The sensor S2 is mounted on the boom 5 and detects the posture angle (hereinafter referred to as the "boom angle") of the base end of the connection portion of the boom 5 and the boom 4 around the rotation axis. The detection signal of the boom angle of the sensor S2 is input to the controller 30. Thus, the controller 30 can grasp the posture state of the boom 5.

[0133] The sensor S3 is mounted on the bucket 6 and detects the posture angle (hereinafter referred to as the "arm angle") of the base end of the connection portion between the bucket 6 and the boom 5 around the rotation axis. The detection signal of the boom angle of the sensor S3 is input to the controller 30. Thus, the controller 30 can grasp the posture state of the bucket 6.

[0134] The sensor S4 detects the tilt state of the machine body (for example, the upper rotating body 3) relative to a predetermined reference plane (for example, a horizontal plane). The sensor S4 is, for example, mounted on the upper rotating body 3, and detects the tilt angles of the two axes of the excavator 100 (that is, the upper rotating body 3) around the front-back direction and the left-right direction (hereinafter referred to as "front-back tilt angle" and "left-right tilt angle"). A detection signal corresponding to the tilt angle (front-back tilt angle and left-right tilt angle) detected by the sensor S4 is input to the controller 30. Thus, the controller 30 can grasp the tilt state of the machine body (the upper rotating body 3).

[0135] The sensor S5 is mounted on the upper rotating body 3, and outputs detection information related to the rotation state of the upper rotating body 3. The sensor S5 detects, for example, the rotation angular velocity and the rotation angle of the upper rotating body 3. The sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, etc. The detection information related to the rotation state detected by the sensor S5 is input to the controller 30. Thus, the controller 30 can grasp the rotation state such as the rotation angle of the upper rotating body 3.

[0136] In addition, when the sensor S4 includes a gyro sensor, a 6-axis sensor, an IMU, etc. that can detect the angular velocity around three axes, the rotation state (for example, the rotation angular velocity) of the upper rotating body 3 can be detected based on the detection signal of the sensor S4. In this case, the sensor S5 can be omitted. In addition, when the posture state of the upper rotating body 3 or the attachment AT can be grasped based on the output of the camera device 40 or the distance sensor, at least a part of the sensors S1 to S5 can be omitted.

[0137] <Hardware structure of information processing device>

[0138] Figure 5 This is a block diagram showing an example of the hardware configuration of the information processing device 200 .

[0139] The functions of the information processing device 200 are implemented by any hardware, or any combination of hardware and software. Figure 5As shown, the information processing device 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a high-speed calculation device 205, a communication interface 206, an input device 207, a display device 208 and a sound output device 209 connected by a bus BS2.

[0140] The external interface 201 functions as an interface for reading data from or writing data to the recording medium 201A. The recording medium 201A includes, for example, a floppy disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, a USB memory, and the like. Thus, the information processing device 200 can read various data used in processing through the recording medium 201A, store them in the auxiliary storage device 202, or install programs that realize various functions.

[0141] Furthermore, the information processing device 200 can acquire various data or programs used in processing from an external device via the communication interface 206 .

[0142] The auxiliary storage device 202 stores various installed programs, and stores files and data required for various processes, etc. The auxiliary storage device 202 includes, for example, a HDD (Hard Disc Drive), an SSD (Solid State Disc), a flash memory, and the like.

[0143] When there is an instruction to start a program, the memory device 203 reads and stores the program from the auxiliary storage device 202. The memory device 203 includes, for example, a DRAM (Dynamic Random Access Memory) or an SRAM.

[0144] The CPU 204 executes various programs loaded from the auxiliary storage device 202 to the memory device 203 , and realizes various functions related to the information processing device 200 according to the programs.

[0145] The high-speed computing device 205 performs computing at a relatively high speed in conjunction with the CPU 204. The high-speed computing device 205 includes, for example, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).

[0146] In addition, the high-speed computing device 205 can be omitted according to the required computing processing speed.

[0147] The communication interface 206 is used as an interface for connecting to an external device so as to be communicable. Thus, the information processing device 200 can communicate with an external device such as the shovel 100 through the communication interface 206. In addition, the communication interface 206 can also have multiple communication interfaces according to the communication method between the connected device and the like.

[0148] The input device 207 receives various inputs from the user. The input device 207 includes an operating device for remote operation for remotely operating the shovel 100 .

[0149] The input device 207 includes, for example, an input device that receives mechanical operation input from the user (hereinafter referred to as an "operation input device"). The operation device for remote operation may be an operation input device. The operation input device includes, for example, a button, a switch key, a joystick, a keyboard, a mouse, a touch panel installed on the display device 208, and a touch pad provided separately from the display device 208.

[0150] Furthermore, the input device 207 may include a voice input device capable of receiving voice input from the user. The voice input device includes, for example, a microphone capable of collecting the user's voice.

[0151] Furthermore, the input device 207 may also include a gesture input device capable of receiving gesture input from the user. The gesture input device may include, for example, a camera capable of capturing the gesture state of the user.

[0152] Furthermore, the input device 207 may include a biometric input device capable of receiving biometric input from the user. The biometric input device includes, for example, a camera capable of acquiring image data containing information related to the user's fingerprint or iris.

[0153] The display device 208 displays an information screen or an operation screen for a user of the information processing device 200. The display device 208 is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

[0154] The sound output device 209 transmits various information by sound to the user of the information processing device 200. The sound output device 209 is, for example, a buzzer, an alarm, a speaker, or the like.

[0155] [Functional structure related to the generation of the trajectory of the working part]

[0156] Then, except Figure 1 to Figure 5 In addition, refer to Figure 6 to Figure 11, a functional configuration related to generation of a trajectory of a working location of the shovel 100 in the operation support system SYS will be described.

[0157] Figure 6 This is a functional block diagram showing an example of a functional configuration related to generation of a trajectory of a working location of the shovel 100 in the operation support system SYS. Figure 7 and Figure 8 1 is a diagram showing an example of the configuration of a model for extracting feature quantities of a trajectory of a working portion of the shovel 100 . Fig. 9 and Fig.10 1 is a diagram showing an example of a configuration of a model for generating a trajectory of a working area in a complex operation of the shovel 100 . Fig.11 It is a diagram showing an example of a track of a working area during a leveling operation of the shovel 100 .

[0158] Hereinafter, the term "track" is used with the intention of including both the path (i.e., trajectory) along which the working part of the excavator 100 has moved and the path along which it may move in the future. Furthermore, a model for extracting the characteristic amount of the track of the working part of the excavator 100 is referred to as "learning model M2" regardless of whether it is in the learning process before learning, during learning, or after learning, and is referred to as "learned model LM2" when referring to an already learned model. The working part of the excavator 100 is, for example, the tip and back of the bucket 6. Furthermore, a model for generating the track of the working part in the composite action of the excavator 100 is referred to as "learning model M3" regardless of whether it is in the learning process before learning, during learning, or after learning, and is referred to as "learned model LM3" when referring to an already learned model. Furthermore, the composite action of the excavator 100 is an action formed by combining two or more prescribed actions of the excavator 100 (hereinafter, "basic actions") of different types from each other.

[0159] The shovel 100 includes a support device 150 . The support device 150 performs support related to the operation of the shovel 100 .

[0160] like Figure 6 As shown, the support device 150 includes the operating device 26, the controller 30, the camera device 40, the output device 50, and the input device 52. Furthermore, when the shovel 100 is remotely operated, the support device 150 may include the communication device 60.

[0161] The controller 30 includes an operation log providing unit 301 and a work supporting unit 302 as functional units.

[0162] In addition, when there are a plurality of shovels 100 included in the operation support system SYS, there may be a shovel 100 in which the controller 30 includes only the former of the operation log providing unit 301 and the operation support unit 302, and a shovel 100 in which the controller 30 includes only the latter. In this case, the former shovel 100 only has a function of acquiring the operation log of the shovel 100 and providing it to the information processing device 200 for the operation support function of the latter shovel 100.

[0163] The information processing device 200 includes, as functional units, an action log acquisition unit 2001 , an action log storage unit 2002 , a training data generation unit 2003 , a machine learning unit 2004 , a learned model storage unit 2005 , and a transmission unit 2006 .

[0164] The action log providing unit 301 is a functional unit for obtaining the action log of the shovel 100, which is the raw data for realizing the function of generating the track of the working part of the shovel 100, and providing it to the information processing device 200. Specifically, the action log when an operator with a long driving experience and relatively experience of the shovel 100 (hereinafter referred to as "experienced operator") operates the shovel 100 is obtained and provided to the information processing device 200.

[0165] The operation log of the shovel 100 includes data related to the shape of the work object around the shovel 100 and data related to the operation of the shovel 100 performed on the shape of the work object. The data related to the shape of the work object around the shovel 100 is, for example, data related to the topographic shape of the ground at the construction site which is the work object of the shovel 100. The data related to the shape of the work object of the shovel 100 is, for example, image data of the camera 40 or three-dimensional data of the work object obtained from the image data. The data related to the operation of the shovel 100 is, for example, data indicating the operation content of the operator. The data indicating the operation content of the operator is, for example, output data of the operation pressure sensor 29 corresponding to the hydraulic pilot type operation device 26, or output data (operation signal data) of the operation device 26 corresponding to the electric type operation device 26. In addition, the data related to the operation of the shovel 100 may be data indicating the operation state of the shovel 100 actually performed according to the operation of the operator. The data indicating the operation state of the shovel 100 is, for example, output data of the sensors S1 to S5 or data on the posture state of the shovel 100 acquired based on the output data of the sensors S1 to S5 .

[0166] The operation log providing unit 301 includes an operation log recording unit 301A, an operation log storage unit 301B, and an operation log transmitting unit 301C.

[0167] The action log recording unit 301A obtains the action log of the shovel 100 and records it in the action log storage unit 301B. For example, each time the shovel 100 performs an action, the action log recording unit 301A records data related to the shape of the work object around the shovel 100 at the start of the action or immediately before the action is performed, and data related to the action of the shovel 100 in the action log storage unit 301B.

[0168] The action log storage unit 301B stores the action logs of the shovel 100 in an accumulated form. For example, the action log storage unit 301B stores data related to the shape of the work object around the shovel 100 and data related to the action of the shovel 100 in an associated form for each action of the shovel 100. Specifically, the action log storage unit 301B stores record data indicating the correspondence between the data related to the shape of the work object around the shovel 100 and the data related to the action of the shovel 100 for each action of the shovel 100, and constructs a database of the action logs.

[0169] In addition, the operation log transmitted to the operation log storage unit 301B of the information processing device 200 by the operation log transmission unit 301C described later may be deleted afterwards.

[0170] The action log sending unit 301C sends the action log of the shovel 100 stored in the action log storage unit 301B to the information processing device 200 through the communication device 60. In addition, the action log sending unit 301C may also send record data of the correspondence between the shape of the work object around the shovel 100 and the data related to the action of the shovel 100 for each action of the shovel 100 to the information processing device 200.

[0171] For example, the operation log transmission unit 301C transmits the operation log of the shovel 100 that has not been transmitted and stored in the operation log storage unit 301B to the information processing device 200 in response to a signal (hereinafter, "transmission request signal") requesting transmission of the operation log of the shovel 100 from the information processing device 200. Furthermore, the operation log transmission unit 301C may automatically transmit the operation log of the shovel 100 that has not been transmitted and stored in the operation log storage unit 301B to the information processing device 200 at a predetermined time. The predetermined time is, for example, when the operation of the shovel 100 is stopped (the key switch is turned "off"), when the operation is started (the key switch is turned "on"), etc.

[0172] The operation log acquisition unit 2001 acquires the operation log of the shovel 100 received from the shovel 100 .

[0173] The operation log acquisition unit 2001 acquires the operation log of the shovel 100 by automatically transmitting a transmission request signal to the shovel 100 in response to an operation of the user of the information processing device 200 or at a predetermined time. Alternatively, the operation log acquisition unit 2001 may acquire the operation log of the shovel 100 automatically transmitted from the shovel 100 at a predetermined time.

[0174] The action log storage unit 2002 stores the action logs of the shovel 100 acquired by the action log acquisition unit 2001 in an accumulated form. For example, similarly to the action log storage unit 301B, the action log storage unit 2002 stores data on the shape of the work object around the shovel 100 and data on the action of the shovel 100 in an associated form for each action of the shovel 100.

[0175] The training data generating unit 2003 generates training data for machine learning based on the operation log of the shovel 100 in the operation log storage unit 2002, and outputs a set of a plurality of training data, namely, a training data set. The training data generating unit 2003 may automatically generate training data by batch processing, or may generate training data based on input from a user of the information processing device 200. The training data generating unit 2003 includes training data generating units 2003A to 2003C.

[0176] The training data generation unit 2003A generates training data for generating a learned model LM1 described later. For example, the training data generation unit 2003A generates training data composed of a combination of input data related to the shape of the work object around the excavator 100 and output data corresponding to the input data indicating the correct answer of the track (trajectory) of the working part of the excavator 100 generated by the operation of a skilled person.

[0177] The training data generation unit 2003B generates training data for generating the learned model LM2. For example, the training data generation unit 2003B generates training data that sets the same data representing the trajectory (track) of the working part in the prescribed action (basic action) of the excavator 100 based on the operation of a skilled person as a combination of input data and correct output data. At this time, the training data generation unit 2003B can generate training data for a plurality of basic actions (hereinafter referred to as "a plurality of basic actions") of different types from each other, and output a training data set for each of the plurality of basic actions. Furthermore, the training data generation unit 2003B can generate training data without depending on the difference between the plurality of basic actions, and output a training data set in which the training data corresponding to the plurality of basic actions coexist.

[0178] The multiple basic actions of the excavator 100 include, for example, at least two of the sweeping action, horizontal pulling action, rolling action, cleaning action, digging action, and soil discharge action used in the leveling operation. The sweeping action is, for example, an action of sweeping out sand and soil forward with the back of the bucket 6 by operating the attachment AT and pushing the bucket 6 forward along the ground. In the sweeping action, for example, the attachment AT performs a lowering action of the boom 4 and an opening action of the dipper 5. The horizontal pulling action is, for example, an action of leveling the unevenness of the ground (surface of the terrain) by operating the attachment AT and moving the tip of the bucket 6 in a manner of being pulled toward the near side along the ground in a substantially horizontal manner. In the horizontal pulling action, for example, the attachment AT performs an ascending action of the boom 4 and a closing action of the dipper 5. The rolling action is, for example, an action of operating the attachment AT and rolling the ground with the back of the bucket 6. In addition, the rolling action may be an action of rolling the ground by causing the back of the bucket 6 to hit the ground while moving the bucket 6 up and down. Furthermore, the rolling action may be an action of rolling the ground at a predetermined position with the back of the bucket 6 after pushing the bucket 6 forward along the ground and sweeping the sand and soil out to a predetermined position in front with the back of the bucket 6. In the rolling action, for example, the attachment AT performs a lowering action of the boom 4 while rolling the ground. The cleaning action may be an action of, for example, moving the upper rotating body 3 and rotating the bucket 6 left and right along the ground. Furthermore, the cleaning action may be an action of, for example, moving the attachment AT and the upper rotating body 3 and pushing the bucket 6 forward while rotating the bucket 6 left and right alternately along the ground. In the cleaning action, for example, the upper rotating body 3 alternately repeats the left and right rotating action. Furthermore, in the cleaning action, for example, in addition to the left and right alternate rotating action of the upper rotating body 3, the attachment AT may also perform a lowering action of the boom 4 and an opening action of the dipper arm 5 as in the case of the sweeping action. The excavation action is, for example, an action of operating the attachment AT to excavate the soil at a certain part of the ground and scoop it out into the bucket 6. The soil discharge action is an action of operating the attachment AT to discharge the soil scooped out into the bucket 6 by the excavation action to another part of the ground. In addition, in the soil discharge action, the excavator 100 can also operate the upper rotating body 3 in addition to the attachment AT. In addition, the multiple basic actions of the excavator 100 include, for example, at least two of the excavation action, boom raising and rotating action, boom lowering and rotating action, soil discharge action, and cleaning action used in excavation work. In addition, the multiple basic actions of the excavator 100 include, for example, soil cutting action and rolling action used in slope construction work.

[0179] The training data generation unit 2003C generates training data for generating the learned model LM3. For example, the training data generation unit 2003C generates data representing the track (trajectory) of the working part of the excavator 100 generated by the operation of a skilled person. The training data set output by the training data generation unit 2003C may include tracks of working parts in different basic actions of the excavator 100. Furthermore, the training data set output by the training data generation unit 2003C may include tracks of working parts in compound actions of the excavator 100 instead of or in addition to the tracks of working parts in the basic actions of the excavator 100.

[0180] The data indicating the trajectory (track) of the working part of the shovel 100 is generated based on the output data of the sensors S1 to S5 included in the data related to the operation of the shovel 100 , for example.

[0181] The machine learning unit 2004 generates a learned model by performing machine learning on the basic learning model based on the training data set generated by the training data generation unit 2003. The learned model (basic learning model) includes, for example, a neural network such as DNN (Deep Neural Network).

[0182] The machine learning unit 2004 includes machine learning units 2004A to 2004C.

[0183] The machine learning unit 2004A performs machine learning on the basic learning model M1 based on the training data set output from the training data generation unit 2003A. As a result, the machine learning unit 2004A can generate a learned model LM1 that can generate (output) data representing the trajectory of the working part in the basic action of the shovel 100 by taking data representing the shape of the working object around the shovel 100 as input data. Specifically, the machine learning unit 2004A generates a learned model LM1 for each of the multiple basic actions based on the training data sets for each of the multiple basic actions output from the training data generation unit 2003A.

[0184] The machine learning unit 2004B performs machine learning on the basic learning model M2 based on the training data set output from the training data generation unit 2003B to generate a learned model LM2. As described above, the learned model LM2 is a model for extracting feature quantities of the trajectory of the working part of the shovel 100.

[0185] For example, Figure 7As shown, the learning model M2 is a neural network that inputs data representing the track of the working part of the shovel 100, repeatedly performs downsampling and then repeatedly performs upsampling, and outputs data representing the track of the working part of the shovel 100. In the first half of the learning model M2, data of the same dimension as the input data (latent variable: Latent Variable) of the learning model M3 described later is generated by sequential downsampling. In addition, in the second half of the learned model LM2, data of the track of the working part of the shovel 100 is output by sequential upsampling.

[0186] The machine learning unit 2004B performs machine learning on the learning model M2 based on a training data set in which the input data and the output data are a combination of data of the same track representing the working part of the shovel 100. Thus, the machine learning unit 2004B can use the data of the track representing the working part of the shovel 100 as input data to generate a learned model LM2 that can output the same output data as the input data. At this time, the machine learning unit 2004B can perform machine learning in the upsampling process of the second half of the learning model M2 so as to reflect the result of the downsampling of the corresponding first half. Thus, the machine learning unit 2004B can perform machine learning on the learning model M2 so as to appropriately output the same output data as the input data.

[0187] As described above, in the upsampling process (second half) of the learning model M2, features for generating data identical to the input data from intermediate data of a dimension equivalent to the latent variable, that is, generating data representing the trajectory of the working part of the shovel 100 from the intermediate data are reflected. Therefore, various network parameters of the second half of the learned model LM2 can be extracted as feature quantities of the trajectory of the working part of the shovel 100 equivalent to the input data of the learned model LM2.

[0188] For example, Figure 8 As shown, in this example, the learned model LM2 can be used to extract the network parameters A1 to A4 of each level of the second half of the learned model LM2 as the feature quantity F_A of the track of the working part in the sweeping action of the shovel 100. Similarly, in this example, the learned model LM2 can be used to extract the network parameters B1 to B4 of each level of the second half of the learned model LM2 as the feature quantity F_B of the track of the working part in the horizontal pulling action of the shovel 100.

[0189] The learned model LM2 may be generated for each of a plurality of basic actions, or a common one may be generated for a plurality of basic actions.

[0190] The machine learning unit 2004C performs machine learning on the basic learning model M3 based on the training data set output from the training data generation unit 2003C to generate a learned model LM3. As described above, the learned model LM3 is a model for generating a trajectory of a working part in the combined motion of the shovel 100.

[0191] For example, Fig. 9 As shown, the learning model M3 is a neural network that repeatedly upsamples the latent variable z as input data to generate data of a dimension equivalent to the trajectory of the working part of the shovel 100. The learned model LM3 is equivalent to the generator in the GAN (Generative Adversarial Network). In addition, the machine learning unit 2004C uses the training data output from the training data generation unit 2003C to make the generator (learning model M3) and the discriminator (Discriminator) perform adversarial learning. Specifically, the discriminator is learned to distinguish the data representing the trajectory of the working part of the shovel 100 by a skilled person input as training data as "real data", and the data representing the trajectory of the working part of the shovel 100 generated by the generator as "false data". In addition, the generator is learned to generate data representing the trajectory of the working part of the shovel 100 that the discriminator cannot identify as training data (data representing the trajectory of the working part of the shovel 100 by a skilled person). The machine learning unit 2004C can generate a learned model LM3 capable of generating the trajectory of the working part of the shovel 100 from the latent variable z by causing the identifier and the generator to learn alternately.

[0192] Here, if Figure 7 to Figure 10 As shown in the figure, as described above, the structure of the second half of the learned model LM2 (learning model M2) is the same as the structure of the learned model LM3 (learning model M3). Figure 8 The feature quantity extracted by the learned model LM2 is reflected in the learned model LM3, and the trajectory of the working part of the shovel 100 corresponding to the feature quantity can be generated. Therefore, the learned model LM3 can generate data representing the trajectory of the working part in the complex action of the shovel 100 by appropriately reflecting the feature quantity of each basic action extracted from the learned model LM2 and different from each other.

[0193] For example, Fig.10As shown, in this example, the feature quantity FV obtained by coordinating the feature quantities F_A and F_B of the track of the working part in each of the sweeping action and the horizontal pulling action of the excavator 100 is reflected in the learned model LM3. Specifically, the feature quantity FV includes feature quantities FV1 to FV4 corresponding to network parameters A1, B1, network parameters A2, B2, network parameters A3, B3 and network parameters A4, B4, respectively. The feature quantity FV1 is adjusted within the range between network parameter A1 and network parameter B1 by coordinating network parameters A1 and B1 corresponding to the same level of the network, and is input to the level of the object of the learned model LM3. Similarly, the feature quantity FV2 is adjusted within the range between network parameter A2 and network parameter B2 by coordinating network parameters A2 and B2 corresponding to the same level of the network, and is input to the level of the object of the learned model LM3. Similarly, the feature value FV3 is adjusted between the network parameter A3 and the network parameter B3 by matching the network parameters A3 and B3 corresponding to the same layer of the network, and is input to the layer of the object of the learned model LM3. Similarly, the feature value FV4 is adjusted within the range between the network parameter A4 and the network parameter B4 by matching the network parameters A4 and B4 corresponding to the same layer of the network, and is input to the layer of the object of the learned model LM3. Thus, the learned model LM3 can generate the track of the working part in the composite action of the combination of the sweeping action and the horizontal pulling action of the excavator 100. And in this example, in addition to the feature value FV4, the learned model LM3 can adjust the details of the track of the working part in the composite action of the combination of the sweeping action and the horizontal pulling action of the excavator 100 by reflecting (inputting) the adjustment parameters in each layer (refer to the solid arrow in the figure). Furthermore, the adjustment parameter can be reflected in the output of the learned model LM3 to adjust the composite trajectory of the shovel 100 output from the learned model LM3 (see the dotted arrow in the figure).

[0194] For example, Fig.11 As shown, when viewed from the excavator 100 (upper rotating body 3), there are sometimes sand accumulations 1102 and 1103 in front and behind the depression 1101 of the ground. In this case, using the track 1100 of the working part in the composite action composed of the sweeping action and the horizontal pulling action of the excavator 100, the excavator 100 can move the sand accumulations 1102 and 1103 to the depression 1101 at one time and make the ground flat.

[0195] Return to Figure 6The learned models LM1 to LM3 output by the machine learning unit 2004 are stored in the learned model storage unit 2005. When the learned model LM1 is relearned or additionally learned by the machine learning unit 2004A, the learned model LM1 in the learned model storage unit 2005 is updated. The same is true for the cases where the learned models LM2 and LM3 are relearned or additionally learned by the machine learning units 2004B and 2004C.

[0196] The transmitting unit 2006 transmits the data of the learned models LM1 to LM3 to the shovel 100 .

[0197] For example, when the learned model LM1 is generated or updated by the machine learning unit 2004A, the transmission unit 2006 transmits the most recently generated or updated learned model LM1 to the shovel 100. Furthermore, the transmission unit 2006 may transmit the latest learned model LM1 in the learned model storage unit 2005 to the shovel 100 in response to a signal requesting the transmission of the learned model LM1 received from the shovel 100. The same may be applied to the learned models LM2 and LM3.

[0198] The work support unit 302 is a functional unit for supporting the operator's operation on the shovel 100 .

[0199] The work support unit 302 includes a learned model storage unit 302A, a work object shape acquisition unit 302B, a work selection unit 302C, a trajectory generation unit 302D, a specification setting unit 302E, a trajectory generation unit 302F, an action control unit 302G, and a display processing unit 302H.

[0200] The learned models LM1 to LM3 transmitted from the information processing device 200 and received via the communication device 60 are stored in the learned model storage unit 302A.

[0201] The work object shape acquisition unit 302B acquires data indicating the shape (topographic shape) of the work object around the shovel 100 based on the output of the imaging device 40 or the distance sensor.

[0202] The operation selection unit 302C selects the operation to be performed by the excavator 100 from a plurality of candidate operations of different types, based on the input from the user (operator) received through the input device 52. Thus, the user can generate a track of the operation part of the excavator 100 corresponding to the type of operation performed by the excavator 100. The plurality of candidate operations include, for example, leveling operation, excavation operation, slope construction operation, etc. Furthermore, when the excavator 100 is remotely operated, the operation selection unit 302C can select the operation to be performed by the excavator 100 from a plurality of candidate operations, based on the input from the user using the remote operation support device 300 received through the communication device 60. For each operation selected by the operation selection unit 302C, basic actions of the excavator 100 that can be combined as a composite action are predefined. For example, as described above, in the case of the leveling operation, the basic actions that can be combined as a composite action are a cleaning action, a horizontal pulling action, a rolling action, an excavation action, a soil discharge action, etc. Therefore, the work selection unit 302C selects a basic operation to be combined with the complex operation of the shovel 100 by selecting the type of work.

[0203] The track generation unit 302D generates data representing a track of the working part of the excavator 100 using the learned model LM1 for each of the multiple basic actions corresponding to the work selected by the work selection unit 302C, and based on data related to the shape of the work object around the excavator 100. The multiple basic actions of each object of the multiple candidate works are predefined as described above. For example, the multiple basic actions corresponding to the leveling work include, for example, a sweeping action, a horizontal pulling action, and a rolling action. In addition, the multiple basic actions corresponding to the leveling work may include an excavation action, a soil discharge action, a cleaning action, and the like.

[0204] The specification setting unit 302E sets (adjusts) the specification related to the combination of the plurality of basic actions in the compound action of the shovel 100 based on the input from the user (operator) received through the input device 52. Furthermore, when the shovel 100 is remotely operated, the specification setting unit 302E may set the specification related to the combination of the plurality of basic actions of the shovel 100 based on the input from the user using the remote operation support device 300 received through the communication device 60.

[0205] For example, the specification setting unit 302E sets (selects) basic actions to be combined as the composite action of the excavator 100 from the plurality of basic actions specified for the operation selected by the operation selection unit 302C. Furthermore, the specification setting unit 302E can set the overall composite action of the excavator 100 to 100%, and adjust the composition ratio (hereinafter referred to as "compound ratio") of each of the plurality of basic actions in the composite action. Thus, when the user generates the trajectory of the operation part in the composite action of the excavator 100, the user can adjust the distribution of each of the basic actions constituting the composite action. Furthermore, the specification setting unit 302E can set the specifications for the distribution of the characteristic quantities of each of the plurality of basic actions. At this time, for example, Fig.10 As shown, in the case where there are multiple feature quantities reflected (input) in different layers of the learned model LM3, the specifications for the allocation of feature quantities of each of the multiple basic actions can be set for each of the multiple feature quantities. In addition, the specification setting unit 302E can also adjust the implementation order of each of the multiple basic actions in the compound action. Thus, when the user generates the trajectory of the working part in the compound action of the excavator 100, the implementation order of the basic actions that constitute the compound action can be adjusted. In addition, the specification setting unit 302E can also adjust the conditions such as the passing point of the working part of each of the multiple basic actions that constitute the compound action. Thus, when the user generates the trajectory of the working part in the compound action of the excavator 100, the user can limit the position where the working part should pass for each of the multiple basic actions while confirming the image of the terrain shape around the excavator 100 displayed on the display device 50A.

[0206] The trajectory generation unit 302F generates data indicating the trajectory of the working part in the complex motion of the shovel 100 using the learned models LM2 and LM3 based on the generation data for each of the plurality of basic motions by the trajectory generation unit 302D and the data of the setting result by the specification setting unit 302E.

[0207] Specifically, the track generation unit 302F extracts the feature value of each of the plurality of basic actions using the learned model LM2 based on the generated data of each of the plurality of basic actions by the track generation unit 302D. Furthermore, the track generation unit 302F generates data representing the track of the working part in the composite track of the shovel 100 using the learned model LM3 based on the feature value of each of the plurality of basic actions and the setting result of the specification setting unit 302E.

[0208] For example, Fig.10As shown, the trajectory generation unit 302F adjusts the feature amount FV according to the composite ratio of each of the plurality of basic actions set by the specification setting unit 302E, and generates data representing the trajectory of the working part in the composite trajectory of the shovel 100 using the learned model LM3. Furthermore, the trajectory generation unit 302F can reflect the setting result of the specification setting unit 302E in the data representing the trajectory of the working part in the composite trajectory of the shovel 100 by adjusting the adjustment parameter.

[0209] Return to Figure 6 The motion control unit 302G operates the shovel 100 in such a manner that the working part of the bucket 6 moves along the track corresponding to the data generated by the track generation unit 302F based on the input from the user (operator) received through the input device 52. Specifically, the motion control unit 302G can control the hydraulic control valve 31 while grasping the position of the working part of the bucket 6 based on the output of the sensors S1 to S5, thereby operating the shovel 100 in such a manner that the working part of the bucket 6 moves along the target track.

[0210] For example, the motion control unit 302G operates the shovel 100 in such a manner that the working part of the bucket 6 moves along the track corresponding to the data generated by the track generation unit 302F according to the motion execution instruction input from the user through the input device 52. In addition, when the shovel 100 is remotely operated, the motion control unit 302G can operate the shovel 100 according to the motion execution instruction input received from the user using the remote operation support device 300 through the communication device 60.

[0211] Furthermore, the motion control unit 302G may operate the shovel 100 by assisting the operator's operation based on the operation of the operating device 26 or a remote operation signal so that the working part of the bucket 6 moves along a trajectory corresponding to the data generated by the trajectory generation unit 302F.

[0212] The display processing unit 302H causes the display device 50A to display a screen related to the generation of a target trajectory of the working area of ​​the shovel 100 .

[0213] The screen related to the generation of the trajectory of the working part of the excavator 100 includes, for example, a screen that displays an image representing the shape of the construction object around the excavator 100 based on the data acquired by the working object shape acquisition unit 302B. As a result, the user can confirm the shape of the construction object around the excavator 100 and determine the action that the excavator 100 needs to perform in accordance with the work content. The image representing the shape of the construction object around the excavator 100 may be, for example, a camera image of the camera device 40 representing the shape of the construction object around the excavator 100 or a processed image thereof, or may be an image of three-dimensional data representing the shape of the construction object around the excavator 100. The processed image is, for example, an image obtained by performing viewpoint conversion processing on the camera image acquired by the camera device 40. For example, Fig.12 As shown, an image TG representing the shape of the construction object around the shovel 100 as viewed from the viewpoint of the cab 10 of the shovel 100 is displayed on the screen. The image TG corresponds to an image of three-dimensional data of the construction object around the shovel 100. At this time, an image CG representing the shovel 100 (attachment AT) may be displayed on the screen in a form corresponding to the positional relationship of the image TG representing the shape of the construction object around the shovel 100. In addition, an image representing the shape of the construction object around the shovel 100 as viewed from a predetermined viewpoint around the shovel 100 may be displayed on the screen. Thus, the user can confirm the shape of the construction object around the shovel 100 from a viewpoint different from the viewpoint of the normal cab 10. The viewpoint of the image representing the shape of the construction object around the shovel 100 displayed on the screen can be arbitrarily changed according to a predetermined input inputted from the user through the input device 52. Thus, the user can confirm the image representing the shape of the construction object around the shovel 100 from a favorite viewpoint.

[0214] In addition, the screen related to the generation of the trajectory of the working part of the shovel 100 includes, for example, an operation screen for the user to perform an operation input for selecting a target work to be performed by the shovel 100 from a plurality of works through the work selection unit 302C. In this operation screen, the user can use the input device 52 to perform an operation to select a target work to be performed by the shovel 100 from a plurality of works.

[0215] In addition, the screen related to the generation of the trajectory of the working part of the shovel 100 includes, for example, an operation screen (setting screen) on which the user performs operation input for setting the specifications related to the combination of multiple basic actions in the compound action of the shovel 100 through the specification setting unit 302E. In this operation screen, the user can use the input device 52 to perform an operation to set the specifications related to the combination of multiple basic actions in the compound action of the shovel 100.

[0216] Furthermore, the screen related to the generation of the track of the working part of the shovel 100 includes, for example, an operation screen for the user to input an operation to cause the track production unit 302F to generate the track of the working part of the shovel 100 according to the specifications set by the specification setting unit 302E. In this operation screen, the user can use the input device 52 to perform an operation to issue an instruction to cause the track production unit 302F to generate the track of the working part of the shovel 100 according to the set specifications.

[0217] Furthermore, in the screen related to the generation of the track of the working part of the excavator 100, a screen is included that displays the track of the working part of the excavator 100 corresponding to the data generated by the track generation unit 302F. At this time, the track of the working part of the excavator 100 can be displayed superimposed on the image representing the shape of the construction object around the excavator 100. Furthermore, a simulated dynamic image of the excavator 100 that moves in a manner in which the working part moves along the track can also be displayed on the screen. Thus, the user can grasp in advance how the compound action of the excavator 100 is performed based on the track of the working part in the compound action of the excavator 100 corresponding to the data generated by the track generation unit 302F. At this time, an image representing the shape of the working object predicted after the working part moves along the track when the simulated dynamic image of the excavator 100 ends can also be displayed on the screen. Thus, the user can understand in advance how the shape of the work object changes by making the shovel 100 perform a compound action according to the track of the working part in the compound action of the shovel 100 corresponding to the data generated by the track generation unit 302F. At this time, at the end of the dynamic image of the simulation of the shovel 100, an image showing the shape of the work object predicted after the working part moves along the track can also be displayed on the screen. Thus, the user can understand in advance how the shape of the work object changes by making the shovel 100 perform a compound action according to the track of the working part in the compound action of the shovel 100 corresponding to the data generated by the track generation unit 302F.

[0218] In addition, the screen related to the generation of the track of the working part of the shovel 100 includes an operation screen for automatically operating the shovel 100 in such a manner that the working part of the shovel 100 moves along the track corresponding to the data generated by the track generation unit 302F. In this operation screen, the user can use the input device 52 to perform an operation to issue an instruction to operate the shovel 100 in such a manner that the working part moves along the track corresponding to the data generated by the track generation unit 302F. The action control unit 302G operates the shovel 100 in such a manner that the working part of the bucket 6 moves along the track corresponding to the data generated by the track generation unit 302F according to the instruction of the action execution outputted from the user by the operation on the operation screen. For example, the operation screen is the same as the screen displaying the track of the working part of the shovel 100 corresponding to the data generated by the track generation unit 302F, and includes an icon corresponding to the instruction of the action execution. Thus, after confirming the trajectory of the working area of ​​the shovel 100 , the user can input an instruction to execute the operation of the shovel 100 through the input device 52 or the like.

[0219] Furthermore, when the shovel 100 is remotely operated, the display processing unit 302H may also transmit data related to a screen related to the generation of a trajectory of the working part of the shovel 100 to the remote operation support device 300 via the communication device 60. Thus, the display processing unit 302H can display a screen related to the generation of a target trajectory of the working part of the shovel 100 on the remote operation support device 300 (display device).

[0220] In addition, when the shovel 100 is remotely operated, the function of the display processing unit 302H may be provided in the remote operation support device 300. Thus, a screen related to the generation of the track of the working part of the shovel 100 can be displayed on the display device of the remote operation support device 300. Therefore, the user (operator) of the remote operation support device 300 can use the screen displayed on the display device of the remote operation support device 300 to cause the track generation unit 302F to generate data representing the track of the working part in the composite track of the shovel 100. Furthermore, when the shovel 100 is remotely operated, part or all of the functions of the work object shape acquisition unit 302B, the work selection unit 302C, the track generation unit 302D, the specification setting unit 302E, the track generation unit 302F, and the action control unit 302G may be provided in the remote operation support device 300. Furthermore, part or all of the functions of the work object shape acquisition unit 302B, the work selection unit 302C, the trajectory generation unit 302D, the specification setting unit 302E, the trajectory generation unit 302F, and the motion control unit 302G can be transferred to the information processing device 200. As a result, the processing load of the shovel 100 or the remote operation support device 300 related to the processing related to the generation of the trajectory of the work part in the complex motion of the shovel 100 or the motion control of the shovel 100 can be reduced.

[0221] [Setting screen for specifications related to the combination of multiple basic actions]

[0222] Next, refer to Fig.13 , Fig.14 , a specification setting screen related to a combination of a plurality of basic operations in a compound operation of the shovel 100 will be described.

[0223] <Example 1>

[0224] Fig.13 1 is a diagram showing a first example (setting screen 1300 ) of a setting screen for specifications related to a combination of a plurality of basic operations in a complex operation of the shovel 100 .

[0225] In addition, when the shovel 100 is remotely operated, Fig.13 The setting screen 1300 of the remote operation support device 300 is displayed on the display device of the remote operation support device 300, and is configured to be operable through the input device of the remote operation support device 300. Fig.14 The setting screen 1400 may also be the same.

[0226] like Fig.13 As shown, setting screen 1300 includes image 1301 , operation image 1302 , and operation image 1303 .

[0227] The image 1301 schematically shows the basic operation of the shovel 100. The image 1301 includes images 1301A to 1301C.

[0228] The image 1301A schematically shows the horizontal pulling operation of the shovel 100 .

[0229] The image 1301B schematically shows the compaction operation of the shovel 100 .

[0230] The image 1301C schematically shows the sweeping operation of the shovel 100 .

[0231] The images 1301A to 1301C are arranged vertically on the left side of the center in the left-right direction of the setting screen 1300 .

[0232] The operating images 1302 and 1303 are images that can be operated via the input device 52 .

[0233] The operation image 1302 is an operation image for adjusting the allocation of the combination of the horizontal pull action, the rolling action, and the swipe action. The operation image 1302 includes operation images 1302A to 1302C.

[0234] The operating image 1302A is a slide bar that can adjust the composition ratio (compound rate) of the horizontal pulling motion in the compound motion of the shovel 100 between 0% and 100%. The operating image 1302A is arranged on the right side of the image 1301A.

[0235] The operating image 1302B is a slide bar that can adjust the composition ratio (compound ratio) of the rolling action in the compound action of the shovel 100 between 0% and 100%. The operating image 1302B is arranged on the right side of the image 1301B.

[0236] The operating image 1302C is a slide bar that can adjust the composition ratio (compound ratio) of the sweeping action in the compound action of the shovel 100 between 0% and 100%. The operating image 1302C is arranged on the right side of the image 1301C.

[0237] Operation images 1302A to 1302C are arranged vertically on the right side of the center in the left-right direction of setting screen image 1300 .

[0238] The composite ratios of the horizontal pull action, the rolling action, and the sweeping action are adjusted so that the total composite ratio of the horizontal pull action, the rolling action, and the sweeping action is 100%. For example, all the sliders of the operation images 1302A to 1302C are aligned to 0% in the initial state. If a percentage greater than 0% is specified for one of the sliders, the other sliders are restricted to be movable only within a range not exceeding the total 100%.

[0239] The operating image 1303 is an operating image for determining the composite ratio of the horizontal pulling action, the rolling action, and the sweeping action among the composite actions of the shovel 100 specified in the operating image 1302, and executing the generation processing of the data representing the track of the working part based on the track generation unit 302F.

[0240] In this example, the slider of the operation image 1302B is set to 0%, and the operation images 1302A and 1302B are both set to about 50%. In this state, by operating the operation image 1303, the user can create data representing the trajectory of the working part in the composite action of the sweeping action and the horizontal pulling action of the excavator 100 according to the composite ratio specified in the operation image 1302 (refer to Fig.11 ).

[0241] If one of the sliders in the operation images 1302A to 1302C is designated as 100% and the other two sliders are designated as 0%, the trajectory generation unit 302F can generate a trajectory for the working area based not on the complex motion of the shovel 100 but on the basic motion.

[0242] <Example 2>

[0243] Fig.14 14 is a diagram showing a first example (setting screen 1400 ) of a setting screen for specifications related to a combination of a plurality of basic operations in a compound operation of the shovel 100 .

[0244] like Fig.13 As shown, setting screen 1400 includes image 1401 , operation image 1402 , and operation image 1403 .

[0245] The image 1401 schematically shows the basic operation of the shovel 100. The image 1401 includes images 1401A to 1401C.

[0246] The image 1401A schematically shows the horizontal pulling operation of the shovel 100. The image 1401A is arranged above the center of the setting screen 1400 in the left-right direction and the center in the up-down direction.

[0247] The image 1401B schematically shows the rolling operation of the shovel 100. The image 1401B is arranged on the right side of the left-right center of the setting screen 1400 and on the lower side of the up-down center.

[0248] The image 1401C schematically shows the sweeping operation of the shovel 100. The image 1401C is arranged on the left side of the center in the left-right direction of the setting screen 1400 and on the lower side than the center in the up-down direction.

[0249] The images 1401A to 1401C are arranged in a substantially equilateral triangle on the setting screen 1400 .

[0250] The operating images 1402 and 1403 are images that can be operated via the input device 52 .

[0251] The operation image 1402 is an operation image for adjusting the allocation of the combination of the horizontal pull action, the rolling action, and the swipe action. The operation image 1402 includes operation images 1402A to 1402D.

[0252] The operating image 1402A is a circular image showing a state in which the composition ratio (compounding rate) of the horizontal pulling action of the shovel 100 in the compound action of the shovel 100 is 100%, that is, the shovel 100 is not performing the compound action but performing the horizontal pulling action. The operating image 1402A is arranged adjacent to the lower side of the image 1401A.

[0253] The operating image 1402B is a circular image showing that the ratio of the rolling action in the combined action of the shovel 100 is 100%, that is, the shovel 100 is not performing the combined action but performing the rolling action. The operating image 1402B is arranged on the left side of the image 1401A.

[0254] The operating image 1402C is a circular image showing that the ratio of the sweeping motion in the combined motion of the shovel 100 is 100%, that is, the shovel 100 is not performing the combined motion but performing the sweeping motion. The operating image 1402C is arranged on the right side of the image 1401C.

[0255] Operation images 1402A to 1402C are arranged in a substantially equilateral triangle, and operation images 1402A and 1402B, operation images 1402B and 1402C, and operation images 1402C and 1402A are connected by straight lines.

[0256] The operating image 1402D is an image of a circular cursor for designating a composition ratio (compound ratio) of the horizontal pulling motion, the rolling motion, and the sweeping motion in the compound motion of the shovel 100. The operating image 1402D can be moved on or within the triangle formed by the operating images 1402A to 1402C.

[0257] The composition ratio (compounding ratio) of the horizontal pulling action, the rolling action, and the sweeping action in the compound action of the excavator 100 is uniquely specified at the cursor position of the operating image 1402D on the triangle formed by the operating images 1402A to 1402C or inside thereof. Specifically, the smaller the distance between the operating image 1402X designated as one and the operating image 1402D becomes in the relative relationship with the distances between the other two operating images 1402X and the operating image 1402D, the greater the compounding ratio of the basic action corresponding to the one operating image becomes. Conversely, the larger the distance between the operating image 1402X designated as one and the operating image 1402D becomes in the relative relationship with the distances between the other two operating images 1402X and the operating image 1402D, the smaller the compounding ratio of the basic action corresponding to the one operating image becomes.

[0258] The operating image 1403 is an operating image for determining the composite ratio of the horizontal pulling action, the rolling action, and the sweeping action among the composite actions of the shovel 100 specified in the operating image 1402, and executing the generation processing of the data representing the track of the working part by the track generation unit 302F.

[0259] For example, Fig.14 As shown, when the operating image 1402D is located on one side of the triangle between the operating images 1402A and 1402C, the compounding rate of the rolling action is specified as 0%. Furthermore, the compounding rates of the horizontal pulling action and the sweeping action in the compound action of the excavator 100 are specified according to the position of the operating image 1402D on the line segment (edge) between the operating images 1402A and 1402C. In this state, if the operating image 1403 is operated, the track generating unit 302F generates data representing the track of the working part in the compound action of the combination of the horizontal pulling action and the sweeping action of the excavator 100 according to the compounding rate corresponding to the position of the operating image 1402D. Thus, the user can create data representing the track of the working part in the compound action based on the combination of the horizontal pulling action, the sweeping action and the rolling action of the excavator 100 (refer to Fig.11 ).

[0260] Furthermore, when the operating image 1402D is located on one side of the triangle between the operating images 1402A and 1402B, the composite rate of the sweeping action is specified as 0%. Furthermore, the composite rates of the horizontal pulling action and the rolling action in the composite action of the excavator 100 are specified according to the position of the operating image 1402D located on the line segment (one side) between the operating images 1402A and 1402B. In this state, if the operating image 1403 is operated, the track generation unit 302F generates data representing the track of the working part in the composite action of the combination of the horizontal pulling action and the rolling action of the excavator 100 according to the composite rate corresponding to the position of the operating image 1402D. Thus, the user can create data representing the track of the working part in the composite action of the combination of the horizontal pulling action and the rolling action in the horizontal pulling action, sweeping action, and rolling action of the excavator 100.

[0261] Furthermore, when the operating image 1402D is located on one side of the triangle between the operating images 1402B and 1402C, the compounding rate of the horizontal pulling action is specified as 0%. Then, the compounding rates of the rolling action and the sweeping action in the compound action of the excavator 100 are specified according to the position of the operating image 1402D on the line segment (one side) between the operating images 1402B and 1402C. In this state, if the operating image 1403 is operated, the track generation unit 302F generates data representing the track of the working part in the compound action of the rolling action and the sweeping action of the excavator 100 according to the compounding rate corresponding to the position of the operating image 1402D. Thus, the user can create data representing the track of the working part in the compound action of the rolling action and the sweeping action in the horizontal pulling action, the sweeping action, and the rolling action of the excavator 100.

[0262] Furthermore, when the operating image 1402D is located inside the triangle formed by the operating images 1402A to 1402C, the composite ratios of the horizontal pulling action, the rolling action, and the sweeping action are all specified as being greater than 0%. Specifically, the composite ratios of the horizontal pulling action, the rolling action, and the sweeping action are specified as being 100% in total according to the relative size relationship of the distances between the operating image 1402D and the operating images 1402A to 1402C. In this state, if the operating image 1403 is operated, the track generation unit 302F generates data representing the track of the working part in the composite action of the combination of the horizontal pulling action, the rolling action, and the sweeping action of the excavator 100 according to the composite ratio corresponding to the position of the operating image 1402D. Thus, the user can create data representing the track of the working part of the composite action based on the horizontal pulling action, the sweeping action, and the rolling action of the excavator 100.

[0263] Furthermore, if the operating image 1402D is designated at the same position as any of the operating images 1402A to 1402C, the trajectory generating unit 302F can generate a trajectory of the working area based not on the complex motion of the shovel 100 but on the basic motion.

[0264] [Processing related to generation of trajectory of working area of ​​shovel]

[0265] Next, refer to Fig.15 , a process related to the generation of a trajectory of a working area of ​​the shovel 100 will be described.

[0266] Fig.15 This is a flowchart schematically showing an example of processing related to the generation of a trajectory of a working area of ​​the shovel 100 .

[0267] When the controller 30 receives a predetermined input for executing a process related to the generation of a trajectory of the working part of the shovel 100 from the input device 52 or the remote operation support device 300, the operation starts. Fig.15 Flowchart of the process.

[0268] like Fig.15 As shown, in step S102 , the work object shape acquisition unit 302B acquires data of the latest captured image of the imaging device 40 , and acquires data representing the shape of the work object around the shovel 100 based on the captured image.

[0269] When the process of step S102 is completed, the controller 30 proceeds to step S104 .

[0270] In step S104, the display processing unit 302H displays an image showing the shape of the work object around the shovel 100 on the display device 50A or the display device of the remote operation support device 300 (see FIG. 104 ). Fig.12 ).

[0271] When the process of step S104 is completed, the controller 30 proceeds to step S106 .

[0272] In step S106, the display processing unit 302H displays a screen for selecting one operation from among the plurality of candidate operations on the display device 50A or the display device of the remote operation support device 300. For example, the display processing unit 302H superimposes images representing the shapes of the operation objects around the shovel 100 to display options for the plurality of candidate operations.

[0273] When the process of step S106 is completed, the controller 30 proceeds to step S108 .

[0274] In step S108 , the work selection unit 302C selects one work from candidate works (eg, leveling work, excavation work, slope construction work, etc.) based on a predetermined input in the selection screen of step S106 received via the input device 52 or the like.

[0275] When the process of step S108 is completed, the controller 30 proceeds to step S110 .

[0276] In step S110 , the trajectory generation unit 302D generates trajectories of work sites of each of the plurality of basic movements corresponding to the work selected in step S106 using the learned model LM1 based on the data acquired in step S102 .

[0277] When the process of step S110 is completed, the controller 30 proceeds to step S112 .

[0278] In step S112 , the display processing unit 302H displays a setting screen of specifications related to a combination of a plurality of basic operations on the display device 50A or the display device of the remote operation support device 300 .

[0279] When the processing of step S112 is completed, the controller 30 proceeds to step S114.

[0280] In step S114 , the specification setting unit 302E sets specifications related to the combination of the plurality of basic operations based on a predetermined input received on the setting screen of step S112 via the input device 52 or the like.

[0281] If the process of step S114 is completed, the controller 30 proceeds to step S116 .

[0282] In step S116 , the trajectory generation unit 302F generates data indicating a trajectory of a working area by the combined motion of the shovel 100 using the learned models LM2 and LM3 based on the trajectory data generated in step S110 and the setting content of step S114 .

[0283] When the process of step S116 is completed, the controller 30 proceeds to step S118 .

[0284] In step S118 , the display processing unit 302H displays an image showing the trajectory of the working area based on the combined operation of the shovel 100 corresponding to the data generated in step S116 on the display device 50A or the display device of the remote operation support device 300 .

[0285] When the process of step S118 is completed, the controller 30 proceeds to step S120 .

[0286] In step S120, the controller 30 determines whether an input for instructing execution of the action of the shovel 100, which is to move the working part of the bucket 6 along the track corresponding to the data generated in step S118, has been received. If the input for instructing execution of the action of the shovel 100 has been received, the controller 30 proceeds to step S122. On the other hand, if other inputs have been received, specifically, if input for regenerating the track of the working part based on the complex action of the shovel 100 has been received, the controller 30 returns to step S106.

[0287] In step S122 , the operation control unit 302G controls the hydraulic control valve 31 to automatically operate the shovel 100 so that the working part of the bucket 6 moves along a trajectory corresponding to the data generated in the most recent process of step S116 .

[0288] When the controller 30 completes the process of step S122 , it ends the process of this flowchart.

[0289] When the process of step S122 is completed, the shovel 100 (attachment AT) may be in a state where the working part of the bucket 6 is located at the end of the target trajectory. The shovel 100 (attachment AT) may return to the posture state before the process of step S122 is started.

[0290] Thus, in this example, the support device 150 (controller 30) can generate a trajectory of a working part based on a composite motion of the shovel 100 in accordance with the specification of a combination of a plurality of basic motions set by the user. Therefore, the working efficiency of the shovel 100 can be improved.

[0291] [effect]

[0292] Next, the functions of the support device, construction machine, support system, and program according to the present embodiment will be described.

[0293] In the present embodiment, the support device includes an input unit and a display unit. The support device is, for example, the support device 150, the information processing device 200, or the remote operation support device 300. The input unit is, for example, the input device 52 or the input device of the remote operation support device 300. Specifically, the input unit receives input from the user. Moreover, the display unit determines the specifications related to the combination of multiple actions of different types of the construction machine based on the input from the input unit, and displays the first operation screen, which is used to generate a track of the working part of the construction machine based on the composite action composed of the multiple actions. The construction machine is, for example, the excavator 100 described above. In addition, the construction machine may also be the crane, forklift, or road machine described above. The display unit is, for example, the display device 50A described above or the display device of the remote operation support device 300. The first operation screen is, for example, the setting screen 1300 or the setting screen 1400 described above.

[0294] Furthermore, in this embodiment, the program causes an information processing device having an input unit and a display unit to implement a display step. Specifically, in the display step, a specification related to a combination of a plurality of actions of different types of the construction machine is determined based on an input from the input unit, and a first operation screen is displayed on the display unit, the first operation screen being used to generate a track of a working part of the construction machine based on a composite action composed of the plurality of actions. The display step is, for example, step S112.

[0295] More specifically, in the first operation screen, the user can also perform an operation to confirm the specification through the input unit.

[0296] Thus, the user can generate a track of the working part of the composite action of the excavator 100 based on the combination of multiple actions by operating the first operation screen using the input unit. Therefore, the user can operate the construction machine in a manner that the working part moves along the track, or operate the automatic operation function of the construction machine in a manner that the working part moves along the track, thereby promoting the operation of the construction machine. Therefore, the operation efficiency when the multiple basic actions of the construction machine are combined to perform the operation can be improved.

[0297] Furthermore, in the present embodiment, the plurality of actions are actions that the construction machine performs using the construction attachment in a predetermined operation. The construction attachment is, for example, the terminal attachment AT described above.

[0298] This allows the user to generate a trajectory of a work site based on a complex operation of the construction machine in accordance with a predetermined work to be performed by the construction machine.

[0299] Furthermore, in this embodiment, the prescribed operation may be a leveling operation, an excavation operation, or an inclined surface construction operation. When the prescribed operation is a leveling operation, the plurality of actions may include at least two of a horizontal pulling operation, a rolling operation, a cleaning operation, an excavation operation, and a soil discharge operation. When the prescribed operation is an excavation operation, the plurality of actions may include at least two of an excavation operation, a boom raising and rotating operation, a boom lowering and rotating operation, a soil discharge operation, and a cleaning operation. When the prescribed operation is an inclined surface construction operation, the plurality of actions may include a soil cutting operation and a rolling operation.

[0300] This allows the user to generate a trajectory of a work site based on a complex motion of a construction machine in accordance with leveling work, excavation work, or slope construction work.

[0301] Furthermore, in the present embodiment, the track of the working part may be a track of a predetermined part of an end attachment at the front end of a construction attachment of a construction machine. The end attachment may be, for example, the bucket 6.

[0302] Thereby, the user can generate a track of a predetermined portion of the end attachment of the front end of the working attachment which is a working portion that comes into contact with a work object in the construction machine.

[0303] Furthermore, in the present embodiment, the above-mentioned specifications may include allocation of a combination of a plurality of actions in a complex action of the construction machine.

[0304] Thus, the user can cause the support device to generate a more appropriate trajectory for the working area based on the complex motion of the shovel 100 by appropriately adjusting the allocation of the combination of the plurality of motions in the complex motion of the construction machine (for example, the composition ratio).

[0305] Furthermore, in the present embodiment, the display unit may display an image showing the shape of the work object around the construction machine, and may superimpose the track generated by the first operation screen on the image showing the shape of the work object around the construction machine.

[0306] More specifically, the image representing the shape of the work object may be a captured image of the surroundings of the construction machine or a processed image thereof, or a three-dimensional data image of the work object surrounding the construction machine.

[0307] This allows the user to check the rationality of the generated trajectory, etc., while comparing the image representing the shape of the work object around the construction machine with the generated trajectory.

[0308] Furthermore, in the present embodiment, the display unit may superimpose on the image representing the shape of the work object a simulated dynamic image of the construction machine in which the work site moves along the track generated by the first operation screen.

[0309] Thereby, the user can confirm in advance the state in which the shovel 100 performs the compound operation so that the working part moves along the generated trajectory.

[0310] Furthermore, in the present embodiment, the display unit may display an image showing the shape of the work object predicted after the work site moves on the trajectory generated by the first operation screen.

[0311] Thereby, the user can confirm in advance how the shape of the work object changes after the shovel 100 performs the complex operation so that the work site moves along the generated trajectory.

[0312] Furthermore, in the present embodiment, the display unit may display a second operation screen for operating the construction machine so that the working area moves along a track generated by the first operation screen, based on the input from the input unit.

[0313] More specifically, on the second operation screen, the user can use the input unit to perform an operation for instructing the construction machine to operate so that the work site moves along the track generated on the first operation screen.

[0314] Thus, the user can make the excavator 100 automatically perform a compound action in such a way that the working part moves along the generated track. Therefore, for example, even if the user (operator) is not a skilled person, the working part of the construction machine can be moved along the generated track, and as a result, the working efficiency of the construction machine can be further improved.

[0315] Furthermore, in this embodiment, the construction machine may also include the above-mentioned support device. That is, the construction machine may include: an input unit that receives input from a user; and a display unit that determines specifications related to a combination of multiple actions of different types of the construction machine based on the input from the input unit, and displays a first operation screen for generating a trajectory of the working part of the construction machine based on a composite action composed of the multiple actions.

[0316] Thus, a user (operator) riding on the construction machine can generate a trajectory of a working area based on the complex motion of the shovel 100 using the support device.

[0317] Furthermore, in the present embodiment, the support system may include the construction machine and the support device. That is, the support system is, for example, the operation system SYS.

[0318] Thus, a user (operator) outside the construction machine can generate a trajectory of a working area based on the complex motion of the shovel 100 using the support device.

[0319] [Further disclosure]

[0320] Regarding the above-mentioned embodiment, the following contents are further disclosed. (1)

[0322] A work support system comprising:

[0323] an acquisition unit that acquires data related to a shape of a work object around the construction machine; and

[0324] The first generating unit generates, based on the data acquired by the acquiring unit, a trajectory of a working location of the construction machine based on a composite motion formed by combining a plurality of motions of the construction machine.

[0325] The work support system is, for example, the operation support system SYS. The construction machine is, for example, the shovel 100. The acquisition unit is, for example, the work object shape acquisition unit 302B. The first generation unit is the track generation unit 302F. (2)

[0327] The work support system according to (1) further comprising a second generation unit configured to generate a trajectory of the work site for each of the plurality of actions based on the data acquired by the acquisition unit.

[0328] The first generation unit generates the trajectory of the working part based on the combined motion, based on the trajectories of the working part in each of the plurality of motions generated by the second generation unit.

[0329] The second generation unit is, for example, the above-mentioned track generation unit 302D. (3)

[0331] The work support system according to (1) or (2), further comprising a selection unit configured to select the plurality of actions from actions of the construction machine based on the data acquired by the acquisition unit.

[0332] The selection unit is, for example, the above-mentioned operation selection unit 302C. (4)

[0334] The work support system according to (2) includes an input unit that receives input from a user.

[0335] The first generating unit generates a trajectory of the working part based on the combined motion, based on the input related to the specification of the combination of the plurality of motions from the input unit.

[0336] The input unit is, for example, the input device 52 described above or an input device of the remote operation support device 300 . (5)

[0338] The work support system according to (4) further comprises an extraction unit configured to extract a feature value of each of the plurality of actions generated by the second generation unit,

[0339] The first generating unit adjusts the matching ratio of the characteristic amounts of each of the plurality of actions based on the input related to the specification of the combination of the plurality of actions from the input unit, and generates the trajectory of the working part of the composite action based on the adjusted characteristic amounts.

[0340] The extraction unit is, for example, the above-mentioned trajectory generation unit 302F. (6)

[0342] The work support system according to any one of (1) to (5), comprising a display unit configured to display an image representing a shape of the work object based on the data acquired by the acquisition unit,

[0343] The display unit displays the trajectory generated by the first generating unit in a superimposed manner on an image of the shape of the work object.

[0344] The display unit is, for example, the display device 50A or the display device of the remote operation support device 300 . (7)

[0346] The work support system according to (6), wherein:

[0347] The display unit displays a dynamic image of a movement of the working part along the trajectory generated by the first generating unit in a superimposed manner on an image representing a shape of the working object. (8)

[0349] The work support system according to (6) or (7), wherein:

[0350] The display unit displays an image showing a shape of the work object predicted after the work site moves on the trajectory generated by the first generation unit. (9)

[0352] The support system according to any one of (1) to (8), comprising:

[0353] An input unit for receiving input from a user; and

[0354] The control unit automatically operates the construction machine based on the trajectory generated by the first generation unit in response to a predetermined input from the input unit.

[0355] The control unit is, for example, the above-mentioned action control unit 302G. (10)

[0357] An information processing device comprises a first generating unit which generates a trajectory of a working part of the construction machine based on a composite action formed by combining multiple actions of the construction machine, based on data acquired by an acquiring unit which acquires data related to the shape of a working object around the construction machine.

[0358] The information processing device is, for example, the controller 30 , the information processing device 200 , or the remote operation support device 300 described above. (11)

[0360] A construction machine comprising:

[0361] an acquisition unit, which acquires data related to a shape of a work object around the construction machine,

[0362] The first generating unit generates a trajectory of a working location of the construction machine based on a composite motion formed by combining a plurality of motions of the construction machine, based on the data acquired by the acquiring unit. (12)

[0364] A program that causes an information processing device to execute the following first generation step:

[0365] A trajectory of a working portion of the construction machine based on a composite motion combining a plurality of motions of the construction machine is generated based on data acquired by an acquisition unit that acquires data on a shape of a working object around the construction machine.

[0366] As mentioned above, although embodiment is described in detail, this invention is not limited to this specific embodiment, Various deformation|transformation and change are possible within the range of the summary described in a claim.

[0367] Finally, this application claims priority based on Japanese Patent Application No. 2022-127437 filed on August 9, 2022. The entire contents of the Japanese application are incorporated herein by reference.

[0368] Explanation of symbols

[0369] 1-Lower walking body, 1C, 1CL, 1CR-Crawler, 3-Upper rotating body, 4-Boom, 5-Arm, 6-Bucket, 10-Cockpit, 26-Operating device, 30-Controller, 31-Hydraulic control valve, 40-Camera, 40B, 40F, 40L, 40R-Camera, 50-Output device, 50A-Display device, 52-Input device, 60-Communication device, 100-Excavator, 150-Support device, 200-Information processing device , 300-remote operation support device, 301-action log providing unit, 301A-action log recording unit, 301B-action log storage unit, 301C-action log sending unit, 302-job support unit, 302A-model storage unit, 302B-job object shape acquisition unit, 302C-job selection unit, 302D-track generation unit, 302E-specification setting unit, 302F-track generation unit, 302G-action control unit, 302H-display processing 1100-track, 1300-setting screen, 1301, 1301A to 1301C-image, 1302, 1302A to 1302C-operation image, 1303-operation image, 1400-setting screen, 1401, 1401A to 1401C-image, 1402, 1402A to 1402D-operation image, 1403-operation image, 2001-action log acquisition unit, 2002-action log storage unit, 2 003, 2003A~2003C-training data generation unit, 2004, 2004A~2004C-machine learning unit, 2005-model storage unit, 2006-transmission unit, A1~A4-network parameters, AT-accessory equipment, B1~B4-network parameters, F_A, F_B-feature quantities, FV, FV1~FV4 feature quantities, LM1~LM3 models, M1~M3-learning models, S1~S5-sensors, SYS-operation support system.

Claims

1. A support device comprising: An input unit for receiving input from a user; and The display unit determines specifications related to the combination of multiple different types of actions of the construction machine based on the input from the input unit, and displays a first operation screen, wherein the first operation screen is used to generate a trajectory of the working part of the construction machine based on a composite action formed by combining the multiple actions.

2. The support device according to claim 1, wherein: In the first operation screen, the user can perform an operation to determine the specification through the input unit.

3. The support device according to claim 2, wherein: The plurality of actions are actions that the construction machine performs using the construction attachments in a predetermined operation.

4. The support device according to claim 3, wherein: The prescribed operation is leveling operation, excavation operation or slope construction operation, When the prescribed operation is a leveling operation, the multiple actions include at least two of a horizontal pulling action, a rolling action, a cleaning action, an excavating action, and a soil discharge action. When the prescribed operation is an excavating operation, the multiple actions include at least two of an excavating action, a boom raising and rotating action, a boom lowering and rotating action, a soil discharge action, and a cleaning action. When the prescribed operation is a slope construction operation, the multiple actions include a soil cutting action and a rolling action.

5. The support device according to any one of claims 1 to 4, wherein: The track of the working part is a track set at a predetermined part of the end attachment of the front end of the construction attachment of the construction machine.

6. The support device according to any one of claims 1 to 4, wherein: The specification includes an allocation of a combination of the plurality of actions in the composite action.

7. The support device according to any one of claims 1 to 4, wherein: The display unit displays an image showing a shape of a work object around the construction machine, and displays a track generated through the first operation screen in a superimposed manner on the image showing the shape of the work object around the construction machine.

8. The support device according to claim 7, wherein: The image representing the shape of the work object is a captured image of the periphery of the construction machine or a processed image thereof, or an image of three-dimensional data of the work object in the periphery of the construction machine.

9. The support device according to claim 7, wherein: The display unit displays a dynamic image of a simulation of the construction machine in which the work site moves along a track generated by the first operation screen, superimposed on the image representing the shape of the work object.

10. The support device according to claim 7, wherein: The display unit displays an image showing a shape of the work object predicted after the work site moves on the trajectory generated by the first operation screen.

11. The support device according to any one of claims 1 to 4, wherein: The display unit displays a second operation screen for causing the construction machine to operate so that the work site moves along a track generated by the first operation screen, based on the input from the input unit.

12. The support device according to claim 11, wherein: In the second operation screen, the user can use the input unit to perform an operation for issuing an instruction to operate the construction machine so that the work site moves along the trajectory generated by the first operation screen.

13. A construction machine comprising: An input unit for receiving input from a user; and The display unit determines specifications related to the combination of multiple different types of actions of the construction machine based on the input from the input unit, and displays a first operation screen, wherein the first operation screen is used to generate a trajectory of the working part of the construction machine based on a composite action formed by combining the multiple actions.

14. A support system comprising a construction machine and a support device capable of communicating with the construction machine, wherein: The supporting device comprises: An input unit for receiving input from a user; and The display unit determines specifications related to the combination of multiple actions of different types of the construction machine based on the input from the input unit, and displays a first operation screen, wherein the first operation screen is used to generate a track of the working part of the construction machine based on a composite action formed by combining the multiple actions.

15. A program that causes an information processing device having an input unit and a display unit to perform the following processing: Specifications related to a combination of multiple actions of different types of the construction machine are determined based on input from the input unit, and the display unit is caused to display a first operation screen for generating a trajectory of the working part of the construction machine based on a composite action formed by combining the multiple actions.

Images