Assistance device, work machine, and program

Abstract

Provided is a technology for enabling data relating to an intended shape of a processing target to be acquired more easily. An assistance device 150 according to one embodiment of the present disclosure comprises a topographic shape acquisition unit 301 that acquires image data relating to the shape of a processed portion of a processing target, an intended shape estimation unit 302 that estimates the intended shape of the processing target on the basis of the image data acquired by the topographic shape acquisition unit 301, and an intended shape correction unit 304 that corrects the intended shape of the processing target included in the result of estimation by the intended shape estimation unit 302 in response to input by a user.

Description

Support devices, work machines, and programs

[0001] The present disclosure relates to an assistance device for a work machine.

[0002] BACKGROUND ART Techniques relating to machine guidance and machine control of work machines are known, which use data on the target shape of a work object to assist an operator in operation or to perform work automatically (see, for example, Patent Document 1).

[0003] JP 2016-084663 A

[0004] However, machine guidance and machine control technologies require the preparation of target shape data for the workpiece in advance and the manual input of target shape parameters by an operator, etc. This can make them difficult to implement at small-scale work sites, for example, and can be time-consuming for operators.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a technology that makes it possible to more easily acquire data regarding the target shape of a construction target.

[0006] In order to achieve the above object, one embodiment of the present disclosure provides an assistance device including: an acquisition unit that acquires data regarding the shape of a portion of a construction target that has already been constructed; and an estimation unit that estimates a target shape of the construction target based on the data acquired by the acquisition unit.

[0007] In another embodiment of the present disclosure, a work machine is provided that includes: an acquisition unit that acquires data relating to the shape of a portion of a construction target that has already been constructed; and an estimation unit that estimates a target shape of the construction target based on the data acquired by the acquisition unit.

[0008] In yet another embodiment of the present disclosure, a program is provided that causes an assistance device to execute an acquisition step of acquiring data regarding the shape of a portion of a construction target that has already been constructed, and an estimation step of estimating a target shape of the construction target based on the data acquired in the acquisition step.

[0009] According to the above-described embodiment, data relating to the target shape of the construction target can be acquired more easily.

[0010] 1 is a side view showing an example of a shovel. FIG. 2 is a top view showing an example of a shovel. FIG. 3 is a diagram showing an example of a configuration related to remote operation of a shovel. FIG. 4 is a block diagram showing an example of a hardware configuration of a shovel. FIG. 5 is a diagram showing an example of a method for estimating a target shape of a construction target in slope work. FIG. 6 is a diagram showing an example of a method for estimating a target shape of a construction target in bed excavation work. FIG. 7 is a functional block diagram showing a first example of a functional configuration related to estimating the target shape of a construction target. FIG. 8 is a flowchart showing a first example of a process related to estimating the target shape of a construction target. FIG. 9 is a diagram showing an example of the display content of a display device showing the topographical shape around a shovel. FIG. 10 is a diagram showing an example of the display content of a display device showing an estimation result of the target shape of a construction target. FIG. 11 is a diagram showing another example of the display content of a display device showing an estimation result of the target shape of a construction target. FIG. 12 is a diagram showing an example of an operation support system. FIG. 13 is a diagram showing an example of the hardware configuration of an information processing device. FIG. 14 is a functional block diagram showing a second example of a functional configuration related to estimating the target shape of a construction target. FIG. 15 is a flowchart showing a first example of a process related to estimating the target shape of a construction target.

[0011] Hereinafter, an embodiment will be described with reference to the drawings.

[0012] [Outline of Shovel] First, an outline of a shovel 100 according to this embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0013] 1 and 2 are side views showing an example of a shovel 100. FIG. 2 is a top view showing an example of the shovel 100. FIG. 3 is a diagram showing an example of a configuration related to remote control of a shovel. Hereinafter, the direction in which the attachment AT extends when viewed from above the shovel 100 (the upward direction in FIG. 2) will be defined as "front," and directions in the shovel 100 or directions seen from the shovel 100 may be described.

[0014] As shown in FIGS. 1 and 2 , the excavator 100 includes a lower traveling body 1 , an upper rotating body 3 , an attachment AT including a boom 4 , an arm 5 , and a bucket 6 , and a cabin 10 .

[0015] The undercarriage 1 uses crawlers 1C to travel the excavator 100. The crawlers 1C include a left crawler 1CL and a right crawler 1CR. The crawler 1CL is hydraulically driven by a traveling hydraulic motor 1ML. Similarly, the crawler 1CL is hydraulically driven by a traveling hydraulic motor 1MR. This allows the undercarriage 1 to travel independently.

[0016] The upper rotating body 3 is rotatably mounted on the lower traveling body 1 via a rotating mechanism 2. For example, the upper rotating body 3 rotates relative to the lower traveling body 1 when the rotating mechanism 2 is hydraulically driven by a rotating hydraulic motor 2M.

[0017] The boom 4 is attached to the front center of the upper rotating body 3 so as to be able to tilt up and down about a rotation axis that extends in the left-right direction. The arm 5 is attached to the tip of the boom 4 so as to be able to rotate about a rotation axis that also extends in the left-right direction. The bucket 6 is attached to the tip of the arm 5 so as to be able to rotate about a rotation axis that also extends in the left-right direction.

[0018] The bucket 6 is an example of an end attachment and is used, for example, for excavation work.

[0019] The bucket 6 is attached to the tip of the arm 5 in a manner that allows it to be appropriately replaced depending on the work content of the shovel 100. In other words, instead of the bucket 6, a bucket of a different type from the bucket 6, such as a relatively large bucket, a slope bucket, or a dredging bucket, may be attached to the tip of the arm 5. Also, a type of end attachment other than a bucket, such as a mixer, breaker, or crusher, may be attached to the tip of the arm 5. Also, a spare attachment such as a quick coupling or a tiltrotator may be provided between the arm 5 and the end attachment.

[0020] The boom 4, arm 5, and bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0021] The cabin 10 is a control room where an operator sits and operates the excavator 100. The cabin 10 is mounted on the front left side of the upper rotating body 3, for example.

[0022] For example, the excavator 100 operates driven elements such as the lower running body 1 (i.e., a pair of left and right crawlers 1CL, 1CR), upper rotating body 3, boom 4, arm 5, and bucket 6 in response to operations by an operator seated in the cabin 10.

[0023] Furthermore, instead of or in addition to being configured to be operable by an operator inside the cabin 10, the shovel 100 may be configured to be remotely operable from outside the shovel 100. When the shovel 100 is remotely operated, the inside of the cabin 10 may be unmanned. The following description will be given on the assumption that the operation of the operator includes at least one of operation of the operating device 26 by the operator inside the cabin 10 and remote operation by an external operator.

[0024] For example, as shown in FIG. 3 , remote operation includes a mode in which the shovel 100 is operated by operation input related to the actuator of the shovel 100 performed by the remote operation support device 300 .

[0025] The remote operation support device 300 is provided, for example, in a management center or the like that externally manages the work of the shovel 100. The remote operation support device 300 may also be a portable operation terminal, in which case the operator can remotely operate the shovel 100 while directly checking the work status of the shovel 100 from the vicinity of the shovel 100.

[0026] The shovel 100 may transmit to the remote operation support device 300, for example, via a communication device 60 described below, an image (hereinafter referred to as a "peripheral image") showing the surroundings including the area in front of the shovel 100, based on an image captured by an imaging device 40 described below. The remote operation support device 300 may then display the image (peripheral image) received from the shovel 100 on a display device. Furthermore, various information images (information screens) displayed on an output device 50 (display device) inside the cabin 10 of the shovel 100 may also be displayed on the display device of the remote operation support device 300. This allows the operator using the remote operation support device 300 to remotely operate the shovel 100 while checking, for example, the contents of an image, information screen, or the like showing the surroundings of the shovel 100 displayed on the display device. The excavator 100 may operate actuators and drive driven elements such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6 in accordance with a remote control signal indicating the content of the remote control, which is received by the communication device 60 from the remote control support device 300.

[0027] Furthermore, remote control may include, for example, a mode in which the shovel 100 is operated by an external voice input, gesture input, or the like to the shovel 100 by a person (e.g., a worker) around the shovel 100. Specifically, the shovel 100 recognizes voices uttered by surrounding workers and gestures made by the workers through a voice input device (e.g., a microphone) or a gesture input device (e.g., an imaging device) mounted on the shovel 100. Then, the shovel 100 may operate actuators in accordance with the content of the recognized voices, gestures, or the like to drive driven elements such as the lower traveling body 1 (left and right crawlers 1C), upper rotating body 3, boom 4, arm 5, and bucket 6.

[0028] The excavator 100 may also automatically operate the actuators regardless of the operation by the operator. This allows the excavator 100 to realize a function ("automatic driving function" or "MC (Machine Control) function") that automatically operates at least some of the driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6.

[0029] The automatic driving function includes, for example, a function ("semi-automatic driving function" or "operation-assisted MC function") that automatically operates driven elements (actuators) other than the driven element (actuator) that is the target of operation in response to an operator's operation of the operating device 26 or remote operation. The automatic driving function may also include a function ("fully automatic driving function" or "fully automatic MC function") that automatically operates at least some of the multiple driven elements (actuators) without the operator's operation of the operating device 26 or remote operation. When the fully automatic driving function is enabled in the excavator 100, the interior of the cabin 10 may be unmanned. The semi-automatic driving function, the fully automatic driving function, etc. may also include a mode in which the operation content of the driven element (actuator) that is the target of automatic driving is automatically determined according to predetermined rules. The semi-automatic driving function, the fully automatic driving function, etc. may also include a mode ("autonomous driving function") in which the excavator 100 autonomously makes various decisions and autonomously determines the operation content of the driven element (actuator) that is the target of automatic driving based on the decision results.

[0030] Furthermore, the work of the shovel 100 may be remotely monitored. In this case, a remote monitoring support device having the same functions as the remote operation support device 300 may be provided. The remote monitoring support device is, for example, the information processing device 200 described below. This allows a monitor, who is a user of the remote monitoring support device, to monitor the status of the work of the shovel 100 while checking a peripheral image displayed on a display device of the remote monitoring support device. Furthermore, for example, if the monitor determines it is necessary from a safety standpoint, the monitor can intervene in the operation by the operator of the shovel 100 and cause an emergency stop by making a predetermined input using an input device of the remote monitoring support device.

[0031] [Hardware Configuration of Shovel] Next, the hardware configuration of the shovel 100 will be described with reference to FIG.

[0032] FIG. 4 is a block diagram showing an example of the hardware configuration of the shovel 100.

[0033] In FIG. 4, the paths through which mechanical power is transmitted are indicated by double lines, the paths through which high-pressure hydraulic oil that drives the hydraulic actuator flows are indicated by solid lines, the paths through which pilot pressure is transmitted are indicated by dashed lines, and the paths through which electrical signals are transmitted are indicated by dotted lines.

[0034] The shovel 100 includes various components, such as a hydraulic drive system for hydraulically driving the driven elements, an operation system for operating the driven elements, a user interface system for exchanging information with the user, a communication system for communicating with the outside world, and a control system for various controls.

[0035] 4, the hydraulic drive system of the excavator 100 includes hydraulic actuators HA that hydraulically drive each of the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, and the attachment AT, as described above. The hydraulic drive system of the excavator 100 according to this embodiment also includes an engine 11, a regulator 13, a main pump 14, and a control valve 17.

[0036] The hydraulic actuator HA includes traveling hydraulic motors 1ML, 1MR, a swing hydraulic motor 2M, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and the like.

[0037] Note that the hydraulic actuators HA of the shovel 100 may be partially or entirely replaced with electric actuators. In other words, the shovel 100 may be a hybrid shovel or an electric shovel.

[0038] The engine 11 is the prime mover of the excavator 100 and the main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, on the rear of the upper rotating body 3. The engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15.

[0039] It should be noted that instead of or in addition to the engine 11, another prime mover (for example, an electric motor) may be mounted on the excavator 100.

[0040] The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter referred to as the "tilt angle") in response to a control command from the controller 30.

[0041] The main pump 14 supplies hydraulic oil to the control valve 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13, thereby adjusting the stroke length of the piston and controlling the discharge flow rate and discharge pressure.

[0042] The control valve 17 drives the hydraulic actuators HA in response to an operator's operation of the operating device 26, the details of remote operation, or an operation command corresponding to the automatic operation function. The control valve 17 is mounted, for example, in the center of the upper rotating body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line, and selectively supplies hydraulic oil supplied from the main pump 14 to each hydraulic actuator in response to an operator's operation or an operation command corresponding to the automatic operation function. Specifically, the control valve 17 includes a plurality of control valves (also referred to as "directional control valves") that control the flow rate and flow direction of hydraulic oil supplied from the main pump 14 to each hydraulic actuator HA.

[0043] <Operation System> As shown in FIG. 4 , the operation system of the excavator 100 includes the pilot pump 15 , the operation device 26 , the hydraulic control valve 31 , the shuttle valve 32 , and the hydraulic control valve 33 .

[0044] The pilot pump 15 supplies pilot pressure to various hydraulic devices via a pilot line 25. The pilot pump 15 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11. The pilot pump 15 is, for example, a fixed displacement hydraulic pump, and is driven by the engine 11 as described above.

[0045] The pilot pump 15 may be omitted. In this case, the relatively high-pressure hydraulic oil discharged from the main pump 14 may be reduced in pressure by a predetermined pressure reducing valve, and the resulting relatively low-pressure hydraulic oil may be supplied to various hydraulic devices as pilot pressure.

[0046] The operating device 26 is provided near the cockpit of the cabin 10 and is used by the operator to operate the various driven elements. Specifically, the operating device 26 is used by the operator to operate the hydraulic actuators HA that drive the respective driven elements, thereby enabling the operator to operate the driven elements that are driven by the hydraulic actuators HA. The operating device 26 includes pedal devices and lever devices for operating the respective driven elements (hydraulic actuators HA).

[0047] For example, as shown in FIG. 4 , the operating device 26 is of a hydraulic pilot type. Specifically, the operating device 26 uses hydraulic oil supplied from the pilot pump 15 through a pilot line 25 and a pilot line 25A branching from the pilot line 25, and outputs a pilot pressure corresponding to the operation to a secondary pilot line 27A. The pilot line 27A is connected to one inlet port of a shuttle valve 32 and is connected to the control valve 17 via a pilot line 27 connected to the outlet port of the shuttle valve 32. This allows pilot pressure corresponding to the operation of various driven elements (hydraulic actuators HA) in the operating device 26 to be input to the control valve 17 via the shuttle valve 32. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation of the operating device 26 by an operator or the like.

[0048] Alternatively, the operating device 26 may be electric. In this case, the pilot line 27A, the shuttle valve 32, and the hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electric signal (hereinafter referred to as an "operation signal") corresponding to the operation content, and the operation signal is input to the controller 30. The controller 30 then outputs a control command corresponding to the operation signal, i.e., a control signal corresponding to the operation content of the operating device 26, to the hydraulic control valve 31. As a result, a pilot pressure corresponding to the operation content of the operating device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26.

[0049] Furthermore, the control valves (directional control valves) that are built into the control valve 17 and drive the hydraulic actuators HA may be of an electromagnetic solenoid type. In this case, the operation signal output from the operating device 26 may be input directly to the control valve 17, i.e., the electromagnetic solenoid type control valve.

[0050] As described above, some or all of the hydraulic actuators HA may be replaced with electric actuators. In this case, the controller 30 may output control commands to the electric actuators or a driver that drives the electric actuators, depending on the operation content of the operation device 26 and the remote operation content specified by the remote operation signal. Furthermore, when the excavator 100 is remotely operated, the operation device 26 may be omitted.

[0051] A hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) operated by the operating device 26 and for each drive direction (e.g., the raising and lowering directions of the boom 4) of the driven element (hydraulic actuator HA). In other words, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA. The hydraulic control valve 31 may be provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17 and configured to be able to change its flow path area (i.e., the cross-sectional area through which hydraulic oil can flow). This allows the hydraulic control valve 31 to output a predetermined pilot pressure to the secondary pilot line 27B using hydraulic oil from the pilot pump 15 supplied through the pilot line 25B. Therefore, the hydraulic control valve 31 can indirectly apply a predetermined pilot pressure to the control valve 17 in response to a control signal from the controller 30 via a shuttle valve 32 between the pilot line 27B and the pilot line 27B. Therefore, the controller 30 can supply pilot pressure according to the operation command corresponding to the automatic driving function from the hydraulic control valve 31 to the control valve 17, thereby realizing operation of the excavator 100 using the automatic driving function.

[0052] Furthermore, the controller 30 may, for example, control the hydraulic control valve 31 to realize remote operation of the shovel 100. Specifically, the controller 30 outputs a control signal corresponding to the content of remote operation specified in a remote operation signal received from the remote operation assistance device 300 to the hydraulic control valve 31 via the communication device 60. As a result, the controller 30 causes the hydraulic control valve 31 to supply a pilot pressure corresponding to the content of remote operation to the control valve 17, thereby realizing operation of the shovel 100 based on remote operation by the operator.

[0053] In addition, if the operating device 26 is electric, the controller 30 can supply pilot pressure corresponding to the operation content (operation signal) of the operating device 26 directly to the control valve 17 from the hydraulic control valve 31, thereby realizing operation of the shovel 100 based on the operation of the operator.

[0054] The shuttle valve 32 has two inlet ports and one outlet port, and outputs hydraulic fluid having a higher pilot pressure of the two pilot pressures input to the two inlet ports to the outlet port. A shuttle valve 32 is provided for each driven element (hydraulic actuator HA) operated by the operating device 26 and for each drive direction of the driven element (hydraulic actuator HA). One of the two inlet ports of the shuttle valve 32 is connected to a secondary pilot line 27A of the operating device 26 (specifically, the lever device or pedal device included in the operating device 26), and the other is connected to a secondary pilot line 27B of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of a corresponding control valve of the control valve 17 via the pilot line 27. The corresponding control valve is a control valve that drives the hydraulic actuator that is operated by the lever device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, each of these shuttle valves 32 can apply the higher of the pilot pressure in the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure in the pilot line 27B on the secondary side of the hydraulic control valve 31 to the pilot port of the corresponding control valve. In other words, by outputting a pilot pressure higher than the pilot pressure on the secondary side of the operating device 26 from the hydraulic control valve 31, the controller 30 can control the corresponding control valve regardless of the operation of the operating device 26 by the operator. Therefore, the controller 30 can control the operation of the driven elements (undercarriage 1, upper revolving body 3, and attachment AT) regardless of the operating state of the operating device 26 by the operator, thereby realizing a remote operation function and an automatic driving function.

[0055] The hydraulic control valve 33 is provided in a pilot line 27A connecting the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured, for example, to be able to change its flow path area. The hydraulic control valve 33 operates in response to a control signal input from the controller 30. As a result, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is operated by an operator. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. Furthermore, for example, even when the operating device 26 is being operated, the controller 30 can reduce the pilot pressure output from the operating device 26 to make it lower than the pilot pressure output from the hydraulic control valve 31. Therefore, by controlling the hydraulic control valves 31 and 33, the controller 30 can reliably apply a desired pilot pressure to the pilot port of the control valve in the control valve 17, for example, regardless of the operation of the operating device 26. Therefore, the controller 30 can more appropriately realize the remote control function and automatic operation function of the excavator 100 by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31, for example.

[0056] <User Interface System> As shown in FIG. 4 , the user interface system of the shovel 100 includes the operation device 26 , the output device 50 , and the input device 52 .

[0057] The output device 50 outputs various information to the user of the shovel 100 (e.g., the operator of the cabin 10 or an external remote operator) and people in the vicinity of the shovel 100 (e.g., workers or drivers of work vehicles).

[0058] For example, the output device 50 includes a lighting device or a display device 50A (see FIG. 5 ) that outputs various types of information visually. The lighting device is, for example, a warning light (indicator lamp). The display device 50A is, for example, a liquid crystal display or an organic EL (electroluminescence) display. For example, as shown in FIG. 2 , the lighting device or the display device 50A may be provided inside the cabin 10 and output various types of information visually to an operator or the like inside the cabin 10. Furthermore, the lighting device or the display device 50A may be provided, for example, on the side of the upper rotating body 3 and output various types of information visually to workers or the like around the excavator 100.

[0059] Furthermore, for example, the output device 50 includes a sound output device that outputs various types of information by an auditory means. Examples of sound output devices include a buzzer, a speaker, etc. The sound output device may be provided, for example, inside or outside the cabin 10, and may output various types of information by an auditory means to an operator inside the cabin 10 or to people (workers, etc.) around the excavator 100.

[0060] Furthermore, for example, the output device 50 may include a device that outputs various types of information in a tactile manner, such as by vibrating the cockpit.

[0061] The input device 52 receives various inputs from the user of the excavator 100, and signals corresponding to the received inputs are taken into the controller 30. The input device 52 is provided, for example, inside the cabin 10, and receives inputs from an operator or the like inside the cabin 10. The input device 52 may also be provided, for example, on the side of the upper rotating body 3, and receive inputs from workers or the like around the excavator 100.

[0062] For example, the input device 52 includes an operation input device that accepts input from a user through mechanical operation. The operation input device may include a touch panel mounted on the display device, a touch pad installed around the display device, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), and the like.

[0063] Furthermore, for example, the input device 52 may include an audio input device that accepts audio input from the user. The audio input device includes, for example, a microphone.

[0064] Furthermore, for example, the input device 52 may include a gesture input device that accepts gesture inputs from the user. The gesture input device includes, for example, an imaging device that captures an image of a gesture made by the user.

[0065] Furthermore, for example, the input device 52 may include a biometric input device that accepts biometric input from the user. The biometric input includes input of biometric information such as the user's fingerprint or iris.

[0066] <Communication System> As shown in FIG. 4 , the communication system of the shovel 100 according to this embodiment includes a communication device 60 .

[0067] The communication device 60 connects to an external communication line and communicates with devices provided separately from the shovel 100. The devices provided separately from the shovel 100 may include devices external to the shovel 100 as well as portable terminal devices (mobile terminals) brought into the cabin 10 by the user of the shovel 100. The communication device 60 may be, for example, a 4G (4 th Generation) and 5G (5 th The communication device 60 may include a mobile communication module conforming to standards such as the Universal Serial Bus (UART) Standard (UART) Generation. The communication device 60 may also include, for example, a satellite communication module. The communication device 60 may also include, for example, a Wi-Fi communication module or a Bluetooth (registered trademark) communication module. The communication device 60 may also include multiple communication devices in accordance with the communication lines to be connected.

[0068] For example, the communication device 60 communicates with external devices such as the information processing device 200 and the remote operation support device 300 at the work site through a local communication line established at the work site. The local communication line is, for example, a local 5G (so-called local 5G) mobile communication line established at the work site or a local network (LAN: Local Area Network) using Wi-Fi 6.

[0069] Furthermore, for example, the communication device 60 communicates with the information processing device 200 and the remote operation support device 300 outside the work site through a wide area communication line that includes the work site, i.e., a wide area network (WAN). The wide area network includes, for example, a wide area mobile communication network, a satellite communication network, the Internet network, etc.

[0070] 4, the control system of the shovel 100 includes a controller 30. The control system of the shovel 100 according to this embodiment also includes an operating pressure sensor 29, an imaging device 40, and sensors S1 to S5.

[0071] The controller 30 performs various controls related to the shovel 100 .

[0072] The functions of the controller 30 may be realized by any hardware or any combination of hardware and software, etc. For example, as shown in Fig. 4, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D, which are connected by a bus B1.

[0073] The auxiliary storage device 30A is a non-volatile storage means that stores the programs to be installed as well as necessary files, data, etc. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory.

[0074] For example, when a program start instruction is received, the memory device 30B loads the program from the auxiliary storage device 30A so that it can be read by the CPU 30C. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

[0075] The CPU 30C executes, for example, a program loaded into the memory device 30B, and realizes various functions of the controller 30 according to instructions of the program.

[0076] The interface device 30D functions as, for example, a communication interface for connecting to a communication line inside the shovel 100. The interface device 30D may include a plurality of different types of communication interfaces in accordance with the types of communication lines to be connected.

[0077] The interface device 30D also functions as an external interface for reading data from a recording medium and writing data to the recording medium. The recording medium is, for example, a dedicated tool connected to a connector installed inside the cabin 10 via a detachable cable. The recording medium may also be a general-purpose recording medium, such as an SD memory card or a USB (Universal Serial Bus) memory. As a result, a program that realizes various functions of the controller 30 can be provided, for example, by a portable recording medium and installed in the auxiliary storage device 30A of the controller 30. The program may also be downloaded from another computer external to the excavator 100 via the communication device 60 and installed in the auxiliary storage device 30A.

[0078] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

[0079] The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot type operating device 26, i.e., the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. A detection signal of the pilot pressure by the operating pressure sensor 29 corresponding to the operating state of each driven element (hydraulic actuator HA) in the operating device 26 is input to the controller 30.

[0080] If the operating device 26 is an electric type, the operating pressure sensor 29 is omitted because the controller 30 can grasp the operating state of each driven element through the operating device 26 based on the operating signal received from the operating device 26.

[0081] The imaging device 40 acquires an image of the periphery of the shovel 100. Furthermore, the imaging device 40 may acquire (generate) three-dimensional data (hereinafter simply referred to as "three-dimensional data of objects") representing the positions and outer shapes of objects around the shovel 100 within the imaging range (angle of view) based on the acquired image and data related to distances, which will be described later. The three-dimensional data of objects around the shovel 100 is, for example, data of coordinate information of a point cloud representing the surface of the object, distance image data, etc.

[0082] 2 , the imaging device 40 includes a camera 40F that images the area in front of the upper rotating body 3, a camera 40B that images the area behind the upper rotating body 3, a camera 40L that images the area to the left of the upper rotating body 3, and a camera 40R that images the area to the right of the upper rotating body 3. As a result, the imaging device 40 can capture images of the entire circumference around the shovel 100, i.e., a range spanning a 360-degree angular direction, when viewed from above. Furthermore, the operator can visually recognize images captured by the cameras 40B, 40L, and 40R and peripheral images such as processing images generated based on the captured images via the output device 50 (display device 50A) and the remote operation display device, and can check the conditions to the left, right, and rear of the upper rotating body 3. Furthermore, by visually recognizing images captured by the camera 40F and peripheral images such as processing images generated based on the captured images via the remote operation display device, the operator can remotely operate the shovel 100 while checking the operation of the attachment AT including the bucket 6. Hereinafter, the cameras 40F, 40B, 40L, and 40R may be collectively or individually referred to as "camera 40X."

[0083] The camera 40X may be, for example, a monocular camera. Alternatively, the camera 40X may be, for example, a stereo camera, a TOF (Time Of Flight) camera, or the like (hereinafter collectively referred to as a "3D camera") that can acquire data related to distance (depth) in addition to two-dimensional images.

[0084] Output data (e.g., image data and three-dimensional data of objects around the shovel 100) from the imaging device 40 (camera 40X) is input to the controller 30 via a one-to-one communication line or an in-vehicle network. This allows the controller 30 to, for example, monitor objects around the shovel 100 based on the output data from the camera 40X. Also, for example, the controller 30 can determine the surrounding environment of the shovel 100 based on the output data from the camera 40X. Also, for example, the controller 30 can determine the attitude state of the attachment AT shown in the captured image based on the output data from the camera 40X (camera 40F). Also, for example, the controller 30 can determine the attitude state of the body (upper rotating body 3) of the shovel 100 based on the output data from the camera 40X, using objects around the shovel 100 as a reference.

[0085] Note that some of the cameras 40F, 40B, 40L, and 40R may be omitted. For example, if the shovel 100 is not remotely operated, the cameras 40F and 40L may be omitted. This is because the situation in front of and to the left of the shovel 100 is relatively easy to see from the operator in the cabin 10. Furthermore, instead of or in addition to the imaging device 40 (camera 40X), a distance sensor may be provided on the upper rotating body 3. The distance sensor is attached, for example, to the upper part of the upper rotating body 3 and acquires data regarding the distance and direction of surrounding objects relative to the shovel 100. Furthermore, the distance sensor may acquire (generate) three-dimensional data (e.g., point cloud coordinate information data) of objects around the shovel 100 within its sensing range based on the acquired data. The distance sensor is, for example, a light detection and ranging (LIDAR). Furthermore, for example, the distance sensor may be, for example, a millimeter-wave radar, an ultrasonic sensor, an infrared sensor, or the like.

[0086] The sensor S1 is attached to the boom 4 and detects the attitude angle (hereinafter referred to as the "boom angle") around the rotation axis of the base end of the boom 4, which corresponds to the connection part between the boom 4 and the upper rotating body 3. The sensor S1 includes, for example, a rotary potentiometer, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), etc. The same may be true for the sensors S2 and S4 below. The sensor S1 may also include a cylinder sensor that detects the extension / retraction position of the boom cylinder 7. The same may be true for the sensor S2 below. The boom angle detection signal from the sensor S1 is input to the controller 30. This allows the controller 30 to grasp the attitude state of the boom 4.

[0087] The sensor S2 is attached to the arm 5 and detects the attitude angle (hereinafter referred to as "arm angle") around the rotation axis of the base end of the arm 5, which corresponds to the connection part between the arm 5 and the boom 4. The detection signal of the arm angle by the sensor S2 is input to the controller 30. This enables the controller 30 to grasp the attitude state of the arm 5.

[0088] Sensor S3 is attached to bucket 6 and detects the attitude angle (hereinafter referred to as "arm angle") around the rotation axis of the base end of bucket 6, which corresponds to the connection part between bucket 6 and arm 5. The arm angle detection signal from sensor S3 is input to controller 30. This enables controller 30 to grasp the attitude state of bucket 6.

[0089] The sensor S4 detects the inclination state of the machine body (e.g., the upper rotating body 3) with respect to a predetermined reference plane (e.g., a horizontal plane). The sensor S4 is attached to the upper rotating body 3, for example, and detects the inclination angles of the excavator 100 (i.e., the upper rotating body 3) about two axes in the front-to-rear and left-to-right directions (hereinafter referred to as the "front-to-rear inclination angle" and the "left-to-right inclination angle"). The detection signals corresponding to the inclination angles (front-to-rear inclination angle and left-to-right inclination angle) detected by the sensor S4 are input to the controller 30. This allows the controller 30 to grasp the inclination state of the machine body (upper rotating body 3).

[0090] The sensor S5 is attached to the upper rotating body 3 and outputs detection information related to the rotation state of the upper rotating body 3. The sensor S5 detects, for example, the rotation angular velocity and rotation angle of the upper rotating body 3. The sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, etc. The detection information related to the rotation state detected by the sensor S5 is input into the controller 30. This allows the controller 30 to grasp the rotation state of the upper rotating body 3, such as the rotation angle.

[0091] If sensor S4 includes a gyro sensor, a six-axis sensor, an IMU, or the like that can detect angular velocity around three axes, the rotation state (e.g., rotation angular velocity) of the upper rotating body 3 may be detected based on the detection signal of sensor S4. In this case, sensor S5 may be omitted. Also, if it is possible to grasp the posture state of the upper rotating body 3, the attachment AT, etc. based on the output of the imaging device 40 or a distance sensor, at least some of sensors S1 to S5 may be omitted.

[0092] [Outline of Target Shape Estimation Function] Next, with reference to FIGS. 5 and 6 in addition to FIGS. 1 to 4, an outline of the function of the shovel 100 related to estimating the target shape of the work target (hereinafter, for convenience, referred to as the "target shape estimation function") will be described.

[0093] Fig. 5 is a diagram schematically showing an example of a method for estimating a target shape of a construction target in slope work. Fig. 6 is a diagram schematically showing an example of a method for estimating a target shape of a construction target in bed excavation work.

[0094] The construction target is an object that is formed on the ground or the like by construction work by the shovel 100. Construction work is, for example, slope work, bed excavation work, ground leveling work, etc. The construction target is, for example, a horizontal surface, a slope, a ditch, an embankment, etc.

[0095] The target shape of the construction target is a target shape of the construction target that is expected to be finally realized by the work of the shovel 100. The target shape of the construction target is, for example, a target construction surface formed as a flat surface. The target shape of the construction target may also be a target construction surface formed as a curved surface with a predetermined curvature.

[0096] In this embodiment, the shovel 100 (controller 30) estimates a target shape of the construction target based on the shape of a portion of the construction target that has been completed by operation of the shovel by an expert. The estimated target shape of the construction target may be the target shape of the entire construction target, or may be the target shape of a partial region of the construction target that is larger than the completed portion. The shovel used by the expert for construction work on the construction target may be the shovel 100, or may be another shovel different from the shovel 100. An expert is, for example, an operator who has relatively long experience in operating a shovel and has a relatively high level of experience in operating a shovel.

[0097] For example, in a slope work, a part of the slope to be worked on is formed as a worked area 501, as shown in Fig. 5. As described above, the area 501 is worked on in advance based on the operation of a shovel by a skilled worker.

[0098] First, the shovel 100 (controller 30 ) uses the imaging device 40 to capture an image of the shape of the area 501 , and acquires shape data of the area 501 based on the output of the imaging device 40 .

[0099] Next, the shovel 100 (controller 30) estimates a target construction surface 502 corresponding to the target shape of the construction target based on the shape data of the area 501.

[0100] For example, the controller 30 estimates the target construction surface 502 by extending the planar shape of the area 501 in the width direction of the slope or by duplicating and arranging them in the width direction, and acquires data related to the target construction surface 502. Hereinafter, this estimation method may be referred to as a "first estimation method" of the target shape of the work object.

[0101] The controller 30 may also use a trained model that has been machine-learned using a training data set that is a combination of data on the shape of a portion of the slope of the construction target that has already been constructed and data on the shape of the target construction surface of the slope of the construction target. In this case, the controller 30 applies the shape data of the area 501 as input data to the trained model, thereby estimating the target construction surface 502 based on the output data of the trained model and acquiring data on the target construction surface 502.

[0102] 6, for example, a completed trench 601 is formed as part of a trench to be constructed during a bed excavation operation. The trench 601 is constructed in advance based on the operation of a shovel by a skilled worker, as described above.

[0103] First, the shovel 100 (controller 30 ) uses the imaging device 40 to capture an image of the shape of the trench 601 , and acquires shape data of the trench 601 based on the output of the imaging device 40 .

[0104] Next, the shovel 100 (controller 30) estimates a target construction surface 602 corresponding to the target shape of the trench to be constructed, based on the shape data of the trench 601. The target construction surface 602 includes a target construction surface corresponding to the side surfaces at both ends in the width direction of the trench, a target construction surface corresponding to the side surfaces at both ends in the length direction of the trench, and a target construction surface corresponding to the bottom of the trench.

[0105] For example, the controller 30 estimates the target construction surface 602 using a first estimation method. Specifically, the controller 30 may estimate the target construction surface 602 corresponding to the side surfaces at both ends in the width direction of the groove by extending the shape of the side surfaces at both ends in the width direction of the groove 601 in the length direction of the groove to be constructed, or by duplicating and arranging them in the length direction. The controller 30 may also estimate the target construction surface 602 corresponding to the bottom of the groove by extending the shape of the bottom of the groove 601 in the length direction of the groove to be constructed, or by duplicating and arranging them in the length direction. The controller 30 may also estimate the target construction surface 602 corresponding to the side surfaces at both ends in the length direction of the groove by using the shape of both end faces in the length direction of the groove 601 as is, or by offsetting the position in the length direction of the groove to be constructed.

[0106] The controller 30 may also estimate the target construction surface 602 using a second estimation method. Specifically, the controller 30 may use a trained model that has been machine-learned using a training data set that combines data on the shape of a portion of the trench to be constructed that has already been constructed and data on the target construction surface of the trench to be constructed. In this case, the controller 30 uses the shape data of the trench 601 as input data and applies the trained model, thereby estimating the target construction surface 602 based on the output data of the trained model and acquiring data on the target construction surface 602.

[0107] As described above, in this embodiment, the controller 30 can estimate the target shape of the construction target based on shape data of a portion of the construction target that has already been constructed by an expert shovel operator, and acquire data related to the target shape. This allows the controller 30 to more easily acquire data related to the target shape of the construction target. Therefore, since there is no need to prepare data related to the target shape in advance, there is no need for the user to manually input parameters related to the target shape. Furthermore, even at small-scale construction sites where the system and funds for preparing data related to the target shape in advance are not secured, it is possible to acquire data related to the target shape and employ machine guidance and machine control technologies to improve work efficiency.

[0108] For example, the target shape estimation function is applied to the shovel 100 operated by an operator in the cabin 10 or a remote-controlled operator. In this case, the shovel 100 can assist the operator in operation by using a machine guidance function or an operation-assistance MC function based on data related to the target shape estimated by the target shape estimation function. Furthermore, the target shape estimation function may be applied to the shovel 100 that operates by a fully automatic driving function (fully automatic MC function). In this case, the shovel 100 can automatically perform construction so that the work object assumes the target shape by using the fully automatic driving function based on data related to the target shape estimated by the target shape estimation function.

[0109] [First Example of Functional Configuration Related to Estimation of Target Shape of Construction Object] Next, a first example of a functional configuration related to estimation of a target shape of a construction object will be described with reference to FIG. 7 in addition to FIGS. 1 to 6.

[0110] FIG. 7 is a functional block diagram showing a first example of a functional configuration related to estimation of a target shape of a construction object.

[0111] As shown in FIG. 7 , the excavator 100 includes a support device 150 .

[0112] The support device 150 supports the work of the shovel 100. The support device 150 includes a controller 30, an imaging device 40, a display device 50A, an input device 52, and a communication device 60.

[0113] The controller 30 includes, as functional units, a topographical shape acquisition unit 301 , a target shape estimation unit 302 , a display processing unit 303 , a target shape correction unit 304 , a target shape data storage unit 305 , and a work support control unit 306 .

[0114] The terrain shape acquisition unit 301 acquires data on the terrain shape of the location where the construction target will be formed around the shovel 100, including some completed areas of the construction target, based on the outputs of the imaging device 40 and the distance sensor. The data on the terrain shape includes, for example, both some completed areas of the construction target and areas that have not yet been constructed. The data on the terrain shape may be, for example, image data or three-dimensional data.

[0115] The target shape estimation unit 302 estimates the target shape of the construction object using the first estimation method described above based on the data acquired by the topographical shape acquisition unit 301, and acquires data relating to the target shape of the construction object.

[0116] The display processing unit 303 displays the target shape of the construction target as the estimation result of the target shape estimation unit 302 on the display device 50A. Furthermore, when the excavator 100 is remotely operated or remotely monitored, the display processing unit 303 may transmit image data of the target shape of the construction target as the estimation result of the target shape estimation unit 302 to the remote operation support device 300 or the remote monitoring support device via the communication device 60. This allows the display processing unit 303 to display the target shape of the construction target as the estimation result of the target shape estimation unit 302 on the display device of the remote operation support device 300 or the remote operation support device. This allows the user (operator) to visually check the data related to the target shape of the construction target.

[0117] Furthermore, the display processing unit 303 may display on the display device 50A the target shape of the construction target as the estimation result of the target shape estimation unit 302 and the current terrain shape based on the output of the imaging device 40 so that they can be compared (see FIGS. 10 and 11 ). Similarly, the display processing unit 303 may transmit to the remote operation support device 300 or the remote monitoring support device an image that allows comparison of the target shape of the construction target as the estimation result of the target shape estimation unit 302 and the current terrain shape based on the output of the imaging device 40. This allows the user (operator) to determine whether an appropriate target shape has been generated (estimated) by comparing the current terrain shape with the target shape of the construction target.

[0118] The target shape correction unit 304 corrects data related to the target shape of the construction target, which is the estimation result of the target shape estimation unit 302, in response to a predetermined input from the user (operator). The input from the user is received by the input device 52. Furthermore, when the excavator 100 is remotely operated or remotely monitored, the input from the user is received from the remote operation support device 300 or the remote monitoring support device via the communication device 60. For example, the target shape correction unit 304 corrects data related to the target shape of the construction target with a total of six degrees of freedom in a three-dimensional Cartesian coordinate system, including translational movement along the X-axis, Y-axis, and Z-axis, and rotational movement around the X-axis, Y-axis, and Z-axis. Furthermore, the target shape correction unit 304 may correct data related to the target shape of the construction target with a total of nine degrees of freedom, including expansion or contraction along the X-axis, Y-axis, and Z-axis.

[0119] Data relating to the target shape of the construction target is stored in the target shape data storage unit 305. Specifically, data relating to the target shape of the construction target as an estimation result by the target shape estimation unit 302 or data relating to the target shape of the construction target as a correction result by the target shape correction unit 304 may be stored.

[0120] The work support control unit 306 performs control to support work related to the construction of the construction object based on data related to the target shape of the construction object.

[0121] For example, the work support control unit 306 controls the machine guidance based on data related to the target shape of the work target. Specifically, the work support control unit 306 may notify the operator of information such as the distance between the target shape and a work location such as the tip or back of the bucket 6, via the output device 50 such as the display device 50A or the communication device 60.

[0122] The work support control unit 306 may also perform control related to machine control (automatic driving function) based on data related to the target shape of the work target. Specifically, the work support control unit 306 may control the hydraulic control valve 31 to operate the attachment AT etc. so as to support the operation of the operator, or so that the working parts such as the tip and back of the bucket 6 move on a trajectory that follows the target shape, regardless of the operation of the operator.

[0123] [First Example of Processing Related to Estimation of Target Shape of Construction Object] Next, a first example of processing related to estimation of a target shape of a construction object will be described with reference to FIGS. 8 to 11 in addition to FIGS. 1 to 7.

[0124] Fig. 8 is a flowchart schematically illustrating a first example of processing related to estimation of the target shape of the construction target. Fig. 9 is a diagram showing an example (screen 900) of the display content of the display device 50A showing the topographical shape around the shovel 100. Figs. 10 and 11 are diagrams showing one example and another example of the display content of the display device 50A showing the estimation result of the target shape of the construction target. Specifically, Fig. 10 is a diagram showing the display content of the display device 50A when the target shape of the construction target as the estimation result is appropriate, and Fig. 11 is a diagram showing the display content of the display device 50A when the target shape of the construction target as the estimation result is inappropriate.

[0125] The flowchart of FIG. 8 starts when a predetermined input is received from a user (an operator, a supervisor, or the like) via the input device 52 or the communication device 60, for example.

[0126] As shown in Figure 8, in step S102 (an example of an acquisition step), the terrain shape acquisition unit 301 acquires image data of the area around the shovel 100 or three-dimensional data representing the state of the area around the shovel 100 based on the image data, based on the output of the imaging device 40.

[0127] When the process of step S102 is completed, the controller 30 proceeds to step S104.

[0128] In step S104, the display processing unit 303 displays an image representing the shape of the terrain around the shovel 100 at the location where the construction target is to be formed on the display device 50A, the remote operation support device 300, etc., based on the data acquired in step S102 (see Figure 9).

[0129] For example, as shown in FIG. 9, images 901 to 903 are displayed on a screen 900 of the display device 50A.

[0130] Image 901 is an image that represents the shape of the terrain around shovel 100 at a location where a construction object (in this example, a slope) is to be formed. Specifically, in image 901, the shape of the terrain around shovel 100 is expressed by three-dimensional mesh data. This allows the user to recognize the current shape of the terrain at the location where the construction object is to be formed.

[0131] The image 902 is an icon for operation to execute a process for estimating a target shape of the construction target. This allows the user to execute the process for estimating a target shape of the construction target by operating the icon of the image 902 via the input device 52 or the like (see step S106 in FIG. 8 ).

[0132] Image 903 is an icon for operation to end the processing of the flowchart in Fig. 8 and return to a predetermined screen. The same applies to images 1004 and 1104 described below. This allows the processing of the flowchart in Fig. 8 to be stopped midway.

[0133] Returning to FIG. 8, when the process of step S104 is completed, the controller 30 proceeds to step S106.

[0134] In step S106, the controller 30 determines whether or not an operation to execute a process to estimate a target shape of the construction target has been performed via the input device 52, etc. If an operation to execute a process to estimate a target shape of the construction target has been performed, the controller 30 proceeds to step S108, and otherwise (for example, if an operation is performed on the image 903 or if no operation is performed even after a certain period of time has elapsed) the controller 30 ends the processing of this flowchart.

[0135] The processes of steps S104 and S106 may be omitted.

[0136] In step S108 (an example of an estimation step), the target shape estimation unit 302 estimates the target shape of the construction object using the first estimation method described above based on the data acquired in step S102, and acquires data regarding the target shape of the construction object.

[0137] When the process of step S108 is completed, the controller 30 proceeds to step S110.

[0138] In step S110, the target shape of the construction target, which is the estimation result in step S108, is displayed on the display device 50A, the remote operation support device 300 (display device), or the like (see FIGS. 10 and 11).

[0139] For example, as shown in Fig. 10, a screen 1000 includes images 1001 to 1005. Similarly, as shown in Fig. 11, a screen 1100 includes images 1101 to 1105.

[0140] Similar to image 901 in FIG. 9, images 1001 and 1101 each represent the shape of the terrain around the excavator 100 at the location where the construction object (in this example, a slope) is to be formed.

[0141] The images 1002 and 1102 are images that represent the target shape (target construction surface) of the construction object.

[0142] Images 1003 and 1103 are icons for operations to fix the target shape (target construction surface) of the construction object to the currently displayed content.

[0143] Images 1004 and 1104 are operation icons for transitioning from the currently displayed content to a screen for correcting the target shape of the construction object.

[0144] On the screen 1000, the target shape of the construction target (image 1002) is appropriately estimated based on the current topographical shape (image 1001) around the excavator 100. Therefore, the user can confirm the target shape of the construction target using the currently displayed data by operating the icon in the image 1003 via the input device 52, the remote operation support device 300, or the like.

[0145] On the other hand, on the screen 1100, the target shape of the construction target (image 1102) is estimated in a form that does not match at all with the current shape of the periphery of the shovel 100. Therefore, the user can correct the target shape of the construction target by operating the operation icon on the image 1104 via the input device 52, the remote operation support device 300, or the like.

[0146] Returning to FIG. 8, when the process of step S110 is completed, the controller 30 proceeds to step S112.

[0147] In step S112, the controller 30 determines whether or not an operation for modifying the target shape of the construction object has been performed via the input device 52, the remote operation support device 300 (input device), etc. If an operation for modifying the target shape of the construction object has been performed, the controller 30 proceeds to step S114, and if an operation for finalizing the target shape of the construction object has been performed, the controller 30 proceeds to step S116.

[0148] In step S114 , the target shape correction unit 304 corrects data relating to the target shape of the construction object in accordance with input from the user via the input device 52 or the remote operation support device 300 .

[0149] When the process of step S114 is completed, the controller 30 proceeds to step S116.

[0150] In step S116, the controller 30 determines the target shape and stores data relating to the target shape in the target shape data storage unit 305.

[0151] When the process of step S116 is completed, the controller 30 ends the process of this flowchart.

[0152] In this way, in this example, the support device 150 can estimate the target shape of the construction target based on the shape of some areas that have already been constructed by an experienced worker operating a shovel, making it easier to obtain data regarding the target shape of the construction target.

[0153] In addition, in this example, the support device 150 can correct the target shape of the construction target as an estimation result in response to an input from the user.

[0154] [Outline of Operation Support System] Next, the operation support system SYS will be described with reference to FIG. 12 in addition to FIGS. 1 to 3.

[0155] 12 , the shovel 100 may be a component of an operation support system SYS. Specifically, the operation support system SYS includes the shovel 100 and an information processing device 200.

[0156] The operation support system SYS uses the information processing device 200 to cooperate with the shovel 100 and provide support regarding the operation of the shovel 100.

[0157] The operation support system SYS may include one or more excavators 100.

[0158] The excavator 100 is a work machine that receives support for operation in the operation support system SYS.

[0159] The information processing device 200 communicates with the shovel 100 to cooperate with each other and provide support regarding the operation of the shovel 100.

[0160] The information processing device 200 is, for example, a server or a management terminal device installed in an administration office within the work site of the shovel 100, or in a management center that manages the operating status of the shovel 100 and is located in a location different from the work site of the shovel 100. The management terminal device may be, for example, a fixed terminal device such as a desktop personal computer (PC), or a portable terminal device (mobile terminal) such as a tablet terminal, a smartphone, or a laptop PC. In the latter case, workers at the work site, supervisors who supervise the work, managers who manage the work site, etc. can move around the work site carrying the portable information processing device 200. In addition, in the latter case, an operator can, for example, bring the portable information processing device 200 into the cabin of the shovel 100.

[0161] The information processing device 200 acquires, for example, data relating to the operating state from the shovel 100. This enables the information processing device 200 to grasp the operating state of the shovel 100 and monitor the presence or absence of abnormalities in the shovel 100. Furthermore, the information processing device 200 displays data relating to the operating state of the shovel 100 via a display device 208 (described later) so that the user can check it.

[0162] Furthermore, the information processing device 200 transmits to the shovel 100, for example, various data such as programs and reference data used in the processing of the controller 30, etc., to the shovel 100. This allows the shovel 100 to perform various processes related to the operation of the shovel 100 using the various data downloaded from the information processing device 200.

[0163] [Hardware Configuration of Operation Support System] Next, the hardware configuration of the operation support system SYS will be described with reference to FIG. 13 in addition to FIGS. 1 to 4 and 12.

[0164] The hardware configuration of the excavator 100 is the same as that shown in FIG. 4, and therefore a description thereof will be omitted.

[0165] FIG. 5 is a block diagram showing an example of the hardware configuration of the information processing device 200.

[0166] The functions of the information processing device 200 are realized by any hardware or any combination of hardware and software, etc. For example, as shown in Fig. 5, the information processing device 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a high-speed calculation device 205, a communication interface 206, an input device 207, and a display device 208, which are connected by a bus B2.

[0167] The external interface 201 functions as an interface for reading data from the recording medium 201A and writing data to the recording medium 201A. Examples of the recording medium 201A include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, a USB memory, etc. This allows the information processing device 200 to read various data used in processing through the recording medium 201A, store the data in the auxiliary storage device 202, and install programs that realize various functions.

[0168] The information processing device 200 may acquire various data and programs used in processing from an external device via the communication interface 206 .

[0169] The auxiliary storage device 202 stores various installed programs, as well as files and data necessary for various processes. The auxiliary storage device 202 includes, for example, a hard disk drive (HDD), a solid state disk (SSD), a flash memory, etc.

[0170] When an instruction to start a program is received, the memory device 203 reads and stores the program from the auxiliary storage device 202. The memory device 203 includes, for example, a dynamic random access memory (DRAM) or an SRAM.

[0171] The CPU 204 executes various programs loaded from the auxiliary storage device 202 to the memory device 203, and realizes various functions related to the information processing device 200 in accordance with the programs.

[0172] The high-speed arithmetic unit 205 performs arithmetic processing at a relatively high speed in cooperation with the CPU 204. The high-speed arithmetic unit 205 includes, for example, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0173] The high-speed calculation unit 205 may be omitted depending on the required calculation processing speed.

[0174] The communication interface 206 is used as an interface for communicatively connecting with an external device. This allows the information processing device 200 to communicate with an external device such as the excavator 100 through the communication interface 206. The communication interface 206 may have multiple types of communication interfaces depending on the communication method between the connected device and the like.

[0175] The input device 207 accepts various inputs from the user.

[0176] The input device 207 includes, for example, an operation input device that accepts mechanical operation input from a user. The operation input device includes, for example, a button, a toggle, a lever, etc. The operation input device also includes, for example, a touch panel mounted on the display device 208, a touch pad provided separately from the display device 208, etc.

[0177] The input device 207 also includes, for example, a voice input device capable of receiving voice input from the user, and a microphone capable of collecting the user's voice.

[0178] The input device 207 also includes, for example, a gesture input device capable of receiving gesture input from a user. The gesture input device includes, for example, a camera capable of capturing images of the user's gestures.

[0179] The input device 207 also includes, for example, a biometric input device capable of accepting biometric input from a user. The biometric input device includes, for example, a camera capable of acquiring image data containing information about a user's fingerprint or iris.

[0180] The display device 208 displays an information screen or an operation screen to the user. For example, the display device 208 includes the above-mentioned remote control display device. The display device 208 is, for example, a liquid crystal display or an organic EL (Electroluminescence) display.

[0181] Like the information processing device 200, the remote operation support device 300 may be realized by any hardware or any combination of hardware and software, and may have a similar hardware configuration. For example, like the information processing device 200 ( FIG. 6 ), the remote operation support device 300 is primarily configured around a computer including a CPU, a memory device, an auxiliary storage device, an interface device, an input device, and a display device. The memory device is, for example, an SRAM or DRAM. The auxiliary storage device is, for example, an HDD, an SSD, an EEPROM, or a flash memory. The interface device includes an external interface for connecting to an external recording medium and a communication interface for communicating with the outside, such as the shovel 100. The input device includes, for example, a lever-type operation input device. As a result, the operator can use the operation input device to input operations related to the actuators of the shovel 100, and the remote operation support device 300 can transmit signals corresponding to the operation input to the shovel 100 using the communication interface. This allows the operator to remotely control the shovel 100 using the remote operation support device.

[0182] [Second example of functional configuration related to estimation of target shape of construction object] Next, a second example of functional configuration related to estimation of target shape of construction object will be described with reference to Figure 14 in addition to Figures 1 to 4, Figure 12, and Figure 13.

[0183] The following description will focus on the differences from the first example (FIG. 7) described above, and the description of the same or corresponding content as the first example described above may be simplified or omitted.

[0184] FIG. 14 is a functional block diagram showing a second example of a functional configuration related to estimation of a target shape of a construction object.

[0185] As shown in FIG. 14 , the excavator 100 includes a support device 150 .

[0186] The support device 150 includes the controller 30, the imaging device 40, the display device 50A, the input device 52, and the communication device 60, as in the first example (FIG. 7) described above.

[0187] The controller 30 includes, as functional units, a terrain shape acquisition unit 301, a target shape estimation unit 302, a display processing unit 303, a target shape correction unit 304, a target shape data storage unit 305, a work support control unit 306, a learned model storage unit 307, and a transmission unit 308.

[0188] The learned model storage unit 307 stores the learned model LM.

[0189] The trained model LM is used to estimate the target shape of the construction target by the second estimation method described above. The trained model LM is distributed from the information processing device 200.

[0190] The target shape estimation unit 302 estimates the target shape of the construction target and acquires data related to the target shape of the construction target using the second estimation method described above, based on the data acquired by the topographical shape acquisition unit 301. Specifically, the target shape estimation unit 302 may use the data acquired by the topographical shape acquisition unit 301 as input data, and apply the learned model LM to estimate the target shape of the construction target based on the output data of the learned model LM.

[0191] The transmitting unit 308 transmits to the information processing device 200 a combination of data acquired by the terrain shape acquisition unit 301 and data corresponding to this data regarding the target shape of the construction object stored (registered) in the target shape data storage unit 305.

[0192] The information processing device 200 includes a teacher data storage unit 2001, a machine learning unit 2002, a trained model storage unit 2003, and a distribution unit 2004 as functional units related to estimating the target shape of the construction object.

[0193] The teacher data storage unit 2001 stores (registers) teacher data for generating the learned model LM. The teacher data storage unit 2001 may also include teacher data for re-learning or additional learning and updating the learned model LM. The teacher data is a combination of data on the topographical shape including a portion of the construction target that has already been constructed and data on the target shape of the construction target.

[0194] For example, the latter training data includes a combination of data acquired by the terrain shape acquisition unit 301 received from the shovel 100 (transmission unit 308) and data corresponding to this data regarding the final determined target shape of the construction target.

[0195] The machine learning unit 2002 uses the teacher data set in the teacher data storage unit 2001 to perform machine learning on a predetermined model, thereby generating a trained model LM.

[0196] In addition, the machine learning unit 2002 may use the teacher data set in the teacher data storage unit 2001 to re-learn or additionally train the learned model LM, thereby updating the learned model LM.

[0197] The learned model storage unit 2003 stores (registers) the learned model LM generated by the machine learning unit 2002. In addition, the learned model LM in the learned model storage unit 2003 may be updated by re-learning or additional learning by the machine learning unit 2002.

[0198] The distribution unit 2004 distributes the learned model LM to the excavator 100.

[0199] [Second Example of Processing Related to Estimation of Target Shape of Construction Object] Next, a second example of processing related to estimation of a target shape of a construction object will be described with reference to FIG. 15 in addition to FIGS. 1 to 4 and 12 to 14 .

[0200] The following description will focus on the differences from the first example (FIG. 8) described above, and the description of the same or corresponding content as the first example described above may be simplified or omitted.

[0201] As shown in FIG. 14, the processes in steps S202 and S204 are the same as steps S102 and S104 in FIG. 8, and therefore a description thereof will be omitted.

[0202] When the process of step S204 is completed, the controller 30 proceeds to step S206.

[0203] In step S206, the controller 30 determines whether or not an operation to execute a process to estimate a target shape of the construction object has been performed via the input device 52, etc., similarly to step S106 in Fig. 8. If an operation to execute a process to estimate a target shape of the construction object has been performed, the controller 30 proceeds to step S208; otherwise, the controller 30 ends the processing of this flowchart.

[0204] In step S208, the target shape estimation unit 302 estimates the target shape of the construction object using the second estimation method described above based on the data acquired in step S102, and acquires data related to the target shape of the construction object.

[0205] When the process of step S208 is completed, the controller 30 proceeds to step S210.

[0206] The processing in steps S210 to S216 is the same as steps S110 to S116 in FIG. 8, and therefore a description thereof will be omitted.

[0207] When the process of step S216 is completed, the controller 30 proceeds to step S218.

[0208] In step S218, the transmitting unit 308 transmits the data acquired in step S202 and the data on the target shape of the construction object confirmed in step S216 to the information processing device 200. Specifically, the transmitting unit 308 transmits to the information processing device 200 a combination of data on the current topographical shape of the location where the construction object is to be formed, including a portion of the construction object where construction has already been completed, and the finally confirmed data on the target shape of the construction object.

[0209] When the process of step S216 is completed, the controller 30 ends the process of this flowchart.

[0210] The process of step S216 may be omitted, and the transmitting unit 308 may transmit the data for multiple times of this flowchart together to the information processing device 200 as a batch process.

[0211] In this way, in this example, the support device 150 can estimate the target shape of the construction object using the learned model LM based on data regarding the current topographical shape of the location where the construction object is to be formed, including areas where part of the construction object has already been constructed.

[0212] Furthermore, in this example, the information processing device 200 can update the learned model LM using training data that is a combination of data on the topographical shape of the construction object, including a portion of the construction object that has already been constructed, and data on the target shape of the construction object. This allows the results of the correction of the target shape of the construction object by the target shape correction unit 304 to be reflected in the learned model LM. This can improve the estimation accuracy of the target shape of the construction object by the learned model LM.

[0213] [Other Embodiments] Next, other embodiments will be described.

[0214] The above-described embodiments may be combined, modified, or changed as appropriate.

[0215] For example, in the above-described embodiment, when the shovel 100 performs construction work on a portion of the construction target, the pre-processing process related to the construction work and the process related to the target shape estimation (see FIGS. 8 and 15 ) may be executed as a series of processes. In this case, in the pre-processing process related to the construction work, or in a process parallel to this process, the trajectories of the working parts, such as the toe and back of the bucket 6, during the construction work may be recorded in the auxiliary storage device 30A or the like. This allows the terrain shape acquisition unit 301 to acquire data related to the terrain shape including the portion of the construction target that has already been constructed, based on the data on the trajectories of the working parts. This is because the data on the trajectories of the working parts is considered to represent the relative terrain shape of the portion that has already been constructed, as seen from the shovel 100. In this case, the target shape estimation unit 302 may estimate the target shape of the construction target based on the data on the trajectories of the working parts. In this case, the target shape correction unit 304 may correct (correct) the data related to the target shape of the construction target, which is the estimation result of the target shape estimation unit 302, based on the data on the trajectories of the working parts.

[0216] In addition, in the above-described embodiment and its modifications, some or all of the functions of the assistance device 150 may be transferred to the remote operation assistance device 300 .

[0217] In addition, in the above-described embodiment and its modifications, some or all of the functions of the assistance device 150 may be transferred to the information processing device 200 .

[0218] In addition, in the above-described embodiment and its modified examples, some or all of the functional units of the information processing device 200 related to the target shape estimation function may be transferred to the shovel 100.

[0219] [Operation] Next, the operation of the support device according to this embodiment will be described.

[0220] In this embodiment, the support device includes an acquisition unit that acquires data on the shape of a portion of the construction target that has already been constructed, and an estimation unit that estimates a target shape of the construction target based on the data acquired by the acquisition unit. The support device is, for example, the support device 150. The acquisition unit is, for example, a topographical shape acquisition unit 301. The estimation unit is, for example, a target shape estimation unit 302.

[0221] This allows data on the target shape of the construction target to be acquired by simply performing construction on a portion of the construction target, which makes it easier for the support device to acquire data on the target shape of the construction target.

[0222] In the present embodiment, the estimation unit may estimate the target shape of the construction target using a trained model that has been machine-learned using training data that is a combination of the shape of a portion of the construction target that has already been constructed and the target shape of the construction target. The trained model is, for example, a trained model LM.

[0223] This allows the support device to acquire data on the target shape of the construction object from the shapes of some of the already constructed portions of the construction object.

[0224] In addition, in this embodiment, the estimation unit may estimate the target shape of the construction object by duplicating or extending the shape of a portion of the construction object that has already been constructed in a direction corresponding to the construction object.

[0225] This allows the support device to acquire data on the target shape of the construction object from the shapes of some of the already constructed portions of the construction object.

[0226] In this embodiment, the estimation unit may estimate the target shape of the construction target based on data relating to the trajectory of a work part when the construction machine performs construction work on a part of the construction target that has already been constructed. The construction machine is, for example, a shovel 100, and the work part is the tip or back of the bucket 6 of the shovel 100.

[0227] This allows the support device to acquire data on the target shape of the construction object from the shapes of some of the already constructed portions of the construction object.

[0228] In the present embodiment, the support device may include a display unit that displays the target shape of the construction target as the estimation result by the estimation unit. The display unit is, for example, a display device 50A.

[0229] This allows the assistance device to allow the user to visually confirm the target shape of the construction target as an estimation result, thereby improving user convenience.

[0230] In the present embodiment, the support device may further include a correction unit that corrects the target shape of the construction target, which is the estimation result of the estimation unit, in response to an input from a user. The correction unit is, for example, the target shape correction unit 304.

[0231] This allows the user to operate the assistance device and correct the target shape of the construction object when the target shape of the construction object as an estimated result is inappropriate.

[0232] In the present embodiment, the assistance device may further include a correction unit and an update unit that updates the trained model using a combination of the data acquired by the acquisition unit and data on the target shape of the construction target as a result of correction by the correction unit as training data. The update unit is, for example, the machine learning unit 2002.

[0233] This allows the correction results by the correction unit to be reflected in the trained model, thereby improving the accuracy of estimating the target shape of the construction target using the trained model.

[0234] In this embodiment, the work machine may also be equipped with the above-mentioned assistance device.

[0235] This allows the work machine to use the assistance device to more easily estimate and acquire the target shape of the work target.

[0236] Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist described in the claims.

[0237] Finally, this application claims priority based on Japanese Patent Application No. 2022-058416, filed on March 31, 2022, the entire contents of which are incorporated herein by reference.

[0238] REFERENCE SIGNS LIST 1 Lower traveling body 1C, 1CL, 1CR Crawler 1ML, 1MR Travel hydraulic motor 2 Swing mechanism 2M Swing hydraulic motor 3 Upper rotating body 4 Boom 5 Arm 6 Bucket 7 Boom cylinder 8 Arm cylinder 9 Bucket cylinder 10 Cabin 11 Engine 13 Regulator 14 Main pump 15 Pilot pump 17 Control valve 26 Operation device 29 Operation pressure sensor 30 Controller 40 Imaging device 50 Output device 50A Display device 52 Input device 60 Communication device 100 Excavator 150 Support device 200 Information processing device 206 Communication interface 207 Input device 208 Display device 300 Remote operation support device 301 Topographic shape acquisition unit 302 Target shape estimation unit 303 Display processing unit 304 Target shape correction unit 305 Target shape data storage unit 306 Work support control unit 307 Learned model storage unit 308 Transmission unit 2001 Teacher data storage unit 2002 Machine learning unit 2003 Learned model storage unit 2004 Distribution unit AT Attachment HA Hydraulic actuator LM Learned model S1 to S5 Sensors SYS Operation support system

Claims

1. An acquisition unit that acquires data on the shape of a portion of a construction target that has already been constructed; An estimation unit that estimates a target shape of a construction object based on the data acquired by the acquisition unit, Support equipment.

2. The estimation unit estimates the target shape of the construction object using a trained model that has been machine-learned using training data that is a combination of a shape of a part of the construction object that has been constructed and a target shape of the construction object. The support device according to claim 1 .

3. The estimation unit estimates a target shape of the construction object by duplicating or extending a shape of a part of the construction object that has already been constructed in a direction corresponding to the construction object. The support device according to claim 1 .

4. The estimation unit estimates a target shape of the construction target based on data related to a trajectory of a work site when a work machine performs construction work on a part of the construction target that has already been constructed.

4. An assistance device according to any one of claims 1 to 3.

5. A display unit that displays a target shape of a construction target as an estimation result by the estimation unit, 4. An assistance device according to any one of claims 1 to 3.

6. A correction unit is provided that corrects a target shape of a construction target as an estimation result of the estimation unit in response to an input from a user.

4. An assistance device according to any one of claims 1 to 3.

7. A correction unit that corrects a target shape of a construction target as an estimation result of the estimation unit in response to an input from a user; An update unit that updates the trained model using a combination of the data acquired by the acquisition unit and the data of the target shape of the construction target as a result of the correction by the correction unit as teacher data. The support device according to claim 2 .

8. An acquisition unit that acquires data on the shape of a portion of a construction target that has already been constructed; An estimation unit that estimates a target shape of the object to be processed based on the data acquired by the acquisition unit, Working machinery.

9. Support equipment: An acquisition step of acquiring data on the shape of a portion of the construction target that has already been constructed; an estimation step of estimating a target shape of the construction object based on the data acquired in the acquisition step; program.