Buried object estimation system, and excavator

Abstract

This technology provides a way to accurately estimate buried objects being excavated during shovel excavation work. [Solution] The shovel 100 includes a lower traveling body 1, an upper rotating body 3 that is rotatably mounted on the lower traveling body 1, an attachment AT mounted on the upper rotating body 3 for excavating the target to be excavated, and a controller 30 that controls the operation of the attachment AT. The buried object estimation system 200 includes an information processing device 210 that acquires information related to the excavation reaction force during the excavation work of the attachment AT in order to estimate the possibility of buried objects being present in the target to be excavated, learns a learning model for detecting buried objects, and transmits the learning model to the controller 30. The controller 30 estimates the presence or absence of buried objects in the target to be excavated based on the excavation reaction force acquired in the actual excavation work and the learning model.

Description

Technical Field

[0001] The present disclosure relates to an embedded object estimation system and an excavator.

Background Art

[0002] In excavation work, an excavator may damage buried objects buried in an excavation target such as the ground. Therefore, at the work site, work is carried out to check for the presence or absence of buried objects in the excavation target. For example, in Patent Document 1, ground measurement data is acquired by a geophysical exploration device, and the position of a buried object in the ground corresponding to the measurement data for teaching is learned, and the position of the buried object is estimated from the measurement data using a learned model. An excavation system is disclosed. [[ID=!]]

[0003] When estimating buried objects by a geophysical exploration device as described above during the excavation work of an excavator, the excavation work of the excavator is often interrupted, resulting in poor work efficiency. From this, at the work site, it is required that buried objects that may be buried in the excavation target can be estimated during the excavation work of the excavator.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present disclosure provides a technique capable of accurately estimating buried objects in an excavation target during the excavation work of an excavator. [[ID=!]]

Means for Solving the Problems

[0006] Note: There seem to be some tags with "!" in the original text which might be errors. I've translated them as-is while keeping the tags intact. If these are meant to be something else, please clarify.According to one aspect of the present disclosure, a buried object estimation system is provided for an excavator including a lower traveling body, an upper rotating body rotatably mounted on the lower traveling body, an attachment mounted on the upper rotating body for excavating an excavation target, and a control unit for controlling the operation of the attachment, wherein the buried object estimation system estimates the possibility of buried objects being present in the excavation target, and includes an information processing unit that acquires information related to the excavation reaction force during the excavation work of the attachment to learn a learning model for detecting the buried objects, and transmits the learning model to the control unit, and the control unit estimates the presence or absence of buried objects in the excavation target based on the excavation reaction force acquired in the actual excavation work and the learning model. [Effects of the Invention]

[0007] According to one embodiment, buried objects to be excavated can be accurately estimated during excavation work with a shovel. [Brief explanation of the drawing]

[0008] [Figure 1] This is a side view showing an excavator according to an embodiment. [Figure 2] This is a side view showing various physical quantities related to the drilling attachment. [Figure 3] This is an explanatory diagram showing the basic system of an excavator. [Figure 4] This figure shows an example of the configuration of an excavation control system mounted on the excavator shown in Figure 1. [Figure 5] This diagram shows a cross-section of the ground where water pipes are buried. [Figure 6] This graph shows the relationship between excavation reaction force and approach distance. [Figure 7] This is a block diagram of the detection system. [Figure 8] This is a block diagram showing the functional components of an information processing device when generating a learning model. [Figure 9] This is an explanatory diagram showing how the excavator acquires, tunes, and uses the learned model during excavation work. [Figure 10]This diagram illustrates the image information displayed on the display device when the buried object estimation function is executed. [Figure 11] Figure 11(A) is a flowchart showing the processing flow before actually performing excavation work. Figure 11(B) is a flowchart showing the method for estimating buried objects when actually performing excavation work. [Figure 12] This is a schematic diagram showing an example of the configuration of the operating system. [Modes for carrying out the invention]

[0009] The following describes embodiments for implementing this disclosure with reference to the drawings. In each drawing, the same reference numerals are used for identical components, and redundant explanations may be omitted.

[0010] Figure 1 is a side view showing an excavator 100 according to an embodiment. The excavator 100 comprises a lower traveling body 1 and an upper rotating body 3 that is mounted on the lower traveling body 1 so as to be rotatable via a slewing mechanism 2.

[0011] Furthermore, the shovel 100 is equipped with an attachment AT, which is an example of an attachment, as a working element. Attachment AT includes a boom 4 attached to the upper slewing body 3, an arm 5 attached to the tip of the boom 4, and a bucket 6 attached to the tip of the arm 5. For convenience, in this specification, the side of the upper slewing body 3 to which the boom 4 is attached is considered the front, and the side to which the counterweight is attached is considered the rear. The boom 4 is driven by a boom cylinder 7. The arm 5 is driven by an arm cylinder 8. The bucket 6 is driven by a bucket cylinder 9.

[0012] The upper rotating body 3 is equipped with a cabin 10 and a power source such as an engine 11 or an electric motor. Inside the cabin 10 are an operating device 26, a controller 30, a display device 40, and a sound output device 45, etc.

[0013] The excavator 100 has a posture detection device M1 that detects the posture of the attachment AT. The posture detection device M1 also serves as a detection device that detects information regarding the excavation reaction force. Specifically, the posture detection device M1 includes a boom angle sensor M1a, an arm angle sensor M1b, and a bucket angle sensor M1c. For example, as the boom angle sensor M1a, a rotation angle sensor that detects the rotation angle of the boom foot pin, a stroke sensor that detects the stroke amount of the boom cylinder 7, an inclination (acceleration) sensor that detects the inclination angle of the boom 4, etc. can be applied. Also, the same type of sensors can be applied to the arm angle sensor M1b and the bucket angle sensor M1c.

[0014] Furthermore, the excavator 100 is equipped with an object detection device 70, etc. on the upper revolving body 3. The object detection device 70 detects objects existing around the excavator 100. The object is, for example, a person, an animal, a vehicle, a construction machine, a building, or a hole, etc. The object detection device 70 is configured by combining, for example, one or a plurality of an ultrasonic sensor, a millimeter-wave radar, an imaging device, or an infrared sensor, etc. The imaging device is, for example, a monocular camera, a stereo camera, a LIDAR, or a distance image sensor, etc. In the illustrated example, the object detection device 70 includes a rear camera 70B attached to the rear end of the upper surface of the upper revolving body 3, a front camera 70F attached to the front end of the upper surface of the cab 10, a left camera 70L attached to the left end of the upper surface of the upper revolving body 3, and a right camera 70R attached to the right end of the upper surface of the upper revolving body 3.

[0015] The object detection device 70 can be configured to detect an object (for example, a person) within a region set around the excavator 100. For example, the object detection device 70 may be configured to distinguish and detect a person and an object other than a person.

[0016] Figure 2 is a side view showing various physical quantities related to the attachment AT. The boom angle sensor M1a detects the boom angle θ1. The boom angle θ1 is the angle of the line segment P1 - P2 connecting the boom foot pin position P1 and the arm connection pin position P2 with respect to the horizontal line in the XZ plane. The arm angle sensor M1b detects the arm angle θ2. The arm angle θ2 is the angle of the line segment P2 - P3 connecting the arm connection pin position P2 and the bucket connection pin position P3 with respect to the horizontal line in the XZ plane. The bucket angle sensor M1c detects the bucket angle θ3. The bucket angle θ3 is the angle of the line segment P3 - P4 connecting the bucket connection pin position P3 and the bucket tip position P4 with respect to the horizontal line in the XZ plane. Note that the bucket angle θ3 may be calculated based on the operation content of the operating device 26. For example, the bucket angle θ3 may be calculated based on the outputs of the pilot pressure sensors 15a, 15b (Figure 3), etc. In this case, the bucket angle sensor M1c may be omitted.

[0017] Figure 3 is an explanatory view showing the basic system of the excavator 100. The basic system of the excavator 100 includes an engine 11, a main pump 14, a pilot pump 15, a control valve unit 17, an operating device 26, a controller 30, a display device 40, a sound output device 45, an engine control device 74, an operation mode changeover switch 75, a buried object estimation mode switch 76, an attitude detection device M1, and an excavation pressure sensor S1, etc.

[0018] The engine 11 is a drive source of the excavator 100, and is, for example, a diesel engine that operates so as to maintain a predetermined rotational speed. The output shaft of the engine 11 is connected to the input shafts of the main pump 14 and the pilot pump 15.

[0019] The main pump 14 is a hydraulic pump that supplies hydraulic fluid to the control valve unit 17 via the hydraulic fluid line 16, and is, for example, a swashplate variable displacement hydraulic pump. In a swashplate variable displacement hydraulic pump, the stroke length of the piston that determines the displaced volume changes in accordance with the change in the swashplate tilt angle, and the discharge flow rate per revolution changes. The swashplate tilt angle is controlled by a regulator 14a. The regulator 14a changes the swashplate tilt angle in accordance with the change in the control current from the controller 30. For example, the regulator 14a increases the swashplate tilt angle in accordance with an increase in the control current to increase the discharge flow rate of the main pump 14. Conversely, the regulator 14a decreases the swashplate tilt angle in accordance with a decrease in the control current to reduce the discharge flow rate of the main pump 14. The discharge pressure sensor 14b detects the discharge pressure of the main pump 14. The oil temperature sensor 14c detects the temperature of the hydraulic fluid drawn in by the main pump 14.

[0020] The pilot pump 15 is a hydraulic pump for supplying hydraulic fluid to various hydraulic control devices such as the operating device 26 via the pilot line 25, and for example, a fixed-displacement hydraulic pump can be used.

[0021] The control valve unit 17 controls the flow of hydraulic fluid to the hydraulic actuators. In the illustrated example, the control valve unit 17 includes a plurality of flow control valves. The control valve unit 17 selectively supplies hydraulic fluid received from the main pump 14 through the hydraulic fluid line 16 to one or more hydraulic actuators in response to changes in pressure (pilot pressure) corresponding to the operating direction and amount of the operating device 26. The hydraulic actuators include, for example, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left-travel hydraulic motor 1A, a right-travel hydraulic motor 1B, and a slewing hydraulic motor 2A. In the illustrated example, the hydraulic motors (left-travel hydraulic motor 1A, right-travel hydraulic motor 1B, and slewing hydraulic motor 2A) are swashplate piston motors. However, the hydraulic motors may be replaced with electric motors.

[0022] The operating device 26 is a device used by an operator to operate a hydraulic actuator and includes levers 26A, 26B, and pedals 26C, etc. The operating device 26 receives hydraulic fluid from the pilot pump 15 via the pilot line 25 and generates pilot pressure. The operating device 26 applies this pilot pressure to the pilot port of the corresponding flow control valve via the pilot line 25a. The pilot pressure changes according to the operating direction and amount of the operating device 26. The operating device 26 may be remotely operated. In remote operation, the operating device 26 generates pilot pressure based on information regarding the operating direction and amount of operation received via wireless communication.

[0023] The operating device 26 may be an electric operating device instead of a hydraulic operating device as described above. In this case, a solenoid valve for adjusting the pilot pressure may be placed between the flow control valve in the control valve unit 17 and the pilot pump 15. Information regarding the operating direction and amount of the electric operating device is transmitted from the electric operating device to the controller 30 as an electrical signal. The controller 30 can adjust the magnitude of the pilot pressure acting on the flow control valve by adjusting the opening area of ​​the solenoid valve in response to the electrical signal received from the electric operating device.

[0024] The controller 30 functions as a control unit that drives and controls the shovel 100. The controller 30's functions may be realized by any hardware, or a combination of hardware and software. For example, the controller 30 is mainly composed of a microcomputer that includes a processor such as a CPU (Central Processing Unit), memory such as RAM (Random Access Memory) and ROM (Read Only Memory), and various input / output interface devices. The controller 30 realizes various functions by executing various programs stored in ROM, etc., on the CPU. For example, the controller 30 changes the magnitude of the control current to the regulator 14a according to the pressure of the hydraulic fluid in the negative control valve, and controls the discharge flow rate of the main pump 14 via the regulator 14a.

[0025] The display device 40 is a device that displays various information and is located near the driver's seat in the cabin 10. In the illustrated example, the display device 40 has an image display unit 41 and an input unit 42. The image display unit 41 is a liquid crystal display. The input unit 42 is a membrane switch. The operator can input information and commands to the controller 30 using the input unit 42. The operator can also understand the operating status and control information of the shovel 100 by looking at the image display unit 41. The display device 40 is connected to the controller 30 via a communication network such as CAN. However, the display device 40 may also be connected to the controller 30 via a dedicated line.

[0026] The display device 40 operates on power supplied from the storage battery 90. The storage battery 90 is charged with electricity generated by the alternator 11a. Power from the storage battery 90 is also supplied to other devices besides the controller 30 and the display device 40, such as the electrical components 72 of the shovel 100. The starter 11b of the engine 11 is driven by power from the storage battery 90 to start the engine 11.

[0027] The sound output device 45 is a device that outputs sound information. In the illustrated example, the sound output device 45 is a speaker located near the driver's seat inside the cabin 10. The sound output device 45 may also be a buzzer.

[0028] The engine control device 74 is a device that controls the engine 11. For example, the engine control device 74 controls the fuel injection amount and the like so that the engine speed set via the input device is achieved.

[0029] The engine control device 74 transmits various data indicating the state of the engine 11 (for example, data related to physical quantities such as data indicating the coolant temperature detected by the water temperature sensor 11c) to the controller 30. The controller 30 stores the various data in the memory 30a and transmits it to the display device 40 or the like as needed. The same applies to data indicating the swash plate tilt angle output by the regulator 14a, data indicating the discharge pressure of the main pump 14 output by the discharge pressure sensor 14b, data indicating the hydraulic oil temperature output by the oil temperature sensor 14c, and data indicating the pilot pressure output by the pilot pressure sensors 15a and 15b.

[0030] The operating mode selector switch 75 is a switch for switching the operating mode of the shovel 100 and is located inside the cabin 10. The operating modes are, for example, M (manual) mode and SA (semi-automatic) mode. The controller 30 switches the operating mode of the shovel 100 according to the output of the operating mode selector switch 75.

[0031] The M (Manual) mode is a mode in which the shovel 100 is operated in response to the operator's input to the control device 26. For example, this mode operates the boom cylinder 7, arm cylinder 8, and bucket cylinder 9, etc., according to the content of the operator's input to the control device 26. The SA (Semi-Automatic) mode is a mode in which the shovel 100 is operated automatically when predetermined conditions are met, regardless of the content of the input to the control device 26. For example, when predetermined conditions are met, at least one of the boom cylinder 7, arm cylinder 8, and bucket cylinder 9 is operated automatically regardless of the content of the input to the control device 26. Note that the operating modes may also include a fully automatic mode in which the entire lower traveling body 1, slewing mechanism 2, boom cylinder 7, arm cylinder 8, and bucket cylinder 9, etc., are operated autonomously.

[0032] The buried object estimation mode switch 76 is a switch for activating the buried object estimation function mode and is located inside the cabin 10. The buried object estimation function mode performs a process to estimate buried objects in the ground to be excavated. In this embodiment, the buried object estimation function mode of the shovel 100 estimates the presence or absence of buried objects based on the excavation reaction force. The operator switches the buried object estimation function mode on and off by operating the buried object estimation mode switch 76.

[0033] The controller 30 executes the buried object estimation function mode in response to an activation command from the buried object estimation mode switch 76, and stops the buried object estimation function mode in response to a stop command from the buried object estimation mode switch 76. However, the controller 30 may also activate the buried object estimation function mode if it determines that excavation is being performed based on the posture of the attachment AT, etc., regardless of the operation of the buried object estimation mode switch 76. For example, the controller 30 can continuously execute the buried object estimation function mode from the time the excavation operation starts until the time the boom raising operation is performed.

[0034] The drilling pressure sensor S1 is an example of a detection device that detects information related to drilling reaction force. It detects the pressure of the hydraulic fluid in a hydraulic cylinder such as the boom cylinder 7 and outputs the detected data to the controller 30. The drilling pressure sensor S1 according to this embodiment is composed of a combination of drilling pressure sensors S11 to S18. Drilling pressure sensor S11 detects the boom bottom pressure, which is the pressure of the hydraulic fluid in the bottom-side oil chamber of the boom cylinder 7. Drilling pressure sensor S12 detects the boom rod pressure, which is the pressure of the hydraulic fluid in the rod-side oil chamber of the boom cylinder 7. Similarly, drilling pressure sensor S13 detects the arm bottom pressure, drilling pressure sensor S14 detects the arm rod pressure, drilling pressure sensor S15 detects the bucket bottom pressure, and drilling pressure sensor S16 detects the bucket rod pressure. Drilling pressure sensor S17 detects the left slewing pressure, which is the pressure of the hydraulic fluid in the first port (left-side port) of the slewing hydraulic motor 2A. The drilling pressure sensor S18 detects the rightward rotation pressure, which is the pressure of the hydraulic fluid at the second port (right port) of the swing hydraulic motor 2A.

[0035] The control valve E1 is a valve that operates in response to commands from the controller 30. In the illustrated example, the control valve E1 is used to forcibly operate a flow control valve for a predetermined hydraulic cylinder, regardless of the content of the operating input to the operating device 26. When the above-described electric operating device is used, the control valve E1 corresponds to a solenoid valve placed between the flow control valve and the pilot pump 15.

[0036] Figure 4 shows an example of the configuration of the excavation control system mounted on the excavator 100 shown in Figure 1. The excavation control system consists of an attitude detection device M1, an excavation pressure sensor S1, an operation mode switching switch 75, a buried object estimation mode switch 76, a controller 30, a control valve E1, a display device 40, and a sound output device 45, etc. The controller 30 has an excavation reaction force calculation unit 31 and a buried object estimation unit 32 built into it by a processor reading and executing a program stored in memory.

[0037] The excavation reaction force calculation unit 31 is a functional element that calculates the excavation reaction force. The excavation reaction force calculation unit 31 is configured to calculate the excavation reaction force based at least on the output of the excavation pressure sensor S1. In the embodiment, the excavation reaction force calculation unit 31 calculates the excavation reaction force based on the output of the excavation pressure sensor S1 and the attitude of the attachment AT detected by the attitude detection device M1. The excavation reaction force calculation unit 31 may additionally utilize the output of the vehicle body tilt sensor. The vehicle body tilt sensor may be composed of, for example, an acceleration sensor or a gyro sensor.

[0038] The output of the drilling pressure sensor S1 includes, for example, at least one of the boom bottom pressure, boom rod pressure, arm bottom pressure, arm rod pressure, bucket bottom pressure, and bucket rod pressure detected by drilling pressure sensors S11 to S16.

[0039] The excavation reaction force calculation unit 31 may calculate the cylinder thrust based on the output of the excavation pressure sensor S1. The cylinder thrust is calculated, for example, based on the excavation pressure and the pressure-receiving area of ​​the piston sliding inside the cylinder. The cylinder thrust includes, for example, the boom cylinder thrust (f1), the arm cylinder thrust (f2), and the bucket cylinder thrust (f3). Specifically, the boom cylinder thrust (f1) is expressed as the difference between the cylinder extension force, which is the product of the boom bottom pressure and the pressure-receiving area of ​​the piston in the boom bottom side oil chamber, and the cylinder contraction force, which is the product of the boom rod pressure and the pressure-receiving area of ​​the piston in the boom rod side oil chamber. The same applies to the arm cylinder thrust (f2) and the bucket cylinder thrust (f3).

[0040] The drilling reaction force calculation unit 31 may calculate the drilling torque based on the attitude of the attachment AT and the cylinder thrust. The magnitude of the bucket drilling torque (τ3) is expressed as shown in Figure 2, by multiplying the magnitude of the bucket cylinder thrust (f3) by the distance G3 between the line of action of the bucket cylinder thrust (f3) and the bucket connecting pin position P3. The distance G3 is a function of the bucket angle θ3 and is an example of link gain. The same applies to the boom drilling torque (τ1) and the arm drilling torque (τ2). Note that distance G1 is the distance between the line of action of the boom cylinder thrust (f1) and the boom foot pin position P1, and distance G2 is the distance between the line of action of the arm cylinder thrust (f2) and the arm connecting pin position P2.

[0041] The digging reaction force is calculated, for example, as the product of a mechanism function with boom angle θ1, arm angle θ2, and bucket angle θ3 as arguments, as shown in Figure 2, and a function with boom digging torque (τ1), arm digging torque (τ2), and bucket digging torque (τ3) as arguments. The function with boom digging torque (τ1), arm digging torque (τ2), and bucket digging torque (τ3) as arguments may also be a function with boom cylinder thrust (f1), arm cylinder thrust (f2), and bucket cylinder thrust (f3) as arguments. The function with boom angle θ1, arm angle θ2, and bucket angle θ3 as arguments may be based on force equilibrium equations, on the Jacobian, or on the principle of virtual work.

[0042] Thus, the value of the drilling reaction force can be derived based on the current detection values ​​of various sensors. However, the detection value of the drilling pressure sensor S1 may be used directly as information on the drilling reaction force. Alternatively, the cylinder thrust value calculated based on the detection value of the drilling pressure sensor S1 may be used as information on the drilling reaction force. The value of the drilling reaction force may also be derived from the drilling torque value calculated from the cylinder thrust value calculated based on the detection value of the drilling pressure sensor S1 and the attitude value of the attachment AT derived based on the detection value of the attitude detection device M1.

[0043] The drilling reaction force calculation unit 31 may calculate the drilling reaction force acting in the rotation direction based on the outputs of the drilling pressure sensors S17 and S18. When the left rotation pressure (P17) detected by the drilling pressure sensor S17 is greater than the right rotation pressure (P18) detected by the drilling pressure sensor S18, the upper rotating body 3 will attempt to rotate to the left. Conversely, when the right rotation pressure (P18) detected by the drilling pressure sensor S18 is greater than the left rotation pressure (P17) detected by the drilling pressure sensor S17, the upper rotating body 3 will attempt to rotate to the right. For example, the drilling reaction force calculation unit 31 may calculate the left rotation pressure (P17) when the left rotation pressure (P17) is greater than the right rotation pressure (P18) as the drilling reaction force acting in the left rotation direction. Furthermore, the excavation reaction force calculation unit 31 may, for example, calculate the right-hand rotation pressure (P18) as the excavation reaction force acting in the right-hand rotation direction when the right-hand rotation pressure (P18) is greater than the left-hand rotation pressure (P17). In addition, if an electric rotation motor is installed instead of the hydraulic rotation motor 2A, the excavation reaction force calculation unit 31 may calculate the excavation reaction force acting in the rotation direction based on information about power, such as the direction and magnitude of the current supplied to the electric rotation motor.

[0044] The buried object estimation unit 32 is configured to detect buried objects based on information regarding the excavation reaction force. In this embodiment, the buried object estimation unit 32 estimates the presence or absence of buried objects based on the excavation reaction force calculated by the excavation reaction force calculation unit 31 and a pre-owned learning model for buried object estimation. This learning model will be described in detail later.

[0045] For example, the buried object estimation unit 32 outputs a control command to the control valve E1 when it estimates the presence of buried objects. When the control valve E1 receives a control command from the buried object estimation unit 32, it forcibly operates a flow control valve for a predetermined hydraulic cylinder, regardless of the content of the operation input to the operating device 26, thereby forcibly extending or retracting the predetermined hydraulic cylinder. For example, even if the boom operation lever is not being operated, the control valve E1 forcibly moves the flow control valve for the boom cylinder 7, thereby forcibly extending the boom cylinder 7. As a result, the shovel 100 can forcibly raise the boom 4 and reduce the excavation depth (change the trajectory). Alternatively, even if the arm operation lever is being operated, the control valve E1 may forcibly move the flow control valve for the arm cylinder 8, thereby forcibly stopping the arm cylinder 8. By forcibly stopping the arm 5, the shovel 100 can avoid contact between the bucket 6 and the buried object. In this way, the control valve E1 can suppress contact between the attachment AT and the buried object by forcibly extending or stopping at least one of the boom cylinder 7, arm cylinder 8, and bucket cylinder 9 in response to a control command from the buried object estimation unit 32.

[0046] The buried object estimation unit 32 may output a control command to the display device 40 if it estimates the presence of a buried object. When the display device 40 receives a control command from the buried object estimation unit 32, it displays the estimated location of the buried object. For example, the display device 40 may display a virtual viewpoint image representing the view of the shovel 100 from a virtual viewpoint directly above the shovel 100, and superimpose the buried object on that virtual viewpoint image. The virtual viewpoint image is generated based on images acquired by the front camera 70F, rear camera 70B, left camera 70L, and right camera 70R, respectively. Alternatively, the display device 40 may display an image representing a cross-section of the ground where the shovel 100 is located, and superimpose the buried object on that virtual viewpoint image. Furthermore, the buried object estimation unit 32 may output a control command to the sound output device 45 if it estimates the presence of a buried object.

[0047] The controller 30 activates the buried object estimation function mode in response to an activation command from the buried object estimation mode switch 76. When the buried object estimation function mode is activated, the buried object estimation unit 32 estimates the presence or absence of buried objects based on the excavation reaction force calculated by the excavation reaction force calculation unit 31. On the other hand, the controller 30 stops the buried object estimation function mode in response to a stop command from the buried object estimation mode switch 76. This prevents the shovel 100 from mistakenly estimating the presence of buried objects in response to fluctuations in the excavation reaction force, even when it is clear that there are no buried objects, and outputting control commands to the control valve E1, display device 40, or sound output device 45. When the buried object estimation function mode is stopped, the excavation reaction force calculation unit 31 does not need to calculate the excavation reaction force in order to reduce the computational load.

[0048] The buried object estimation function mode can be activated regardless of whether the shovel 100 is operating in M ​​(manual) mode or SA (semi-automatic) mode. However, the buried object estimation function mode may only be activated when SA (semi-automatic) mode is selected. When SA (semi-automatic) mode is selected, the operator can improve the accuracy of buried object detection by moving the attachment AT along a pre-set target trajectory.

[0049] Next, referring to Figure 5, we will provide a representative explanation of the movement of shovel 100 when it estimates the location of the buried water pipe U1. Figure 5 shows a cross-section of the ground where the water pipe U1 is buried. In Figure 5, the ground ES before excavation is shown by a dashed line.

[0050] First, the operator of the shovel 100 switches the operating mode of the shovel 100 to SA (semi-automatic) mode by operating the operating mode switch 75. The operator then manually operates the operating device 26 to move the tip of the bucket 6 to the desired position (first position PS1). After moving the tip of the bucket 6 to the desired position, the operator activates the buried object estimation function mode by operating the buried object estimation mode switch 76.

[0051] In SA (semi-automatic) mode, the controller 30 autonomously operates the attachment AT. Specifically, the controller 30 automatically extends and retracts at least one of the boom cylinder 7, arm cylinder 8, and bucket cylinder 9 to move a predetermined part of the attachment AT along a preset target trajectory TP. However, even in the buried object estimation function mode, the controller 30 may not autonomously operate the attachment AT, but instead operate the attachment AT according to the operation of the control device 26 by the operator.

[0052] The controller 30 automatically operates the attachment AT so that the tip of the bucket 6 moves along a preset first target trajectory TP1 (dash-dotted line) during excavation work. As the tip of the bucket 6 moves along the first target trajectory TP1, the excavation reaction force calculation unit 31 repeatedly calculates the excavation reaction force based on the output of the attitude detection device M1 and the output of the excavation pressure sensor S1. The buried object estimation unit 32 also repeatedly estimates the presence or absence of buried objects based on the excavation reaction force calculated by the excavation reaction force calculation unit 31.

[0053] When the tip of the bucket 6 reaches the end of the first target trajectory TP1, the controller 30 stops the autonomous operation of the attachment AT. This means that the buried object estimation unit 32 did not detect any buried objects until the tip of the bucket 6 reached the end of the first target trajectory TP1.

[0054] Subsequently, the operator stops the buried object estimation function by operating the buried object estimation mode switch 76. The controller 30 may automatically stop the buried object estimation function if the operator manually operates the operating device 26 to perform a boom raising operation or a boom raising and slewing operation. After performing a soil removal operation or a boom lowering and slewing operation, the operator moves the tip of the bucket 6 to the next desired position (second position PS2) in order to perform the next excavation operation. For example, the second position PS2 is at a depth D1 from the ground ES before excavation is started, and is approximately directly below the first position PS1.

[0055] The operator moves the tip of the bucket 6 to the second position PS2 and then activates the buried object estimation function mode. When the buried object estimation function mode is activated, the controller 30 automatically operates the attachment AT so that the tip of the bucket 6 moves along the preset second target trajectory TP2 (dash-dotted line). As the tip of the bucket 6 moves along the second target trajectory TP2, the excavation reaction force calculation unit 31 repeatedly calculates the excavation reaction force based on the output of the attitude detection device M1 and the output of the excavation pressure sensor S1. The buried object estimation unit 32 also repeatedly estimates the presence or absence of buried objects based on the excavation reaction force calculated by the excavation reaction force calculation unit 31.

[0056] When the tip of bucket 6 reaches the end of the second target trajectory TP2, the controller 30 stops the autonomous operation of attachment AT. This means that the buried object estimation unit 32 did not detect any buried objects until the tip of bucket 6 reached the end of the second target trajectory TP2.

[0057] Subsequently, the operator performs the soil removal operation and boom lowering and slewing operation as described above, and then moves the tip of the bucket 6 back to the next desired position (third position PS3) to perform the next excavation operation. The third position PS3 is located at a depth D2 from the first exposed surface and is approximately directly below the second position PS2.

[0058] The operator moves the tip of the bucket 6 to the third position PS3, and then activates the buried object estimation function. Once the buried object estimation function is activated, the controller 30 automatically operates the attachment AT so that the tip of the bucket 6 moves along the pre-set third target trajectory TP3 (dashed line).

[0059] As the tip of the bucket 6 moves along the third target trajectory TP3, the excavation reaction force calculation unit 31 repeatedly calculates the excavation reaction force based on the output of the attitude detection device M1 and the output of the excavation pressure sensor S1. The buried object estimation unit 32 also repeatedly estimates the presence or absence of buried objects based on the excavation reaction force calculated by the excavation reaction force calculation unit 31.

[0060] For example, the buried object estimation unit 32 estimates the presence of a buried object when the tip of the bucket 6 reaches the fourth position PS4. The fourth position PS4 is located at a depth D3 from the second exposed surface and is on the third target trajectory TP3. This fourth position PS4 is the position where the distance between the tip of the bucket 6 and the buried object, which is a water pipe U1 located along the third target trajectory TP3, is valued as AD1.

[0061] Here, referring to Figure 6, we will explain the process of estimating the presence or absence of buried objects based on the excavation reaction force. Figure 6 is a graph showing the relationship between the excavation reaction force F and the approach distance AD. The vertical axis of Figure 6 corresponds to the excavation reaction force F calculated by the excavation reaction force calculation unit 31, and the horizontal axis of Figure 6 corresponds to the approach distance AD. The approach distance AD ​​is the distance between the current position of the tip of the bucket 6 and the buried object (water pipe U1) in the direction along the target trajectory TP. In Figure 6, the approach distance AD ​​decreases from left to right on the horizontal axis until the approach distance AD ​​becomes zero. That is, when the approach distance AD ​​is value AD0, the tip of the bucket 6 is located further from the water pipe U1 than when the approach distance AD ​​is value AD1.

[0062] Specifically, the dashed line in Figure 6 illustrates a learning model that serves as the basis for the excavation reaction force. That is, the learning model is represented by a function that expresses the relationship between the change in approach distance AD ​​during excavation work and the excavation reaction force. The function of the learning model may be a linear function, a polynomial function, a logarithmic function, etc. This learning model has learned the excavation reaction force that the attachment AT receives from the excavation target (in the ground) when there are no buried objects in the excavation target. The learning model increases the excavation reaction force F as the approach distance AD ​​decreases. This is because the amount of soil taken into the bucket 6 increases as the bucket 6 approaches the machine body (upper rotating body 3), and therefore the excavation reaction force increases.

[0063] The buried object estimation unit 32 compares this learned model with the excavation reaction force repeatedly calculated during the excavation work to estimate the presence or absence of buried objects. For example, in Figure 5, the excavation reaction force when the tip of the bucket 6 moves along the third target trajectory TP3 is shown by the solid line in Figure 6.

[0064] In this case, within the approach distance AD ​​range of AD0 to AD1, the excavation reaction force fluctuates in roughly accordance with the learned model. However, when the approach distance AD ​​exceeds AD1, the excavation reaction force deviates from the learned model and increases significantly. This is because the water pipe U1 is located in the third target trajectory TP3 of bucket 6, and as the tip of bucket 6 approaches the water pipe U1, the soil between bucket 6 and the water pipe U1 is compressed between them.

[0065] The buried object estimation unit 32 estimates the presence or absence of a buried object (water pipe U1) by, for example, calculating the difference between the learned model and the actual excavation reaction force, and determining whether the difference between the learned model and the actual excavation reaction force exceeds a threshold. Figure 6 shows an example where the difference between the learned model and the actual excavation reaction force exceeds a threshold at a position AD2 where the approach distance AD ​​is. In other words, the position AD2 corresponds to the timing in Figure 5 when the tip of the bucket 6 reaches the fourth position PS4 on the third target trajectory TP3. This position AD2 is, for example, about 20 cm away from the buried object. Note that the position AD3 where the approach distance AD ​​is is the position where the buried object (water pipe U1) is located.

[0066] However, the method for comparing the learned model with the actual excavation reaction force is not limited to the above and various methods can be used. For example, the buried object estimation unit 32 may compare the average rate of increase of the actual excavation reaction force with respect to the approach distance AD ​​and the average rate of increase of the learned model. If the average rate of increase of the actual excavation reaction force exceeds the average rate of increase of the learned model above a predetermined level, it can be estimated that there is a buried object.

[0067] Next, the buried object estimation system 200, which generates a learning model for detecting the buried objects mentioned above and provides the learning model, will be explained with reference to Figure 7. Figure 7 is a block diagram of the buried object estimation system 200.

[0068] The buried object estimation system 200 includes an information processing device 210, a communication network 220, a cloud server 230, and a controller 30 for the excavator 100.

[0069] The information processing device 210 is an information processing unit that creates a learning model used for detecting buried objects with the excavator 100 described above. This information processing device 210 can be a well-known computer equipped with a processor 211, memory 212, input / output interface 213, and communication interface 214. Furthermore, the information processing device 210 may be composed of one computer or multiple computers.

[0070] The information processing device 210 is installed, for example, in a company or organization that manufactures, manages, or uses work machinery such as an excavator 100. The information processing device 210 manages information such as the usage status and condition of multiple excavators 100 or other work machinery, and provides information to the excavators 100 or other work machinery. The information processing device 210 is connected to a cloud server 230 via a communication network 220, and provides and retrieves information via the cloud server 230. Alternatively, the information processing device 210 may be configured to communicate information with (or directly with) a mobile terminal such as a computer, smartphone, or tablet held by a worker via the cloud server 230.

[0071] The communication network 220 of the buried object estimation system 200 can be the Internet or a dedicated line such as Ethernet. As described above, the buried object estimation system 200 according to the embodiment sends and receives information between the information processing device 210 and each excavator 100 by accessing the cloud server 230 via the communication network 220. For example, when various excavator information, including the excavation reaction force when the excavator 100 performs excavation work at the work site, is uploaded to the cloud server 230, the information processing device 210 retrieves this information from the cloud server 230 at an appropriate time.

[0072] Conversely, when the information processing device 210 uploads work information (not shown) to the cloud server 230, the shovel 100 downloads the work information from the cloud server 230 at an appropriate time before starting work. This work information may include location information of the work site, information on a 3D or 2D model of the work site, information on the work machinery used at the work site, and a learning model for detecting buried objects.

[0073] Furthermore, the buried object estimation system 200 may be configured to communicate information directly (or via another computer) between the information processing device 210 and the excavator 100 without going through the cloud server 230. Alternatively, the operator may store information about the excavator 100 in an external storage device and connect this external storage device to the information processing device 210 to provide the information about the excavator 100 to the information processing device 210. The same applies to the information provided from the information processing device 210 to the excavator 100.

[0074] Next, the learning model for detecting buried objects generated by the information processing device 210 will be explained with reference to Figure 8. Figure 8 is a block diagram showing the functional parts of the information processing device 210 when generating the learning model. The information processing device 210 generates a learning model for detecting buried objects, for example, by an unsupervised learning method. As an example, the information processing device 210 constructs a sensor data storage unit 215, a learning data extraction unit 216, a reaction force model generation unit 217, an optimization unit 218, an evaluation unit 219, etc., internally by having the processor 211 read and execute a program stored in the memory 212.

[0075] The sensor data storage unit 215 stores information acquired by various sensors of the shovel 100. In this process, the information processing device 210 links information from shovels 100 of the same model and stores the sensor values ​​and time information in the sensor data storage unit 215. The information stored in the sensor data storage unit 215 includes information related to excavation reaction force (sensor data from the excavation pressure sensor S1 and attitude detection device M1). The information stored in the sensor data storage unit 215 may also include information about the external environment of the shovel 100 detected by the object detection device 70 (camera, LiDAR, etc.) of the shovel 100. Furthermore, when providing operational support for the shovel 100, the sensor data storage unit 215 may store simulation data that simulates the operation of the shovel 100, including the excavation work of the attachment AT. This simulation data may be data actually used by the shovel 100 in excavation work, or it may be data that has been simulated in advance by the information processing device 210 and is planned to be provided to the shovel 100.

[0076] The learning data extraction unit 216 is a machine learning input layer that extracts data from various types of information stored in the sensor data storage unit 215 to be used in generating a learning model for detecting buried objects, and provides it to the reaction force model generation unit 217. The data used to generate this learning model includes the above-mentioned information related to excavation reaction force (such as the pressure from the excavation pressure sensor S1), information related to the posture of the shovel 100, and information from the object detection device 70.

[0077] The reaction force model generation unit 217 generates a learning model for detecting buried objects based on the data provided by the learning data extraction unit 216. Here, the excavation targets for the shovel 100 have different soil types, such as soil, sand, soil and sand, hard bedrock, and soft bedrock. Since the excavation reaction force will also change depending on the soil type of the excavation target, the reaction force model generation unit 217 generates a learning model for each different soil type. Therefore, the learning model is generated to include a soil learning model used when excavating soil, a sand learning model used when excavating sand, a gravel learning model used when excavating gravel, and so on.

[0078] For example, the reaction force model generation unit 217 uses the extracted excavation reaction force features to recognize the pattern of sequential data of excavation reaction force using a time-series analysis neutral network. The reaction force model generation unit 217 also extracts terrain data features based on camera and LiDAR information (imaging information) or construction data from when the shovel 100 actually performed the excavation work. The reaction force model generation unit 217 then generates a reaction force prediction model by combining the excavation reaction force pattern and terrain data features in a fully connected layer. The reaction force prediction model is, for example, a function or table information that shows the change in excavation reaction force in response to the passage of time or the change in approach distance AD. The reaction force prediction model is also calculated for each type of soil (soil, sand, soil and sand, etc.) based on the terrain data features.

[0079] The optimization unit 218 optimizes the reaction force prediction model and generates a learning model (soil learning model, sand learning model, soil and sediment learning model, etc.) based on the input of one or more reaction force prediction models generated by the reaction force model generation unit 217. For example, the optimization unit 218 performs Bayesian optimization processing based on the reaction force prediction model, terrain data features, or other excavation work conditions. The optimization unit 218 can obtain a highly accurate learning model by using multiple reaction force prediction models generated by the reaction force model generation unit 217. Note that the optimization processing of the optimization unit 218 is not limited to Bayesian optimization processing; various methods such as grid search may be employed.

[0080] Furthermore, the evaluation unit 219 evaluates the quality of the generated learning model and outputs the evaluation result to the reaction force model generation unit 217. For example, the evaluation unit 219 may evaluate the learning model based on information about the presence or absence of actual buried objects confirmed by the exposure work described later. Alternatively, the evaluation of the learning model may be performed by optimizing hyperparameters that can be set or adjusted by the user of the information processing device 210. This makes it possible to generate a learning model that satisfies any requirements, such as improving the performance of the learning model or enabling the smooth acquisition of the learning model.

[0081] The information processing device 210 generates one or more learning models (soil learning model, sand learning model, soil and sediment learning model) for detecting buried objects through the processing described above, and then uploads these learning models to the cloud server 230. As a result, the excavator 100 at the work site can download the learning models by accessing the cloud server 230.

[0082] The controller 30 of the shovel 100 uses a downloaded learning model during excavation to estimate the presence or absence of buried objects during excavation, as described above. In this process, the shovel 100 can also estimate buried objects based on soil type by selecting a learning model from several types of learning models (soil learning model, sand learning model, soil and sand learning model) that corresponds to the soil type being excavated. For example, when the operator of the shovel 100 recognizes the soil type being excavated, they input the soil type by operating the input unit 42 of the display device 40. This allows the controller 30 to select the learning model to use from among several types of learning models.

[0083] Alternatively, the controller 30 may be configured to automatically select the soil type to be excavated based on imaging information captured by the camera of the object detection device 70 of the shovel 100. This makes it possible to immediately switch to an appropriate learning model, for example, even if the soil type changes during excavation and the operator does not notice.

[0084] However, in the actual operational phase, the hardness and viscosity of the material excavated by the shovel 100 will change due to the influence of temperature, humidity, and the resulting soil moisture content. Therefore, if the shovel 100 simply uses its existing learning model, it may not be able to detect buried objects because the excavation reaction force will not be appropriate for the actual work site. For this reason, the controller 30 of the shovel 100 checks the condition of the material to be excavated by actually excavating it at the work site, and then tunes (adjusts) the learning model to match the actual material to be excavated.

[0085] Figure 9 is an explanatory diagram illustrating how the shovel 100 acquires, tunes, and uses the learned model during excavation work. As shown in Figure 9, the shovel 100 performs preliminary excavation work in areas of the work site where it has been determined that there are no buried objects. During the preliminary excavation work, the shovel 100 detects the actual excavation reaction force using various sensors (attitude detection device M1, excavation pressure sensor S1).

[0086] The controller 30 can determine whether tuning of the learning model is necessary by comparing the degree of change in the excavation reaction force at the actual work site with the learning model it possesses. If it determines that tuning of the learning model is necessary, the controller 30 tunes the learning model using the actual excavation reaction force at the work site. For example, in learning model tuning, the learning model (predicted excavation reaction force), which is represented by a linear function, polynomial function, logarithmic function, etc., is fitted to data plotting the actual excavation reaction force at the work site. As a result, the learning model is appropriately corrected to a model that shows the change in excavation reaction force according to the actual work site. Hereinafter, the corrected learning model will also be referred to as the corrected learning model.

[0087] Tuning for the work site may be performed only on a pre-selected learning model (e.g., a soil learning model), or it may be performed on all of the multiple types of learning models available (e.g., a soil learning model, a sand learning model, and a gravel learning model). At the work site, the soil type may change during the excavation process. Therefore, if the controller 30 calculates corrected learning models for multiple types of learning models, it can smoothly switch between models.

[0088] After performing preliminary excavation work, the controller 30 uses this corrective learning model to perform the excavation work shown in Figure 5. That is, the shovel 100 performs the excavation work in SA (semi-automatic) mode, autonomously operating the attachment AT, during which the excavation pressure sensor S1 and attitude detection device M1 acquire their respective detection values. The excavation reaction force calculation unit 31 of the controller 30 calculates the excavation reaction force based on these sensors.

[0089] The buried object estimation unit 32 of the controller 30 then compares the calculated excavation reaction force with the corrected learning model to estimate the presence or absence of buried objects during excavation work by the attachment AT. As described above, because the corrected learning model is tuned according to the actual work site, the controller 30 can accurately estimate the presence of buried objects inside the excavation target.

[0090] The excavator 100 may autonomously control the movement of the attachment AT to prevent it from coming into contact with buried objects when it estimates the presence of buried objects in the excavation area. Specifically, the controller 30 may disable the arm closing operation and stop the operation of the attachment AT based on the detection of buried objects. Alternatively, the controller 30 may automatically extend or retract the boom cylinder 7 to raise the boom 4, thereby changing the trajectory so that the tip of the bucket 6 does not come into contact with buried objects.

[0091] Furthermore, when the controller 30 estimates the presence of buried objects in SA (semi-automatic) mode or M (manual) mode, it is preferable to alert the operator to prevent the attachment AT from approaching the buried objects. For example, the controller 30 can use the sound output device 45 to inform the operator of the distance between the tip of the bucket 6 and the buried object. In this case, the controller 30 should shorten the interval between intermittent sounds as the distance to the buried object decreases. Also, if the tip of the bucket 6 approaches the buried object, the controller 30 may issue a loud alarm to the operator via the sound output device 45.

[0092] If the controller 30 of the shovel 100 estimates the presence of buried objects, the presence or absence of buried objects may be confirmed by performing an exposure operation at the work site. For example, in an exposure operation, workers at the work site use tools to excavate the area where buried objects are estimated to be located, thereby exposing the buried objects from the excavation target. Alternatively, in an exposure operation, the operator of the shovel 100 may confirm the presence or absence of buried objects by carefully operating the attachment AT to excavate the area where buried objects are estimated to be located. When the presence or absence of buried objects is actually confirmed during the exposure operation at the location where the presence or absence of buried objects is estimated, the worker may transmit this information to the controller 30 of the shovel 100 and / or the cloud server 230 using their information terminal device. As a result, the controller 30 of the shovel 100 and / or the cloud server 230 store the data of the excavation reaction force when buried objects were estimated and the result of the exposure operation regarding the presence or absence of buried objects.

[0093] If information is received indicating the presence of buried objects during the exposure work, the controller 30 or cloud server 230 can recognize that the detection of buried objects using the learning model (corrected learning model) functioned correctly. On the other hand, if information is received indicating that there were no buried objects during the exposure work, the controller 30 or cloud server 230 can recognize that the detection of buried objects using the learning model (corrected learning model) was incorrect. In this case, the controller 30 can make new corrections to the learning model (or the threshold for determining buried objects) based on the information obtained. As a result, the shovel 100 can execute the buried object estimation function mode using the newly corrected corrected learning model or threshold.

[0094] Furthermore, when the controller 30 executes the buried object estimation function mode, it may display information related to the estimation of buried objects on the user interface display device 40. Figure 10 is an example of image information 400 that is displayed on the display device 40 when the buried object estimation function mode is executed.

[0095] Image information 400 may be displayed automatically when the buried object estimation function mode is executed, regardless of whether SA (semi-automatic) mode or M (manual) mode is selected. Image information 400 includes a mode selection display unit 410, a selected model display unit 420, an abnormality indicator 430, model parameters 440, a model accuracy display unit 450, a reaction force display unit 460, an environment display unit 470, an external connection setting display unit 480, and a detailed setting display unit 490.

[0096] The mode selection display unit 410 allows the operator to select from multiple learning modes, a tuning mode for adjusting the learning mode, or a buried object estimation function mode for performing actual excavation work. For example, the operator turns on the mode selection display unit 410 by operating a switch provided on lever 26A or lever 26B of the operating device 26. Upon being turned on, the mode selection display unit 410 displays a selection window with various modes, allowing the operator to select a mode of their choice.

[0097] The selected model display unit 420 displays various learning models (soil learning model, sand learning model, gravel learning model, etc.) selected by the operator. After the tuning mode is executed, the selected model display unit 420 may also display an indication that tuning is complete (a pictogram indicating the corrected learning model, etc.).

[0098] The anomaly indicator 430 converts the probability of the presence of buried objects into an anomaly score and notifies the user when performing buried object estimation during actual excavation work. For example, the anomaly score is calculated by the ratio of the difference in excavation reaction force relative to the learned model to the threshold value. In the illustrated example, the anomaly score is notified by a circular graph and numerical values. The color of the circular graph may change according to the anomaly score, such as green for an anomaly score in the range of 0% to 50%, yellow for 50% to 80%, and red for 80% to 100%.

[0099] The controller 30 can notify the operator of the buried object via the abnormality indicator 430 if the abnormality level is high (in other words, if the difference from the threshold is high). If the abnormality level is high, the controller 30 may automatically stop the operation of the attachment AT or change its trajectory as described above. Alternatively, the operator may monitor the abnormality level and make a decision, for example, to stop the operation of the attachment AT at 70% or 90%.

[0100] Model parameter 440 displays hyperparameters within the learning model (corrected learning model) that can be set by the operator. Examples of these hyperparameters include the soil type of the work site, the detection accuracy in the buried object estimation function, and the operating mode of the shovel 100.

[0101] The model accuracy display unit 450 evaluates the currently applied learning model (corrected learning model) and displays the evaluation results. For example, the evaluation of the learning model increases the score if there are no buried objects when the system is detecting a state without buried objects, and decreases the score if there are buried objects when the system is detecting a state without buried objects. Alternatively, the evaluation of the learning model increases the score if there are buried objects when the system is detecting a state with buried objects, and decreases the score if there are no buried objects when the system is detecting a state with buried objects.

[0102] The reaction force display unit 460 displays the learning model (corrected learning model) actually used in excavation work and the excavation reaction force detected by each sensor of the shovel 100 (attitude detection device M1, excavation pressure sensor S1) in a graph format. For example, the graph shows the change in excavation reaction force according to the change in excavation distance, with the excavation distance on the horizontal axis and the excavation reaction force on the vertical axis. Alternatively, the graph shows the change in excavation reaction force according to the passage of time, with the excavation time on the horizontal axis and the excavation reaction force on the vertical axis. This allows the operator to operate the shovel 100 while actually recognizing the changes in the learning model and the actual excavation reaction force.

[0103] The environmental display unit 470 displays environmental factor variables. Examples of these environmental factor variables include the date and time, weather conditions, and operator ID when the shovel 100 performs excavation work.

[0104] The external connection settings display unit 480 displays the communication environment between the controller 30 and the external communication network. For example, when connecting to the cloud server 230, the external connection settings display unit 480 allows the connection status with the cloud server 230 to be set based on the operator's actions.

[0105] The detailed settings display unit 490 allows the operator to change settings other than those mentioned above when executing the buried object estimation function.

[0106] By displaying the above image information 400 on the display device 40, the operator of the shovel 100 can set the learning model for the buried object estimation function and also clearly understand the relationship between the learning model and the excavation reaction force when actually performing excavation work. In particular, the shovel 100 according to this embodiment can inform the operator that the presence of a buried object has been determined because the excavation reaction force during excavation work does not conform to the learning model, by displaying the learning model.

[0107] The excavator 100 according to this embodiment is basically configured as described above, and its operation will be explained below with reference to the flowcharts in Figures 11(A) and 11(B). Figure 11(A) is a flowchart of the processing flow before actually performing excavation work. Figure 11(B) is a flowchart of the buried object estimation method when actually performing excavation work.

[0108] The controller 30 of the shovel 100 executes steps S101 to S109 in the buried object estimation method shown in Figures 11(A) and 11(B).

[0109] The controller 30 accesses the cloud server 230 and retrieves various learning models stored in the cloud server 230 (step S101). Note that the pattern for retrieving learning models is not limited to this; the controller 30 may also retrieve the learning models by connecting a memory device containing the learning models to the controller 30. Alternatively, the controller 30 may store the learning models in advance, in which case step S101 may be omitted.

[0110] The controller 30 prompts the operator of the shovel 100 to tune the learning model at the actual work site and tunes the learning model by having the operator perform preliminary excavation work (step S102). By performing this tuning, the controller 30 can obtain a corrected learning model that corresponds to the actual work site. However, the operator may choose not to perform tuning, in which case the learning model held by the controller 30 will be used as is. Alternatively, for example, if multiple shovels 100 are used at the same site, tuning may be performed on one shovel 100 and the information of the tuning result (corrected learning model) may be saved to the cloud server 230. Then, the other shovels 100 can obtain the tuning result of the one shovel 100 from the cloud server 230. This eliminates the need for multiple shovels 100 to perform tuning. Alternatively, tuning may be performed on multiple shovels 100, the tuning results may be saved to the cloud server 230, the learning model may be corrected using the multiple tuning results in the cloud server 230, and then sent to each shovel 100. This will also make it possible to improve the accuracy of tuning the learning model for each of the 100 shovels.

[0111] Then, the shovel 100 moves on to excavation work at the actual work site. The controller 30 determines whether the operator has selected the operating mode and the buried object estimation function mode (step S103). If the buried object estimation function mode has been selected (step S103: YES), the process proceeds to step S104.

[0112] In step S104, the controller 30 reads out the learning model (corrected learning model) it possesses. At this time, the controller 30 also displays the image information 400 described above so that the operator can visually confirm which learning model to use.

[0113] Then, the shovel 100 operates the attachment AT based on the control of the controller 30 or the operator's input to perform the excavation work on the target to be excavated (step S105).

[0114] In this excavation operation, the controller 30 acquires sensor data from the attitude detection device M1 and the excavation pressure sensor S1, calculates the excavation reaction force from this data, and compares the calculated excavation reaction force with the learned model to determine whether or not there are buried objects (step S106). If it is determined that there are no buried objects (step S106: NO), the process proceeds to step S107.

[0115] In step S107, the controller 30 determines whether the excavation work is complete. If the excavation work is not to be completed (step S107: NO), the process returns to step S105 and continues the excavation work. On the other hand, if the excavation work is to be completed (step S107: YES), the current processing flow is terminated.

[0116] Furthermore, if it is determined in step S106 that there are buried objects (step S106: YES), the process proceeds to step S108. In step S108, the controller 30 stops the operation of attachment AT or changes its trajectory as described above, and displays information about the presence of buried objects on the display device 40. This allows the operator to recognize that the operation of the shovel 100 has been stopped because there are buried objects in the excavation target.

[0117] In this case, the operator of the shovel 100 may notify workers in the vicinity of the work site that buried objects have been detected, allowing the workers to carry out the work of exposing the buried objects. The workers then provide feedback to the controller 30 or cloud server 230 regarding whether or not buried objects were actually found (step S109).

[0118] This allows the controller 30 to evaluate the learning model (corrected learning model) it is using based on information from the worker about the actual presence or absence of buried objects, and thereby correct the learning model or threshold used in the next excavation operation. For example, if no buried objects are found, the controller 30 can correct the function of the learning model (corrected learning model) by taking into account the change in the excavation reaction force in this operation.

[0119] As described above, the shovel 100 and the buried object estimation method can accurately detect buried objects in the excavation target by performing excavation work using the learned model acquired in the information processing device 210. Furthermore, the shovel 100 uses a corrected learned model that is suitable for the actual work site by tuning the learned model acquired from the cloud server 230 at the actual work site. As a result, the shovel 100 can estimate the presence or absence of buried objects with greater accuracy.

[0120] Next, with reference to Figure 12, an example configuration of the operation system SYS according to another embodiment will be described. Figure 12 is a schematic diagram showing an example configuration of the operation system SYS. The operation system SYS includes an excavator 100, a remote control room RC, and a management center MC. Note that the excavator 100 shown in Figure 12 has the same configuration as the excavator 100 shown in Figure 1.

[0121] The excavator 100, the remote control room RC, and the management center MC are connected to each other so that data can be sent and received via a communication network NW. Alternatively, the excavator 100, the remote control room RC, and the management center MC may be connected to each other directly so that data can be sent and received without using the communication network NW. In the illustrated example, the excavator 100 transmits information about the work site to the remote control room RC. This allows the remote operator RO in the remote control room RC to understand the situation at the work site based on the information from the excavator 100.

[0122] Shovel 100 is equipped with sensors capable of recognizing the position and shape of objects present at the work site in three dimensions. For example, shovel 100 is equipped with a spatial recognition device. Therefore, shovel 100 can transmit the results of three-dimensional measurements of the work site to the remote control room RC.

[0123] The spatial recognition device is a device for recognizing the space surrounding the shovel 100. In the illustrated example, the spatial recognition device is a LiDAR. The LiDAR measures, for example, the distance between each of more than 1 million points within the monitoring range and the LiDAR itself. Note that the spatial recognition device can be any device capable of measuring the distance to an object. For example, the spatial recognition device may be a stereo camera, or a combination of an imaging device S6 and a ranging device such as a millimeter-wave radar.

[0124] The operating system SYS may include one or more excavators 100. If it includes multiple excavators 100, the remote operator RO of a particular excavator 100 can obtain information about the work sites obtained by that particular excavator 100, as well as information about the work sites obtained by the other one or more excavators 100.

[0125] The remote control room RC is equipped with a communication device T2, a remote controller RCC, an operating device 26E, an operating sensor 43, a display device D1E, an internal sound output device SP2E, and an internal sound collection device M2E. The remote control room RC also has an operating seat DS where the remote operator RO sits to remotely control the shovel 100.

[0126] The communication device T2 is configured to communicate with the communication device T1 attached to the shovel 100.

[0127] The remote controller RCC is a computing device that performs various calculations. In this embodiment, the remote controller RCC is composed of a microcomputer including a CPU and memory. The various functions of the remote controller RCC are realized by the CPU executing a program stored in memory.

[0128] The display device D1E is a device capable of displaying various types of information. The display device D1E displays images based on information transmitted from the shovel 100 so that the remote operator RO in the remote control room RC can visually inspect the area around the shovel 100. In the illustrated example, the display device D1E is a liquid crystal display that displays images captured by the imaging device S6 mounted on the shovel 100. The display device D1E may also be a display or projector that enables naked-eye stereoscopic viewing, or it may be a VR goggle or the like.

[0129] The internal sound output device SP2E is a device capable of outputting sound information. The internal sound output device SP2E outputs sound based on information transmitted from the shovel 100 so that the remote operator RO in the remote control room RC can hear the sounds emitted at the work site.

[0130] The operating device 26E is equipped with an operation sensor 43 for detecting the operation of the operating device 26E. The operation sensor 43 is, for example, a tilt sensor that detects the tilt angle of the operating lever, or an angle sensor that detects the oscillation angle of the operating lever around its pivot axis. The operation sensor 43 may also consist of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor 43 outputs information regarding the detected operation of the operating device 26E to the remote controller RCC. The remote controller RCC generates an operation signal based on the received information and transmits the generated operation signal to the shovel 100. The operation sensor 43 may be configured to generate the operation signal. In this case, the operation sensor 43 may output the operation signal to the communication device T2 without going through the remote controller RCC. With this configuration, the remote operator RO can remotely operate the shovel 100 from the remote control room RC.

[0131] The control center MC is a facility equipped with various devices for managing the remote operation of the excavator 100, either by the excavator 100 located at the work site or by the remote operator RO in the remote control room RC. In the illustrated example, the control center MC is installed at a distance from both the excavator 100's work site and the remote control room RC. The control center MC is equipped with a control device 300, an internal sound output device SP2C, and an internal sound collection device M2C.

[0132] The management device 300 is an example of a control unit, and is, for example, a server computer (a so-called cloud server) or an edge server. The management device 300 is typically a fixed terminal device, but may also be a portable terminal device (for example, a laptop computer, tablet, or smartphone).

[0133] Even with the above operating system SYS, the shovel 100 can estimate buried objects using a learned model. When the remote operator RO performs excavation work at the work site, the controller 30 of the shovel 100 or the remote controller RCC reads the learned model described above and compares it with the excavation reaction force. This makes it possible to accurately determine the presence or absence of buried objects during excavation work at the work site and perform the excavation work accordingly.

[0134] In this embodiment, the buried object estimation system 200 is configured to have an information processing unit for generating a learning model located in an external information processing device 210 of the shovel 100. However, this information processing unit may also be installed in the controller 30 of the shovel 100; in other words, the buried object estimation system 200 may be realized using only the configuration of the shovel 100.

[0135] The technical concept and effects of this disclosure, as described in the embodiments above, are described below.

[0136] A first aspect of this disclosure is a buried object estimation system 200 for estimating the possibility of buried objects being present in an excavation target, in an excavator 100 including a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1, an attachment AT mounted on the upper rotating body 3 for excavating an excavation target, and a control unit (controller 30) for controlling the operation of the attachment AT, the system comprising an information processing unit (information processing device 210) that acquires information related to the excavation reaction force during the excavation work of the attachment AT, learns a learning model for detecting buried objects, and transmits the learning model to the control unit, the control unit estimates the presence or absence of buried objects in the excavation target based on the excavation reaction force acquired in the actual excavation work and the learning model.

[0137] As described above, the buried object estimation system 200 can accurately estimate buried objects during excavation work by using a learning model that has learned information related to excavation reaction force. In other words, compared to a configuration that simply monitors the excavation reaction force acquired from the sensors of the excavator 100, the buried object estimation system 200 can estimate the presence or absence of buried objects with a high probability from changes in excavation reaction force according to the learning model. Moreover, the buried object estimation system 200 can improve its learning model by accumulating sensor data each time excavation work is performed. As a result, it becomes possible to estimate buried objects with greater accuracy during excavation work by the excavator 100.

[0138] Furthermore, the control unit (controller 30) acquires information related to the condition of the excavation target during the actual excavation work and adjusts its learning model based on this information. As a result, the buried object estimation system 200 can adjust its learning model according to the condition of the excavation target at the actual work site (soil type, moisture content, viscosity, weather, etc.). Consequently, it can estimate the presence or absence of buried objects according to the condition of the excavation target, and estimate buried objects with greater accuracy.

[0139] Furthermore, the information related to the excavation target is the excavation reaction force information obtained when the target to be excavated was excavated in advance. This allows the buried object estimation system 200 to adjust its learning model to suit the actual work site based on the excavation reaction force information obtained when the target to be excavated was excavated in advance.

[0140] Furthermore, the information processing unit (information processing device 210) generates multiple learning models for each type of soil to be excavated, and the control unit (controller 30) selects the learning model to be used based on the soil type to be excavated during the actual excavation work. As a result, the shovel 100 can accurately monitor the excavation reaction force during the actual excavation work using the learning model corresponding to the soil type to be excavated.

[0141] Furthermore, the control unit (controller 30) or information processing unit (information processing unit 210) acquires information confirming the presence or absence of buried objects after determining the presence or absence of buried objects during actual excavation work, and evaluates the learning model based on the information confirming the presence or absence of buried objects. As a result, the buried object estimation system 200 can improve its learning model based on the presence or absence of buried objects confirmed by the exposure work, thereby increasing the accuracy of subsequent buried object estimations.

[0142] Furthermore, the control unit (controller 30) calculates the difference between the excavation reaction force of the actual excavation work and that of the learned model. If the difference is less than a threshold, it determines that there are no buried objects, and if the difference is greater than or equal to the threshold, it determines that there are buried objects. As a result, the buried object estimation system can easily and accurately estimate the presence or absence of buried objects.

[0143] Furthermore, the control unit (controller 30) stops the operation of the attachment or changes its trajectory when it determines the presence of buried objects. This allows the shovel 100 to avoid contact between the attachment AT and the estimated buried objects.

[0144] Furthermore, it is equipped with a display device 40 that shows the learning model used in actual excavation work. This allows the operator of the shovel 100 to clearly recognize the learning model used in estimating buried objects.

[0145] Furthermore, the display device 40 displays image information 400 that includes information related to the results of the estimation of the presence or absence of buried objects. This allows the operator of the shovel 100 to immediately recognize the results of the estimation of the presence or absence of buried objects and take necessary actions.

[0146] Furthermore, a second aspect of this disclosure is a shovel 100 including a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1, an attachment AT mounted on the upper rotating body 3 for excavating an excavation target, and a control unit (controller 30) for controlling the operation of the attachment AT, wherein the control unit acquires information related to the excavation reaction force during the excavation work of the attachment AT and acquires a learning model for detecting buried objects, and estimates the presence or absence of buried objects in the excavation target based on the excavation reaction force acquired in the actual excavation work and the learning model. Even in this case, buried objects in the excavation target can be accurately estimated during the excavation work of the shovel 100.

[0147] The buried object estimation system 200 and the shovel 100 according to the embodiments disclosed herein are illustrative and not restrictive in all respects. The embodiments can be modified and improved in various ways without departing from the scope and spirit of the appended claims. The matters described in the above embodiments can be otherwise configured and combined in a non-consistent manner. [Explanation of Symbols]

[0148] 1. Lower running body 3. Upper rotating body 30 controllers 40 Display device 100 Shovel 200 Buried Object Estimation System 210 Information Processing Device 400 Image Information AT attachment

Claims

1. Lower running body and An upper rotating body is provided on the lower traveling body so as to be rotatable, An attachment provided on the upper rotating body for excavating the target to be excavated, An excavator including a control unit for controlling the operation of the attachment, wherein a buried object estimation system estimates the possibility of buried objects being present in the excavation target, The system includes an information processing unit that acquires information related to the excavation reaction force during the excavation work of the attachment, learns a learning model for detecting the buried object, and transmits the learning model to the control unit. The control unit estimates the presence or absence of buried objects in the excavation target based on the excavation reaction force obtained during the actual excavation work and the learned model. Buried object estimation system.

2. The control unit acquires information relating to the state of the target to be excavated during the actual excavation work, and adjusts the learning model it possesses based on the information relating to the state of the target to be excavated. The buried object estimation system according to claim 1.

3. The information relating to the excavation target is information on the excavation reaction force when the excavation target, which will be used for the actual excavation work, was excavated in advance. The buried object estimation system according to claim 2.

4. The information processing unit generates a plurality of learning models for each type of soil to be excavated, The control unit selects the learning model to be used based on the soil type of the target to be excavated in the actual excavation work. A buried object estimation system according to any one of claims 1 to 3.

5. The control unit or the information processing unit, after determining the presence or absence of the buried object in the actual excavation work, acquires information confirming the presence or absence of the buried object, and evaluates the learning model based on the information confirming the presence or absence of the buried object. A buried object estimation system according to any one of claims 1 to 3.

6. The control unit calculates the difference between the excavation reaction force of the actual excavation work and the learned model, determines that there is no buried object if the difference is less than a threshold, and determines that there is a buried object if the difference is equal to or greater than the threshold. A buried object estimation system according to any one of claims 1 to 3.

7. The control unit, upon determining the presence of the buried object, stops the operation of the attachment or changes its trajectory. The buried object estimation system according to claim 6.

8. The system includes a display device that displays the learning model used in the actual excavation work, A buried object estimation system according to any one of claims 1 to 3.

9. The display device displays image information including information related to the results of estimating the presence or absence of the buried object. The buried object estimation system according to claim 8.

10. Lower running body and An upper rotating body is provided on the lower traveling body so as to be rotatable, An attachment provided on the upper rotating body for excavating the target to be excavated, A shovel including a control unit for controlling the operation of the attachment, The control unit, By acquiring information related to the excavation reaction force during the excavation work of the aforementioned attachment, a learning model for detecting buried objects is obtained. Based on the excavation reaction force obtained during the actual excavation work and the learning model, the presence or absence of the buried object in the excavation target is estimated. Shovel.

Images