Buried object inference system and excavator

Abstract

A buried object inference system and an excavator are provided, providing a technique for accurately inferring buried objects in an excavation target during excavation operations. The excavator (100) includes: a lower traveling body (1); an upper slewing body (3) rotatably mounted on the lower traveling body (1); an attachment (AT) mounted on the upper slewing body (3) and used for excavating the target; and a controller (30) for controlling the movement of the attachment (AT). The buried object inference system (200) has an information processing device (210) that learns a learning model for detecting buried objects by acquiring information related to the excavation reaction force of the attachment (AT) during the excavation operation in order to infer the possibility of buried objects in the target, and sends the learning model to the controller (30). The controller (30) infers the presence of buried objects in the target based on the excavation reaction force acquired during the actual excavation operation and the learning model.

Description

Technical Field

[0001] This application claims priority based on Japanese Patent Application No. 2024-225498, filed on December 20, 2024, the entire contents of which are incorporated herein by reference.

[0002] This invention relates to a buried object detection system and an excavator. Background Technology

[0003] Excavators may damage buried objects within the excavation target during excavation operations. Therefore, it is necessary to check the presence of buried objects within the excavation target at the work site. For example, Patent Document 1 discloses an excavation system that acquires ground measurement data using an underground detection device and infers the location of buried objects based on the measurement data using a learned model that has learned the locations of buried objects corresponding to the measured data.

[0004] As mentioned above, during excavation operations, the use of underground detection devices to infer buried objects frequently interrupts the excavation process, thus reducing operational efficiency. Therefore, at the work site, it is required to be able to infer potential buried objects within the excavated area while the excavator is digging.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-43107 Summary of the Invention

[0006] This invention provides a technique for accurately identifying buried objects within an excavated object during excavation operations using an excavator.

[0007] According to one embodiment of the present invention, a buried object inference system is provided, which infers the possibility of the presence of buried objects in an excavation object during excavation. The excavator includes: a lower traveling body; an upper rotating body rotatably disposed on the lower traveling body; an attachment disposed on the upper rotating body and excavating the excavation object; and a control unit that controls the action of the attachment. The buried object inference system includes an information processing unit that acquires information related to the excavation reaction force of the attachment during excavation to learn a learning model for detecting buried objects, and sends the learning model to the control unit. The control unit infers whether buried objects exist in the excavation object based on the excavation reaction force acquired during actual excavation and the learning model.

[0008] Invention Effects According to one implementation, when an excavator is performing an excavation operation, it is possible to accurately deduce the buried objects within the excavated object. Attached Figure Description

[0009] Figure 1 This is a side view of the excavator involved in the implementation method.

[0010] Figure 2 It is a side view that represents the various physical quantities related to the excavation attachments.

[0011] Figure 3 This is an explanatory diagram showing the basic system of an excavator.

[0012] Figure 4 It means that it is carried on Figure 1 A diagram illustrating the structure of an excavator's control system.

[0013] Figure 5 It is a diagram showing the cross-section of the foundation where water pipes are buried.

[0014] Figure 6 It is a graph showing the relationship between the excavation reaction force and the approach distance.

[0015] Figure 7 This is a block diagram representing the detection system.

[0016] Figure 8 It is a block diagram representing the functional parts of the information processing device when generating a learning model.

[0017] Figure 9 This is an illustration of how an excavator uses a learning model during the acquisition, optimization, and excavation operations.

[0018] Figure 10 This is an example of image information displayed on a display device when the buried object inference function mode is executed.

[0019] Figure 11 (A) is a flowchart representing the processing flow before the actual excavation operation is carried out. Figure 11 (B) is a flowchart illustrating the method for inferring buried objects during actual excavation operations.

[0020] Figure 12 This is a schematic diagram representing an example of the structure of an operating system.

[0021] In the diagram: 1-lower traveling body, 3-upper rotating body, 30-controller, 40-display device, 100-excavator, 200-buried object inference system, 210-information processing device, 400-image information, AT-accessories. Detailed Implementation

[0022] Hereinafter, the embodiments for carrying out the present invention will be described with reference to the accompanying drawings. In the drawings, the same symbols are sometimes used to denote the same structural parts and repeated descriptions are omitted.

[0023] Figure 1 This is a side view of the excavator 100 according to the embodiment. The excavator 100 includes a lower traveling body 1 and an upper rotating body 3 rotatably mounted on the lower traveling body 1 via a rotating mechanism 2.

[0024] Furthermore, the excavator 100 is equipped with an accessory AT as an example of an accessory for operation. The accessory AT includes: a boom 4 mounted on the upper slewing body 3, a stick 5 mounted on the front end of the boom 4, and a bucket 6 mounted on the front end of the stick 5. In this specification, for convenience, the side of the upper slewing body 3 where the boom 4 is mounted is designated as the front, and the side where the counterweight is mounted is designated as the rear. The boom 4 is driven by a boom cylinder 7. The stick 5 is driven by a stick cylinder 8. The bucket 6 is driven by a bucket cylinder 9.

[0025] Furthermore, the upper rotating body 3 is equipped with a cockpit 10, an engine 11 or an electric motor, and other power sources. Inside the cockpit 10 are operating devices 26, controllers 30, display devices 40 and sound output devices 45, etc.

[0026] The excavator 100 has a posture detection device M1 for detecting the posture of the attachment AT. The posture detection device M1 is also a detection device for detecting information related to the digging reaction force. Specifically, the posture detection device M1 includes a boom angle sensor M1a, a stick angle sensor M1b, and a bucket angle sensor M1c. For example, the boom angle sensor M1a can be a rotation angle sensor that detects the rotation angle of the boom foot pin, a stroke sensor that detects the stroke amount of the boom cylinder 7, or a tilt (acceleration) sensor that detects the tilt angle of the boom 4. Furthermore, the same sensors can be used for the stick angle sensor M1b and the bucket angle sensor M1c.

[0027] Furthermore, the excavator 100 is equipped with an object detection device 70 in its upper rotating body 3. The object detection device 70 detects objects present around the excavator 100. These objects include, for example, people, animals, vehicles, construction machinery, buildings, or pits. The object detection device 70 is configured by combining one or more of the following: ultrasonic sensors, millimeter-wave radar, camera devices, or infrared sensors. The camera devices include, for example, monocular cameras, stereo cameras, LiDAR, or distance image sensors. Specifically, the object detection device 70 includes a rear camera 70B mounted at the rear end of the upper surface of the upper rotating body 3, a front camera 70F mounted at the front end of the upper surface of the cab 10, a left camera 70L mounted at the left end of the upper surface of the upper rotating body 3, and a right camera 70R mounted at the right end of the upper surface of the upper rotating body 3.

[0028] The object detection device 70 can be configured to detect objects (e.g., people) within a defined area around the excavator 100. For example, the object detection device 70 can be configured to distinguish between detecting people and objects other than people.

[0029] Figure 2 This is a side view showing various physical quantities related to the AT attachment. The boom angle sensor M1a detects the boom angle θ1. The boom angle θ1 is the angle of the line segment P1-P2 connecting the boom foot pin position P1 and the stick connecting pin position P2 on the XZ plane relative to the horizontal line. The stick angle sensor M1b detects the stick angle θ2. The stick angle θ2 is the angle of the line segment P2-P3 connecting the stick connecting pin position P2 and the bucket connecting pin position P3 on the XZ plane relative to the horizontal line. The bucket angle sensor M1c detects the bucket angle θ3. The bucket angle θ3 is the angle of the line segment P3-P4 connecting the bucket connecting pin position P3 and the bucket tip position P4 on the XZ plane relative to the horizontal line. Additionally, the bucket angle θ3 can also be calculated based on the operation of the operating device 26. For example, the bucket angle θ3 can be calculated based on the pilot pressure sensors 15a and 15b (… Figure 3 The angle is calculated from the output of sensors such as the bucket angle sensor M1c. In this case, the bucket angle sensor M1c can be omitted.

[0030] Figure 3 This is an explanatory diagram showing the basic system of the excavator 100. The basic system of the excavator 100 includes an engine 11, a main pump 14, a pilot pump 15, a control valve unit 17, an operating device 26, a controller 30, a display device 40, a sound output device 45, an engine control device 74, an operation mode switching switch 75, a buried object inference mode switch 76, a posture detection device M1, and an excavation pressure sensor S1, etc.

[0031] Engine 11 is the drive source of excavator 100, such as a diesel engine that operates at a predetermined speed. The output shaft of engine 11 is connected to the input shaft of main pump 14 and pilot pump 15.

[0032] The main pump 14 is a hydraulic pump that supplies working oil to the control valve unit 17 via the working oil line 16; for example, it is a swashplate variable capacity hydraulic pump. In the swashplate variable capacity hydraulic pump, the stroke length of the piston, which determines the displacement, varies according to the swashplate deflection angle, thereby changing the discharge flow rate per revolution. The swashplate deflection angle is controlled by a regulator 14a. The regulator 14a changes the swashplate deflection angle according to changes in the control current from the controller 30. For example, the regulator 14a increases the swashplate deflection angle according to an increase in the control current, thereby increasing the discharge flow rate of the main pump 14. Conversely, the regulator 14a decreases the swashplate deflection angle according to a decrease in the control current, thereby decreasing the discharge flow rate of the main pump 14. A discharge pressure sensor 14b detects the discharge pressure of the main pump 14. An oil temperature sensor 14c detects the temperature of the working oil drawn into the main pump 14.

[0033] The pilot pump 15 is a hydraulic pump used to supply working oil to various hydraulic controllers such as the operating device 26 via the pilot line 25. For example, a fixed capacity hydraulic pump can be used.

[0034] Control valve unit 17 controls the flow of working oil related to the hydraulic actuators. In the example shown, control valve unit 17 includes multiple flow control valves. Control valve unit 17 selectively supplies working oil received from main pump 14 via working oil line 16 to one or more hydraulic actuators based on changes in pressure (pilot pressure) corresponding to the operating direction and amount of operation of operating device 26. Hydraulic actuators include, for example, boom cylinder 7, stick cylinder 8, bucket cylinder 9, left travel hydraulic motor 1A, right travel hydraulic motor 1B, and swing hydraulic motor 2A. In the example shown, the hydraulic motors (left travel hydraulic motor 1A, right travel hydraulic motor 1B, and swing hydraulic motor 2A) are swashplate piston motors. However, the hydraulic motors can also be replaced with electric motors.

[0035] Operating device 26 is a device for an operator to operate the hydraulic actuator, including joystick 26A, joystick 26B, and pedal 26C. Operating device 26 receives a supply of working oil from pilot pump 15 via pilot line 25 to generate pilot pressure. Operating device 26 applies this pilot pressure to the pilot port of the corresponding flow control valve through pilot line 25a. The pilot pressure varies depending on the direction and amount of operation of operating device 26. Operating device 26 can also be operated remotely. In remote operation, operating device 26 generates pilot pressure based on information related to the direction and amount of operation received via wireless communication.

[0036] The operating device 26 can also be an electrically operated device, instead of a hydraulically operated device as described above. In this case, a solenoid valve for adjusting the pilot pressure can be arranged between the flow control valve and the pilot pump 15 within the control valve unit 17. Information related to the operating direction and amount of the electrically operated device is transmitted from the electrically operated device to the controller 30 in the form of an electrical signal. The controller 30 adjusts the opening area of ​​the solenoid valve based on the electrical signal received from the electrically operated device, thereby adjusting the magnitude of the pilot pressure acting on the flow control valve.

[0037] The controller 30 functions as a control unit for driving and controlling the excavator 100. The functions of the controller 30 can be implemented using any hardware or a combination of hardware and software. For example, the controller 30 is centered around a microcomputer, including a processor such as a CPU (Central Processing Unit), memory such as RAM (Random Access Memory) and ROM (Read Only Memory), and various input / output interface devices. The controller 30 implements various functions by executing various programs stored in ROM on the CPU. For example, the controller 30 changes the control current to the regulator 14a based on the pressure of the working oil in the negative control valve, and controls the discharge flow rate of the main pump 14 through the regulator 14a.

[0038] Display device 40 is a device for displaying various information and is located near the driver's seat in the cab 10. In the example shown, display device 40 has an image display unit 41 and an input unit 42. The image display unit 41 is a liquid crystal display. The input unit 42 is a diaphragm switch. The operator can use the input unit 42 to input information or commands to the controller 30. Furthermore, the operator can observe the image display unit 41 to monitor the operating status or control information of the excavator 100. Display device 40 is connected to controller 30 via a communication network such as CAN. However, display device 40 can also be connected to controller 30 via a dedicated line.

[0039] The display device 40 operates by receiving power from the storage battery 90. The storage battery 90 is charged by power generated by the alternator 11a. Power from the storage battery 90 is also supplied to other devices besides the controller 30 and the display device 40, such as the electrical installations 72 of the excavator 100. The starting device 11b of the engine 11 can be driven by power from the storage battery 90, thereby starting the engine 11.

[0040] The sound output device 45 is a device for outputting sound information. In the example shown, the sound output device 45 is a speaker located near the driver's seat within the cockpit 10. The sound output device 45 can also be a buzzer.

[0041] The engine control unit 74 is a device for controlling the engine 11. The engine control unit 74 controls, for example, the fuel injection quantity, to achieve the engine speed set by the input device.

[0042] The engine control unit 74 sends various data indicating the state of the engine 11 (e.g., data related to physical quantities such as the coolant temperature detected by the water temperature sensor 11c) to the controller 30. The controller 30 pre-stores the various data in the memory 30a and sends them to the display device 40, etc., as needed. The same applies to data indicating the swashplate deflection angle output by the regulator 14a, the discharge pressure of the main pump 14 output by the discharge pressure sensor 14b, the working oil temperature output by the oil temperature sensor 14c, and the pilot pressure output by the pilot pressure sensors 15a and 15b.

[0043] The operating mode switch 75, located within the operator's cab 10, is used to switch the operating mode of the excavator 100. Operating modes include, for example, M (manual) mode and SA (semi-automatic) mode. The controller 30 switches the operating mode of the excavator 100 according to the output of the operating mode switch 75.

[0044] The M (manual) mode is a mode in which the excavator 100 operates based on the operator's input to the operating device 26. For example, it operates the boom cylinder 7, stick cylinder 8, and bucket cylinder 9 based on the operator's input to the operating device 26. The SA (semi-automatic) mode is a mode in which the excavator 100 operates automatically, regardless of the input to the operating device 26, when predetermined conditions are met. For example, at least one of the boom cylinder 7, stick cylinder 8, and bucket cylinder 9 operates automatically, regardless of the input to the operating device 26, when predetermined conditions are met. Additionally, the operating modes may also include a fully automatic mode in which the lower traveling body 1, slewing mechanism 2, boom cylinder 7, stick cylinder 8, and bucket cylinder 9 operate autonomously as a whole.

[0045] The buried object estimation mode switch 76 is a switch for activating the buried object estimation function mode, and it is located inside the operator's cab 10. The buried object estimation function mode estimates buried objects located in the foundation of the excavation target. The buried object estimation function mode of the excavator 100 according to the embodiment estimates the presence or absence of buried objects based on the excavation reaction force. The operator switches the buried object estimation function mode on and off by operating the buried object estimation mode switch 76.

[0046] The controller 30 executes the buried object estimation function mode according to the start command from the buried object estimation mode switch 76, and stops the buried object estimation function mode according to the stop command from the buried object estimation mode switch 76. However, the controller 30 may also start the buried object estimation function mode when it is determined that digging is in progress based on the posture of the accessory AT, etc., regardless of the operation of the buried object estimation mode switch 76. For example, the controller 30 can continuously execute the buried object estimation function mode from the moment the digging operation begins until the moment the boom lifting operation is performed.

[0047] The digging pressure sensor S1 is an example of a detection device that detects information related to digging reaction force. It detects the pressure of the working oil in hydraulic cylinders such as the boom cylinder 7 and outputs the detected data to the controller 30. In this embodiment, the digging pressure sensor S1 is constructed by combining digging pressure sensors S11 to S18. Digging pressure sensor S11 detects the pressure of the working oil in the bottom chamber of the boom cylinder 7, i.e., the boom cylinder bottom pressure. Digging pressure sensor S12 detects the pressure of the working oil in the rod-side chamber of the boom cylinder 7, i.e., the boom rod pressure. Similarly, digging pressure sensor S13 detects the stick cylinder bottom pressure, digging pressure sensor S14 detects the stick rod pressure, digging pressure sensor S15 detects the bucket cylinder bottom pressure, and digging pressure sensor S16 detects the bucket rod pressure. Digging pressure sensor S17 detects the pressure of the working oil in the first port (left port) of the swing hydraulic motor 2A, i.e., the left swing pressure. Digging pressure sensor S18 detects the pressure of the working oil in the second port (right port) of the swing hydraulic motor 2A, i.e., the right swing pressure.

[0048] Control valve E1 is a valve that operates according to instructions from controller 30. In the example shown, control valve E1 is used to force the flow control valve associated with a predetermined hydraulic cylinder to operate independently of the input to the operating device 26. In the case of the electric operating device described above, control valve E1 is equivalent to a solenoid valve disposed between the flow control valve and the pilot pump 15.

[0049] Figure 4 It means that it is carried on Figure 1 The diagram illustrates the structure of the excavation control system of an excavator 100. The excavation control system comprises a posture detection device M1, an excavation pressure sensor S1, an operation mode switch 75, a buried object estimation mode switch 76, a controller 30, a control valve E1, a display device 40, and a sound output device 45. The controller 30 internally forms an excavation reaction force calculation unit 31 and a buried object estimation unit 32 by reading and executing a program stored in memory through a processor.

[0050] The digging reaction force calculation unit 31 is a functional component for calculating the digging reaction force. The digging reaction force calculation unit 31 is configured to calculate the digging reaction force based at least on the output of the digging pressure sensor S1. In the embodiment, the digging reaction force calculation unit 31 calculates the digging reaction force based on the output of the digging pressure sensor S1 and the posture of the attachment AT detected by the posture detection device M1. The digging reaction force calculation unit 31 may also utilize the output of a vehicle tilt sensor. The vehicle tilt sensor may be, for example, an accelerometer or a gyroscope sensor.

[0051] The output of the digging pressure sensor S1 includes, for example, at least one of the following detected by the digging pressure sensors S11 to S16: boom cylinder bottom pressure, boom rod pressure, stick cylinder bottom pressure, stick rod pressure, bucket cylinder bottom pressure, and bucket rod pressure.

[0052] The digging reaction force calculation unit 31 can calculate the cylinder thrust based on the output of the digging pressure sensor S1. The cylinder thrust is calculated, for example, based on the digging pressure and the pressure-bearing area of ​​the piston sliding in the cylinder. The cylinder thrust includes, for example, the boom cylinder thrust (f1), the stick cylinder thrust (f2), and the bucket cylinder thrust (f3). Specifically, the boom cylinder thrust (f1) is represented by the difference between the cylinder extension force (i.e., the product of the boom cylinder bottom pressure and the pressure-bearing area of ​​the piston in the boom cylinder bottom oil chamber) and the cylinder contraction force (i.e., the product of the boom rod pressure and the pressure-bearing area of ​​the piston in the boom rod side oil chamber). The same applies to the stick cylinder thrust (f2) and the bucket cylinder thrust (f3).

[0053] The digging reaction force calculation unit 31 can calculate the digging torque based on the posture of the accessory AT and the cylinder thrust. For example... Figure 2 As shown, the magnitude of the bucket digging torque (τ3) is represented by the product of the magnitude of the bucket cylinder thrust (f3) and the distance G3 between the line of action of the bucket cylinder thrust (f3) and the bucket connecting pin position P3. Distance G3 is a function of the bucket angle θ3 and is an example of link gain. The same applies to the boom digging torque (τ1) and stick digging torque (τ2). Additionally, distance G1 is the distance between the line of action of the boom cylinder thrust (f1) and the boom foot pin position P1, and distance G2 is the distance between the line of action of the stick cylinder thrust (f2) and the stick connecting pin position P2.

[0054] Extracting reaction forces, for example, as... Figure 2The mechanism function, with boom angle θ1, stick angle θ2, and bucket angle θ3 as parameters, is calculated by multiplying it with a function using boom digging torque (τ1), stick digging torque (τ2), and bucket digging torque (τ3) as parameters. The function using boom digging torque (τ1), stick digging torque (τ2), and bucket digging torque (τ3) as parameters can also be a function using boom cylinder thrust (f1), stick cylinder thrust (f2), and bucket cylinder thrust (f3) as parameters. The function using boom angle θ1, stick angle θ2, and bucket angle θ3 as parameters can be a function based on force balance, a function based on the Jacobian matrix, or a function based on the principle of virtual work.

[0055] Thus, the value of the digging reaction force can be derived from the current detection values ​​of various sensors. However, the detection value of the digging pressure sensor S1 can also be used directly as information about the digging reaction force. Alternatively, the value of the cylinder thrust calculated based on the detection value of the digging pressure sensor S1 can also be used as information about the digging reaction force. The value of the digging reaction force can also be calculated using the value of the cylinder thrust calculated based on the detection value of the digging pressure sensor S1 and the value related to the posture of the accessory AT derived from the detection value of the posture detection device M1.

[0056] The digging reaction force calculation unit 31 can also calculate the digging reaction force acting in the rotation direction based on the outputs of the digging pressure sensors S17 and S18. When the left rotation pressure (P17) detected by the digging pressure sensor S17 is greater than the right rotation pressure (P18) detected by the digging pressure sensor S18, the upper rotating body 3 intends to rotate to the left. Conversely, when the right rotation pressure (P18) detected by the digging pressure sensor S18 is greater than the left rotation pressure (P17) detected by the digging pressure sensor S17, the upper rotating body 3 intends to rotate to the right. For example, the digging reaction force calculation unit 31 can calculate the left rotation pressure (P17) when the left rotation pressure (P17) is greater than the right rotation pressure (P18) as the digging reaction force acting in the left rotation direction. Furthermore, the digging reaction force calculation unit 31 can, for example, calculate the right slewing pressure (P18) when the right slewing pressure (P18) is greater than the left slewing pressure (P17) as the digging reaction force acting in the right slewing direction. Additionally, when a slewing electric motor is installed instead of the slewing hydraulic motor 2A, the digging reaction force calculation unit 31 can calculate the digging reaction force acting in the slewing direction based on information related to electrical energy, such as the direction and magnitude of the current supplied to the slewing electric motor.

[0057] The buried object inference unit 32 is configured to detect buried objects based on information related to the excavation reaction force. In this embodiment, the buried object inference unit 32 infers the presence or absence of buried objects based on the excavation reaction force calculated by the excavation reaction force calculation unit 31 and a pre-existing buried object inference learning model. This learning model will be described in detail later.

[0058] Then, for example, if it is deduced that there is a buried object, the buried object deduction unit 32 outputs a control command to the control valve E1. Upon receiving the control command from the buried object deduction unit 32, the control valve E1, regardless of the operation input to the operating device 26, forcibly actuates the flow control valve associated with the predetermined hydraulic cylinder, thereby forcibly extending or retracting the predetermined hydraulic cylinder. For example, even without operating the boom control lever, the control valve E1 forcibly extends the boom cylinder 7 by forcibly moving the flow control valve associated with the boom cylinder 7. As a result, the excavator 100 can forcibly raise the boom 4, thereby reducing the digging depth (changing the trajectory). Alternatively, even if the stick control lever is operated, the control valve E1 can forcibly stop the stick cylinder 8 by forcibly moving the flow control valve associated with the stick cylinder 8. By forcibly stopping the stick 5, the excavator 100 can prevent the bucket 6 from contacting the buried object. Thus, by forcibly extending or stopping at least one of the boom cylinder 7, stick cylinder 8, and bucket cylinder 9 according to the control command from the buried object inference unit 32, the control valve E1 can prevent the accessory AT from contacting the buried object.

[0059] If the presence of a buried object is deduced, the buried object deduction unit 32 can output a control command to the display device 40. Upon receiving the control command from the buried object deduction unit 32, the display device 40 displays the deduced location of the buried object. For example, the display device 40 can display a virtual viewpoint image showing the state of the excavator 100 as viewed from a virtual viewpoint directly above it, and overlay the buried object onto this virtual viewpoint image. The virtual viewpoint image is generated based on images acquired by the front camera 70F, the rear camera 70B, the left camera 70L, and the right camera 70R. Alternatively, the display device 40 can also display an image showing a cross-section of the foundation where the excavator 100 is located, and overlay the buried object onto this virtual viewpoint image. Furthermore, if the presence of a buried object is deduced, the buried object deduction unit 32 can output a control command to the sound output device 45.

[0060] The controller 30 activates the buried object estimation function mode based on the start command from the buried object estimation mode switch 76. When the buried object estimation function mode is activated, the buried object estimation unit 32 estimates the presence or absence of buried objects based on the digging reaction force calculated by the digging reaction force calculation unit 31. On the other hand, the controller 30 deactivates the buried object estimation function mode based on the stop command from the buried object estimation mode switch 76. Thus, the excavator 100 can prevent the erroneous estimation of the presence of buried objects based on changes in the digging reaction force, even when it is clearly known that there are no buried objects, and output control commands to the control valve E1, the display device 40, or the sound output device 45. When the buried object estimation function mode is deactivated, the digging reaction force calculation unit 31 can stop calculating the digging reaction force to reduce the computational load.

[0061] The buried object detection function can be executed in either the M (manual) or SA (semi-automatic) operating mode of the excavator 100. However, it can also be executed only when the SA (semi-automatic) mode is selected. When the SA (semi-automatic) mode is selected, the operator can improve the detection accuracy of buried objects by moving the attachment AT along a pre-set target track.

[0062] Next, refer to Figure 5 A representative description of the actions of the excavator 100 when it deduces that the water pipe U1 is the buried object is provided. Figure 5 This is a diagram showing a cross-section of the foundation where the water pipe U1 is buried. Figure 5 The ground surface (ES) before excavation is shown in dashed lines.

[0063] The operator of the excavator 100 first operates the operation mode switch 75 to switch the operation mode of the excavator 100 to SA (semi-automatic) mode. The operator manually operates the operating device 26 to move the tip of the bucket 6 to the desired position (position 1 PS1). After moving the tip of the bucket 6 to the desired position, the operator operates the buried object estimation mode switch 76 to activate the buried object estimation function mode.

[0064] In SA (semi-automatic) mode, the controller 30 causes the attachment AT to operate autonomously. Specifically, the controller 30 automatically extends or retracts at least one of the boom cylinder 7, stick cylinder 8, and bucket cylinder 9 to move a predetermined part of the attachment AT along a pre-set target trajectory TP. However, even in the buried object estimation function mode, the controller 30 may not cause the attachment AT to operate autonomously, but rather to operate the attachment AT according to the operator's operation of the operating device 26.

[0065] The controller 30 automatically activates the accessory AT to move the tip of the bucket 6 along a pre-set first target trajectory TP1 (single-dot line) during the digging operation. As the tip of the bucket 6 moves along the first target trajectory TP1, the digging reaction force calculation unit 31 repeatedly calculates the digging reaction force based on the output of the posture detection device M1 and the output of the digging pressure sensor S1. Furthermore, the buried object inference unit 32 repeatedly infers the presence or absence of buried object based on the digging reaction force calculated by the digging reaction force calculation unit 31.

[0066] If the tip of the bucket 6 reaches the end of the first target track TP1, the controller 30 stops the accessory AT from operating autonomously. This means that the buried object inference unit 32 did not detect the buried object before the tip of the bucket 6 reached the end of the first target track TP1.

[0067] Then, the operator operates the buried object estimation mode switch 76 to stop the buried object estimation function mode. If the operator manually operates the operating device 26 to perform a boom lifting or boom lifting slewing action, the controller 30 can automatically stop the buried object estimation function mode. After performing the soil removal action and boom lowering slewing action, in order to perform the next digging action, the operator moves the tip of the bucket 6 to the next desired position (position 2 PS2). For example, position 2 PS2 is located at a depth D1 from the ground surface ES before the start of digging, and is approximately directly below position 1 PS1.

[0068] After the operator moves the tip of the bucket 6 to position 2 PS2, the operator activates the buried object detection function mode. When the buried object detection function mode is activated, the controller 30 automatically activates the accessory AT to move the tip of the bucket 6 along a pre-set second target track TP2 (single-dot line). While the tip of the bucket 6 moves along the second target track TP2, the digging reaction force calculation unit 31 repeatedly calculates the digging reaction force based on the output of the posture detection device M1 and the output of the digging pressure sensor S1. Furthermore, the buried object detection unit 32 repeatedly detects the presence or absence of buried object based on the digging reaction force calculated by the digging reaction force calculation unit 31.

[0069] If the tip of the bucket 6 reaches the end of the second target track TP2, the controller 30 stops the accessory AT from operating autonomously. This means that the buried object inference unit 32 did not detect any buried object before the tip of the bucket 6 reached the end of the second target track TP2.

[0070] Then, similarly as above, after the operator performs the soil removal and boom lowering / slewing actions, in order to perform the next digging action, the tip of the bucket 6 is moved again to the next desired position (position 3 PS3). Position 3 PS3 is located at a depth D2 from the first exposed surface and is approximately directly below position 2 PS2.

[0071] After the operator moves the tip of the bucket 6 to position 3 PS3, the buried object prediction function mode is activated. If the buried object prediction function mode is activated, the controller 30 causes the accessory AT to automatically move so that the tip of the bucket 6 moves along the pre-set third target track TP3 (single-dot line).

[0072] As the tip of the bucket 6 moves along the third target track TP3, the digging reaction force calculation unit 31 repeatedly calculates the digging reaction force based on the output of the posture detection device M1 and the output of the digging pressure sensor S1. Furthermore, the buried object inference unit 32 repeatedly infers the presence or absence of buried objects based on the digging reaction force calculated by the digging reaction force calculation unit 31.

[0073] For example, when the tip of the bucket 6 reaches position 4 PS4, the object inference unit 32 infers that there is an object buried. Position 4 PS4 is located at a depth D3 from the second exposed surface and is on the third target track TP3. This position 4 PS4 is the position where the distance between the water pipe U1 of the object buried along the direction of the third target track TP3 and the tip of the bucket 6 is defined as AD1.

[0074] Here, for reference Figure 6 The method of inferring the presence or absence of buried objects based on the reaction force of excavation is explained. Figure 6 It is a graph showing the relationship between the excavation reaction force F and the approach distance AD. Figure 6 The vertical axis corresponds to the excavation reaction force F calculated by the excavation reaction force calculation unit 31. Figure 6 The horizontal axis corresponds to the approach distance AD. The approach distance AD ​​is the distance between the current position of the bucket tip 6 and the buried object (water pipe U1) along the target track TP. Figure 6 The diagram shows the situation where the approach distance AD ​​decreases from left to right on the horizontal axis before it becomes zero. That is, when the approach distance AD ​​is AD0, the tip of the bucket 6 is located further away from the water pipe U1 than when the approach distance AD ​​is AD1.

[0075] Specifically, Figure 6The double-dotted line in the diagram illustrates the learning model that serves as the benchmark for the digging reaction force. That is, the learning model is represented by a function, such as a function, that expresses the relationship between the change in the approach distance AD ​​during digging operations and the digging reaction force. The function of the learning model can be any of a linear function, a polynomial function, a logarithmic function, etc. This learning model is a model that learns the digging reaction force experienced by the attachment AT from the digging object when there are no buried objects in the digging object (within the ground). In the learning model, the digging reaction force F increases as the approach distance AD ​​decreases. This is because as the bucket 6 approaches the machine body (upper rotating body 3), the amount of sand loaded into the bucket 6 increases, thus increasing the digging reaction force.

[0076] The buried object inference unit 32 compares the learned model with the excavation reaction force repeatedly calculated during excavation operations to infer the presence or absence of buried objects. For example, using... Figure 6 The solid line is shown in Figure 5 The digging reaction force when the tip of the middle bucket 6 moves along the third target track TP3.

[0077] At this point, within the approach distance AD ​​of AD0 to AD1, the digging reaction force varies roughly along the learned model. However, if the approach distance AD ​​exceeds AD1, the digging reaction force deviates significantly from the learned model. This is because a water pipe U1 exists on the third target track TP3 of bucket 6; therefore, as the tip of bucket 6 approaches the water pipe U1, the sand between bucket 6 and the water pipe U1 is compressed between them.

[0078] The buried object inference unit 32, for example, calculates the difference between the learning model and the actual excavation reaction force, and determines whether the difference between the learning model and the actual excavation reaction force exceeds a threshold, thereby inferring whether there is a buried object (water pipe U1). Figure 6 The example shown illustrates an instance where the difference between the learned model and the actual mining reaction force exceeds a threshold at a position close to AD by AD2. That is, a position close to AD2 is equivalent to... Figure 5 The moment when the tip of the middle bucket 6 reaches position PS4 on the third target track TP3. This position AD2 is, for example, about 20cm away from the buried object. In addition, the position close to AD3 is where the buried object (water pipe U1) is located.

[0079] However, the method for comparing the learning model with the actual excavation reaction force is not limited to the above, and various methods can be used. For example, the buried object inference unit 32 can compare the average increase rate of the actual excavation reaction force relative to the proximity distance AD ​​with the average increase rate of the learning model. Then, if the average increase rate of the actual excavation reaction force exceeds a predetermined value of the average increase rate of the learning model, it can be inferred that there is a buried object.

[0080] Next, refer to Figure 7 The above-described learning model for detecting buried objects and the buried object inference system 200 that provides the learning model will be explained. Figure 7 This is a block diagram representing the buried object inference system 200.

[0081] The buried object inference system 200 includes an information processing device 210, a communication network 220, a cloud server 230, and a controller 30 for the excavator 100.

[0082] The information processing device 210 is an information processing unit used to create a learning model for detecting the buried objects of the excavator 100. This information processing device 210 can be a known computer equipped with a processor 211, a memory 212, an input / output interface 213, and a communication interface 214. Furthermore, the information processing device 210 can be configured as a single computer or as multiple computers.

[0083] Information processing device 210 may be installed, for example, in companies or organizations that manufacture, manage, or use excavators 100 or other work machinery. Information processing device 210 manages information such as the usage status and condition of multiple excavators 100 or other work machinery, and provides information to the excavators 100 or other work machinery. Information processing device 210 is connected to cloud server 230 via communication network 220, and provides and retrieves information via cloud server 230. Alternatively, information processing device 210 may be configured to communicate with mobile terminals such as computers, smartphones, and tablets held by workers via cloud server 230 (or directly).

[0084] The communication network 220 of the buried object estimation system 200 can utilize a dedicated line such as the Internet or Ethernet. As described above, the buried object estimation system 200 of the embodiment accesses the cloud server 230 via the communication network 220, thereby transmitting and receiving information between the information processing device 210 and each excavator 100. For example, if various excavator information, including the digging reaction force of the excavator 100 during digging operations at the work site, is uploaded to the cloud server 230, the information processing device 210 will retrieve this information from the cloud server 230 at an appropriate time.

[0085] Conversely, if the information processing device 210 uploads the work information (not shown) to the cloud server 230, the excavator 100 downloads the work information from the cloud server 230 at an appropriate time before the work begins. This work information may include, for example, the location information of the work site, information on a 3D or 2D model of the work site, information on the machinery used at the work site, and a learning model used to detect buried objects.

[0086] Alternatively, the buried object inference system 200 can be configured to communicate directly (or via another computer) between the information processing device 210 and the excavator 100 without using the cloud server 230. Furthermore, it can also employ a structure where an operator stores the excavator 100's information in an external storage device and connects this external storage device to the information processing device 210, thereby providing the excavator 100's information to the information processing device 210. The information provided from the information processing device 210 to the excavator 100 is also the same.

[0087] Next, refer to Figure 8 The learning model for detecting buried objects generated in the information processing device 210 is explained. Figure 8 This is a block diagram showing the functional units of the information processing device 210 when generating the learning model. For example, the information processing device 210 generates a learning model for detecting buried objects using an unsupervised learning method. As an example, the information processing device 210 internally constructs a sensor data storage unit 215, a learning data extraction unit 216, a reaction force model generation unit 217, an optimization unit 218, an evaluation unit 219, etc., by reading and executing a program stored in the memory 212 by the processor 211.

[0088] The sensor data storage unit 215 stores information acquired by various sensors of the excavator 100. At this time, the information processing unit 210 establishes associations between information from excavators 100 of the same type and stores sensor values ​​and time information in the sensor data storage unit 215. The information stored in the sensor data storage unit 215 includes information related to digging reaction force (sensor data from the digging pressure sensor S1 and the posture detection device M1). Furthermore, the information stored in the sensor data storage unit 215 may also include information about the external environment detected by the object detection device 70 (camera, LiDAR, etc.) of the excavator 100. Moreover, when providing operational support for the excavator 100, the sensor data storage unit 215 can store simulation data obtained from simulating the excavation operation, including the AT attachment. This simulation data may be data actually used by the excavator 100 during the digging operation, or data simulated in advance in the information processing unit 210 and planned to be provided to the excavator 100.

[0089] The learning data extraction unit 216 is the input layer for machine learning. It extracts data from various information stored in the sensor data storage unit 215 that is used to generate the learning model for detecting buried objects, and provides it to the reaction force model generation unit 217. Examples of data used in generating this learning model include information related to excavation reaction force (such as the pressure of the excavation pressure sensor S1), information related to the posture of the excavator 100, and information from the object detection device 70.

[0090] The reaction force model generation unit 217 generates a learning model for detecting buried objects based on data provided by the learning data extraction unit 216. Here, the objects excavated by the excavator 100 have different soil types, such as soil, sand, sandy soil, hard rock, and soft rock. If the soil type of the excavated object is different, the excavation reaction force will also change. Therefore, the reaction force model generation unit 217 generates a learning model for each different soil type. Thus, a learning model is generated that includes: a soil learning model used when excavating soil, a sand learning model used when excavating sand, and a gravel learning model used when excavating gravel, etc.

[0091] For example, the reaction force model generation unit 217 uses the extracted features of the excavation reaction force to identify patterns in the sequence data of the excavation reaction force through a time-series analysis neural network. Furthermore, the reaction force model generation unit 217 extracts terrain data features based on information from cameras or LiDAR (video information) or construction data from the actual excavation work performed by the excavator 100. Then, the reaction force model generation unit 217 generates a reaction force prediction model by connecting the patterns of the excavation reaction force and the terrain data features in a fully connected layer. The reaction force prediction model is, for example, a function or table representing the change in excavation reaction force relative to time or the change in proximity distance AD. Furthermore, based on the terrain data features, a reaction force prediction model is calculated for each type of soil (soil, sand, sandy soil, etc.).

[0092] The optimization unit 218 optimizes the reaction force prediction model based on the input of one or more reaction force prediction models generated by the reaction force model generation unit 217, and generates a learning model (soil learning model, sand learning model, sandy soil learning model, etc.). For example, the optimization unit 218 performs Bayesian optimization processing based on the reaction force prediction model and terrain data features or other mining operation conditions. By using multiple reaction force prediction models generated by the reaction force model generation unit 217, the optimization unit 218 can obtain a high-precision learning model. Furthermore, the optimization processing of the optimization unit 218 is not limited to Bayesian optimization processing; various methods such as grid search can also be used.

[0093] Furthermore, the evaluation unit 219 evaluates the quality of the generated learning model and outputs the evaluation result to the reaction force model generation unit 217. For example, the evaluation unit 219 can evaluate the learning model based on information about the actual presence or absence of buried objects confirmed through the exposure operation described later. Alternatively, the evaluation of the learning model can be performed by optimizing hyperparameters that can be set or adjusted by the user of the information processing device 210. Thus, a learning model can be generated to meet any requirement such as improving the performance of the learning model or smoothly obtaining the learning model.

[0094] If the information processing device 210 generates one or more learning models (soil learning model, sand learning model, and sandy soil learning model) for detecting buried objects through the above processing, it uploads the learning model to the cloud server 230. Thus, the excavator 100 at the work site can download the learning model by accessing the cloud server 230.

[0095] The controller 30 of the excavator 100 uses a learning model downloaded during the excavation operation to infer the presence or absence of buried objects during the excavation operation, as described above. Furthermore, the excavator 100 can infer buried objects based on soil type by selecting a learning model corresponding to the soil type of the excavation object from multiple types of learning models (soil learning model, sand learning model, and sandy soil learning model). For example, when the operator of the excavator 100 identifies the soil type of the excavation object, they operate the input unit 42 of the display device 40 to input the soil type of the excavation object. Thus, the controller 30 can select the learning model to be used from multiple types of learning models.

[0096] Alternatively, the controller 30 can be configured to automatically select the soil type of the excavation object based on video information captured by the camera or the like of the object detection device 70 of the excavator 100. Thus, for example, even if the soil type changes during excavation without the operator's notice, it can immediately switch to the appropriate learning model.

[0097] However, in actual application, the hardness or viscosity of the object being excavated by the excavator 100 can change due to factors such as temperature, humidity, and soil moisture content. Therefore, simply using the existing learning model will not generate excavation reaction forces adapted to the actual work site, potentially failing to detect buried objects. Therefore, the controller 30 of the excavator 100 confirms the state of the object being excavated by excavating it at the actual work site and optimizes the learning model to adapt it to the actual excavation conditions.

[0098] Figure 9 This is an explanatory diagram illustrating the acquisition, optimization, and use of the learning model by the excavator 100 during excavation operations. For example... Figure 9As shown, the excavator 100 performs pre-excavation work in an area determined to be free of buried objects at the work site. Then, during the pre-excavation work, the excavator 100 detects the actual excavation reaction force through various sensors (posture detection device M1, excavation pressure sensor S1).

[0099] The controller 30 determines whether model tuning is needed by comparing the actual variation in excavation reaction force at the work site with its existing learning model. If tuning is deemed necessary, the controller 30 tunes the model using the actual excavation reaction force at the work site. For example, the tuning may involve fitting a model (predicted excavation reaction force) represented by a linear function, polynomial function, logarithmic function, etc., to data showing the actual excavation reaction force at the work site. This appropriately corrects the learning model to represent the variation in excavation reaction force corresponding to the actual work site. Hereinafter, the corrected learning model will also be referred to as the corrected learning model.

[0100] On-site optimization can be performed on only the pre-selected learning model (e.g., the soil learning model) or on all available learning models (e.g., soil learning model, sand learning model, and gravel learning model). On-site, soil conditions may change during excavation. Therefore, if the controller 30 pre-calculates calibration learning models for multiple learning models, it can smoothly switch between models.

[0101] After performing preliminary mining operations, controller 30 uses the correction learning model to... Figure 5 The excavation operation is shown. That is, the excavator 100 performs the excavation operation in SA (semi-automatic) mode, which enables the attachment AT to move autonomously. At this time, the excavation pressure sensor S1 and the posture detection device M1 acquire their respective detection values. The excavation reaction force calculation unit 31 of the controller 30 calculates the excavation reaction force based on these sensors.

[0102] Then, the embedded object inference unit 32 of the controller 30 infers whether there are embedded objects in the excavation object during excavation operations based on attachment AT by comparing the calculated excavation reaction force with the calibration learning model. As described above, since the calibration learning model has been optimized according to the actual work site, the controller 30 is able to infer with high accuracy whether there are embedded objects inside the excavation object.

[0103] If it is inferred that there is buried material in the excavation target, the excavator 100 can autonomously control the movement of the attachment AT to prevent the attachment AT from contacting the buried material. Specifically, the controller 30 stops the movement of the attachment AT by invalidating the boom closing operation based on the detection of buried material. Alternatively, the controller 30 can also raise the boom 4 by automatically extending and retracting the boom cylinder 7, thereby changing the trajectory to prevent the tip of the bucket 6 from contacting the buried material.

[0104] Furthermore, if a buried object is detected in SA (semi-automatic) or M (manual) mode, the controller 30 preferably alerts the operator to prevent the accessory AT from approaching the buried object. For example, the controller 30 can convey the distance between the tip of the bucket 6 and the buried object to the operator via the sound output device 45. In this case, the controller 30 can shorten the interval of the intermittent sound as the distance to the buried object decreases. Moreover, if the tip of the bucket 6 approaches the buried object significantly, the controller 30 can issue a loud alarm to the operator via the sound output device 45.

[0105] If the controller 30 of the excavator 100 deduces the presence of a buried object, an exposure operation can be performed at the work site to confirm its presence. For example, during the exposure operation, workers at the work site use tools to excavate the area where the buried object is suspected, thereby exposing the buried object from the excavation target. Alternatively, during the exposure operation, the operator of the excavator 100 can carefully operate the attachment AT to excavate the area where the buried object is suspected, thereby confirming its presence. If the presence of a buried object is confirmed during the exposure operation at the suspected location, the worker can use their information terminal device to send this information to the controller 30 of the excavator 100 and / or the cloud server 230. Thus, the controller 30 of the excavator 100 and / or the cloud server 230 establish and store the data of the excavation reaction force when the buried object was suspected, in association with the result of the exposure operation regarding the presence or absence of a buried object.

[0106] When information indicating the presence of buried objects is received during the exposure operation, the controller 30 or cloud server 230 can identify that the detection of buried objects using the learning model (correction learning model) has functioned correctly. Conversely, when information indicating the absence of buried objects is received during the exposure operation, the controller 30 or cloud server 230 can identify that the detection of buried objects using the learning model (correction learning model) is incorrect. In this case, the controller 30 can recalibrate the learning model (or the threshold for determining buried objects) based on the obtained information. Thus, the excavator 100 can use the recalibrated correction learning model or threshold to execute the buried object inference function mode.

[0107] Furthermore, when the buried object inference function mode is executed, the controller 30 can display information related to the inference of buried objects on the display device 40, which serves as the user interface. Figure 10 This is a diagram illustrating image information 400 displayed on display device 40 when the buried object inference function mode is executed.

[0108] The image information 400 can be automatically displayed in both SA (semi-automatic) and M (manual) modes, accompanying the execution of the buried object inference function mode. The image information 400 includes a mode selection display unit 410, a model selection display unit 420, an anomaly indicator 430, model parameters 440, a model accuracy display unit 450, a reaction force display unit 460, an environment display unit 470, an external connection setting display unit 480, and a detailed setting display unit 490.

[0109] The mode selection display unit 410 allows the operator to select from multiple learning modes, an optimization mode for adjusting the learning mode, or a buried object inference function mode for actual excavation work. For example, the operator can turn on the mode selection display unit 410 by operating the switch provided on the joystick 26A or joystick 26B of the operating device 26. The mode selection display unit 410 displays a selection window with various modes along with the turning operation, allowing the operator to select any mode.

[0110] The selection model display unit 420 displays various learning models (soil learning model, sand learning model, gravel learning model, etc.) selected by the operator. Furthermore, after executing the tuning mode, the selection model display unit 420 can display a notification indicating that tuning has been achieved (an icon indicating the corrected learning model, etc.).

[0111] When the buried object inference function mode is activated during actual excavation operations, the anomaly indicator 430 converts the probability of the buried object's existence into an anomaly level and notifies the user. For example, the anomaly level is calculated based on the difference in excavation reaction force against the learned model and the proportion of the available threshold. In the graphical example, the anomaly level is indicated using a pie chart and numerical values. The color of the pie chart can change according to the anomaly level, such as green for anomalies between 0% and 50%, yellow for anomalies between 50% and 80%, and red for anomalies between 80% and 100%.

[0112] In cases of high anomaly (in other words, when the difference from the threshold is large), the controller 30 can notify the operator of the presence of a buried object via the anomaly indicator 430. In cases of high anomaly, the controller 30 can also automatically stop the accessory AT's operation or change its trajectory as described above. Alternatively, the operator can monitor the anomaly level and make decisions such as stopping the accessory AT's operation at 70% or at 90%.

[0113] Model parameter 440 displays hyperparameters within the learning model (correction learning model) in a manner that can be set by the operator. These hyperparameters may include, for example, the soil conditions at the work site, the detection accuracy in the buried object inference function mode, and the operating mode of the excavator 100.

[0114] The model accuracy display unit 450 evaluates the currently applied learning model (corrected learning model) and displays the evaluation results. For example, regarding the evaluation of the learning model, if the system detects no buried object and there is actually no buried object, the score is increased; if the system detects no buried object but there is actually a buried object, the score is decreased. Alternatively, regarding the evaluation of the learning model, if the system detects buried object and there is actually a buried object, the score is increased; if the system detects buried object but there is actually no buried object, the score is decreased.

[0115] The reaction force display unit 460 displays the digging reaction force detected by the learning model (correction learning model) actually used in the digging operation and by various sensors (posture detection device M1, digging pressure sensor S1) of the excavator 100 in graphical form. For example, the graph shows the change in digging reaction force corresponding to the change in digging distance, with digging distance as the horizontal axis and digging reaction force as the vertical axis. Alternatively, the graph shows the change in digging reaction force corresponding to the passage of time, with digging time as the horizontal axis and digging reaction force as the vertical axis. Thus, the operator can operate the excavator 100 while simultaneously recognizing the changes in the learning model and the actual digging reaction force.

[0116] The environmental display unit 470 displays environmental element variables. Examples of these environmental element variables include the date and time when the excavator 100 is performing excavation work, the weather, and the operator ID.

[0117] The external connection setting display unit 480 displays the communication environment between the controller 30 and the external communication network. For example, when the external connection setting display unit 480 is connected to the cloud server 230, it can set the connection status with the cloud server 230 according to the operator's operation.

[0118] The detailed setting display unit 490 allows the operator to change settings other than those mentioned above during the execution of the buried object estimation function mode.

[0119] By displaying the above image information 400 on the display device 40, the operator of the excavator 100 can set the learning model in the buried object inference function mode and can effectively identify the relationship between the learning model and the digging reaction force during actual digging operations. In particular, by displaying the learning model, the excavator 100 according to the embodiment can inform the operator of situations where buried objects are determined to exist because the digging reaction force during digging operations does not conform to the learning model.

[0120] The excavator 100 involved in the implementation method is basically configured as described above. Hereinafter, reference will be made to... Figure 11 (A) and Figure 11 The flowchart of (B) describes its actions. Figure 11 (A) is a flowchart representing the processing flow before the actual excavation operation is carried out. Figure 11 (B) is a flowchart illustrating the method for inferring buried objects during actual excavation operations.

[0121] The controller 30 of the excavator 100 executes the buried object inference method. Figure 11 (A) and Figure 11 Steps S101 to S109 of (B).

[0122] The controller 30 accesses the cloud server 230 to obtain various learning models stored in the cloud server 230 (step S101). However, the method of obtaining the learning models is not limited to this; it can also be configured to allow the controller 30 to obtain the learning models by connecting a storage device containing the learning models to the controller 30. Alternatively, the controller 30 can pre-store the learning models, in which case step S101 can be omitted.

[0123] The controller 30 displays a prompt to optimize the learning model at the actual work site and instructs the operator of the excavator 100 to perform pre-defined excavation work, thereby optimizing the learning model (step S102). By performing this optimization, the controller 30 obtains a corrected learning model corresponding to the actual work site. However, the operator may choose not to perform optimization; in this case, the controller 30 directly uses the existing learning model. Furthermore, for example, if multiple excavators 100 are used at the same site, one excavator 100 can perform the optimization and store the optimization result (corrected learning model) in the cloud server 230. Then, the other excavators 100 retrieve the optimization result from the cloud server 230. This eliminates the need for optimization by multiple excavators 100. Alternatively, multiple excavators 100 can be used for optimization, and the optimization results can be stored in a cloud server 230. The cloud server 230 can then use multiple optimization results to correct the learning model and send it to each excavator 100. This also improves the optimization accuracy of the learning model of each excavator 100.

[0124] Then, the excavator 100 moves to the actual work site for excavation. The controller 30 determines whether the operator has selected the operating mode and the buried object inference function mode, etc. (step S103). Then, if the buried object inference function mode has been selected (step S103: yes), proceed to step S104.

[0125] In step S104, the controller 30 reads out the learning model it possesses (correction learning model). At this time, the controller 30 displays the aforementioned image information 400, enabling the operator to visually identify the learning model to be used.

[0126] Then, the excavator 100 causes the attachment AT to move according to the control of the controller 30 or the operation of the operator, thereby performing the excavation operation of the excavation object (step S105).

[0127] In this excavation operation, the controller 30 acquires sensor data from the posture detection device M1 and the excavation pressure sensor S1, calculates the excavation reaction force based on this data, and compares the calculated excavation reaction force with the learning model to determine whether there is any buried object (step S106). Then, if it is determined that there is no buried object (step S106: No), proceed to step S107.

[0128] In step S107, the controller 30 determines that the excavation operation has ended. If the excavation operation has not ended (step S107: No), the process returns to step S105 and continues. On the other hand, if the excavation operation has ended (step S107: Yes), the current processing flow ends.

[0129] Furthermore, in step S106, if it is determined that there is a buried object (step S106: Yes), the process proceeds to step S108. In step S108, as described above, the controller 30 stops the operation of the accessory AT or changes the track, and displays information about the buried object on the display device 40. Thus, the operator can recognize that the excavator 100 has stopped operating because there is a buried object in the excavation target.

[0130] At this point, the operator of the excavator 100 can inform the workers around the work site that a buried object has been detected, so that the workers can perform the exposure operation. Then, the workers will report the information on whether there is actually a buried object to the controller 30 or the cloud server 230 (step S109).

[0131] Therefore, the controller 30 can evaluate the learning model (corrected learning model) used based on information from the workers regarding the actual presence or absence of buried objects, thereby correcting the learning model or threshold used in the next excavation operation. For example, in the absence of buried objects, the controller 30 can correct the function of the learning model (corrected learning model) by taking into account the changes in the excavation reaction force.

[0132] As described above, the excavator 100 and the buried object inference method use a learning model learned in the information processing device 210 to perform excavation operations, thereby enabling accurate detection of buried objects within the excavation target. Furthermore, the excavator 100 optimizes the learning model obtained from the cloud server 230 at the actual work site, thereby using a calibrated learning model suitable for the actual work site. As a result, the excavator 100 can infer the presence or absence of buried objects with high accuracy.

[0133] Next, refer to Figure 12 An example of the structure of the operating system SYS according to another embodiment will be described. Figure 12 This is a schematic diagram representing the structure of the operating system SYS. The operating system SYS includes the excavator 100, the remote control room (RC), and the management center (MC). Additionally, Figure 12 The excavator 100 shown has the same Figure 1 It has the same structure as the excavator 100 shown.

[0134] The excavator 100, the remote control room (RC), and the management center (MC) are connected to each other via a communication network (NW) for data transmission and reception. Alternatively, the excavator 100, RC, and MC can also be connected to each other directly without using the NW communication network. In the example shown, the excavator 100 sends information related to the work site to the remote control room (RC). Thus, the remote operator (RO) in the RC can monitor the work site status based on the information from the excavator 100.

[0135] The excavator 100 is equipped with sensors capable of three-dimensionally identifying the position and shape of objects present at the work site. For example, the excavator 100 is equipped with a spatial recognition device. Therefore, the excavator 100 can send the results of three-dimensional measurement of the work site to a remote control room (RC).

[0136] A spatial identification device is used to identify the space surrounding the excavator 100. In the example shown, the spatial identification device is a LiDAR. The LiDAR, for example, measures the distance between itself and more than one million points within the monitoring range. Alternatively, the spatial identification device can be any device capable of measuring distances to objects. For example, it could be a stereo camera or a combination of a camera device S6 and a ranging device such as millimeter-wave radar.

[0137] The operating system SYS may include one or more excavators 100. In the case of multiple excavators 100, the remote operator RO of a specific excavator 100 can obtain information about the work site obtained by that specific excavator 100, as well as information about the work site obtained by one or more other excavators 100.

[0138] The remote control room RC is equipped with a communication device T2, a remote controller RCC, an operating device 26E, an operating sensor 43, a display device D1E, an internal sound output device SP2E, and an internal sound acquisition device M2E. Furthermore, the remote control room RC is equipped with an operator's seat DS where the remote operator RO sits to remotely operate the excavator 100.

[0139] The communication device T2 is configured to communicate with the communication device T1 installed on the excavator 100.

[0140] A remote controller (RCC) is a computing device that performs various calculations. In this embodiment, the RCC is composed of a microcomputer including a CPU and memory. Furthermore, the various functions of the RCC are implemented by the CPU executing programs stored in memory.

[0141] Display device D1E is a device capable of displaying various information. Display device D1E displays images based on information sent from excavator 100, enabling remote operator RO located in remote control room RC to visually identify the area around excavator 100. In the example shown, display device D1E is a liquid crystal display showing images captured by camera device S6 mounted on excavator 100. Alternatively, display device D1E can also be a display or projector enabling naked-eye stereoscopic viewing, or VR goggles, etc.

[0142] The internal sound output device SP2E is a device capable of outputting sound information. The internal sound output device SP2E outputs sound based on the information sent from the excavator 100, so that the remote operator RO located in the remote control room RC can hear the sounds emitted at the work site.

[0143] An operation sensor 43 is provided in the operating device 26E for detecting the operation content of the operating device 26E. The operation sensor 43 may be, for example, a tilt sensor that detects the tilt angle of the operating lever or an angle sensor that detects the swing angle of the operating lever around its swing axis. The operation sensor 43 may also be composed of other sensors such as a pressure sensor, current sensor, voltage sensor, or distance sensor. The operation sensor 43 outputs the detected information related to the operation content of the operating device 26E to the remote controller RCC. The remote controller RCC generates an operation signal based on the received information and sends the generated operation signal to the excavator 100. Alternatively, the operation signal may be generated by the operation sensor 43. In this case, the operation sensor 43 can output the operation signal to the communication device T2 without going through the remote controller RCC. With this structure, the remote operator RO can remotely operate the excavator 100 from the remote control room RC.

[0144] The management center (MC) is a facility equipped with various devices for managing the excavator 100 located at the work site or for remote operation of the excavator 100 by a remote operator (RO) located in the remote control room (RC). In the example shown, the management center (MC) is located at locations far from both the work site and the remote control room (RC). Furthermore, the management center (MC) includes a management device (300), an internal sound output device (SP2C), and an internal sound acquisition device (M2C).

[0145] The management device 300 is an example of a control unit, such as a server computer (a so-called cloud server) or an edge server. The management device 300 is typically a fixed terminal device, but it can also be a portable terminal device (e.g., a laptop computer, tablet computer, or smartphone).

[0146] Even with the aforementioned operating system SYS, the excavator 100 can infer the presence of buried objects using a learned model. When a remote operator RO performs excavation work at the work site, the buried object inference function mode is executed. The excavator 100's controller 30 or the remote controller RCC reads the learned model and compares it with the excavation reaction force. Therefore, it can accurately determine the presence or absence of buried objects during excavation work at the work site.

[0147] Furthermore, the buried object inference system 200 according to the embodiment is configured to have an information processing unit that generates a learning model in an information processing device 210 outside the excavator 100. However, this information processing unit may also be provided in the controller 30 of the excavator 100; in other words, the buried object inference system 200 can be implemented using only the structure of the excavator 100.

[0148] The technical concept and effects of the present invention described in the above embodiments are described below.

[0149] The first embodiment of the present invention is a buried object inference system 200, which infers the possibility of buried objects in an excavation object in an excavator 100. The excavator 100 includes: a lower traveling body 1; an upper rotating body 3 rotatably disposed on the lower traveling body 1; an attachment AT disposed on the upper rotating body 3 and excavating the excavation object; and a control unit (controller 30) that controls the operation of the attachment AT. The buried object inference system 200 includes an information processing unit (information processing device 210), which acquires information related to the excavation reaction force of the attachment AT during the excavation operation to learn a learning model for detecting buried objects, and sends the learning model to the control unit. The control unit infers whether buried objects exist in the excavation object based on the excavation reaction force acquired in the actual excavation operation and the learning model.

[0150] Based on the above, by using a learning model that has learned information related to the excavation reaction force, the buried object inference system 200 can accurately infer the buried object in the excavation target when the excavator 100 is performing an excavation operation. That is, compared to simply monitoring the structure based on the excavation reaction force acquired from the excavator 100's sensors, the buried object inference system 200 can infer the presence or absence of buried objects with high probability based on changes in the excavation reaction force corresponding to the learning model. Furthermore, the buried object inference system 200 can store sensor data each time an excavation operation is performed to gradually improve the learning model. As a result, the excavator 100 can infer buried objects with higher accuracy during excavation operations.

[0151] Furthermore, the control unit (controller 30) acquires information related to the state of the object being excavated during the actual excavation operation and adjusts its learning model based on this information. Thus, the buried object inference system 200 can adjust its learning model according to the state of the excavated object at the actual work site (soil type, moisture content, viscosity, weather, etc.). As a result, it can infer the presence or absence of buried objects based on the state of the excavated object, thereby enabling more accurate inference of buried objects.

[0152] Furthermore, the information related to the object to be excavated is the information about the excavation reaction force when the object to be excavated is excavated in advance for the actual excavation operation. As a result, the buried object inference system 200 can adjust a learning model suitable for the actual work site based on the information about the excavation reaction force obtained from the advance excavation of the object to be excavated.

[0153] Furthermore, the information processing unit (information processing device 210) generates multiple learning models for each type of soil in the excavation object, and the control unit (controller 30) selects the learning model to be used based on the soil type of the excavation object to be excavated. As a result, the excavator 100 can effectively monitor the excavation reaction force during the actual excavation operation using the learning model corresponding to the soil type of the excavation object.

[0154] Furthermore, after the control unit (controller 30) or information processing unit (information processing device 210) determines that there is a buried object during the actual excavation operation, it acquires information confirming the presence or absence of the buried object and evaluates the learning model based on this information. Thus, the buried object inference system 200 can improve the learning model based on the presence or absence of buried objects confirmed through the excavation operation, thereby further improving accuracy in the next buried object inference.

[0155] Furthermore, the control unit (controller 30) calculates the difference between the actual excavation reaction force and the learned model. If the difference is less than a threshold, it is determined that there is no buried object; if the difference is greater than the threshold, it is determined that there is buried object. Thus, the buried object inference system can easily and accurately infer the presence or absence of buried objects.

[0156] Furthermore, if a buried object is detected, the control unit (controller 30) stops the operation of the attachment or changes its trajectory. Thus, the excavator 100 can prevent the attachment AT from coming into contact with the detected buried object.

[0157] Furthermore, it is equipped with a display device 40 that displays the learning model used in actual excavation operations. As a result, the operator of the excavator 100 can accurately identify the learning model used in the deduction of buried objects.

[0158] Furthermore, the display device 40 displays image information 400, which contains information related to the result of the inference of whether or not there is a buried object. Thus, the operator of the excavator 100 can immediately identify the result of the inference of whether or not there is a buried object and take necessary countermeasures.

[0159] Furthermore, the second embodiment of the present invention is an excavator 100, comprising: a lower traveling body 1; an upper rotating body 3 rotatably disposed on the lower traveling body 1; an attachment AT disposed on the upper rotating body 3 and used for excavating an object; and a control unit (controller 30) that controls the operation of the attachment AT. The control unit acquires information related to the excavation reaction force of the attachment AT during excavation to obtain a learning model for detecting buried objects, and infers the presence of buried objects in the object being excavated based on the excavation reaction force acquired during actual excavation and the learning model. In this case, the excavator 100 can accurately infer buried objects in the object being excavated during excavation operations.

[0160] The buried object inference system 200 and excavator 100 disclosed herein are illustrative and not limiting in all respects. The embodiments can be modified and improved in various ways without departing from the appended technical solutions and their spirit. Other structures can be adopted and combinations can be made with respect to the matters described in the foregoing embodiments without contradiction.

Claims

1. A system for inferring the possibility of buried objects within an excavation target while operating an excavator. The excavator includes: Lower walking body; An upper rotating body is rotatably mounted on the lower traveling body; The attachment, which is located on the upper rotating body, excavates the object to be excavated; and The control unit controls the operation of the accessory, wherein, The buried object inference system includes an information processing unit, which acquires information related to the excavation reaction force of the attachment during the excavation operation to learn a learning model for detecting the buried object, and sends the learning model to the control unit. The control unit infers whether there are buried objects in the excavation object based on the excavation reaction force obtained in the actual excavation operation and the learning model.

2. The buried object inference system according to claim 1, wherein, The control unit acquires information related to the state of the excavation object during the actual excavation operation, and adjusts its learning model based on the information related to the state of the excavation object.

3. The buried object inference system according to claim 2, wherein, The information relating to the object being excavated is information about the excavation reaction force when the object being excavated is pre-excavated for the actual excavation operation.

4. The buried object detection system according to any one of claims 1 to 3, wherein, The information processing unit generates multiple learning models for each type of soil in the excavation object. In the control unit, the learning model to be used is selected based on the soil properties of the excavation object where the actual excavation operation is being carried out.

5. The buried object detection system according to any one of claims 1 to 3, wherein, After the control unit or the information processing unit determines that the buried object exists during the actual excavation operation, it obtains information confirming the presence or absence of the buried object and evaluates the learning model based on the information confirming the presence or absence of the buried object.

6. The buried object detection system according to any one of claims 1 to 3, wherein, The control unit calculates the difference between the actual excavation reaction force and the learning model. If the difference is less than a threshold, it determines that there is no buried object; if the difference is greater than the threshold, it determines that there is a buried object.

7. The buried object inference system according to claim 6, wherein, If the presence of the buried object is detected, the control unit stops the operation of the accessory or changes its trajectory.

8. The buried object detection system according to any one of claims 1 to 3, wherein, The buried object inference system includes a display device that displays the learning model used in the actual excavation operation.

9. The buried object inference system according to claim 8, wherein, The display device displays image information, which includes information related to the result of inferring whether the buried object exists.

10. An excavator, comprising: Lower walking body; An upper rotating body is rotatably mounted on the lower traveling body; The attachment, which is located on the upper rotating body, is used to excavate the object being excavated; and The control unit controls the operation of the accessory, wherein, The control unit acquires information related to the excavation reaction force of the attachment during the excavation operation to obtain a learning model for detecting buried objects. The presence of the buried object in the excavation object is inferred based on the excavation reaction force obtained in the actual excavation operation and the learning model.

Images